Code-First Couchbase and .NET
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Zablocki, John
Couchbase Server 2.0 is anopensource, distributedNoSQL database. It is a document-oriented data store with a key/value API.Couchbase features a map/reduceengine that allows for complex document indexing and querying.This talk will introduce development with Couchbase Serverusingthe.NET Client Library. After a brief overview of serverdeployment andarchitecture, a detailed look at the key/value and document APIs will be covered. The discussion will conclude with a demonstration of using the code-first approach to building an ASP.NET MVC application withCouchbase.
 Alright, I think we're about to get started. Thanks everyone for coming out. Just a quick check in the audience to see where people are with NoSQL and CouchBase and that sort of genre. How many people have heard of NoSQL before? This is good. A couple of years ago there were very few hands, so obviously it's becoming a little more pervasive. How many people have heard of CouchBase? It's pretty good. CouchDB, MongoDB. So that's sort of the broad range of NoSQL technologies that you've probably encountered. Cassandra might be another one you've encountered. So what I'll do in the beginning of this talk, the talk is really about.NET and CouchBase and writing apps with the.NET client library that I previously supported. And what I'm going to start out with though for those of you who are new to CouchBase is just a quick overview of CouchBase so you can see how it kind of fits in with all those other NoSQL systems. So a little bit about me. I've recently changed positions. I used to work for CouchBase. I was the developer advocate doing the.NET SDK. I decided to kind of take a different path a couple of weeks ago, so I actually worked for a company that's headquartered out here in Europe. Education first. If any of you have been lucky enough to spend a year abroad during high school, you might have gone through our program. I have run the Boston Code Camp. I'm from Boston Mass, so I run the Code Camp out there. Do you guys have Code Camps out here? So the all day, Saturday, Dev events, local community stuff. And if you have any questions for me, you can find me on Twitter at CodeVoyer or you can just find me at johnslablaki.com and there's a link to a form where you can contact me if you have any questions about anything with either CouchBase or NoSQL after this talk. And feel free to drop me a note. So like I said, I'm just going to take about five minutes to cover what CouchBase is and kind of how that, how CouchBase differs from some of the other databases that you may or may not have heard of. But my first question for the audience is how many people have used a dictionary? Not the physical book, the actual. How many people have used Link? So fundamentally it looks like everyone in this room has the background knowledge for working with a database like CouchBase. The concepts from Link are going to key in heavily to the way we do indexing with MapReduce and the concepts from dictionaries are going to obviously play into the key value store aspect of CouchBase. So CouchBase is a hybrid. So a lot of databases kind of, a lot of NoSQL databases tend to fit into one or two, I'm sorry, one bucket only. And that bucket might be a key value store or it might be a document store or a multi-column or some specific genre. But CouchBase is a hybrid database as of 2.0 which came out last December. So CouchBase is both a key value store and a document store. And it's a key value store in that the CRUD operations, the primary CRUD operations are all done through simple key value, simple key value API. Has anyone worked with Memcache D? Is that what it's called now? It used to be Velocity, Microsoft's app fabric, the distributed caching tier on Azure. So it's basically, Memcache D was the original distributed cache across multiple nodes in a cluster of Memcache servers. And CouchBase is fully compatible with Memcache D. So if you've used Memcache D, you understand the primary API for working with CouchBase. But again, if you've worked with the dictionary, you also understand how to work with key value pairs. So it's a very simple API for putting data in and getting data out. It's a little more interesting when you want to create a secondary index. And that's what we're going to talk about later on, basically. So you put everything in by key value, but now you want to find users by last name. And their key is not their last name. So a couple other concepts. So again, everything is done with the key value API. You store documents in a bucket. So if I use the term bucket, think of it as a database. It's not precisely a database, but you can kind of think anywhere in SQL Server where you would have right-clicked new database, you would create a new bucket in CouchBase. And then again, secondary indexes are going to be created using JavaScript, and I'll show you a couple examples of that. So rather than, I'm going to try to minimize how many slides I look at. I want to just actually dig right in. I did call this talk code first, so I want to jump right into some actual simple code. So before I jump into the complete example, I think it's useful to start with the Hello World so you can understand what is this CouchBase thing if you haven't seen it. How many people have actually seen CouchBase? I know a bunch of you have heard of it, but has anyone actually used the product at all? So a lot fewer people. So I'll jump in with a simple Hello World, and then we'll kind of work our way up into a more interesting example. How many people have heard the term code first before? So code first for those of you who haven't heard generally refers to the idea that before you design your schema, you design your business objects, and the business objects become your database schema. Now in Ruby on Rails, it's the active record model where you basically define some active records and then auto-generate the schema. Well, I think with NoSQL, it's an interesting approach because everything is code first. You don't define objects, you don't define a schema. So CouchBase is a schema-less database as we'll see as we go along. There's no definition of a table. There's no definition of something that has a schema. So you define your objects and that's how they get saved. Does anyone feel like that's scary, not having a schema? People don't want to admit it. So I'd ask you to consider things like, okay, so one of the things that really bothers people at first instinctively is there's no schema in this database. How do I control what goes in there? But let me ask a couple of questions. How many people have gotten their schema right in the first try before? Usually at least one person says they have. How many people have denormalized their database to make it perform better? So one thing I like to say about NoSQL, a lot of people look at NoSQL and think it's creating a new class of problems. I like to think that it's creating a new class of solutions. So we don't have new problems. We just have the same problems with different solutions. So you've denormalized that database because that query that joined to 10 tables that were perfectly normalized took 10 minutes to retrieve one row. So NoSQL databases tend to start from the other side. Let's optimize first, not worry about having a table with 10 million rows in a column that's null in 900,000 of them. So that's not artificially impose a schema on our data when it doesn't really have one, or let's not artificially enforce these constraints that really don't perform well. So NoSQL takes the other approach. We need to perform first, and then we'll figure out how to deal with referential integrity and that sort of stuff. And it gets bubbled up to the application in general. This is the other scary part. There's no referential integrity or transactions beyond a single document insert in a NoSQL database. And there are a couple of fringe databases, and fringes aren't to discount them, but they haven't really caught on that are actually ACID compliant. That would be NoSQL databases. But generally speaking, the major players do not support ACID properties. So you don't get cross-insert, cross-update transactions. But a lot of this stuff goes away when you think about some of the common use cases. The way you store your data in a relational database is I insert into five tables because they're normalized versus one document that contains all my data. So that sort of need to have a transaction boundary across a lot of tables goes away because you don't have a lot of tables. So again, a lot of these problems are not new problems. They're just solved in a different way. So now with that said, I'm going to go back to my Hello World example here so we can get a quick view of how I create an instance of the CouchBase client, how I configure it, how I insert a key, and how I read the key. So the most basic operations. So I haven't actually, I think I have internet access that's working right now. This doesn't always work depending on the venue. So I'm going to look for the CouchBaseNet client. Does everyone use NuGet? Does anyone not use NuGet in the room? So if you haven't used NuGet, it's a package manager for downloading assemblies and Visual Studio into your project. So basically a reference manager. Oh, yes, that would, let me, I don't need my slides anymore, sorry. Oops, there we go. Can everyone see that okay now? So I'm going to search for the CouchBaseNet client, which is the official SDK. I'm in my installed packages, let me go to online here. There is a second CouchBaseNet client that is an older version. I don't know why NuGet is keeping that up there twice. I've removed this one as far as I know, but 1.26 is the most recent version. And it supports.NET 3.5 and 4.0. So now that I've added that reference, what I get is a reference to three different assemblies. The only dependency that the CouchBaseNet client has is JSON.NET, which pretty much everything has a dependency on. Previous versions had a dependency on Rust Sharp and Hamac, and those are no longer there. Oops, that wasn't right. So you can see this, that's my magnifier in the way, of course, Enium.caching. If you've used memcached.net, you've used enium.caching. This is basically where most of the functionality exists inside of the CouchBase driver, and that is a fork of the original enium.caching. So if you encounter anything out in the wild that's enium with CouchBase, there is a particular fork that ships with a CouchBase client, not the actual enium.caching. And then the CouchBaseAssembly is the other one that has all of the CouchBase specific wrappers around the enium stuff. So now, once you have those references, you can either configure the client in code or configure it in app config. I prefer code. I think more and more I've seen as I've gone out into the wild, people do prefer code configuration. So there's a class called the CouchBaseClientConfiguration, and the minimum configuration you need is a Bootstrap URL, and I'll explain this in just a second. So when the CouchBaseClient initializes, what it does, and this is unlike other NoSQL and SQL databases, so what it does is the CouchBaseClient starts up, it spins a thread that goes and listens to a streaming HTTP connection. So basically the cluster, the CouchBase cluster, which is just a bunch of CouchBase nodes that are acting as peers, it connects to one of those nodes which sends down topology updates. So the actual client that's running in your application space knows about the topology of the cluster. So cryptographically it hashes each key that you set for a value, and it sends it to a particular server. So as you add nodes and remove nodes, the client knows how to go directly to the right server. So it maintains a map. There's actually a layer of abstraction in between the client and those keys and how they're mapped, so that's a much more finite because you wouldn't want to have a map with millions and millions of keys potentially on the client. But the client always knows how to go right to the right server that owns the key. So in a CouchBase cluster, the nodes are all peers except one of the nodes owns the key as a master. So the key can be replicated up to three times on other nodes, but it's only on one node as a primary key. I'm not primary key in the obviously SQL sense, but it lives as a master instance of that key and that way CouchBase is consistent. There's different types of consistency levels for no SQL databases. Some databases are called eventually consistent. Has anyone heard that term before? So eventually consistent, kind of the common analogy or the common example is the DNS system. You can update your DNS record, but that DNS record has to propagate to the other nodes in the DNS system before all requests see the most recent update. CouchBase is consistent because when you say get by this key, only one server in the cluster can answer that question versus other databases where Cassandra is eventually consistent. It's tunable. You can force consistency a little bit more. But basically what happens is when you request from an eventually consistent database, you could get a dirty read. You cannot get a dirty read with CouchBase. Hopefully you can, but the.NET SDK doesn't expose the replica reads, which is how you would do that. Sure. So the question was about the failover scenario. And what happens is, so assuming that you've replicated your nodes, replicated to another node, at least two nodes, there's an election of one of those servers to become the new master for that key, and then the server rebalances the keys. So the keys, another thing I'll mention about the keys is they're auto-sharded. Is anyone not familiar with the term sharding? So sharding meaning distributing keys across a number of nodes in your system. So you can shard a SQL table so that you basically spread the rows across many nodes in a cluster. So sharding in MongoDB, for example, was kind of controversial because famously, sorry, Foursquare picked the wrong shard key, kind of overloaded some set of their shards, and that kind of brought Foursquare down. This is many years ago. This isn't like a recent thing, but it kind of was a classic example of why sharding is hard because you can't predict that first names are going to distribute, or last names are going to distribute evenly across your data, versus a cryptographically hashed key in your key value pair. It's pretty safe that you're going to get, if you have a three node cluster and 100 keys, you're going to have about 33 on each node. So you can easily, you can almost guarantee that with Couchbase, if you have n number of nodes, you're going to have even distribution across the n nodes for your keys. And you don't have to do any kind of predicting because it's all done automatically. So that's the bootstrap URL. So there's basically like a JSON handshake that happens when the client starts up, it gets a JSON response. It reads it, gets an X URL to go to to get a little more information, and then it kind of goes through that once or twice. Because of that, you don't want to create multiple instances of your Couchbase client. It's not the typical database pattern where it's using connection equals new connection and starting over. You're going to end up kind of read handshaking every time. So that does mean that you have to have a stateful client. Generally speaking, you keep a static member somewhere that you reuse throughout your application. There's also a socket pool. So it's not that there's, you can fine tune how many connections are in the socket pool. So basically it'll maintain by default a set of 10 connections to your cluster, to each node in your cluster. The other thing you need to tell Couchbase is which bucket to connect to. So again, buckets are databases. They're analogous to databases. So there's a default bucket that's installed when you run through the wizard and install Couchbase. So it's just an off list. So the off model in Couchbase is SASL. So it's just basically, if you've used, it's a simple binary protocol. It's not encrypting anything. It's not user defined object level security. It's just, I have a bucket. I can either put a password on it or I don't. The username is the name of the bucket. You can set a password on the bucket. And that's, again, that's not unique to Couchbase. Most node SQL databases don't have granular security. You may have object, like database level users, and you may have read only versus write accessible. But generally speaking, no SQL databases do not have the same security model as yet as SQL databases. So now that I have my configuration, I can create a new client. So it's the classes Couchbase client. And if you use the parameter list constructor, the client will expect that you have an app config section defined for Couchbase. Or I can just pass my config instance. So now we'll ignore return values for now and assume everything's going to work because this is the Hello World example. So I'm going to create a message. Equals. Did I get that right? Was that right? Close enough. And so then I'm going to save that message. So the store method is the basic method for doing ads updates and replaces. There are three types of ads in Couchbase. Three types of stores in Couchbase. You can add, which does an insert. It'll fail if the key exists. Replace does an update. It'll fail if the key doesn't exist. Or a set, which is basically like a save, which does an insert if it doesn't exist or an update if it does exist. So add is useful. Generally speaking, I usually use just the set mode. So generally, if you have things that you want to fail, like creating a user based on the email address and you want the user property, whatever to be unique, add is a way of guaranteeing that. And you get a status back that says the key exists or something to that effect. So you can kind of check to make sure that you are ensuring uniqueness and getting the proper behavior. So I'm going to set the store mode to set the key to message and the value to message. So it's slightly more complicated than a hash table because I have to set the store mode. The other SDKs, the Java, Ruby and everything else, they don't have this idea of a store mode. They have straight methods that are set, replaced and add. This is an artifact of the enum.caching library. I personally would, at some point, would like to see three unique methods there. So then we'll make sure that it worked by getting the value back. So the key is message. So when I set it, I set it with a key. When I get it, I get it with a key. And if all worked, I should get my almost no region hello world. So any questions on the very basics of hello there then? I'm trying. I feel like because I'm reasonably tall and mostly blonde, I've been mistaken for no region quite a bit since I've been here. So now I want to move on to a more interesting example, which will kind of walk through a full crud application, some of the more advanced usages and basically just show how you do code-first development with a no-SQL database. So I think everyone's familiar with the idea of a task app. You have a task. It has a due date, that sort of thing. So I'm going to start out by creating my model. This is the code-first aspect. If you haven't done quote unquote code-first development with entity framework or something, the basic idea is you define a model class. So I'm going to define a model called task. Can you guys, is the font size big enough? My task will have an ID, a description, we'll say notes, we'll give it a is-complete flag and a due date. So very basic model. Any questions or seems pretty straightforward. So now let's go ahead and how many, I shouldn't make any assumptions, how many people have used, not used MVC before? Excellent. It is interesting. Over the past couple of years, I would say it's increasing by about 25% a year the number of people who have gotten exposed to MVC, which is a good thing. So I'm going to create a new controller. I'm going to use the wizards as I go through this demo just so it speeds things up. So I'm going to create an MVC controller with empty read-write actions. And I'm going to call the controller tasks controller. So all I'm doing is creating a very basic CRUD application for managing tasks at this point. So the index is actually going to be the hardest piece that we do. So I'm going to do that last. It's hardest because we haven't seen how to work with secondary indexes yet. So I'm going to start with the create form. And I'm not going to worry about things like validation or that sort of stuff. We're just going to keep this very simple. So I'm going to use the, has anyone, has everyone used the scaffolding in Visual Studio for MVC? That's all the stuff that I'm doing now is part of the scaffolding feature of MVC where you can basically point it out of model and generate forms and things like that in controllers. So I'm going to create a new view under the tasks folder. And we're going to start with create. And then it's a strongly typed view. And you have to compile your application before it actually shows up in that list. So I'm going to call this view create again, strongly typed. So now my task shows up. So now this is going to give me a nice create. If I pick the check box that says to use the create. So I forgot to set the template to create. So what this did for me at this point, in case you haven't seen this before, if I navigate now to task slash create, I have a very simple create form. I think there's no validation because I haven't put validation attributes on or anything. So it's just a very basic create form that I can work with from now on. Before I do that, what I want to do is jump over to the couch base. I have, if I wave at my laptop, it detects that and starts to do things. So it may move on its own. I did cheat for the Hello World. So this is the couch base console. So it seemed like only a couple people had seen this. One of the big differentiators between the other NoSQL products and couch bases, couch bases has in my not so unbiased opinion the best admin tools. This is what you get when you install a couch base. It's a web-based, fully exposed API. So everything you see, this is a jQuery app. This isn't like a server doing complicated stuff in the back end. This is a full, restful API, interface to a fully restful API. So if you're ever curious to see what is actually happening, you can just use Firebug or whatever tool to go look at the calls that are being made to generate these graphs and things like that. So when you install it, you get some basic information. So I have a couple of buckets active. You can see ops per second. If I actually had ongoing ops per second, I would get to see the moving graph. You can see if you had any servers down in your cluster. You can drill into the node level. So if I had multiple nodes, so if I wanted to create a couch base, a true cluster, a cluster is just a single node or more nodes. If I wanted to have a multi-node cluster, I click on add server. I add the IP address and then I add the cluster username and password. So when you install the first node, you set a cluster account and that cluster auth is used to then basically bring all the other nodes online in the cluster. If I had in that failover case, it would show up as pending rebalance because basically the keys need to be rebalanced. Data buckets, excuse me. So the beer sample bucket, the beer sample is based on the OpenBearDB. This is the couch base version of what was Northwind. So it's not the most up-to-date database. So if you're a beer drinker, you probably won't find the beer you're looking for in there. But there's some interesting data and we wrote a bunch of sample apps around it, which if there's time at the end, I'll show you quickly the.NET one. Views I'm going to cover in a bit. Cross data center replication. This is a nice feature. You can actually pump data bidirectionally out of couch base and bidirectional, cross data center replication is set up by basically setting up two unidirectional cross data center replication. So if you have, I don't know what the coasts are here, but like in America, if you have an east coast and west coast data center, you can send data one way and have it sent the other way. Or you can just use unidirectional and have a single feed into another data center for backup or disaster recovery. The neat thing about it is it's all done with HTTP. So you can, if you wanted to write your own basically HTTP endpoint for the couch base data, across data center replication, you could. And we've done that to work with Elasticsearch. So if you want to pump data into an Elasticsearch cluster and do full tech search on your documents, you can do that. Does anyone use Nancy, the micro framework for ASP.NET? So it's kind of inspired by Ruby's Sinatra. So it's a very simple handler. I wrote on blog.couchbase.com, you can find a little example where I created a Nancy endpoint which listens to the change feed and then just spits out data into an XML file as a dummy example. But if you wanted to move data from couch base into SQL server, that's one way to do it. We're into some other back end processing. The log is what it sounds like. Then you can set some settings like re-adding some sample buckets, whether you want to have compaction and basically cleaning up the disk, auto failover and that sort of thing. What I want to do now is add a data bucket. So for this to-do application, I'm going to create a new bucket. So this is a new database. And when I create this, I'll just give it the name, NDC Oslo. Now again, we're fully compatible with Memcache D. So if you wanted to create a Memcache D bucket, you can, but there's no persistence. If you wanted to use couch base, and we do have lots of clients who use couch base as a caching layer because we have that Memcache D support, so you can actually just drop out your Memcache D cluster and replace it with a couch base cluster and use Memcache D buckets. But again, there's no persistence with those. And the performance really isn't going to be better. So you could just create couch base buckets and set expirations on- there is the ability to add an expiration to a key so you can have it expire and that sort of thing. You can change the port, I'm sorry, you can't change the port with a standard authenticated, if you create an authenticated bucket, it has to be on port 11 to 11. If you want to have an off-less bucket, you can do that just on a different port. So I'm going to leave the password blank for now. I only have a single node cluster so there's no reason to enable replicas. Flush is useful if you want to have a test bucket. So basically you want to just flush all the data out of the bucket. When you enable flush, you basically have a one-click, I can now delete all the data option. So I have that bucket now, you can see it in my bucket list. So now, are there any questions on the UI before I start with the code? So now the first thing I want to do is implement the create method. So I don't need to do anything on the create get, but for the create post, I need to write the insert method basically. So what I'm going to do is start off by creating a static private field of type couch-based client. And I had previously added the couch-based client reference through NuGet to this project because I wasn't sure about the internet access, so that's where that came from in case you're wondering. And what I'm going to do is, in a static constructor, so static constructors, if you're not familiar with them, are guaranteed to execute once before any instance of the classes created. So in the static constructor, I'm going to create an instance of this couch-based client. I also need to set that configuration like I did before, so var config equals newCatchBasedClient configuration. Again, set the URI so it knows where to bootstrap against. And then the bucket name. What I call ndc.oslue. And then I pass that config to the client. So now I have a static field that I can now use across all my methods here. And that was, again, just what we saw with the Hello World stuff. So now in create, what I'm going to do is, let's see. I'm going to change this. Instead of taking a forms collection, I'm just going to have it auto-bind to the task using the kind of bi-directional data binding that comes with MVC out of the box. So now that I have a task, actually, first site, so we're a key value store, right? So I need to consider what the key is going to be. Now a lot of times, predictable keys are useful because you can do ordering on keys and things like that with views that we'll see in just a few minutes. But the important thing is that you pick a key that makes sense for your scenario. Now there's really not value in having a key. There's no auto key, first of all. So there's no sort of, I do an insert and I don't have a key, so it automatically assigns one. You have to pick a key. And so generally speaking, if I were doing a to-do app like the one I'm demoing now, I would just use a GUID or some unique identifier. There's not really any value in having a descriptive key for a task. If someone has a better reason to do that, I'm open to that. But I think the best way to do this is just GUID.newGUID.2string. All keys in catchbase are strings, UTF-8 strings. Oops, not result key. So now I'm going to call the store method, but I'm going to call a different store method. So one of the interesting legacy items I had to deal with when I first took over the catchbase client was that it was, again, it was a caching library under the covers. So the core of the library was a caching library. And in a caching library, arguably you don't want exceptions thrown if something is a cache miss or something is a cache fail, cache write fail. You either got it out of cache or you didn't and your application responds appropriately. So what was happening in the internals of the client is every single exception was being swallowed and booleans were being bubbled up to the top. So if you called store and it failed, you got a false. Now, if you're storing a transactional financial data or something, you probably don't want false as the reason why something failed. So there's this sort of parallel API. So every store get remove method has an associated execute dot dot dot method. So execute store returns more detail than store. So store returns a boolean. Execute store returns an operation result, which I'll show you in just a second what that contains. So we'll do store mode. We're doing a create. So I'm going to do store mode.add. The key again is that GUID. And the thing we're going to save is the task. So then that result object. So you can do things like check the success of the result. So a common pattern, if it's successful, or if it's not successful, I should say, you can check things like the message, the status code, or the exception. So if there was an exception way down in the socket layer, it basically gets wrapped up and passed up through this exception object. So there's no exception that's thrown, but you can check to see if, excuse me, if exception is not null, then throw that exception. So I won't go through that exercise of all that detail, but just so you know there is a bunch of detail about what went wrong or what didn't go wrong inside of that result. So let me get claim some space back here. So now on the next line, I'm just going to redirect to the details. So we'll kind of work our way. We'll create it and we'll show the results in the details. Oops. So redirect to action, details. So I want to redirect with the ID set to the value of the key. Oops. Return. So there are a couple of issues I have to fix that I just know from doing this a bunch of times. You have to save serializable objects through the client. So if you're saving a class, it has to be serializable because it gets serialized and saved. And the other thing is because we're dealing with string IDs, with an MVC template you get int IDs so you just have to change so on edit and on details, these int IDs have to be string IDs or the routes don't match. And since we're going to details, let me just find that. Okay. So now let me compile that. So now, oops, where did my, this should still be good. So I'm going to just create a very simple task here. And there's my gesture thing popping up because it sees me waving or something. It's not complete. We'll give it a due date of 6.14, 2013. Apologies. Okay. So this I expected, right? So I haven't created the details for me yet. So no exception was thrown. Here's my GUID. If I go, well, I'm going to go through details first and then I'll show you what's actually in the database. There are any questions on create so far? Pretty straightforward. I have a key. I have an object that's serializable. I save it. There's not much more to it than that. Now in the details, I'm just going to get by key. So the key is passed, oops, the key was passed here in the query string. So I have the key available to me. And that's coming in through the route as this string ID. So var result equals client.excuteget. And I can use the generic execute get, which will automatically cast it to a task on the way out. And then the key is the ID. And then I just pass that. So the execute get method returns an I get operation result, which has a value property, which is the thing I saved. It also has the exception property, the success property and all those things. So that just wraps all that stuff. The get method returns either null or the object with no detail about what went wrong. So now I'm going to create the details view. I'm going to name it details, scaffolding template of details. So now if I compile and refresh this YSOD, then I should get, so now I have the description showing up. So pretty straightforward getting set. It's really just what we've seen so far in our console app. But what I want to show you now is what's actually in the database. So you can see in NDC Oslo we have one up per second. So we're doing pretty well there. If I click on documents, one you can see over here that I have an item count of one. And if I click in here, you can see here's that document. Here's the key. So this is a little strange though. I've said before couch base is a document oriented database. That doesn't really look like a document. Anyone have an idea of what that is? Base 64 encoded version of the binary object. Now I didn't store JSON. There's a big difference between couch base and MongoDB. MongoDB forces everything into Bson, binary JSON. Couch base does not. Couch base stores binary objects. It stores JSON as just a special recognition object you can think of it as. So on the server, if you store valid JSON, the server can do stuff to that JSON document. The client has to make a decision as to whether it's going to pass JSON or not. And to make that easier, what you can do is use some extension methods. So we've thought a lot about whether we should force people to store JSON. We have a legacy of a lot of binary data being stored. So we couldn't just release 2.0 and say, oh, all that data that you stored binary, now it has to be JSON. So we have, plus some people don't want to store JSON. They want to store just objects in their binary form and retrieve them quickly. People are storing images, people are storing all sorts of different things in couch base clusters. So there are lots of use cases for not storing binary objects. JSON is very expensive. We heard from a former CTO of Zynga that they were spending 30% of their CPU cycles just doing JSON serialization and deserialization. That's not insignificant, obviously. I do have some extension methods, though. If you want to always, under couch base.extensions, you can basically take any of the execute methods and tack the word JSON to the end of it. So if I say execute get JSON or execute store JSON, if I compile that and go back to my form, we can see something else that happens, create a task 2. So everything worked as it did before. But now if I go back to my documents, you'll see that I have JSON stored here. So I have a useful thing as far as the server is concerned. I have a JSON document against which I can write a secondary index. So I'm just going to store a couple of these tasks quickly and just change some dates here. And we'll create a task 4. We'll call this. OK. So now over here, I've just basically created some more data to demonstrate the list, which is the next thing we'll do. Before I jump into the list, which is going to switch to secondary indexing, does anyone have any questions? What happens if you go back to the document with the JSON? It's still non-JSON in there. Yeah, but when you're in your report, we're trying to do the... Oh, yeah. That'll break because, yeah. If I took this... I apologize that I didn't bring music. So if I tried to drop this in, I'll get a deserialization problem. Well, the model doesn't have an ID, so the form is actually breaking, but it won't deserialize because it's not a string. Yeah, backwards compatibility. And there are a bunch of challenges with backwards compatibility because you can store a document that doesn't deserialize properly into your class. You can have binary data that's not going to store properly. There are solutions for all that stuff, whether it's versioning your documents or just having... You can create something called a custom transcoder. It's pluggable, so you can basically tell Couchface every time you deserialize something, if it's of some particular format, then deserialize it into binary and then JSON. You can look up transcoders and see some videos that we had done on how to do some of that sort of special mapping. My question would be then, so everyone knows how to work with key value stores. How would you get a list of all the keys for a particular... How would you get a list of all the tasks with a key value store? Can anyone imagine how you might have done this? If all you had was this key value API, how would you go find all the keys? Kind of a trick question. Yeah, there's definitely... And I'll show you how to do it without... This is not using the key value API to do that. So with Couchface 1.8, which was a pure key value store, the sort of approach that people took was to store keys in another key. So if you wanted to say all of my tasks are... When I create a task, also insert the key into the tasks list. So you're basically storing references and other keys, pulling them all back. There were some other... Like if you had sequential keys, you could do basically a bunch of gets until one failed by incrementing up one. And then you kind of knew the one that failed, that's the top one, and now grab them all. None of it's pretty. Yeah, everyone kind of realizes that. So secondary indexing really is much prettier, it's much nicer. So let me jump right into that. So the way secondary indexing is done, it's something called a view. Has anyone heard the term map reduce before? So map reduce, if you've used link, you've probably had some sort of map reduce exposure. Think of a very basic idea, you have some collection of things, in this case all of the documents. You have a map function. The map function is going to be applied over a big four each loop, over each of your documents, and the input to the map function is the current document, the output is a key value pair. The key is something you want to search on, or something you want to create an index on. So it's basically just transforming a set of data into a projection, which is a key value pair. And you can do that with a dot where dot select in link. So if you want to think of a link analogy, the map function is kind of like the dot where, and the application of the dot select to shape it into a key value pair. Let me show you in JavaScript what that means though. So we have this idea of development versus production views. So development views are done with basically taking a small set of your production data. Let's say you want to create an index on a million records. You don't want to start with all one million records. You want to start with a finite subset of that. So development views are deterministic. They create a small subset. They basically grab a small set of your data and mark it as usable in a development view. So it's a useful way not to break things. Design documents are namespaces for your index definitions. So I'm creating a design document called tasks, in this case. The dev underscore is what makes it that deterministic dev database. I'm sorry, design document. And the view is just going to be all. So I want to get all tasks. So once I create that, I click in and I have this editor where I have a map function which is just a JavaScript function that by default creates an index on the ID. Now, the only reason you would ever create an index on the ID, so the ID is the key. So one of the confusing things to kind of get your head around is in the context of key value crud operations, the key is the key in the key value operation. In the context of a view, the key is the meta.id because views actually create a new key value pair, which is the index. So when I'm talking about keys in terms of views, I'm talking about the thing I'm indexing. So I'm creating an index on a property that property is called a key. So if I run this, you see I just get all documents along with their, the index created on their ID. Now this is the primary key. So this is just like creating a secondary index on a primary index. You wouldn't do that, obviously. Unless you want to do a range query on your primary key. You can't do range queries on keys without using, without creating an index on them. You can't say give me all the keys that start with A and go to Z if you don't have an index on that secondary index. Now, but what I want is really just null. So I want all tasks. So I don't care about the key. I just want to, I want to create an index that will just return all of the data for a set of tasks. So let me show you what, how this actually works by showing you the JSON that comes back to the SDK. When you create an index, you create a triplet of rows. You create a set of rows where each row is a triplet. And the triplet consists of the original ID. So the key from the key value, insert, the key, so the thing you're indexing. So if I had a first name or last name, that would show up in the key. And the value is potentially a projection over which I'm going to do aggregation, which I probably don't have time to get into today. But if I were doing like a count of tags, I would project something into that, or some of tags, I may, or some of karma points, I may project that into a value and then do aggregation on it. But the important thing is that with every row in an index, and this is just a standard B plus tree database index, with every row I have the ID of the original document. So what I do with that ID generally is go back to the original document in memory and pull it out, because Catchbase has persistence. So views are basically indexes that are stored on disk, right off of disk. But you go back to the memcached API, because your document is 99% likely to be in memory. So all of those documents are resident in memory if you have enough memory. So any most recently used document will be there. So it's kind of a different access pattern than your typical database where you wouldn't generally create an index to get the primary key to go back to the original row. But that's what you do in Catchbase, because you create an index to get the ID to go back to memory. So it's just a different sort of workflow. But that's abstracted from you and the client. I'll show you that now. So let me do the all query here, and then I'm going to change it to something that will hopefully illustrate what that key and the key value is a little more easily. So now if I go back to the client, like, what? That's going to be an issue I'll show you in just a second. That was a good catch. So the index method, where is it? So what I want to do is call the client's getView method. There's a generic getView which will convert your documents into an instance of t. So my design document was named tasks, and my view name was all. And there's a third argument that says go pull it out of memory instead of trying to deserialize that value. And then I'm going to pass that view to the index view. Again, we have a name collision with view and view here. So I'm going to create an MVC view called index with a list scaffolding. So one more quick thing I have to do over here is I have to promote this from production, from dev to production. So when I publish, now you can see I have a production view. So basically I just took the dev underscore off the name so it will go against my entire dataset. So now if I go back to list, oops. So this is part of the problem that a couple of people have mentioned. So I have this binary thing sitting in there. I can't convert that. My view just emitted everything. There's no way for me to know that a document is a task. We don't have a concept of a collection or some kind of container for documents. In Couchface there's a bucket, and the bucket just contains documents. So the convention is to include a type on your documents. So what I usually do is define a base class that forces you to implement an abstract type. So we'll just return task. So this demonstrates the schema-less idea. So now I can just go create new documents with a new schema, quote unquote, because each document has an implicit schema. But if I go back, let me create just a couple of quick documents here. Let me make this. Now if I go back to my view, now I can check that I just pull up a random somewhere in here. So now you can see, does everyone see this type equals task? Now in my map function I can emit only tasks. So anything you can do in JavaScript, you can do, oops, you can't edit in production. You have to go to Dev to edit. Anything you can do in JavaScript, you could theoretically embed jQuery in here. I've looked at, I've basically taken a base64 decoder, put it in here, and pulled out like original strings that have been base64 encoded as a demo. I would never recommend doing that. So now I can check if doc.type equals task, then emit it. So if I save that and I show the results, now you see that I'm only getting the two records that I created. So I've given taxonomy to my documents, even though they don't have it. So I'm republishing to production. So now when I go back to list, hopefully everything should work. Now you see that I have my documents. But I want to show one last thing, then I'll take questions. So if I create a new document, and I give it, so if we look back at my list, you see I have 114, 314, but let's say I do 12, 2013, 2012. So I'm intentionally creating these in the past. So a couple things to demonstrate. These are eventually consistent. So basically when you create the view, you create an index on all the documents that exist. There are different tuning settings to say that you can force a view to be not eventually consistent. I don't have time to demo that, but it's basically you can set a flag that says, on a right, make sure it goes to disk, on the view, make sure that you update the index, which is incremental. It's not going to update all of your documents. It's just anything that's changed. And that gives you a fully consistent read. So when I refresh, it shows up. So it just took a second for the indexer to kick off and update that. So basically requesting a view triggers the next index of your view. But you can tell the view index first and then give me a result. But notice that these are not quite in order. 12, 12, 2012 should be the first task in my list. So the way I solve that problem, I can create an index on the date. So if I go back here, oops, we're going to add a new view. And we're going to call this by date. And this is going to be very similar to my all. So I'm just going to copy this. So let me make that a little bigger. So now in addition to checking that the type is a task, I want to make sure that because we're schemalists, I want to have some protection on my index creation. I want to make sure that the doc has a due date. Because I don't want to index a document by due date if it doesn't have a due date. And I can just use JavaScript's implicit null check there. So now I'm emitting the due date. So you can think of all these map functions if you want to think of a SQL analogy, create index. All this is doing is creating indexes on properties inside of documents. So if I show the results, you can see that now I'm ordering correctly by date time. Let me publish this to production. Even if I come back here and change my method to by date. So now when I refresh this, we should see this order by date. So now 1212, 2012 comes first, which coincidentally is the date that catchbase 2 was launched. One other thing I'll mention that's worth knowing about is out on github.com. There's a much more complete example of using Couchbase with MVC. And this is, it runs against the beer sample database. And I've extensively commented this thing. I wrote like a 40 page tutorial on it. If you picked up the NDC developer book, I wrote an article on basically here's the beer sample database, here's the view that gets you data out of it. So that's in there. But this says everything from basic cred forms that we just looked at. If you wanted to see, it doesn't look like we have any. If you wanted to see how to do aggregation, this is all in there. If you wanted to see, again, this is github.com slash couchbase labs. And there's also geospatial stuff. So if you wanted to see how to do geospatial indexing, this app is kind of goes through some best practices too. So has anyone, is anyone familiar with the repository pattern? So I implemented a repository of T. I'm at zero here. So there's a repository of T and it's fully commented. You can see how to create subclasses out of a base repository class, which will then give you access to all of your handling of errors and how to reuse a lot of this code. There's also one last thing I'll mention. That JavaScript stuff that I was showing you, you can actually not do that. And instead use a framework I wrote called couchbase model views, which you decorate your plain old C sharp objects with attributes and it will auto generate the views for you. So if you wanted to create that all view, a by name query or that sort of stuff, it will just automatically generate that. So I know that was a lot of information, but are there any questions? In this framework? Oh, sure. So the sorting, it's ascending by default. You can set descending to false. There's a parameter. So all the views, the API has things like descending. You can set stale as the parameter to turn off. You can do an exact query by passing in a key. So if you wanted to find the user with the last name of X. All in the query. Yep. Question? So multiple buckets, I would generally say there are resources associated with each bucket and there are limitations where you can't cross buckets to get data. So if you have a document in one and you want to somehow reference that document, you can't do joins or anything like that in no SQL database, but you can, with the same client, you can get to multiple documents within a bucket. So there's client affinity to a bucket. So if you create an instance of, like if you have five buckets, you're one going to create a lot of client resources. So it's per node per bucket, 10 connections per server. So that adds up quickly. But more importantly, there are things like indexers. So the thing that actually increments all those design documents or implement, that thing that actually indexes all those documents is happening per bucket. So a lot of those resources are expensive. So the general rule of thumb I always follow is if I would create a database for this in SQL server, I would create a bucket for it in CouchBase. With that type property. So again, it's that convention of including a type property or a doc type property or something. And that's kind of inherited from CouchDB where the same sort of general collection of just one thing existed. Yeah, you can, so there's some limits. When you set the bucket memory size, it has to be equal, there has to be basically the same amount of RAM on each node in the cluster because the sharding is distributed equally. So you can't have one bucket, one node can't have, so if you set it to three gigs, for example, all of your servers have to have three gigs available. So if you had like a 96 gig large instance on Amazon and you wanted to have that coupled with a small instance with 24 gigs or something, that wouldn't work. But you can upgrade all of the nodes. So you can have nested objects. There's no referential integrity. So generally the pattern is to store the ID of another document as a property of another document. You need to combine them manually. And in that bear sample application I showed you, I do walk through how you do master detail. So it's a little tricky. And I actually walked through that in the developer article that I wrote. Like what? So the comment was about what it looks like Lotus Notes. Damian Katz, who is the CTO of Couchbase and the creator of CouchDB, worked at Lotus Notes. So no coincidence, I'm sure. Question? No, so you specified, the question is, you know, when you handshake to that first node, generally you want to specify more than one node. So if that node isn't available, you can get to the next one. But once it connects to one node, it gets a list of all of the URIs. So if a node goes down, it knows how to get to the next one. So you can get to the next one. In the config is generally the preferred approach because it maintains an internal list of nodes to connect to. Question? Yes, so for large tables that's being queried in lots of different ways, do you think there's a way to get to the next one? You know, I think one of the, so NoSQL actually refers to not only SQL, so I think anyone at Couchbase or any NoSQL company would probably suggest that it's a complementary technology. You know, SQL will always have a place for like data warehousing for highly transactional systems. You know, you can do, you can mimic sort of after the fact transactional systems, like do verifications after things go wrong, you know, after things are inserted and see if they went wrong. But I definitely think there's room for both. You know, I don't think that anyone's going to stop using SQL because of Couchbase, but they may stop using some of their SQL for Couchbase. So is performance one of the issues? Performance is, yeah, performance is definitely a big issue. So we, Couchbase is probably, I mean, this is obviously a somewhat biased opinion, but is generally considered probably the fastest NoSQL database out there. You know, we, so a good example of what we can achieve, LinkedIn, I'm sure most people know of, they have a four node cluster, they're doing 400,000 ops per second. Cisco, who has ridiculously fast hardware, has a five node cluster doing one and a half million operations per second. So you can't get that out of SQL server, you know, you can't get that, you could, it would just cost you a fortune. So it, you know, for people who need to scale massively and easily and quickly, you know, one of our biggest success stories is that you guys have drawn something, did everyone, does everyone remember that game, the little Pictionary app? It went from like 3 million to 100 million users in a matter of weeks and they scaled out on us. So there's, scale is definitely our best story. You know, we have, you know, I think we have just, we've spent a lot of time in making sure that this thing will grow from one node to 100 nodes and have no downtime because you can, you add nodes, take nodes offline without your application having to go offline. Question? Yes. The replication story, you talked about how you could replicate the non-seed, but there was a talk about how you could replicate the non-seed, but there was a talk about how you could replicate the non-seed to calculate the stator or EPL. Oh yeah, so cross-data center replication is cluster to cluster. That is the standard, but you can, because it's an open interface, you can actually implement other endpoints. So, but the replication is actually just, I have three nodes in a cluster. I specify that two of them are replicas and one of them is a primary. So it just, it will replicate automatically for you. You could, yes. Yeah, so if you wanted to take, so a common use case is in the advertising industry, the online advertising industry, where people, they pull data into CouchBase, pump it out into Hadoop, do some analytics and pump it back into CouchBase. Any other questions? Thank you.
 Alright, I think we're about to get started.  Thanks everyone for coming out.  Just a quick check in the audience to see where people are with NoSQL and CouchBase  and that sort of genre.  How many people have heard of NoSQL before?  This is good.  A couple of years ago there were very few hands, so obviously it's becoming a little  more pervasive.  How many people have heard of CouchBase?  It's pretty good.  CouchDB, MongoDB.  So that's sort of the broad range of NoSQL technologies that you've probably encountered.  Cassandra might be another one you've encountered.  So what I'll do in the beginning of this talk, the talk is really about.NET and CouchBase  and writing apps with the.NET client library that I previously supported.  And what I'm going to start out with though for those of you who are new to CouchBase is  just a quick overview of CouchBase so you can see how it kind of fits in with all those  other NoSQL systems.  So a little bit about me.  I've recently changed positions.  I used to work for CouchBase.  I was the developer advocate doing the.NET SDK.  I decided to kind of take a different path a couple of weeks ago, so I actually worked  for a company that's headquartered out here in Europe.  Education first.  If any of you have been lucky enough to spend a year abroad during high school, you might  have gone through our program.  I have run the Boston Code Camp.  I'm from Boston Mass, so I run the Code Camp out there.  Do you guys have Code Camps out here?  So the all day, Saturday, Dev events, local community stuff.  And if you have any questions for me, you can find me on Twitter at CodeVoyer or you can  just find me at johnslablaki.com and there's a link to a form where you can contact me  if you have any questions about anything with either CouchBase or NoSQL after this talk.  And feel free to drop me a note.  So like I said, I'm just going to take about five minutes to cover what CouchBase is and  kind of how that, how CouchBase differs from some of the other databases that you may or  may not have heard of.  But my first question for the audience is how many people have used a dictionary?  Not the physical book, the actual.  How many people have used Link?  So fundamentally it looks like everyone in this room has the background knowledge for  working with a database like CouchBase.  The concepts from Link are going to key in heavily to the way we do indexing with MapReduce  and the concepts from dictionaries are going to obviously play into the key value store  aspect of CouchBase.  So CouchBase is a hybrid.  So a lot of databases kind of, a lot of NoSQL databases tend to fit into one or two, I'm  sorry, one bucket only.  And that bucket might be a key value store or it might be a document store or a multi-column  or some specific genre.  But CouchBase is a hybrid database as of 2.0 which came out last December.  So CouchBase is both a key value store and a document store.  And it's a key value store in that the CRUD operations, the primary CRUD operations are  all done through simple key value, simple key value API.  Has anyone worked with Memcache D?  Is that what it's called now?  It used to be Velocity, Microsoft's app fabric, the distributed caching tier on Azure.  So it's basically, Memcache D was the original distributed cache across multiple nodes in  a cluster of Memcache servers.  And CouchBase is fully compatible with Memcache D.  So if you've used Memcache D, you understand the primary API for working with CouchBase.  But again, if you've worked with the dictionary, you also understand how to work with key value  pairs.  So it's a very simple API for putting data in and getting data out.  It's a little more interesting when you want to create a secondary index.  And that's what we're going to talk about later on, basically.  So you put everything in by key value, but now you want to find users by last name.  And their key is not their last name.  So a couple other concepts.  So again, everything is done with the key value API.  You store documents in a bucket.  So if I use the term bucket, think of it as a database.  It's not precisely a database, but you can kind of think anywhere in SQL Server where  you would have right-clicked new database, you would create a new bucket in CouchBase.  And then again, secondary indexes are going to be created using JavaScript, and I'll show  you a couple examples of that.  So rather than, I'm going to try to minimize how many slides I look at.  I want to just actually dig right in.  I did call this talk code first, so I want to jump right into some actual simple code.  So before I jump into the complete example, I think it's useful to start with the Hello  World so you can understand what is this CouchBase thing if you haven't seen it.  How many people have actually seen CouchBase?  I know a bunch of you have heard of it, but has anyone actually used the product at all?  So a lot fewer people.  So I'll jump in with a simple Hello World, and then we'll kind of work our way up into  a more interesting example.  How many people have heard the term code first before?  So code first for those of you who haven't heard generally refers to the idea that before  you design your schema, you design your business objects, and the business objects become your  database schema.  Now in Ruby on Rails, it's the active record model where you basically define some active  records and then auto-generate the schema.  Well, I think with NoSQL, it's an interesting approach because everything is code first.  You don't define objects, you don't define a schema.  So CouchBase is a schema-less database as we'll see as we go along.  There's no definition of a table.  There's no definition of something that has a schema.  So you define your objects and that's how they get saved.  Does anyone feel like that's scary, not having a schema?  People don't want to admit it.  So I'd ask you to consider things like, okay, so one of the things that really bothers  people at first instinctively is there's no schema in this database.  How do I control what goes in there?  But let me ask a couple of questions.  How many people have gotten their schema right in the first try before?  Usually at least one person says they have.  How many people have denormalized their database to make it perform better?  So one thing I like to say about NoSQL, a lot of people look at NoSQL and think it's  creating a new class of problems.  I like to think that it's creating a new class of solutions.  So we don't have new problems.  We just have the same problems with different solutions.  So you've denormalized that database because that query that joined to 10 tables that were  perfectly normalized took 10 minutes to retrieve one row.  So NoSQL databases tend to start from the other side.  Let's optimize first, not worry about having a table with 10 million rows in a column that's  null in 900,000 of them.  So that's not artificially impose a schema on our data when it doesn't really have one,  or let's not artificially enforce these constraints that really don't perform well.  So NoSQL takes the other approach.  We need to perform first, and then we'll figure out how to deal with referential integrity  and that sort of stuff.  And it gets bubbled up to the application in general.  This is the other scary part.  There's no referential integrity or transactions beyond a single document insert in a NoSQL  database.  And there are a couple of fringe databases, and fringes aren't to discount them, but they  haven't really caught on that are actually ACID compliant.  That would be NoSQL databases.  But generally speaking, the major players do not support ACID properties.  So you don't get cross-insert, cross-update transactions.  But a lot of this stuff goes away when you think about some of the common use cases.  The way you store your data in a relational database is I insert into five tables because  they're normalized versus one document that contains all my data.  So that sort of need to have a transaction boundary across a lot of tables goes away  because you don't have a lot of tables.  So again, a lot of these problems are not new problems.  They're just solved in a different way.  So now with that said, I'm going to go back to my Hello World example here so we can get  a quick view of how I create an instance of the CouchBase client, how I configure it,  how I insert a key, and how I read the key.  So the most basic operations.  So I haven't actually, I think I have internet access that's working right now.  This doesn't always work depending on the venue.  So I'm going to look for the CouchBaseNet client.  Does everyone use NuGet?  Does anyone not use NuGet in the room?  So if you haven't used NuGet, it's a package manager for downloading assemblies and Visual  Studio into your project.  So basically a reference manager.  Oh, yes, that would, let me, I don't need my slides anymore, sorry.  Oops, there we go.  Can everyone see that okay now?  So I'm going to search for the CouchBaseNet client, which is the official SDK.  I'm in my installed packages, let me go to online here.  There is a second CouchBaseNet client that is an older version.  I don't know why NuGet is keeping that up there twice.  I've removed this one as far as I know, but 1.26 is the most recent version.  And it supports.NET 3.5 and 4.0.  So now that I've added that reference, what I get is a reference to three different assemblies.  The only dependency that the CouchBaseNet client has is JSON.NET, which pretty much  everything has a dependency on.  Previous versions had a dependency on Rust Sharp and Hamac, and those are no longer there.  Oops, that wasn't right.  So you can see this, that's my magnifier in the way, of course, Enium.caching.  If you've used memcached.net, you've used enium.caching.  This is basically where most of the functionality exists inside of the CouchBase driver, and  that is a fork of the original enium.caching.  So if you encounter anything out in the wild that's enium with CouchBase, there is a particular  fork that ships with a CouchBase client, not the actual enium.caching.  And then the CouchBaseAssembly is the other one that has all of the CouchBase specific  wrappers around the enium stuff.  So now, once you have those references, you can either configure the client in code or  configure it in app config.  I prefer code.  I think more and more I've seen as I've gone out into the wild, people do prefer code configuration.  So there's a class called the CouchBaseClientConfiguration, and the minimum configuration you need is a Bootstrap  URL, and I'll explain this in just a second.  So when the CouchBaseClient initializes, what it does, and this is unlike other NoSQL and  SQL databases, so what it does is the CouchBaseClient starts up, it spins a thread that goes and  listens to a streaming HTTP connection.  So basically the cluster, the CouchBase cluster, which is just a bunch of CouchBase nodes that  are acting as peers, it connects to one of those nodes which sends down topology updates.  So the actual client that's running in your application space knows about the topology  of the cluster.  So cryptographically it hashes each key that you set for a value, and it sends it to a particular  server.  So as you add nodes and remove nodes, the client knows how to go directly to the right  server.  So it maintains a map.  There's actually a layer of abstraction in between the client and those keys and how  they're mapped, so that's a much more finite because you wouldn't want to have a map with  millions and millions of keys potentially on the client.  But the client always knows how to go right to the right server that owns the key.  So in a CouchBase cluster, the nodes are all peers except one of the nodes owns the key  as a master.  So the key can be replicated up to three times on other nodes, but it's only on one node  as a primary key.  I'm not primary key in the obviously SQL sense, but it lives as a master instance of that  key and that way CouchBase is consistent.  There's different types of consistency levels for no SQL databases.  Some databases are called eventually consistent.  Has anyone heard that term before?  So eventually consistent, kind of the common analogy or the common example is the DNS  system.  You can update your DNS record, but that DNS record has to propagate to the other nodes  in the DNS system before all requests see the most recent update.  CouchBase is consistent because when you say get by this key, only one server in the cluster  can answer that question versus other databases where Cassandra is eventually consistent.  It's tunable.  You can force consistency a little bit more.  But basically what happens is when you request from an eventually consistent database, you  could get a dirty read.  You cannot get a dirty read with CouchBase.  Hopefully you can, but the.NET SDK doesn't expose the replica reads, which is how you  would do that.  Sure.  So the question was about the failover scenario.  And what happens is, so assuming that you've replicated your nodes, replicated to another  node, at least two nodes, there's an election of one of those servers to become the new  master for that key, and then the server rebalances the keys.  So the keys, another thing I'll mention about the keys is they're auto-sharded.  Is anyone not familiar with the term sharding?  So sharding meaning distributing keys across a number of nodes in your system.  So you can shard a SQL table so that you basically spread the rows across many nodes  in a cluster.  So sharding in MongoDB, for example, was kind of controversial because famously, sorry,  Foursquare picked the wrong shard key, kind of overloaded some set of their shards, and  that kind of brought Foursquare down.  This is many years ago.  This isn't like a recent thing, but it kind of was a classic example of why sharding is  hard because you can't predict that first names are going to distribute, or last names  are going to distribute evenly across your data, versus a cryptographically hashed key  in your key value pair.  It's pretty safe that you're going to get, if you have a three node cluster and 100 keys,  you're going to have about 33 on each node.  So you can easily, you can almost guarantee that with Couchbase, if you have n number of  nodes, you're going to have even distribution across the n nodes for your keys.  And you don't have to do any kind of predicting because it's all done automatically.  So that's the bootstrap URL.  So there's basically like a JSON handshake that happens when the client starts up, it  gets a JSON response.  It reads it, gets an X URL to go to to get a little more information, and then it kind  of goes through that once or twice.  Because of that, you don't want to create multiple instances of your Couchbase client.  It's not the typical database pattern where it's using connection equals new connection  and starting over.  You're going to end up kind of read handshaking every time.  So that does mean that you have to have a stateful client.  Generally speaking, you keep a static member somewhere that you reuse throughout your application.  There's also a socket pool.  So it's not that there's, you can fine tune how many connections are in the socket pool.  So basically it'll maintain by default a set of 10 connections to your cluster, to each  node in your cluster.  The other thing you need to tell Couchbase is which bucket to connect to.  So again, buckets are databases.  They're analogous to databases.  So there's a default bucket that's installed when you run through the wizard and install  Couchbase.  So it's just an off list.  So the off model in Couchbase is SASL.  So it's just basically, if you've used, it's a simple binary protocol.  It's not encrypting anything.  It's not user defined object level security.  It's just, I have a bucket.  I can either put a password on it or I don't.  The username is the name of the bucket.  You can set a password on the bucket.  And that's, again, that's not unique to Couchbase.  Most node SQL databases don't have granular security.  You may have object, like database level users, and you may have read only versus write accessible.  But generally speaking, no SQL databases do not have the same security model as yet as  SQL databases.  So now that I have my configuration, I can create a new client.  So it's the classes Couchbase client.  And if you use the parameter list constructor, the client will expect that you have an app  config section defined for Couchbase.  Or I can just pass my config instance.  So now we'll ignore return values for now and assume everything's going to work because  this is the Hello World example.  So I'm going to create a message.  Equals.  Did I get that right?  Was that right?  Close enough.  And so then I'm going to save that message.  So the store method is the basic method for doing ads updates and replaces.  There are three types of ads in Couchbase.  Three types of stores in Couchbase.  You can add, which does an insert.  It'll fail if the key exists.  Replace does an update.  It'll fail if the key doesn't exist.  Or a set, which is basically like a save, which does an insert if it doesn't exist or  an update if it does exist.  So add is useful.  Generally speaking, I usually use just the set mode.  So generally, if you have things that you want to fail, like creating a user based on  the email address and you want the user property, whatever to be unique, add is a way of guaranteeing  that.  And you get a status back that says the key exists or something to that effect.  So you can kind of check to make sure that you are ensuring uniqueness and getting the  proper behavior.  So I'm going to set the store mode to set the key to message and the value to message.  So it's slightly more complicated than a hash table because I have to set the store mode.  The other SDKs, the Java, Ruby and everything else, they don't have this idea of a store  mode.  They have straight methods that are set, replaced and add.  This is an artifact of the enum.caching library.  I personally would, at some point, would like to see three unique methods there.  So then we'll make sure that it worked by getting the value back.  So the key is message.  So when I set it, I set it with a key.  When I get it, I get it with a key.  And if all worked, I should get my almost no region hello world.  So any questions on the very basics of hello there then?  I'm trying.  I feel like because I'm reasonably tall and mostly blonde, I've been mistaken for no region  quite a bit since I've been here.  So now I want to move on to a more interesting example, which will kind of walk through a  full crud application, some of the more advanced usages and basically just show how you do  code-first development with a no-SQL database.  So I think everyone's familiar with the idea of a task app.  You have a task.  It has a due date, that sort of thing.  So I'm going to start out by creating my model.  This is the code-first aspect.  If you haven't done quote unquote code-first development with entity framework or something,  the basic idea is you define a model class.  So I'm going to define a model called task.  Can you guys, is the font size big enough?  My task will have an ID, a description, we'll say notes, we'll give it a is-complete flag  and a due date.  So very basic model.  Any questions or seems pretty straightforward.  So now let's go ahead and how many, I shouldn't make any assumptions, how many people have  used, not used MVC before?  Excellent.  It is interesting.  Over the past couple of years, I would say it's increasing by about 25% a year the number  of people who have gotten exposed to MVC, which is a good thing.  So I'm going to create a new controller.  I'm going to use the wizards as I go through this demo just so it speeds things up.  So I'm going to create an MVC controller with empty read-write actions.  And I'm going to call the controller tasks controller.  So all I'm doing is creating a very basic CRUD application for managing tasks at this  point.  So the index is actually going to be the hardest piece that we do.  So I'm going to do that last.  It's hardest because we haven't seen how to work with secondary indexes yet.  So I'm going to start with the create form.  And I'm not going to worry about things like validation or that sort of stuff.  We're just going to keep this very simple.  So I'm going to use the, has anyone, has everyone used the scaffolding in Visual Studio for MVC?  That's all the stuff that I'm doing now is part of the scaffolding feature of MVC where  you can basically point it out of model and generate forms and things like that in controllers.  So I'm going to create a new view under the tasks folder.  And we're going to start with create.  And then it's a strongly typed view.  And you have to compile your application before it actually shows up in that list.  So I'm going to call this view create again, strongly typed.  So now my task shows up.  So now this is going to give me a nice create.  If I pick the check box that says to use the create.  So I forgot to set the template to create.  So what this did for me at this point, in case you haven't seen this before, if I navigate  now to task slash create, I have a very simple create form.  I think there's no validation because I haven't put validation attributes on or anything.  So it's just a very basic create form that I can work with from now on.  Before I do that, what I want to do is jump over to the couch base.  I have, if I wave at my laptop, it detects that and starts to do things.  So it may move on its own.  I did cheat for the Hello World.  So this is the couch base console.  So it seemed like only a couple people had seen this.  One of the big differentiators between the other NoSQL products and couch bases, couch  bases has in my not so unbiased opinion the best admin tools.  This is what you get when you install a couch base.  It's a web-based, fully exposed API.  So everything you see, this is a jQuery app.  This isn't like a server doing complicated stuff in the back end.  This is a full, restful API, interface to a fully restful API.  So if you're ever curious to see what is actually happening, you can just use Firebug  or whatever tool to go look at the calls that are being made to generate these graphs and  things like that.  So when you install it, you get some basic information.  So I have a couple of buckets active.  You can see ops per second.  If I actually had ongoing ops per second, I would get to see the moving graph.  You can see if you had any servers down in your cluster.  You can drill into the node level.  So if I had multiple nodes, so if I wanted to create a couch base, a true cluster, a  cluster is just a single node or more nodes.  If I wanted to have a multi-node cluster, I click on add server.  I add the IP address and then I add the cluster username and password.  So when you install the first node, you set a cluster account and that cluster auth is  used to then basically bring all the other nodes online in the cluster.  If I had in that failover case, it would show up as pending rebalance because basically  the keys need to be rebalanced.  Data buckets, excuse me.  So the beer sample bucket, the beer sample is based on the OpenBearDB.  This is the couch base version of what was Northwind.  So it's not the most up-to-date database.  So if you're a beer drinker, you probably won't find the beer you're looking for in  there.  But there's some interesting data and we wrote a bunch of sample apps around it, which  if there's time at the end, I'll show you quickly the.NET one.  Views I'm going to cover in a bit.  Cross data center replication.  This is a nice feature.  You can actually pump data bidirectionally out of couch base and bidirectional, cross  data center replication is set up by basically setting up two unidirectional cross data  center replication.  So if you have, I don't know what the coasts are here, but like in America, if you have  an east coast and west coast data center, you can send data one way and have it sent  the other way.  Or you can just use unidirectional and have a single feed into another data center for  backup or disaster recovery.  The neat thing about it is it's all done with HTTP.  So you can, if you wanted to write your own basically HTTP endpoint for the couch base  data, across data center replication, you could.  And we've done that to work with Elasticsearch.  So if you want to pump data into an Elasticsearch cluster and do full tech search on your documents,  you can do that.  Does anyone use Nancy, the micro framework for ASP.NET?  So it's kind of inspired by Ruby's Sinatra.  So it's a very simple handler.  I wrote on blog.couchbase.com, you can find a little example where I created a Nancy endpoint  which listens to the change feed and then just spits out data into an XML file as a  dummy example.  But if you wanted to move data from couch base into SQL server, that's one way to do  it.  We're into some other back end processing.  The log is what it sounds like.  Then you can set some settings like re-adding some sample buckets, whether you want to have  compaction and basically cleaning up the disk, auto failover and that sort of thing.  What I want to do now is add a data bucket.  So for this to-do application, I'm going to create a new bucket.  So this is a new database.  And when I create this, I'll just give it the name, NDC Oslo.  Now again, we're fully compatible with Memcache D. So if you wanted to create a Memcache  D bucket, you can, but there's no persistence.  If you wanted to use couch base, and we do have lots of clients who use couch base as  a caching layer because we have that Memcache D support, so you can actually just drop out  your Memcache D cluster and replace it with a couch base cluster and use Memcache D buckets.  But again, there's no persistence with those.  And the performance really isn't going to be better.  So you could just create couch base buckets and set expirations on- there is the ability  to add an expiration to a key so you can have it expire and that sort of thing.  You can change the port, I'm sorry, you can't change the port with a standard authenticated,  if you create an authenticated bucket, it has to be on port 11 to 11.  If you want to have an off-less bucket, you can do that just on a different port.  So I'm going to leave the password blank for now.  I only have a single node cluster so there's no reason to enable replicas.  Flush is useful if you want to have a test bucket.  So basically you want to just flush all the data out of the bucket.  When you enable flush, you basically have a one-click, I can now delete all the data  option.  So I have that bucket now, you can see it in my bucket list.  So now, are there any questions on the UI before I start with the code?  So now the first thing I want to do is implement the create method.  So I don't need to do anything on the create get, but for the create post, I need to write  the insert method basically.  So what I'm going to do is start off by creating a static private field of type couch-based  client.  And I had previously added the couch-based client reference through NuGet to this project  because I wasn't sure about the internet access, so that's where that came from in case you're  wondering.  And what I'm going to do is, in a static constructor, so static constructors, if you're  not familiar with them, are guaranteed to execute once before any instance of the classes  created.  So in the static constructor, I'm going to create an instance of this couch-based client.  I also need to set that configuration like I did before, so var config equals newCatchBasedClient  configuration.  Again, set the URI so it knows where to bootstrap against.  And then the bucket name.  What I call ndc.oslue.  And then I pass that config to the client.  So now I have a static field that I can now use across all my methods here.  And that was, again, just what we saw with the Hello World stuff.  So now in create, what I'm going to do is, let's see.  I'm going to change this.  Instead of taking a forms collection, I'm just going to have it auto-bind to the task  using the kind of bi-directional data binding that comes with MVC out of the box.  So now that I have a task, actually, first site, so we're a key value store, right?  So I need to consider what the key is going to be.  Now a lot of times, predictable keys are useful because you can do ordering on keys and things  like that with views that we'll see in just a few minutes.  But the important thing is that you pick a key that makes sense for your scenario.  Now there's really not value in having a key.  There's no auto key, first of all.  So there's no sort of, I do an insert and I don't have a key, so it automatically assigns  one.  You have to pick a key.  And so generally speaking, if I were doing a to-do app like the one I'm demoing now,  I would just use a GUID or some unique identifier.  There's not really any value in having a descriptive key for a task.  If someone has a better reason to do that, I'm open to that.  But I think the best way to do this is just GUID.newGUID.2string.  All keys in catchbase are strings, UTF-8 strings.  Oops, not result key.  So now I'm going to call the store method, but I'm going to call a different store method.  So one of the interesting legacy items I had to deal with when I first took over the catchbase  client was that it was, again, it was a caching library under the covers.  So the core of the library was a caching library.  And in a caching library, arguably you don't want exceptions thrown if something is a cache  miss or something is a cache fail, cache write fail.  You either got it out of cache or you didn't and your application responds appropriately.  So what was happening in the internals of the client is every single exception was being  swallowed and booleans were being bubbled up to the top.  So if you called store and it failed, you got a false.  Now, if you're storing a transactional financial data or something, you probably don't want  false as the reason why something failed.  So there's this sort of parallel API.  So every store get remove method has an associated execute dot dot dot method.  So execute store returns more detail than store.  So store returns a boolean.  Execute store returns an operation result, which I'll show you in just a second what  that contains.  So we'll do store mode.  We're doing a create.  So I'm going to do store mode.add.  The key again is that GUID.  And the thing we're going to save is the task.  So then that result object.  So you can do things like check the success of the result.  So a common pattern, if it's successful, or if it's not successful, I should say, you  can check things like the message, the status code, or the exception.  So if there was an exception way down in the socket layer, it basically gets wrapped up  and passed up through this exception object.  So there's no exception that's thrown, but you can check to see if, excuse me, if exception  is not null, then throw that exception.  So I won't go through that exercise of all that detail, but just so you know there is  a bunch of detail about what went wrong or what didn't go wrong inside of that result.  So let me get claim some space back here.  So now on the next line, I'm just going to redirect to the details.  So we'll kind of work our way.  We'll create it and we'll show the results in the details.  Oops.  So redirect to action, details.  So I want to redirect with the ID set to the value of the key.  Oops.  Return.  So there are a couple of issues I have to fix that I just know from doing this a bunch  of times.  You have to save serializable objects through the client.  So if you're saving a class, it has to be serializable because it gets serialized and  saved.  And the other thing is because we're dealing with string IDs, with an MVC template you get  int IDs so you just have to change so on edit and on details, these int IDs have to be string  IDs or the routes don't match.  And since we're going to details, let me just find that.  Okay.  So now let me compile that.  So now, oops, where did my, this should still be good.  So I'm going to just create a very simple task here.  And there's my gesture thing popping up because it sees me waving or something.  It's not complete.  We'll give it a due date of 6.14, 2013.  Apologies.  Okay.  So this I expected, right?  So I haven't created the details for me yet.  So no exception was thrown.  Here's my GUID.  If I go, well, I'm going to go through details first and then I'll show you what's actually  in the database.  There are any questions on create so far?  Pretty straightforward.  I have a key.  I have an object that's serializable.  I save it.  There's not much more to it than that.  Now in the details, I'm just going to get by key.  So the key is passed, oops, the key was passed here in the query string.  So I have the key available to me.  And that's coming in through the route as this string ID.  So var result equals client.excuteget.  And I can use the generic execute get, which will automatically cast it to a task on the  way out.  And then the key is the ID.  And then I just pass that.  So the execute get method returns an I get operation result, which has a value property,  which is the thing I saved.  It also has the exception property, the success property and all those things.  So that just wraps all that stuff.  The get method returns either null or the object with no detail about what went wrong.  So now I'm going to create the details view.  I'm going to name it details, scaffolding template of details.  So now if I compile and refresh this YSOD, then I should get, so now I have the description  showing up.  So pretty straightforward getting set.  It's really just what we've seen so far in our console app.  But what I want to show you now is what's actually in the database.  So you can see in NDC Oslo we have one up per second.  So we're doing pretty well there.  If I click on documents, one you can see over here that I have an item count of one.  And if I click in here, you can see here's that document.  Here's the key.  So this is a little strange though.  I've said before couch base is a document oriented database.  That doesn't really look like a document.  Anyone have an idea of what that is?  Base 64 encoded version of the binary object.  Now I didn't store JSON.  There's a big difference between couch base and MongoDB.  MongoDB forces everything into Bson, binary JSON.  Couch base does not.  Couch base stores binary objects.  It stores JSON as just a special recognition object you can think of it as.  So on the server, if you store valid JSON, the server can do stuff to that JSON document.  The client has to make a decision as to whether it's going to pass JSON or not.  And to make that easier, what you can do is use some extension methods.  So we've thought a lot about whether we should force people to store JSON.  We have a legacy of a lot of binary data being stored.  So we couldn't just release 2.0 and say, oh, all that data that you stored binary, now  it has to be JSON.  So we have, plus some people don't want to store JSON.  They want to store just objects in their binary form and retrieve them quickly.  People are storing images, people are storing all sorts of different things in couch base  clusters.  So there are lots of use cases for not storing binary objects.  JSON is very expensive.  We heard from a former CTO of Zynga that they were spending 30% of their CPU cycles just  doing JSON serialization and deserialization.  That's not insignificant, obviously.  I do have some extension methods, though.  If you want to always, under couch base.extensions, you can basically take any of the execute  methods and tack the word JSON to the end of it.  So if I say execute get JSON or execute store JSON, if I compile that and go back to my  form, we can see something else that happens, create a task 2.  So everything worked as it did before.  But now if I go back to my documents, you'll see that I have JSON stored here.  So I have a useful thing as far as the server is concerned.  I have a JSON document against which I can write a secondary index.  So I'm just going to store a couple of these tasks quickly and just change some dates here.  And we'll create a task 4.  We'll call this.  OK.  So now over here, I've just basically created some more data to demonstrate the list, which  is the next thing we'll do.  Before I jump into the list, which is going to switch to secondary indexing, does anyone  have any questions?  What happens if you go back to the document with the JSON?  It's still non-JSON in there.  Yeah, but when you're in your report, we're trying to do the...  Oh, yeah.  That'll break because, yeah.  If I took this...  I apologize that I didn't bring music.  So if I tried to drop this in, I'll get a deserialization problem.  Well, the model doesn't have an ID, so the form is actually breaking, but it won't deserialize  because it's not a string.  Yeah, backwards compatibility.  And there are a bunch of challenges with backwards compatibility because you can store a document  that doesn't deserialize properly into your class.  You can have binary data that's not going to store properly.  There are solutions for all that stuff, whether it's versioning your documents or just having...  You can create something called a custom transcoder.  It's pluggable, so you can basically tell Couchface every time you deserialize something,  if it's of some particular format, then deserialize it into binary and then JSON.  You can look up transcoders and see some videos that we had done on how to do some of that  sort of special mapping.  My question would be then, so everyone knows how to work with key value stores.  How would you get a list of all the keys for a particular...  How would you get a list of all the tasks with a key value store?  Can anyone imagine how you might have done this?  If all you had was this key value API, how would you go find all the keys?  Kind of a trick question.  Yeah, there's definitely...  And I'll show you how to do it without...  This is not using the key value API to do that.  So with Couchface 1.8, which was a pure key value store, the sort of approach that people  took was to store keys in another key.  So if you wanted to say all of my tasks are...  When I create a task, also insert the key into the tasks list.  So you're basically storing references and other keys, pulling them all back.  There were some other...  Like if you had sequential keys, you could do basically a bunch of gets until one failed  by incrementing up one.  And then you kind of knew the one that failed, that's the top one, and now grab them all.  None of it's pretty.  Yeah, everyone kind of realizes that.  So secondary indexing really is much prettier, it's much nicer.  So let me jump right into that.  So the way secondary indexing is done, it's something called a view.  Has anyone heard the term map reduce before?  So map reduce, if you've used link, you've probably had some sort of map reduce exposure.  Think of a very basic idea, you have some collection of things, in this case all of  the documents.  You have a map function.  The map function is going to be applied over a big four each loop, over each of your documents,  and the input to the map function is the current document, the output is a key value pair.  The key is something you want to search on, or something you want to create an index on.  So it's basically just transforming a set of data into a projection, which is a key  value pair.  And you can do that with a dot where dot select in link.  So if you want to think of a link analogy, the map function is kind of like the dot where,  and the application of the dot select to shape it into a key value pair.  Let me show you in JavaScript what that means though.  So we have this idea of development versus production views.  So development views are done with basically taking a small set of your production data.  Let's say you want to create an index on a million records.  You don't want to start with all one million records.  You want to start with a finite subset of that.  So development views are deterministic.  They create a small subset.  They basically grab a small set of your data and mark it as usable in a development view.  So it's a useful way not to break things.  Design documents are namespaces for your index definitions.  So I'm creating a design document called tasks, in this case.  The dev underscore is what makes it that deterministic dev database.  I'm sorry, design document.  And the view is just going to be all.  So I want to get all tasks.  So once I create that, I click in and I have this editor where I have a map function which  is just a JavaScript function that by default creates an index on the ID.  Now, the only reason you would ever create an index on the ID, so the ID is the key.  So one of the confusing things to kind of get your head around is in the context of key  value crud operations, the key is the key in the key value operation.  In the context of a view, the key is the meta.id because views actually create a new key  value pair, which is the index.  So when I'm talking about keys in terms of views, I'm talking about the thing I'm indexing.  So I'm creating an index on a property that property is called a key.  So if I run this, you see I just get all documents along with their, the index created on their  ID.  Now this is the primary key.  So this is just like creating a secondary index on a primary index.  You wouldn't do that, obviously.  Unless you want to do a range query on your primary key.  You can't do range queries on keys without using, without creating an index on them.  You can't say give me all the keys that start with A and go to Z if you don't have an index  on that secondary index.  Now, but what I want is really just null.  So I want all tasks.  So I don't care about the key.  I just want to, I want to create an index that will just return all of the data for a set  of tasks.  So let me show you what, how this actually works by showing you the JSON that comes  back to the SDK.  When you create an index, you create a triplet of rows.  You create a set of rows where each row is a triplet.  And the triplet consists of the original ID.  So the key from the key value, insert, the key, so the thing you're indexing.  So if I had a first name or last name, that would show up in the key.  And the value is potentially a projection over which I'm going to do aggregation, which  I probably don't have time to get into today.  But if I were doing like a count of tags, I would project something into that, or some  of tags, I may, or some of karma points, I may project that into a value and then do  aggregation on it.  But the important thing is that with every row in an index, and this is just a standard  B plus tree database index, with every row I have the ID of the original document.  So what I do with that ID generally is go back to the original document in memory and  pull it out, because Catchbase has persistence.  So views are basically indexes that are stored on disk, right off of disk.  But you go back to the memcached API, because your document is 99% likely to be in memory.  So all of those documents are resident in memory if you have enough memory.  So any most recently used document will be there.  So it's kind of a different access pattern than your typical database where you wouldn't  generally create an index to get the primary key to go back to the original row.  But that's what you do in Catchbase, because you create an index to get the ID to go back  to memory.  So it's just a different sort of workflow.  But that's abstracted from you and the client.  I'll show you that now.  So let me do the all query here, and then I'm going to change it to something that will  hopefully illustrate what that key and the key value is a little more easily.  So now if I go back to the client, like, what?  That's going to be an issue I'll show you in just a second.  That was a good catch.  So the index method, where is it?  So what I want to do is call the client's getView method.  There's a generic getView which will convert your documents into an instance of t.  So my design document was named tasks, and my view name was all.  And there's a third argument that says go pull it out of memory instead of trying to  deserialize that value.  And then I'm going to pass that view to the index view.  Again, we have a name collision with view and view here.  So I'm going to create an MVC view called index with a list scaffolding.  So one more quick thing I have to do over here is I have to promote this from production,  from dev to production.  So when I publish, now you can see I have a production view.  So basically I just took the dev underscore off the name so it will go against my entire  dataset.  So now if I go back to list, oops.  So this is part of the problem that a couple of people have mentioned.  So I have this binary thing sitting in there.  I can't convert that.  My view just emitted everything.  There's no way for me to know that a document is a task.  We don't have a concept of a collection or some kind of container for documents.  In Couchface there's a bucket, and the bucket just contains documents.  So the convention is to include a type on your documents.  So what I usually do is define a base class that forces you to implement an abstract type.  So we'll just return task.  So this demonstrates the schema-less idea.  So now I can just go create new documents with a new schema, quote unquote, because each  document has an implicit schema.  But if I go back, let me create just a couple of quick documents here.  Let me make this.  Now if I go back to my view, now I can check that I just pull up a random somewhere in  here.  So now you can see, does everyone see this type equals task?  Now in my map function I can emit only tasks.  So anything you can do in JavaScript, you can do, oops, you can't edit in production.  You have to go to Dev to edit.  Anything you can do in JavaScript, you could theoretically embed jQuery in here.  I've looked at, I've basically taken a base64 decoder, put it in here, and pulled out like  original strings that have been base64 encoded as a demo.  I would never recommend doing that.  So now I can check if doc.type equals task, then emit it.  So if I save that and I show the results, now you see that I'm only getting the two  records that I created.  So I've given taxonomy to my documents, even though they don't have it.  So I'm republishing to production.  So now when I go back to list, hopefully everything should work.  Now you see that I have my documents.  But I want to show one last thing, then I'll take questions.  So if I create a new document, and I give it, so if we look back at my list, you see  I have 114, 314, but let's say I do 12, 2013, 2012.  So I'm intentionally creating these in the past.  So a couple things to demonstrate.  These are eventually consistent.  So basically when you create the view, you create an index on all the documents that  exist.  There are different tuning settings to say that you can force a view to be not eventually  consistent.  I don't have time to demo that, but it's basically you can set a flag that says, on a right,  make sure it goes to disk, on the view, make sure that you update the index, which is incremental.  It's not going to update all of your documents.  It's just anything that's changed.  And that gives you a fully consistent read.  So when I refresh, it shows up.  So it just took a second for the indexer to kick off and update that.  So basically requesting a view triggers the next index of your view.  But you can tell the view index first and then give me a result.  But notice that these are not quite in order.  12, 12, 2012 should be the first task in my list.  So the way I solve that problem, I can create an index on the date.  So if I go back here, oops, we're going to add a new view.  And we're going to call this by date.  And this is going to be very similar to my all.  So I'm just going to copy this.  So let me make that a little bigger.  So now in addition to checking that the type is a task, I want to make sure that because  we're schemalists, I want to have some protection on my index creation.  I want to make sure that the doc has a due date.  Because I don't want to index a document by due date if it doesn't have a due date.  And I can just use JavaScript's implicit null check there.  So now I'm emitting the due date.  So you can think of all these map functions if you want to think of a SQL analogy, create  index.  All this is doing is creating indexes on properties inside of documents.  So if I show the results, you can see that now I'm ordering correctly by date time.  Let me publish this to production.  Even if I come back here and change my method to by date.  So now when I refresh this, we should see this order by date.  So now 1212, 2012 comes first, which coincidentally is the date that catchbase 2 was launched.  One other thing I'll mention that's worth knowing about is out on github.com.  There's a much more complete example of using Couchbase with MVC.  And this is, it runs against the beer sample database.  And I've extensively commented this thing.  I wrote like a 40 page tutorial on it.  If you picked up the NDC developer book, I wrote an article on basically here's the  beer sample database, here's the view that gets you data out of it.  So that's in there.  But this says everything from basic cred forms that we just looked at.  If you wanted to see, it doesn't look like we have any.  If you wanted to see how to do aggregation, this is all in there.  If you wanted to see, again, this is github.com slash couchbase labs.  And there's also geospatial stuff.  So if you wanted to see how to do geospatial indexing, this app is kind of goes through  some best practices too.  So has anyone, is anyone familiar with the repository pattern?  So I implemented a repository of T. I'm at zero here.  So there's a repository of T and it's fully commented.  You can see how to create subclasses out of a base repository class, which will then give  you access to all of your handling of errors and how to reuse a lot of this code.  There's also one last thing I'll mention.  That JavaScript stuff that I was showing you, you can actually not do that.  And instead use a framework I wrote called couchbase model views, which you decorate  your plain old C sharp objects with attributes and it will auto generate the views for you.  So if you wanted to create that all view, a by name query or that sort of stuff, it will  just automatically generate that.  So I know that was a lot of information, but are there any questions?  In this framework?  Oh, sure.  So the sorting, it's ascending by default.  You can set descending to false.  There's a parameter.  So all the views, the API has things like descending.  You can set stale as the parameter to turn off.  You can do an exact query by passing in a key.  So if you wanted to find the user with the last name of X.  All in the query.  Yep.  Question?  So multiple buckets, I would generally say there are resources associated with each  bucket and there are limitations where you can't cross buckets to get data.  So if you have a document in one and you want to somehow reference that document, you can't  do joins or anything like that in no SQL database, but you can, with the same client,  you can get to multiple documents within a bucket.  So there's client affinity to a bucket.  So if you create an instance of, like if you have five buckets, you're one going to create  a lot of client resources.  So it's per node per bucket, 10 connections per server.  So that adds up quickly.  But more importantly, there are things like indexers.  So the thing that actually increments all those design documents or implement, that  thing that actually indexes all those documents is happening per bucket.  So a lot of those resources are expensive.  So the general rule of thumb I always follow is if I would create a database for this in  SQL server, I would create a bucket for it in CouchBase.  With that type property.  So again, it's that convention of including a type property or a doc type property or  something.  And that's kind of inherited from CouchDB where the same sort of general collection of just  one thing existed.  Yeah, you can, so there's some limits.  When you set the bucket memory size, it has to be equal, there has to be basically the  same amount of RAM on each node in the cluster because the sharding is distributed equally.  So you can't have one bucket, one node can't have, so if you set it to three gigs, for  example, all of your servers have to have three gigs available.  So if you had like a 96 gig large instance on Amazon and you wanted to have that coupled  with a small instance with 24 gigs or something, that wouldn't work.  But you can upgrade all of the nodes.  So you can have nested objects.  There's no referential integrity.  So generally the pattern is to store the ID of another document as a property of another  document.  You need to combine them manually.  And in that bear sample application I showed you, I do walk through how you do master detail.  So it's a little tricky.  And I actually walked through that in the developer article that I wrote.  Like what?  So the comment was about what it looks like Lotus Notes.  Damian Katz, who is the CTO of Couchbase and the creator of CouchDB, worked at Lotus Notes.  So no coincidence, I'm sure.  Question?  No, so you specified, the question is, you know, when you handshake to that first node,  generally you want to specify more than one node.  So if that node isn't available, you can get to the next one.  But once it connects to one node, it gets a list of all of the URIs.  So if a node goes down, it knows how to get to the next one.  So you can get to the next one.  In the config is generally the preferred approach because it maintains an internal list of nodes  to connect to.  Question?  Yes, so for large tables that's being queried in lots of different ways, do you think there's  a way to get to the next one?  You know, I think one of the, so NoSQL actually refers to not only SQL, so I think anyone  at Couchbase or any NoSQL company would probably suggest that it's a complementary technology.  You know, SQL will always have a place for like data warehousing for highly transactional  systems.  You know, you can do, you can mimic sort of after the fact transactional systems, like  do verifications after things go wrong, you know, after things are inserted and see if  they went wrong.  But I definitely think there's room for both.  You know, I don't think that anyone's going to stop using SQL because of Couchbase, but  they may stop using some of their SQL for Couchbase.  So is performance one of the issues?  Performance is, yeah, performance is definitely a big issue.  So we, Couchbase is probably, I mean, this is obviously a somewhat biased opinion, but  is generally considered probably the fastest NoSQL database out there.  You know, we, so a good example of what we can achieve, LinkedIn, I'm sure most people  know of, they have a four node cluster, they're doing 400,000 ops per second.  Cisco, who has ridiculously fast hardware, has a five node cluster doing one and a half  million operations per second.  So you can't get that out of SQL server, you know, you can't get that, you could, it  would just cost you a fortune.  So it, you know, for people who need to scale massively and easily and quickly, you know,  one of our biggest success stories is that you guys have drawn something, did everyone,  does everyone remember that game, the little Pictionary app?  It went from like 3 million to 100 million users in a matter of weeks and they scaled  out on us.  So there's, scale is definitely our best story.  You know, we have, you know, I think we have just, we've spent a lot of time in making  sure that this thing will grow from one node to 100 nodes and have no downtime because  you can, you add nodes, take nodes offline without your application having to go offline.  Question?  Yes.  The replication story, you talked about how you could replicate the non-seed, but there  was a talk about how you could replicate the non-seed, but there was a talk about how  you could replicate the non-seed to calculate the stator or EPL.  Oh yeah, so cross-data center replication is cluster to cluster.  That is the standard, but you can, because it's an open interface, you can actually implement  other endpoints.  So, but the replication is actually just, I have three nodes in a cluster.  I specify that two of them are replicas and one of them is a primary.  So it just, it will replicate automatically for you.  You could, yes.  Yeah, so if you wanted to take, so a common use case is in the advertising industry, the  online advertising industry, where people, they pull data into CouchBase, pump it out  into Hadoop, do some analytics and pump it back into CouchBase.  Any other questions?  Thank you.
John_Zablocki_-_Code-First_NoSQL_with__0_107.png John_Zablocki_-_Code-First_NoSQL_with__108_7103.png John_Zablocki_-_Code-First_NoSQL_with__7104_7151.png John_Zablocki_-_Code-First_NoSQL_with__7152_13475.png John_Zablocki_-_Code-First_NoSQL_with__13476_13547.png John_Zablocki_-_Code-First_NoSQL_with__13548_13667.png John_Zablocki_-_Code-First_NoSQL_with__13668_14759.png John_Zablocki_-_Code-First_NoSQL_with__14760_15155.png John_Zablocki_-_Code-First_NoSQL_with__15156_15227.png John_Zablocki_-_Code-First_NoSQL_with__15228_15347.png John_Zablocki_-_Code-First_NoSQL_with__15348_16343.png John_Zablocki_-_Code-First_NoSQL_with__16344_30683.png John_Zablocki_-_Code-First_NoSQL_with__30684_30755.png John_Zablocki_-_Code-First_NoSQL_with__30756_30815.png John_Zablocki_-_Code-First_NoSQL_with__30816_36923.png John_Zablocki_-_Code-First_NoSQL_with__36924_37043.png John_Zablocki_-_Code-First_NoSQL_with__37044_38963.png John_Zablocki_-_Code-First_NoSQL_with__38964_39011.png John_Zablocki_-_Code-First_NoSQL_with__39012_39059.png John_Zablocki_-_Code-First_NoSQL_with__39060_40703.png John_Zablocki_-_Code-First_NoSQL_with__40704_40763.png John_Zablocki_-_Code-First_NoSQL_with__40764_43151.png John_Zablocki_-_Code-First_NoSQL_with__43152_43331.png John_Zablocki_-_Code-First_NoSQL_with__43332_45479.png John_Zablocki_-_Code-First_NoSQL_with__45480_45647.png John_Zablocki_-_Code-First_NoSQL_with__45648_56699.png John_Zablocki_-_Code-First_NoSQL_with__56700_56747.png John_Zablocki_-_Code-First_NoSQL_with__56748_56807.png John_Zablocki_-_Code-First_NoSQL_with__56808_57707.png John_Zablocki_-_Code-First_NoSQL_with__57708_58319.png John_Zablocki_-_Code-First_NoSQL_with__58320_58367.png John_Zablocki_-_Code-First_NoSQL_with__58368_60599.png John_Zablocki_-_Code-First_NoSQL_with__60600_60887.png John_Zablocki_-_Code-First_NoSQL_with__60888_62579.png John_Zablocki_-_Code-First_NoSQL_with__62580_64499.png John_Zablocki_-_Code-First_NoSQL_with__64500_64847.png John_Zablocki_-_Code-First_NoSQL_with__64848_64907.png John_Zablocki_-_Code-First_NoSQL_with__64908_65759.png John_Zablocki_-_Code-First_NoSQL_with__65760_65807.png John_Zablocki_-_Code-First_NoSQL_with__65808_70247.png John_Zablocki_-_Code-First_NoSQL_with__70248_70367.png John_Zablocki_-_Code-First_NoSQL_with__70368_73139.png John_Zablocki_-_Code-First_NoSQL_with__73140_73271.png John_Zablocki_-_Code-First_NoSQL_with__73272_74231.png John_Zablocki_-_Code-First_NoSQL_with__74232_74519.png John_Zablocki_-_Code-First_NoSQL_with__74520_75527.png John_Zablocki_-_Code-First_NoSQL_with__75528_75575.png John_Zablocki_-_Code-First_NoSQL_with__75576_75623.png John_Zablocki_-_Code-First_NoSQL_with__75624_78455.png John_Zablocki_-_Code-First_NoSQL_with__78456_80459.png John_Zablocki_-_Code-First_NoSQL_with__80460_80831.png John_Zablocki_-_Code-First_NoSQL_with__80832_81035.png John_Zablocki_-_Code-First_NoSQL_with__81036_81083.png John_Zablocki_-_Code-First_NoSQL_with__81084_81131.png John_Zablocki_-_Code-First_NoSQL_with__81132_81191.png John_Zablocki_-_Code-First_NoSQL_with__81192_81395.png John_Zablocki_-_Code-First_NoSQL_with__81396_81443.png John_Zablocki_-_Code-First_NoSQL_with__81444_81875.png John_Zablocki_-_Code-First_NoSQL_with__81876_82595.png John_Zablocki_-_Code-First_NoSQL_with__82596_83087.png John_Zablocki_-_Code-First_NoSQL_with__83088_84203.png John_Zablocki_-_Code-First_NoSQL_with__84204_84479.png John_Zablocki_-_Code-First_NoSQL_with__84480_84563.png John_Zablocki_-_Code-First_NoSQL_with__84564_84743.png John_Zablocki_-_Code-First_NoSQL_with__84744_84899.png John_Zablocki_-_Code-First_NoSQL_with__84900_87035.png John_Zablocki_-_Code-First_NoSQL_with__87036_87167.png John_Zablocki_-_Code-First_NoSQL_with__87168_87227.png John_Zablocki_-_Code-First_NoSQL_with__87228_87407.png John_Zablocki_-_Code-First_NoSQL_with__87408_87491.png John_Zablocki_-_Code-First_NoSQL_with__87492_87611.png John_Zablocki_-_Code-First_NoSQL_with__87612_88499.png John_Zablocki_-_Code-First_NoSQL_with__88500_88775.png John_Zablocki_-_Code-First_NoSQL_with__88776_88871.png John_Zablocki_-_Code-First_NoSQL_with__88872_88991.png John_Zablocki_-_Code-First_NoSQL_with__88992_89195.png John_Zablocki_-_Code-First_NoSQL_with__89196_89495.png John_Zablocki_-_Code-First_NoSQL_with__89496_90023.png John_Zablocki_-_Code-First_NoSQL_with__90024_90359.png John_Zablocki_-_Code-First_NoSQL_with__90360_93359.png John_Zablocki_-_Code-First_NoSQL_with__93360_93551.png John_Zablocki_-_Code-First_NoSQL_with__93552_93611.png John_Zablocki_-_Code-First_NoSQL_with__93612_104399.png John_Zablocki_-_Code-First_NoSQL_with__104400_104447.png John_Zablocki_-_Code-First_NoSQL_with__104448_104519.png