Actually, the slide is 12 databases in 25 minutes (and 20 minutes of theory) by Eric Redmond (@inviite).
Complex data: A lot of data isn’t really complex, it’s just modeled in a complex way.
“Complexity is a symptom of confusion, not a cause” Jeff Hawkins.
NoSQL
Linear Scalability
Ability to be Distributed
Low Latency
SQL
Not NoSQL
ACID (transaction-based)
Atomic – Transactions are “all or nothing”
Consistent – system data will have integrity
Isolated – Transactions can’t see each other
Durability – changes aren’t lost
BASE (request based)
Basically Available
Soft state
Eventual consistency
Redmond says: ACID is negative (avoid negative things), BASE is mostly positive, things will be good, not great.
CAP theorem – it’s a fact 😀
Consistent, Available, Partition-tolerant web services. “It is impossible to reliably provide atomic, consistent data when there are partitions in the network. It is feasible, however, to achieve
Note that “consistent” is not the same as “consistent” in ACID, it’s more like Atomicity.
Strong consistency: when an update completes, subsequent access returns the new result. [to my mind this is actually durability]
Weak consistency – eventual consistency
“Correct consistency” – is the most important part. DNS, for example, is eventual consistency.
Common patterns:
Replication
CouchDB has an amazing ability to do this, Mongo is also good but not as good.
– copying data amongst nodes in a distributed database. Lazy (optimistic) replication, gossip (nodes communicate to stay in sync). – master/slave (mongo)
– master/master (riak, couch)
– vector clocks (keep track of write order per client
– mvcc (mysql)
N/R/W
N – Nodes to write to (per bucket)
R – Nodes read from before success
W – Nodes written to before success
Amazon Dynamo does this (Cassandra and Riak do this) – supports both CP and AP in one db (from the CAP theorem)
Consistent Hashing
Balance your servers, and when you hash your keys, if a server goes down or is added you don’t have to rebalance ALL nodes, just some % of them.
Mapreduce
Relational Models:
“Nothing beats relational databases for raw queryability.” The tradeoff — you have to structure your data and tell the system how it is structured.
PostgreSQL (full featured) – http://bitbucket.org/ged/ruby-pg, http://github.com/Casecommons/pg_search, http://github.com/tenderlove/texticle
MySQL (lighter) – http://gitub.com/oldmoe/mysqlplus, http://github.com/brianmario/mysql2
Drizzle (lightest) – http://www.drizzle.org
Bigtable/Columnar Style
What makes it columnar? well, a primary key is really a row key, and then you have column families, which are columns, stored together. (each column’s values are stored together as opposed to the row being stored together.) You can set expiry for a column family too, after which the data expires (which is why it’s great for Google).
HBase – http://hbase.apache.org – Google’s BigTable implementation, which was born of Hadoop (Java mapreduce engine). If you want to use HBase in production, use Thrift (http://thirft.apache.org) which Cassandra also uses). This is CP, but configurable to AP. Does sequential reads and column versioning, strong but flexible columnar schema.
Cassandra – hybrid. Node architecture like dynamo – data structure like BigTable w/column families – http://cassandra.apache.org – Good for hundreds of nodes in the same data center, if there is more than that or different data centers, use HBase (that’s what Digg and Facebook are running into). In cassandra you set up your schemas with an XML file, not with DDL. Benefits – sequential reads of ordered keys, also has versioning. It’s AP, configurable to CP.
Documentation Datastores:
MongoDB (AP focused – master/slave)
http://www.mongodb.org – created to be huge (huMONGous). Made to be partitioned, distributed, needed ad hoc queries. Wasn’t built to be durable.
CouchDB
Not made to be distributed, originally, was meant to be very durable. AP focused (master/master)
http://couchdb.apache.org
http://tilgovi.github.com/couchdb-lounge (clustering)
MapReduce in Mongo is an ad hoc query, comfortable for relational db ppl. In CouchDB, you make views and then request data from those views.
Riak – The most “architecturally cool” database out there. It’s a dynamo implementation that is purely REST based. It’s a key-value store, but it’s not descriptive enough — it has map-reduce built in, metadata and links you can walk. You can store ANYTHING in riak — not just text. example: getting a JPG file from the web and putting it as the value for the key “firefox.jpg”. Neat demo.
Riak has a ring, eventual consistency, can pull nodes in and take nodes out, without having to invalidate all the ids. It has quorum consistency, which blows Eric’s mind, but we didn’t have
Key/value stores
memcached – don’t use it
Kyoto Cabinet – don’t use it
Redis – use it – http://redis.io – it can handle lists, hashes, can intersect the value of 2 keys (such as person and pet, to find out who owns which set).
Graph datastores – you walk the graph instead of querying or doing mapreduce.
Neo4j
FlockDB – distributed, “unless you’re twitter, you don’t need to use it”. It’s not really possible to distribute a graph database, you can’t walk it and do node traversals, you can just walk edges (you can do friends, but not friends of friends, etc).
Slides are available at https://github.com/coderoshi/holy-grail-dbs