1 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

III B.Tech II Sem

Subject: NoSQL Databases Code: 23AD08

UNIT-4

Topic: Column Family Data store

2 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

UNIT-IV: Column-oriented NoSQL databases using Apache HBASE, Column-oriented NoSQL databases using Apache Cassandra, Architecture of HBASE, Column-Family Data Store Features, Consistency, Transactions, Availability, Query Features, Scaling, Suitable Use Cases, Event Logging, Content Management Systems, Blogging Platforms, Counters, Expiring Usage.

Course Outcomes: At the end of the Course the student will be able to

CO1: Explain and compare different types of NoSQL Databases

CO2: Compare and contrast RDBMS with different NoSQL databases.

CO3: Demonstrate the detailed architecture and performance tune of Document-oriented NoSQL databases.

CO4 : Explain performance tune of Key-Value Pair NoSQL databases.

CO5: Apply NoSQL development tools on different types of NoSQL Databases.

NOSQL DATABASES

3 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Database are classified into 2 types
Relational Database
Non Relational Database

4 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Data store are classified into 2 types
Row Oriented Data store or Row Oriented Database
Column Oriented store or Column Oriented Database

5 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Row vs Column Oriented Databases

6 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

APACHE HBASE

Since 1970, RDBMS is the solution for data storage and maintenance related problems.
After the advent of big data, companies realized the benefit of processing big data and started opting for solutions like Hadoop.
Hadoop uses distributed file system for storing big data, and MapReduce to process it.
Hadoop excels in storing and processing of huge data of various formats such as arbitrary, semi-, or even unstructured.

Limitations of Hadoop

Hadoop can perform only batch processing, and data will be accessed only in a sequential manner. That means one has to search the entire dataset even for the simplest of jobs.

A huge dataset when processed results in another huge data set, which should also be processed sequentially. At this point, a new solution is needed to access any point of data in a single unit of time (random access).

7 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

APACHE HBASE

Apache HBase is a distributed, scalable, NoSQL database built on top of the Hadoop Distributed File System (HDFS).
It is modeled after Google's Bigtable and is designed for storing large volumes of sparse, unstructured, or semi-structured data across clusters.
HBase is column-oriented, supports horizontal scaling, and allows for real-time read/write access, making it an essential component of the Hadoop ecosystem.
Example: Facebook migrated from Cassandra to HBase in 2010 to power its messaging infrastructure, needing a scalable, real-time system to unify chat, email, and SMS conversations.

8 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

HBase evolution flowchart:

Year	Event
Nov 2006	Google released the paper on BigTable.
Feb 2007	Initial HBase prototype was created as a Hadoop contribution.
Oct 2007	The first usable HBase along with Hadoop 0.15.0 was released.
Jan 2008	HBase became the sub project of Hadoop.
Oct 2008	HBase 0.18.1 was released.
Jan 2009	HBase 0.19.0 was released.
Sept 2009	HBase 0.20.0 was released.
May 2010	HBase became Apache top-level project.

9 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Key Features of Apache Hbase

Column-Oriented Storage: Efficient for queries on large datasets with flexible schemas.
Horizontal Scalability: Easily scales across multiple nodes by adding more servers.
Strong Consistency: Provides consistent reads and writes (no eventual consistency).
Atomic Operations: Ensures atomicity at the row level.
Rich APIs: Offers Java APIs and supports Thrift & REST APIs for integration with non-Java platforms.
Built-in Fault Tolerance: Automatically recovers from node failures using region server replication.
Caching Support: Includes Block Cache and Bloom Filters for faster data retrieval.
Metric Export: Supports exporting performance metrics through Hadoop's metrics subsystem.

10 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Architecture of Apache HBase

Apache HBase follows a master-slave architecture and is built on top of Hadoop HDFS. Here's how its major components work together:

11 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Here’s how each part of the Apache HBase architecture works in detail:

1. HMaster

Acts as the master node of the HBase cluster.

Its main responsibilities include:

Coordinating RegionServers
Assigning regions to RegionServers
Monitoring RegionServer health
Handling schema changes, like creating or deleting tables
Performing load balancing so no RegionServer is overloaded

If the HMaster fails, a backup HMaster can take over to ensure high availability.

12 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Here’s how each part of the Apache HBase architecture works in detail:

2. RegionServer

A worker node in HBase, serving client requests.

Each RegionServer manages multiple regions, meaning chunks of tables. Internally, RegionServers have:

MemStore: stores data in memory for fast writes before persisting to disk
HFile: permanent storage format in HDFS
BlockCache: caches frequently read data to improve read performance

RegionServers also manage WAL (Write Ahead Log) for crash recovery.If a RegionServer fails, the HMaster reassigns its regions to other RegionServers.

13 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Here’s how each part of the Apache HBase architecture works in detail:

3. Region

A region is a horizontal partition of an HBase table (like a subset of rows).
Each region contains data for a continuous range of row keys.
When a region grows too large (default around 10GB), it automatically splits into smaller regions.
Only one RegionServer handles a given region at a time to avoid conflicts.

14 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Here’s how each part of the Apache HBase architecture works in detail:

4. ZooKeeper

It's an external, reliable coordination service.

HBase uses ZooKeeper to:

Track available RegionServers
Help clients locate regions quickly
Manage master election in case of failure
Provide distributed configuration and synchronization
Without ZooKeeper, the cluster cannot coordinate properly.

15 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Here’s how each part of the Apache HBase architecture works in detail:

5. HDFS (Hadoop Distributed File System)

HBase uses HDFS to store actual data on disk. It stores HFiles which are compressed files with the actual data and also WAL (Write Ahead Logs) files for durability. It Provides fault-tolerant, distributed storage so data is protected even if hardware fails.

Massive Scalability – It Can handle extremely large datasets and seamlessly expand by adding more nodes without affecting performance.
High Throughput – Supports fast read/write operations, making it suitable for applications requiring quick processing of huge volumes of data.
Schema Flexibility – Allows storing data without a fixed schema, enabling easy handling of unstructured and semi-structured data.
Fault Tolerant – Ensures data reliability and system availability by automatically replicating data across multiple nodes.
Flexible Distributed Setup – Designed for large scale systems, offering high control for experts in Hadoop and distributed architectures.
Efficient Query Processing for Big Data – Optimized for large datasets using non-SQL approaches suited for scalable applications.
High Performance Without Heavy Transaction Overhead – Ideal for scenarios where speed is prioritized over strict transactional consistency.
Optimized for Simple, High-Speed Operations – Best suited for workloads that do not require complex joins or real-time streaming.

16 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Difference Between Hadoop and HBase

Hadoop: Hadoop is an open source framework from Apache that is used to store and process large datasets distributed across a cluster of servers. Four main components of Hadoop are Hadoop Distributed File System(HDFS), Yarn, MapReduce, and libraries. It involves not only large data but a mixture of structured, semi-structured, and unstructured information. Amazon, IBM, Microsoft, Cloudera, ScienceSoft, Pivotal, Hortonworks are some of the companies using Hadoop technology.

HBase: HBase is an open source database from Apache that runs on Hadoop cluster. It falls under the non-relational database management system. Three important components of HBase are HMaster, Region server, Zookeeper. CapitalOne, JPMorganchase, apple, MTB, AT& T, Lockheed Martin are some of the companies using HBase.

17 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

S.No.	Hadoop	HBase
1	Hadoop is a collection of software tools	HBase is a part of hadoop eco-system
2	Stores data sets in a distributed environment	Stores data in a column-oriented manner
3	Hadoop is a framework	HBase is a NOSQL database
4	Data are stored in form of chunks	Data are stored in form of key/value pair
5	Hadoop does not allow run time changes	HBase allows run time changes
6	File can be written only once, can be read many times	File can be read and write multiple times
7	Hadoop has low latency operations	HBase has high latency operations
8	HDFS can be accessed through MapReduce	HBase can be accessed through shell commands, Java API, REST

Below is a table of differences between Hadoop and HBase:

18 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

19 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Column-Family Stores

Column-family stores, such as Cassandra [Cassandra], HBase [Hbase], Hypertable [Hypertable], and Amazon SimpleDB [Amazon SimpleDB], allow you to store data with keys mapped to values and the values grouped into multiple column families, each column family being a map of data.

20 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

What Is a Column-Family Data Store?

There are many column-family databases.
In this chapter, we will talk about Cassandra but also reference other column-family databases to discuss features that may be of interest in particular scenarios.
Column-family databases store data in column families as rows that have many columns associated with a row key (Figure 10.1).
Column families are groups of related data that is often accessed together.
For a Customer, we would often access their Profile information at the same time, but not their Orders.

21 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Cassandra is one of the popular column-family databases;
there are others, such as HBase, Hypertable, and Amazon DynamoDB [Amazon DynamoDB].
Cassandra can be described as fast and easily scalable with write operations spread across the cluster.
The cluster does not have a master node, so any read and write can be handled by any node in the cluster.

22 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Features

Let’s start by looking at how data is structured in Cassandra.
The basic unit of storage in Cassandra is a column.
A Cassandra column consists of a name-value pair where the name also behaves as the key.
Each of these key-value pairs is a single column and is always stored with a timestamp value.
The timestamp is used to expire data, resolve write conflicts, deal with stale data, and do other things.
Once the column data is no longer used, the space can be reclaimed later during a compaction phase.

23 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Features

The column has a key of firstName and the value of Martin and has a timestamp attached to it.
A row is a collection of columns attached or linked to a key; a collection of similar rows makes a column family.
When the columns in a column family are simple columns, the column family is known as standard column family.

24 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Each column family can be compared to a container of rows in an RDBMS table where the key identifies the row and the row consists on multiple columns.
The difference is that various rows do not have to have the same columns, and columns can be added to any row at any time without having to add it to other rows.
We have the pramod-sadalage row and the martin-fowler row with different columns; both rows are part of the column family.
When a column consists of a map of columns, then we have a super column.
A super column consists of a name and a value which is a map of columns. Think of a super column as a container of columns.

25 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

When we use super columns to create a column family, we get a super column family.

26 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

27 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Super column families are good to keep related data together, but when some of the columns are not needed most of the time, the columns are still fetched and deserialized by Cassandra, which may not be optimal.
Cassandra puts the standard and super column families into keyspaces.
A keyspace is similar to a database in RDBMS where all column families related to the application are stored.
Keyspaces have to be created so that column families can be assigned to them:
create keyspace ecommerce

28 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Consistency

When a write is received by Cassandra, the data is first recorded in a commit log, then written to an in-memory structure known as memtable.
A write operation is considered successful once it’s written to the commit log and the memtable.
Writes are batched in memory and periodically written out to structures known as SSTable.
SSTables are not written to again after they are flushed; if there are changes to the data, a new SSTable is written.
Unused SSTables are reclaimed by compactation.

29 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Let’s look at the read operation to see how consistency settings affect it.
If we have a consistency setting of ONE as the default for all read operations, then when a read request is made, Cassandra returns the data from the first replica, even if the data is stale.
If the data is stale, subsequent reads will get the latest (newest) data; this process is known as read repair.
The low consistency level is good to use when you do not care if you get stale data and/or if you have high read performance requirements

Consistency

Similarly, if you are doing writes, Cassandra would write to one node’s commit log and return a response to the client.
The consistency of ONE is good if you have very high write performance requirements and also do not mind if some writes are lost, which may happen if the node goes down before the write is replicated to other nodes.

30 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Using the QUORUM consistency setting for both read and write operations ensures that majority of the nodes respond to the read and the column with the newest timestamp is returned back to the client, while the replicas that do not have the newest data are repaired via the read repair operations.
During write operations, the QUORUM consistency setting means that the write has to propagate to the majority of the nodes before it is considered successful and the client is notified.

31 of 42

DEPT.OF AI&DS V.SOWJANYA, SR.ASST.PROF.,

Using ALL as consistency level means that all nodes will have to respond to reads or writes, which will make the cluster not tolerant to faults—even when one node is down, the write or read is blocked and reported as a failure.
It’s therefore upon the system designers to tune the consistency levels as the application requirements change.
Within the same application, there may be different requirements of consistency; they can also change based on each operation, for example showing review comments for a product has different consistency requirements compared to reading the status of the last order placed by the customer.

1 of 42

2 of 42

3 of 42

4 of 42

5 of 42

6 of 42

7 of 42

8 of 42

9 of 42

10 of 42

11 of 42

12 of 42

13 of 42

14 of 42

15 of 42

16 of 42

17 of 42

18 of 42

19 of 42

20 of 42

21 of 42

22 of 42

23 of 42

24 of 42

25 of 42

26 of 42

27 of 42

28 of 42

29 of 42

30 of 42

31 of 42

32 of 42

33 of 42

34 of 42

35 of 42

36 of 42

37 of 42

38 of 42

39 of 42

40 of 42

41 of 42

42 of 42