Sri Raghavendra Educational Institutions Society (R)

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

www.skit.org.in

Title: Searching, Sorting, Compression.

CO addressed: CO2

Course: Big Data Analytics

Presented by: Mr. P. Kiran kumar

Department: ISE

2

28/08/2026

/skit.org.in

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

Searching using MapReduce:

Implement a MapReduce program to search for a specific keyword in a file.

Input file: student.csv

Keyword: Jack

Output contains the matching line, file name, and position.

Input Data

1001,John,45

1002,Jack,39

1003,Alex,44

1004,Smith,38

1005,Bob,33

​

MapReduce Components

• Driver: WordSearcher.java
• Mapper: WordSearchMapper.java
• Reducer: WordSearchReducer.java

Flow: Input → Mapper → Shuffle & Sort → Reducer → Output

/ Setting the keyword

job.getConfiguration().set("keyword", "Jack");

​

// Input and output paths

FileInputFormat.setInputPaths(job,

new Path("/mapreduce/student.csv"));

​

FileOutputFormat.setOutputPath(job,

new Path("/mapreduce/output/search/"));

​

job.setNumReduceTasks(1);

​

System.exit(job.waitForCompletion(true) ? 0 : 1);

3

12/16/2024

/skit.org.in

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

Searching: Driver Program

WordSearcher.java

​

The Driver program configures the Hadoop job, sets the keyword, and specifies input/output paths.

​

Configuration conf = new Configuration();

Job job = new Job(conf, "Word Search");

job.setJarByClass(WordSearcher.class);

job.setMapperClass(WordSearchMapper.class);

job.setReducerClass(WordSearchReducer.class);

​

job.setOutputKeyClass(Text.class);

job.setOutputValueClass(Text.class);

​

job.setInputFormatClass(TextInputFormat.class);

job.setOutputFormatClass(TextOutputFormat.class);

4

28/08/2026

/skit.org.in

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

Searching: Mapper Program

WordSearchMapper.java

​

The Mapper:

• Gets the keyword from configuration.
• Reads each input line.
• Checks whether the line contains the keyword.
• Gets the input file name.
• Tracks the position of the line.
• Emits the matching line with file name and position.

​

Reducer – WordSearchReducer.java

​

The Reducer simply writes the filtered results.

​

5

28/08/2026

/skit.org.in

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

What is Compression?

Compression reduces the size of MapReduce output files.

Benefits

• Reduces storage space
• Speeds up data transfer across the network

Compression Codec

A codec is an implementation of a compression and decompression algorithm.

Example

GzipCodec → Provides Gzip compression.

Process

MapReduce Output

↓

Compression

↓

Smaller File

↓

Faster Transfer

6

28/08/2026

/skit.org.in

(Approved by AICTE, Accredited by NAAC, Affiliated to VTU, Karnataka)

Sri Krishna Institute of Technology

Compression: Configuration & Summary

​

Compression can be configured in the Driver program:

​

conf.setBoolean(

"mapred.output.compress", true);

​

conf.setClass(

"mapred.output.compression.codec",

GzipCodec.class,

CompressionCodec.class);