Secondary Data in the Age of Big Data
Zikmund (2013) · Cooper & Schindler · Sekaran
01 · FOUNDATIONS
Secondary data: someone else collected it first
Data that others gathered and recorded before your project, and for a different purpose than yours (Zikmund, 2013).
Why use it
Faster and cheaper than collecting your own. Sometimes the only source, such as historical or national statistics. Helps sharpen the problem before primary research.
Watch out
It was built for someone else's question, so definitions, units, or time periods may not fit. Its accuracy is only as good as the original collector.
Two families
Internal: sales, CRM, accounting, data warehouses.
External: government statistics, industry reports, syndicated data, and the internet.
Source: Zikmund et al. (2013), Business Research Methods
2
02 · BIG DATA
Big data: secondary data, only bigger and messier
Volume
Millions of posts, pages, and images
Velocity
Created continuously, in near real time
Variety
Mostly unstructured: text, images, video
Textual data
News, posts, reviews, speeches, annual reports. Analyzed as words, topics, named entities, and sentiment.
Visual data
Photos, satellite images, video, charts, screenshots. Analyzed with image classification, object detection, and OCR.
WHERE TO START
1. Start from the research question, not from the dataset.
2. Pull a small sample and read it before you automate.
3. Check access, terms of service, and privacy before scaling up.
Secondary Data in the Age of Big Data
3
03 · SOURCES
Where to find data on the internet
News
Get: articles, headlines, dates, outlets
Access: library databases (Factiva, LexisNexis), GDELT, outlet archives, scraping
Mind: editorial slant, paywalls
Social media
Get: posts, comments, hashtags, engagement
Access: platform APIs (X, Reddit, YouTube), researcher access programs
Mind: bots, users who are not the population, terms of service
Websites
Get: company pages, prices, reviews, government portals (BPS, data.go.id)
Access: downloads, or scraping with BeautifulSoup or rvest
Mind: robots.txt, pages change
Wayback Machine
Get: past versions of web pages at web.archive.org
Access: browse by date, or list snapshots in bulk via the CDX API
Mind: gaps in coverage, missing images
Secondary Data in the Age of Big Data
4
04 · PROCESSING
From raw text to a dataset you can analyze
Collect
API, scraper, archive
→
Clean
HTML, duplicates, spam
→
Tokenize
words, stopwords, stems
→
Tag (NER)
ML finds names
→
Structure
rows and variables
EXAMPLE (FICTIONAL SENTENCE)
PT
Sinar
Kopi
opened
a
new
outlet
in
Bandung
on
3
March
2025
PT Sinar Kopi = ORG
Bandung = LOCATION
3 March 2025 = DATE
Tools: spaCy, NLTK, Hugging Face models (IndoBERT for Bahasa Indonesia), tidytext or quanteda in R
Secondary Data in the Age of Big Data
5
05 · TOOLS & ANALYSIS
Where you work, and how you test it
Tool | Best for |
Python | Scraping, NLP, machine learning |
R | Text mining and statistics in one language |
Jupyter Notebook | Code, output, and notes in one file |
Google Colab | Notebooks in the browser, nothing to install |
VS Code | Larger, reusable projects and scripts |
Stata | Regression and econometrics, clean output |
Text becomes variables
Mentions, entity counts, or sentiment per firm per month enter a regression like any other variable.
Sales = β0 + β1·Sentiment + β2·Price + ε
Stata
regress sales sentiment price, robust
R
lm(sales ~ sentiment + price, data = df)
Secondary Data in the Age of Big Data
6
06 · EVALUATION
Before you analyze anything, ask two questions
Applicability
Is it relevant to my specific research question?
Accuracy
Is it credible and trustworthy enough to use?
7
FRAMEWORK 1 · COOPER & SCHINDLER
The 5-factor evaluation matrix
Factor | Ask yourself | In web data, for example |
Purpose | Why does it exist? What commercial or political agenda could bias it? | A brand's own site vs an independent outlet |
Scope | Which time period, geography, population, and inclusion rules? | A hashtag covers only people who post publicly |
Authority | Who is the author or publisher? Is it primary, secondary, or tertiary? | An official BPS release vs a blog quoting it |
Audience | Who was it written for, and how does that shape tone, detail, and bias? | Investor report vs consumer ad copy |
Format | Can you find the details and load them into analysis software? | CSV or API access vs tables locked in a PDF image |
Source: Cooper & Schindler, Business Research Methods
8
FRAMEWORK 2 · ZIKMUND
Applicability: does it fit, or can it be reworked?
Definitions
Do key terms and categories match your population? Income bands or broad labels like "percent nonwhite" may not.
Units
Are measures comparable? Sales in cases, pounds, and dollars are not the same variable.
Time and detail
Does it cover your exact period, with enough detail rather than broad totals?
Reworking
If it doesn't match, can it be converted? Dollar sales ÷ unit price = physical volume.
In big data: a "follower" on one platform is not a "subscriber" on another, and a "post" may include reposts. Align definitions before you merge sources.
Source: Zikmund et al. (2013), Business Research Methods
9
FRAMEWORKS 3 & 4 · SEKARAN, ZIKMUND, SCHINDLER
Accuracy: can you trust it, and is it worth it?
External data: Sekaran & Zikmund
Timeliness
When was it collected? Fast-changing markets make old data useless.
Methodology
Who collected it, with what design, and what bias could creep in?
Cross-checking
Compare independent sources and explain the gaps between them.
Cost-benefit
Buying data vs the cost and effort of collecting primary data.
Internal data: Schindler
Compatibility
How closely do old records fit today's management dilemma?
Original quality
Were proper collection standards and protocols followed?
Compiler experience
How experienced were the people who recorded it?
Sources: Sekaran; Zikmund et al. (2013); Cooper & Schindler
10