DS161 Introduction to Data Science & Artificial Intelligence
Self-Study 2
AI Evolution: Expert Systems to Generative AI
Krishnendu Ghosh
What is Artificial Intelligence?
AI is the science of building machines that can perform tasks requiring human-like intelligence. Understanding its evolution — from rigid rule-based logic to creative generative models — helps us grasp where the technology is headed and why it matters today.
AI Evolution Timeline
Five decades of innovation:
from hand-coded rules to AI that creates, reasons, and generates
RULE-BASED SYSTEMS
GENERATIVE AI
MACHINE LEARNING
DEEP LEARNING
EXPERT SYSTEMS
Ai Evolution: Early Foundations
01
02
03
Symbolic AI (1950s–60s)
Approach: Logic & symbols
Features: Hand-coded rules
Data: None required
Capability: Theorem proving
App: Chess, puzzles
Limit: Brittle, no learning
Rule-Based Systems (1960s–70s)
Approach: IF-THEN rules
Features: Explicit knowledge
Data: Expert input only
Capability: Simple reasoning
App: Early automation
Limit: Rule explosion
Expert Systems (1970s–80s)
Approach: Knowledge bases
Features: Inference engine
Data: Domain expert rules
Capability: Decision support
App: MYCIN, DENDRAL
Limit: Maintenance cost
Symbolic & Expert Systems Era
Key Approach & Features
Symbolic reasoning using formal logic and predicate calculus. Knowledge stored in hand-crafted rule bases (IF-THEN). Expert knowledge encoded by knowledge engineers.
Data & Learning Requirements
No statistical learning from data. All knowledge manually coded by domain experts. Required extensive human effort to build and maintain rule sets.
01
02
Capabilities & Applications
Medical diagnosis: MYCIN (bacterial infections). Chemical analysis: DENDRAL. Planning, theorem proving, and natural language parsing in narrow domains.
Limitations
Brittle outside defined rules. Could not handle uncertainty or incomplete data. Knowledge acquisition bottleneck. Failed to scale — led to the first AI Winter (1970s–80s).
03
04
1950s–1980s: AI built on logic, rules, and hand-coded knowledge. Machines reasoned like experts — without learning from data.
Machine Learning Revival
04
05
06
AI Winter
1980s–1990s
Approach: Rule-based collapse
Cause: Overpromising, underfunding
Limitation: Brittle, no learning
Transition: RULES → DATA
Machine Learning
1990s–2000s
Approach: Statistical learning
Feature: Learns from examples
Data: Labeled datasets needed
Apps: Spam filters, search
Big Data Era
2000s–2010s
Approach: Data-driven models
Feature: Scale & computation
Data: Massive datasets
Apps: Recommendations, ads
Machine Learning Era
Key Approach: Statistical Learning
Algorithms infer rules from data using probability, optimization, and regression — replacing hand-crafted logic with learned models (SVMs, decision trees, k-NN).
Data Requirement: Labeled Datasets
Supervised learning demands large volumes of manually labeled examples. Quality and quantity of training data directly determine model accuracy and generalization.
01
02
Main Feature: Pattern Recognition
Models identify statistical regularities across features. Applications include spam filters, recommendation engines, medical diagnosis, and image classification.
Limitation: Feature Engineering
Human experts must manually select and transform raw inputs into informative features. This bottleneck is costly, domain-specific, and hard to scale.
03
04
DEEP LEARNING REVOLUTION
01
02
03
Deep Learning (2010s)
Approach: Multi-layer neural networks
Features: CNNs, RNNs, backprop
Data: Massive labeled datasets
Capabilities: Image & speech recognition
Apps: Vision, translation, games
Limit: Black box, compute-heavy
Transformers (2017–2019)
Approach: Attention mechanisms
Features: Self-attention, parallelism
Data: Large text corpora
Capabilities: Context-aware language
Apps: BERT, GPT-2, translation
Limit: Huge memory & data needs
Foundation Models (2020)
Approach: Large-scale pre-training
Features: Transfer learning, fine-tune
Data: Web-scale unlabeled data
Capabilities: General-purpose tasks
Apps: GPT-3, CLIP, Codex
Limit: Bias, cost, interpretability
Deep Learning & Transformers
Key Approach & Features
Deep neural networks (CNNs, RNNs, LSTMs); self-attention & Transformer architecture; end-to-end learning from raw data.
Data & Learning Requirements
Massive labeled & unlabeled datasets; GPU/TPU clusters; pre-training on billions of tokens; fine-tuning for downstream tasks.
01
02
Capabilities & Applications
Image recognition (ResNet), machine translation, speech synthesis; BERT for NLP; GPT series for text generation; ViT for vision.
Limitations
High compute & energy costs; limited interpretability (black-box); data-hungry; prone to hallucination; poor out-of-distribution generalization.
03
04
Generative AI vs Multimodal
AI vs Agentic AI
01
02
Generative AI (2022–2024)
• Approach: Large-scale generative modeling
• Features: Text, image & code synthesis; creative output
• Data: Massive web-scale datasets (trillions of tokens)
• Capabilities: GPT-4, DALL·E, Stable Diffusion, Codex
• Applications: Writing, art, coding, Q&A, summarization
• Limitations: Hallucinations, bias, high compute cost, no persistent memory
Multimodal & Agentic AI (2024–2026)
• Approach: Cross-modal reasoning + autonomous action
• Features: Vision, audio, text unified; tool use & planning
• Data: Paired multimodal corpora; reinforcement feedback
• Capabilities: GPT-4o, Gemini, Claude 3; AI agents & copilots
• Applications: Autonomous research, robotics, personal agents
• Limitations: Alignment risk, safety gaps, unpredictable behavior
Key Takeaways