History of AI

1. Inception of AI (1943 – 1955)

• The first work that is now generally recognized as AI was done by Warren McCulloch and Walter Pitts(1943).
• They drew on three sources:
• Knowledge of the basic physiology and function of neurons in the brain.
• A formal analysis of propositional logic (AND,OR,NOT,IMPLICATION).
• Turing’s theory of computation.

They proposed a model of artificial neurons in which each neuron is characterized by a sufficient number of neighbouring neurons.

HEBBIAN LEARNING: McCulloch and Pitts also suggested that suitably defined networks could learn.

Donald Hebb(1949) demonstrated a simple updating rule for modifying the connection strengths between neurons.

1. Inception of AI (1943 – 1955)

• 1950: Two UG students at Harward, Marvin Minsky and Dean Edmonds, Built the first neural network called SNARC which used 3000 vacuum tubes and a surplus automatic pilot mechanism from a B-24 bomber to simulate a network of 40 neurons.

​

• 1950: Alan Turing stressed on computing machine and intelligence, where he introduced Turing test, machine learning, genetic algorithms, reinforcement learning – it would be easy to create human level AI by learning algorithms.

2. The Birth of AI (1956)

• McCarthy convinced Minsky, Claude Shannon, and Nathaniel Rochester to help him bring together U.S. researchers interested in automata theory, neural nets, and the study of intelligence.
• John McCarthy introduced the term "Artificial Intelligence (AI)" for the first time. 1956 is called the Birth Year of Artificial Intelligence.
• An attempt was made to find how to make machines use language - NLP, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.

​

2. 1956: Logic Theorist-Allen Newell and Herbert Simon

“We have invented a computer program capable of thinking non-numerically and thereby solved the venerable mind–body problem.”

​

3. Early enthusiasm, great expectations (1952-1969)

Computers where designed for arithmetic operations and nothing else but AI researchers intellectual establishment, by large, preferred to believe that

“a machine can never do X”.

They focused on the tasks including games, puzzles, mathematics and IQ.

General Problem Solver, or GPS, Newell and Simon’s early success was followed up the GPS, unlike Logic Theorist, this program was designed from the start to imitate human problem-solving protocols.

Physical symbol system: hypothesis that suggest for “general intelligent action.” What they meant is that any system (human or machine) exhibiting intelligence must operate by manipulating data structures composed of symbols.

3. Early enthusiasm, great expectations(1952-1969)

At IBM, N Rochester and his colleagues produced some of the first AI programs.

1952, Arthur Samuel wrote a series of programs for checkers that eventually learned to play at a strong amateur level.

1958, In MIT AI lab, Mc-Carthy defined the high-level language Lisp, that has dominated the next 30 years programming language.

1958, Mc-Carthy published a paper entitled programs with common sense, in which he described the Advice Taker, a hypothetical program that can be seen as the first complete AI system.

1959: Herbert Gelernter constructed the Geometry Theorem Prover, which was able to prove theorems of mathematics that students would find quite tricky.

3. Early enthusiasm, great expectations(1952-1969)

4. A dose of reality (1966-1973)

• “Visible Future” Simon made more concrete predictions: that within 10 years a computer would be chess champion, and a significant mathematical theorem would be proved by machine. These predictions came true (or approximately true) within 40 years rather than 10.
• Simon’s overconfidence was due to the promising performance of early AI systems on simple examples. In almost all cases, however, these early systems turned out to fail miserably when tried out on wider selections of problems and on more difficult problems.

Reasons for failure?

1. Heuristic-based approach: Relied on human reasoning instead of efficient algorithms.
2. Intractable problems: Many AI problems were too computationally complex.
3. Microworld limitation: Worked only in small, controlled environments; failed to scale to the real world.
4. Perceptron limitation: Early neural networks could learn only simple patterns and had limited representation capability.

5. Knowledge-based systems: The key to power?(1969-1979)

• Early AI systems had adopted the general purpose search mechanism to solve the problem which proved to be weak as it did not scale up on large data.
• The first kind of difficulty arose because most early programs knew nothing of their subject matter, they succeeded by means of simple syntactic manipulations.
• The alternative to weak methods is to use more powerful, domain-specific knowledge that allows larger reasoning steps and can more easily handle typically occurring cases in narrow areas of expertise.
• DENDRAL program was an example of this type.
• Expert systems are more Knowledge-intensive systems, Stanford began Heuristic programming project(HPP) to understand how new method of expert system can be applied to other areas.
• MYCIN was system to diagnose the blood infection based on 450 rules and was better than junior doctors which employed calculus of uncertainty called certainty factors to fit well how doctors diagnose on impact of evidence on the diagnosis.

​

​

6. AI becomes an industry(1980-present)

• The first successful commercial expert system, R1, began operation at the Digital Equipment Corporation (McDermott, 1982).
• The program helped configure orders for new computer systems; by 1986, it was saving the company an estimated $40 million a year.
• In 1981, the Japanese announced the “Fifth Generation” project, a 10-year plan to build intelligent computers running Prolog.
• Overall, the AI industry boomed from a few million dollars in 1980 to billions of dollars in 1988, including hundreds of companies building expert systems, vision systems, robots, and software and hardware specialized for these purposes.
• Soon after that came a period called the “AIWinter,” in which many companies fell by the wayside as they failed to deliver on extravagant promises.

​

7. The return of neural networks (1986-present)

• Backpropagation algorithm revived neural networks.
• Neural networks learn from examples by adjusting weights.
• Competed with symbolic AI (Newell & Simon, McCarthy).
• Laid the foundation for Deep Learning.

8. AI adopts the scientific method (1987-present)

• Shift from expert systems to data-driven AI.
• Probability replaced Boolean logic.
• Machine Learning replaced hand-coded rules.
• Standard benchmarks introduced:
• UCI Repository, MNIST, ImageNet & COCO, LibriSpeech, SQuAD

9. The emergence of intelligent agents (1995-present)

• Focus shifted to complete intelligent agents.
• SOAR architecture is a cognitive architecture that models human problem-solving by representing the current situation (state), selecting an action (operator), and producing a new situation (result) until the goal is achieved.
• AI widely adopted in Internet applications:
• Search Engines
• Recommender Systems
• Web Bots
• Vision expanded toward:
• Human-Level AI
• Artificial General Intelligence (AGI)
• Friendly AI

10. The availability of very large data sets (2001-present)

• Data became more important than algorithms.
• Massive datasets improved AI performance.
• Examples: Web text , Images , Genomic data
• Success stories:
• Word Sense Disambiguation
• Banko & Brill (2001): More data > Better algorithms
• Learning from data reduced the knowledge bottleneck.
• AI became deeply embedded across all industries.