1 of 16

Introduction to Bayesian Networks

By Ram Reddy, Eric Feng

2 of 16

Introduction

Not bae; bayes

3 of 16

What is a Bayesian Network?

  • They can be used to build models from data
    • Probabilistic graphical model
  • Mechanizes use of Bayes’ theorem
  • Comparison to neural networks
    • Bayesian Networks
      • Better accuracy, less sensitive to small datasets
      • Easily adaptable
      • More easily understood and thus evaluated
    • Neural Networks
      • Faster model evaluation time
      • Can be constructed with incomplete data

4 of 16

What can it be used for?

  • Bayesian Networks are good for taking an event and predicting what a possible cause is
  • Predicting a disease based on the symptoms
    • Probabilities of diseases based on symptoms
  • Decision making under uncertainty
  • Anomaly detection
  • Great way to understand a model
  • Efficient

5 of 16

Bayesian Statistics

6 of 16

Bayesian Probability

  • Each variable is represented by an upper case letter like A and B
    • Number of possible values in variable is denoted by |A|
  • A set of variables is denoted by a bold uppercase letter like X
    • Number of variables in set is denoted by |X|
  • All variables in bayesian network is denoted by U
  • P(A) is probability of A
  • P(A, B) is joint probability of A and B
    • Union notation
  • P(A | B) is the conditional probability of A given B
    • P(A ∩ B) P(B)

​

​

7 of 16

Bayesian Graphs

  • Based on DAGs
    • Directed acyclic graphs are directed, with no directed cycles, hence acyclic
    • All edges travel down: topographically ordered, forms no closed loops
    • Reflects causal knowledge
  • Made up of a set of nodes (called structural specification)
    • Each node represents a variable like age and gender
    • Discrete variable (gender)
      • Can be male or female
    • Continuous variable
      • Age

8 of 16

Bayes’ Theorem

  • Factors are independent
  • Factors have an equal effect on the outcome
  • Finds the probability of A happening, given that B has occurred

​

​

​

  • Expanded using the chain rule:

​

​

  • Proportionality introduced

9 of 16

Implementing Bayes’ Theorem

10 of 16

Training Bayesian Networks

  • No weights or biases (only based on the data)
  • Manual construction
    • Making a DAG that uses the probability distributions for every node
    • Knowledge-driven
  • Automatic Construction
    • Inferring unobserved variables
    • Parameter learning
    • Structure learning
    • Data-driven
  • Rarely used by themselves for classification or regressive problems. Used with other algorithms.

11 of 16

Implementation

  • BNNs and BCNNs
  • Generalizes a model for real-world situations
    • Anomaly detection
    • Decreases standard variation
    • Increases confidence

12 of 16

Implementation continued

  • Bayes by Backprop
    • Introduced by Blundell et al (2015)
    • Gaussian distribution
    • Hyper-parameters: mean and standard deviation

13 of 16

Summary

14 of 16

Summary

  • Network is made up of nodes and edges
  • Can be constructed manually or automatically
  • Great way to represent random variables that may have relationships
  • Network uses distributions to model probabilities
    • Bayes’ theorem

​

15 of 16

Sources

Bayes Server. (n.d.). Bayesian networks - an introduction. Bayes Server. Retrieved April 7, 2021, from https://www.bayesserver.com/docs/introduction/bayesian-networks

Gandhi, R. (2018, May 5). Naive Bayes Classifier. Towards Data Science. https://towardsdatascience.com/naive-bayes-classifier-81d512f50a7c

Horný, M. (2014, April 18). Bayesian Networks [PDF]. https://www.bu.edu/sph/files/2014/05/bayesian-networks-final.pdf

Laumann, F. (2018, December 12). Bayesian Convolutional Neural Networks with Bayes by Backprop. Neural Space. https://medium.com/neuralspace/bayesian-convolutional-neural-networks-with-bayes-by-backprop-c84dcaaf086e

16 of 16

Sources

Shehu, A., Ph.D. (2018, May 2). Lecture 10: Bayesian Networks and Inference CS 580 (001) - Spring 2018 [PDF]. https://cs.gmu.edu/~ashehu/sites/default/files/cs580_Spring2018/LecBayesNetsAndInference.pdf

Wikipedia. (n.d.). Bayesian network. Wikipedia. Retrieved April 7, 2021, from https://en.wikipedia.org/wiki/Bayesian_network

Woolf, A. (2020, October 6). Bayesian Neural Networks: 3 Bayesian CNN. Towards Data Science. https://towardsdatascience.com/bayesian-neural-networks-3-bayesian-cnn-6ecd842eeff3

Zhang, R., & Bivens, A. J. (2007). Comparing the use of bayesian networks and neural networks in response time modeling for service-oriented systems. SOCP '07: Proceedings of the 2007 workshop on Service-oriented computing performance: aspects, issues, and approaches, 67-74. https://doi.org/10.1145/1272457.1272467