User Tools

Site Tools


notes:bayesian_classification

This is an old revision of the document!


Bayesian Classification

This page discusses the application of Bayes Theorem as a simple classifier for text and outlines the mathematical basis and the algorithmic approach.

Bayes' Theorem

A reference to the meaning of the notation described below:

$P(A)$ The unconditional probability of event A occurring.
$P(A \cap B)$ The unconditional probability of both events A and B occurring.
$P(A|B)$ The probability of event A occurring given that event B also occurs.

We start with the axiom of conditional probability:

$$ P(A \cap B) = P(A|B)P(B) $$

This encapsulates the multiplicative nature of conditional probabilities. Note that A and B can be swapped without affecting the meaning due to the commutativity of $P(A \cap B)$:

$$ P(A \cap B) = P(B|A)P(A) $$

Setting these two equal yields:

$$ P(A|B)P(B) = P(B|A)P(A) $$ $$ \Rightarrow P(A|B) = \frac{P(B|A)P(A)}{P(B)} $$

This assumes that $P(A) \not= 0$ and $P(B) \not= 0$. This is a simple statement of Bayes' Theorem.

notes/bayesian_classification.1363262218.txt.gz · Last modified: 2013/03/14 11:56 by andy