Basic Probability Rules and Concepts

Introduction

Key Concepts

1. Probability Fundamentals

Probability quantifies the chance of an event happening, ranging from 0 (impossible) to 1 (certain). It is expressed as:

$$ P(A) = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}} $$

For example, the probability of rolling a four on a fair six-sided die is:

$$ P(4) = \frac{1}{6} \approx 0.1667 \text{ or } 16.67\% $$

2. Sample Space and Events

The sample space, denoted as $S$, is the set of all possible outcomes of an experiment. An event is a subset of the sample space. For instance, when flipping a coin, the sample space is $S = \{ \text{Heads, Tails} \}$, and the event "getting Heads" is $A = \{ \text{Heads} \}$.

3. Complementary Events

The complement of an event $A$, denoted as $A'$, consists of all outcomes in the sample space that are not in $A$. The probability of the complement is:

$$ P(A') = 1 - P(A) $$

If the probability of event $A$ is 0.3, then:

$$ P(A') = 1 - 0.3 = 0.7 $$

4. Mutually Exclusive Events

Two events are mutually exclusive if they cannot occur simultaneously. In other words, if event $A$ occurs, event $B$ cannot, and vice versa. The probability of either event $A$ or event $B$ occurring is:

$$ P(A \text{ or } B) = P(A) + P(B) $$

For example, when rolling a die, events "rolling a 2" and "rolling a 5" are mutually exclusive:

$$ P(2 \text{ or } 5) = P(2) + P(5) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = \frac{1}{3} $$

5. Independent Events

Two events are independent if the occurrence of one does not affect the probability of the other. The probability of both independent events $A$ and $B$ occurring is:

$$ P(A \text{ and } B) = P(A) \times P(B) $$

For example, flipping a coin and rolling a die are independent events. The probability of getting Heads and rolling a 4 is:

$$ P(\text{Heads and } 4) = P(\text{Heads}) \times P(4) = 0.5 \times \frac{1}{6} = \frac{1}{12} $$

6. Dependent Events

Events are dependent if the occurrence of one event affects the probability of the other. The probability of both dependent events $A$ and $B$ occurring is:

$$ P(A \text{ and } B) = P(A) \times P(B|A) $$

Where $P(B|A)$ is the conditional probability of $B$ given that $A$ has occurred. For example, drawing cards from a deck without replacement makes the events dependent.

7. Conditional Probability

Conditional probability is the probability of an event occurring given that another event has already occurred. It is defined as:

$$ P(B|A) = \frac{P(A \text{ and } B)}{P(A)} $$>

Continuing the card example, if event $A$ is drawing an Ace and event $B$ is drawing a King after an Ace has been drawn, then:

$$ P(King|Ace) = \frac{P(Ace \text{ and } King)}{P(Ace)} = \frac{0}{\frac{4}{52}} = 0 $$>

Since you cannot draw a King immediately after an Ace in a single draw without replacement.

8. Addition Rule

The addition rule calculates the probability of either of two events occurring. For any two events $A$ and $B$:

$$ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $$>

This rule accounts for the overlap where both events occur.

9. Multiplication Rule

The multiplication rule determines the probability of two events occurring together. For dependent events:

$$ P(A \text{ and } B) = P(A) \times P(B|A) $$>

For independent events, it simplifies to:

$$ P(A \text{ and } B) = P(A) \times P(B) $$>

10. Permutations and Combinations

Permutations and combinations are techniques used to count the number of ways events can occur.

• Permutations consider the order of events. The number of permutations of $n$ distinct objects taken $r$ at a time is: $$nP r = \frac{n!}{(n-r)!}$$
• Combinations do not consider the order of events. The number of combinations of $n$ distinct objects taken $r$ at a time is: $$nC r = \frac{n!}{r!(n-r)!}$$

For example, the number of ways to arrange 3 books out of 5 is: $$ 5P3 = \frac{5!}{(5-3)!} = \frac{120}{2} = 60 $$>

11. Probability Distributions

A probability distribution assigns probabilities to each possible outcome in a sample space. There are discrete and continuous probability distributions.

• Discrete Probability Distributions deal with countable sample spaces. Examples include the Binomial and Poisson distributions.
• Continuous Probability Distributions handle uncountable sample spaces. An example is the Normal distribution.

Understanding probability distributions is crucial for modeling real-world phenomena and conducting statistical analyses.

12. Expected Value

The expected value is the long-term average outcome of a probability distribution. For a discrete random variable $X$ with possible values $x_i$ and corresponding probabilities $P(x_i)$:

$$ E(X) = \sum_{i} x_i \times P(x_i) $$>

For example, the expected value of rolling a fair six-sided die is:

$$ E(X) = \frac{1}{6}(1 + 2 + 3 + 4 + 5 + 6) = \frac{21}{6} = 3.5 $$>

13. Variance and Standard Deviation

Variance measures the dispersion of a set of probabilities from the expected value. It is calculated as:

$$ \text{Var}(X) = E[(X - E(X))^2] = \sum_{i} (x_i - E(X))^2 \times P(x_i) $$>

The standard deviation is the square root of the variance:

$$ \sigma_X = \sqrt{\text{Var}(X)} $$>

Both metrics are essential for understanding the variability in probability distributions.

14. Law of Large Numbers

The Law of Large Numbers states that as the number of trials increases, the experimental probability of an event will get closer to the theoretical probability. This principle underpins many statistical methods and ensures the reliability of probability-based predictions over numerous trials.

15. Bayes' Theorem

Bayes' Theorem relates the conditional and marginal probabilities of events. It is expressed as:

$$ P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} $$>

This theorem is pivotal in various fields, including statistics, machine learning, and decision-making processes.

Comparison Table

Concept	Definition	Application
Mutually Exclusive Events	Events that cannot occur simultaneously.	Determining the probability of either event A or event B occurring.
Independent Events	Events where the occurrence of one does not affect the other.	Calculating joint probabilities in separate trials, such as coin tosses.
Conditional Probability	The probability of an event given that another event has occurred.	Updating probability estimates based on new information.
Permutations	Ordered arrangements of objects.	Determining the number of possible sequences in arranging books.
Combinations	Unordered selections of objects.	Selecting committee members from a larger group.

Summary and Key Takeaways