Accept $\theta^*$ with probability $a$; otherwise, $\theta^{(t+1)} = \theta^{(t)}$.

Pymc Results: Provides sampled posterior distributions, posterior means, standard deviations, and credible intervals.

Situation	Preferred Approach
Need explicit uncertainty about parameters	Bayesian (posterior distribution)
Have strong prior knowledge from past data or theory	Bayesian
Small datasets $\rightarrow$ need regularization from priors	Bayesian
Large datasets, computation expensive	Frequentist (MLE, quick optimization)
Model has conjugate priors $\rightarrow$ easy computation	Bayesian
High-dimensional models without easy priors	Frequentist (regularized regression)
Want simple hypothesis tests, p-values	Frequentist
MCMC or sampling computationally feasible	Bayesian

Posterior $\propto$ Likelihood $\times$ Prior.
Prior matters most when $n$ is small; as $n \rightarrow \infty$, likelihood dominates.
Conjugate priors $\rightarrow$ closed-form posterior (Beta-Binomial, Normal-Normal).
Posterior predictive integrates over parameter uncertainty.
Hierarchical models “borrow strength” across groups.
MCMC used when posterior lacks closed-form solution.
Credible interval = probability statement about parameter.

Frequentist example: "Estimate $\beta$ by maximizing likelihood (MLE)."
Bayesian example: "Prior: $\beta \sim N(0,1)$. Posterior shrinks toward prior when $n$ is small."
Hierarchical example: "Group-level intercepts $\alpha_j$ shrink toward population mean $\mu_\alpha$."

Geometry: Partitions feature space with axis-parallel lines/hyperplanes.
Splitting Criteria: Selects predictor and threshold to maximize "purity" gain (reduce "impurity").
- Gini Index: $1 - \sum_k (\text{Proportion of Class } k \text{ in Region } r)^2$. Favors larger partitions.
- Entropy: $-\sum_k (\text{Proportion of Class } k \text{ in Region } r) \log_2 (\text{Proportion of Class } k \text{ in Region } r)$. Penalizes impurity more.
- Classification Error: $1 - \max_k (\text{Proportion of Class } k \text{ in Region } r)$. Least sensitive to impurity.
$$ \text{Gain}(R) = m(R) - \frac{N_1}{N}m(R_1) - \frac{N_2}{N}m(R_2) $$ ($m$ is impurity measure, $N$ is number of samples).
Stopping Conditions: Prevents overfitting.
- Max depth, min samples per leaf, min impurity decrease.
- Increasing tree depth $\rightarrow$ lower bias, higher variance.

Impurity change: Parent node has 8 positives, 2 negatives: Gini = $1 - (0.8^2 + 0.2^2) = 0.32$. Split into: Left: 4 pos, 1 neg (Gini = 0.32); Right: 4 pos, 1 neg (Gini = 0.32). $\rightarrow$ No impurity reduction $\rightarrow$ bad split.
Identical X values with opposite labels: If $X = 3$ occurs for $Y = 0$ and $Y = 1 \rightarrow$ tree cannot achieve zero training error, even with infinite depth.