In [1]:
import graphviz
import numpy as np

1¶

Solve for the $p_{ij}$'s in terms of the $\beta$'s in Example 19.3.

Solution:

$$ \begin{align*} p_{00} &= \exp\{\beta_1\} \\ p_{10} &= \exp\{\beta_2\} \cdot p_{00} = \exp\{\beta_1 + \beta_2\} \\ p_{01} &= \exp\{\beta_3\} \cdot p_{00} = \exp\{\beta_1 + \beta_3\} \\ p_{02} &= \exp\{\beta_4\} \cdot p_{00} = \exp\{\beta_1 + \beta_4\} \\ p_{11} &= \frac{\exp\{\beta_5\} \cdot p_{01} \cdot p_{10}}{p_{00}} = \exp\{\beta_1 + \beta_2 + \beta_3 + \beta_5\} \\ p_{12} &= \frac{\exp\{\beta_6\} \cdot p_{02} \cdot p_{10}}{p_{00}} = \exp\{\beta_1 + \beta_2 + \beta_4 + \beta_6\} \\ \end{align*} $$

2¶

Prove Lemma 19.5:

A partition $(X_a, X_b, X_c)$ satisfies $X_b \amalg X_c \mid X_a$ if and only if $f(x_a, x_b, x_c) = g(x_a, x_b) h(x_a, x_c)$ for some functions $g$ and $h$.

Solution:

($\Rightarrow$): Suppose $X_b \amalg X_c \mid X_a$. Then,

$$ \begin{align*} f_{X_a, X_b, X_c}(x_a, x_b, x_c) &= f_{X_b, X_c \mid X_a}(x_b, x_c \mid x_a) f_{X_a}(x_a) \\ &= f_{X_b \mid X_a}(x_b \mid x_a) f_{X_c \mid X_a}(x_c \mid x_a) f_{X_a}(x_a) \tag{definition of conditional independence}\\ &= g(x_a, x_b) h(x_a, x_c), \end{align*} $$

letting $g(x_a, x_b) = f_{X_b \mid X_a}(x_b \mid x_a)$ and $h(x_a, x_c) = f_{X_c \mid X_a}(x_c \mid x_a) f_{X_a}(x_a)$.

($\Leftarrow$): Suppose $f(x_a, x_b, x_c) = g(x_a, x_b) h(x_a, x_c)$ for some functions $g$ and $h$. Then, for each fixed $X_a = x_a$, $f(x_b, x_c) = g'(x_b) h'(x_c)$ for some functions $g'$ and $h'$. By Theorem 2.33, $X_b \amalg X_c \mid X_a$.

3¶

Prove Lemma 19.9:

A graphical model is hierarchical but the reverse need not be true.

Solution:

Suppose a model is graphical with graph $\mathcal{G}$. We want to show $\psi_A = 0$ and $A \subset B$ implies $\psi_B = 0$.

Suppose $\psi_A = 0$. Since the model is graphical, there is an edge $\{i, j\} \subset A$ missing from $\mathcal{G}$. Suppose $A \subset B$. Then, $\{i,j\} \in B$. Therefore, since the model is graphical, $\psi_B = 0$.

The reverse need not be true: take Example 19.11. The graph is hierarchical but not graphical.

4¶

Consider random variables $(X_1, X_2, X_3, X_4)$. Suppose the log-density is

$$\log f(x) = \psi_{\emptyset}(x) + \psi_{12}(x) + \psi_{13}(x) + \psi_{24}(x) + \psi_{34}(x).$$

(a) Draw the graph $G$ for these variables.

(b) Write down all independence and conditional independence relations implied by the graph.

(c) Is this model graphical? Is it hierarchical?

Solution:

In [2]:
g = graphviz.Graph()
g.edge("X1", "X2")
g.edge("X1", "X3")
g.edge("X2", "X4")
g.edge("X3", "X4")
g
Out[2]:
%3 X1 X1 X2 X2 X1--X2 X3 X3 X1--X3 X4 X4 X2--X4 X3--X4

The independence and conditional independence relations implied by the graph are:

$$ \begin{align*} X_1 \amalg X_4 \mid X_2, X_3 \\ X_2 \amalg X_3 \mid X_1, X_4 \\ \end{align*} $$

The model is graphical. The missing edges are $\{1, 4\}$ and $\{2, 3\}$. Observe that no term in the log density expression contains either pair of indices (i.e., there is no $\psi_{14}$, $\psi_{124}$, $\psi_{23}$, etc.). Therefore, the model is graphical.

The model is not hierarchical. Observe $\psi_1 = 0$, but $\psi_{12} \neq 0$. Since $\{1\} \subset \{1,2\}$, this violates Definition 19.8.

5¶

Suppose that parameters $p(x_1, x_2, x_3)$ are proportional to the following values:

$$ \begin{array}{cccccc} \hline & x_2 & 0 & 0 & 1 & 1 \\ & x_3 & 0 & 1 & 0 & 1 \\ \hline x_1 & 0 & 2 & 8 & 4 & 16 \\ & 1 & 16 & 128 & 32 & 256 \\ \hline \end{array} $$

Find the $\psi$-terms for the log-linear expansion. Comment on the model.

Solution:

The log-linear expansion takes the form

$$ \log f(x) = \psi_{\emptyset}(x) + \psi_1(x_1) + \psi_2(x_2) + \psi_3(x_3) + \psi_{12}(x_1, x_2) + \psi_{13}(x_1, x_3) + \psi_{23}(x_2, x_3) + \psi_{123}(x_1, x_2, x_3). $$

where

$$ \psi_{\emptyset}(x) = \log p_{000} $$

6¶

Let $X_1, \dots, X_4$ be binary. Draw the independence graphs corresponding to the following log-linear models. Also, identify whether each is graphical and/or hierarchical (or neither).

(a) $\log f = 7 + 11 x_1 + 2 x_2 + 1.5 x_3 + 17 x_4$

(b) $\log f = 7 + 11 x_1 + 2 x_2 + 1.5 x_3 + 17 x_4 + 12 x_2 x_3 + 78 x_2 x_4 + 3 x_3 x_4 + 32 x_2 x_3 x_4$

(C) $\log f = 7 + 11 x_2 + 2 x_2 + 1.5 x_3 + 17 x_4 + 12 x_2 x_3 + 3 x_3 x_4 + x_1 x_4 + 2 x_1 x_2$

(d) $\log f = 7 + 5055 x_1 x_2 x_3 x_4$