diff --git a/lectures/likelihood_bayes.md b/lectures/likelihood_bayes.md index d85aaae6a..2787dbe1b 100644 --- a/lectures/likelihood_bayes.md +++ b/lectures/likelihood_bayes.md @@ -196,7 +196,13 @@ l_seq_f = np.cumprod(l_arr_f, axis=1) -## Likelihood Ratio Process and Bayes’ Law +## Likelihood Ratio Processes and Bayes’ Law + +Let $\pi_0 \in [0,1]$ be a Bayesian statistician's prior probability that nature generates $w^t$ as a sequence of i.i.d. draws from +distribution $f$. + +* here "probability" is to be interpreted as a way to summarize or express a subjective opinion +* it does **not** mean an anticipated relative frequency as sample size grows without limit Let $\pi_{t+1}$ be a Bayesian posterior probability defined as @@ -225,11 +231,26 @@ With no data in hand, our Bayesian statistician thinks that the probability dens $$ -{\rm Prob}(w^{t+1} |\emptyset) = \pi_0 f(w^{t+1})+ (1 - \pi_0) +{\rm Prob}(w^{t+1} |\emptyset) = \pi_0 f(w^{t+1})+ (1 - \pi_0) g(w^{t+1}) +$$ + +Laws of probability say that the joint distribution ${\rm Prob}(AB)$ of events $A$ and $B$ are connected to the conditional distributions +${\rm Prob}(A |B)$ and ${\rm Prob}(B |A)$ by + +$$ +{\rm Prob}(AB) = {\rm Prob}(A |B) {\rm Prob}(B) = {\rm Prob}(B |A) {\rm Prob}(A) . +$$ (eq:problawAB) + +We are interested in events + +$$ +A = \{q=f\}, \quad B = \{w^{t+1}\}, \quad $$ -Probability laws connecting joint probability distributions and conditional probability distributions imply that +where braces $\{\cdot\}$ are our shorthand for "event". +So in our setting, probability laws {eq}`eq:problawAB` imply that + $$ {\rm Prob}(q=f |w^{t+1}) {\rm Prob}(w^{t+1} |\emptyset) = {\rm Prob}(w^{t+1} |q=f) {\rm Prob}(q=f | \emptyset) $$ @@ -293,7 +314,7 @@ Dividing both the numerator and the denominator on the right side of the equat ```{math} :label: eq_recur1 -\pi_{t+1}=\frac{\pi_{t} l_t(w_{t+1})}{\pi_{t} l_t(w_t)+1-\pi_{t}} +\pi_{t+1}=\frac{\pi_{t} l_t(w_{t+1})}{\pi_{t} l_t(w_{t+1})+1-\pi_{t}} ``` with $\pi_{0}$ being a Bayesian prior probability that $q = f$, @@ -412,7 +433,7 @@ np.abs(π_seq - π_seq_f).max() < 1e-10 ``` We thus conclude that the likelihood ratio process is a key ingredient of the formula {eq}`eq_Bayeslaw1033` for -a Bayesian's posterior probabilty that nature has drawn history $w^t$ as repeated draws from density +a Bayesian's posterior probability that nature has drawn history $w^t$ as repeated draws from density $f$. @@ -425,8 +446,11 @@ Until now we assumed that before time $1$ nature somehow chose to draw $w^t$ as Nature's decision about whether to draw from $f$ or $g$ was thus **permanent**. -We now assume a different timing protocol in which before **each period** $t =1, 2, \ldots$ nature flips an $x$-weighted coin and with probability -$x \in (0,1)$ draws from $f$ in period $t$ and with probability $1 - x $ draws from $g$. +We now assume a different timing protocol in which before **each period** $t =1, 2, \ldots$ nature + +* flips an $x$-weighted coin, then +* draws from $f$ if it has drawn a "head" +* draws from $g$ if it has drawn a "tail". Under this timing protocol, nature draws permanently from **neither** $f$ **nor** $g$, so a statistician who thinks that nature is drawing i.i.d. draws **permanently** from one of them is mistaken. @@ -479,7 +503,7 @@ Let's generate a sequence of observations from this mixture model with a true mi We will first use this sequence to study how $\pi_t$ behaves. ```{note} -Later, we can use it to study how a statistician who knows that an $x$-mixture of $f$ and $g$ could construct maximum likelihood or Bayesian estimators of $x$ along with the free parameters of $f$ and $g$. +Later, we can use it to study how a statistician who knows that nature generates data from an $x$-mixture of $f$ and $g$ could construct maximum likelihood or Bayesian estimators of $x$ along with the free parameters of $f$ and $g$. ``` ```{code-cell} ipython3 @@ -563,7 +587,7 @@ print(f'KL(m, f) = {KL_f:.3f}\nKL(m, g) = {KL_g:.3f}') Since $KL(m, f) < KL(m, g)$, $f$ is "closer" to the mixture distribution $m$. Hence by our discussion on KL divergence and likelihood ratio process in -{doc}`likelihood_ratio_process`, $log(L_t) \to \infty$ as $t \to \infty$. +{doc}`likelihood_ratio_process`, $\log(L_t) \to \infty$ as $t \to \infty$. Now looking back to the key equation {eq}`eq_Bayeslaw1033`. @@ -611,7 +635,7 @@ The worker's initial beliefs induce a joint probability distribution Bayes' law is simply an application of laws of probability to compute the conditional distribution of the $t$th draw $w_t$ conditional on $[w_0, \ldots, w_{t-1}]$. -After our worker puts a subjective probability $\pi_{-1}$ on nature having selected distribution $F$, we have in effect assumes from the start that the decision maker **knows** the joint distribution for the process $\{w_t\}_{t=0}$. +After our worker puts a subjective probability $\pi_{-1}$ on nature having selected distribution $F$, we have in effect assumed from the start that the decision maker **knows** the joint distribution for the process $\{w_t\}_{t=0}$. We assume that the worker also knows the laws of probability theory. @@ -632,7 +656,7 @@ $$ Let $a \in \{ f, g\} $ be an index that indicates whether nature chose permanently to draw from distribution $f$ or from distribution $g$. After drawing $w_0$, the worker uses Bayes' law to deduce that -the posterior probability $\pi_0 = {\rm Prob} ({a = f | w_0}) $ +the posterior probability $\pi_0 = {\rm Prob}({a = f | w_0}) $ that the density is $f(w)$ is $$ @@ -691,7 +715,7 @@ Because $\{\pi_t\}$ is a bounded martingale sequence, it follows from the **mart Practically, this means that probability one is attached to sample paths $\{\pi_t\}_{t=0}^\infty$ that converge. -According to the theorem, it different sample paths can converge to different limiting values. +According to the theorem, different sample paths can converge to different limiting values. Thus, let $\{\pi_t(\omega)\}_{t=0}^\infty$ denote a particular sample path indexed by a particular $\omega \in \Omega$. @@ -908,7 +932,7 @@ $w_t$'s and the $\pi_t$ sequences that gave rise to them. Notice that one of the paths involves systematically higher $w_t$'s, outcomes that push $\pi_t$ upward. -The luck of the draw early in a simulation push the subjective distribution to draw from +The luck of the draw early in a simulation pushes the subjective distribution to draw from $F$ more frequently along a sample path, and this pushes $\pi_t$ toward $0$. ```{code-cell} ipython3 @@ -938,7 +962,7 @@ In the following table, the left column in bold face reports an assumed value of The second column reports the fraction of $N = 10000$ simulations for which $\pi_{t}$ had converged to $0$ at the terminal date $T=500$ for each simulation. -The third column reports the fraction of $N = 10000$ simulations for which $\pi_{t}$ had converged to $1$ as the terminal date $T=500$ for each simulation. +The third column reports the fraction of $N = 10000$ simulations for which $\pi_{t}$ had converged to $1$ at the terminal date $T=500$ for each simulation. ```{code-cell} ipython3 # create table @@ -994,7 +1018,7 @@ ax.set_ylabel(r'$\sigma^{2}(\pi_{t}\vert \pi_{t-1})$') plt.show() ``` -The shape of the the conditional variance as a function of $\pi_{t-1}$ is informative about the behavior of sample paths of $\{\pi_t\}$. +The shape of the conditional variance as a function of $\pi_{t-1}$ is informative about the behavior of sample paths of $\{\pi_t\}$. Notice how the conditional variance approaches $0$ for $\pi_{t-1}$ near either $0$ or $1$. diff --git a/lectures/likelihood_ratio_process.md b/lectures/likelihood_ratio_process.md index daa23ed6d..a2037eae6 100644 --- a/lectures/likelihood_ratio_process.md +++ b/lectures/likelihood_ratio_process.md @@ -31,14 +31,14 @@ kernelspec: This lecture describes likelihood ratio processes and some of their uses. -We'll study the same setting that is also used in {doc}`this lecture on exchangeability `. +We'll study the same setting that is also used in {doc}`this lecture on exchangeability `. Among the things that we'll learn are * How a likelihood ratio process is a key ingredient in frequentist hypothesis testing * How a **receiver operator characteristic curve** summarizes information about a false alarm probability and power in frequentist hypothesis testing * How a statistician can combine frequentist probabilities of type I and type II errors to form posterior probabilities of mistakes in a model selection or in an individual-classification problem -* How likelihood ratios helped Lawrence Blume and David Easley formulate an answer to ''If you're so smart, why aren't you rich?'' {cite}`blume2006if` +* How likelihood ratios helped Lawrence Blume and David Easley formulate an answer to ''If you're so smart, why aren't you rich?'' {cite}`blume2006if` * How to use a Kullback-Leibler divergence to quantify the difference between two probability distributions with the same support * How during World War II the United States Navy devised a decision rule for doing quality control on lots of ammunition, a topic that sets the stage for {doc}`this lecture ` * A peculiar property of likelihood ratio processes @@ -441,7 +441,7 @@ moves toward $-\infty$ when $g$ is the data generating process, while log$(L(w^t))$ goes to $\infty$ when data are generated by $f$. -That disparate behavior of log$(L(w^t))$ under $f$ and $q$ +That disparate behavior of log$(L(w^t))$ under $f$ and $g$ is what makes it possible eventually to distinguish $q=f$ from $q=g$. @@ -783,18 +783,18 @@ In the [next section](hetero_agent), we will see an application of these ideas. (hetero_agent)= ## Heterogeneous Beliefs and Financial Markets -A likelihood ratio process lies behind Lawrence Blume and David Easley's answer to their question +A likelihood ratio process lies behind Lawrence Blume and David Easley's answer to their question ''If you're so smart, why aren't you rich?'' {cite}`blume2006if`. -Blume and Easley constructed formal models to study how differences of opinions about probabilities governing risky income processes would influence outcomes and be reflected in prices of stocks, bonds, and insurance policies that individuals use to share and hedge risks. +Blume and Easley constructed formal models to study how differences of opinions about probabilities governing risky income processes would influence outcomes and be reflected in prices of stocks, bonds, and insurance policies that individuals use to share and hedge risks. ```{note} -{cite}`alchian1950uncertainty` and {cite}`friedman1953essays` can conjectured that, by rewarding traders with more realistic probability models, competitive markets in financial securities put wealth in the hands of better informed traders and help -make prices of risky assets reflect realistic probability assessments. +{cite}`alchian1950uncertainty` and {cite}`friedman1953essays` can conjectured that, by rewarding traders with more realistic probability models, competitive markets in financial securities put wealth in the hands of better informed traders and help +make prices of risky assets reflect realistic probability assessments. ``` -Here we'll provide an example that illustrates basic components of Blume and Easley's analysis. +Here we'll provide an example that illustrates basic components of Blume and Easley's analysis. We'll focus only on their analysis of an environment with complete markets in which trades in all conceivable risky securities are possible. @@ -803,14 +803,18 @@ We'll study two alternative arrangements: * perfect socialism in which individuals surrender their endowments of consumption goods each period to a central planner who then dictatorially allocates those goods * a decentralized system of competitive markets in which selfish price-taking individuals voluntarily trade with each other in competitive markets -The fundamental theorems of welfare economics will apply and assure us that these two arrangements end up producing exactly the same allocation of consumption goods to individuals **provided** that the social planner assigns an appropriate set of **Pareto weights**. +The fundamental theorems of welfare economics will apply and assure us that these two arrangements end up producing exactly the same allocation of consumption goods to individuals **provided** that the social planner assigns an appropriate set of **Pareto weights**. +```{note} +You can learn about how the two welfare theorems are applied in modern macroeconomic models in {doc}`this lecture on a planning problem ` and {doc}`this lecture on a related competitive equilibrium `. +``` +### The setting -Let the random variable $s_t \in (0,1)$ at time $t =0, 1, 2, \ldots$ be distributed according to the same Beta distribution with parameters +Let the random variable $s_t \in (0,1)$ at time $t =0, 1, 2, \ldots$ be distributed according to the same Beta distribution with parameters $\theta = \{\theta_1, \theta_2\}$. -We'll denote this probability density as +We'll denote this probability density as $$ \pi(s_t|\theta) @@ -818,7 +822,7 @@ $$ Below, we'll often just write $\pi(s_t)$ instead of $\pi(s_t|\theta)$ to save space. -Let $s_t \equiv y_t^1$ be the endowment of a nonstorable consumption good that a person we'll call "agent 1" receives at time $t$. +Let $s_t \equiv y_t^1$ be the endowment of a nonstorable consumption good that a person we'll call "agent 1" receives at time $t$. Let a history $s^t = [s_t, s_{t-1}, \ldots, s_0]$ be a sequence of i.i.d. random variables with joint distribution @@ -841,7 +845,7 @@ Nature draws i.i.d. sequences $\{s_t\}_{t=0}^\infty$ from $\pi_t(s^t)$. * so $\pi$ without a superscript is nature's model * but in addition to nature, there are other entities inside our model -- artificial people that we call "agents" * each agent has a sequence of probability distributions over $s^t$ for $t=0, \ldots$ -* agent $i$ thinks that nature draws i.i.d. sequences $\{s_t\}_{t=0}^\infty$ from $\pi_t^i(s^t)$ +* agent $i$ thinks that nature draws i.i.d. sequences $\{s_t\}_{t=0}^\infty$ from $\{\pi_t^i(s^t)\}_{t=0}^\infty$ * agent $i$ is mistaken unless $\pi_t^i(s^t) = \pi_t(s^t)$ ```{note} @@ -905,7 +909,7 @@ $$ As for attitudes toward bearing risks, agent $i$ has a one-period utility function $$ -u(c_t^i) = u(c_t^i) = \ln (c_t^i) +u(c_t^i) = \ln (c_t^i) $$ with marginal utility of consumption in period $i$ @@ -939,7 +943,7 @@ Notice how social welfare criterion {eq}`eq:welfareW` takes into account both ag This means that the social planner knows and respects -* each agent's one period utility function $u(\cdot) = \ln(\cdot)$ +* each agent's one period utility function $u(\cdot) = \ln(\cdot)$ * each agent $i$'s probability model $\{\pi_t^i(s^t)\}_{t=0}^\infty$ Consequently, we anticipate that these objects will appear in the social planner's rule for allocating the aggregate endowment each period. @@ -1003,16 +1007,17 @@ $$ ### If you're so smart, $\ldots$ -Let's compute some values of limiting allocations {eq}`eq:allocationrule1` for some interesting possible limiting +Let's compute some values of limiting allocations {eq}`eq:allocationrule1` for some interesting possible limiting values of the likelihood ratio process $l_t(s^t)$: $$l_\infty (s^\infty)= 1; \quad c_\infty^1 = \lambda$$ - * In the above case, both agents are equally smart (or equally not smart) and the consumption allocation stays put at a $\lambda, 1 - \lambda $ split between the two agents. + * In the above case, both agents are equally smart (or equally not smart) and the consumption allocation stays put at a $\lambda, 1 - \lambda$ split between the two agents. $$l_\infty (s^\infty) = 0; \quad c_\infty^1 = 0$$ -* In the above case, agent 2 is smarter than agent 1, and agent 1's share of the aggregate endowment converges to zero. +* In the above case, agent 2 is ''smarter'' than agent 1, and agent 1's share of the aggregate endowment converges to zero. + @@ -1020,22 +1025,31 @@ $$l_\infty (s^\infty)= \infty; \quad c_\infty^1 = 1$$ * In the above case, agent 1 is smarter than agent 2, and agent 1's share of the aggregate endowment converges to 1. +```{note} +These three cases are somehow telling us about how relative **wealths** of the agents evolve as time passes. +* when the two agents are equally smart and $\lambda \in (0,1)$, agent 1's wealth share stays at $\lambda$ perpetually. +* when agent 1 is smarter and $\lambda \in (0,1)$, agent 1 eventually "owns" the continuation entire continuation endowment and agent 2 eventually "owns" nothing. +* when agent 2 is smarter and $\lambda \in (0,1)$, agent 2 eventually "owns" the continuation entire continuation endowment and agent 1 eventually "owns" nothing. +Continuation wealths can be defined precisely after we introduce a competitive equilibrium **price** system below. +``` + Soon we'll do some simulations that will shed further light on possible outcomes. But before we do that, let's take a detour and study some "shadow prices" for the social planning problem that can readily be converted to "equilibrium prices" for a competitive equilibrium. -Doing this will allow us to connect our analysis with an argument of {cite}`alchian1950uncertainty` and {cite}`friedman1953essays` that competitive market processes can make prices of risky assets better reflect realistic probability assessments. +Doing this will allow us to connect our analysis with an argument of {cite}`alchian1950uncertainty` and {cite}`friedman1953essays` that competitive market processes can make prices of risky assets better reflect realistic probability assessments. ### Competitive Equilibrium Prices -The two fundamental welfare theorems for general equilibrium models lead us to anticipate that there is a connection between the allocation that solves the social planning problem we have been studying and the allocation in a **competitive equilibrium** with complete markets in history-contingent commodities. +Two fundamental welfare theorems for general equilibrium models lead us to anticipate that there is a connection between the allocation that solves the social planning problem we have been studying and the allocation in a **competitive equilibrium** with complete markets in history-contingent commodities. ```{note} For the two welfare theorems and their history, see . +Again, for applications to a classic macroeconomic growth model, see {doc}`this lecture on a planning problem ` and {doc}`this lecture on a related competitive equilibrium ` ``` Such a connection prevails for our model. @@ -1048,43 +1062,68 @@ Instead, there is a comprehensive centralized market that meets at one point i There are **prices** at which price-taking agents can buy or sell whatever goods that they want. -Trade is multilateral in the sense that all that there is a "Walrasian auctioneer" who lives outside the model and whose job is to verify that +Trade is multilateral in the sense that that there is a "Walrasian auctioneer" who lives outside the model and whose job is to verify that each agent's budget constraint is satisfied. That budget constraint involves the total value of the agent's endowment stream and the total value of its consumption stream. +These values are computed at price vectors that the agents take as given -- they are "price-takers" who assume that they can buy or sell +whatever quantities that they want at those prices. + Suppose that at time $-1$, before time $0$ starts, agent $i$ can purchase one unit $c_t(s^t)$ of consumption at time $t$ after history $s^t$ at price $p_t(s^t)$. Notice that there is (very long) **vector** of prices. -We want to study how agents' diverse beliefs influence equilibrium prices. + * there is one price $p_t(s^t)$ for each history $s^t$ at every date $t = 0, 1, \ldots, $. + * so there are as many prices as there are histories and dates. + +These prices determined at time $-1$ before the economy starts. + +The market meets once at time $-1$. + +At times $t =0, 1, 2, \ldots$ trades made at time $-1$ are executed. + + + +* in the background, there is an "enforcement" procedure that forces agents to carry out the exchanges or "deliveries" that they agreed to at time $-1$. + + + +We want to study how agents' beliefs influence equilibrium prices. Agent $i$ faces a **single** intertemporal budget constraint $$ -\sum_{t=0}\sum_{s^t} p_t(s^t) c_t^i (y_t(s^t)) \leq \sum_{t=0}\sum_{s^t} p_t(s^t) y_t^i (y_t(s^t)) +\sum_{t=0}^\infty\sum_{s^t} p_t(s^t) c_t^i (s^t) \leq \sum_{t=0}^\infty\sum_{s^t} p_t(s^t) y_t^i (s^t) $$ (eq:budgetI) -Agent $i$ puts a Lagrange multiplier $\mu^i$ on {eq}`eq:budgetI` and once-and-for-all chooses a consumption plan $\{c^i_t(s^t)\}_{t=0}^\infty$ +According to budget constraint {eq}`eq:budgetI`, trade is **multilateral** in the following sense + +* we can imagine that agent $i$ first sells his random endowment stream $\{y_t^i (s^t)\}$ and then uses the proceeds (i.e., his "wealth") to purchase a random consumption stream $\{c_t^i (s^t)\}$. + +Agent $i$ puts a Lagrange multiplier $\mu_i$ on {eq}`eq:budgetI` and once-and-for-all chooses a consumption plan $\{c^i_t(s^t)\}_{t=0}^\infty$ to maximize criterion {eq}`eq:objectiveagenti` subject to budget constraint {eq}`eq:budgetI`. -```{note} -For convenience, let's remind ourselves of criterion {eq}`eq:objectiveagenti`: -$ -V^i = \sum_{t=0}^{\infty} \sum_{s^t} \delta^t u_t(c_t^i(s^t)) \pi_t^i(s^t)$ -``` +This means that the agent $i$ chooses many objects, namely, $c_t^i(s^t)$ for all $s^t$ for $t = 0, 1, 2, \ldots$. + -First-order conditions for maximizing with respect to $c_t^i(s^t)$ are +For convenience, let's remind ourselves of criterion $V^i$ defined in {eq}`eq:objectiveagenti`: $$ -\delta^t u'(c^i(s^t)) \pi_t^i(s^t) = \mu_i p_t(s^t) , +V^i = \sum_{t=0}^{\infty} \sum_{s^t} \delta^t u_t(c_t^i(s^t)) \pi_t^i(s^t) +$$ + +First-order necessary conditions for maximizing objective $V^i$ defined in {eq}`eq:objectiveagenti` with respect to $c_t^i(s^t)$ are + +$$ +\delta^t u'(c^i_t(s^t)) \pi_t^i(s^t) = \mu_i p_t(s^t) , $$ which we can rearrange to obtain $$ -p_t(s^t) = \frac{ \delta^t \pi_t^i(s^t)}{\mu^i c^i(s^t)} +p_t(s^t) = \frac{ \delta^t \pi_t^i(s^t)}{\mu_i c^i_t(s^t)} $$ (eq:priceequation1) for $i=1,2$. @@ -1115,14 +1154,20 @@ relative prices, see . If we substitute formula {eq}`eq:allocationce` for $c_t^1(s^t)$ into formula {eq}`eq:priceequation1` and rearrange, we obtain $$ -p_t(s^t) = \frac{\delta^t \pi_t^2(s^t)}{1 - \lambda + \lambda l_t(s^t)} +p_t(s^t) = \frac{\delta^t}{\lambda(1-\lambda)} \pi_t^2(s^t) \bigl[1 - \lambda + \lambda l_t(s^t)\bigr] +$$ + +or + +$$ +p_t(s^t) = \frac{\delta^t}{\lambda(1-\lambda)} \bigl[(1 - \lambda) \pi_t^2(s^t) + \lambda \pi_t^1(s^t)\bigr] $$ (eq:pformulafinal) According to formula {eq}`eq:pformulafinal`, we have the following possible limiting cases: -* when $l_\infty = 0$, $c_\infty^2 = 0 $ and tails of competitive equilibrium prices reflect agent $2$'s probability model $\pi_t^2(s^t)$ -* when $l_\infty = 1$, $c_\infty^1 = 0 $ and tails competitive equilibrium prices reflect agent $1$'s probability model $\pi_t^2(s^t)$ -* for small $t$'s, competitive equilbrium prices reflect both agents' probability models. +* when $l_\infty = 0$, $c_\infty^1 = 0 $ and tails of competitive equilibrium prices reflect agent $2$'s probability model $\pi_t^2(s^t)$ according to $p_t(s^t) \propto \delta^t \pi_t^2(s^t) $ +* when $l_\infty = \infty$, $c_\infty^1 = 1 $ and tails of competitive equilibrium prices reflect agent $1$'s probability model $\pi_t^1(s^t)$ according to $p_t(s^t) \propto \delta^t \pi_t^1(s^t) $ +* for small $t$'s, competitive equilibrium prices reflect both agents' probability models. ### Simulations @@ -1403,7 +1448,7 @@ We assume that $f$ and $g$ both put positive probabilities on the same intervals -In the simulations below, we specify that $f$ is a $\text{Beta}(1, 1)$ distribution and that $g$ is $\text{Beta}(3, 1.2)$ distribution. +In the simulations below, we specify that $f$ is a $\text{Beta}(1, 1)$ distribution and that $g$ is $\text{Beta}(3, 1.2)$ distribution. We consider two alternative timing protocols. @@ -1465,9 +1510,9 @@ def protocol_2(π_minus_1, T, N=1000): ``` **Remark:** Under timing protocol 2, the $\{w_t\}_{t=1}^T$ is a sequence of IID draws from $h(w)$. Under timing protocol 1, the $\{w_t\}_{t=1}^T$ is -not IID. It is **conditionally IID** -- meaning that with probability $\pi_{-1}$ it is a sequence of IID draws from $f(w)$ and with probability $1-\pi_{-1}$ it is a sequence of IID draws from $g(w)$. For more about this, see {doc}`this lecture about exchangeability `. +not IID. It is **conditionally IID** -- meaning that with probability $\pi_{-1}$ it is a sequence of IID draws from $f(w)$ and with probability $1-\pi_{-1}$ it is a sequence of IID draws from $g(w)$. For more about this, see {doc}`this lecture about exchangeability `. -We again deploy a **likelihood ratio process** with time $t$ component being the likelihood ratio +We again deploy a **likelihood ratio process** with time $t$ component being the likelihood ratio $$ \ell (w_t)=\frac{f\left(w_t\right)}{g\left(w_t\right)},\quad t\geq1. @@ -1636,11 +1681,11 @@ plt.tight_layout() plt.show() ``` -To the left of the green vertical line $g < f$, so $l_t < 1$; therefore a $w_t$ that falls to the left of the green line is classified as a type $g$ individual. +To the left of the green vertical line $g < f$, so $l_t < 1$; therefore a $w_t$ that falls to the left of the green line is classified as a type $g$ individual. * The shaded orange area equals $\beta$ -- the probability of classifying someone as a type $g$ individual when it is really a type $f$ individual. -To the right of the green vertical line $g > f$, so $l_t >1 $; therefore a $w_t$ that falls to the right of the green line is classified as a type $f$ individual. +To the right of the green vertical line $g > f$, so $l_t >1 $; therefore a $w_t$ that falls to the right of the green line is classified as a type $f$ individual. * The shaded blue area equals $\alpha$ -- the probability of classifying someone as a type $f$ when it is really a type $g$ individual. @@ -1721,9 +1766,9 @@ From the figure above, we can see: ## Measuring discrepancies between distributions -A plausible guess is that the ability of a likelihood ratio to distinguish distributions $f$ and $g$ depends on how "different" they are. +A plausible guess is that the ability of a likelihood ratio to distinguish distributions $f$ and $g$ depends on how "different" they are. -But how should we measure discrepancies between distributions? +But how should we measure discrepancies between distributions? We've already encountered one discrepancy measure -- the Kullback-Leibler (KL) divergence. @@ -1743,13 +1788,13 @@ $$ C(f,g) = - \log \min_{\phi \in (0,1)} \int f^\phi(x) g^{1-\phi}(x) dx $$ -An upper bound on model selection error probabilty is +An upper bound on model selection error probability is $$ e^{-C(f,g)T} . $$ -Thus, Chernoff entropy is an upper bound on the exponential rate at which the selection error probability falls as sample size $T$ grows. +Thus, Chernoff entropy is an upper bound on the exponential rate at which the selection error probability falls as sample size $T$ grows. Let's compute Chernoff entropy numerically with some Python code @@ -1916,7 +1961,7 @@ latex_code = rf""" display(Math(latex_code)) ``` -The above table indicates how Jensen-Shannon divergence, and Chernoff entropy, and KL divergence covary as we alter $f$ and $g$. +The above table indicates how Jensen-Shannon divergence, and Chernoff entropy, and KL divergence covary as we alter $f$ and $g$. Let's also visualize how these diverge measures covary @@ -2156,7 +2201,7 @@ $$ \frac{1}{t} E_h[\log L_t] = K_g - K_f $$ -with finite $K_g, K_f < \infty$, $E_h |\log f(W)| < \infty$ and $E_h |\log g(W)| < \infty$. +with finite $K_g, K_f$, $E_h |\log f(W)| < \infty$ and $E_h |\log g(W)| < \infty$. *Hint:* Start by expressing $\log L_t$ as a sum of $\log \ell(w_i)$ terms and compare with the definition of $K_f$ and $K_g$. ``` @@ -2270,5 +2315,96 @@ Therefore by similar reasoning $L_t \to 0$ almost surely. ```{solution-end} ``` +```{exercise} +:label: lr_ex3 + +Starting from {eq}`eq:priceequation1`, show that the competitive equilibrium prices can be expressed as + +$$ +p_t(s^t) = \frac{\delta^t}{\lambda(1-\lambda)} \pi_t^2(s^t) \bigl[1 - \lambda + \lambda l_t(s^t)\bigr] +$$ + +``` + +```{solution-start} lr_ex3 +:class: dropdown +``` + +Starting from + +$$ +p_t(s^t) = \frac{\delta^t \pi_t^i(s^t)}{\mu_i c_t^i(s^t)}, \qquad i=1,2. +$$ + +Since both expressions equal the same price, we can equate them + +$$ +\frac{\pi_t^1(s^t)}{\mu_1 c_t^1(s^t)} = \frac{\pi_t^2(s^t)}{\mu_2 c_t^2(s^t)} +$$ + +Rearranging gives + +$$ +\frac{c_t^1(s^t)}{c_t^2(s^t)} = \frac{\mu_2}{\mu_1} l_t(s^t) +$$ + +where $l_t(s^t) \equiv \pi_t^1(s^t)/\pi_t^2(s^t)$ is the likelihood ratio process. + +Using $c_t^2(s^t) = 1 - c_t^1(s^t)$: + +$$ +\frac{c_t^1(s^t)}{1 - c_t^1(s^t)} = \frac{\mu_2}{\mu_1} l_t(s^t) +$$ + +Solving for $c_t^1(s^t)$ + +$$ +c_t^1(s^t) = \frac{\mu_2 l_t(s^t)}{\mu_1 + \mu_2 l_t(s^t)} +$$ + + +The planner's solution gives + +$$ +c_t^1(s^t) = \frac{\lambda l_t(s^t)}{1 - \lambda + \lambda l_t(s^t)} +$$ + +To match them, we need the following equality to hold + +$$ +\frac{\mu_2}{\mu_1} = \frac{\lambda}{1 - \lambda} +$$ + +Hence we have + +$$ +\mu_1 = 1 - \lambda, \qquad \mu_2 = \lambda +$$ + + +With $\mu_1 = 1-\lambda$ and $c_t^1(s^t) = \frac{\lambda l_t(s^t)}{1-\lambda+\lambda l_t(s^t)}$, +we have + +$$ +\begin{aligned} +p_t(s^t) &= \frac{\delta^t \pi_t^1(s^t)}{(1-\lambda) c_t^1(s^t)} \\ +&= \frac{\delta^t \pi_t^1(s^t)}{(1-\lambda)} \cdot \frac{1 - \lambda + \lambda l_t(s^t)}{\lambda l_t(s^t)} \\ +&= \frac{\delta^t \pi_t^1(s^t)}{(1-\lambda)\lambda l_t(s^t)} \bigl[1 - \lambda + \lambda l_t(s^t)\bigr]. +\end{aligned} +$$ + +Since $\pi_t^1(s^t) = l_t(s^t) \pi_t^2(s^t)$, we have + +$$ +\begin{aligned} +p_t(s^t) &= \frac{\delta^t l_t(s^t) \pi_t^2(s^t)}{(1-\lambda)\lambda l_t(s^t)} \bigl[1 - \lambda + \lambda l_t(s^t)\bigr] \\ +&= \frac{\delta^t \pi_t^2(s^t)}{(1-\lambda)\lambda} \bigl[1 - \lambda + \lambda l_t(s^t)\bigr] \\ +&= \frac{\delta^t}{\lambda(1-\lambda)} \pi_t^2(s^t) \bigl[1 - \lambda + \lambda l_t(s^t)\bigr]. +\end{aligned} +$$ + +```{solution-end} +``` + diff --git a/lectures/mix_model.md b/lectures/mix_model.md index a7eaeaaa3..635061a6d 100644 --- a/lectures/mix_model.md +++ b/lectures/mix_model.md @@ -25,9 +25,9 @@ kernelspec: ## Overview -This is a sequel to {doc}`this quantecon lecture `. +This is a sequel to {doc}`this quantecon lecture `. -We discuss two ways to create compound lottery and their consequences. +We discuss two ways to create a compound lottery and their consequences. A compound lottery can be said to create a _mixture distribution_. @@ -35,7 +35,7 @@ Our two ways of constructing a compound lottery will differ in their **timing**. * in one, mixing between two possible probability distributions will occur once and all at the beginning of time -* in the other, mixing between the same two possible possible probability distributions will occur each period +* in the other, mixing between the same two possible probability distributions will occur each period The statistical setting is close but not identical to the problem studied in that quantecon lecture. @@ -50,7 +50,7 @@ The agent represented that ignorance by assuming that nature had chosen $f$ or That assumption allowed the agent to construct a subjective joint probability distribution over the random sequence $\{W_t\}_{t=0}^\infty$. -We studied how the agent would then use the laws of conditional probability and an observed history $w^t =\{w_s\}_{t=0}^t$ to form +We studied how the agent would then use the laws of conditional probability and an observed history $w^t =\{w_s\}_{s=0}^t$ to form $$ \pi_t = E [ \textrm{nature chose distribution} f | w^t] , \quad t = 0, 1, 2, \ldots @@ -68,7 +68,7 @@ and $g$ with probability $1 -\alpha$. Thus, nature perpetually draws from the **mixture distribution** with c.d.f. $$ -H(w ) = \alpha F(w) + (1-\alpha) G(w), \quad \alpha \in (0,1) +H(w) = \alpha F(w) + (1-\alpha) G(w), \quad \alpha \in (0,1) $$ We'll study two agents who try to learn about the wage process, but who use different statistical models. @@ -95,7 +95,7 @@ possible. Nevertheless, we'll see that our first type of agent muddles through and eventually learns something interesting and useful, even though it is not *true*. -Instead, it turn out that our type 1 agent who is armed with a wrong statistical model ends up learning whichever probability distribution, $f$ or $g$, +Instead, it turns out that our type 1 agent who is armed with a wrong statistical model ends up learning whichever probability distribution, $f$ or $g$, is in a special sense *closest* to the $h$ that actually generates the data. We'll tell the sense in which it is closest. @@ -232,7 +232,7 @@ Here is pseudo code for a direct "method 1" for drawing from our compound lotter Our second method uses a uniform distribution and the following fact that we also described and used in the quantecon lecture : - * If a random variable $X$ has c.d.f. $F(X)$, then a random variable $F^{-1}(U)$ also has c.d.f. $F(x)$, where $U$ is a uniform random variable on $[0,1]$. + * If a random variable $X$ has c.d.f. $F$, then a random variable $F^{-1}(U)$ also has c.d.f. $F$, where $U$ is a uniform random variable on $[0,1]$. In other words, if $X \sim F(x)$ we can generate a random sample from $F$ by drawing a random sample from a uniform distribution on $[0,1]$ and computing $F^{-1}(U)$. @@ -316,13 +316,13 @@ Bayes’ law implies that $ \{\pi_t\} $ obeys the recursion $$ \pi_t=\frac{\pi_{t-1} l_t(w_t)}{\pi_{t-1} l_t(w_t)+1-\pi_{t-1}} -$$ (equation-eq-recur1) +$$ (eq:recur1) with $ \pi_{0} $ being a Bayesian prior probability that $ q = f $, i.e., a personal or subjective belief about $ q $ based on our having seen no data. Below we define a Python function that updates belief $ \pi $ using -likelihood ratio $ \ell $ according to recursion {eq}`equation-eq-recur1` +likelihood ratio $ \ell $ according to recursion {eq}`eq:recur1` ```{code-cell} ipython3 :hide-output: false @@ -337,7 +337,7 @@ def update(π, l): return π ``` -Formula {eq}`equation-eq-recur1` can be generalized by iterating on it and thereby deriving an +Formula {eq}`eq:recur1` can be generalized by iterating on it and thereby deriving an expression for the time $ t $ posterior $ \pi_{t+1} $ as a function of the time $ 0 $ prior $ \pi_0 $ and the likelihood ratio process $ L(w^{t+1}) $ at time $ t $. @@ -389,20 +389,18 @@ and the initial prior $ \pi_{0} $ $$ \pi_{t+1}=\frac{\pi_{0}L\left(w^{t+1}\right)}{\pi_{0}L\left(w^{t+1}\right)+1-\pi_{0}}. -$$ (equation-eq-bayeslaw103) +$$ (eq:bayeslaw103) -Formula {eq}`equation-eq-bayeslaw103` generalizes formula {eq}`equation-eq-recur1`. +Formula {eq}`eq:bayeslaw103` generalizes formula {eq}`eq:recur1`. -Formula {eq}`equation-eq-bayeslaw103` can be regarded as a one step revision of prior probability $ \pi_0 $ after seeing +Formula {eq}`eq:bayeslaw103` can be regarded as a one step revision of prior probability $ \pi_0 $ after seeing the batch of data $ \left\{ w_{i}\right\} _{i=1}^{t+1} $. ## What a type 1 Agent Learns when Mixture $H$ Generates Data -We now study what happens when the mixture distribution $h;\alpha$ truly generated the data each period. +We now study what happens when the mixture distribution $h;\alpha$ truly generated the data each period. -A submartingale or supermartingale continues to describe $\pi_t$ - -It raises its ugly head and causes $\pi_t$ to converge either to $0$ or to $1$. +The sequence $\pi_t$ continues to converge, despite the agent's misspecified model, and the limit is either $0$ or $1$. This is true even though in truth nature always mixes between $f$ and $g$. @@ -615,9 +613,9 @@ plt.show() Evidently, our type 1 learner who applies Bayes' law to his misspecified set of statistical models eventually learns an approximating model that is as close as possible to the true model, as measured by its Kullback-Leibler divergence: -- When $\alpha$ is small, the $KL_g < KL_f$ meaning the divergence of $g$ from $h$ is smaller than that of $f$ and so the limit point of $\pi_t$ is close to $0$. +- When $\alpha$ is small, $KL_g < KL_f$ meaning the divergence of $g$ from $h$ is smaller than that of $f$ and so the limit point of $\pi_t$ is close to $0$. -- When $\alpha$ is large, the $KL_f < KL_g$ meaning the divergence of $f$ from $h$ is smaller than that of $g$ and so the limit point of $\pi_t$ is close to $1$. +- When $\alpha$ is large, $KL_f < KL_g$ meaning the divergence of $f$ from $h$ is smaller than that of $g$ and so the limit point of $\pi_t$ is close to $1$. ## Type 2 Agent @@ -634,7 +632,7 @@ $$ but does not know $\alpha$. -We'll assume that the person starts out with a prior probabilty $\pi_0(\alpha)$ on +We'll assume that the person starts out with a prior probability $\pi_0(\alpha)$ on $\alpha \in (0,1)$ where the prior has one of the forms that we deployed in {doc}`this quantecon lecture `. @@ -658,7 +656,7 @@ $t \rightarrow + \infty$. Let us try a uniform prior first. -We use the `Mixture` class in Numpyro to construct the likelihood function. +We use the `Mixture` class in numpyro to construct the likelihood function. ```{code-cell} ipython3 α = 0.8 @@ -730,7 +728,7 @@ Thus, the following situation prevails quite generally in empirical work. A scientist approaches the data with a manifold $S$ of statistical models $ s (X | \theta)$ , where $s$ is a probability distribution over a random vector $X$, $\theta \in \Theta$ is a vector of parameters, and $\Theta$ indexes the manifold of models. -The scientist with observations that he interprests as realizations $x$ of the random vector $X$ wants to solve an **inverse problem** of somehow _inverting_ +The scientist with observations that he interprets as realizations $x$ of the random vector $X$ wants to solve an **inverse problem** of somehow _inverting_ $s(x | \theta)$ to infer $\theta$ from $x$. But the scientist's model is misspecified, being only an approximation to an unknown model $h$ that nature uses to generate $X$. @@ -750,7 +748,7 @@ In that lecture, we used a model selection algorithm to study the case where the In this lecture, we studied how to correctly "learn" a model generated by a mixing process using a Bayesian approach. -To fix the algorithm we used in {doc}`likelihood_bayes`. A correct Bayesian approach should directly model the uncertainty about $x$ and update beliefs about it as new data arrives. +To fix the algorithm we used in {doc}`likelihood_bayes`, a correct Bayesian approach should directly model the uncertainty about $x$ and update beliefs about it as new data arrives. Here is the algorithm: