## Introduction

This lecture studies a problem that we study from another angle in this quantecon lecture

{doc}`calvo`.

Both lectures compute a Ramsey plan for a version of a model of Calvo {cite}`Calvo1978`.

This lecture uses what we call a ``machine learning`` approach to

compute a Ramsey plan for a version of a model of Calvo {cite}`Calvo1978`.

We use another approach to compute a Ramsey plan for Calvo's model in another quantecon lecture

{doc}`calvo`.

The {doc}`calvo` lecture uses an analytic approach based on ``dynamic programming squared`` to guide computations.

Dynamic programming squared provides information about the structure of mathematical objects in terms of which a Ramsey plan can be represented recursively.

That paves the way to computing a Ramsey plan efficiently.

Using that information paves the way to computing a Ramsey plan efficiently.

Included in the structural information that dynamic programming squared provides in quantecon lecture {doc}`calvo` are descriptions of

Included in the structural information that dynamic programming squared provides in quantecon lecture {doc}`calvo` are

* a **state** variable that confronts a continuation Ramsey planner, and

* two **Bellman equations**

* one that describes the behavior of the representative agent

* another that describes decision problems of a Ramsey planner and of a continuation Ramsey planner

In this lecture, we approach the Ramsey planner in a less sophisticated way.

We proceed without knowing the mathematical structure imparted by dynamic programming squared.

In this lecture, we approach the Ramsey planner in a less sophisticated way that proceeds without knowing the mathematical structure imparted by dynamic programming squared.

Instead, we use a brute force approach that simply chooses a pair of infinite sequences of real numbers that maximizes a Ramsey planner's objective function.

We simply choose a pair of infinite sequences of real numbers that maximizes a Ramsey planner's objective function.

The pair consists of

* a sequence $\vec \theta$ of inflation rates

* a sequence $\vec \mu$ of money growh rates

Because it fails to take advantage of the structure recognized by dynamic programming squared and instead proliferates parameters, we take the liberty of calling this a **machine learning** approach.

Because it fails to take advantage of the structure recognized by dynamic programming squared and, relative to the dynamic programming squared approach, proliferates parameters, we take the liberty of calling this a **machine learning** approach.

This is similar to what other machine learning algorithms also do.

Comparing the calculations in this lecture with those in our sister lecture {doc}`calvo` provides us

with a laboratory that can help us appreciate promises and limits of machine learning approaches

more generally.

We'll actually deploy two machine learning approaches.

In this lecture, we'll actually deploy two machine learning approaches.

* the first is really lazy

* it just writes a Python function to computes the Ramsey planner's objective as a function of a money growth rate sequence and then hands it over to a gradient descent optimizer

* it writes a Python function that computes the Ramsey planner's objective as a function of a money growth rate sequence and hands it over to a ``gradient descent`` optimizer

* the second is less lazy

* it exerts the effort required to express the Ramsey planner's objective as an affine quadratic form in $\vec \mu$, computes first-order conditions for an optimum, arranges them into a system of simultaneous linear equations for $\vec \mu$ and then $\vec \theta$, then solves them.

* it exerts the mental effort required to express the Ramsey planner's objective as an affine quadratic form in $\vec \mu$, computes first-order conditions for an optimum, arranges them into a system of simultaneous linear equations for $\vec \mu$ and then $\vec \theta$, then solves them.

While both of these machine learning (ML) approaches succeed in recovering the Ramsey plan that we also compute in quantecon lecture {doc}`calvo` by using dynamic programming squared, they don't reveal the recursive structure of the Ramsey plan described in that lecture.

Each of these machine learning (ML) approaches recovers the same Ramsey plan that shall compute in quantecon lecture {doc}`calvo` by using dynamic programming squared.

That recursive structure lies hidden within some of the objects calculated by our ML approach.

However, they conceal the recursive structure of the Ramsey plan.

We can ferret out some of that structure if we ask the right questions.

That recursive structure lies hidden within some of the objects calculated by our ML approaches.

At the end of this lecture we describe some of those questions are and how they can be answered by running particular linear regressions on components of

$\vec \mu, \vec \theta$.

Nevertheless, we can ferret out some of that structure by asking the right questions.

Human intelligence, not the artificial intelligence deployed in our machine learning approach, is a key input into choosing which regressions to run.

We pose those questions at the end of this lecture and answer them by running particulars some linear regressions on components of $\vec \mu, \vec \theta$.

Human intelligence, not the ``artificial intelligence`` deployed in our machine learning approach, is a key input into choosing which regressions to run.

## The Model

We study a linear-quadratic version of a model that Guillermo Calvo {cite}`Calvo1978` used to illustrate the **time inconsistency** of optimal government plans.

The model focuses attention on intertemporal tradeoffs between

The model focuses on intertemporal tradeoffs between

- utility that a representative agent's anticipations of future deflation generate by lowering the costs of holding real money balances and thereby increasing the agent's *liquidity*, as measured by holdings of real money balances, and

- social costs associated with the distorting taxes that a government levies to acquire the paper money that it destroys in order to generate anticipated deflation

- utility that a representative agent's anticipations of future deflation delivered by lowering the agent's cost of holding real money balances and thereby increasing the agent's *liquidity*, as ultimately measured by the agent's holdings of real money balances, and

- social costs associated with the distorting taxes that a government levies to acquire the paper money that it destroys in order to generate prospective deflation

The model features

- rational expectations

- costly government actions at all dates $t \geq 1$ that increase household utilities at dates before $t$

The model combines ideas from papers by Cagan {cite}`Cagan` and Calvo {cite}`Calvo1978`.

The model combines ideas from papers by Cagan {cite}`Cagan`, {cite}`sargent1973stability`, and Calvo {cite}`Calvo1978`.

Via equation {eq}`eq_grad_old3`, a government plan

$\vec \mu = \{\mu_t \}_{t=0}^\infty$ leads to a

sequence of inflation outcomes

sequence of inflation rates

$\vec \theta = \{ \theta_t \}_{t=0}^\infty$.

We assume that the government incurs social costs $\frac{c}{2} \mu_t^2$ at

The Ramsey planner chooses

a vector of money growth rates $\vec \mu$

to maximize criterion {eq}`eq:RamseyV` subject to equations {eq}`eq_grad_old3`.

to maximize criterion {eq}`eq:RamseyV` subject to equations {eq}`eq_grad_old3` and a restriction

requiring that

$$

\vec \theta \in L^2

$$ (eq:thetainL2)

Notice equations {eq}`eq_grad_old3` and {eq}`eq:thetainL2` imply that $\vec \theta$ is a function

of $\vec \mu$.

In particular, the inflation rate $\theta_t$ satisfies

$$

\theta_t = (1-\lambda) \sum_{j=0}^\infty \lambda^j \mu_{t+j}, \quad t \geq 0

$$ (eq:inflation101)

where

$$

\lambda = \frac{\alpha}{1+\alpha} .

$$

## Parameters and Variables

**Parameters** are

**Parameters:**

* Demand for money parameter is $\alpha > 0$; we set its default value $\alpha = 1$

**Variables** are

**Variables:**

* $\theta_t = p_{t+1} - p_t$ where $p_t$ is log of price level

\end{aligned}

$$

The inflation rate $\theta_t$ satisfies

$$

\theta_t = (1-\lambda) \sum_{j=0}^\infty \lambda^j \mu_{t+j}, \quad t \geq 0

$$ (eq:inflation101)

where

$$

\lambda = \frac{\alpha}{1+\alpha}

$$

A Ramsey planner chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue`

subject to equation {eq}`eq:inflation101`.

subject to equations {eq}`eq:inflation101`.

A solution $\vec \mu$ of this problem is called a **Ramsey plan**.

**Formula for $V$**

Having computed the truncated vectors $\tilde \mu$ and $\tilde \theta$

as described above, we want to write a function that computes

Having specified a truncated vector $\tilde \mu$ and and having computed $\tilde \theta$

by using formula {eq}`eq:thetaformula102`, we want to write a Python function that computes

$$

\tilde V = \sum_{t=0}^\infty \beta^t (

## A Gradient Descent Algorithm

We now describe code that maximizes the criterion function {eq}`eq:Ramseyvalue` by choice of the truncated vector $\tilde \mu$.

We now describe code that maximizes the criterion function {eq}`eq:Ramseyvalue` subject to equations {eq}`eq:inflation101` by choice of the truncated vector $\tilde \mu$.

We use a brute force or ``machine learning`` approach that just hands our problem off to code that minimizes $V$ with respect to the components of $\tilde \mu$ by using gradient descent.

We'll eventually want to compare the results we obtain here to those that we obtain in those obtained in this quantecon lecture {doc}`calvo`.

To enable us to do that, we copy the class `ChangLQ` that we used in that lecture.

To enable us to do that, we copy the class `ChangLQ` used in that lecture.

We hide the cell that copies the class, but readers can find details of the class in this quantecon lecture {doc}`calvo`.

We take a brief detour to solve a restricted version of the Ramsey problem defined above.

First, recall that a Ramsey planner chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue`subject to equation {eq}`eq:inflation101`.

First, recall that a Ramsey planner chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue` subject to equations {eq}`eq:inflation101`.

We now define a distinct problem in which the planner chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue`subject to equation {eq}`eq:inflation101` and

We now define a distinct problem in which the planner chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue` subject to equation {eq}`eq:inflation101` and

the additional restriction that $\mu_t = \bar \mu$ for all $t$.

The solution of this problem is a time-invariant $\mu_t$ that this quantecon lecture {doc}`calvo` calls $\mu^{CR}$.

print(f"optimized μ = \n{optimized_μ_CR}")

```

Compare it to $\mu^{CR}$ in {doc}`calvo`, we again obtained very close answers.

Comparing it to $\mu^{CR}$ in {doc}`calvo`, we again obtained very close answers.

```{code-cell} ipython3

np.linalg.norm(clq.μ_CR - optimized_μ_CR)

## A More Structured ML Algorithm

By thinking a little harder about the mathematical structure of the Ramsey problem and using some linear algebra, we can simplify the problem that we hand over to a ``machine learning`` algorithm.

By thinking about the mathematical structure of the Ramsey problem and using some linear algebra, we can simplify the problem that we hand over to a ``machine learning`` algorithm.

We start by recalling that the Ramsey problem that chooses $\vec \mu$ to maximize the government's value function {eq}`eq:Ramseyvalue`subject to equation {eq}`eq:inflation101`.

## Some Exploratory Regressions

To help us learn something about the structure of the Ramsey plan, we compute some least squares linear regressions of some components of $\vec \theta$ and $\vec \mu$ on others.

We compute some least squares linear regressions of some components of $\vec \theta$ and $\vec \mu$ on others.

Our hope is that these regressions will reveal structure hidden within the $\vec \mu^R, \vec \theta^R$ sequences associated with a Ramsey plan.

We hope that these regressions will reveal structure hidden within the $\vec \mu^R, \vec \theta^R$ sequences associated with a Ramsey plan.

It is worth pausing to think about roles being played here by **human** intelligence and **artificial** intelligence.

plt.show()

```

Note that $\theta_t$ is less than $\mu_t$for low $t$'s, but that it eventually converges to

the same limit $\bar \mu$ that $\mu_t$ does.

Note that while $\theta_t$ is less than $\mu_t$for low $t$'s, it eventually converges to

the limit $\bar \mu$ of $\mu_t$ as $t \rightarrow +\infty$.

This pattern reflects how formula {eq}`eq_grad_old3` makes $\theta_t$ be a weighted average of future $\mu_t$'s.

print(results1.summary(slim=True))

```

Our regression tells us that along the Ramsey outcome $\vec \mu, \vec \theta$ the linear function

Our regression tells us that the affine function

$$

\mu_t = .0645 + 1.5995 \theta_t

$$

fits perfectly.

fits perfectly along the Ramsey outcome $\vec \mu, \vec \theta$.

```{note}

### Continuation Values

Next, we'll compute a sequence $\{v_t\}_{t=0}^T$ of what we'll call "continuation values" along a Ramsey plan.

Next, we'll compute a sequence $\{v_t\}_{t=0}^T$ of what we'll call ``continuation values`` along a Ramsey plan.

To do so, we'll start at date $T$ and compute

v_t = compute_vt(μs, β=0.85, c=2)

```

The initial continuation value $v_0$ should equals the optimized value of the Ramsey planner's criterion $V$ defined

The initial continuation value $v_0$ should equal the optimized value of the Ramsey planner's criterion $V$ defined

in equation {eq}`eq:RamseyV`.

plt.show()

```

Figure {numref}`continuation_values` shows several interesting patterns:

Figure {numref}`continuation_values` shows interesting patterns:

* The sequence of continuation values $\{v_t\}_{t=0}^T$ is monotonically decreasing

* Evidently, $v_0 > V^{CR} > v_T$ so that

We discovered these relationships by running some carefully chosen regressions and staring at the results, noticing that the $R^2$'s of unity tell us that the fits are perfect.

We have learned something about the structure of the Ramsey problem.

We have learned much about the structure of the Ramsey problem.

However, it is challenging to say more just by using the methods and ideas that we have deployed in this lecture.

However, by using the methods and ideas that we have deployed in this lecture, it is challenging to say more.

There are many other linear regressions among components of $\vec \mu^R, \theta^R$ that would also have given us perfect fits.