\chapter{Methodology}
\label{ch:method}


\section{General Approach}

Given an distorted image, by the model $D=I+W$, where $W \sim N(0,\sigma^2_2)$, the approach must obtain an estimate for the posterior density of the image, $I \in \mathbf{R}^{m \times n}$.   The image of $I$, a matrix of R.V's forming a Markov Random field, is a model that has prior knowledge.  Each pixel, $I_{j,k}$ is a conditional density that is only dependent on the values of its vertical and horizontal neighbors of the given pixel.    Given that $\mu$ is the mean of the horizontal and vertical pixels of $I_{j,k}$, if is known that $I_{j,k} \sim N(\mu,\sigma^2_1)$


\subsection{Bayesian Analysis and Methods}  

Using Bayesian methods, we can estimate the posterior density of I, by sampling from the posterior of each $I_{j,k}$, given $D_{j,k}$ by using the following Bayesian rules.

\begin{equation}
f(I|D) = f(D|I)\frac{f(I)}{f(y)}
\end{equation}

where $f(y) = \int f(D)f(I)dy$.  


As stated above, since the conditional density is dependent only on certian neighbor of values, can form the following

\begin{equation}
f(I_{j,k}|D_{j,k},I_{j+1,k},I_{j-1,k},I_{j,k+1},I_{j,k-1}) = f(D_{j,k}|I_{i,j})f(I_{j,k}
\end{equation}

Since $I_{j,k} \sim N(\mu, \sigma^2_1)$ and $D_{j,k} \sim N(I_{j,k}, \sigma^2_2)$ then

\begin{equation}
f(I_{j,k}|D_{j,k},I_{j+1,k},I_{j-1,k},I_{j,k+1},I_{j,k-1}) \sim N(\mu,\sigma^2_1) \cdot N(I_{i,j},\sigma^2_2)
\end{equation}



\subsection{Metropolis Hastings}

Metropolis Hastings is used in to approximate sampling from complicated distributions.  In general, the goal is to generate samples of a random variable distributed according to the density, say $f(x)$.  Moreover, we assume that the conditional density, say $q(y|x)$ with the following properities

\begin{enumerate}
\item $\forall \, x$, sampling from $q(y|x)$ is possible
\item The support of $q$ contains the support of $f(x)$
\item $q(y|x)$ is known and symmetric in $x$ and $y$.  
\end{enumerate}

Given a function and a conditional density with the above properities, the M-H algorithm is the following

\begin{enumerate}
\item  Choose an intial condition $x_0$ in support of $f(x)$
\\
Construct $x_n$ using the following steps:
\item Generate $y \sim q(y|x_t)$
\item Update the state to $x_{t+1}$ by using

\end{enumerate}

\begin{equation}
x_{t+1} = \left\{\begin{array}{cc}
y & \textrm{probabilty} \hspace{.15cm} \rho(x_t,y) \\
x_t & \textrm{probabilty} \hspace{.15cm} 1-\rho(x_t,y)
\end{array}\right.
\end{equation}

where $\rho(x,y) = \min \left(\frac{f(y)q(x|y)}{f(x)q(y|x)},1 \right)$.  Under certain conditions, $\rho$ can be simplifed, as such the case when

\begin{enumerate}
\item  In cases where the density is independent of the current state, $q(y|x)=q(y)$, then becomes an independent M-H.  Therefore the function becomes
\begin{equation}
\rho(x,y) = \min \left(\frac{f(y)q(x)}{f(x)q(x)},1 \right)
\end{equation}

\item When $q(y|x)$ is symmetric in $x$ and $y$, then the likelihood rate appears, because the function becomes

\begin{equation}
\rho(x,y) = \min \left(\frac{f(y)}{f(x)},1 \right)
\end{equation}

\end{enumerate}









\subsection{Gibbs Sampler}

Another technique for generating Markov Chains is the process of Gibbs Sampling.  The goal is to generate samples by constructing a MC in $\mathbf{R}^n$, from a random vector, $(x_1, x_2, \ldots, x_n)$, with joint pdf, $f(x_1, x_2, \ldots, x_n)$.  In order to use Gibbs sampling for this problem, will assume the conditional densities are known, so $f(x_i|y_i)$ for $i \neq j$.  Therefore will obtain univariate densities to apply the algorithm to update from $x^t$ to $x^{t+1}$


\begin{enumerate}
\item Generate $X_1^{t+1} \sim f_1(x_1|X_2^t, X_3^t, X_4^t)$
\item Generate $X_2^{t+1} \sim f_2(x_2|X_2^t, X_3^t, X_4^t)$
\item Generate $x_1^{t+1}, x_2^{t+1}$
\end{enumerate}

In the procedure, each pixel, $I_{j,k}$ is processed until a complete sweep which will result in a new prior distrubtion, which in turn will be used in the next iteration.  Therefore, in order to do a complete sweep, will sample from the posterior using the Gibbs sampling method, and update the posterior on each squenece.



