\documentclass{beamer}
\usetheme[secheader]{Boadilla}

\title{Nonparametric Curve Estimation}
\author{Jaime Frade}
\institute[Florida State Univeristy]{
\inst{}Department of Statistics\\
Florida State Univeristy
\and
\inst{}Victor Patrangenaru\\
STA5334
}
\date{\today}

\begin{document}

\frame{\titlepage}

\section[Outline]{}
\frame{\tableofcontents}


\section{Introduction}
\frame
{
\begin{center}
Nonparametric Density Estimation
\end{center}
Probabilty density estimation goes hand in hand with nonparametric estimation of the cumulative distribution.  The density function provides a better visual summary of how the random variable is distributed across its support.  Skewness, kurtosis, disperseness are just a few properities can ascertiantined from the density plot.
\vspace{.25cm}

\begin{figure}
\includegraphics[scale=0.25]{example1.png} 
\includegraphics[scale=0.25]{example2.png} 
%\caption{}
\end{figure}
}

\subsection{Objective}
\frame
{
\begin{center}
\underline{Goal}
\end{center}
Unlike using \textit{empirical density function}, which places probabilty mass $\dfrac{1}{n}$ on each observation, the project focuses on the kernel density esimator that more fairly spreads out the probabilty mass of each observation, not arbitrarily in a fixed interval, but smoothly around the observation, typically in a symmetric way.
\vspace{.25cm}

From there, will illustrate four basic kernels, as well as an adaptive kernel function, to estimate the distribution of returns on a certian asset.  Simulation will be completed.
}

\section{Theory}

\subsection{Definitions}
\frame
{
With a sample of $X_1, X_2, \ldots , X_n$, write the density estimator
\begin{equation}\label{re:one}
\hat{f}(x)  =  \sum^n_{i=1}{K\left(\dfrac{x-x_i}{h_n}\right)}
\end{equation}
for $X_i =x_i$, $i=1, \ldots , n$.  The \textit{kernel function K} represents how the probabilty mass is assigned.  For example, for the histogram, in any particular interval, $K$ is constant.  The smoothing function $h_n$ is a positive sequence of bandwidths analoguous to the bin width in a histogram.
}

\frame
{
From lecture notes, to estimate $f$, one may use the density of the random variable $\widehat{X}+hZ$, where $\widehat{X}$ has the distribution (conditionally, given $X_1, \ldots, X_n$) and $Z$ is independent of $\widehat{X}$.  From $\eqref{re:one}$, the \textit{bandwidth} satisfies
\begin{equation}
h \equiv h_n \longrightarrow \textrm{as} \hspace{.15cm} n \rightarrow \infty
\end{equation}

$\widehat{X} + hZ$ has the density.

\begin{equation}\label{re:two}
\widehat{f_n}(x) = \frac{1}{n}\sum^n_{i=1}{K_h(x-X_i)}, 
\end{equation}
where $K_h$ is the density of $hZ$.  From $\eqref{re:two}$, using the following definition for $K_h$ below, will obtain \eqref{re:one}.  
\begin{equation}\label{re:three}
K_h(y) = \frac{1}{h}K\left(\dfrac{y}{h}\right)
\end{equation}
}

\frame
{
The kernel function $K$ has five important properities
\begin{itemize}
\item  $K(x)  \geq 0$ \,\, $\forall \, x$ 
\item  $K(x)  =  K(-x)$ \,\, $\forall$\, $x>0$ 
\item  $\int K(u)du  =  1$ 
\item  $\int uK(u)du =  0$ 
\item  $\int u^2K(v)dv  =   \sigma^2_k < \infty $
\end{itemize}
}

\frame
{
Basic idea is that $K$ controls the shape, $h_n$ controls the spread of the kernel.  The accuracy of a density estimator can be evaluated using he mean intergrated squared error, defined as 
\begin{eqnarray} \label{re:four}
	\textrm{MISE} & = & \mathbb{E}\left( \int{(f(x) - \hat{f}(x))^2}dx\right) \nonumber\\
								& = & \int{\textrm{Bias}^2(\hat{f}(x))dx} + \int{\mathbb{V}ar(\hat{f}(x))dx}
\end{eqnarray}
}

\frame
{
To find a density estimator that minimizes tehe MISE under the five constraints, also will assume that $f(x)$ is continuous and twice differentiable, $h_n \rightarrow 0$ and $nh_n \rightarrow \infty$ as $n \rightarrow \infty$.  Under these conditions, 
\begin{eqnarray}
	\textrm{Bias} & = & \frac{\sigma^2_K}{2}f''(x) + O(h^4_n) \nonumber \\
	\; \nonumber\\
	\textrm{Var}\left(\hat{f}(x)\right)	& = & \frac{f(x)R(K)}{nh_n} + O(n^{-1}) 
\end{eqnarray}
where $R(g) = \int{g(u)^2du}$
}

\frame
{
Determine the minimum MISE by the choice of $h_n$.  However, choosing $h_n$ to reduce the bias will increase the variance and vice versa, there is a tradeoff.  The choice of the \textit{bandwidth} is important to the construction of $\hat{f}(x)$.  
\vspace{.25cm}

If $h$ is chosen to be small, the minor difference in main part of the density will be apparent.  If $h$ is choose to be large, the tails of the distribution are better modeled, but fail to see important charateristics of the middle quartiles of the data.
}

\frame
{
By subsituting the bias and variance into the formula for \eqref{re:four}, minimize MISE with
\begin{equation} 
	h^*_n  =  \left(\frac{R(K)}{\sigma^4_K R(f')}\right)^{\frac{1}{5}}\,n^{-\frac{1}{5}} \nonumber
\end{equation}
From here, still can choose $K(x)$ and insert a \textit{representative} density for $f(x)$ to solve for the bandwidth.  
}

\frame
{
Epanechnickov (1969) showed that, upon subsituting $f(x)=\phi(x)$, the kernel that minimizes MISE is 
\begin{equation*} 
	K_E(x)  = \left\{
\begin{array}{rl}
-\frac{3}{4}(1-x^2) &  |x| \leq 1\\
0 & \text{if }  |x| > 1
\end{array} \right.
\end{equation*}  
The resulting bandwidth becomes $h^* \approx 1.06\hat{\sigma}n^{-\frac{1}{5}}$, where $\hat{\sigma}$ is the sample standard deviation.  The choice relies on the approximation for $\sigma$ for $f(x)$, can obtain different answers.
}

\subsection{Adapative kernels}
\frame
{
Adapative kernels were derived to alleviate this problem.  If use mroe general smoothing function tied to the density at $x_j$, could generalize the denisty estimator as
\begin{equation} \label{re:adap}
	\hat{f}(x)   =  \frac{1}{n} \sum^n_{i=1}{\frac{1}{h_{n,i}}K\left(\frac{x-x_i}{h_{n,i}}\right)} 
\end{equation}
where $h_n$ is a decreasing function of $n$, under $|\hat{f}(x)-f(x)| \stackrel{P}{\rightarrow} 0$
}

\frame
{
In a majority of practical applications the parameter of smoothness is
unknown. After all, nonparametric estimation is typically a first glance at
the data at hand. Thus the aim of one paper I researched is to discuss estimates that
are adaptive to an unknown smoothness, i.e., data-driven.  While the MISE of adaptive estimators can attain the minimax rate, that is, it is not necessary to know the parameter, (can visualize in data).  See Efromovich (1999) pg 281.
}


\section{Financial Application}
\subsection{Setup of Problem}
\frame
{
In finance, there exists a need to analyze the distribution of returns of various indexes.  From these calculations, can obtain measurements of risk.  The value-at-risk (VaR) is a measurement which accounts for a confidence interval for the amount of loss one may expect.  There exists a percentage of certainity that the portfolio manager will not lose more than the value of the Var of the portfolio in the next $N$ days.
}
\subsection{Setup of Problem: VaR}
\frame
{
if a portfolio of stocks has a one-day $5\%$ VaR of $\$1$ million, there is a $5\%$ probability that the portfolio will decline in value by more than $\$1$ million over the next day, assuming markets are normal and there is no trading.
\vspace{.15cm}

The reason for assuming normal markets and no trading, and to restricting loss to things measured in daily accounts, is to make the loss observable. In some extreme financial events it can be impossible to determine losses, either because market prices are unavailable or because the loss-bearing institution breaks up.
\begin{figure}
\includegraphics[scale=0.15]{VaR_graph.png} 
\caption{Illustration of the $10\%$ Value at Risk with normally distibuted portfolio value}
\end{figure}
}

\subsection{Financial Definitions}
\frame
{
Collected historical daily last price for 10 year Treasury notes, 3 month Treasury Bills, and 1 month Treasury bills.
\begin{itemize}
\item \underline{Treasury bills:} (T-bill) A short-term debt obligation backed by the U.S. government with a maturity of less than one year. (maturity is one year or less)
\item \underline{Treasury notes:} (T-notes) similary as above, just mature in two to ten years
\end{itemize} 
}

\subsection{Purpose}
\frame
{
\begin{flushleft}
\underline{Goal:}
\end{flushleft}
Stochastic dynamics of stock prices is commonly described by a geometric (multiplicative) Brownian motion, which gives a log-normal pdf for returns. However, numerous observations show that the tails of the PDF decays lower than the log–normal distribution predicts (the so-called ‘fat-tails’ effect).  Which does not provide a substantial estimate for for calcaulting VaR, area under the curve, when returns are high spreads.  
\begin{figure}
\includegraphics[scale=0.25]{10yrhist.png} 
\end{figure}
}

\section{Matlab Programs}
\frame
{
\begin{flushleft}
\textit{ksdensity}
\end{flushleft}
I specified a kernel function of four pre-selected built-in functions in matlab statistical toolbox, 'normal', 'epanechinikov', 'box', and 'triangle', which are all scaled to have standard deviation equal to one, so the bandwidth parameter means roughly the same thing regardless of kernel function.
}

\frame
{
The default estimator is based on a Normal kernel, Box, Triangle, and Epanechnikov.  The next figure shows how the normal kernel compares to the each other kernel.  The optimal bandwidth for 1 month T-bills (0.0036),  3 month T-bills (0.0027), and 10 year T-notes (0.0028) was determined.  

}

\section{Simulation Results}
\frame
{
\begin{figure}
\includegraphics[scale=0.25]{1mongraphs.png} 
\includegraphics[scale=0.25]{3mongraphs.png} 
\includegraphics[scale=0.25]{10mongraphs.png} 
\end{figure}
The density estimates are roughly comparable, but the box kernel produces a density that is rougher than the others. 
}


\section{Conclusion}
\frame
{
Futher computational research needs to be completed with the above obtained density curve estimations.  Now that a density curve has been approximated, a computational calculation of the area under the curve can done by using a Monte Carlo approach.
}

\section{References}
\frame
{
\begin{itemize}
\item A Course in Mathematical Statistics and Large Sample Theory, by Rabi Bhattacharya
and Victor Patrangenaru, 2009, Springer.
\item V. A. Epanechnikov, Non-Parametric Estimation of a Multivariate Probability Density,  Theory Probab. Appl. Volume 14, Issue 1, pp. 153-158 (January 1969) Issue Date: January 1969 
\item Efromovich, S.  (1999).  Nonparametric Curve Estimation: Methods, Theory and Applications. Springer Series in Statistics.  New York, N.Y.: Springer.
\item Kvam, P. H., \& Vidakovic, B. (2007). Nonparametric statistics with applications to science and engineering. Wiley series in probability and statistics. Hoboken, N.J.: Wiley-Interscience. 
\item Tapia, R. A., \& Thompson, J. R. (1978). Nonparametric probability density estimation. Baltimore: Johns Hopkins University Press.
\item Hull, J. (1989). Options, futures, and other derivative securities. Englewood Cliffs, N.J.: Prentice Hall. 
\item Hull, J. (2007). Risk management and financial institutions. Upper Saddle River, NJ: Pearson Prentice Hall.

\end{itemize}
}

\end{document}