Kurssin kuvaus
Tilastollinen päättely II on 10 op:n tilastotieteen syventävien opintojen kurssi.
Kurssilla keskitytään lähinnä parametrisiin malleihin liittyviin päättelyperiaatteisiin, erityisesti uskottavuusfunktioon (likelihood function) perustuvaan päättelyyn:
- konvergenssikäsitteet, raja-arvolauseita (heikko suurten lukujen laki ja keskeinen raja-arvolause)
- uskottavuuspäättelyä: suurimman uskottavuuden estimaattoreiden asymptoottiset ominaisuudet (tarkentuvuus ja asymptoottinen normaalisuus), luottamusvälit, eksponenttiperheet
- pienimmän neliösumman menetelmä ja momenttimenetelmä: tarkentuvuus ja asymptoottinen normaalisuus
- testiteoriaa: LR-, score- ja Wald- testien asymptoottiset ominaisuudet, sovelluksia lineaariseen malliin
Esitietoina suositellaan 800120P Analyysi I, 800322A Analyysi II, 805310A Tilastollinen päättely I.
Luennot
Luennot pidetään keväällä 2015. Luennot ovat tiistaisin 10-12 salissa M204 ja torstaisin 14-16 salissa M304.
Luentokurssi kestää 14 viikkoa siten, että kurssiin kuuluu yhteensä 28 kahden tunnin luentokertaa, eli yhteensä 56 tuntia luentoja.
- Luento 1, 13.1 ti: 2. Variation; 2.1 Statistics and Sampling Variation; 2.1.1 Data Summaries: sample moments, order statistics, quantiles, histogram, empirical distribution function, scatter plot, box plot, QQ plots.
- Luento 2, 15.1 to: 2.1.2 Random Sample: examples of distributions: normal, exponential, double exponential, gamma, Bernoulli, and Poisson. 2.1.3 Sampling Variation: mean and variance of sample mean, mean of sample variance. 2.2 Convergence; 2.2.1 Modes of convergence: convergence in probability, convergence in distribution, weak law of large numbers, central limit theorem.
- Luento 3, 20.1 ti: convergence in probability of empirical distribution function, Slutsky's lemma and related results, convergence of sample variance and covariance, convergence in probability and in distribution for random vectors, Studentized statistic. 2.2.2 Delta method; the proof in the univariate case.
- Luento 4, 22.1 to: The delta-method in the multivariate case, the approximate distribution of a ratio of statistics, the approximate distribution of a Sharpe ratio. The big-O and little-o notation.
- Luento 5, 27.1 ti: CLT for a random vector, uniform stochastic convergenece, CLT for dependent random variables. 2.3 Order statistics; derivation of the density function of the r:th order statistics.
- Luento 6, 29.1 to: density function of the r:th order statistics of the uniform distribution, expected value of the r:th order statistics with delta method, the limit distribution of the r:th order statistics without proof, the limit distribution of the r:th order statistics of a normal distribution, Renyi's representation, the asymptotic distribution of the minimum of a sample from a Pareto distribution. 2.4 Moments and Cumulants; Moment generating function, Cumulant generating function, Cumulant generating function of a normal distribution.
- Luento 7, 3.2 ti: Cumulant generating function of a linear combination of independent random variables and of a linear combination of normal random variables. Cumulant generating function of a random vector. Cumulant generating function and the central limit theorem, skewness and kurtosis. 3. Uncertainty: 3.1 Confidence intervals; 3.1.1 Standard errors and pivots; pivotal quantity, normal and exponential distribution as examples, asymptotically pivotal quantity, a confidence interval for an asymptotically normal pivot.
- Luento 8, 5.2 to: confidence intervals for the mean of a binomial and a gamma distribution. 3.1.2 Choice of Scale; binomial distribution and the logarithmic transform, Poisson distribution and the square root transform, general case of scale choice with the delta-method. 3.1.3 Tests; p-value of a test.
- Luento 9, 10.2 ti: 3.2 Normal Model; 3.2.1 Normal and related distributions; moment generating function of a univariate normal distribution, chi-squared distribution, moment generating function of a chi-squared distribution, t-distribution, expectation and variance of the t-distribution, F-distribution. 3.2.2 Normal random sample; studentized sample mean from a normal distribution. 3.2.3 Multivariate normal distribution; covariance matrix, covariance matrix of a linear transform.
- Luento 10, 12.2 to: derivation of marginal distributions with the moment generating function, moment generating function of independent random vectors, conditional distributions for normal distribution, distribution of linear combinations with the moment generating function, two sample problem. 3.3 Simulation; 3.3.2 Variance reduction; importance sampling.
- Luento 11, 17.2 ti: 4. Likelihood; 4.1 Likelihood; 4.1.1 Definition and examples; likelihood function of i.i.d data, likelihood function for linear regression and panel data, likelihood function of dependent data, likelihood function of AR(1) process. 4.1.2 Basic properties; log-likelihood, transformations in sample space, transformations in parameter space.
- Luento 12, 19.2 to: 4.2 Summaries; 4.2.2 Sufficient statistics; definition of sufficient statistics, factorization criterion, a proof of the factorization criterion for a discrete distribution. derivation of sufficient statistics for samples from Bernoulli and exponential distributions using the definition of sufficiency and the factorization criterion, sufficiency of ordered sample, minimal sufficiency. 4.3 Information; 4.3.1 Observed and expected information; Definition of observed and expected information, expected information for i.i.d sample, observed and expected information for a sample from a Bernoulli distribution (binomial distribution).
- Luento 13, 24.2 ti: Observed and expected information for a sample from 1D normal distribution, observed and expected information for censored data. 4.4 Maximum likelihood estimator; 4.4.1 Computation; solving the score equation, normal distribution as an example, Newton-Raphson algorithm, one-step estimator.
- Luento 14, 26.2 to: construction of confidence intervals and confidence regions using ML estimator, regular satistical model, expectation and covariance matrix of score vector, Kullback-Leibler distance, proof of weak consistency of ML estimator (P.S. Lause 2.2) proof of asymptotic normality of ML estimator (P.S. Lause 2.3).
- Luento 15, 3.3 ti: 4.5 Likelihood ratio statistics; 4.5.1 Basic ideas; asymptotic distribution of the likelihood ratio statistics, confidence region based on the likelihood ratio statistics. 4.5.2 Profile likelihood; asymptotic distribution of the generalized likelihood ratio statistics, confidence region based on the generalized likelihood ratio statistics.
- Luento 16, 5.3 to: Testing linear restrictions, Wald's test (P.S. Luku 2.6.1). 4.6 Non-regular models; parameter identifiability, normal mixture model and the nonexistence of the maximum likelihood estimator.
- Luento 17, 17.3 ti: 5. Models; 5.1 Straight-line regression; distribution of the maximum likelihood estimator in the normal linear model, asymptotic distribution of the least squares estimator in the linear model (P.S. Luku 2.5.2).
- Luento 18, 19.3 to: 5.2 Exponential family models; 5.2.1 Basic notions; Definition of an exponential family and of a natural exponential family, Hölder's inequality, a proof that the parameter space is convex, the family of exponential distributions as an example, the family of binomial distributions as an example, the cumulant generating function of a natural observation, expectation and variance of a distribution in an exponential family.
- Luento 19, 24.3 ti: Binary response model. 5.2.2 Families of order p; exponential family with p-dimensional parameter. 5.2.3 Inference; Sufficient statistics of an exponential family, distribution of the sufficient statistics belongs to an exponential family. 7. Estimation and Hypothesis Testing; 7.1 Estimation; 7.1.1 Mean squared error; Bias-variance decomposition of the mean squared error, the mean squared error of the maximum likelihood estimator of the variance, and the mean squared error of the biased estimator of the variance, in the normal model. a proof of the Cramer-Rao lower bound for a one dimensional parameter, a statment of the Cramer-Rao lower bound for a vector parameter.
- Luento 20, 26.3 to: 7.1.2 Kernel density estimation; the definition of kernel density estimator, analogy to the histogram estimator, asymptotic bias and variance of the kernel density estimator at one point. minimizing the asymptotic mean squared error of a kernel density estimator at one point, plug-in smoothing parameter selection, normal reference rule, cross validation.
- Luento 21, 31.3 ti: 7.1.3 Minimum variance unbiased estimation; Rao-Blackwell theorem, complete statistics, Poisson distribution as an example, construction of a minimum variance unbiased estimator. 7.2 Estimating functions; 7.2.1 Basic notions; the definition of the method of estimating functions, maximum likelihood method and the method of moments as examples,
- Luento 22, 2.4 to: estimation in the normal family and in the Weibull family with the method of moments, the proof of the consistency of the estimator in the method of estimating functions. 7.3 Hypothesis testing; 7.3.1 Significance levels; likelihood ratio test for exponential densities, estimation of the size of a test with Monte Carlo. 7.3.2 Comparison of tests; critical region, type I and type II error, the size and power of a test, the size and power when testing the normal mean.
- Luento 23, 7.4 ti: proof of Neyman-Pearson lemma, Neyman-Pearson test for exponential families, uniformly most powerful test. 7.3.4 Link with confidence intervals.
- Luento 24, 9.4 to: 10.7 Semiparametric regression; nonparametric regression: regressogram, kernel regression estimator, nearest neighborhood regression estimator, local linear estimator, weighted linear regression.
- Luento 25, 14.4 ti: Time series prediction: sum of squared prediction errors (in-sample vs. out-of-sample), nonparametric generalized likelihood ratio test, bootstrap.
- Luento 26, 16.4 to: 6.4 Prediction in AR and MA models, moving averages.
- Luento 27, 21.4 ti: 6.1 Markov chains, homogeneous Markov chains, transition matrix, stationary distribution, pageRank algorithm of Google, ergodic Markov chains.
- Luento 28, 23.4 to: Summary
Laskuharjoitukset
Laskuharjoituksia pidetään 14 kertaa. Laskuharjoitukset ovat keskiviikkoisin 16-18 salissa M304. Harjoitukset alkavat 21.1.2015.
- Harjoitus 1, 21.1. Laskuharjoitus 1 Tietokoneharjoitusten 1 vastaukset
- Harjoitus 2, 28.1. Laskuharjoitus 2 Tietokoneharjoitusten 2 vastaukset
- Harjoitus 3, 4.2. Laskuharjoitus 3 Tietokoneharjoitusten 3 vastaukset
- Harjoitus 4, 11.2. Laskuharjoitus 4 Tietokoneharjoitusten 4 vastaukset
- Harjoitus 5, 18.2. Laskuharjoitus 5 Tietokoneharjoitusten 5 vastaukset
- Harjoitus 6, 25.2. Laskuharjoitus 6 Tietokoneharjoitusten 6 vastaukset
- Harjoitus 7, 4.3. Laskuharjoitus 7
- Harjoitus 8, 18.3. Laskuharjoitus 8
- Harjoitus 9, 25.3. Laskuharjoitus 9 Tietokoneharjoitusten 9 vastaukset
- Harjoitus 10, 1.4. Laskuharjoitus 10
- Harjoitus 11, 8.4. Laskuharjoitus 11
- Harjoitus 12, 15.4. Laskuharjoitus 12
- Harjoitus 13, 22.4. Laskuharjoitus 13
- Harjoitus 14, 29.4. Laskuharjoitus 14
Harjoituksista voi saada lisäpisteitä. Lisäpisteitä annetaan suorassa suhteessa tehtyjen tehtävien määrään. Lisäpisteiden suurin määrä on 10. Koko kurssin maksimipistemäärä on 30, koska tentissä on 5 tehtävää ja jokaisesta tehtävästä voi saada 6 pistettä. Täydet pisteet voi siis saada pelkästään tenttiin osallistumalla.
Tentti
Tentti pidetään laitoksen yleistentissä 11.5.2015.
Kirjallisuus
Kurssin pohjana oleva kirjallisuus
- Davison, A.C. (2003). Statistical Models, Cambridge Series in Statistical and Probabilistic Mathematics.
Kurssiin liittyvää suositeltavaa kirjallisuutta
- https://wiki.helsinki.fi/download/attachments/59061003/Paattely_II(10).pdf
- http://cc.oulu.fi/~mrahiala/mtt2.pdf
- E. Lehmann & G. Casella: Theory of Point Estimation.
- L. Wasserman: All of Nonparametric Statistics, Springer, 2006.
- A. B. Tsybakov: Introduction to Nonparametric Estimation, Springer, 2008
- A. Korostelev & O. Korosteleva: Mathematical Statistics: Asymptotic Minimax Theory, AMS, 2011