Masalah pada estimasi parameter

Biarkan dan menjadi empat variabel acak sehingga , di mana adalah parameter yang tidak diketahui. Juga asumsikan bahwa ,Lalu yang mana yang benar?θ 1 , θ 2 , θ 3 V a r ( Y i ) = σ 2 i = 1 , 2 , 3 , $Y_1,Y_2,Y_3$$Y_4$$E(Y_1)=\theta_1-\theta_3;\space\space E(Y_2)=\theta_1+\theta_2-\theta_3;\space\space E(Y_3)=\theta_1-\theta_3;\space\space E(Y_4)=\theta_1-\theta_2-\theta_3$$\theta_1,\theta_2,\theta_3$$Var(Y_i)=\sigma^2$$i=1,2,3,4.$

A. dapat diperkirakan.$\theta_1,\theta_2,\theta_3$

B. dapat diperkirakan.$\theta_1+\theta_3$

C. dapat diperkirakan dan adalah estimasi linear tidak bias terbaik dari .$\theta_1-\theta_3$$\dfrac{1}{2}(Y_1+Y_3)$$\theta_1-\theta_3$

D. dapat diperkirakan.$\theta_2$

Jawaban yang diberikan adalah C yang terlihat aneh bagi saya (karena saya mendapat D).

Kenapa saya mendapat D? Sejak, .$E(Y_2-Y_4)=2\theta_2$

Mengapa saya tidak mengerti bahwa C bisa menjadi jawaban? Oke, saya bisa melihat, adalah penaksir yang tidak bias dari , dan kurang dari .$\dfrac{Y_1+Y_2+Y_3+Y_4}{4}$$\theta_1-\theta_3$$\dfrac{Y_1+Y_3}{2}$

Tolong beritahu saya di mana saya melakukan kesalahan.

Juga diposting di sini: /math/2568894/a-problem-on-estimability-of-parameters

Jawaban ini menekankan verifikasi kemungkinan. Properti varians minimum adalah pertimbangan sekunder saya.

Untuk memulainya, rangkumlah informasi dalam bentuk matriks dari model linier sebagai berikut: mana

$$\begin{align} Y := \begin{bmatrix} Y_1 \\ Y_2 \\ Y_3 \\ Y_4 \end{bmatrix} = \begin{bmatrix} 1 & 0 & -1 \\ 1 & 1 & -1 \\ 1 & 0 & -1 \\ 1 & -1 & -1 \\ \end{bmatrix} \begin{bmatrix} \theta_1 \\ \theta_2 \\ \theta_3 \end{bmatrix} + \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \varepsilon_3 \\ \varepsilon_4 \end{bmatrix}:= X\beta + \varepsilon, \tag{1} \end{align}$$ (untuk membahas estimasi, asumsi spherity tidak diperlukan. Tetapi untuk membahas properti Gauss-Markov, kita perlu mengasumsikan spherity ε ).$E(\varepsilon) = 0, \text{Var}(\varepsilon) = \sigma^2 I$$\varepsilon$

Jika desain matriks adalah peringkat penuh, maka parameter asli β mengakui unik kuadrat-memperkirakan β = ( X ' X ) - 1 X ' Y . Akibatnya, setiap parameter φ , yang didefinisikan sebagai fungsi linear φ ( β ) dari β adalah diduga dalam arti bahwa hal itu dapat jelas diperkirakan oleh data melalui kuadrat-memperkirakan β sebagai φ = p ' β .$X$$\beta$$\hat{\beta} = (X'X)^{-1}X'Y$$\phi$$\phi(\beta)$$\beta$$\hat{\beta}$$\hat{\phi} = p'\hat{\beta}$

Kehalusan muncul ketika tidak dari peringkat penuh. Untuk melakukan diskusi yang menyeluruh, kami memperbaiki beberapa notasi dan istilah pertama di bawah ini (Saya mengikuti konvensi Pendekatan Bebas Koordinat untuk Model Linier , Bagian 4.8. Beberapa istilah terdengar teknis tidak perlu). Selain itu, diskusi ini berlaku untuk model linear umum Y = X β + ε dengan X ∈ R n × k dan β ∈ R k .$X$$Y = X\beta + \varepsilon$$X \in \mathbb{R}^{n \times k}$$\beta \in \mathbb{R}^k$

1. Sebuah berjenis regresi adalah koleksi vektor mean sebagai bervariasi dari R k : M = { X β : β ∈ R k } $\beta$$\mathbb{R}^k$ $$M = \{X\beta: \beta \in \mathbb{R}^k\}.$$
2. Sebuah fungsional parametrik adalah fungsional linear dari β , φ ( β ) = p ' β = p 1 β 1 + ⋯ + p k β k$\phi = \phi(\beta)$$\beta$ $$\phi(\beta) = p'\beta = p_1\beta_1 + \cdots + p_k\beta_k.$$

Seperti disebutkan di atas, ketika , tidak setiap fungsional parametrik ϕ ( β ) dapat diperkirakan. Tapi, tunggu, apa definisi dari istilah yang dapat diperkirakan secara teknis? Tampaknya sulit untuk memberikan definisi yang jelas tanpa mengganggu aljabar linear kecil. Salah satu definisi, yang saya pikir paling intuitif, adalah sebagai berikut (dari referensi yang sama):$\text{rank}(X) < k$$\phi(\beta)$

Definisi 1. Fungsi parametrik dapat diperkirakan jika secara unik ditentukan oleh X β dalam arti bahwa ϕ ( β 1 ) = ϕ ( β 2 ) setiap kali β 1 , β 2 ∈ R k memenuhi X β 1 = X β 2 .$\phi(\beta)$$X\beta$$\phi(\beta_1) = \phi(\beta_2)$$\beta_1,\beta_2 \in \mathbb{R}^k$$X\beta_1 = X\beta_2$

Penafsiran. Definisi di atas menetapkan bahwa pemetaan dari manifold regresi ke ruang parameter ϕ harus satu-ke-satu, yang dijamin ketika peringkat ( X ) = k (yaitu, ketika X itu sendiri adalah satu-ke-satu). Ketika peringkat ( X ) < k , kita tahu bahwa ada β 1 ≠ β 2 sehingga X β 1 = X β $M$$\phi$$\text{rank}(X) = k$$X$$\text{rank}(X) < k$$\beta_1 \neq \beta_2$$X\beta_1 = X\beta_2$. Definisi yang dapat diperkirakan di atas dalam efek mengesampingkan fungsi-fungsi parametrik kekurangan struktural yang menghasilkan nilai yang berbeda sendiri bahkan dengan nilai yang sama pada , yang tidak masuk akal secara alami. Di sisi lain, fungsional parametrik yang diperkirakan ϕ ( ⋅ ) memang memungkinkan kasus ϕ ( β 1 ) = ϕ ( β 2 ) dengan β 1 ≠ β 2 , selama kondisi X β 1 = X β 2 terpenuhi.$M$$\phi(\cdot)$$\phi(\beta_1) = \phi(\beta_2)$$\beta_1 \neq \beta_2$$X\beta_1 = X\beta_2$

Ada kondisi lain yang setara untuk memeriksa kemungkinan fungsional parametrik yang diberikan dalam referensi yang sama, Proposisi 8.4.

Setelah pengenalan latar belakang yang begitu jelas, mari kembali ke pertanyaan Anda.

A. sendiri tidak dapat diperkirakan karena peringkat ( X ) < 3 , yang mencakup X β 1 = X β 2 dengan β 1 ≠ β 2 . Meskipun definisi di atas diberikan untuk fungsi skalar, ia mudah digeneralisasikan ke fungsional bernilai vektor.$\beta$$\text{rank}(X) < 3$$X\beta_1 = X\beta_2$$\beta_1 \neq \beta_2$

B. $\phi_1(\beta) = \theta_1 + \theta_3 = (1, 0, 1)'\beta$ is non-estimable. To wit, consider $\beta_1 = (0, 1, 0)'$ and $\beta_2 = (1, 1, 1)'$, which gives $X\beta_1 = X\beta_2$ but $\phi_1(\beta_1) = 0 + 0 = 0 \neq \phi_1(\beta_2) = 1 + 1 = 2$.

C. $\phi_2(\beta) = \theta_1 - \theta_3 = (1, 0, -1)'\beta$ is estimable. Because $X\beta_1 = X\beta_2$ trivially implies $\theta_1^{(1)} - \theta_3^{(1)} = \theta_1^{(2)} - \theta_3^{(2)}$, i.e., $\phi_2(\beta_1) = \phi_2(\beta_2)$.

D. $\phi_3(\beta) = \theta_2 = (0, 1, 0)'\beta$ is also estimable. The derivation from $X\beta_1 = X\beta_2$ to $\phi_3(\beta_1) = \phi_3(\beta_2)$ is also trivial.

After the estimability is verified, there is a theorem (Proposition 8.16, same reference) claims the Gauss-Markov property of $\phi(\beta)$. Based on that theorem, the second part of option C is incorrect. The best linear unbiased estimate is $\bar{Y} = (Y_1 + Y_2 + Y_3 + Y_4)/4$, by the theorem below.

Theorem. Let $\phi(\beta) = p'\beta$ be an estimable parametric functional, then its best linear unbiased estimate (aka, Gauss-Markov estimate) is $\phi(\hat{\beta})$ for any solution $\hat{\beta}$ to the normal equations $X'X\hat{\beta} = X'Y$.

The proof goes as follows:

Proof. Straightforward calculation shows that the normal equations is

$$\begin{equation} \begin{bmatrix} 4 & 0 & -4 \\ 0 & 2 & 0 \\ -4 & 0 & 4 \end{bmatrix} \hat{\beta} = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 0 & 1 & 0 & -1 \\ -1 & -1 & -1 & -1 \end{bmatrix} Y, \end{equation}$$ which, after simplification, is $$\begin{equation} \begin{bmatrix} \phi(\hat{\beta}) \\ \hat{\theta}_2/2 \\ -\phi(\hat{\beta}) \end{bmatrix} = \begin{bmatrix} \bar{Y} \\ (Y_2 - Y_4)/4 \\ -\bar{Y} \end{bmatrix}, \end{equation}$$ i.e., $\phi(\hat{\beta}) = \bar{Y}$.

Therefore, option D is the only correct answer.

---

Addendum: The connection of estimability and identifiability

When I was at school, a professor briefly mentioned that the estimability of the parametric functional $\phi$ corresponds to the model identifiability. I took this claim for granted then. However, the equivalance needs to be spelled out more explicitly.

According to A.C. Davison's monograph Statistical Models p.144,

Definition 2. A parametric model in which each parameter $\theta$ generates a different distribution is called identifiable.

For linear model $(1)$, regardless the spherity condition $\text{Var}(\varepsilon) = \sigma^2 I$, it can be reformulated as

$$\begin{equation} E[Y] = X\beta, \quad \beta \in \mathbb{R}^k. \tag{2} \end{equation}$$

It is such a simple model that we only specified the first moment form of the response vector $Y$. When $\text{rank}(X) = k$, model $(2)$ is identifiable since $\beta_1 \neq \beta_2$ implies $X\beta_1 \neq X\beta_2$ (the word "distribution" in the original definition, naturally reduces to "mean" under model $(2)$.).

Now suppose that $\text{rank}(X) < k$ and a given parametric functional $\phi(\beta) = p'\beta$, how do we reconcile Definition 1 and Definition 2?

Well, by manipulating notations and words, we can show that (the "proof" is rather trivial) the estimability of $\phi(\beta)$ is equivalent to that the model $(2)$ is identifiable when it is parametrized with parameter $\phi = \phi(\beta) = p'\beta$ (the design matrix $X$ is likely to change accordingly). To prove, suppose $\phi(\beta)$ is estimable so that $X\beta_1 = X\beta_2$ implies $p'\beta_1 = p'\beta_2$, by definition, this is $\phi_1 = \phi_2$, hence model $(3)$ is identifiable when indexing with $\phi$. Conversely, suppose model $(3)$ is identifiable so that $X\beta_1 = X\beta_2$ implies $\phi_1 = \phi_2$, which is trivially $\phi_1(\beta) = \phi_2(\beta)$.

Intuitively, when $X$ is reduced-ranked, the model with $\beta$ is parameter redundant (too many parameters) hence a non-redundant lower-dimensional reparametrization (which could consist of a collection of linear functionals) is possible. When is such new representation possible? The key is estimability.

To illustrate the above statements, let's reconsider your example. We have verified parametric functionals $\phi_2(\beta) = \theta_1 - \theta_3$ and $\phi_3(\beta) = \theta_2$ are estimable. Therefore, we can rewrite the model $(1)$ in terms of the reparametrized parameter $(\phi_2, \phi_3)'$ as follows

$$\begin{equation} E[Y] = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 0 \\ 1 & - 1 \end{bmatrix} \begin{bmatrix} \phi_2 \\ \phi_3 \end{bmatrix} = \tilde{X}\gamma. \end{equation}$$

Clearly, since $\tilde{X}$ is full-ranked, the model with the new parameter $\gamma$ is identifiable.

Apply the definitions.

I will provide details to demonstrate how you can use elementary techniques: you don't need to know any special theorems about estimation, nor will it be necessary to assume anything about the (marginal) distributions of the $Y_i$. We will need to supply one missing assumption about the moments of their joint distribution.

Definitions

All linear estimates are of the form

$$t_\lambda(Y) = \sum_{i=1}^4 \lambda_i Y_i$$ for constants $\lambda = (\lambda_i)$.

An estimator of $\theta_1-\theta_3$ is unbiased if and only if its expectation is $\theta_1-\theta_3$. By linearity of expectation,

$$\eqalign{ \theta_1 - \theta_3 &= E[t_\lambda(Y)] = \sum_{i=1}^4 \lambda_i E[Y_i]\\ & = \lambda_1(\theta_1-\theta_3) + \lambda_2(\theta_1+\theta_2-\theta_3) + \lambda_3(\theta_1-\theta_3) + \lambda_4(\theta_1-\theta_2-\theta_3) \\ &=(\lambda_1+\lambda_2+\lambda_3+\lambda_4)(\theta_1-\theta_3) + (\lambda_2-\lambda_4)\theta_2. }$$

Comparing coefficients of the unknown quantities $\theta_i$ reveals

$$\lambda_2-\lambda_4=0\text{ and }\lambda_1+\lambda_2+\lambda_3+\lambda_4=1.\tag{1}$$

In the context of linear unbiased estimation, "best" always means with least variance. The variance of $t_\lambda$ is

$$\operatorname{Var}(t_\lambda) = \sum_{i=1}^4 \lambda_i^2 \operatorname{Var}(Y_i) + \sum_{i\ne j}^4 \lambda_i\lambda_j \operatorname{Cov}(Y_i,Y_j).$$

The only way to make progress is to add an assumption about the covariances: most likely, the question intended to stipulate they are all zero. (This does not imply the $Y_i$ are independent. Furthermore, the problem can be solved by making any assumption that stipulates those covariances up to a common multiplicative constant. The solution depends on the covariance structure.)

Since $\operatorname{Var}(Y_i)=\sigma^2,$ we obtain

$$\operatorname{Var}(t_\lambda) =\sigma^2(\lambda_1^2 + \lambda_2^2 + \lambda_3^2 + \lambda_4^2).\tag{2}$$

The problem therefore is to minimize $(2)$ subject to constraints $(1)$.

Solution

The constraints $(1)$ permit us to express all the $\lambda_i$ in terms of just two linear combinations of them. Let $u=\lambda_1-\lambda_3$ and $v=\lambda_1+\lambda_3$ (which are linearly independent). These determine $\lambda_1$ and $\lambda_3$ while the constraints determine $\lambda_2$ and $\lambda_4$. All we have to do is minimize $(2)$, which can be written

$$\sigma^2(\lambda_1^2 + \lambda_2^2 + \lambda_3^2 + \lambda_4^2) = \frac{\sigma^2}{4}\left(2u^2 + (2v-1)^2 + 1\right).$$

No constraints apply to $(u,v)$. Assume $\sigma^2 \ne 0$ (so that the variables aren't just constants). Since $u^2$ and $(2v-1)^2$ are smallest only when $u=2v-1=0$, it is now obvious that the unique solution is

$$\lambda = (\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1/4,1/4,1/4,1/4).$$

Option (C) is false because it does not give the best unbiased linear estimator. Option (D), although it doesn't give full information, nevertheless is correct, because

$$\theta_2 = E[t_{(0,1/2,0,-1/2)}(Y)]$$

is the expectation of a linear estimator.

It is easy to see that neither (A) nor (B) can be correct, because the space of expectations of linear estimators is generated by $\{\theta_2, \theta_1-\theta_3\}$ and none of $\theta_1,\theta_3,$ or $\theta_1+\theta_3$ are in that space.

Consequently (D) is the unique correct answer.