Q:

Learning prior distribution from data

Suppose I have a dataset. How can I learn the prior distributions of the parameters of a model from this data? I want to learn the prior from this data in order to use them in a Bayesian model. Sorry if the question is too simple. 

A:

From a dataset $x_1,\ldots,x_n$, generated from a fixed distribution $f(\cdot;\theta_0)$, one can learn about the single value $\theta_0$ of the parameter that led to its generation, that is, the generation of this dataset, but close to nothing of the prior distribution of this single value, because precisely it is single: even observing $\theta_0$ exactly does not allow me to learn about its distribution $\pi$.
To construct a posterior, one need to make an informed or uninformed choice of a prior. Posteriors cannot be created ex nihilo. However, combining any prior $\pi_0$ and the data $X$ to construct a posterior $\pi_0(\theta|x)$ and then re-start with this posterior as if it was a prior is not part of the Bayesian approach because it uses the data twice:
$$\pi^\text{prop}(\theta)=\frac{\pi_0(\theta) f(x|\theta) f(x|\theta)}{\int  \pi_0(\theta) f^2(x|\theta)\text{d}\theta}$$
which is a proper Bayesian posterior with the data duplicated and thus multiplies by two the information brought by the data. 
To drive the point about the problematic features of this proposal, repeating the "posterior-turns-prior" process over and over actually leads to a prior that is a Dirac mass at the maximum likelihood estimator associated with the data and the model. (This may also prove an alternative scheme to obtain the maximum likelihood estimator. That we called State-Augmentation for Marginal Estimation (SAME), lated copied under the new name of data cloning.)
This illustrates most forcibly how the substitution creates a bias or an over-fit in the analysis and there are several drawbacks well documented in the literature, even though some speak of "empirical Bayes" (Robbins, 1953; Carlin and Louis, 1996) "data-dependent priors" (Wasserman, 2000) or  "integrated likelihoods" (Aitkin, 2009). To quote from a recent thesis by W. Darnieder (Ohio State U):

Curiously, the data-dependent strategy attempts to circumvent the
  often difficult task of establishing prior belief while still
  masquerading as a Bayesian technique.  In so doing, the data-dependent
  Bayesian will incorporate aspects of the observed data into  his 
  assessment  of  prior  information  before  experimentation  (or  at 
  least  before analysis), and in the formal posterior updating in
  accordance with Bayes Theorem. This  double  use  of  the  data  within
  the  Bayesian  paradigm  is  highly  objectionable because it can
  easily be abused to produce virtually any outcome the researcher
  de-sires (Gelman et al., 2003).  This is true even when the researcher
  is not performing a deliberately deceptive analysis, but when the
  researcher may unwittingly be using a data-dependent prior so strong
  that it completely drives posterior inference (...)
Similarly,  those  critical  of  data-dependent  techniques  can 
  originate  their  arguments with the Likelihood Principle which states
  that all experimental information is contained in the data likelihood.
  Generally speaking, empirical Bayes techniques strongly violate this
  principle. Many objective Bayesian methods also violate the Likelihood
  Principle, as we have seen in the examples of Jeffreys, Zellner, and
  Wasserman,but the scope of offense is frequently venial by comparison,
  e.g.  when an objective prior depends only on the sample size. (...)
  And yet the explicitly data-dependent approach still seems worse
  because of its brazen disregard of such a principled approach while
  still invoking the name Bayes.
The above criticisms notwithstanding, the most odious feature of the
  typical use of data-based priors is how the elegant simplicity of
  Bayes’ Theorem has been perverted.  The statement $p(θ|x)∝f(x|θ)π(θ)$
  is true and the usual rules of probability remain in force, while
  $p(θ|x)∝f(x|θ)π(θ|T(x))$ is patently false.  Deely and Lindley (1981) 
  humorously  commented  that,  “It  is  this  use  of  Bayes  formula 
  that  presumably accounts for the reference to Bayes in the term,
  empirical Bayes.”  Worse, the data-dependent formulation completely
  abandons the rules of probability.

A side remark is that Bayesian posteriors can be sequentially updated in the sense that the posterior $\pi(\theta|x_1,\ldots,x_n)$ is also the posterior associated with the prior $\pi(\theta|x_1,\ldots,x_{n-1})$ and the additional observation $x_n$, with $\pi(\theta|x_1,\ldots,x_{n-1})$ itself being the posterior associated with the prior $\pi(\theta|x_1,\ldots,x_{n-2})$ and the additional observation $x_{n-1}$, &tc., &tc.

