unsupervised learning a priori compression mutual information posterior probability estimation clustering statistical distributions compression filtering supervised learning parameter parameters statistic cost function
In unsupervised learning, input data is given along with the cost function, some function of the data 




x



{\displaystyle \textstyle x}

 and the network's output. The cost function is dependent on the task (the model domain) and any a priori assumptions (the implicit properties of the model, its parameters and the observed variables). As a trivial example, consider the model 




f
(
x
)
=
a



{\displaystyle \textstyle f(x)=a}

 where 




a



{\displaystyle \textstyle a}

 is a constant and the cost 




C
=
E
[
(
x
−
f
(
x
)

)

2


]



{\displaystyle \textstyle C=E[(x-f(x))^{2}]}

. Minimizing this cost produces a value of 




a



{\displaystyle \textstyle a}

 that is equal to the mean of the data. The cost function can be much more complicated. Its form depends on the application: for example, in compression it could be related to the mutual information between 




x



{\displaystyle \textstyle x}

 and 




f
(
x
)



{\displaystyle \textstyle f(x)}

, whereas in statistical modeling, it could be related to the posterior probability of the model given the data (note that in both of those examples, those quantities would be maximized rather than minimized). Tasks that fall within the paradigm of unsupervised learning are in general estimation problems; the applications include clustering, the estimation of statistical distributions, compression and filtering.
