What is the difference between variable, argument and parameter? I'm sure that these terms should be different since there exists a difference between parameter and argument in computer science but I'm not sure about their differences in math.

Variables : A variable is a quantity that may change within the context of a mathematical problem or experiment. Typically, we use a single letter to represent a variable. The letters $~x,~ y~$ and $~z~$ are common generic symbols used for variables. Sometimes, we will choose a letter that reminds us of the quantity it represents, such as $~t~$ for time, $~v~$ for voltage etc.
Parameters : A parameter is a quantity that influences the output or behavior of a mathematical object but is viewed as being held constant.
Arguments : The word argument is used in several differing contexts in mathematics. The most common usage refers to the argument of a function, but is also commonly used to refer to the complex argument or elliptic argument.
An argument of a function $~f(x_1,...,x_n)~$ is one of the $~n~$ parameters on which the function's value depends. For example, the $~\sin x~$ is a one-argument function, the binomial coefficient $~\binom{n}{m}~$ is a two-argument function, and the hypergeometric function $~_2F_1(a,b;c;z)~$ is a four-argument function.
Note: In general, mathematical functions may have a number of arguments. Arguments that are typically varied when plotting, performing mathematical operations, etc., are termed variables, while those that are not explicitly varied in situations of interest are termed parameters. In some contexts, one can imagine performing multiple experiments, where the variables are changing through each experiment, but the parameters are held fixed during each experiment and only change between experiments. One place parameters appear is within functions. 
Examples :
Ex -$\bf(1)~:~$ A function might a generic quadratic function as $$~f(x)=ax^2+bx+c~.$$
Here, the variable $~x~$ is regarded as the input to the function. The symbols $~a,~ b ~$and $~c~$ are parameters that determine the behavior of the function $~f~$. For each value of the parameters, we get a different function. 
Ex -$\bf(2)~:~$In the standard equation of an ellipse
$$\dfrac{x^2}{a^2}+\dfrac{y^2}{b^2}=1~,$$
$x~$ and $~y~$ are generally considered variables and $~a~$ and $~b~$ are considered parameters.
The decision on which arguments to consider variables and which to consider parameters may be historical or may be based on the application under consideration. However, the nature of a mathematical function may change depending on which choice is made.
