Automorphism group of tree as a topological group? I'm reading the paper (https://link.springer.com/article/10.1007%2Fs10711-006-9113-9) and in this paper, author define 'arboreal representation' as the following.

An arboreal representation of a proﬁnite group G is a continuous homomorphism  G→Aut$(T)$, where $T$ is the complete rooted $d$-ary tree for some $d$.

But to define the term 'continuous homomorphism', we need to equip the topology to $G$ and Aut$(T)$. What topolgy? Discrete or any special one? 
And $G$ is already an inverse limit of topological group. Am I right?

Profiniteness is a property of topological groups, not of groups!  So when you say $G$ is a profinite group, that already means it has a topology.  Explicitly, is profinite group is by definition a topological group that is an inverse limit (as a topological group) of a system of finite discrete groups.  So, the topology is the natural inverse limit topology coming from this inverse system of finite groups.
The standard topology to put on $\operatorname{Aut}(T)$ in this context is as the inverse limit of the finite discrete groups $\operatorname{Aut}(T_n)$, where $T_n$ is the truncation of $T$ to height $n$, making $\operatorname{Aut}(T)$ a profinite group.  Explicitly, this is just the topology of pointwise convergence of functions $T\to T$, giving $T$ the discrete topology.
