Derivative w.r.t orthogonal matrix Let $A$ be an orthogonal matrix with elements $a_{ij}$ so that $\sum_k a_{ik} a_{jk}  = \delta_{ij}$. I'd like to know what is $\frac{ \partial a_{ij} }{ \partial a_{kl}}$. If $A$ was a generic matrix (with no constraints on its elements) then the answer would be $\delta_{ik} \delta_{jl}$. However, now we have the quadratic constraint on the matrix. What is the answer in this case?

Your question is non-sense but it is difficult to explain why. That follows is an example for $n=3$.
There is a one to one local diffeomorphism $f:K\in SK_3\rightarrow (I_3-K)(I_3+K)^{-1}\in SO_3$ where $SK_3$ is the set of skew symmetric matrices. In particular,
let $U=\begin{pmatrix}-18/23& 14/23& 3/23\\6/23& 3/23& 22/23\\13/23& 18/23& -6/23\end{pmatrix}\in O_3=f(\begin{pmatrix}0&-4&5\\4&0&-2\\-5&2&0\end{pmatrix})$.
Thus $U=[u_{i,j}]\in O_3$ depends on $3$ independent parameters. Locally, we can choose these parameters amongst the $(u_{i,j})$, but they cannot stand on the same row or column. We are interested by the derivative $\frac{ \partial u_{1,2} }{ \partial u_{1,1}}$. Then $u_{1,1}$ must be a parameter and $u_{1,2}$ must not (otherwise, the result is obvious).
We consider the $2$ following parametrizations
Choice 1. $u_{1,1},u_{2,2},u_{3,3}$. Then the derivative in our $U$ is $\frac{ \partial u_{1,2} }{ \partial u_{1,1}}\approx 1.46246$.
Choice 2. $u_{1,1},u_{2,3},u_{3,2}$. Then the derivative in our $U$ is $\frac{ \partial u_{1,2} }{ \partial u_{1,1}}\approx 0.666603$.
You can see that the result depends on the choice of the chosen local parametrization of $SO_3$.
EDIT. Answer to @Adam . Yes, in $SO(2)$, there is no problem because the parametrization contains only one parameter; for example, if $U(\theta)=\begin{pmatrix}\cos(\theta)&-\sin(\theta)\\\sin(\theta)&\cos(\theta)\end{pmatrix}$ with $u_{1,1}=f(u_{2,1})>0$, then $\dfrac{ \partial u_{1,1} }{ \partial u_{2,1}}=\dfrac{-\sin(\theta)}{\cos(\theta)}=\dfrac{-u_{2,1}}{\sqrt{1-u_{2,1}^2}}$. Yet, an element of $SO(3)$ depends on $3$ parameters and you must choose these parameters to calculate a partial derivative with respect to one of these parameters.
