I'd like to set-up X11 Forwarding to run remote X applications on the server and X11 client can handle the UX interaction.However I would like to configure the X11 client or fake it to actual do the user interaction programatically (from the script).For example I want to run an app which requires some mouse clicks or keyboard interaction (such as installers), so I could programatically send these clicks or keyboard key press signals to the app from the script until it finishes.Is it something possible? How this can be achieved or where to start? Or how I can re-use/hijack X11 protocol to inject my own non-user interaction?
How to programatically control X11 forwarded apps?
ssh;scripting;x11;application;xforwarding
So far I've found xdotool tool which can fake input from the mouse and keyboard quite easily.Some examples with simple keyboard interaction:xdotool key axdotool key Downxdotool key Tabxdotool key Returnxdotool key Control_L+txdotool key Alt+1xdotool key ctrl+vxdotool key Superxdotool key KP_Enterxdotool key ctrl+Page_Upxdotool key ctrl+U005Cxdotool key ctrl+shift+exdotool key --delay 1000  shift+minus # for underscorexdotool key --clearmodifiers shift+Insertxdotool key --clearmodifiers --window 0x2600006 alt+1 0 9 8 7 6 5 4 3Example script with mouse interaction:WINDOWID=$(xdotool selectwindow)xdotool set_window --overrideredirect 1 $WINDOWID windowunmap $WINDOWID windowmap $WINDOWIDxdotool windowsize $WINDOWID 10 100%# Set behaviorsxdotool behave $WINDOWID mouse-enter windowfocus windowsize --usehints 80 100% &xdotool behave $WINDOWID mouse-leave windowsize 4 100% &Another example: How to send F5 keystroke to the first Chrome window.More script examples please find them at GitHub or on the official page.You can install it either from apt repository like: sudo apt-get install xdotool or compile from the sources.WineIn case the X11 app is run under Wine, you can also use Winetricks. Check the source file (it's a shell script) for the guidance how to control the apps using the AutoHotkey tool.More advance method can include using winedbg debugger and attaching to the process:$ winedbgWine-dbg>info process 00000008 3        'terminal.exe'Wine-dbg>attach 80xf7709c0e __kernel_vsyscall+0xe in [vdso].so: int  $0x80then you can interact directly by using the debugger (see: man winedbg for help).