1. A method of controlling backing of a trailer coupled to a tow vehicle for a trailer assist system for assisting with backing up the trailer connected to the tow vehicle, said method comprising: acquiring with a camera video images of a user proximate to the trailer and tow vehicle, wherein the camera is located on one of the trailer and tow vehicle; processing the video images to determine a gesture command by the user; and controlling the tow vehicle to back the trailer based on the determined gesture command, wherein the gesture command includes a command for changing trajectory of the trailer.