In the picture below, I'm trying to figure out why there's a need of epsilon in there and what would be the difference if we didn't put epsilon.
Why do these NFAs need epsilon transitions?
automata;finite automata;nondeterminism
They do not make a difference. In fact, we don't need epsilon transitions, in the strict sense. Even though they can not be just left out in general, they can always be replaced by several edges using the epsilon closure (cf. the powerset construction).These examples most likely stem from Thompson's construction, a simple algorithm that creates an NFA equivalent to a given regular expression. It uses epsilon transitions to link recursively constructed subautomata together. 