Dividing by zero and imaginary numbers If we decide to call the square root of -1 its own number, then why not do the same for dividing by zero? Just make an entirely new type of number?
If you introduce a division by zero, you can only retain a subset of the usual rules of arithmetic or you run into contradictions. If $0\times x=1$ then associativity would give $$2\times 0 \times x = 0 \times x = 3 \times 0 \times x,\,2 \times 1 = 3\times 1.$$
Let's put this in a more helpful way: rather than saying that $\sqrt{-1}=i$, say that $i$ is a (particular consistent choice of) thing so that $i^2=-1$. Phrased this way, your question does have a sensible interpretation: we suppose there is a symbol $\varepsilon$ so that $\varepsilon^2=0$. We then form the ring of things of the form $a+b\varepsilon$, where $a$ and $b$ are reals; these are called dual numbers.

It is easy to show that the dual numbers are not a field: it has elements that multiply to zero (indeed, it has lots of them), so they cannot have inverses ("$1/\varepsilon$" would be the element that you multiply $\varepsilon$ by to get $1$, but there is no such element). For a fuller discussion of which dual numbers have inverses, have a look at the appropriate section of the Wikipedia article.

A concrete way to represent $i$ is the matrix $ J = \begin{pmatrix} 0 &amp; -1 \\ 1 &amp; 0 \end{pmatrix}$, while $1$ is represented by the identity matrix $I=\begin{pmatrix} 1 &amp; 0 \\ 0 &amp; 1 \end{pmatrix}$; one can show that the collection of matrices of the form $aI+bJ$ with $a,b \in \mathbb{R}$ form a field. Similarly, a concrete way to represent $\varepsilon$ is $\begin{pmatrix} 0 &amp; 1 \\ 0 &amp; 0 \end{pmatrix}$.

What do functions of dual numbers look like? Quite boring, actually. Let $p(x)$ be a real polynomial. Then by using the binomial theorem, we find 
$$p(x+y\varepsilon)=p(x) + \varepsilon y p'(x).$$
This extends to any function with a Taylor expansion. This simplicity does actually have applications in computer algebra, as one way of implementing automatic differentiation.