If $\mu$ is a finite measure on $E$, then $\mu(E)=\sum_{x\in E}\mu(\{x\})$ Remember that if $I$ is a set, then $(a_i)_{i\in I}\subseteq[0,\infty)$ is summable if and only if $$a:=\sup\left\{\sum_{j\in J}a_j:J\subseteq I\text{ and }|J|\in\mathbb N\right\}<\infty\tag1$$ and if $(1)$ holds, then $\sum_{i\in I}a_i=a$. Moreover, if $(b_i)_{i\in I}\subseteq[0,\infty)$ is Cauchy (e.g. summable), then $\left\{i\in I:b_i>0\right\}$ is countable.
Now let $(E,\mathcal E,\mu)$ be a finite measure space with $\{x\}\in\mathcal E$ for all $x\in E$. Then, clearly, $$B=\biguplus_{x\in B}\{x\}\in\mathcal E\tag2$$ and $$\mu(B)=\sum_{x\in B}\mu(\{x\})\tag3$$ for all countable $B\subseteq E$.
Using $(3)$, we easily see that $$c:=\sup\left\{\sum_{x\in B}\mu(\{x\}):B\subseteq E\text{ and }|B|\in\mathbb N\right\}\le\mu(E)<\infty\tag4$$ and hence $\left(\mu(\{x\})\right)_{x\in E}$ is summable with $$\sum_{x\in E}\mu(\{x\})=c.\tag5$$

Are we able to show that $c=\mu(E)$?

EDIT: I don't know whether it's useful or not, but by my remark above $\left\{x\in E:\mu(x)>0\right\}$ is countable. Moreover, it's easy to see that $$\left|\left\{x\in E:\mu(\{x\})>\frac1n\right\}\right|<n\mu(E)\;\;\;\text{for all }n\in\mathbb N.\tag2$$

This is trivially false. Let $\mu (A)=\lambda ((0,1) \cap A)$ where $E=\mathbb R$ and $\lambda$ is the Lebesgue measure.  Then $c=0$ and $\mu (E)=1$.
