Can I use sups and infs on an increasing function on a closed, bounded interval and state that it is continuous? In Royden's book, we are given the following proposition:

Let $C$ be a countable subset of the open interval $(a,b)$. Then there is an increasing function $f$ on $(a,b)$ such that it is continuous only at points in $(a,b) \sim C$.

Then, he gives a problem which is similarly stated as

Let $C$ be a countable subset of the nondegenerate closed, bounded invterval $[a,b]$. Show that there is an increasing function on $[a,b]$ that is continuous only at points in $[a,b] \sim C$. 

This problem seems very trivial because I could define a function $g(x)$ where $g = f$ on $(a,b)$, and $g(a) = \inf \{ f(x) : x \in (a,b) \}$, if $C$ does not contain the endpoint $a$. The same construction would be used on $b$, but with a supremum. Then what would remain to be shown is that using this definition of $f$ for the endpoints, is continuous on at either endpoint. Is this sort of construction the right idea?
Your question does not mention Royden's construction in the open interval case; that makes it a bit confusing, since whether the construction extends easily to the closed interval case depends on what the open interval construction is.

I don't have a copy of Royden's book, so I don't know what his construction is.  But there is a standard construction here: the case in which $C$ is finite is a very easy one, so suppose $C$ is countably infinite and let $C = \{c_n\}_{n=1}^{\infty}$ be an enumeration.  Then we define 

$f(x) = \sum_{n \ \mid \ c_n \leq x} 2^{-n}$.  

This function is (weakly) increasing and has a jump discontinuity at each point of $C$.  (If you want a function which is strictly increasing, just add the function $g(x) = x$ to it.)  Moreover, this function has $\lim_{x \rightarrow a^+} f(x) = 0$ and $\lim_{x \rightarrow b^-} f(x) = 1$.  So if you extend it $[a,b]$ by taking the value $0$ at $a$ and $1$ at $b$, then yes, this construction works to answer your question.

Note also that the extension to $[a,b]$ follows the recipe that you give in your question.  However, for this to work we needed $f$ to be bounded on $(a,b)$.  There are certainly going to be functions which satisfy the property on $(a,b)$ but are not bounded: e.g. add to $f$ a function like the arctangent which has limit $-\infty$ at $a$ and $\infty$ at $b$, and then the extension procedure would not work (unless you allow your functions to take values in $[-\infty,\infty]$, which seems unlikely).
Let us start with the open interval (a,b).

(1)  suppose the number of points in C is finite.  Obviously we can make f a function which takes a jump at any point of C, but is continuous otherwise.  Then all the points in S = (a,b) ~ C have a neighborhood in which they are continuous by definition, so it is done.

(2) Next, consider C to be a countable set which is not dense in (a,b). We can use the same f as above, one which jumps at each element of C but is continuous elsewhere.  Then again, all the points in (a,b) ~ C are in open neighborhoods on which f is continuous by definition, so it is continuous at each such point.

(3) The issue is the C which is dense in (a,b). Since C is countable, it can be described as the limit a sequence of finite sets: $R_1 \subset R_2 \subset ...$.  On each $R_n$ we can define $f_n$ as we did in part 1, so that it is continuous on $S_n = (a,b) ~ R_n$ and discontinuous on $S_n$.  Let us only specify in addition that the jump chosen, which we will call q, should be the same for each $R_n$.  We claim the {$f_n$} converge and will call that limit function f.

Why the convergence.  Because each $f_{n+1}$ differs from $f_n$ only at a finite number of points, and contains all the points of discontinuity of $f_n$.

We can hope that this f is our desired function, one which is discontinuous at all elements of C and continuous on S = C ~ (a,b).  Certainly it is continuous at all elements of S, because every $f_n$ is continuous there.  The danger is that it might have become continuous on C as well. However, because the jump at r $\in R_n$ is always q, there is always a sequence of $r_n \in C$ such that $r_n \rightarrow r$ but $f(r_n)$ does not converge to f(r).

Now let us look at the closed interval [a,b].  Since we are constructing f, we can set f(a) to be anything we want, whether a $\in$ C or not.  If we specify f(a), the difficulty is at the point b.  That is, by using a constant jump of q to make f discontinuous on C, we have defined an f which $\rightarrow \infty$ at b.

To deal with this, we are going to have to drop the constant q, and replace it with a sequence {$q_n$} which $\rightarrow 0$ as $r_n$ goes to b. The resulting function f will still be discontinuous at every point of S except possibly b itself.  If b is not in S, then f is okay.  

But b could be in S.  I do not have an answer to this.  Perhaps someone else can answer.