
  Some people say since "so" is a transitional word…


There aren't traditional words, there are traditional senses. Compare:


  Mix all of these ingredients together. Next, add the milk.
  
  I love Firefly. Next to Twin Peaks it is my favourite programme.


I've deliberately picked a different "transitional word" to hopefully make it easier to see the point if you're perhaps over-thinking so.

In the first case here, next is indeed in a transitional sense—it serves to introduce the next sentence while also stating they are in a temporal order—and the comma makes sense.

In the second case next is not doing this transitional job, but telling us something about how the topic of discussion (Firefly) relates to another thing (Twin Peaks) in the writer's opinion. We don't want to separate it from the to; what would the now separated "to Twin Peaks it is…" even mean?

So, back to so.

So's transitional sense is "used after a pause for thought to introduce a new topic, question or story." In this sense you can remove it and not really lose much meaning, though might you lose impact. If you want your sentence to mean "I am confident that …" but with so serving this introductory role, then the comma might be a good idea for that reason. (It might also be a good idea to just cut it, but that's another matter).

Another sense of so is "therefore" or "with the result that". If you want so to say that the studies you mention in the previous sentence (along, perhaps, with earlier statements) is the reason why you have this confidence, then the comma is probably a bad idea.

I imagine you want the second of these two readings, and I'd therefore recommend that you don't use a comma.

I am not sure whether to put a comma after "so", which is at the beginning of a sentence
I'm following Rasesh Mori's instructions to install Hadoop on a multinode cluster, and have gotten to the point where jps shows the various nodes are up and running. I can copy files into hdfs; I did so with

$HADOOP_HOME/bin/hdfs dfs -put ~/in /in

and then tried to run the wordcount example program on it with

$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /in /out

but I get the error

15/06/16 00:59:53 INFO mapreduce.Job: Task Id : attempt_1434414924941_0004_m_000000_0, Status : FAILED
Rename cannot overwrite non empty destination directory /home/hduser/hadoop-2.6.0/nm-local-dir/usercache/hduser/appcache/application_1434414924941_0004/filecache/10
java.io.IOException: Rename cannot overwrite non empty destination directory /home/hduser/hadoop-2.6.0/nm-local-dir/usercache/hduser/appcache/application_1434414924941_0004/filecache/10
    at org.apache.hadoop.fs.AbstractFileSystem.renameInternal(AbstractFileSystem.java:716)
    at org.apache.hadoop.fs.FilterFs.renameInternal(FilterFs.java:228)
    at org.apache.hadoop.fs.AbstractFileSystem.rename(AbstractFileSystem.java:659)
    at org.apache.hadoop.fs.FileContext.rename(FileContext.java:909)
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:364)
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)


How can I fix this?
