How to construct Dataframe from a Excel (xls,xlsx) file in Scala Spark?

Question: I have a large Excel(xlsx and xls) file with multiple sheet and I need convert it to RDD or Dataframe so that it can be joined to other dataframe later. I was thinking of using Apache POI and save it as a CSV and then read csv in dataframe. But if there is any libraries or API that can help in this Process would be easy. Any help is highly appreciated.
Comment: Check this answer for newbies with steps
https://stackoverflow.com/a/47721326/2112382
Comment: Actually, I've to store a spark dataframe in a excel file format with few column as a read only nature? Can you guide me regarding the same?
Answer: The solution to your problem is to use Spark Excel dependency in your project.
Spark Excel has flexible options to play with.
I have tested the following code to read from excel and convert it to dataframe and it just works perfect
def readExcel(file: String): DataFrame = sqlContext.read
    .format("com.crealytics.spark.excel")
    .option("location", file)
    .option("useHeader", "true")
    .option("treatEmptyValuesAsNulls", "true")
    .option("inferSchema", "true")
    .option("addColorColumns", "False")
    .load()

val data = readExcel("path to your excel file")

data.show(false)

you can give sheetname as option if your excel sheet has multiple sheets
.option("sheetName", "Sheet2")

I hope its helpful
Comment: `'sheetName'` doesn't work anymore. You have to use `'dataAddress'` - https://github.com/crealytics/spark-excel/issues/118
Comment: @Regressor try not using location and using path in load as mentioned in https://github.com/crealytics/spark-excel
Comment: So it processes all tabs if sheetName not specified?
Comment: I used `spark.read.format("com.crealytics.spark.excel").option("location","/home/mylocation/myfile.xlsx").load()` but got `java.lang.IllegalArgumentException: Parameter "path" is missing in options.`
Answer: Here are read and write examples to read from and write into excel with full set of options...
Source spark-excel from crealytics
Scala API
Spark 2.0+:
Create a DataFrame from an Excel file
    import org.apache.spark.sql._

val spark: SparkSession = ???
val df = spark.read
         .format("com.crealytics.spark.excel")
        .option("sheetName", "Daily") // Required
        .option("useHeader", "true") // Required
        .option("treatEmptyValuesAsNulls", "false") // Optional, default: true
        .option("inferSchema", "false") // Optional, default: false
        .option("addColorColumns", "true") // Optional, default: false
        .option("startColumn", 0) // Optional, default: 0
        .option("endColumn", 99) // Optional, default: Int.MaxValue
        .option("timestampFormat", "MM-dd-yyyy HH:mm:ss") // Optional, default: yyyy-mm-dd hh:mm:ss[.fffffffff]
        .option("maxRowsInMemory", 20) // Optional, default None. If set, uses a streaming reader which can help with big files
        .option("excerptSize", 10) // Optional, default: 10. If set and if schema inferred, number of rows to infer schema from
        .schema(myCustomSchema) // Optional, default: Either inferred schema, or all columns are Strings
        .load("Worktime.xlsx")

Write a DataFrame to an Excel file
    df.write
      .format("com.crealytics.spark.excel")
      .option("sheetName", "Daily")
      .option("useHeader", "true")
      .option("dateFormat", "yy-mmm-d") // Optional, default: yy-m-d h:mm
      .option("timestampFormat", "mm-dd-yyyy hh:mm:ss") // Optional, default: yyyy-mm-dd hh:mm:ss.000
      .mode("overwrite")
      .save("Worktime2.xlsx")

Note: Instead of sheet1 or  sheet2 you can use their names as well..
  in this example given above Daily is sheet name.

If you want to use it from spark shell...

This package can be added to Spark using the --packages command line option. For example, to include it when starting the spark shell:
    $SPARK_HOME/bin/spark-shell --packages com.crealytics:spark-excel_2.11:0.13.1

Dependencies needs to be added (in case of maven etc...):

groupId: com.crealytics
artifactId: spark-excel_2.11
version: 0.13.1

Further reading : See my article (How to do Simple reporting with Excel sheets using Apache Spark, Scala ?) of how to write in to excel file after an aggregations in to many excel sheets

Tip : This is very useful approach  particularly for writing
  maven test cases where you can place excel sheets with sample data in excel
  src/main/resources folder and you can access them in your unit test cases(scala/java), which creates DataFrame[s] out of excel sheet...

Another option you could consider is
spark-hadoopoffice-ds

A Spark datasource for the HadoopOffice library. This Spark datasource
  assumes at least Spark 2.0.1. However, the HadoopOffice library can
  also be used directly from Spark 1.x. Currently this datasource
  supports the following formats of the HadoopOffice library:
Excel Datasource format: org.zuinnote.spark.office.Excel Loading and
  Saving of old Excel (.xls) and new Excel (.xlsx) This datasource is
  available on Spark-packages.org and on Maven Central.
Comment: I used `spark.read.format("com.crealytics.spark.excel").option("location","/home/mylocation/myfile.xlsx").load()` but got `java.lang.IllegalArgumentException: Parameter "path" is missing in options.`
Comment: Actually, I've to store a spark dataframe in a excel file format with few column as a read only nature? Can you guide me regarding the same?
Answer: Alternatively, you can use the HadoopOffice library (https://github.com/ZuInnoTe/hadoopoffice/wiki), which supports also encrypted Excel documents and linked workbooks, amongst other features. Of course Spark is also supported.
Comment: Hello All,Can we use the above to write data to multiple tabs in an excel sheet?.
Comment: I assume you mean multiple sheets in an Excel workbook. Yes, it can write to multiple sheets. Basically you define a SpreadSheetCellDAO which specifies formattedValue, Comment, Formula, Address and Sheet. However, to support you more I would need to know more about your use case. Feel free to provide the information as Github issue: https://github.com/ZuInnoTe/hadoopoffice/issues
Comment: I have a column that has the values with double quotes eg: "xxxxx,yyy,zzz". Because of this, the value is not considered as a single column, if I see the dataframe, instead of one column, it is showing as 3 columns
Comment: That is strange. There is no logic to split that column based on commas or double quotes. Can you please check with the Apache POI people: https://poi.apache.org/help/index.html ? Can you also please verify that it is indeed just one column and provide an example file?
Answer: I have used com.crealytics.spark.excel-0.11 version jar and created in spark-Java, it would be the same in scala too, just need to change javaSparkContext to SparkContext.
tempTable = new SQLContext(javaSparkContxt).read()
    .format("com.crealytics.spark.excel")
    .option("sheetName", "sheet1")
    .option("useHeader", "false") // Required
    .option("treatEmptyValuesAsNulls","false") // Optional, default: true
    .option("inferSchema", "false") //Optional, default: false
    .option("addColorColumns", "false") //Required
    .option("timestampFormat", "MM-dd-yyyy HH:mm:ss") // Optional, default: yyyy-mm-dd hh:mm:ss[.fffffffff] .schema(schema)
    .schema(schema)
    .load("hdfs://localhost:8020/user/tester/my.xlsx");

Comment: Actually, I've to store a spark dataframe in a excel file format with few column as a read only nature? Can you guide me regarding the same?
Answer: Hope this should help.
val df_excel= spark.read.
                   format("com.crealytics.spark.excel").
                   option("useHeader", "true").
                   option("treatEmptyValuesAsNulls", "false").
                   option("inferSchema", "false").
                   option("addColorColumns", "false").load(file_path)

display(df_excel)

Comment: Actually, I've to store a spark dataframe in a excel file format with few column as a read only nature? Can you guide me regarding the same?


Custom Ubuntu 18.04 LTS Desktop ISO won't boot in UEFI

Question: So I've been banging my head against the wall trying to figure out what I'm doing wrong.
I have 2 different machines I'm trying to get this to work on. A Dell Precision 5820 & a Dell Precision 5810. I take the standard ISO of Ubuntu 18.04.1 LTS Desktop (from here: https://www.ubuntu.com/download/desktop), burn it to a DVD using Brasero (from an Ubuntu 16.04 machine), and use that DVD to install Ubuntu on both machines just fine (UEFI). Two things to note. Both machines are using a RAID controller and the 5820 has an NVMe M.2 drive while the 5810 has an SSD.
Now what I want to do is customize the ISO to make it an unattended install with the latest package updates. I was able to successfully do this with Ubuntu 16.04 (BIOS/Legacy boot) using a preseed file and following the instructions here (Live CD Customization), here (preseeding), and here.
So now I'm taking the 18.04 ISO, doing the same thing I did with 16.04 and following instructions here to make it UEFI bootable. This seems to work when I boot to a VM in VirtualBox (and enabling EFI), but not on the Dell computers I have. The same computers were able to boot the default Ubuntu 18.04 image, but not my custom one. I'm sure I'm doing something wrong.
When I check the file system of both ISO's with fdisk like this:
$ sudo fdisk -l ubuntu-18.04.1-desktop-amd64.iso
Disk ubuntu-18.04.1-desktop-amd64.iso: 1.8 GiB, 1953349632 bytes, 3815136 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x663eb4c4

Device                            Boot   Start     End Sectors  Size Id Type
ubuntu-18.04.1-desktop-amd64.iso1 *          0 3815135 3815136  1.8G  0 Empty
ubuntu-18.04.1-desktop-amd64.iso2      3737268 3741939    4672  2.3M ef EFI (FAT-12/16/32)

They both look identical, with the exception of the size of the first partition. The custom ISO is a little bigger since I installed some updates.
Answer: Ok, I followed the instructions here & was successful:
https://linuxconfig.org/legacy-bios-uefi-and-secureboot-ready-ubuntu-live-image-customization
I think what got it working for me was creating the isohdpfx.bin file from the original ISO:
$ sudo dd if=ubuntu-16.04-desktop-amd64.iso bs=512 count=1 of=custom-iso/isolinux/isohdpfx.bin

Whereas before I had installed the isolinux package to get it from /usr/lib/ISOLINUX/isohdpfx.bin


Can sleep become addictive?

Question: Can a person become dependent on sleeping (more than they otherwise physically need) in a way that fits the definition of addiction, in the same way some psychology professionals may describe a person as being addicted to other standard needs, such as food, work, exercise?
If so, is there a way to determine whether the "need" for more sleep is due to actual physical needs or due to an addictive response?
Have there been any studies on this topic?
Comment: There is something called Hypersomnia, which is the opposite of insomnia. I don't think it is an addiction as much as it is a disorder. There is also Narcolepsy, where people tend to sleep a lot, but only because they are not sleeping very well in the first place
Comment: There is no "sleep addiction". Excessive sleepiness is a disease called hypersomnia. Sleeping to avoid "facing the day" is a symptom of depression.
Comment: Thank you SOO much Marie for sharing your struggles. I have the same problem as you have described. I'm 26 and I do all the same things. I don't always have to be asleep as long as I'm laying down and resting. It's not the sleep itself that I get pleasure from but I feel euphoric when I'm lying down and drifting to sleep.
Comment: Sleep is as necessary as breathing. Would you call absolute necessity an addiction? Otherwise I agree to @what@.
Comment: Don't completely overlook the correlation to vivid dreaming, many a sailor lay fortnites under their covers to once again glimpse the sirens their minds were once able to picture for them.
Answer: Not aware of any studies on this topic.
I think this is a tricky question because of the nature of sleeping.
Hypersomnia does not fit in with physiological nor psychological dependence because of the following (from the proposed DSM-V revision, but similar enough to the DSM-IV-TR) symptom:
"The sleep periods are non-restorative (unrefreshing) or so prolonged in length that this causes clinically significant distress or impairment in social, occupational, or other important areas of functioning."
Now, sleep would not fit in with the class of substance-related disorders. I cannot see one having impaired control over or compulsive use of sleep. Can one really derive that much pleasure, and thus, a reward response from sleep? Would one have a tendency to keep increasing the dose, to chase "a high"? Abrupt cessation of sleep after excessive "use" may trigger negative feelings but if one were to, say, give the sleep-craving individual stimulant medication, I doubt serious withdrawal effects will take place.
The only proposed behavioral addiction for the DSM-V (at least I think at this point) is Gambling Disorder, or pathological gambling. The proposed criteria are as follows.
A.    Persistent and recurrent maladaptive gambling behavior as indicated by five (or more) of the following:

is preoccupied with gambling (e.g., preoccupied with reliving past gambling experiences, handicapping or planning the next venture, or thinking of ways to get money with which to gamble
needs to gamble with increasing amounts of money in order to achieve the desired excitement
has repeated unsuccessful efforts to control, cut back, or stop gambling
is restless or irritable when attempting to cut down or stop gambling
gambles as a way of escaping from problems or of relieving a dysphoric mood (e.g., feelings of helplessness, guilt, anxiety, depression)
[Not relevant.]
lies to family members, therapist, or others to conceal the extent of involvement with gambling
has jeopardized or lost a significant relationship, job, or educational or career opportunity because of gambling
[Not relevant.]

Can a psychological dependence on sleep fit in with the above criteria if one were to mold it to pertain to "pathological" sleeping?
I could not think of analogous "Sleep Addiction" criteria to match that of Gambling Addiction's A-6/A-9 symptom points, but from the above, it appears that a psychological dependence /can/ apply to a dependence on sleep(ing). From this loose argument alone.

Preoccupied with sleep. [Sure.]
Needs get an increasing amount of hours of sleep to achieve the same response. [Eh. I was thinking "resorts to increasing doses of sedatives to achieve longer periods of sleep" but felt that ventured into substance abuse territory.]
Has repeated unsuccessful efforts to control, cut back, or stop (ha) [excessive] sleep. [Sure, why not.]
Is restless or irritable when attempting to cut down on sleep. [Can see this happening.]
Sleeps as a way of avoiding problems or of relieving dysphoric mood. [Most definitely, this symptom reminds me of some subtypes of Major Depressive Disorder. But I digress.]
Lies to family members, others, to conceal the extent of excessive sleep. [Why not.]
Has jeopardized or lost a significant relationship, job, or educational or career opportunity because of excessive sleep. [Can easily see this.]

If I can interject my own opinion on the matter, I just cannot see anyone developing a physical need for sleep. Our basic biological or evolutionary make-up does not allow for that, I feel. But I know little about, say, narcolepsy - but to my knowledge they find their excessive sleepiness unwanted and detrimental. Nor can I see potential for one to become psychologically dependent or addicted to sleep. I just can't see someone deriving pleasure from more and more sleep.
Comment: I like the comparison to gambling, but I would suggest moving the sleep based example to the front of the question (for us TL;DR folks) and then explaining it and the gambling example it was based on after-the-fold. Also, links would be nice!
Answer: "Nor can I see potential for one to become psychologically dependent or addicted to sleep."
I believe someone can become "psychologically dependent on sleep".
I am 47 and have used sleep for 40 years to escape from life.   I typically sleep 4-6 hours too much each day.   I don't really physically need this sleep, since I then am often awake the next night for oversleeping that day, or I get a very strange tingling feeling in my body when I am forcing myself to sleep more and my body simply can't sleep any more.  Also, I wake up after a normal 8hrs, feel like I can't face the day and force myself to go back to sleep with more breaks of waking up and forcing myself back to sleep for the next 4-5 hours.
My sleep gets in the way of living a full day and being productive.  When I finish my day, I am frustrated that I lost so many hours to sleep.
I have tried everything for the last 25 years to address this.   I have tried 7 years of psychotherapy, personal will power, all kinds of sleep and activity schedules, full physical examinations, talking with a sleep specialist, etc.
I always seem to eventually drift back into using sleep as a way of getting out of life.
I am generally an intelligent, capable person, have a college degree, reached a decent level professionally, travelled around the world, etc.    I look like a normal person but the sleep problem makes functioning difficult.
I have had some childhood trauma, depression, anxiety which I have worked with extensively with psychotherapy.
Sleep feels like an addiction to me because
.  I crave it several times a day and am looking forward to how I can sneak it in.
.  I don't seem to be able to control it with will power for very long.
.  If I have a successful period, I seem to always eventually slip back into it.
.  I only have short periods when this isn't a problem.
.  When I am under stress it is at it's worse.
.  If I have any free or unstructured time, I can't control how much I sleep excessively.
.  When my time is heavily scheduled, I really struggle with keeping a full schedule and crave the time off when I can sleep for hours.
.  If I know I'll have a few hours in between activities free, I will find ways to sneak in some sleep.
.  I am embarrassed about this, don't tell the people around me the extent of the problems and devise ways to sneak in sleep without people knowing.
.  If I am anxious, I use sleep as a way to reboot my system, calm myself down or get another perspective.
.  After I have chosen excessive sleep over being up, I struggle with self-recriminations about how much of my life I have wasted, how I didn't accomplish the things I wanted, why it was not good for me, how most of my day was gone, etc.
The upside is my addition only hurts me.
For me, sleep addiction (and perhaps food addiction for others) are different than gambling, alcohol, smoking, drugs.   Sleep is something you must do each day to live.   I have no problem with total abstinence of things that aren't good for me (like drugs, etc).   However, I really struggle day after day with getting up out of sleep.
I have researched this and I have not found any studies on "sleep addiction".    However, in medical sleep science journals and among a few psychiatric journals there are recent studies on "delayed sleep phase syndrome" and "non- 24 hour sleep phase syndrome", whereby people who appear to sleep excessively simply have a malfunction of how melantonin is secreted in the system (perhaps due to the recent advent of the electric lightbulb) and do not sleep when most humans sleep.  Studies are showing that a much larger percentage of the population unknowingly struggle with this disorder in isolation all their lives.  This is turning out to be the be the biggest sleep disorder affecting the most number of people, even above insomnia and apnea.   This is very little understood and has only been discovered and documented in science journals since 1981.    DSPS is a recognized disability in the government disability system.
It seems that there is a huge gulf between the studies and expertise of medical sleep science and that of psychology/psychiatric studies on the sleep difficulties and the use of excessive sleep as a psychological coping mechanism.
Comment: Yes, I have researched it.   No there aren't any studies.   Too bad I am downgraded, doesn't someone have to be the first to start somewhere?   This has been considered an "invisible disability" (see invisibledisabilities.org) which people don't want to admit to.  Your posting guidelines state personal experiences can be submitted.   Too bad I can't get help or have an intelligent forum to discuss this.
Comment: Please don't be offended Marie. I downvoted you because  it wasn't clear that there were no studies that you could reference, and because [good answers on this site reference studies, not only personal experience](http://meta.cogsci.stackexchange.com/questions/237). Now, that having been said, I will happily remove my downvote if you can edit your question... perhaps just add a note saying "I have researched this but couldn't find any relevant studies"
Comment: Also, please be aware of a few things: **1.** This site isn't a "forum" in the traditional sense. Our goal is to provide *objective* answers to *objective* questions. So answers based on personal experience may tend to be downvoted. We're better suited to Q&A than discussion. **2.** our community is still trying to decide as a whole who we are and what the rules are here. If you feel we're going in the wrong direction, please [start a new discussion on Meta](http://meta.cogsci.stackexchange.com/). I want to be sure you know I am not trying to be rude, just trying to help!
Comment: Just because there aren't any professional studies done on the subject of sleep addiction does not mean that it isn't a science. Galileo was ridiculed and jailed for his "science." And the first doctors to suggest airborne germs were invisible to the human eye yet present on a microscopic scale were also derided. This forum is functionally inept if one can't explore what is thought to be the scientific fringe. A question doesn't always have to be answered with an answer. Marie I up-voted your response because it was detailed, in-depth and scientifically apt, even if it failed to include actual
Comment: I downvoted this answer... which I hated to do, because it was really interesting! However, it's lacking any scientific references. DO you have any you can provide? have you researched your condition yourself and can you provide links to studies? I think you have some great content here and it just needs a little bit more. Thanks!
Comment: This sounds like a plain old depression to me.
Answer: According to Dr Piotr Wozniak, no.
Oversleeping
To summarize what Dr Wozniak says, people need sleep to optimize memory. While there is an evolutionary benefit to overeating (conserving fat for the future), there is none to oversleeping.
Sleepiness is a combination of

a circadian component (the hours you usually sleep)
a homeostatic component (e.g. staying awake too long, learning too much).

What causes oversleeping is people who often sleep while they are tired before their bedtime, and the circadian component launches hormones that keep you asleep.
For example, if your usual bedtime is about 10 PM, your usual wake time is around 4-6 AM, which is when the circadian sleepiness wears off. If you are extremely tired and fall asleep at 6 PM, your circadian sleepiness hormones kick in at about 10 PM, leaving you sleepy until around 4-6 AM. You will fall asleep for about 12 hours and feel like you've slept for only 6, which is what many people interpret as oversleeping.
If you happen to sleep at 4 PM that day and sleep off the homeostatic component, you'll lack enough 'sleepiness' to fall asleep when the circadian part kicks in, and you'll get insomnia.
Thus, early sleepers will either get insomnia or oversleep, depending on how different from their usual circadian rhythm it is. Eventually, people's circadian rhythm will adjust to the new cycle, but people who keep adjusting their cycles and travel often (e.g. 2 AM one week, 6 PM the next) will often face oversleeping issues.
Can sleep become addictive?
I'm no expert on how addiction works, so someone should correct me on this if mistaken.
Addiction is a habitual psychological or physiological dependence on a practice that is beyond voluntary control. Quite often, it occurs when you have a reward at the end of a cue.
E.g. someone starts drinking or smoking when they get stressed out. As they repeat this, they gain a psychological addiction to drinking or smoking every time they are stressed. The reward of nicotine/alcohol triggers and enforces the habit loop. After enough of the substance in the system, they become physiologically addicted to alcohol or nicotine.
Or someone begins gambling every time they are bored and have money in their pocket. The reward of getting money enforces a habit loop, creating psychological addiction. The cue of being bored and having money triggers the addiction, the reward of getting money enforces it.
Being unable to retrieve these rewards creates craving and thus, psychological addiction.
However, with sleep is difficult to do as a reaction to anything. One can easily overeat as a response to stress, but one cannot easily sleep as a response to stress.
There's little euphoria associated to sleeping; when you wake up, you'd normally feel a little drowsy instead of 'satisfied'. While someone lacking sleep can have a craving for sleep, it's difficult to form a habit loop of sleeping as a reaction to anything.
So, I'd argue that while very rare circumstances would make one addicted to sleep, it'd have to be a deliberately developed habit. The most likely case would be a craving to sleep whenever one opens a textbook. A 'craving' to sleep at night is quite normal, much like the 'craving' to eat several times a day caused by low blood sugar levels.

Can sleep become addictive?


Why do I burp sometimes?

Question: Sometimes I burp after eating food in Minecraft (Java edition 1.13.2 on Linux in a singleplayer world). First, the food will be consumed as normal, restoring the expected amount of hunger. Then I hear a "burp" sound and additional hunger is restored. Since this usually happens unexpectedly, I don't know whether additional food has been consumed or not. What causes the burp? What are its effects? And can it be done intentionally?
Comment: I don't have enough reputation to see votes on this site, but +8 reputation implies 2 up and 1 down. Would the downvoter care to provide some feedback?
Answer: The second food consumption is caused by the bug MC-849. Under certain circumstances, it's possible to accidentally eat a second food item, without seeing it, slowing down your movement or a chewing sound.
The burping sound still plays, since it's caused by the server telling the client that eating is finished. It also plays after the first food item, but it's hard to hear due to the eating sound.
Comment: This does seem to be the correct explanation for the second food disappearing. But it seems from my experience that the burp sound only* happens when food is eaten twice, and not during regular eating. Are the two really unrelated?
Comment: It also plays after the first one, but it's hard to hear. I put that into my answer.
Answer: From the list of sound effects the sound is purely random.

random.burp

The extra food can sometimes be attributed to latency when consuming food, and may sometimes try to "catch up", leading to consuming extra items of food.
Comment: Latency shouldn't be an issue as I'm not not playing on a multiplayer server.
Answer: It happens, when you eat some food that fills your hunger bar entirely.
The hunger bar you see on screen is just 1/2 of the bar. Actually your maximum hunger level is 20.
When it reaches 20, then the burp sound plays but not every time.

Why do I burp sometimes?


Quantile regression vs probability density estimation

Question: If you want to predict a range for a regression problem using a deep network, you can do quantile regression and go with (for example) 5% and 95% quantiles. The other option is predicting a probability density function (using something like tensorflow probability) and use its cdf to get a range. What is the advantage/disadvantage of one versus the other? Are there circumstances that there is a clear choice?
Comment: Isn't the TensorFlow probability the predicted probability of class membership in a classification problem?
Comment: It might be useful for that as well (I don't know tbh) but the context of my question is using tensorflow probability layers (something like this https://www.tensorflow.org/probability/api_docs/python/tfp/layers/DistributionLambda) and setting the output as a distribution.
Answer: There are several ways to issue probabilistic forecasts. Using a predictive density function is the parametric approach. For example, if you assume errors are Gaussian, you only need to estimate their mean and variance in order to calculate prediction intervals. This requires less computation, but you have to make assumptions about the underlying distribution, so it might be inflexible.
Alternatively, you can estimate separate quantiles via quantile regression. If you estimate a large set, say ${0.01, 0.02, .., 0.99}$ you can approximate the probability distribution function. This is considered a non-parametric approach and it's more flexible, since you avoid making any assumptions about the distribution. However, it might require a different model to be trained for each quantile, which increases the computational burden.
There are also other ways to solve this problem, for example bootstrapping is a popular approach. I believe it depends on the problem at hand; if a known distribution fits your problem well, I would follow the parametric approach. In general, I think the non-parametric is more flexible and popular in practical applications (at least in my domain). I have also seen applications where quantile regression is used to estimate the range from $5\%-95\%$ and exponential distributions are used for modelling the tails.


How to ground a fixture on a conduit?

Question: I am replacing a line-voltage path light that screws onto the top of a 1/2" metal conduit. I am pulling new wire through the conduit. Since the conduit is metal, I expect to use it as a ground path. However, I don't see how I make a connection from a ground wire in the light fixture to the conduit. I am sort of imagining some sort of clip that would surround the wires and make contact with the conduit to which the fixture wire could be attached, but I've never seen same. Or, for a metal fixture, may I assume that the ground wire need not be connected if the fixture is threaded onto metal?
Answer: First, check all the directions that came with the fixture to see if they have special instructions on how to install it. You'll have to verify that the conduit is a continuous run eventually entering the main panel and insure it's grounded. Then, screwing the metal fixture to the conduit will ground it. Check to see how the fixture ground wire is connected to the fixture, you can probably remove the wire if it's just screwed into the fixture. Depending on the fixture, the instructions might call for you to run a ground wire with your other wires.

How to ground a fixture on a conduit?


Windows 10 NOT asking for password for Ubuntu samba shares

Question: I have recently made a clean install of 16.04.3LTS on a new laptop.  I am trying to share files from my ubuntu laptop on my home network.  I want only specific users to be able to access the samba shares, and require password authentication.  Currently, either Windows 10 has full access with no password required????  What is wrong with my samba config?
#======================= Global Settings =======================

[global]

## Browsing/Identification ###

# Change this to the workgroup/NT-domain name your Samba server will part of
    workgroup = home

# server string is the equivalent of the NT Description field
    server string = %h server (Samba, Ubuntu)

# Windows Internet Name Serving Support Section:
# WINS Support - Tells the NMBD component of Samba to enable its WINS Server
#   wins support = no

# WINS Server - Tells the NMBD components of Samba to be a WINS Client
# Note: Samba can be either a WINS Server, or a WINS Client, but NOT both
;   wins server = w.x.y.z

# This will prevent nmbd to search for NetBIOS names through DNS.
    dns proxy = no

#### Networking ####

# The specific set of interfaces / networks to bind to
# This can be either the interface name or an IP address/netmask;
# interface names are normally preferred
;   interfaces = 127.0.0.0/8 eth0

# Only bind to the named interfaces and/or networks; you must use the
# 'interfaces' option above to use this.
# It is recommended that you enable this feature if your Samba machine is
# not protected by a firewall or is a firewall itself.  However, this
# option cannot handle dynamic or non-broadcast interfaces correctly.
;   bind interfaces only = yes

####### Authentication #######

# Server role. Defines in which mode Samba will operate. Possible
# values are "standalone server", "member server", "classic primary
# domain controller", "classic backup domain controller", "active
# directory domain controller".
#
# Most people will want "standalone sever" or "member server".
# Running as "active directory domain controller" will require first
# running "samba-tool domain provision" to wipe databases and create a
# new domain.
    server role = standalone server

# If you are using encrypted passwords, Samba will need to know what
# password database type you are using.
;   passdb backend = tdbsam

    obey pam restrictions = yes

# This boolean parameter controls whether Samba attempts to sync the Unix
# password with the SMB password when the encrypted SMB password in the
# passdb is changed.
    unix password sync = yes

# For Unix password sync to work on a Debian GNU/Linux system, the following
# parameters must be set (thanks to Ian Kahan <<firstname.lastname@example.com> for
# sending the correct chat script for the passwd program in Debian Sarge).
    passwd program = /usr/bin/passwd %u
    passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .

# This boolean controls whether PAM will be used for password changes
# when requested by an SMB client instead of the program listed in
# 'passwd program'. The default is 'no'.
    pam password change = yes

# This option controls how unsuccessful authentication attempts are mapped
# to anonymous connections
    map to guest = bad user
############ Misc ############

# Using the following line enables you to customise your configuration
# on a per machine basis. The %m gets replaced with the netbios name
# of the machine that is connecting
;   include = /home/samba/etc/smb.conf.%m

# Some defaults for winbind (make sure you're not using the ranges
# for something else.)
;   idmap uid = 10000-20000
;   idmap gid = 10000-20000
;   template shell = /bin/bash

# Setup usershare options to enable non-root users to share folders
# with the net usershare command.

# Maximum number of usershare. 0 (default) means that usershare is disabled.
;   usershare max shares = 100

# Allow users who've been granted usershare privileges to create
# public shares, not just authenticated ones
    usershare allow guests = yes
    username map = /etc/samba/smbusers
    security = user
;   encrypt passwords = yes
;   guest ok = no
;   guest account = nobody

#======================= Share Definitions =======================

# Un-comment the following (and tweak the other settings below to suit)
# to enable the default home directory shares. This will share each
# user's home directory as \\server\username
;[homes]
;   comment = Home Directories
;   browseable = no

# By default, the home directories are exported read-only. Change the
# next parameter to 'no' if you want to be able to write to them.
;   read only = yes

# File creation mask is set to 0700 for security reasons. If you want to
# create files with group=rw permissions, set next parameter to 0775.
;   create mask = 0700

# Directory creation mask is set to 0700 for security reasons. If you want to
# create dirs. with group=rw permissions, set next parameter to 0775.
;   directory mask = 0700

# By default, \\server\username shares can be connected to by anyone
# with access to the samba server.
# Un-comment the following parameter to make sure that only "username"
# can connect to \\server\username
# This might need tweaking when using external authentication schemes
;   valid users = %S

# Un-comment the following and create the netlogon directory for Domain Logons
# (you need to configure Samba to act as a domain controller too.)
;[netlogon]
;   comment = Network Logon Service
;   path = /home/samba/netlogon
;   guest ok = yes
;   read only = yes

# Un-comment the following and create the profiles directory to store
# users profiles (see the "logon path" option above)
# (you need to configure Samba to act as a domain controller too.)
# The path below should be writable by all users so that their
# profile directory may be created the first time they log on
;[profiles]
;   comment = Users profiles
;   path = /home/samba/profiles
;   guest ok = no
;   browseable = no
;   create mask = 0600
;   directory mask = 0700

[printers]
    comment = All Printers
    browseable = no
    path = /var/spool/samba
    printable = yes
;   guest ok = no
;   read only = yes
    create mask = 0700

# Windows clients look for this share name as a source of downloadable
# printer drivers
[print$]
    comment = Printer Drivers
    path = /var/lib/samba/printers
;   browseable = yes
;   read only = yes
;   guest ok = no
# Uncomment to allow remote administration of Windows print drivers.
# You may need to replace 'lpadmin' with the name of the group your
# admin users are members of.
# Please note that you also need to set appropriate Unix permissions
# to the drivers directory for these users to have write rights in it
;   write list = root, @lpadmin

[Music]
    path = /home/chad/Music
    writeable = yes
;   browseable = yes
;   valid users = chad
    guest ok = no
    security = user

[Downloads]
    path = /home/chad/Downloads
    writeable = yes
;   browseable = yes
    valid users = chad

[Videos]
    path = /home/chad/Videos
    writeable = yes
    valid users = chad

Before someone says its must be a problem with Windows 10 credentials...I have already tried removing all net credentials (actually there were none), and still have the same issue.
Comment: yes created only password for one user "chad"
Comment: Maybe? it is a Windoesn't issue?  I just logged out of my account on Windows, then logged in another user. I was then asked for a user and password.  I logged out of that user and logged in to MY user, am still able to access shares, without having to enter a  user/password.
Answer: This usually happens when the samba user matches the currently logged on user. So if you create a samba share for CHAD and on the remote PC, your username is also CHAD it wont ask for a password and let you in. (I believe the passwords have to match as well, it turns into like an auto login, so to speak).
Now if you want to make the share require a password no matter what, what you could try is add the line security = user to your smb.conf and it should require a password now.
This is all assuming both usernames on the client and server are the same.
Comment: "security = user" in Global?


How can I maximize the number of cantrips for an Eldritch Knight?

Question: An eldritch knight only gets 2 cantrips to start.  This is very limiting. Is there a way to get more cantrips to increase the eldritch knight's utility and flexibility?  What is the maximum number of cantrips possible for a character that is a level 4 eldritch knight?
Comment: @Christopher We have a strict policy of not guessing the system, no matter how obvious it may seem. You can discuss that policy [here](https://rpg.meta.stackexchange.com/q/9805/3548).
Answer: Assuming a single class Eldritch Knight (EK) for base of 2 cantrips.
Start as a High Elf for +2 dex, +1 int, and any cantrip from the wizards list. This is more optimal for an EK than starting as a Tiefling.
At level 4 select the Magic Initiate feat to gain 2 cantrips and 1 first level spell.
For a total of 5 cantrips at level 4 plus a bonus spell from magic initiate.
Answer: Multi-classing, feats, and racial features will get you more cantrips.
Being a tiefling grants the cantrip thaumaturgy.  On top of that, taking a level in other classes that grant cantrips or taking the magic initiate feat are the most efficient methods of gaining additional cantrips.
Max cantrips for a character level 4 with all levels in fighter
A tiefling taking the magic initiate feat at L4 would have an additional three cantrips for a total of five.  They would get thaumaturgy from their racial feature and an additional two cantrips from the feat.
Classes granting cantrips at 1st level
Wizard, sorcerer, cleric, druid, bard, warlock all grant cantrips:

At 1st level, you know two cantrips of your choice from the [class] spell list

Feats
Some feats grant additional cantrips known: aberrant dragon mark, spell sniper, and magic initiate.
Racial Features
Some races have a feature that grants a cantrip.  E.g. the Infernal Legacy trait of the tiefling:

You know the thaumaturgy cantrip.

Maximum cantrips innate to the character: 33
The following build uses a racial features, feats, and a ludicrous amount of multiclassing to maximize the number of cantrips known for a "level 4 eldritch knight."  The build only gets two classes to level 4, thus only gets two feats.
\begin{array}{|l|l|l|}
\hline
\textbf{Class} & \textbf{Level} & \textbf{Cantrips} \\ \hline
  \text{bard} & \text{1} & \text{2} \\ \hline
  \text{cleric} & \text{1} & \text{2} \\ \hline
  \text{druid} & \text{1} & \text{2} \\ \hline
  \text{sorc} & \text{4} & \text{5} \\ \hline
  \text{wizard} & \text{2 illusion tradition} & \text{3+1} \\ \hline
  \text{warlock celestial} & \text{3 pact of the tome} & \text{4+3} \\ \hline
  \text{rogue} & \text{3 arcane trickster} & \text{0+3} \\ \hline
  \text{artificer} & \text{1} & \text{2} \\ \hline
  \text{fighter} & \text{4 eldritch knight} & \text{0+2} \\ \hline
  \\ \hline
  \textbf{total} & \text{20} & \text{29} \\ \hline
\end{array}
\begin{array}{|l|l|}
\hline
  \textbf{Feat or Feature} & \textbf{Cantrips} \\ \hline
  \text{Racial Feature e.g. Infernal Legacy} & \text{1} \\ \hline
  \text{Spell Sniper or Aberrant Dragon Mark feats} & \text{1} \\ \hline
  \text{Magic Initiate feat} & \text{2} \\ \hline
  \\ \hline
  \textbf{total} & \text{4}  \\ \hline
\end{array}
Grand total of 33 cantrips known at level 20.
Interesting honorary mention: Mizzium Apparatus
This is a wondrous item described in Guildmaster's Guide to Ravnica on page 179 that allows the wearer to attempt to cast cantrips on their class list despite not knowing them with a DC10 arcana check.  Coupled with rogue's expertise and a decent int bonus, it would allow reasonably reliable casting any cantrip from the character's class lists.
Sauces:

Celestial Warlock is from Xanathar's Guide to Everything p.54
Aberrant Dragonmark is from Eberron: Rising from the Last War p.52
Artificer class is from Eberron: Rising from the Last War p.55
Mizzium Apparatus is from Guildmaster's Guide to Ravnica p.179
Comment: @Medix2 Flipping through it appears the backgrounds add cantrips to your spell list, but don't grant them to you directly.  Interestingly there is an item that lets the user cast cantrips they don't know, but are on their class list.
Comment: Celestial Warlock starts with 4 cantrips.
Comment: Don't forget, High Elves get any Wizard cantrip. Also, why did you mention multiclassing with *Magic Initiate*? In regards to number of cantrips known, it's strictly better than *Spell Sniper*.
Comment: Do the Ravnica backgrounds grant cantrips?
Comment: @Medix2 ooo... that's a good point. I didn't think to check there.  Also, eberron artificer might get cantrips.  I haven't used that source book enough yet.
Comment: @RevenantBacon good point about magic initiate.  I'll update that.  There are a number of races that can get a cantrip.  I figured tiefling was a decent example as it's in the basic rules.
Comment: @GcL There are only three races in the PHB that give cantrips, Forest Gnomes, High Elves, and the Tieflings. You're already listing all of the classes and feats that grant cantrips and how many they grant, I don't know why you wouldn't do the same with races as well. Oh, and also if our Celestial Warlock selects *Pact of the Tome*, that's another 3 cantrips known, from any list.
Comment: @RevenantBacon there are a number of racial variants in the Eberron expansion that grant a cantrip.  Pact of the Tome!  good catch.
Comment: @GcL I don't actually have the Eberron book, so I don't actually know what's in there. I only brought up the High Elf and the Gnome because they're in the PHB (And the elf option is strictly better :p ) Pact of the Tome is actually really good too, because if you pump Warlock to 4 and drop Arcane Trickster, you net one extra cantrip (3 base Warlock level 4, 2 Celestial, 3 Tome), AND one more ASI.


How do I change the default file manager back to Nautilus?

Question: A while back, I set my default file manager to Nemo. I like it a lot, but it constantly crashes. I remember editing a text file to set it as default, but I forgot which file it was. How do I switch back to the Nautilus file manager?
Comment: Opposite of this question: http://askubuntu.com/questions/260244/make-nemo-the-default-file-browser
Comment: Official documentation for changing and restoring file manager: [Ubuntu documentation for Default File Manager ](https://help.ubuntu.com/community/DefaultFileManager)
Answer: This did it for me, after I google for a while:
xdg-mime default nautilus.desktop inode/directory application/x-gnome-saved-search

I also typed:
gsettings set org.gnome.desktop.background show-desktop-icons true

Comment: After that my Linux Mint failed to log in with X-session error `g_key_file_free: assertion 'key_file != NULL' failed`. To recover use Ctrl-Alt-F1 & `sudo apt-get cinnamon` and `sudo reboot`
Comment: The `xdg-mime` command worked for me in Ubuntu 16.04.  I replaced `nautilus.desktop` with `pcmanfm.desktop` since I wanted to use that as my default file manager.  You can find the name of the file manager you want to use by using the following command `ls /usr/share/applications` and finding its corresponding `.desktop` file.
Comment: 5/5/20, it works on Ubuntu 19.10
Answer: I spent AGES trying to find a method that would work. Eventually I found this advice, which is incredibly easy, and worked for me:
Install exo-utils (this package contains the Xfce settings plugin and the utility files for libexo-2-0). Open a terminal and type:
sudo apt install exo-utils

Then run:
exo-preferred-applications

then switch to Utilities tab and select File Manager you prefer.
exo-preferred-applications was dropped entirely in version 4.15.3-1. Install xfce4-settings with sudo apt install xfce4-settings and use this instead:
xfce4-mime-settings

Comment: Is there any reason not to install exo-utils? (I use Ubuntu 16.04 and this method of changing the File Manager back to Nautilus, from Nemo, was the only method that worked, out of several I tried.)
Comment: This is the only way I could make `xdg-open` open directories with `nautilus` in Debian, thanks.
Comment: Worked great for me at Ubuntu 18.04.1 LTS!
Comment: It'll only work in Xubuntu or systems that have exo-utils installed
Comment: Worked well for krusader after I installed it
Comment: `xdg-open` seems to ignore this on 16.04.
Comment: Works fine for me. Just to mention that root and users have different settings so you need to run exo as user and/or as root respectivelly.
Comment: in Ubuntu 21.10 there is no `exo-preferred-applications` tool inside `exo-utils`
Answer: The answers here didn't work for me on Debian Bookworm - so after figuring it out myself:
Premise
Every time I attempt to leverage xdg-open on a directory (typically through an application; like Keybase's "open in file explorer" option, or something similar in Steam), what opens is Visual Studio Code.
What doesnt work
Attempting to "Open with other application" a directory via the context menu in "Files" [Nautilus] (file explorer when using Gnome on Debian) and then "Forget association" on Visual Studio Code (right click on it in the list of applications), subsequently choosing "Files" to open the directory with - does not fix what is used for directories when xdg-open is invoked; it only changes what opens a directory from within "Files" [Nautilus].
What does work
Run the following:
xdg-mime default org.gnome.Nautilus.desktop inode/directory

The standard, as explained, is as follows:

xdg-mime default

We're setting a default application

org.gnome.Nautilus.desktop

We're setting the default application to be Nautilus, by its fully-qualified desktop menu name; It's registered as org.gnome.Nautilus.desktop.

inode/directory

For the mimetype inode/directory (found using xdg-mime query filetype /path/to/target, with target in my case being a directory.)

The accepted answer results in Nautilus.desktop and application/x-gnome-saved-search becoming the default application(s); which is not correct, it must needs be org.gnome.Nautilus.desktop.
Also, setting desktop icons to be shown won't set the default application when opening directories via xdg-open.
Alternatively, you can also set it as follows:
nano ~/.config/mimeapps.list

Scroll down to inode/directory under [Added Associations], as well under [Default Applications], and ensure that the line(s) read:
inode/directory=org.gnome.Nautilus.desktop

You can additionally add, under [Removed Associations]:
inode/directory=code.desktop;

This is just the manual, longer, way of doing what I suggested first - together with the "Forget association" bit I portrayed I had attempted
Restore Behavior in Terminal
A user provided edit surmises that one should source the default list of file associations "for immediate effect". This answer, however, focuses on file associations for when xdg-open is invoked by another application.
With that in mind, and in reviewing the default file associations (on Debian and Ubuntu) at /usr/share/applications/defaults.list;
The default file does not seem to provide the aforementioned association. However, in the event that it did, and/or in the event that you wanted an immediate reflection of the default file associations in terminal (i.e. you're xdg-open'ing something in terminal) - you could source the default list in your current terminal session:
source /usr/share/applications/defaults.list

Comment: This has got to be the oldest thread I've seen revived, 7 years later
Comment: @nick It's a good question because it can stump people who've used Debian/Ubuntu for even longer; and 7 years later the answer(s) can/will change. You should never hesitate to answer a question regardless of its age; especially if the question exists already - and the historic answer doesn't work any longer. People are encouraged *not* to repeat questions; stack has evolved its products so that answers may stack according to most recent upvotes (recent preference changes, you may have noticed). In a sense revival isn't really a thing, because ongoing support is supported/expected.
Answer: The file manager choice is handled by a file in ~/.local/share/applications. They are mimeapps.list and mimeinfo.cache, deleting both of those files will reset your file manager to the default, which is Nautilus.
Comment: I'm afraid that solution didn't work.
Answer:
sudo mv /usr/bin/nemo /usr/bin/nemo.backup
Open any folder. You could use xdg-open /path/to/folder/ while in the command line, for example. Or you could use a X application that requests a file.
A dialog should appear to you requesting a default application to be your file manager. Type /usr/bin/nautilus there and confirm.

After that, as long as you don't rename nemo to its original name/location, nautilus will be your default file manager again.


is it possible to make fields of inline visualforce page mandatory

Question: I have one detail page where i have inserted inline VF page which is used for text search and save the results of text search into some custom field of detail VF page.There are few text fields inside inline Vf page.I want to make those fields mandatory inside inline VF page.is it possible.Or any other solution.
Comment: Can you post the inline VF and outside VF code ?
Answer: I can think of 4 options:

Create a validation rule for new records.
Use a Before Insert Trigger. Either of these two will cause the fields to become required.
You can also add code to the inline VF fields to apply the required attribute before the page can be saved.
Finally, if wrapped in a form tag, you can use the features from something like the jQuery validation plug-in that won't allow it to "submit" until those fields have been completed.

Since you've not provided code, those are the best options I can think of that would apply to the situation you've presented.
Comment: Door #3 methinks.
Comment: @AdrianLarson  #3 would be my choice as well, but many people like smorgasbords. ;)
Comment: Yes, it's the only way to answer such a question.
Answer: Yes. It is absolutely possible. You can make use of required attribute on text fields which will make the fields mandatory.Please find the screenshot which shows my test implemented inline contact's VF page where I have made Home Phone mandatory.
Comment: @shailesh-will those fields be visible in standard edit page.No right ?
Comment: @RobertBarnett Sorry if I misunderstood the question. Could you please post some of your code?
Comment: I have one standard page layout for creating a record in which i am adding inline VF page.Now when i am trying to create record that time inline Vf page wont be available.Once record is created then only we can see inline vf page as inline vf page is available on detail page only.So what i want is how to check for validation on fields of inline page while creating a new records.


How do I boot into Xubuntu after dual booting along-side Windows

Question: I recently installed Xubuntu 14.04 from USB on a netbook running Windows 7. It installed fine, but before I restarted the machine, I removed the USB stick (out of habit). I couldn't shutdown the live version properly then, but the hard disk installation of the OS apparently went just fine, so I didn't think there would be any problems. I did a hard reset, but whenever the machine boots, it always goes straight to Windows, no grub menu.
Before I started messing with it, there were 4 primary partitions on the machine:

/dev/sda1 contains the windows installation (~100GB)
/dev/sda2 is just another non-system partition that's being used by windows (~130GB)
/dev/sda3 contains some windows recovery stuff (10GB)
/dev/sda4 was a 16MB partition and I have no idea what it was for (gparted could not identify the file system - possibly some ASUS emergency browser thing).

For the installation, I freed up 40GB from /dev/sda2, deleted /dev/sda4 (a bit silly not knowing exactly what is was for, but I don't mind) and created a new extended partition, which in turn contains 2 logical partitions (2GB swap and 38GB for Xubuntu installation/data).
Did the installation not complete because I didn't shutdown the live installation properly? Do I have to manually install grub myself or did I forget to do something else?
Answer: It turns out that I accidentally chose some partition other than /dev/sda as the location for the boot loader during the Xubuntu installation, and that was the reason why I never saw the grub menu at boot time. I re-installed Xubuntu with /dev/sda as the boot loader location and it works fine now.


Lyapunov function

Question: My lecturer gave us the definition of a strong Lyapunov function. She then said that if $V$ is positive definite but $dV/dt$ is also positive definite (instead of negative definite) in a region containing an equilibrium point then the equilibrium point is unstable. If $dV/dt$ is positive definite can we conclude straightaway that the equilibrium point is unstable or do we also require $V$ being positive definite?
Answer: No,  function $V$ cannot be arbitrary. Only $dV/dt\geq 0$ is not sufficient to prove that it the equilibrium is unstable.
For example let the stable simple linear system
$$\dot{x}=-x$$
with
$$V=-x^2$$
then
$$\frac{dV}{dt}=-x\dot{x}=x^2$$
which is positive definite.
Comment: Sorry, I think you've misunderstood the question. I was asking if we can find a function V such that $dV/dt$ is positive definite in a region containing an equilibrium point then is it true that the equilibrium point is unstable?
Comment: @user152440 Well this is equivalent. I have edited my answer.
Answer: No you do not require $V$ to be positive definite.
Read the instability theorem by Chetaev

Lyapunov function


What does S1 and S2 mean with respect to quarterly reports?

Question: Was looking through earnings reports on Bloomberg Terminal and noticed in addition to Q1 Q2 Q3 Q4 and annual reports, there are also S1 and S2 reports. They seem to be semi-annual but what exactly are they?
Answer: Most likely they are the two halves of the year: S1 is Q1+Q2 and S2 is Q3+Q4. More typically these would be called H1 and H2. But it seems to be S for semester, whose original meaning from Latin is a six-month period.
Answer: There are two other possible meanings of S1 and S2 reports.

S1 and S2 might refer to subsidiary companies.
S1 and S2 (but more commonly S-1 and S-2) might refer to SEC Form S-1 -- a form filed in anticipation of an Initial Public Offering -- and SEC Form S-2 -- a simplified version of the S-1. But S-2 was discontinued in 2005, so that's probably not it.


Aptana 3 plugin for eclipse and publish

Question: What I am after is the ability to right click on an edited file and send it to either an SFTP site or a mapped location from within Eclipse so if there is another plugin besides Aptana which allows me to do that it would work as well.
I have been using the Aptana 2 plugin with Eclipse for a long time for its ability to push files to either a server (via SFTP) or mapped drive.
I just updated my Eclipse install to Juno and Aptana 2 will no longer install due to changes in Eclipse.
I then installed Aptana 3 and am not able to set up publish options for my projects.
I have seen the wiki on the Aptana site which tells you to use the App Explorer then click the left button on the toolbar to set deployment options.
Unfortunately in eclipse (at least my install) the left button does nothing.
I have also tried right clicking the project and choosing publish (no options there either)
Is this broken in the plugin but working in Studio?
I have tried Remote system explorer as well but I don't want to edit remote files, I want to edit locally commit to source when ready and deploy to remote site.
PS
I had installed Aptana 3 previously in Eclipse Indigo but then removed it and reverted to 2.0 due to the same issue but that doesn't seem to be an option now due to the changes in Juno.
Any help would be greatly appreciated as I can't seem to find an answer in Google or here.
Thanks.
Comment: Found an open bug on the Aptana support site.
Root of the issue seems to be with Eclipse but there is a patch posted on the bug thread which corrected it

https://jira.appcelerator.org/browse/APSTUD-4998
Answer: Found an open bug on the Aptana support site. Root of the issue seems to be with Eclipse but there is a patch posted on the bug thread which corrected it jira.appcelerator.org/browse/APSTUD-4998

Aptana 3 plugin for eclipse and publish


Bad corners after using Unite from Illustrator's Pathfinder

Question: Very new to Illustrator, trying to sort it out, created a few shapes, ellipse and rectangle, and then tried to join them together through Illustrator's Pathfinder > Unite effect and ended up with rough jagged corners where the two meet together. How can I clean this up? or use the same effect without the flaws?
Answer: That looks like the shapes weren't meeting properly before you applied the unite.
To ensure that you get a nice clean join you need to make sure that there is a good overlap between the shapes before uniting them. In this case, that would mean extending the square shape upwards so that it protrudes further into the ellipse.
Comment: Yeah, that did the trick. Sure am loving Illustrator now! Used GIMP before and after briefly got to know Inkscape, but Illustrator runs circles around both of em. Looking forward to learning more.


Efficient way to print Q&A?

Question: What I do is copying the Q&A to a Word document, rearrange it as needed (if needed), and then print it. Is this the best way possible? How to easily and efficiently get a printable version?
Answer: Why don't you try an Ad-on for that. There is a Firefox Ad-on called Print Edit.

Allows editing of web page content while in Print Preview mode, prior to printing or saving as HTML or PDF.
Compacts the layout and removes unwanted content such as adverts, sidebars and blank pages. Any element can be formatted, hidden or deleted.

I've just tried it for couple of Q+As on DA. It's working good.

Efficient way to print Q&A?


reducing word2vec dimension from Google News Vector Dataset

Question: I loaded google's news vector -300 dataset.  Each word is represented with a 300 point vector.  I want to use this in my neural network for classification.  But 300 for one word seems to be too big.  How can i reduce the vector from 300 to say 100 without compromising on the quality.
Answer: tl;dr Use a dimensionality reduction technique like PCA or t-SNE.
This is not a trivial operation that you are attempting. In order to understand why, you must understand what these word vectors are.
Word embeddings are vectors that attempt to encode information about what a word means, how it can be used, and more. What makes them interesting is that they manage to store all of this information as a collection of floating point numbers, which is nice for interacting with models that process words. Rather than pass a word to a model by itself, without any indication of what it means, how to use it, etc, we can pass the model a word vector with the intention of providing extra information about how natural language works.
As I hope I have made clear, word embeddings are pretty neat. Constructing them is an area of active research, though there are a couple of ways to do it that produce interesting results. It's not incredibly important to this question to understand all of the different ways, though I suggest you check them out. Instead, what you really need to know is that each of the values in the 300 dimensional vector associated with a word were "optimized" in some sense to capture a different aspect of the meaning and use of that word. Put another way, each of the 300 values corresponds to some abstract feature of the word. Removing any combination of these values at random will yield a vector that may be lacking significant information about the word, and may no longer serve as a good representation of that word.
So, picking the top 100 values of the vector is no good. We need a more principled way to reduce the dimensionality. What you really want is to sample a subset of these values such that as much information as possible about the word is retained in the resulting vector. This is where a dimensionality reduction technique like Principle Component Analysis (PCA) or t-distributed Stochastic Neighbor Embeddings (t-SNE) come into play. I won't describe in detail how these methods work, but essentially they aim to capture the essence of a collection of information while reducing the size of the vector describing said information. As an example, PCA does this by constructing a new vector from the old one, where the entries in the new vector correspond to combinations of the main "components" of the old vector, i.e those components which account for most of the variety in the old data.
To summarize, you should run a dimensionality reduction algorithm like PCA or t-SNE on your word vectors. There are a number of python libraries that implement both (e.g scipy has a PCA algorithm). Be warned, however, that the dimensionality of these word vectors is already relatively low. To see how this is true, consider the task of naively representing a word via its one-hot encoding (a one at one spot and zeros everywhere else). If your vocabulary size is as big as the google word2vec model, then each word is suddenly associated with a vector containing hundreds of thousands of entries! As you can see, the dimensionality has already been reduced significantly to 300, and any reduction that makes the vectors significantly smaller is likely to lose a good deal of information.
Answer: @narasimman I suggest that you simply keep the top 100 numbers in the output vector of the word2vec model. The output is of type numpy.ndarray so you can do something like:
<code>>>> word_vectors = KeyedVectors.load_word2vec_format('modelConfig/GoogleNews-vectors-negative300.bin', binary=True)
>>> type(word_vectors["hello"])
<type 'numpy.ndarray'>
>>> word_vectors["hello"][:10]
array([-0.05419922,  0.01708984, -0.00527954,  0.33203125, -0.25      ,
       -0.01397705, -0.15039062, -0.265625  ,  0.01647949,  0.3828125 ], dtype=float32)
>>> word_vectors["hello"][:2]
array([-0.05419922,  0.01708984], dtype=float32)
</code>
I don't think that this will screw up the result if you do it to all the words (not sure though!)


Linearization before curve fitting

Question: I am currently studying at college curve fitting to a set of experimental data. One of our activities/homework was to fit the curve $y=ax^b$ to the set of points of the position of a free-falling object we filmed. My professor stated that it was always necessary to first linearize the data, by taking the logarithm of the $x$ and $y$ values, and then do a least squares regression on this new data, and that the result of this linear regression would be the final correct value. However, doesn't this change the final result compared to a non-linear regression? Not only that, wouldn't the result change depending on the units, because $\log(y)$ is the logarithm of a quantity which has units? If it changes the final result of the regression, which method is better to use in an experimental context such as the one mentioned above?
Comment: Sorry for the delay. I just edited my answer. Cheers :-)
Answer: Your professor is totally correct. The model
$$y=a \,x^b$$ is nonlinear because of $b$. So, in a first step, you linearize
$$\log(y)=\log(a)+b\log(x)=c+b\log(x)$$ and a linear regression gives the estimates of $b$ and $a=e^c$.
Now, you start with them the nonlinear regression what you must do since what is measured is $y$ and not any of its possible transforms.
Edit
When you linearize the model, the residue at point $i$ is
$$\text{res}_i=\log\big[y_i^{\text{(calc)}}\big]-\log\big[y_i^{\text{(exp)}}\big]$$ which can rewrite as
$$\text{res}_i=\log\Bigg[\frac{y_i^{\text{(calc)}} } {y_i^{\text{(exp)}} } \Bigg]=\log\Bigg[1+\frac{y_i^{\text{(calc)}}-y_i^{\text{(exp)}}  } {y_i^{\text{(exp)}} } \Bigg]$$ So, if the error is small
$$\text{res}_i \sim \frac{y_i^{\text{(calc)}}-y_i^{\text{(exp)}}  } {y_i^{\text{(exp)}} }$$ which is the relative error.
In practice, when the range of the $y_i^{\text{(exp)}}$ is very large, it is often preferable to stop at this point. This is the case for physical properties like vapor pressure.
Comment: In general the least squares idea is directly related to the maximal likelihood for the i.i.d. Gaussian noise model though it is rarely discussed in the introductory courses, so which minimization scheme is more physically relevant really depends on which noise model makes more sense. However, LSM for a linear function is computationally trivial and for a non-linear one may be a headache, so there is a tendency to linearize no matter what. @ordptt is right that linearization can change the answer *from the non-linear problem* but the linear LSM commutes with scaling and shifts so units are OK.
Comment: I see! Well, the problem is that my professor stated that the results of the linear regression are the final answer, and doing a nonlinear regression with the found estimates afterwards is incorrect, which, I think, is what is causing my confusion.
Comment: @ordptt. If the data are not too noisy, the linearized model corresponds to the minimization of **relative** errors. If you are concerned, I can elaborate
Comment: If possible, that would be great!
Answer: Let me expand on a couple of aspects of the previous answer that may not be obvious to a novice in curve fitting. The first is to note that a least-squares regression implicitly involves a weighting factor or uncertainty at each data point. If you can estimate the uncertainty in your data at each point and it is approximately gaussian, the statistically sound way to fit the data is to minimize
$$\sum_i \left[\frac{f(x_i; a_i)-y_i}{\sigma_i}\right]^2$$
where $f(x_i; a_i)$ is your model function at the point $x_i$ with fit parameters $a_i$, $y_i$ are your measured data points, and $\sigma_i$ is the uncertainty at the point $i$.
If you ignore the weights in the intial problem (by setting them equal to 1), the weights for your transformed problem will necessarily be different leading to a different fit. For small errors, you have converted the problem from assuming absolute equal weights to one where the weights depend on the position. That's why an unweighted linear fit gives a different answer than an unweighted nonlinear fit.
If the errors are large, there is another complication. The transformed noise in the data may no longer be gaussian, even if it was close to gaussian initially. This could skew your fit so that it isn't the most likely model function to have resulted in your measured data.
Let me address the unit question directly with a concrete example. If you explictly include the units, things will cancel out nicely into dimensionless numbers. Let's say $y$ is in units of meters and $x$ is in units of seconds. Then $a$ will have to have units of meters/seconds$^b$. Treating the units like algebraic quantities, let $v = y$/meters, $\alpha = a$seconds$^b$/meters, $u=x$/seconds. The dimensions all cancel each other and the resulting equation
$$v=\alpha u^b$$
has all dimensionless variables.

Linearization before curve fitting


Is there anyway to make your endorser only inject endorsements for priority 0 blocks?

Question: Is there anyway to make your endorser only inject endorsements for priority 0 blocks, and not inject for priority 1 or higher ?
Answer: There isn't an option to set a max priority like the baker binary does. This is because there shouldn't be a good valid reason to do this. The endorser by design will endorse the highest priority block available since doing so yields the greatest reward. By choosing to not endorse lower priority blocks, you'll miss out a lot more rewards that you could had otherwise earned.
If the goal is to avoid the rare case where your endorser prematurely endorses a lower priority block that yields a losing chain, that fear is not merited. By design, these occurrences should never happen as there is specific time windows where each priority can bake based on the previous block. The time windows for different priority bakes shouldn't overlap, at least your endorser won't acknowledge them as validate. As long as your node has all the propagated operations known to the network, the endorser should not endorse the "losing" chain.


Registering SevenZipLib

Question: I have "legacy" software in C# which uses SevenZipLib.
I have downloaded SevenZipLib, but I dont know how to declare:
<code>using SevenZipLib;
</code>

I have tried adding the reference without any result.
Comment: When happens when you add the dll as reference? Does it fails?
Comment: Have you added a reference to the assembly (dll) of SevenZipLib?
Answer: Make sure that you add a reference to the dll in your project/solution. Right click on your project and click Add Reference. Browse to the directory that you extracted SevenZipLib and click on the dll to add the reference.
Also, if you are able, you might look into installing the SevenZipSharp Nuget Package as an alternative. Installing the Nuget Package will automatically download any required dependencies and add them to your project/solution. Please note that if you go with SevenZipSharp, you will have to rewrite the 7-zip part of your application to use SevenZipSharp instead of SevenZipLib.
Answer: The file 7z64.dll which you are trying to add is a native DLL, not a .NET Assembly and not a COM component. You cannot add this dll as a reference.
Add a reference to a managed DLL or use P/Invoke to load and call 7z64.dll

Registering SevenZipLib


How to change SQL Server Service Account using PS

Question: SQL Server runs under AD Account SQLAccount1. How to make it run under another account, SQLAccount2? How to do it un Powershell?
Answer:
The GUI way to do this is to go to SQL Server Configuration Manager.  Go to the properties of the SQL Server and change the account name on the Log On tab.  Click the Browse button.

Than enter the account that you want to SQL Server Log on as.  Click Check Names, to make sure the account exists.  Than click OK on select user or group and OK on SQL Server Properties.

Than restart.
2.This blog Hey, Scripting Guy! Blog, the author Aaron Nelson - SQLvariant shows how to do this in PowerShell.
<code>[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SqlWmiManagement") | out-null

$SMOWmiserver = New-Object ('Microsoft.SqlServer.Management.Smo.Wmi.ManagedComputer') "WIN7NetBook" #Suck in the server you want

#These just act as some queries about the SQL Services on the machine you specified.
$SMOWmiserver.Services | select name, type, ServiceAccount, DisplayName, Properties, StartMode, StartupParameters | Format-Table

#Same information just pivot the data
$SMOWmiserver.Services | select name, type, ServiceAccount, DisplayName, Properties, StartMode, StartupParameters | Format-List

#Specify the "Name" (from the query above) of the one service whose Service Account you want to change.

$ChangeService=$SMOWmiserver.Services | where {$_.name -eq "MSSQLSERVER"} #Make sure this is what you want changed!

#Check which service you have loaded first

$ChangeService

$UName="DomainName\UserName"

$PWord="YourPassword"

$ChangeService.SetServiceAccount($UName, $PWord)

#Now take a look at it afterwards

$ChangeService
</code>
Answer: Microsoft has published a SQL Server module for Powershell Desired State Configuration (DSC) that you can use to change the SQL Server service account.
I have only tested this out with Powershell 5, so I don't know how well it works with Powershell Core, so you may experience
Please note this will RESTART YOUR SQL SERVER.
<code># THIS WILL RESTART YOUR SQL SERVER SERVICE
$credential = Get-Credential -Message 'Provide the username (domain\username) and password for the SQL Server service account to run under';

$SqlServiceAccountParams = @{
    ServerName     = 'localhost'
    InstanceName   = 'MSSQLSERVER'
    ServiceType    = 'DatabaseEngine'
    ServiceAccount = $credential
    RestartService = $true
};

# Required to be able to run the
Install-Module 'SqlServerDsc';

$scriptBlock = {
    $ErrorActionPreference = 'stop';
    throw "I understand that this will RESTART THE SQL SERVER. Remove this line if you understand that your SQL Server will restart and are ready for the restart.";
    Invoke-DscResource -Name 'SqlServiceAccount' -ModuleName 'SqlServerDsc' -Method Set -Property $SqlServiceAccountParams;
}

invoke-command $scriptBlock

</code>
Powershell DSC -
https://learn.microsoft.com/en-us/powershell/dsc/getting-started/wingettingstarted?view=dsc-1.1
SQL Server Powershell DSC Module - https://github.com/dsccommunity/SqlServerDsc


Vector Autoregression

Question: I have a general question on VAR-methodology. In the case of asymmetric modelling I employ FGLS to exploit off diagonal covariance between residuals due to non-unique regressors between equations. Ok, now. What about the vectors that are by necessity zero, because they were never included? Should they be set to zero later in the Var-modelling or estimated as with restricted paramters (set to zero). This seems to me to take me from FGLS modelling to restricted-LS modelling. Is this even possible? So in a nutshell, which is preferred, to omit a parameter and set it to zero later, or to include it and restrict it to zero during estimation?
Comment: What do you mean by ***vectors*** *that are by necessity zero, because they were never included*? Could you give an example? And how can you *omit a parameter* when setting up OLS or FGLS estimation?
Comment: It would be helpful if you updated the original post with some equations illustrating what is what.
Comment: Hi, and thanks for taking this on! Well, as you know in your VAR-universe we assume to be including "everything". But we really aren't. FGLS is equivalent to LS if regressors are identical, so we let lags vary between equations sometimes. Now, when constructing our companion matrix to check for stability, how do we format the individual A's (of equationwise estimated paramters) to get symmetry in this matrix? Just set the "un-estimated" paramters to zero?
Answer: Since the "universe" is by definition limited anyway (p < infinity), I decided that whatever 1..p-lags,  chosen for the individual equations represent are sufficient. Setting Everything else to zero might not be correct model selection wise, but i cannot upset the model mathematically.

Vector Autoregression


WordPress database error can't create/write to file as there is no space left on device

Question: We recently updated PHP to version 7.2. We had to reboot the server to apply the changes. The site originally ran on apache but was changed over to nginx about a year or two ago by another admin who has now left. After the reboot apache started automatically and the site ran of apache unknown for 2 days.
The OS is Cpanel 11.68 autoinstaller (CentOS 6 64bit) (64bits).
Today I made changes to the wordpress file and folders permissions as we noticed they were incorrect after rebooting the server. Then things started going wrong.
First off was this error in the wordpress error-log
WordPress database error Can't create/write to file '/tmp/#sql_221b_0.MAI' (Errcode: 28 "No space left on device")

Upon investigation I found that Inodes may have been the problem. When I run df -i I got the following output
Filesystem        Inodes   IUsed     IFree IUse% Mounted on
rootfs           3203072 3203072         0  100% /
/dev/root        3203072 3203072         0  100% /
devtmpfs         8241707    2422   8239285    1% /dev
/dev/md3       118865920  193826 118672094    1% /home
tmpfs            8250083       1   8250082    1% /dev/shm
/dev/root        3203072 3203072         0  100% /var/tmp

Based on this article I deleted the /var/tmp folder and rebooted the system but it still shows Iuse at 100% for the var/tmp folder.
When I run df -h I get the following
Filesystem      Size  Used Avail Use% Mounted on
rootfs           48G   24G   22G  53% /
/dev/root        48G   24G   22G  53% /
devtmpfs         32G  508K   32G   1% /dev
/dev/md3        1.8T  147G  1.6T   9% /home
tmpfs            32G     0   32G   0% /dev/shm
/dev/root        48G   24G   22G  53% /var/tmp

As the Inodes is reporting at 100% Use I'm making an assumption that this is where the problem lies and not in the permissions.
I've deleted logs from various directories and even deleted a 122gb backup file that was created on Sunday in /home/backup/weekly/YYYY-MM-DD/accounts/ but still the problem exists.
I've also had numerous errors in the WHM complaining about not being able to write to cache because of permissions at 600 and No space left on device
How can I tell if it's a problem with permissions or Inodes?
If it is a problem with Inodes how can I fix it? There is only 2 files at 4.0k in /var/tmp/ now, plus a symlink to mysql.sock
If it is a problem with permissions how can I find the correct permissions/directory to change?
Answer: What fixed my issue was just clearing the TMP directory and restarting the server. You can also try to just restart the mysql service. We are running CentOS as well so hopefully it's a similar issue.


Discrediting young academic in a conference

Question: I'm an early-career scientist and I attended a conference 2 years ago organised by a local university. I presented the results of one of my first funded research projects and I included my assistants as co-authors. The presentation went well and the rigorous work has impressed the panel and the audience and even one panel are interested to collaborate with me. However, one of the panel, instead of giving feedbacks decided to somehow "discredit" us and our effort by saying "So you are from University X and you must be a colleague of Professor Y. I have high regards of her work and thus I believe in your work." The thing is Professor Y was never involved in my project nor familiar in my work. I wasn't offended by that remark (or should I be?) but I feel that this kind of comment is unfair and shouldn't be tolerated especially when dealing with young scientists starting their career. Today, I was concerned that this panel member will be again present in an upcoming conference and I am sure he will do the same remarks again as he did to other before in past conferences he panelled. And I am also sure that our paths will cross again this time. My question is: Should I "call him out" when he do this behaviour again? Is it polite or proper to "call him out" during the panel discussion? Any advise how will I deal this kind of behaviour without being impolite.
Comment: I don't see at all why that comment would be seen by others to discredit you. The person was essentially trying to make a connection and support you. Perhaps a cultural difference?
Comment: I don't understand why this question is downvoted so many times. This post reflects a sentiment that may seem paranoid at first, but can nevertheless be common in early career professionals.
Comment: There's a small but significant difference between "...and **thus** I believe in your work" and "...and I believe in your work **too**". The first can be taken to mean that your work is good not by your own merit but because of others in your university, but the second is a compliment!
Comment: While a bit odd, I don't understand what's wrong, can you explain?
Comment: @xuq01 as early career and starting to establish your name and reputation, the action of that person (panel) is a little bit discouraging. The way I understand his approach and remarks was like he is trying to overshadow my work and it feels like "Ok I don't care about your work, you get my trust and attention because you are associated to Professor X", but what if I am associated to another professor he doesnt know? But thanks for getting my point.
Comment: @AzorAhai what is wrong with this is the panel is unjustly overshadowing my work by associating me with a random professor he knows who is not actually familiar with my work just because we are on the same department.
Comment: @JonCuster I don't see it he is trying to make connections or support. How it becomes he is trying to support?
Comment: @xavier Maybe this is a language issue, but if you have quoted their statement in English (and not translated it for us), it's not really "unjustly overshadowing." That would be saying something like "Wow, Professor Y must have been a huge help in doing this, no wonder it's such good work." The statement you've relayed to us is as innocuous as saying "University X does great work in this field, and it looks like you do too." Like I said, it's a bit off but I don't think they truly believe you didn't do any of the work.
Comment: @xavier - I have no right telling you how you should take that comment. But, having seen/heard similar sentiments that I know were not meant to be demeaning, I just think they are saying something positive about you carrying on the high standards of your institution. But, your feelings are yours, and yours alone.
Answer: This sounds like positive banter/smalltalk. It's a way of creating a positive link which the OP interprets negatively. I personally would interpret the statement as the person liking your work but not wanting to say so directly, as direct compliments are frowned upon in many cultures or even sound suspicious. Obviously, OP takes it exactly in the opposite spirit - so one never gets it right.
I will hypothesise that calling them out will only:

cause not to want to talk to you anymore;
cause them not wanting to deal with you ever again in the future;
if they know them, probably privately remark to Prof. Y about the strange and borderline impolite behaviour of OP, despite the good work of OP.

To be honest, I fail to see how this statement could possibly be negatively interpreted, but obviously OP did so. Nonetheless, my very strong recommendation to OP is to reconsider whether this is not actually meant as a very complimentary statement.
Answer: Don't read too much into the remark. While some would say such things to lessen you, others wouldn't even think of it in that way and, instead, consider it a neutral or even positive comment. On the face of it, the person is trying to connect himself with you through the other person.
In particular, with no more information than what you write here, I think it would be a mistake to "call him out".
I would take the worst case interpretation of the comment as a person who is a bit insecure and is trying to boost them-self, rather than tear you down. That you suggest this is a common practice of the person seems, to me at least, to support that view.
But even if the person just goes around tearing people down, it isn't worth the mental effort to respond. They are what they are. There is nothing to be gained in that case.
Comment: At first, I received the comment as neutral and it wasn't offending. It was a positive comment, but "believing and applauding" my work because I am associated with a random Professor X (who doesn't even have familiarity with my topic) seems he is trying to unjustly overshadow my work.


Why does the flame of a gas stove not spread backwards?

Question: On lighting a gas stove the flame is active only above the burner. However, the gas input from beneath the burner and all the way to the pipeline contains gas fuel. Why doesn't it all ignite up ? I am guessing it may be due to the pressurised flow of the fuel in one direction but then I am unable to relate it to combustion, i.e., if a combustible substance is in contact of a source of ignition then it should start to burn.
Comment: If it were a mixture of gas and oxygen, that would be bad. No oxygen available, no flame.
Comment: In addition, chemical reactions take some time and it's not instantly flame.
Answer: The methane that goes to the stove contains little or no oxygen, so it will not burn until oxygen gets mixed into it.  This occurs at the burner to the extent that a blue flame is formed in order to obtain complete combustion.  At the point where air is mixed into the methane, the velocity of the flowing methane and air is kept high enough to exceed the methane flame velocity, so the flame cannot go backwards through the burner.  In addition, the metal parts of the burner are substantially colder than flame temperature, and if the gas flow is decreased to the point where the flame tries to go backwards through the burner parts, the close clearances and cold metal temperature of the metal burner parts quenches the flame.
A good example of the type of behavior described above can be seen when a bunsen burner is lit.  The chimney of the bunsen burner is adjusted such that the air register at the bottom of the burner admits enough air to obtain a blue flame.  If you continue opening that air register, the air and gas flowing up the bunsen burner tube increase in velocity until the flame has difficulty burning fast enough to keep up, and eventually, the high velocity up the burner tube blows the flame away from the burner, and the flame goes out.  On the other hand, if the gas and air flow are decreased to a very minimal amount, the flame cannot propagate down the tube because the tube diameter is small, and the flame gets quenched by the "cold" walls of the bunsen burner tube.


SOQL Alias a column

Question: I am wondering how you would do something simple like this in SOQL.
select first_name as name from tableA

I can not find any documentation on Aliasing a column which is very common in all forms of SQL, does anyone know if this is possible?
Comment: I wish this feature existed. I write SOQL queries in R and having to `rename` fields in every tibble I produce gets old fast.
Comment: Ya they make it pretty painful!
Comment: If you need support aliases in SFDC SOQL please up-vote this idea: - https://trailblazer.salesforce.com/ideaView?id=0873A000000lEA2QAM
Answer: SOQL has very limited aliasing support. You can alias the table, e.g. SELECT a.name FROM Account a, but that's of very little use. In most contexts you cannot alias fields.
The one exception is for queries that use aggregations, like SELECT COUNT(Id) recordCount FROM Account allows you to alias the result of the aggregation from it's default expr0 to recordCount.
This probobly doesn't do what you're looking for, and you'll have to use some sort of transform in apex, middleware, or on the API client (depending on what and how you're building) to alias columns in non-aggregate use cases.
Answer: You can use object aliasing in any SOQL query
From the SOQL reference guide.....

Alias Notation
You can use alias notation in SELECT queries: SELECT count() FROM
  Contact c, c.Account a WHERE a.name = 'MyriadPubs'
To establish the alias, first identify the object, in this example a
  contact, and then specify the alias, in this case "c." For the rest of
  the SELECT statement, you can use the alias in place of the object or
  field name.
The following are SOQL keywords that can't be used as alias names:
  AND, ASC, DESC, EXCLUDES, FIRST, FROM, GROUP, HAVING, IN, INCLUDES,
  LAST, LIKE, LIMIT, NOT, NULL, NULLS, OR, SELECT, WHERE, WITH

See page 2 of the reference guide here
You can only use field aliasing with aggregate expressions.  For more info on aliasing with aggregates, take a look here
http://www.salesforce.com/us/developer/docs/dbcom_soql_sosl/Content/sforce_api_calls_soql_select_groupby_alias.htm
Comment: Note that this is an object alias, not a field alias. Salesforce only allows field aliasing in aggregate expressions. i.e. `GROUP BY`
Comment: Indeed.  I didn't make that clear, will edit the answer to reflect.  Thanks
Comment: @ChrisDuncombe what is the point of object aliasing? Is there any practical use case in SOQL
Answer: As Chris mentioned, you can only alias aggregated fields. The syntax to use is like this:
SELECT Status, COUNT(id) Total
FROM Case
GROUP BY Status

Notice, no "AS" keyword like in vanilla SQL
Comment: You can now alias also Status column in your example: https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_groupby_alias.htm
Comment: I see the salesforce page of @Sankozi, but still get "only aggregate expressions use field aliasing" even when going to api-level 58
Answer: Alias notation also works with the toLabel() function in both SOQL and SOSL queries.

The toLabel function supports aliasing. In addition, aliasing is required when the query includes the same field multiple times. For example:
FIND {Joe} RETURNING Lead(company, toLabel(Recordtype.Name) AliasName)
— SOSL Reference: toLabel(fields)

The above excerpt, shows the use of the alias notation with toLabel function in a SOSL query, however, the same syntax works for SOQL queries.
For example, if we have the following Account records in Salesforce:
+--------------+--------------------+
| Name         | BillingCountryCode |
+--------------+--------------------+
| A            | US                 |
| B            | IE                 |
| C            | CA                 |
+--------------+--------------------+

Without the toLabel alias we would only be able to select one of BillingCountryCode or toLabel(BillingCountryCode). If we tried to select both without an alias we would run into an error:

duplicate field selected: BillingCountryCode

Now, if we use alias notiation, we can query for both the Billing Country Code as well as the label as follows:
SELECT
    Name,
    BillingCountryCode,
    toLabel(BillingCountryCode) BillingCountryLabel
FROM Account

Then the above query would produce a query result as follows:
+--------------+--------------------+---------------------+
| Name         | BillingCountryCode | BillingCountryLabel |
+--------------+--------------------+---------------------+
| A            | US                 | United States       |
| B            | IE                 | Ireland             |
| C            | CA                 | Canada              |
+--------------+--------------------+---------------------+

Comment: Interestingly the `toLabel(field) alias` syntax still allows you to return the usual SObject type back instead of `AggregateQuery`. The label data is only accessible via `.get('fieldName')` though of course.
Comment: @RaphaelD You need to use [`SObject.get()`](https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_methods_system_sobject.htm) to access the aliased label. e.g  `String label = (String) opp.get('BillingCountryLabel')`
Comment: Only picklist fields, field history, or platform entity fields (or Record Type Name, Description in the SELECT list) support translation.  =(
Comment: I did the same querry but I am not able to access to Opportunity.BillingCountryLabel
Answer: Aliasing like this:
select first_name as name from tableA

isn't possible in Salesforce SOQL.
Answer: I think the answer is yes you can. Kind of.
The OP is asking whether you can do this like you can in SQL:
select first_name as name from tableA

So whilst SOQL doesn't support the above, you can achieve the same effect like this:
AggregateResult[] people = [select Firstname Name from tableA group by Id, Firstname];

I can see many others have mentioned aggregate queries but nobody has made it clear that you can take advantage of the aliasing possible with the group by query - the trick is to group by something unique like the Id.  All the other fields you want to select out must be in the group by clause as well.
Access the result like so:
for(AggregateResult person:people) {
  String theirName = person.get('Name');
}

Not sure why this isn't used more widely - it's what I do when I need to alias like you do in regular SQL.
Comment: Thanks a lot for showing how to reference the field in Apex!
Comment: I think the reason to avoid this, when not needed, is performance, cause this will require extra database effort to group by Id and Name.
Comment: agreed Jeferson!
Answer: Directly write the alias name
for example:
select AccountID account ,count(id) countid from Account
you can also retrieve using get() function by passing the alias name in ' ' quotes
example: object_reference.get(;
Answer: Finally we have found decision how to "rename" column in SOQL without using standart Aggregate function that requires Group by clause in the end, and unfortunately doesn`t allow to export more that 2000 rows in query.
The best way is using Format(Field) Custom_Name.
It could be tricky way to use this with some date values or etc. cause function changes format of values in field.
The name of Hero who advice it - Roman Bielei

SOQL Alias a column