Quantifying how successful results of clustering are with a test

Question: I have a three types of experiments with many measurements taken. I want to show that using clustering on the unlabelled measurements, I can uncover the three groups. Now using kmeans plus, I get around 60% accuracy. Which if it was binary, would be poor, but how do I quantify how good that is for three categories? Is there a p value I can compute and compare against? I am thinking that if there were 100 categories 60% for the correct labelling is great and far from random, so how can I do this with a test?
Comment: @Vass, indeed (although you could project down to 2d with PCA, to plot). Are any of of the points labelled, for comparison ?
Comment: any specific reason you want to use clustering?
Comment: Are your measurements 1d, 2d ... ? Can you put up a scatter plot / 3x3 confusion matrix for your data or, even better (many eyes) for some synthetic data ?
Comment: @Denis:The data is 112d. Therefore I don't think I can use a scatter plot.
Comment: @Denis, they are not labelled, but I know from which category they were produced
Comment: I did use pca with 2d for a biplot, and the results were not great though. Evades this problem though. People keep using BIC or AIC. I dislike both
Answer: You can get a p-value from a Monte Carlo resampling approach.
Take your number of obserations (n) and randomly distribute them into k groups. Repeat this procedure many times (10,000+). Count the number of times in which this process yields a clustering equal to or better than your own. Put that number over the total number of Monte Carlo trials, and that's your p-value: the chance of getting a clustering as good or better than yours by chance alone.
I agree with Greg Snow, though, that a p-value isn't a measure of prediction strength. Reporting the p-value along with Cohen's kappa and (of course) the descriptive statistics would probably be a good idea.
Comment: To be honest I am not really sure. I think Cohen's kappa (or simply a confusion matrix) can provide a good picture of how well a clustering algorithm performs for -your- data, and not just how well the algorithm performs in general. Mostly I was just suggesting that a p-value derived from this approach, while informative, does not tell the whole story. The reason is that if you know your data is clustered, it's not surprising that a clustering algorithm performs better than random selection (and thus has a low p value).
Comment: that is actually what I ended up doing. Luckily the random fluctuations reach this configuration less than the p-value. thanks!
Comment: is this not related to consensus clustering in a way or consensus statistics? The kappa of Cohen is more useful to see how different algorithms for clustering perform, still very interesting. But unless I have not understood the same algorithm can be used multiple times independently for a Cohen kappa?
Answer: You could use Cohen's kappa test for the 3 x 3 classification table (actual membership by clustering membership).
Answer: Testing for a statistically significant relationship and quantifying the quality of predictions are 2 very different proceedures.  With enough data it is possible to show something is statistically significant even when the effect is to small to care about.  For your example with 3 groups (assuming they are the same size) we would expect a  completely random clasiffication to get the right answer $\frac13$ of the time.  A method that was correct 35% of the time could be found to have a very small p-value, but would not be a practical improvement above 33%.  On the other hand, with small data you may have a classifier that is right 90% of the time but not statistically significant.


Enter text into executable file immediately after it's run

Question: I have an executable file file that I am running by typing ./file into the terminal. When the program runs, I have to type in the text "code". However, the time taken to type it, or paste it into the terminal, results in the program saying "sorry, you took too long!".
I am tying to send the text "code" to the executable file when I run it, as therefore it would be inputted into the program immediately once it's run.
I've tried ./file; "code" and ./file && "code" but have had no luck - the program still wants an input.
Comment: Are you looking for `echo "code" | ./file` ?
Comment: @ArtemS.Tashkinov Brilliant! That's it, thanks very much
Answer: You may want to use the pipeline feature https://en.wikipedia.org/wiki/Pipeline_(Unix) :
echo "code" | ./file


How do i recover my data on a Galaxy Tab after a factory reset?

Question: I forgot my password to unlock my Samsung Galaxy Tab, so I did a factory reset to get into it. Now I would like to recover as much of my previous data that I can. Is this possible? If so, how do I go about recovering it?
Answer:
You still got your SDcard contents
You still got synced data on Google cloud associated with your Google account and other cloud services.
Everything else is wiped out and cannot be restored

All you can do is log in with same Google account on your device and it will restore some data from cloud like Google Contacts, Calendar and, maybe, settings for some apps. Purchased apps are still available from Google Play and you don't need to buy them again. Install Dropbox, Google Drive and other apps which store their data outside your device.
In future, try to keep copy of everything important on other device such as your computer or in cloud. Nowadays you can get lots of free storage on many services like Dropbox, Google Drive, Box, Skydrive etc. You can avoid wiping your device in future but you cannot avoid losing it or breaking it, so backups is very important.
Comment: @david6 which is only true when you have root access to the device in question. And even then only partly, as after the device is started up following a factory-reset, structures will be rebuilt (Dalvik Cache etc.), and new installed/configured apps write to the file system where other data might have been before. Plus, even with good forensic tools it's not "as easy as 1-2-3", but requires at least some, but more likely a lot of manual work.
Comment: *If everything could just be easily recovered after a data wipe/factory reset, that same data would be available to the finder (possessor) or any lost/stolen tablet (or smart-phone).*  That said, there are forensic tools out there (if you can gain access to them) which may be able to recover **ALL** recent data and history.
Comment: I was referring to RAM/ROM-debug tools that (in theory) only LEO and Intelligence persons have access to.


First work in which vampirism was cured

Question: Lots of SCi fi/fantasy universes have vampires. Which was the first of these to have a cure that could turn someone back into a human after they were infected? How did it work?
Comment: How far can the person be infected?  In *Dracula*, Lucy, having actually died, was a lost cause, but Minna was gravely tainted -- she could be kept bound by holy things -- and made it back because she hadn't actually died.
Comment: Would depend on what you're willing to accept as a vampire, I suppose, e.g. Bram Stoker style vampires could not be cured once they died, wheras Buffyverse vampire Angel was cured by absorbing Mohra demon blood and in both cases they were supposedly demonic undead, (which would suggest you'd have to get rid of the demon posessing the corpse before attempting a "cure" which would more correctly be called a resurrection) rather than someone infected with a weird virus. The other popular version, a non-human race born that way, it's not really applicable.
Comment: Is killing the vampire to free his soul considered curing them? Because that's what Dracula was about.
Answer: It's possible the first cure for vampirism appeared in the soap opera Dark Shadows.
Episode 467, broadcast on April 9, 1968 has a doctor reveal to a vampire, Barnabas Collins, that Barnabas has been cured. The method is underwhelming in that the doctor isn't even sure how he did it, just that he used experimental drugs and that they noted which drugs so it can be reproduced.
Scene where the cure is revealed.
It does seem a bit hard to swallow that the first cure was as late as 1968, but it seems that in most previous stories where a cure was even sought, it ended in failure.
We see this search and failure in the penny dreadful Varney the Vampire (1845–1847), the film Dracula's Daughter (1936), and the film House of Dracula (1945).
It's hard to be certain without reviewing the entirety of vampire fiction, but this seems to be the conclusion drawn by https://www.vampires.com/search-for-a-cure/
I can't really say whether a random website is authoritative on the matter, but they've at least reviewed a large breadth of vampire media.
In addition I've looked through The Vampire in Science Fiction Film and Literature in Google Books for instances of the word "cure" with the similar results. Again, I can't really vouch that this is a particularly scholarly and properly researched source.
Answer: Depends on what counts as "cured". The following is cribbed from my answer to the question What was the first story to depict a vampire who does not feed on humans?.
1956: "She Only Goes Out at Night . . ." by William Tenn, first published in Fantastic Universe, October 1956, available at the Internet Archive, is about a young lady vampire named Tatiana
Latianu . . .

He came back about eleven-thirty, looking as old as his father. I was right, all right. When he'd wakened Tatiana and asked her straight, she'd broken down and wept a couple of buckets-full. Yes, she was a vampire, but she'd only got the urge a couple of months ago. She'd fought it until her mind began to crack. Then she'd found that she could make herself invisible, when the craving hit her. She'd only touched kids, because she was afraid of grown-ups—they might wake up and be able to catch her. But she'd kind of worked on a lot of kids at one time, so that no one kid would lose too much blood. Only the craving had been getting stronger . . .

. . . and the country doctor who cures her:

The only thing none of us counted on was Doc. Not enough, that is.Once he'd been introduced to Tatiana and heard her story, his shoulders straightened and the lights came back on in his eyes. The sick children would be all right now. That was most important. And as for Tatiana—"Nonsense," he told her. "Vampirism might have been an incurable disease in the fifteenth century, but I'm sure it can be handled in the twentieth. First, this nocturnal living points to a possible allergy involving sunlight and perhaps a touch of photophobia. You'll wear tinted glasses for a bit, my girl, and we'll see what we can do with hormone injections. The need for consuming blood, however, presents a somewhat greater problem."
But he solved it:
They make blood in a dehydrated, crystalline form these days. So every night before Mrs. Steven Judd goes to sleep, she shakes some powder into a tall glass of water, drops in an ice-cube or two and has her daily blood toddy. Far as I know, she and her husband are living happily ever after.
Comment: Nice answer, but as you say, it depends on what you consider "cured". Since she's just drinking preserved blood, it isn't functionally different to the various fictional cases where vampires use blood banks as a damage-free source. (Although note the Charles Stross caveat for that - vampirism for his world is an infection by magical microorganisms, and drinking someone's blood creates a magical link which is what kills the "donor" slowly. His more ethical vampires prey on the terminally ill to minimise harm.)
Comment: Since it's fiction, I think this counts as even better than cured. The outline is "modern doctor is faced with real vampire, successfully treats it". A shot of penicillin as a cure would have been sloppy and boring, but easy to write. Making it into a manageable, treatable 3-part condition is cleverer and more realistic.
Comment: Managing the symptoms is an interesting angle. In that situation, I Am Legend by Richard Matheson (1954) has the entire society of vampires discover treatments, including allowing them to go out in the day and an alternative to drinking blood.

First work in which vampirism was cured


USB to Arduino to bluetooth

Question: I have a piece of equipment that has a USB output, that a flash drive gets inserted into, to collect data as the equipment completes its cycles. I need to make a device that can replace the Flash Drive and take the data from the USB port and either save onto an SD Card, to re transmit when completed, or to transmit live as the cycles complete.
I was thinking about using a UNO, & a USB HOST Shield. to receive the data. Then transmit with a wireless or Bluetooth shield.
I have a "SANDISK WIRELSS FLASH DRIVE" which works as a flashdrive, and then you can connect it to other devices through wireless transmission, problem with it is, the wireless feature disables when plugged into the usb port. If the wireless stayed connected, then this would be exactly what I needed.
Any help or ideas, sample schematics would be welcomed.
Comment: can you mod the device? if you can make your wifi card disconnected in-situ your problem is solved.
Answer: There once was once something called an "Infinite USB Memory Drive" that acted as a thumb drive that read and wrote data to a networked share over wifi.
They aren't selling those anymore, but their approach might be what you need - network share to resolve the issue of multi- device connected to same drive.
In your shoes, I'd be looking at a Raspberry Pi Zero W (the one with wireless) instead of an arduino.  More computing options means more ways to solve your problems with the same hardware.
Answer: Your plan is flawed.
First, you don't want a USB host - a flash drive is a device, not a host. This unspecified unit that you are connecting to is the host.
Secondly you would require an Arduino a lot more complex than an Uno. You need one that is capable of operating as a USB MSD (Mass Storage Device), and the Uno or any other '328p-based device cannot do that.
The only two kinds of Arduino-like board that I know of that can do that are Teensy and chipKIT.
On top of that is a simple fact that is the reason your existing wireless flash drive doesn't work for you: only one thing can access a "block device" at once. If you had the USB and the wireless both accessing the flash drive at once you would get massive data and filesystem corruption - so it is just disabled.  You need to have some manual way of switching off one point of access and switching on the other - and of course safely "unmounting" the device to ensure that all cached data is written to the disk properly.
Comment: I appreciate to input. That gives me a better avenue to search.

USB to Arduino to bluetooth


Visualising Android AudioTrack from a ByteStream

Question: I'm currently using the AudioTrack to play back recorded music. This involves writing the track data to the audio buffer. How would I go about using the audio data stream to draw a wave form on the screen that represents the audio being played in real time? I've not used any advanced graphics on android before I'm not sure how to get started. Do I have to use openGL or can I implements a View? and also how do I convert the data to something useful to use for drawing.
Answer: Your first step is to call setPositionNotificationPeriod(periodInFrames).  This determines how often your application will receive an onPeriodicNotification call.  So, if you're doing an oscilloscope-type visualization and you want to show 50 milliseconds worth of audio data on the screen at any given moment, you would use periodInFrames value of 2205 (assuming your WAV file is mono, 16 bits per sample and 44,100 Hz sample rate).
Inside your notification event, you can determine where the playback of your AudioTrack object currently is, and then get the corresponding slice of data from the original file or array.  You then use then draw this slice of data to your view using ordinary 2D graphics methods (no need for openGL here).  This answer has a C# sample for drawing a slice of audio data - easy to translate to java.
So your code will end up looking sort of like this:
audioTrack.setPositionNotificationPeriod(2205);
audioTrack.setPlaybackPositionUpdateListener(this);

...

public void onPeriodicNotification(AudioTrack track) {
    int pos = track.getNotificationMarkerPosition();
    short[] slice = Array.copy(_data, pos, _sliceSize) // pseudo-code
    // render the slice to the view
}

Comment: Thanks, this should get me started and moving in the right direction. I then just have to worry about how this will perform with multiple streams.
Comment: Yeah, that will increase your degree-of-difficulty pretty substantially. I'd recommend pre-mixing everything yourself into one stream, but that might not be possible for what you're doing.
Comment: I'm actually planning on running 4 audio tracks with visualisations and latency is going to play a massive factor when recording audio. Who knows how this will turn out but its a start. I might have to move to NDK 2.3 if its too slow.
Comment: @MusiGenesis +1 nice answer... How you found the value of 2205. How to calculate that ? What it should be in my case...http://stackoverflow.com/questions/9413998/live-audio-recording-and-playing-in-android
Answer: Concerning that point, I would suggest that you use android.media.audiofx.Visualizer class that handles the generation of waveform arrays or frequency arrays. So the only thing you have to take care of is the drawing of the graphics.
You can use this class since the release of API 9.


Can a blocking creature tap for its ability while blocking?

Question: I've declared my attack, and my opponent declares their blocker. I then use an instant that boosts and gives first strike to my attacking creature, then in the middle of combat they tap the creature to activate its ability. Can they do this?
Comment: Why is your opponent tapping his creature? Is there an ability on the creature that he is trying to activate? Or is he trying to get out of blocking your buffed creature?
Comment: @Malco The title did say "for its ability", so yes, the former. I edited to add that to the body too.
Comment: @Jefromi Just wanted some clarity, thought it might be something along the lines of "only untapped creatures can block, so if I tap my creature now I am not blocking anymore and it won't die."
Answer: Sure.  As long as the creature is able to block at the beginning of the declare blockers step (typically this means untapped) it can be declared as a blocker.  What happens after that doesn't affect it's status as a blocker.  There are also no restrictions on whether it can tap to use an ability, pay a cost, etc.
Comment: Re "*What happens after that doesn't affect it's status as a blocker*", Unless it's one of the things that removes the creature from combat.
Answer: Yes.
A blocking creature may be tapped to pay a cost or as the result of an effect.[1]
This has no effect on its status as a blocker.[2] It will still deal and receive damage as normal.

It can't be tapped for no reason.
The only way to stop a creature from being a blocker before the end of combat is to remove the creature from combat.

506.4. A permanent is removed from combat if it leaves the battlefield, if its controller changes, if it phases out, if an effect specifically removes it from combat, if it's a planeswalker that's being attacked and stops being a planeswalker, or if it's an attacking or blocking creature that regenerates (see rule 701.13) or stops being a creature. A creature that's removed from combat stops being an attacking, blocking, blocked, and/or unblocked creature. A planeswalker that's removed from combat stops being attacked.
Answer: Yes.
During combat, players each get priority to activate abilities or cast spells after attackers have been declared and after blockers have been declared.
The fact that a creature is blocking doesn't change anything.
Comment: Recommend including a disclaimer that this does not apply to abilities that can be activated only when the player could cast a sorcery.


Why calling setState method doesn't mutate the state immediately?

Question: Ok, i'll try and make this quick because it SHOULD be an easy fix...
I've read a bunch of similar questions, and the answer seems to be quite obvious. Nothing I would ever have to look up in the first place! But... I am having an error that I cannot fathom how to fix or why its happening.
As follows:
class NightlifeTypes extends Component {
constructor(props) {
    super(props);

    this.state = {
        barClubLounge: false,
        seeTheTown: true,
        eventsEntertainment: true,
        familyFriendlyOnly: false
    }
    this.handleOnChange = this.handleOnChange.bind(this);
}

handleOnChange = (event) => {
    if(event.target.className == "barClubLounge") {
        this.setState({barClubLounge: event.target.checked});
        console.log(event.target.checked)
        console.log(this.state.barClubLounge)
    }
}

render() {
    return (
        <input className="barClubLounge" type='checkbox' onChange={this.handleOnChange} checked={this.state.barClubLounge}/>
    )
}

More code surrounds this but this is where my problem lies. Should work, right?
I've also tried this:
handleOnChange = (event) => {
if(event.target.className == "barClubLounge") {
    this.setState({barClubLounge: !this.state.barClubLounge});
    console.log(event.target.checked)
    console.log(this.state.barClubLounge)
}

So I have those two console.log()'s, both should be the same. I'm literally setting the state to be the same as the event.target.checked in the line above it!
But it always returns the opposite of what it should.
Same goes for when I use !this.state.barClubLounge; If it starts false, on my first click it remains false, even though whether the checkbox is checked or not is based off of the state!!
It's a crazy paradox and I have no idea whats going on, please help!
Answer: Reason is setState is asynchronous, you can't expect the updated state value just after the setState, if you want to check the value use a callback method. Pass a method as callback that will be get executed after the setState complete its task.

Why setState is asynchronous ?

This is because setState alters the state and causes re rendering. This can be an expensive operation and making it synchronous might leave the browser unresponsive.
Thus the setState calls are asynchronous as well as batched for better UI experience and performance.
From Doc:

setState() does not immediately mutate this.state but creates a
  pending state transition. Accessing this.state after calling this
  method can potentially return the existing value. There is no
  guarantee of synchronous operation of calls to setState and calls may
  be batched for performance gains.

Using callback method with setState:
To check the updated state value just after the setState, use a callback method like this:
setState({ key: value }, () => {
     console.log('updated state value', this.state.key)
})

Check this:

class NightlifeTypes extends React.Component {
   constructor(props) {
      super(props);

      this.state = {
         barClubLounge: false,
         seeTheTown: true,
         eventsEntertainment: true,
         familyFriendlyOnly: false
      }
   }

   handleOnChange = (event) => {  // Arrow function binds `this`
      let value = event.target.checked;

      if(event.target.className == "barClubLounge") {

         this.setState({ barClubLounge: value}, () => {  //here
             console.log(value);
             console.log(this.state.barClubLounge);
             //both will print same value
         });

      }
   }

   render() {
      return (
          <input className="barClubLounge" type='checkbox' onChange={this.handleOnChange} checked={this.state.barClubLounge}/>
      )
   }
}

ReactDOM.render(<NightlifeTypes/>, document.getElementById('app'))

Answer: Since setState is a async function. That means after calling setState state variable does not immediately change. So if you want to perform other actions immediately after changing the state you should use callback method of setstate inside your setState update function.
handleOnChange = (event) => {
     let inputState = event.target.checked;
      if(event.target.className == "barClubLounge") {
         this.setState({ barClubLounge: inputState}, () => {  //here
             console.log(this.state.barClubLounge);
             //here you can call other functions which use this state
             variable //
         });
      }
   }

Answer: This is by-design due to performance considerations. setState in React is a function guaranteed to re-render Component, which is a costly CPU process. As such, its designers wanted to optimize by gathering multiple rendering actions into one, hence setState is asynchronous.


What to do when we can't trust our human classifiers?

Question: Suppose we want to design a neural network that can diagnose skin cancer. We want this neural network to consider the possibility that the doctor we hired misclassified some of our images while labeling. How can we design our neural network?
Answer: First, neural networks are good in dealing with"label noise". I'm currently on mobile/vacation, so remind me to search the paper on Friday.
Second, the more important question is how to get a good ground truth. Without a good ground truth you can't evaluate your models, no matter how good they might be.
I see the ways:
(1) have multiple experts label the stuff. Then you can make the ground truth a probability, not a simple label. If 9 experts say it is cancer and 1 says it is not, you would label it with 90%
(2) wait. If you can access the patients data, it will likely be more obvious in a year (especially if it was not treated)
(3) other diagnostic methods: I'm not a medical doctor, but I'm pretty sure there are invasive methods to diagnose cancer which are reliable
Answer: I have the following solutions:

If you have abundant data you can shuffle them and make validation and training data. After that, your neural network should exploit generalization techniques not to overfit the training data. By doing so, you may have relatively acceptable performance which works in noisy situations.
The other technique is evaluating Bayes error. This does not have any relation to the neural nets. It just tries to investigate in the feature space of the problem what percentage of your data is misleading, having same input patterns with contradictory labels.
Another approach can be using an existing model for validating the data-set.


Electrical: Can a neutral wire just be capped?

Question: In redoing my main level I have misplaced my electrical diagram showing what was there before I started (so annoying!). Now I am unsure if my new solution is safe or not. I have seen diagrams which confirm, but would greatly appreciate an expert helping me out. So here's what was there before (see my horrible diagram attached):
- power to the fixture
- three wire to a switch (not a two way or three way switch)
- power continues to an outlet
I want to remove the outlet part. But this makes the three wire neutral "stranded". I have tested the wire and there is nothing there (perhaps that is obvious to most!). The second picture attached shows the current setup. So the question is: Is this safe?
Answer: Yes. If you're getting rid of the receptacle, you should cap off the grounded (neutral) conductor in the switch box.
Just make sure there's no exposed wire sticking out of the connector, and you use a twist-on wire connector rated for a single wire.
Answer: Yes, Just cap off the neutral wire aka "grounded conductor". Like the other Chris said, do not connect to the bare ground just leave as is with a wire nut on the end to keep it insulated.
Answer: If the neutral was for an outlet that you're removing from the system, then so long as other outlets in the circuit have a properly grounded neutral wire, everything should be all set.  Be sure to cap off the stranded neutral wire.
Keep in mind that wiring isn't always what you expect, so verify that each wire is indeed hot or neutral as expected.  Code in the US dictates that black is hot, white is neutral and bare copper is ground.

Neutral: The white wire is called the neutral wire. It provides the return path for the current provided by the hot wire. The neutral wire is connected to an earth ground.

See this guide for more info
http://www.dummies.com/how-to/content/alternating-current-in-electronics-
Comment: You should think about editing your answer so that it is more accurate and less misleading. As it stands now there is the direct implication that the neutral wire should be connected to the earth ground wire in the switch or outlet electrical box. This implication is wrong and would be completely against the electrical code requirements.
Comment: Updated.  I can see how that might have been misleading.


Constrained combinations of marbles in jars

Question: I'm fairly new to combinations so I apologize upfront that I don't even know where to start with this one. My problem is of the form:
If I have $N$ marbles, and $M$ jars that can only fit a max of $Z$ marbles each, for how many combinations may I distribute all marbles amongst the jars? (Assume $N$ is between $1$ and $M \cdot Z$, such that you are guaranteed to have at least one jar to place a marble.) Can someone please provide some insight, or point me in the right direction?
Answer: Generating Function Approach
The coefficient of $x^N$ in
$$
\left(1+x+x^2+\cdots+x^Z\right)^M=\left(\frac{1-x^{Z+1}}{1-x}\right)^M
$$
is the number of ways to pick $x^0$, $x^1$, $x^2$, ... , $x^Z$ from each of the $M$ factors so that the exponents sum to $N$. This corresponds to putting $N$ marbles into $M$ bins with a maximum of $Z$ marbles per bin.
Thus, the number of ways to put $N$ marbles into $M$ bins with no more than $Z$ marbles per bin is
$$
\left[x^N\right]\left(\frac{1-x^{Z+1}}{1-x}\right)^M
$$
Applying the Binomial Theorem, we get
$$
\begin{align}
\left[x^N\right]\left(\frac{1-x^{Z+1}}{1-x}\right)^M
&=\left[x^N\right]\left[\sum_{k=0}^M(-1)^k\binom{M}{k}x^{k(Z+1)}\sum_{j=0}^\infty(-1)^j\binom{-M}{j}x^j\right]\\
&=\bbox[5px,border:2px solid #C0A000]{\sum_{k=0}^M(-1)^k\binom{M}{k}\binom{N+M-1-k(Z+1)}{N-k(Z+1)}}
\end{align}
$$
Note that when $Z\ge N$, so that all the marbles can be fit into any jar, the formula above reduces to $\binom{N+M-1}{N}$, which is the same number as given by the "stars and bars" method.
Answer: You need to use the multinomial coefficient:
$$\frac{N!}{k_1!k_2! \ldots k_M!},$$
where $k_i$ denotes the number of marbles put in the $i$-th jar.
The number you are looking for is:
$$\begin{array}{c}
\displaystyle\displaystyle\sum_{k_1 = 0}^Z \sum_{k_2 = 0}^Z \ldots \sum_{k_M = 0}^Z\\
\small{\text{s.t.}~k_1 + k_2 + \ldots k_M = N}\end{array} \frac{N!}{k_1!k_2! \ldots k_M!},$$
with the assumption that each combination of $k_1$, $k_2, \ldots, k_M$ sums up to $N$


Minimum number of components in graph

Question: Minimum number of components in graph where we have 69 vertices and 43 edges. I think the best way is to create a path? One path and the rest would be isolated components. Since in path we use only one edge for 2 vertices. Correct?
Answer: Yes, this is correct.
The easiest way to get to the result is to look at a graph with $v$ vertices and $0$ edges. It obviously has $v$ components. Adding an edge between two vertices $a$ and $b$ reduces the number of components if $a$ was not reachable from $b$ before adding the new edge.
Concerning  "best" solution: Your approach yields an optimal solution, though it is only one way to do it - there are other ways (e.g. 26 components with either 2 or 3 vertices) that work equally well.
Answer: It looks like you are on the right track.
Let us use a bit of arithmetic or counting to find the minimum number of components.
Suppose that graph has $k$ connected components that have $v_1, v_2, \cdots, v_k$ vertices and $e_1, e_2, \cdots, e_k$ edges, respectively. Then we know $v_1 + v_2 + \cdots + v_k = 69$ and $e_1+e_2+\cdots+e_k=43$.
What is the relation between the number of vertices and the number of edges in each connected component? I would assume you have learned the fact that the latter must be at least the former minus one. If you have not learned it, it can be proved easily by mathematical induction on the number of vertices.

The base case when there is only one vertex is clear.
For the induction step where you add a new vertex, at least one new edge is needed to connect the new vertex to the existing vertices.

So $e_i\ge v_i-1$ for all $i$. If you sum all such inequalities, you will get $43\ge 69-k$, which means $k\ge26$.
I will let you check when the equality $k=26$ holds. Hint, it holds if and only if all inequalities are equalities. What does an equality here mean?

Minimum number of components in graph


Is it common for a thesis-based master hasn't got the thesis topic?

Question: The supervisor just tells students to learn knowledge and practice skills that will be required in their research in the first year but doesn't have their topic decided. Is such case common?
Answer: This can happen. The formal thesis topic is quite an open field.
Perhaps, the supervisor doesn't want to spoon-feed and force a topic on to you, but rather would want to have you come up with one. By exposing you to various tools and techniques, and (presumably) side-by-side pointing to some useful literature, and asking you to dig for more works yourself, he/she is essentially doing a good job. Basically, what he's saying is - this is what happens usually in the field, tell me if you come up with any ideas regarding what you want to do. He/she is basically throwing the field open to you.
Since your master's is thesis based, I won't find it very surprising though if he/she has already planned something as a backup option, in case you don't come up with anything reasonable. But he/she will likely not reveal that unless he/she is absolutely convinced that you are struggling to come up with anything doable.
In my experience, this approach is not very common, but not very rare either. And of course, it is a good approach, the goal of course is to train you to think.
Hope that helps. :)


How to correct yellowish pictures under fluorescent lighting on a digital camera?

Question: What can I do about some of my pictures turning yellowish under fluorescent lights - I can not turn the lights off - judo lessons for my grandson in school gym. I am using the Canon Rebel XSI (450D) digital camera. I take all of my pictures on the auto setting, I don't have very good vision. If I am close pictures are fine but ones from any distance are yellow.
Comment: Are you using flash?
Comment: Interesting how this problem seems to be the very common at martial arts places.  The question that I linked was martial arts, yours is martial arts and my first personal experience with this was at my wife's martial arts gym.
Comment: possible duplicate of [Do fluorescent lighting and shutter speed create a problem with color cast?](http://photo.stackexchange.com/questions/4115/do-fluorescent-lighting-and-shutter-speed-create-a-problem-with-color-cast)
Comment: @AJHenderson :) does fast martial arts movements cause light to shift to orange? :)
Comment: @woliveirajr - no, but it does increase the shutter speed you need to use.  The problem is actually that certain types of florescent lights don't emit a consistent color through the power cycle.  At times of lower voltage, they can turn yellow.  What you are actually seeing with the yellow tint is that the camera happened to take the picture while the power was at the low point in the cycle, so the room was actually that color at the moment.
Comment: If you extend your shutter speed to be at least the length of a power cycle (either 1/60 or 1/50 depending on where in the world you are) then you should reliably capture an entire power cycle, (and thus white light), but anything faster than that may end up not getting white light to capture the image by (and thus a yellow bar or general yellow tint).
Comment: It may be that other locations use multiple phases of power for their lighting (which would mean that a faster shutter could be used as one of the lights will give white signal during the time the frame is open, since each phase of power is offset by 1/3 of the wavelength of the power).  You can read more on power phases by searching for 3 phase power.
Comment: @AJHenderson I was just kidding on the coincidence, found it curious because many other pictures are taken under fluorescent light, but in general don´t have the white kimono (? is it the name of those special clothes?) to help setting the white.
Comment: Take a look at this: https://photo.stackexchange.com/questions/61493/color-issue-studio-images-have-a-pink-hue/61497#61497
Answer: Your camera has the ability to manually set the white balance. Under most circumstances, the auto white balance feature works fine, but since it's very likely you're shooting JPEG, correction after the fact when it does get it wrong is probably going to degrade your image quality a bit. at any rate, page 90 of the English version of the manual for your camera is the instructions for setting white balance. This is needed because, while the white under the lights appears white to us, it really isn't and so the camera needs to have some idea how far from that condition it is and sometimes you have to tell it.
You may  also need to learn to shoot in a different mode (such as shutter priority) to control this, I'm not entirely sure on that front (I shoot Nikon, so it's different) reading what the manual has to offer. In any event, I actually recommend you learn to shoot with shutter priority for sports, better control, less likely to have blurry pictures. You can always switch back to auto for other purposes.
One thing to remember... if you change the setting, don't forget to reset it to auto again when done!
Answer: On auto, the camera tries to automatically adjust the overall tone of the picture. I would expect different sources of the problem:

the fluorescent light sources are not all the same, then having mixture of light sources causes troubles
at large distance photos, there is some dominant color, which tricks the camera into wrong choice of white balance

Regardless the cause, using a photo editor it is possible to correct the white balance. Especially at judo, people should have some white stuff on them, which could be used as reference white spot.
Answer: There are several things you can do, but almost all of them require taking the camera off of full auto. If you use the P (Program mode) setting, thw camera will still select the aperture and shutter speed for you, but you will be able to select more options for other things, especially White Balance.
If you are not allowed to use flash at the gym:

Set the White Balance (WB) setting in the camera to White florescent lighting. If that doesn't work, use a specific color temperature. Most gymnasium lighting is in the 3500K to 4300K range. You might also need to use White Balance Correction. How to set a White Balance Correction is covered on page 92-93 of the XSi Instruction Manual. You can also set a Custom WB by taking a picture of a white or grey card and telling the camera to use that reference shot for setting the color balance. How to set a custom WB is explained on pages 90-91 of the XSi Instruction Manual.
Set the Image Recording Quality to RAW. This requires that you post process the pictures on your computer, but gives you much more latitude in adjusting the color and WB after the shot has been taken. You can use Canon's Digital Photo Professional (DPP) that was included on the software disc that came with your camera. There is even a tool in DPP that allows you to click on a white object and the WB will be adjusted so that object appears white.

If you are allowed to use flash at the gym:

The reason the closer shots look fine is the flash has enough power at that distance to overpower the gym lights. When the flash is turned on the camera normally adjusts the WB for the color of the light emitted by the flash. The reason the distance pictures look yellow is because the flash doesn't have enough power to reach that far and even though the camera sets the WB to the color of the flash, the light in the pictures is coming from the gym's lighting that is a different color.
For longer distance shots your flash will be more effective if you increase the camera's ISO setting. Just be aware that the higher ISO you use, the 'grainier' your pictures may look. How to adjust your camera's ISO setting is covered on page 57 of the XSi Instruction Manual.
An external flash will be much more powerful than the one built into your camera. This will allow you to use the flash from a greater distance. The Canon 430EXII would probably be more than enough for your needs. It can be set to use Canon's automatic flash system known as E-TTL. You can also buy a third party flash that is E-TTL compatible, such as the YongNuo YN-468 II E-TTL Speedlite for about 1/3 the price of the 430EX II.


Instruments that are easy on left pinkie fingers

Question: I am interested in learning an instrument but have limited mobility in my left pinkie/little finger due to an old injury. What instruments are least demanding on that finger?
Comment: Not sure this has an answer. You could say guitar (many people don't use the little finger) or piano (as long as you don't use it) or violin etc
Comment: Playing a left-handed guitar might be a possibility, since with normal guitar, the right pinkie is used very little. Not sure if left-handed violin is a possibility either, but I suspect that you don't need great mobility in the pinkie for using the bow.
Comment: @DrMayhem - Piano would be one of the worst possible instruments for a non-functional left pinkie finger.
Comment: @Old John: I know a (right-handed) guitarist who plays left-handed guitar for precisely this reason. I don't think this is such a good idea on string instruments, though: at least on cello the pinky is very important for giving stability and impetus to the bow stroke. (Also, alas, playing left-handed is much more limiting for a string player than it is for a guitarist: you constantly clash with your neighbours in the orchestra...)
Comment: Violin and viola can be adapted to omitting the left pinkie, but cello and double bass wouldn't be such a good idea.
Answer: The obvious ones are -trumpet - no use for pinkie at all, and slide trombone - no use for fingers separately. Followed closely by xylophone/marimba/glock/vibraphone and drums.
Comment: Normally you do not have to operate the spit valves while actually playing. Typically, using it between songs is more than sufficient.
Comment: None of the brass uses the left pinky, aside from horns with a 4th finger stopping valve, which are rare.
Comment: @MattPutnam The horn may NOT be "easy of the left pinkie" depending on the design of the hand-hold: it may use the pinkie for a significant part of the support. (source: 10 years playing the horn)
Comment: Also most brass instruments have a spit-valve that can be operated by the pinkie, though you don't use that all the time, so it might not matter.
Comment: @Brondahl (1) supporting the instrument doesn't require mobility, and (2) you can use a duck's foot and remove all pressure on it entirely.
Answer: Really? Not one mention of Django Reinhardt?
Django lost the use of his fourth and fifth fingers in a fire but that didn't stop him from ripping it up on guitar (and inventing a new genre of music in the process).

A few years ago, I sustained a minor sprain in my fretting hand which made it painful to make certain finger changes (I play bass mostly, but guitar as well). I adapted around that, and I didn't let it stop me from playing.

You can play any instrument you want to! Work around any problems or weaknesses, don't let them stop you completely. And don't let anybody tell you can't because of a disability.
Comment: And Django Reinhardt inspired [Tony Iommi](https://en.wikipedia.org/wiki/Tony_Iommi), who was the first person that came to mind when I read your answer.
Comment: I'm sure I could make anything work, but I don't have my heart set on any particular instrument so I figured it would be best to try to set myself up for success. The 'let nothing hold you back' attitude is admirable, and I know you mean well by it, but I have to work around problems with my pinkie in enough other aspects of my life as it is. I just want to learn for fun and it's much more fun when I don't have to worry about it.
Comment: No discussion of musicians' "let nothing hold you back" attitude would be complete without a mention of [Def Leppard's one-armed drummer Rick Allen](https://en.wikipedia.org/wiki/Rick_Allen_(drummer)) - but of course as the asker says, it's easier to *not* have to overcome adversity if you can avoid it...
Answer: Harmonica, blues or chromatic.
Answer: The typical recorder fits the bill perfectly for you.  It uses all fingers of the right hand, and the thumb and all-but-the-pinkie of the left hand.
Answer: Pretty much all percussion instruments, which includes hammer dulcimer and its ilk.  I don't know whether the harp requires pinky use or not.
I suppose it'd be cheating to suggest the theremin :-) .
Comment: Harp does not use the pinkies, they're not long enough to reach the strings when playing with proper technique.
Comment: Strangely enough, I think most (of the few) proper theremin players _do_ use the right hand fingers, including the pinky, to intonate scales.
Answer: The didgeridoo asks very little of the pinkie finger on either hand.
Answer: The harp.  It's normal to play just using the thumb and first three fingers on each hand.
Answer: As you can see by all the answers so far, you have many options.  One thing that I might point out is that many of the instruments mentioned (horns, harmonica, brass) require you to blow into them.  So they don't work as well if you want to use your instrument to accompany your singing.
You can learn to play guitar "left handed" which means that your right hand would be your fretting hand.  Whether you use a pick or play fingerstyle, very few guitarist use the pinkie of their picking hand for playing.  Learning guitar requires learning to contort your fretting hand into unnatural positions.  Those positions are equally alien to either hand so it might be just as easy to learn from the beginning - to play either way.  In fact many folks who are left handed will learn to play guitar right handed due to the fact that "right handed" guitars are more widely available.  In your case you have a compelling reason to learn to play guitar "left handed".
For an in depth discussion of this concept - see this question on Music SE Left hander learn to play guitar right handed
Another option for a stringed instrument that will work great for accompanying a singer in many musical styles and won't require the use of your pinkie whether playing left or right handed - is a three string cigar box guitar.  These guitars can be homemade or you can buy one commercially produced.  Many of these actually sound amazing and you can even add electronics and play them through an amplifier.  Because of the way they can be tuned, you can actually play thousands of songs with only one finger.  Here is a video on YouTube that discusses a common tuning method and shows how to play with only one finger.
Learn to Play 3 string cigar box guitar
This video also demonstrates how full and resonate a good cigar box guitar can sound (watch other videos for even better sounding cigar box guitars).
If you don't like the idea of a cigar box guitar, you can use the 3 string tuning concept to play a six string guitar with only three strings installed.  Again depending on your tuning, you could play such a guitar with just one finger and use a barre to form entire chords.  Or a finger tip covering two strings to form power chords.
Here is a picture of a professional performer named Chris Ballew (stage name Caspar Babypants) who takes all but two or three strings off all of his stringed instruments (many shown in picture) so he can play them with one or two fingers.

If you are not worried about singing along while playing your instrument, and you want an instrument that is easy to learn, portable and inexpensive, You might want to consider a harmonica.
If you want to try a harmonica, you can buy cheap ones for around $5.00 US.  But if you are serious about trying to learn harmonica, you should get a quality instrument because it will be easier to learn to bend notes on a good quality harmonica.  I like the ten hole diatonic (commonly referred to as "blues harp") harmonica which can be purchased in all 12 keys (and more).
Most professional harp players keep a set so they can play songs in more than one key.  What I like about the diatonic (as opposed to the chromatic) is that the only notes available without bending are guaranteed to be in the correct key, so even if you make a mistake, it's not quite as obvious.
No matter what instrument (or instruments) you decide to learn to play, have fun!
Comment: I do know that many guitarist who play fingerstyle use their pinkie to stabilize their picking hand on the top of the guitar, but the OP said he had "limited mobility"  of the pinkie (not zero use) so he may still be able to do this if fingerpicking was a desired style of play.
Answer: Rockin Cowboy alluded to this, but I'll spell it out separately:
Your own voice!
Singing is a skill that can be developed like playing any instrument, and it's arguably one of the cheapest and most convenient options. You almost never have to tune, polish, or repair the instrument. You don't have to carry a bulky case with you to practice.
That's not to say that you will necessarily find it easy. This article talks with Joanne Rutkowski, professor of music education. I quote,

"Everyone who can speak can learn to use a singing voice, but not everyone will have a fabulous-sounding voice. Once the vocal instrument is working, the ability to sing with good intonation is dependent on musical aptitude and musical achievement."

You will definitely benefit from vocal training. Don't think you can skip taking lessons just because you don't need to buy an expensive instrument. https://www.vocaladvancement.com/can-anyone-learn-to-sing/
I think it's worth a try. You may not ever (or may!) become one of the greatest, but you should be able to get decent results.
Comment: Although I see this as post as OK you may want to add more to it so it is more of answer and less of a comment on another answer.
Answer: The ukulele is great for avoiding pinkie finger use. Also melodica, accordion, banjo…
Comment: Banjo often uses the left pinkie.
Answer: Flute makes no or very little use of the left-hand little finger.
If you play the standard Boehm system there are no keys for the LH 4th finger and even if you play one of the extended systems there are alternative fingerings. Rudall Carte and other 'simple' systems don't use that finger at all.
Also fife and tin whistle, both of those don't use that finger at all.


My sister and I keep switching skins in minecraft!

Question: My sister and I play minecraft(bedrock edition, version 1.14.60) together using LAN(local are network). but whenever we join each other's worlds, we swap skins! We are both using the custom skin, where we both make our own skins, but we always show up with each other's skins. What do I do? Can this be fixed?
Answer: This is an unresolved bug.
Please consider researching the topic a little more before you post here. In this case, you can find the linked page on the 1st result by searching minecraft bedrock skins switching on any simple search engine.
I understand that you did do some more research on the topic, and that you have found items talking about the bug but not fixing it. In that case, I would just like to confirm that no, there is no current fix, I'm afraid. You'll just have to go on without it. I'm terribly sorry about this bug.
Comment: In the meantime, you'll just have to wait it out, and deal with what you've got. Too bad, I'm afraid! Hopefully the devteam will fix it soon.
Comment: yeah, but I have searched it and tried to fix it. It never works, not even restarting minecraft.
Comment: Well you can't fix it. You need to wait for the devteam on the backend to fix it. In the meantime, there's nothing you can do. If it's open and unresolved, chances are, nothing on your end will fix it!
Comment: ok. thanks for letting me know.


Blender 2.8 cycles on GPU compute just shows pink and purple noise?

Question: Any idea what's causing this? I'm set to render cycles with gpu compute. I have 2 geforce rtx 2080 ti's. All I'm getting is this pink and purple garbled mess..
Comment: If you downloaded the file from the internet, chances are that the HDRi Background image wasn't included. You can also switch from Cycles to Eevee and see if stuff starts showing up. If that's the case you probably have a GPU/driver related problem.
Comment: looks like GPU driver related problem.
Answer: Two things;
1: You used textures that you didn't pack into the file. That shade of magenta means the texture is missing. Find the textures and reload them.
2: You're going to need WAY more than 32 samples to get something usable. Start with 100, move up from there to get a more clear image.
Comment: You don't have to pack the textures into the file. This is only useful if you are going to transfer your work between computers. The textures - if there are any - just need to be at their specific locations.
Comment: Packed or not, blender doesn't have the data. Regardless of whether he wants to pack the textures in the future, he needs to find the textures and open them in the image texture node. I mentioned the packed file because I've only seen confusion like this when I file is moved to another system, and the user assumes the textures are packed in the file. Either way, the process is the same. Find them, load them.


How likely is it that photons will interact with electrons?

Question: Electrons are incredibly small, and the "size" of photons (or the distance away from one required to not interact with it) is also incredibly small.  Yet photons still interact with electrons all the time.  Are they attracted to each other in some way, does it have to do with the wave of the electron being larger than the supposed size of the electron, are they actually very unlikely to interact but there is simply an incredibly large amount of them, or a combination of these?  Or maybe something else?
Comment: Consider that the size of an optical photon (~$0.5\times 10^{-6}$ m) is much larger than the distance between atomic nuclei (so also larger than the distance between nearest neighbor electrons) in most solids.
Comment: And that photons with size similar to the atomic spacing (x-rays) are most known because they do penetrate solids without interacting (much) with them.
Comment: The "size" of a photon is given by the heisenberg uncertainty principle and it is a  locus of most probable interaction :Δ(x)Δ(p)>h/2π . where p momentum x space coordinate.
Answer: The idea that charged particles (like electrons) interact with electromagnetic radiation (consisting of photons) is a fundamental fact of nature. When an electrically charged particle is accellerated, it emits photons, and when a photon encounters an electrically charged particle, it can force it to accellerate. This is true even for single photons and single particles; the existence of the interaction does not rely fundamentally on the presence of large numbers of photons or charges.
Experimentally, the strength of this interaction is furnished by the so-called fine structure constant, which is a measure of how likely a given photon is to interact with a charge.
The detailed physical model of this interaction is called quantum electrodynamics, or QED. A good source book on QED for nonspecialists is Richard Feynman's "QED: The Strange Theory of Light And Matter".
Answer: Another view:
Electrons and photons are quantum mechanical entities and have to obey the Heisenberg uncertainty relation

The position and momentum of a particle cannot be simultaneously measured with arbitrarily high precision. There is a minimum for the product of the uncertainties of these two measurements. There is likewise a minimum for the product of the uncertainties of the energy and time.

This means that even though elementary particles are point particles, there is a locus where  they have a probability to manifest, dependent on their momentum. This probability is given by the solution of the wave equation the particle obeys, and is a wavefunction. The deBroglie  wavelength  generally summarizes the quantum mechanical nature of particles.

This is particularly appropriate for comparison with photon wavelengths since for the photon, pc=E and a 1 eV photon is seen immediately to have a wavelength of 1240 nm.

whereas for a one eV electron,

the associated DeBroglie wavelength is 1.23 nm, about a thousand times smaller than a 1 eV photon

Once the electron and the photon are in proximity where the wave nature dominates, the interaction will have  a measurable probability, dependent on the boundary values of the problem.


Survey data validation with inverted questions

Question: I have a survey using a Likert scale and two inverted questions out of twenty. Using R how can I identify (and maybe filter out) the respondents that always tick agree, for example, and thus did not read the questions properly? Thanks.
Answer: I would recommend inverting the scale for both questions. So afterwards, you have two new variables where the polarity is inverted (e.g. before 6 = bad now 1 = bad). Then you can simply iterate of the variables and watch for people who have always 1 or 2 etc....
Additionally, the inverted scales are much better to analyze afterwards - no headaches because 2 out of 10 variables have to be interpreted "ass backwards".
As an additional note: Finding such people is remarkably difficult. Maybe the people are ticking in diagonal patterns or semi-randomly?
There are programs out there who are generating random answers (e.g. ticking a random box out of 6 each time...) - so the question gets even more complicated.
So to filter out bad answers I would suggest your approach + look for the average answer time, candidates with very short times are prime candidates to sort out + check for very short open answers in unclear cases...
Comment: As a side note: If you really have inverted questions, I would not check if the people cliked agree - but if they checked "do not agree". If they clicked only without reading, they would never see that the question has a different meaning...
Answer: First, I preach caution in removing responses for this reason alone. It's possible to hold legitimately ambivalent attitudes on very many subjects, especially when negatively phrased items are not exact opposites of other, positively phrased items. For example, here are three items I work with a lot:

"I have a good sense of what makes my life meaningful."
"My life has no clear purpose."
"My life has a clear sense of purpose."
(Steger, Frazier, Oishi, & Kaler, 2006)

I wouldn't feel justified in excluding a person who rated both the first and second items Somewhat True; life can be meaningful without a clear purpose. Yet if a person rated both the second and third items Absolutely True, this might be reason to treat those two responses as missing. I still wouldn't throw out the other responses on this basis alone though. Where to draw the line between these two example cases is probably a judgment call (or at least a tough methodological question) that depends on many circumstances you haven't described, including your plans for the data after validating them.
I've used exclusion criteria in my work based on response invariance, but I set the bar somewhat lower so that only the worst cases failed to clear it. I.e., only completely invariant responding got a person's responses completely disregarded for a given measure with at least ten items that included at least one negatively phrased item or two subscales. This was easy enough to catch: I just analyzed a subset of my complete dataset that excluded participants with $SD=0$ across all items of a given measure. This might be a clumsy way of doing things, but it works – for example:
for(i in 1:length(MLQ[,1])){MLQ$MLQsd[i]=sd(as.numeric(MLQ[i,paste0('MLQ',1:10)]),na.rm=T)}

This line adds a MLQsd column to a data frame MLQ with ten columns named MLQ1–MLQ10. Each row's value for MLQsd is the $SD$ of that row's values for those other ten columns. Anyone who gives the same response to all ten items including the negatively phrased item has a MLQsd $=0$. Excluding them is then as simple as using subset(MLQ,MLQsd>0).
Depending on how your data are formatted, you can modify the above code to generate other criteria for subsetting. E.g., assign each row a binary code: 1 if all responses =='agree', and 0 otherwise. Then just subset(your data, your binary code==0) if you want to exclude them. If you do, it's probably wise to estimate and report the size of the difference between excluded and retained observation groups. I also second @ChristianSauer's recommendation to look at response time if you have that data. I collected data on Qualtrics, and used completion time as another exclusion criterion.
Reference
Steger, M. F., Frazier, P., Oishi, S., & Kaler, M. (2006). The meaning in life questionnaire: Assessing the presence of and search for meaning in life. Journal of Counseling Psychology, 53(1), 80–93.


How can I resize my chroma key mask?

Question: I am trying to place a green screen video on top of another video. My issue is everytime I add the Keying menu my background image turns black.
What could be causing this issue?

Answer: The size of the output image  is determined by the image connected to the background.

If you plug in a scaled (or differently sized) image as foreground, it will mess up the sizing of the composite.
In this case your setup is correct but the keying node is filling the frame with a value of black information (0) around the image, and a value of 1 in the alpha channel, creating full opacity.

The way to fix this is making the Black clip be a value that is not 0, so that the alpha channel becomes transparent to that value.
Changing the value to 0.00001 will make the transparency come back:
Answer: I believe the problem is that the alpha channel you are creating is the size of the image containing the dog, therefore that is the only portion of the image being overlayed. To solve your problem you need to have an alpha channel that is the same size as the background image.

How can I resize my chroma key mask?


How wide should be the most possible visual spectrum for an organic eye?

Question: A typical human eye reacts to waves with a length of 380 to 750 nanometers, which in frequency corresponds to the strip in the area of 400-790 THz.  At the same time, a portion of 380-400 nm is taken as a shortwave border, and 760-780 nm as long-wave - 760-780 Nm, but how wide there should be a maximum possible spectrum for an organic eye, so many birds and insects can perceive ultraviolet radiation, up to  360 nanometers, is it also possible with infrared radiation?
Comment: Snakes can "see" infrared light using their pit organs and ultraviolet with their eyes. I'd guess it's hard for a single "eye" to have such a wide range and more efficient to evolve separate "eyes" for different ranges. It's also unlikely an infrared eye could perceive much detail so not worth evolving the complex eyeball structure.
Comment: For a hard science tag, I would posit that the purpose and nature of the vision should be specified. Is the 'vision' strictly for localization? Identification by 'heat signature' and 'temperature' the way some eyes see 'Red=bad, RUN"? Humans already have sensors that can detect heat (IR radiation on the skin).
Comment: It is in mind that common eyesight with the eye
Comment: The limits of the wavelengths which can produce visual sensations are not sharp cutoffs, but rather depend on the power of the signal and on the specific conditions of observations. Especially at the long wavelength end, the limit of the visual spectrum is really just an exponential decay. Most people can perceive visually sufficiently strong radiation to 700 nm in the right conditions; and there are reliable reports that even radiation at 1000 nm can be perceived visually in special conditions if extremely powerful, such as laser pulses.
Comment: Related: https://worldbuilding.stackexchange.com/questions/202958/extending-the-visual-spectrum-how-far-is-possible-and-what-would-be-the-benef
Answer: Check out the Mantis Shrimp - https://en.wikipedia.org/wiki/Mantis_shrimp#Eyes
"They are thought to have the most complex eyes in the animal kingdom and have the most complex visual system ever discovered."
"Mantis shrimp can perceive wavelengths of light ranging from deep ultraviolet (UVB) to far-red (300 to 720 nm) and polarized light."
Answer: Humans actually can see ultraviolet - with the right sort of injury.  Post-cataract patients have reported seeing ultraviolet light, and this was reportedly used to a small extent in World War II for signalling.
The reason for this is that the human lens ordinarily contains a pigment 3-hydroxykynurenine, similar to some of the pigments used by butterflies, which blocks UV light, one might suppose to protect the retina.
Infrared vision is substantially harder to do with an "eye" in the narrow sense.  The problem is summed up in this busy curve, which shows that one inch of water in the eye absorbs as much infrared as up to 1 million inches of water would absorb of red.  Now as you look at that, you can see you can go some way out into the infrared before you get to the peak, but with diminishing returns.  Insects have some advantage because their small compound eyes have shorter distances for the light to travel.
Eventually, water becomes clearer again, and terahertz can penetrate some distance.  Refracting terahertz can be done, but the sort of metamaterials used are different from our normal notion of a lens.  It is conceivable to picture such complex elements evolving and being worked in somewhere near the border of cornea and sclera, but per your intent this probably doesn't count as an option.
Comment: Near infrared seems plausible for humans: https://www.njoyvision.com/blog/can-humans-see-near-infrared/ Last I heard though, this experiment was a failure.
Comment: I ruled out air based on the "typical human eye" and "common eyesight with the eye" phrases.  There *could* be an air bubble inside the eye, presumably in place of the vitreous humor only, but the water-air interface would refract light  Engineering the back of the lens to form a fine focus and a lump-free surface over the blood vessels of the retina would make this a *vastly* more difficult exercise in genetic engineering than removing the lens or eating some vitamin A2.
Comment: Very interesting story - I should modify my answer, but I haven't gotten to the bottom of it yet.  The long and short of it is one more double bond in the retinal, which should make it absorb longer light.  There are *many* vitamers of vitamin A, but this one isn't even usually considered with them, but ...
Comment: Is there any physical *law*, or very very strong biological rule, why an eye **has** to be filled with a watery fluid, and thus opaque to InfraRed? Structurally and optically an air-filled eye would work almost as well, be much lighter, and just a bit more fragile. I suspect the only reason we all have fluid-filled eyes is because we evolved them while still being water creatures, and gas-bubbles under water are rather inconvenient.
Answer: Eyes depend on illumination.
The sun delivers the most intensity at the point where usual eyes have their absorption curves at. To prove non-coincidence here is hard, because our sun is at a size/temp amenable to Life itself, and the spectrum directly depends on the temperature -- life, and eyes with it, evolved under this sun, and evolution took the path of lowest resistance using the wavelengths that are most plentiful.
Of course the sun is not the only source of light, just, by orders of magnitude, the most powerful. Any body emits radiation according to it's temperature and then there is chemical luminescence as well, all filtered by the absorption of the media between source/reflector and the receptors (in case of chemoluminescence of a fish and a human observer, that might be the skin of the creature producing the light, surrounding water, plastic of the diving mask, air in the diving mask, the eyes themselves).
Now, to actually detect that radiation, we need something to interfere with it. And to form a picture in our mind ('seeing') we need spatial resolution. BUT : humans posess, for instance, facilities to detect FIR, with some spatial resolution: Close your eyes and have a friend walk around you with a 200°C hotplate, and you will be able to tell, at least, whether the plate is in front,left, of right, up or down, making your forehead a 3x3 resolution `(9 pixel) 'eye', with no color vision (you could not discern a 200°C hotplate at 1 meter from a 30°C hand at 1cm or a 10 000°C plasmaplume at 20m). This reception does depend on heating, as do the pit organs of snakes, and microbolometer thermal cameras. Turning this black-and-white camera into a color version (able to discern different temperatures, not just heat flow) would only take, for instance, a filter wheel. Upping the resolution would take, for instance, shielding (you could place many aluminium-foil rolls on your forehead, pointing in different directions, or have one roll scanning a Z-path from left to right and up-down).
So whether you build your picture by scanning the directionality function of your single receptor, or have an array of receptors with differing directionalities is simply a question of how fast you need your picture (is this about seeing a moving swallow, or knowing where a burning tree is?). How you get from the radiation to the nerve impulse is also up for grabs: Have the radiation heat something, detect the heat, have the radiation exite some molecule, detect the shift in chemical reactivity, have the radiation break a bond, detect the chemical change, etc.
There are some laws to adhere to, e.g. the best resolving parts of the human eye are quite near to the diffraction limit, i.e. the receptor that resolves one pixel is about the size (10^-6 m) of the received wavelenght (0.5*10^-6 m) - but what space-angle that pixel represents is very much up to the optics - if we, for instance, had eyeballs 1.4 times bigger (double the retinal area) we could double our spatial resolution (if we kept the receptor density equal).
Many wavelengths of radiation are not easily manipulated (refracted, reflected) making optics harder, but not impossible
Living tissues can produce both crystals, and metamaterials, Nature is the current MVP in that area, actually. The reason we do not have XRay vision is that there is no natural xray source around (if there were, life as we know it would not have happened). Most wavelenghts of radioactivity are not useful for imaging in a biological setting either: The background radiation of a mountain range is too disperse to give a nice picture of anything standing on front of it, and there are too few hotspots to make 'radioactivity-vision' evolutionaryly vital in avoiding them. Same for hard-UV : were it plentiful, we'd not be here, seeing the short bursts that randomly filter through to us does not pose an evolutionary relevant task.
tl;dr: If your creature has sufficient illumination in a given wavelength, and there is no easier way to get at the information, nature will have found a way to see in that spectrum. (Why the creature is not dead in the 'sufficient illumination' is another Q)


Evaluating commutator of $[\operatorname{sign}(X),\, \operatorname{sign}(P)]$

Question: I wish to evaluate the following commutator: $[\operatorname{sign}(X),\, \operatorname{sign}(P)]$.
Is there a general method for evaluating $[\operatorname{f}(X), \operatorname{f}(P)]$? I thought of a Taylor expansion but $\operatorname{sign}(x)$ is discontinuous on $x=0$.
How would you evaluate this commutator?
Comment: How are you defining the sign of an operator?
Comment: @doetoe As the function $sgn$ is Borel measurable, the sign of a self-adjoint operator $A$ is an operator defined in terms of its spectral measure adopting the standard procedure of Borel functional calculus: $$sgn(A)= \int_{\sigma(A)} sgn(\lambda) dP^{(A)}(\lambda)\:.$$  Since $A$ is self-adjoint and thus densely-defined and closed, it is possible to prove that it is the partial isometry obtained from the *polar decomposition* of $A$: $$A =J |A|$$ where, in fact, $J = sgn(A)$.
Comment: @BrandonEnright I don't think so: we're meant to interpret it as finding a commutator given the QM CCR, not how to do it for a general commutation - at least I think so.
Comment: This seems better suited for Math.SE.
Comment: Thanks VM9, I found a concise reference at http://planetmath.org/spectralmeasure. I guess this also points at an answer to the OP's question: you could evaluate the commutator $[f(A),f(B)]$ loosely speaking by integrating weighted commutators of projection operators over the spectra of $A$ and $B$.
Comment: @doetoe Actually there are several problems with domains of operators when $f$ is not bounded, so computing that commutator is not so easy as it could seem. However it is not the case for the function $sgn$.
Answer: I) The CCR reads
$$\tag{1} [\hat{X},\hat{P}]~=~i\hbar ~{\bf 1}. $$
We can imitate the Dirac delta function and the signum function via the following integral representations$^1$
$$\tag{2}  \delta(\hat{X})~=~ \int_{\mathbb{R}} \! \frac{{\rm d}p}{2\pi\hbar} \exp\left(\frac{p\hat{X}}{i\hbar}\right), \qquad
\delta(\hat{P})~=~ \int_{\mathbb{R}} \! \frac{{\rm d}x}{2\pi\hbar} \exp\left(\frac{i x\hat{P}}{\hbar}\right), $$
$$\tag{3}  {\rm sgn}(\hat{X})~=~ \int_{\mathbb{R}} \! \frac{i{\rm d}p}{\pi p} \exp\left(\frac{p\hat{X}}{i\hbar}\right),  \qquad
{\rm sgn}(\hat{P})~=~ \int_{\mathbb{R}} \! \frac{{\rm d}x}{i\pi x} \exp\left(\frac{ix\hat{P}}{\hbar}\right). $$
The sought-for commutator can e.g. be written in $\hat{X}\hat{P}$-ordered form
$$\tag{4}   [{\rm sgn}(\hat{X}),{\rm sgn}(\hat{P})]
~=~ \iint_{\mathbb{R}^2} \! \frac{{\rm d}x~{\rm d}p}{\pi^2 xp}
\left[1-\exp\left(\frac{px}{i\hbar}\right)\right]
\exp\left(\frac{p\hat{X}}{i\hbar}\right)
\exp\left(\frac{ix\hat{P}}{\hbar}\right). $$
In eq. (4) we have used the following truncated Baker-Campbell-Hausdorff formula
$$\tag{5} e^{\hat{A}}e^{\hat{B}}
~=~e^{[\hat{A},\hat{B}]}e^{\hat{B}}e^{\hat{A}}, $$
which holds if the commutator $[\hat{A},\hat{B}]$ commutes with both the operators $\hat{A}$ and $\hat{B}$.
II) On a wavefunction $\psi(x)=\langle x |\psi\rangle $ in the Schrödinger position representation,
$$\tag{6} \hat{X}~=~x, \qquad \hat{P}~=~\frac{\hbar}{i}\frac{\partial}{\partial x}, $$
we have
$$\tag{7} \langle x |{\rm sgn}(\hat{P}) |\psi\rangle~=~ \int_{\mathbb{R}} \! \frac{{\rm d}y}{i\pi y}\langle x+y |\psi\rangle, $$
and therefore the matrix element of the sought-for commutator becomes
$$\tag{8} \langle x |[{\rm sgn}(\hat{X}),{\rm sgn}(\hat{P})] |\psi\rangle
~=~ \int_{\mathbb{R}} \! \frac{{\rm d}y}{i\pi y}\left({\rm sgn}(x)-{\rm sgn}(x+y)\right)\psi(x+y).$$
--
$^1$ The Cauchy principal value is implicitly assumed in pertinent places.
Comment: Could you please explain where the interpretation of ${\rm sgn}(\hat{p})$ came from. I mean, I know you seem to have interpreted it as the multiplication operator ${\rm sgn}(\hat{p})$ in *momentum* co-ordinates and then Fourier-transformed back to the $P\, 1/x$ in position co-ordinates - am I right? So there seems to be a "known" or "standard" way to interpret ${\rm sgn}$ - it isn't obvious from the question - so what kind of applications does this come up in? If this is too complicated for one line, I'll ask it as a new question.
Comment: I updated the answer.
Answer: Here is my answer (I have had time to answer only today). First of all, in general, if $A$ is a self-adjoint operator in the Hilbert space $H$  with spectrum $\sigma(A)\subset R$ (actually a closed normal operator would be enough), and $f: \sigma(A) \to C$ is a Borel measurable function (so for instance continuous up to a finite number of points would be OK), $f(A)$ is defined as:
$$f(A) := \int_{\sigma(A)} f(\lambda) dP^{(A)}(\lambda)\:,\qquad (1)$$
where $\{P^{(A)}_E\}_{E \in {\cal B}(R)}$ is the so-called  spectral measure of $A$ (for instance see http://planetmath.org/spectralmeasure). The $P_E$ are orthogonal projectors labelled by Borel sets $E$. Actually it turns out that $P_E=0$ if $E\cap \sigma(A)= \emptyset$, in this sense the measure is concentrated on the spectrum of $A$. Physically speaking $\sigma(A)$ is the set of the values that the observable $A$ can assume.
The fact that $A$ is sefl-adjoint guarantees  the existence of the above mentioned notions and the feasibility of the construction I go to summarize.
The integral in (1) is defined in a way similar to that used for Riemann or Lebesgue integrals, first defining the integral of a function $s$ attaining a finite number of values $s_1,\ldots, s_n$ on corresponding sets $E_1,\ldots, E_n$:
$$S(A) := \int_{\sigma(A)} s(\lambda) dP^{(A)}(\lambda) := \sum_{i=1}^n s_i P_{E_i} \qquad (1)'$$
and then taking the limit over a sequence of such functions $S_j$ point-wise tending to $f$ as $j \to +\infty$:
$$\int_{\sigma(A)} f(\lambda) dP^{(A)}(\lambda) \psi := \lim_{j\to +\infty} \int_{\sigma(A)} s(\lambda) dP^{(A)}(\lambda)\psi\:. \quad (2)$$
The notion of convergence is that of the Hilbert space of the theory. The given definition makes precise the domain of $f(A)$: It is given by the vectors $\psi \in H$ such that the limit in (2) exists.
It is worth stressing that:
(a) the operator $f(A)$ is bounded (that is continuous) and defined on the whole Hilbert space, if the function $f$ is bounded over $\sigma(A)$;
(b) it holds: $$A = \int_{\sigma(A)} \lambda dP^{(A)}(\lambda)$$
and this identity completely determines $\{P_E^{(A)}\}_{E\in {\cal B}(R)}$ for a given self-adjoint opertor $A$. It also arises taking (a) into accoutn, that $A$ is bounded if (and only if) the map $\lambda \to \lambda$ is bounded over $\sigma(A)$ that, in turn, means that $\sigma(A)$ is bounded.
To answer to the general question of the OP, the above given definition of $f(A)$ is that one has to use to compute things like $[f(X), f(P)]$.
If $f$ is not bounded the domains of $f(X)$ and $f(P)$ are not the whole Hilbert space, and thus great care has to be used in computing the commutator above, since it is defined only in a common invariant domain.
However, this is not the case for $f= sgn$, since it is bounded.
Let us pass to the computation of $[sgn(X), sgn(P)]$ that, consequently,  is a bounded operator as well.
If $A=X$ (position operator in $L^2(R)$), its spectral measure is quite trivial:
$$(P^{(X)}_E \psi)(x):= \chi_{E}(x)\psi(x)\:,\qquad (3)$$
where $\chi_E(x)=1$ if $x\in E$ and $\chi_E(x)=0$ if $x\not \in E$. Consequently, exploiting (1)' (because $sgn$ assumes only three values (respectively $-1$ in $E_1=(-\infty,0)$, $0$ in  $E_2= \{0\}$, $1$ in $E_3= (0,+\infty)$), one immediately sees that:
$$(sgn(X) \psi)(x) = sgn(x) \psi(x)\:. \qquad (4)$$
We have next to focus on the momentum operator $P$. In the following I will assume $\hbar=1$ for the sake of notational simplicity. Henceforth ${\cal F}: L^2(R) \to L^2(R)$ is the Fourier transform, defined on $L^1$ functions (and than extended by $L^2$ continuity into an unitary map on $L^2$) by the usual integral formula:
$${\cal F}: \psi(x) \mapsto \hat{\psi}(p) := \frac{1}{\sqrt{2\pi}}\int_R e^{-ipx} \psi(x) dx\:.$$
With these definitions, it turns out that the spectral measure of $P$ is
$$(P^{(P)}_E \psi)(x) := {\cal F}^{-1}\left( \chi_E \cdot \hat{\psi}\right)(x)$$
In other words: in momentum representation, the spectral measure of $P$  exactly coincides with that of $X$ in position representation.
As the spectral theory is "covariant" under unitary transformations, it implies in particular that $sgn(P)$ in momentum representation is again defined as:
$$\left(sgn(P)_{momentum} \hat{\psi}\right)(p) = sgn(p) \hat{\psi}(p)\:,$$
so that, coming back to the position representation:
$$sgn(P)\psi = {\cal F}^{-1} \left(sgn(p) \hat{\psi}(p)\right)\:. \quad (5) $$
We are in a position to compute the wanted commutator. I will assume that $\psi \in {\cal S}(R)$ the Schwartz space, because, in this case the Fourier transform can be computed as the usual integral and since that space is dense in $L^2$ so that the final result can be achieved simply taking a limit (as the commutator being bounded as stressed above). With that choice of $\psi$ all integration can be safely swapped. I do not enter into details.
Exploiting (4) and (5) (and interchanging integrals) we have almost immediately:
$$\left(sgn(X)sgn(P) \psi\right)(x)= \int\int \frac{e^{ip(x-y)}}{2\pi} sgn(p) sgn(x) \psi(y) dy dp$$
and
$$\left(sgn(P)sgn(X) \psi\right)(x)=  \int\int \frac{e^{ip(x-y)}}{2\pi} sgn(p)sgn(y) \psi(y) dy dp\:.$$
Taking the difference and inserting an $\epsilon$ prescription to separate integrals, we have:
$$\left([sgn(X),sgn(P)]\psi \right)(x) =  \lim_{\epsilon \to 0^+}\int \left(\int \frac{e^{ip(x-y) -|p| \epsilon}}{2\pi} sgn(p) dp\right)
(sgn(x) -sgn(y)) \psi(y) dy $$
Computing the integral (please check the values of the coefficients) we finally get:
$$\left([sgn(X),sgn(P)]\psi \right)(x) =  \lim_{\epsilon \to 0^+}\frac{1}{i\pi}\int_R
\frac{(x-y)(sgn(x) -sgn(y))}{(x-y)^2 + \epsilon^2} \psi(y) dy \:.$$
Formally, it is possible to introduce the so-called Cauchy principal value:
$$\frac{1}{2}\frac{(x-y)}{(x-y)^2 + 0^2} =  Vp \frac{1}{x-y}$$
so the found identity can be re-arranged in terms of the Cauchy principal value as done in the other answer.
Answer: Why is this commutator needed? I would start by trying to evaluate $$\int\!dx'\,dp'\, \langle \phi |\operatorname{sgn}(x) |x'\rangle\langle x'|p'\rangle\langle p'|\operatorname{sgn}(p) |\psi\rangle$$
and then each integral gets split up, e.g.,
$$-\int_{-\infty}^0\!dx'\,dp'\, \langle \phi |x'\rangle\langle x'|p'\rangle\langle p'|\operatorname{sgn}(p) |\psi\rangle + \int_{0}^\infty\!dx'\,dp'\, \langle \phi |x'\rangle\langle x'|p'\rangle\langle p'|\operatorname{sgn}(p) |\psi\rangle$$
and similarly for $p'$, giving you 4 integrals.  The other half of the commutator is
$$- \int\!dx'\,dp'\, \langle \phi |\operatorname{sgn}(p) |p'\rangle\langle p'|x'\rangle\langle x'|\operatorname{sgn}(x) |\psi\rangle$$
Keep in mind that $\langle p'|x'\rangle = \frac{1}{\sqrt{2\pi\hbar}}e^{-i p x}$ so that when you subtract the last 4 integrals from the first 4, you will need to relabel a coordinate to be able to cancel things out.
Comment: This is just a suggestion. Don't waste your time with it if it doesn't seem to work out.


SOAP Equivelant of RESTs /chatter/users/me/conversations?

Question: Title pretty much says it, I know that in REST we query private conversations using the above endpoint /chatter/users/{ID}/conversations, but I'm not sure where to look for the same thing in SOAP. Does anyone know?
Answer: ChatterConversation and ChatterMessage seem like the API objects you want.
Note that you will need the "Manage Chatter Messages" permission to see these via the API.
This is a bit of a guess, but will hopefully point you in the right direction:
SELECT Id,ConversationId,Body,SentDate FROM ChatterMessage where SenderId = '005100000000000'

Comment: How would I activate this via an application?
Comment: @ehime Are you referring to the permission or how to send the SOQL query via the Partner API?
Comment: I am, since we are working with direct or free signups via chatter.com we are being severly limited on how we can interact with these users, as some (most) do not have an administrative user in place to alter permissions.
Comment: @ehime I don't know how to alter administrative permissions for a user that directly signed up to chatter.com. Seems like a good candidate for a separate question.


SolrNet Fuzzy Search More Than One Word

Question: I am newbie on Solr and i want to full text search with SolrNet. So i used to MySql Full text search before and it is very slowly. I found Solr solution. This is very fast and scalability. But i can't fuzzy search in Solr. How can i do this? My examples and goals below. Excuse my badly english and thank you for advices.
I can fuzzy search but with one word (article):
For Example:
var solr = ServiceLocator.Current.GetInstance<ISolrOperations
>();
List
 search = solr.Query(new SolrQuery("segsource:article~") ).Cast
().ToList();

But i want to search more than a word (article name, bla bla bla - just like >
mysql> SELECT * FROM articles WHERE MATCH (title,body)
    -> AGAINST ('more than one word bla bla bla');

).
For example
var solr = ServiceLocator.Current.GetInstance<ISolrOperations
>();
List
 search = solr.Query(new SolrQuery("segsource:article name ~") ).Cast
().ToList();

Comment: This is answered in this previously asked question: http://stackoverflow.com/questions/8423498/solrnet-how-can-i-perform-fuzzy-search-in-solrnet
Answer: I'll see if there's a nicer way to do this, but in the meantime the following worked for me.
segsource:(article~ name~)
IE, you'd have to split the query on spaces and rejoin them inserting the tilda on each.
Answer: I never used SolrNet, but would something like this produce anything relevant to you:
new SolrQuery("segsource:article~ AND segsource:name~")

Comment: Thank you for response. I want to search sentence or phrase and this solving does not conform to my goal. Because if i searched this way, i wouldn't search "article name~".

SolrNet Fuzzy Search More Than One Word


Are "kernel mode" and "user mode" hardware features or software features?

Question: I read that there are two modes called "kernel mode" and "user mode" to handle execution of processes. (Understanding the Linux Kernel, 3rd Edition.) Is that a hardware switch (kernel/user) that is controlled by Linux, or software feature provided by the Linux kernel?
Comment: See also https://en.wikipedia.org/wiki/CPU_modes
Answer: Kernel mode and user mode are a hardware feature, specifically a feature of the processor. Processors designed for mid-to-high-end systems (PC, feature phone, smartphone, all but the simplest network appliances, …) include this feature. Kernel mode can go by different names: supervisor mode, privileged mode, etc. On x86 (the processor type in PCs), it is called "ring 0", and user mode is called "ring 3".
The processor has a bit of storage in a register that indicates whether it is in kernel mode or user mode. (This can be more than one bit on processors that have more than two such modes.) Some operations can only be carried out while in kernel mode, in particular changing the virtual memory configuration by modifying the registers that control the MMU. Furthermore, there are only very few ways to switch from user mode to kernel mode, and they all require jumping to addresses controlled by the kernel code. This allows the code running in kernel mode to control the memory that code running in user mode can access.
Unix-like operating systems (and most other operating systems with process isolation) are divided in two parts:

The kernel runs in kernel mode. The kernel can do everything.
Processes run in user mode. Processes can't access hardware and can't access the memory of other processes (except as explicitly shared).

The operating system thus leverages the hardware features (privileged mode, MMU) to enforce isolation between processes.
Microkernel-based operating systems have a finer-grained architecture, with less code running in kernel mode.
When user mode code needs to perform actions that it can't do directly (such as access a file, access a peripheral, communicate with another process, …), it makes a system call: a jump into a predefined place in kernel code.
When a hardware peripheral needs to request attention from the CPU, it switches the CPU to kernel mode and jumps to a predefined place in kernel code. This is called an interrupt.
Further reading

Wikipedia
What is the difference between user-level threads and kernel-level threads?
Hardware protection needed for operating system kernel
Answer: "Linux" proper is only the kernel. When we talk about "Linux" as an operating system, it is a conflation of the kernel and all of the other software that is distributed with it in a Linux distribution.
The switch between user and kernel mode happens when you make a system call, which is any of the functions documented in manual section 2. That is, if you say something like man 2 open and get a result, or man creat and see that there is a (2) suffix on the page title, you're looking at a system call, which means it's handled by the kernel.
(The distinction here is with manual section 3, which is for functions implemented entirely in user-space, such as the system's Standard C Library.)
When the system call returns to the user code that called it, it returns that process to user mode.
The Linux kernel uses hardware features to enforce the distinction between kernel and user mode on most processors that it runs on. The Intel protection ring scheme is one way to do this. There are exceptions, such as µCLinux, which runs on small processors without an MMU, which may also lack protection rings. Very old versions of Unix also ran without kernel/user protection.
When we talk about "user mode code," we are talking about code that mostly runs in user space, though pretty much any program is going to contain some system calls, and so will spend some of its time waiting on the kernel, which always runs in kernel mode.
Comment: what is `manual section 2`
Comment: @DScript: I've edited the answer to make it clearer.
Comment: "Linux" usually means a Linux distribution. This has been [debated ad nauseam on Wikipedia](https://en.wikipedia.org/wiki/Talk:Linux#Linux_and_its_status_as_an_operating_system). When it comes to terminology, the 99% can't be wrong.
Comment: @Gilles: While I am no [prescriptivist](http://en.wikipedia.org/wiki/Linguistic_prescription), the original question came up due to a confusion between Linux-the-OS and Linux-the-kernel. The OP clearly had no idea when reading a book on the Linux kernel that Linux-the-OS has no role in maintaining the kernel/user space protection barrier. I know of no better reason for making this careful distinction.
Comment: @Gilles unfortunately 99% can be wrong very easily - and they often are. It's just easier to accept not 100% correct ideas when it doesn't matter that much (i.e. when it is not a matter of life-and-death).

Are “kernel mode” and “user mode” hardware features or software features?


Any Alternate way for writing to a file other than ofstream

Question: I am performing file operations (writeToFile) which fetches the data from a xml and writes into a output file(a1.txt).
I am using MS Visual C++ 2008 and in windows XP.
currently i am using this method of writing to output file..
01.ofstreamhdr   OutputFile;
02./* few other stmts */
03.hdrOutputFile.open(fileName, std::ios::out);
04.
05.hdrOutputFile << "#include \"commondata.h\""<< endl ;
06.hdrOutputFile << "#include \"Commonconfig.h\"" << endl ;
07.hdrOutputFile << "#include \"commontable.h\"" << endl << endl ;
08. hdrOutputFile << "#pragma pack(push,1)" << endl ;
09.hdrOutputFile << "typedef struct \n {" << endl ;
10./* simliar hdrOutputFiles statements... */..

I have around 250 lines to write.. Is any better way to perform this task.
I want to reduce this hdrOutputFile  and use a buffer to do this.
Please guide me how to do that action.
I mean,
buff = "#include \"commontable.h\""  + "typedef struct \n {"  + .......

hdrOutputFile  << buff.

is this way possible?
Thanks
Ramm
Comment: One place to start would be in formatting your code so it's easier to read :p
Answer: How about:
const char * buffer =
    "This is one line of text\n"
    "This is the start of another"
    " and this is the end of the same line\n"
    "and so on\n";

hdrOutputFile << buffer;

In both C and C++, string literals like this are automatically concatenated into a single string.


Can someone calculate my private key if I send the same amount to one address?

Question: If I send the exact same amount to the same address in two separate transactions, can someone calculate my private key from the scriptSig? Will the signature (S) and the message (Z) be same?
Answer: Generally, your wallet software should ensure that you can create and recreate the same or different transactions safely.
However there is one issue that seems related and might have prompted your question.
When someone creates an ECDSA signature, the signature is composed of two integers, r and s. r only depends on a random nonce k, while s depends on the private key, r, and the message. If the same k is reused, it will cause a signature to have the same r value. Usually, your wallet will use a different k for every signature even for the same message, so your signature will be different every time. Some wallets use deterministic nonces, so they'd recreate the same signature for the same message. They will still create a different signature for different messages.
It is important that this k is always picked randomly (unless you're using deterministic nonces): if you sign two different messages (i.e. transactions) with the same private key and the same k value, the two signatures result in a set of equations that can be solved for the private key.
Answer: No, regardless of whether the messages and signatures are the same, it is not possible to calculate the private key from the public information in the blockchain data.
In practise, I believe the signed data in a transaction includes the hash of the previous transaction for each input, these cannot be the same as in any prior confirmed transaction because that would constitute a forbidden double-spend.
I believe cryptographic weaknesses can be quite subtle but the situation you describe, multiple payments of the same amount to the same address, must be quite common. Consequently I'd be very surprised if it had not been anticipated and considered long ago.


Looking for a replacement shimano Tiagra Front Derailleur

Question: What is compatible with the older Shimano Tiagra FD-4403
9 speed triple derailleur
30/42/52 tooth chhainrings
34.9mm clamp on mount.
Answer: It seems like an innocent question, but the situation can be maddening when a replacement FD is needed for any of the 30-42-52 cranks.
Front derailleurs have a spec that has been called by Shimano both "Min. difference between top and intermediate" and "Applicable top-mid tooth difference". It is the minimum difference in tooth count between the middle and large ring that the front derailleur can accommodate without the tail end of it bonking into the middle
ring as it passes over, unless you make it much higher than you'd want to. For a long time, basically since the era of 9-speed front derailleurs that 4403 belongs to, Shimano changed to 30-39-50 for the road triples and the number for that spec is 11, whereas you need it to be 10. Being off by one forces the FD to be mounted much higher than optimal and it usually works poorly and can cause problems with derailment.
The good answers are any of the old Shimano 9-speed road triples where that spec is 10t, so FD-4403, 5503, 5504, 6503, and R443, in either the 34.9 clamp version if they came in one or in braze-on version plus an adapter. There have been a few repair type models that meet this compatibility too, such as Microshift R539 in braze-on configuration. That one appears to still be obtainable new, and I don't know if that can be said for any other compatible matches.
Comment: Thank You Guys I really appreciate your input
Answer: Really most any Shimano triple road front derailleur for 8, 9 or 10 speed will work. Either one that comes with a 31.8mm clamp band or a braze-on type front derailleur can be used with a braze-on adapter, which is a clamp band that will accept the braze-on type derailleur if the bike lacks a braze-on mount.  The adapter clamp band will obviously have to be a 31.8 diameter as well. These are quite common and easy to find.
The reason any of these speed classes of derailleur will work is that there was a consistent design in road front derailleur's that allowed for consistent derailleur movement based on shifter input throughout the speed classes.  Essentially, a road "x-speed" front derailleur is more of a marketing term than a descriptor of the derailleur's spec so many different options over several years are viable replacements.  This coupled with the fact that chainwheel size for road bikes remained consistent throughout those speed generations--most road triple cranks came spec'd with 52 or 53 tooth large chainrings prior to the compact sizing seen today.  This is relevant because for good front shifting and minimal chain rub, the outer plate of a front derailleur should fairly well match the outer circumference of the large chainwheel. That's something to watch out for in the replacement but you'll find ones fitting a 52 or 3 tooth chainwheel far more prevalent.
Here is a line-up chart from the era of the FD-4403.  Model numbers FD-6603, 5603, 4403, 2203, etc would all handle a triple crank with a 52 tooth large ring.  Note the presence of the "3" in the final model number place.  This is Shimano's convention that means it is designed for a triple front crankset as opposed to model numbers terminating in "00" (most common) or "02" (rarely) which infers a derailleur designed for a double crankset.  Avoid these as a double front derailleur's cage is not designed to handle the range of chainwheel sizes found on a triple crankset. They can be set-up to work on a triple, but why? when the proper design is readily available.
Comment: I'll defer to @Nathan Knutson 's excellent point about the large to mid ring tooth difference being a gotcha for a working replacement.  While I haven't seen this as a problem in my more limited experience, I can tell you that for best shifting, the front derailleur's cage should be as close as possible to the teeth of the large chainring without interfering.  Any set-up that forces the mounting of the FD to be higher away from the teeth is less than optimal.
Comment: Thank You Very Much for the information