Python MapReduce on Sun Grid Engine

Question: I am relatively new to distributed computing, so forgive me if I misunderstand some of the basic concepts here.  I am looking for a (preferably) Python-based alternative to Hadoop for processing large data sets via MapReduce on a cluster using an SGE-based grid engine (eg. OpenGrid or Sun of Grid Engine).  I have had good luck running basic distributed jobs with PythonGrid, but I'd really like a more feature-rich framework for running my jobs.  I have read up on tools like Disco and MinceMeatPy, both of which seem to offer true Map-Sort-Reduce job processing, but their does not seem to be any obvious support for SGE.  This makes me wonder if it is possible to achieve true MapReduce functionality using a grid scheduler, or if people just don't support it out-of-the-box because they are not frequently used.  Can you perform Map-Sort-Reduce tasks on a grid engine?  Are their Python tools that support this?  How difficult would it be to rig existing MapReduce tools to use SGE job schedulers?
Answer: I've heard that Jug works. It's using the filesystem for coordination amongst the parallel tasks. In that kind of framework, you'd write your code and run "jug status primes.py" on the machine you're on then start a grid array job with as many workers as you like, all running "jug execute primes.py".
mincemeat.py should be able to function in the same way but looks to use the network for coordination. So that may depend on whether your nodes can talk to a server running the overall script.
There are several release notes about running actual Hadoop MapReduce and HDFS on SGE, but I haven't been able to find good documentation.
If you're used to Hadoop streaming with Python, it's not too bad to replicate on SGE.  I've had some success with this at work: I run an array job that does map + shuffle for each input file. Then another array job that does sort + reduce for each reducer number. The shuffle part just writes files to a network dir like mapper00000_reducer00000, mapper00000_reducer00001, and so on (all pairs of mapper and reducer numbers). Then reducer 00001 sorts all files labeled reducer00001 together and pipes to reducer code.
Unfortunately, Hadoop streaming isn't very full-featured.
Comment: Thanks for the reply, it has given me something to think about.  I'm curious how your solution performs reading/writing so many intermediary files.  Ever tried using a database like Cassandra instead of writing to files?
Comment: I'm not too familiar with Cassandra. From reading though their slides, I doubt I could use it; it seems like it needs dedicated Cassandra boxes. But at a high level, I don't know that it would speed things up unless I had a lot of Cassandra nodes. Doing map-reduce on SGE is entirely sequential reads/writes so I wouldn't use the random-access aspect of Cassandra. So Cassandra would help if I could do an installation with enough disk parallelism to provide high bandwidth than a networked filesystem.

Python MapReduce on Sun Grid Engine


How to improve torque and RPM of a DC motor?

Question: I have an FA-130 motor (DC) with permanent magnet, my power source is a 2 AA batteries (rechargeable) so a total of 2.4v.
Assume that all cases will start from the same specification, theoretically, what would happen if I do the following?
Case 1: Increase/Decrease strength of permanent magnets. What would happen to torque and RPM? Why?
Case 2: Increase/Decrease size of magnet wires. What would happen to torque, power consumption and RPM? Why?
Case 3: Increase/Decrease the size of armature. What would happen to torque, power consumption and RPM? Why?
Case 4: Increase/Decrease the number of turns (coil). What would happen to torque, power consumption and RPM? Why?
In general, how can I increase the torque and RPM of this motor given a constant voltage?
Please, explain it as if you're talking to a 6-year old kid, I'm not knowledgeable in this field but I want to know the concept.
Comment: may be this model will be helpful,
http://robotics.ee.uwa.edu.au/courses/embedded/tutorials/tutorials/tutorial_6/Tutorial_6.htm
Comment: it does typically, but not always, because there are varying parameters in that model. Vemf represents it's back voltage. So I'm thinking how I could explain it to a 6th year kid .
Comment: @sandundhammika thanks, maybe I could put a little more effort, you can consider me a 12 year-old now who knows nothing about electronics...
Comment: I thought you asking how to teach this to 6th year kid,I'm confused, sorry.
Comment: I think I somehow understood EMF now, I saw this, "Anytime an inductor (in this case the coil) passes through an electric field, it creates a voltage. This is how generators work. This is still true when the motor spins under it's own power. But, this voltage is going the opposite direction of the voltage we are putting into the motor to make it spin, so it subtracts. This is called back voltage or back EMF. At a certain speed, the back voltage equals the voltage we put into the motor, and (in a perfect world), when the motor is maxed out in RPM, and no electricity flows, thus no current."
Comment: see the model , there's not only back emf voltage, there are many parameters,such as pure resistance, R, and leakage flux "L".

Normally we control motors using Duty cycle and some feedback.
Comment: I think somebody else will explain that model simply to you.so wait.
Comment: @dpp seems like what you are really interested in is improving speed of a given motor. There are definitely more advanced commutation methods that trade complexity for speed. See FOC, "field weakening", TIs instanspin/powerwarp, etc. Field weakenkng, altho hardly a new idea, definitely allows you to drive bldcs to higher rpms than normal. S.
Comment: Field weakening allows higher speed at the expense of torque, so this isn't really what @dpp is looking for.
Answer: I'm going to assume that this 6 year old has at least a little background in physics. I'm going to start off by answering why each result will occur with a lot of math to describe the physics behind it all. Then I will answer each case individually with the math providing the reasoning behind each result. I will wrap up by answering your "in general" question.

Why?
The answer to all of your "Why?" questions is: Physics! Specifically Lorentz's law and Faraday's law. From here:

The torque of the motor is determined by the equation:
$$\tau = K_t \cdot I~~~~~~~~~~(N \cdot m)$$
Where:
\$\tau = \text{torque}\$
\$K_t = \text{torque constant}\$
\$I = \text{motor current}\$
The torque constant, \$K_t\$, is one of the main motor parameters that describe the specific motor based on the various parameters of its design such as magnetic strength, number of wire turns, armature length, etc. as you've mentioned. Its value is given in torque per amp and is calculated as:
$$K_t = 2 \cdot B \cdot N \cdot l \cdot r~~~~~~~~~~(N \cdot m / A)$$
Where:
\$B = \text{strength of magnetic field in Teslas}\$
\$N = \text{number of loops of wire in the magnetic field}\$
\$l = \text{length of magnetic field acting on wire}\$
\$r = \text{radius of motor armature}\$

The Back-EMF voltage is determined by:
$$V = K_e \cdot \omega~~~~~~~~~~(volts)$$
Where:
\$V = \text{Back-EMF voltage}\$
\$K_e = \text{voltage constant}\$
\$\omega = \text{angular velocity}\$
Angular velocity is the speed of the motor in radians per second (rad/sec) which can be converted from RPM:
$$\text{rad/sec} = \text{RPM}\times\dfrac{\pi}{30}$$
\$K_e\$ is the second main motor parameter. Funnily enough, \$K_e\$ is calculated using the same formula as \$K_t\$ but is given in different units:
$$K_e = 2 \cdot B \cdot N \cdot l \cdot r~~~~~~~~~~(volts/rad/sec)$$

Why does \$K_e = K_t\$? Because of the physical law of Conservation of Energy. Which basically states that the electrical power put into the motor needs to equal the mechanical power got out of the motor. Assuming 100% efficiency:
\$P_{in} = P_{out}\$
\$V \cdot I = \tau \cdot \omega\$
Substituting the equations from above we get:
\$(K_e \cdot \omega) \cdot I = (K_t \cdot I) \cdot \omega\$
\$K_e = K_t\$

Cases
I'm going to assume that each parameter is being changed in isolation.

Case 1: Magnetic field strength is directly proportional to the torque constant, \$K_t\$. So as magnetic field strength is increased or decreased, the torque, \$\tau\$, will increase or decrease proportionally. Which makes sense because the stronger the magnetic field, the stronger the "push" on the armature.
Magnetic field strength is also directly proportional to the voltage constant, \$K_e\$. However \$K_e\$ is inversely proportional to the angular velocity:
$$\omega = \dfrac{V}{K_e}$$
So, as the magnetic field increases, the speed will decrease. This again makes sense because the stronger the magnetic field, the stronger the "push" on the armature so it will resist a change in speed.
Because power out is equal to torque times angular velocity, and power in equals power out (again, assuming 100% efficiency), we get:
$$P_{in} = \tau \cdot \omega$$
So any change to torque or speed will be directly proportional to the power required to drive the motor.

Case 2: (A bit more math here that I didn't explicitly go over above) Going back to Lorentz's law we see that:
$$\tau = 2 \cdot F \cdot r = 2 (I \cdot B \cdot N \cdot l) r$$
Therefore:
$$F = I \cdot B \cdot N \cdot l$$
Thanks to Newton we have:
$$F = m \cdot g$$
So...
$$\tau = 2 \cdot m \cdot g \cdot r$$
If you keep the length of the wire the same but increase its gauge, the mass will increase. As can be seen above, mass is directly proportional to torque just like magnetic field strength so the same result applies.

Case 3: The radius of the armature, \$r\$ in our equations above, is again directly proportional to our motor constants. So, once again, we have the same results as we increase and decrease its length.
Starting to see a pattern here?

Case 4: The number of turns of our wire, \$N\$ in our equations above, is also directly proportional to our motor constants. So, as usual, we have the same results as we increase and decrease the number of turns.

In general
If it isn't obvious by now, torque and speed are inversely proportional:

There is a trade-off to be made in terms of power input to the motor (voltage and current) and power output from the motor (torque and speed):
$$V \cdot I = \tau \cdot \omega$$
If you want to keep the voltage constant, you can only increase current. Increasing current will only increase torque (and the total power being supplied to the system):
$$\tau = K_t \cdot I$$
In order to increase speed, you need to increase voltage:
$$\omega = \dfrac{V}{K_e}$$
If you want to keep the input power constant, then you need to modify one of the physical motor parameters to change the motor constants.
Comment: Thats a hell of a generous answer for a homework question.
Comment: @insta that isn't my homework, that's for my toy car.
Comment: Great answer here! I just want to clarify this: you say increasing current increases torque "at the expense of speed". But increasing current doesnt affect Ke in the speed equation, since Ke and Kt are motor constants. So speed should remain the same but torque has now increased and the overall power supplied is also increased? The inverse relationship between speed and torque only comes into play, when any of the factors of the motor constants is altered? Thanks in advance.
Comment: @TisteAndii Good catch. That was a poor choice of words on my parts. I've edited to hopefully be a little clearer. But you are correct. Increasing current increases input power and output torque but does not effect speed. If power is held constant, then modifying any of the motor parameters would effect both torque *and* speed. This is because \$K_t = K_e\$ and because current is *multiplied* by the constant to get torque and voltage is *divided* by the constant to get speed.
Comment: Thank you so much for this. I have a midterm on this tomorrow and I think I would just have been pasting formulas from rote memorisation if not for you.
Comment: @Asad You're very welcome! Good luck on your test!
Comment: I think this is what I'm looking for, you gave me all I need! I nearly grasp the concept without understanding the formula. I think I should really study more, seems like one cannot create an effective motor using theories alone.
Comment: Can you tell me how the car's fans rotate so quickly with only 12v and since current only increase torque and not speed? (i'm noob) Is there some voltage converter?
Comment: @Ismael Well, the blades themselves are usually quite light, plastic or aluminum, so it doesn't take much torque to move it. Torque and speed are inversely proportional. With the voltage being a constant 12V, to go faster, you need to decrease \$K_e\$ by decreasing any of the component values. To get it going fast you can use weak magnets, like ceramic or more likely Alnico, to decrease \$B\$, and small magnets to decrease \$l\$. Or you could decrease the number of turns of the stator wire to decrease \$n\$. Or have a really short armature to decrease \$r\$.
Comment: What exactly does it mean by "length of the magnetic field acting on the wire"?
Comment: @Tanenthor Does "length of wire in a magnetic field" make any more sense? Take a look at the pictures. They show what is meant by \$l\$.
Comment: @embedded.kyle Ahh, I think I know what you mean now. And what about the section on Case 2, I don't really understand how the Force equation is being implemented and the use of "g" instead of "a" for acceleration.
Comment: @Tanenthor "g" is commonly substituted for "a" in the force formula when the acceleration in question is due to gravitational effects (on Earth 9.8m/s^2). It doesn't really fit here and I'm not sure why I used it (question is 4 years old at this point). What I'm doing is simply trading the electromagnetism definition of force (in terms of current and magnetic field strength) for the physical one (mass and acceleration) to show how the mass of the wire effect torque.
Answer: One explanation is to consider that power \$P\$ is the product of current \$I\$ and voltage \$E\$:
\$ P = IE \$
Power is measured in watts, and is the rate of energy use. Energy is measured in joules, and a watt is convienently defined as one joule per second.
The application of a motor, usually, is to apply a force to a thing to move it. In physics, this is called work, which is equal to the product of force \$F\$ and distance \$d\$:
\$ W = Fd \$
You asked about increasing torque and RPM. Torque is just a rotating force, and RPM is just a rotating speed. So the definition of work is half of what you asked (it has torque in it), and speed and distance are obviously related. It seems like we are really close. You don't want to just do more work with your motor, you want to do work faster. You want to increase force and speed, not force and distance. Is there a physical term for this in a mechanical system?
Yes! It's also called power. In a mechanical system, power is the product of force and velocity:
\$P = Fv\$
Or to use the equivalent terms for a rotational system, power is the product of torque and angular velocity:
\$ P = \tau \omega \$
This is just what you asked. You want the motor to apply more torque and spin faster. You want to increase power. You want to use energy faster.
The law of conservation of energy tells us that if we want to increase mechanical power, we have to increase electrical power also. After all, we can't make the motor spin with magic. If electrical power is the product of voltage and current, then increasing either voltage or current, if the other is held constant, will increase electrical power.
When you change the strength of the magnets, or add or remove turns of wire, you can't increase power. You can, however, trade voltage for current, or current for voltage, just like a mechanical transmission can trade RPM and torque. Lenz's law and other laws of electromagnetic induction explain why this is true, but they aren't really necessary to answer your question, if you simply accept the law of conservation of energy.
Given all that, your question was "How to improve torque and RPM of a DC motor". You can improve it by giving it more energy, or you can make it more efficient. Some sources of loss are:

friction in the bearings
resistance in the windings
magnetic resistance in the winding cores
electromagnetic radiation from commutators
losses in the wires, battery, transistors, and other things supplying electrical energy to the motor

All these serve to make the motor less than a 100% efficient converter of electrical and mechanical energy. Reducing any of them usually increases something else undesirable, frequently cost or size.
An interesting thought: This is why electric hybrid cars can get better mileage in the city. Stopping at a red light converts all the energy of your moving car into heat at the brake pads, which isn't useful. Because a motor is a converter between electrical and mechanical energy, a hybrid car can convert this energy not into heat, but instead into electrical energy, store it in a battery, then convert it back to mechanical energy when the light is green. For further reading, try How can I implement regenerative braking of a DC motor?
Comment: @dpp 2.4V isn't the magnitude of the power source, it's just the voltage. You also have to measure current to know how much power you were delivering. Decreasing the number of turns weakens the magnetic field per amp, which also reduces the back-emf, which improves RPM but reduces torque for a given power. But, you also used bigger wire, which has less resistance, which make the motor more efficient _and_ allows you to deliver more current/power with a given voltage. \$P = E^2/R\$ and \$I = E/R\$. If you measure the current you will find that power went up, and you drain the battery faster.
Comment: Hi, in my experiments I noticed that when I increase the size of wire and decrease the number of turns, I get higher RPM and better torque, why is that? I didn't increase the power source (still 2.4v). By the way, your answer helped me.
Comment: @Thanks Phil, the way you answer makes it easier for me to understand, I mean by saying which causes what etc.
Answer: Although you have received very good and detailed answers, I would like to offer a very simple answer utilizing the formulas already presented:$$\tau = 2.B.N.l.r.I $$ This formula clearly shows that the torque  is directly proportional to the magnetic field strength, the number of turns the length of the loop, the radius of the armature and the current in the wires.  So, as any of these variables increase or decrease, so does the torque.
The other formula, $$\omega = V/2.B.N.l.r.I$$
clearly shows that the RPM is inversely proportional to the same variables.  Therefore, as they increase, the RPM decreases, and vice-versa.
If you increase the wire gauge, you increase the current (I) and thereby the torque. If you also decrease the number of turns, you will decrease the torque.  Whether the total torque increases or decreases, depends on which effect is larger.
Comment: I decided that having brushless motor and adding ball bearings will help. I guess the friction reduces the RPM and the effectiveness of torque.


Why are tracked vehicles rarely used/seen?

Question: Throughout star wars, tracked military vehicles have not been prominent, but other types have- for example, the Clone Turbo tank was a wheeled vehicle that was used in rough terrain. My understanding is that tracked vehicles are often used by armies to scale rough, boggy and uneven terrain that wheeled vehicles would find hard to travel across.
I am aware that there have been tracked vehicles that feature in Star Wars, such as the Combat Assault Tank, used by the Imperial Army, and the MLC-3 light tank, used by the Rebel Alliance. I am also aware that hover vehicles and armored vehicles with legs are also popular, but they perform are meant to go where neither vehicles with caterpillar tracks or wheels are meant to go.So why are tracked vehicles not featured more prominently in Star Wars?
Answer: Treaded vehicles aren't seen because Repulsorlift Technology is cheap, efficient, and prolific in the Universe. This tech takes up all the environments where tracked vehicles would theoretically be the most useful on.
Prolific:
It's used in varying capacities on nearly every world we are introduced to, Speeders and speeder bikes, used as landing engines for spacecraft, even Imperial Star Destroyers can use it to hover in Atmo.
Cheap:
The widespread use even on backwater worlds like Tatooine means that this tech is cheap enough (and energy efficient enough) to be used whenever one needs a vehicle that can go on all terrains.
Efficient:
You don't need to bring certain vehicles to invade certain planets (except submarines for obvious reasons) when  you can bring one tank/transport that hovers over the environment and go on them all.
Anytime you need an all-terrain vehicle you would be better off using Repulsors
The better question might be why are All-Terrain vehicles are walkers when you have a much better All-Terrain tech widely available. Possibly it's not as reliable as we're led to believe. Why you would risk taking an ISD into atmo on only repulsors that aren't as reliable as landing gear like we see on Acclamators would be an interesting case study I guess.
I get what you are saying too, tanks with treads would seem to be more stable/useful than the walkers, but they are used less I think because of coolness factor. I'm also not sure on the breakdown rates of walker leg vs turbo tank wheel vs tank tread vs laat/lambda repulsor.
We do see a number of hover tanks in the star wars uni, most of which are no longer Disney canon.
The droids used MTTs and AATs on the world of Naboo in the swamps where I assume most tanks or wheeled vehicles would get stuck:

The Imperial Assault Tank is present in the comics
The T2B tank is noted for it's speed and versatility, although it also says one of the reasons it lacks heavy weapons is because of the repulsors.
And the TX-225 GAVr "Occupier" is used on Lothal for speed and All-Terrain capabilities in conjuction with walkers on the show Star Wars Rebels.
Answer: The most notable tracked vehicle in the saga is, naturally, the Sandcrawler - the transports the Jawas of Tatooine use in their scavenging pursuits. As we see them in both the Prequels as well as the Original Trilogy, we can surmise that repulsorlift technology for land-based vehicles is relatively new during the end of the Galactic Republic era, but the lack of basically any other kinds of tracked vehicles shows that repulsorlift tech is stable and economical enough for mass use.


Testing objects equality in NodeJS

Question: We are writing tests for a program. We want to write a functionnal test that verifies that the output of the program matches some expectation.
The object returned is a complex JS object (with nested objects, many properties... etc).
We want to test that this obviously matches what we need.
Up until now, we were "browsing" the object and the expected outcome, checking for each property, and each nested object. That is very cumbersome and we were wondering if there was any library that would 'build' all the tests, based just on the object. Something like this for example.
var res = {
  a: {
    alpha: [1,2,3],
    beta: "Hello",
    gamma: "World"
    },
    },
    b: 123,
    c: "It depends"
  }
};
var expectation = {
  a: {
    alpha: [1,2,4],
    beta: "Hello",
    gamma: "World"
    },
    },
    b: 123,
    c: "It depends"
  }
};

assert(res, expectation) // -> Raises an error because res[a][b][2] is different from expectation[a][b][2].

[In the example, I have simplified the complexity of our object...]
I should insist on the fact that we need a piece of code that is smart enough to tell us what is different, rather than just tell us that the 2 objects are different. We now about deep equality, but we haven't found anything that actually tells us the differences.
Comment: http://docs.jquery.com/QUnit/deepEqual
Comment: http://documentcloud.github.com/underscore/#isEqual
Answer: Node has the built in assert module meant for testing. This has a method called deepEqual for deep equality checking.
Function signature is:
assert.deepEqual(actual, expected, [message])

Quickly written function for testing deepEquality and returning diff:
// Will test own properties only
function deepEqualWithDiff(a, e, names){
  var dif = {};
  var aKeys = Object.keys(a);
  var eKeys = Object.keys(e);

  var cKeys = aKeys;
  var dKeys = eKeys;
  var c = a;
  var d = e;
  var names = {
    c: names ? names['a'] : 'Actual',
    d: names ? names['e'] : 'Expected'
  }

  if(eKeys.length > aKeys.length){
    cKeys = eKeys;
    dKeys = aKeys;
    c = e;
    d = a;
    names = {
      d: names ? names['a'] : 'Actual',
      c: names ? names['e'] : 'Expected'
    }
  }

  for(var i = 0, co = cKeys.length; i < co; i++){
    var key = cKeys[i];
    if(typeof c[key] !== typeof d[key]){
      dif[key] = 'Type mismatch ' + names['c'] + ':' + typeof c[key] + '!==' + names['d'] + typeof d[key];
      continue;
    }
    if(typeof c[key] === 'function'){
      if(c[key].toString() !== d[key].toString()){
        dif[key] = 'Differing functions';
      }
      continue;
    }
    if(typeof c[key] === 'object'){
      if(c[key].length !== undefined){ // array
        var temp = c[key].slice(0);
        temp = temp.filter(function(el){
          return (d[key].indexOf(el) === -1);
        });
        var message = '';
        if(temp.length > 0){
          message += names['c'] + ' excess ' + JSON.stringify(temp);
        }

        temp = d[key].slice(0);
        temp = temp.filter(function(el){
          return (c[key].indexOf(el) === -1);
        });
        if(temp.length > 0){
          message += ' and ' + names['d'] + ' excess ' + JSON.stringify(temp);
        }
        if(message !== ''){
          dif[key] = message;
        }
        continue;
      }
      var diff = deepEqualWithDiff(c[key], d[key], {a:names['c'],e:names['d']});
      if(diff !== true && Object.keys(diff).length > 0){
        dif[key] = diff;
      }
      continue;
    }
    // Simple types left so
    if(c[key] !== d[key]){
      dif[key] = names['c'] + ':' + c[key] + ' !== ' + names['d'] + ':' + d[key];
    }
  }
  return Object.keys(dif).length > 0 ? dif : true;
}

Comment: This is sort of unfortunate in that assert's `deepequal` can't be used in a regular `isdeepequal` sense (not without an exception it seems)
Comment: That's really not we're looking for, because it just indicates that the 2 objects are not equal... but fails to tell us what's different!
Answer: JavaScript doesn't support object deep equality out of the box and I am not aware of anything built into the NodeJS API either.
My bet would probably be Underscore and the isEqual function.

npm install underscore

var _ = require('underscore');
var moe   = {name : 'moe', luckyNumbers : [13, 27, 34]};
var clone = {name : 'moe', luckyNumbers : [13, 27, 34]};
moe == clone;
=> false
_.isEqual(moe, clone);
=> true

Although most Node testing frameworks should also contain an object deep equality assertion but you didn't mention which one you are using.
Comment: That's really not we're looking for, because it just indicates that the 2 objects are not equal... but fails to tell us what's different!
Comment: Good point, sorry I missed that. Here is a difference algorithm that is using underscore: http://blog.vjeux.com/2011/javascript/object-difference.html
Answer: deep-diff does what OP is asking for. It can compare two javascript objects / JSON objects and list the differences in a way that your code can access.
This is an old question so this library probably didn't exist at the time the question was asked.
Answer: Note that Node's deepEqual() is deprecated since Node.js v9.9.0, use deepStrictEqual() instead:
const assert = require('assert');

const actual = {
  a: {
    name: 'some',
    b: {
      name: 'thing',
    },
  },
};

const expected = {
  a: {
    name: 'thing',
    b: {
      name: 'some',
    },
  },
};

assert.deepStrictEqual(actual, expected, 'Expect names to be equal.');

Pro Tip: If you don't use a custom message in deepStrictEqual, you will get a nice colorful output of the differences between the two objects:
Answer: While I think the basics of object comparison have already been covered (see here and others), if you're looking for a quick and dirty way to execute these tests and see the diff between two non-equal objects, you can just run your tests using nodeunit within the Webstorm IDE (here)
Webstorm integrates particularly well with nodeunit, and for any test.equals() or test.deepEquals() assertions it provides a viewable diff with highlighting to show your descrepencies. I highly recommend the IDE for how well it integrates testing into my js development cycle.
Now, if you need the results of that test/diff to be accessible within your code, this obviously isn't enough for you and I'd recommend copying a couple of the deepEquals comparators from the first link I listed.
Good luck,
Brian
Answer: This is a simple/flexible deep diff function I wrote using Lodash:

_.merge(obj1, obj2, function (objectValue, sourceValue, key, object, source) {
    if ( !(_.isEqual(objectValue, sourceValue)) && (Object(objectValue) !== objectValue)) {
        console.log(key + "\n    Expected: " + sourceValue + "\n    Actual: " + objectValue);
    }
});

You can replace the console.log() statement with whatever comparison logic you need.  Mine just prints out the differences to the console.

Testing objects equality in NodeJS


Are non-US citizens trading on US stocks/options subject to tax?

Question: I'm a non-US citizen, and not residing in US, who uses an online brokerage firm to trade US stocks and options. Are my capital gains (short term and long term) and dividends subject to taxes? If yes, what are those taxes?
Comment: Related: [Are there capital gains taxes or dividend taxes if I invest in the U.S. stock market from outside of the country?](https://money.stackexchange.com/questions/4939/are-there-capital-gains-taxes-or-dividend-taxes-if-i-invest-in-the-u-s-stock-ma)
Answer: You, lucky sir, if you are in fact a non-resident alien (in the US less than half of one year), are tax-free for capital gains with a few caveats and such.
Generally speaking, if you are a long term investor in non-real estate securities and spend no time in the US, your capital gains are not taxable.
If the short term gains appear to be from trading and not investing, they may be taxable.
Dividends are taxed at a merciless 30% with exceptions, unless there's a tax treaty between your home country and the US to say otherwise.
That said, you're still required to pay the capital gains/income taxes on all your income, home and abroad, in your home country.
Any tax experts feel free to edit for accuracy.
Comment: `Generally speaking, if you are a long term investor in non-real estate securities and spend no time in the US, your capital gains are not taxable.` This is not entirely correct, you will still be taxed, but the deduction will be done by your brokerage. And you will need to report it on your tax returns, in your resident country.
Comment: @DumbCoder no, not for capital gains.
Comment: @littleadv Yep, I was looking at you!  ;))
Comment: when filing taxes, how do you show that you are a resident of, say, Canada (which allows for only 15% dividend tax)?
Answer: If you are trading from outside the US you should pay taxes on any profits in the country you are trading from. You should check with the tax department in the country you are trading from.
Answer: A quick update for people finding this thread through Google. With the help of a few awesome Bogleheads, I compiled all the relevant research done into two Wiki articles:

Nonresident alien taxation
Nonresident alien with no US tax treaty & Irish ETFs

This includes comparing US to Irish domiciled ETFs, how to calculate tax withholding leakage and estate tax concerns.
Hope you find this useful.
Comment: Answers that consist solely of links to external sources are not well-suited to the site; as it stands this doesn't answer the question.


How to create a transparent mask in Love2D

Question: Since Love2D doesn't support SVG, I'm looking for other ways to create solid, compound shapes.  Anything that has straight edges can be drawn using a polygon, but I haven't yet found a way to mask an circle or ellipse, or in other words, subtract a circle or ellipse from the object being drawn.  Is this possible with Love2D?
Answer: I found the answer -- What I was calling a mask is called a stencil in Love2D.  See https://love2d.org/wiki/love.graphics.stencil and Make some black pixels transparent in Love2D.


How to create a hard link in Windows using mklink command?

Question: I've no problem in creating junctions and symbolic links through mklink, but can't create hard link using this command. It returns "Access denied", have tried on both Windows 7 and 8.1 on different machines, cmd started with administrative privileges. D:\>mklink /d /h dirA dirBOf course dirB exists. Is there something I should do to run this command without error?
Comment: If `/d` makes a symlink, and `/h` makes a hardlink **instead** of a symlink, how do you suppose to use them in the same command? At least, reading the current [documentation](https://technet.microsoft.com/en-us/library/cc753194(v=ws.11).aspx)
Comment: re-read the documentation, at least four time: /d and /h option are not compatible;. /d means create a directory junction point (a microsoft construct of ntfs, more closly relted to soft/symbolic links in \*nix), while /h means create a hard link, not a soft link. a hard *AND* soft/symbolic link is irrational as the idea as as polar opposites as good vs evil.
Answer: I'm pretty sure you can't create a hard link to a folder, only files. Symbolic links /D and junctions /J would work for folders though.
Also when creating hard links, keep in mind that you cannot create links between 2 different drives (even on the same physical hard drive).
A short explanation from another SU answer:

A hard link is a file system feature that cannot cross a file system boundary. You can't hard link files on C: to D: because they are separate file systems. They might each contain the same type of file system (eg. NTFS) but they are separate file systems.
Comment: Yes, thank you. You're right, it isn't working on directories, but hope it will be provided in the future as it has really important features.
Comment: it's named 'partition', a part of a disc
Comment: @jarek: it likely will never be added -- the technical issues and problems it create would require extreme amounts of logic (we'd need an AI greater than the Matrix to figure it out) and processing power (we'd need to network *and* __dedicate__ the world's computers all as one to compute it in hours instead of years). a comment lacks space to explain; search Google to understand the problems and issues and why their are no easily feasible solutions.
Answer: As David commented in reply to Braden, Junctions may not technically be "hard folder links," but I think one could say that "/J is harder than /D"
Case in point is that if you mklink /D "..\link_dir" "existing_dir", you will not be able to cd into "link_dir".  However, you can cd if you use /J instead.
Answer: No you CAN make a hard link to a folder/directory. It's called a junction.
mklink /J
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365006(v=vs.85).aspx
http://www.sevenforums.com/tutorials/278262-mklink-create-use-links-windows.html
Also, mklink /D creates a directory symbolic link, not a hard link. In practice, symbolic links are "fancy shortcuts" to files and folders, while hard links are sort of like a "file sync" for files only, where the "shortcut" is a "twin" of the target file; make changes to one, and you make changes to both...but in reality there is actually only one file physically on the drive, so if the target file is 500MB in size, only 500MB are being used. JUNCTIONS are hard links for folders and function the same as hard links.
Comment: Well I have used this numerous times and it works just fine as a solid link that syncs changes to the involved files. It may not be the same thing as a "hard link" but it does the job. There does not seem to be a lot of solid information on how links and junctions and such work in NTFS. Does anyone know a website or book that provides good information on NTFS?
Comment: Yes, and everything I read there indicates that Junction Ponts function in the EXACT same way as hard links. The only difference is a "hard link" is for files and a "directory junction" is for directories. It's purely semantics. Maybe I am missing something, if you are aware of something different please let me know.
Comment: Rubbish. A junction is not a hard link. It is something different (a soft link). From the link in your answer "**A junction** (also called a soft link) **differs from a hard link** in that the storage objects it references are separate directories, and a junction can link directories located on different local volumes on the same computer."
Comment: A bit late, but a major difference is if you delete the Source folder a Junction ceases to function, whereas a hardlink would continue to provide access to the files that were originally in Source.
Comment: @BradenDodge There's really a lot of information available: https://msdn.microsoft.com/en-gb/library/dn466522(v=ws.11).aspx#United Kingdom (English) http://www.ntfs.com https://en.wikipedia.org/wiki/NTFS https://msdn.microsoft.com/de-de/library/windows/desktop/aa365503(v=vs.85).aspx


Install OpenCV on Arch Linux for C++ and Python using `pacman`

Question: I want to install OpenCV (with GTK) latest stable version on my Arch Linux. I know how to manually build and install the library, but I don't want to go through all that. Is there a way to get it via pacman. I know that packages are available for qt but its causing issues for me. I am using Gnome.
Comment: Strictly answering your question, pacman lets you install packages from the official repositories only, without affecting how packages are compiled. You may search the [AUR](https://aur.archlinux.org/) for an alternative version or file bug reports about the issues you are facing with the official package or take advantage of the [ABS](https://wiki.archlinux.org/index.php/Arch_Build_System) to compile your own version. You may want to reword your question a bit, to clarify what kind of solution would be acceptable/preferable to you.
Comment: @fra-san I went through a long list of AUR packages but couldn't find any package with gtk support. But I am not sure about [opencv-git](https://aur.archlinux.org/packages/opencv-git/), it has gtk in its dependencies list. Also, I am not sure about its stability. Can you confirm?
Comment: Sorry, I'm unfortunately not able to comment on that, nor on other AUR packages.
Comment: I am using [opencv-git](https://aur.archlinux.org/packages/opencv-git/) package. I have tweaked the `PKGBUILD` file a bit to support `GTK` instead of `QT`.
Comment: Glad to know that! If that solution works for you it would be great if you could post it as an answer to your own question, to make it clear that your problem is solved (or at least that you've found something working). Thank you!
Answer: I couldn't find a package for opecv in either the main repo or aur that supports GTK by default. So, I decided to use the opencv-git. I added the required flags to the PKGBUILD file in cmake_opts which are:
-DWITH_QT=OFF
-DWITH_GTK=ON

and build the package just as usual.


Best TV brands for privacy

Question: Which are the best and worst smart TV brands with regards to advertising and data collection? I know of course to avoid Samsung and understand LG isn't much better, but how about Philips, Sony, JVC, Panasonic etc?
Comment: This is a very vague question. Define 'best'. Do you want to control which other shows are prioritised relevant to the ones you already watch? This information is already available to the channels you are watching, the TV doesn't need to do that. Advertising… what advertising? I don't see ads other than those during a show I'm watching, which I normally pause/skip anyway, or record to watch later. The trouble with blocking feedback is that you will lose other useful service aspects. Better to just use a good ad-blocker on your compy, so the TV data becomes irrelevant to your other activities.
Answer: I have a Sony smart TV, but I leave the Wifi non-configured.  Meaning, I don't use any of the Smart features.  Not sure you can beat that for privacy.  Hook up rabbit ears and nobody will know you're watching TV.
I've read reviews for LG and Samsung that claim you are required to activate the Smart features upon first boot, and/or are vague about being "allowed" to skip those steps.
In the end, if you really want privacy, buy the dumb versions of the TVs.  They do exist, visit your local electronics store and they'll help you.
However, I have an Amazon FireTV hooked up.  So my privacy may be undone.  It isn't the TV that is spying on me  :-D
Comment: Thanks but I want the Youtube and Netflix apps on there! Those apps are free to conduct telemetry and make recommendations. But like you say, there's no reason for Samsung etc to do so (other than the obvious in their own interest).
Comment: I see, so you're trying to pick the least evil.  Do you trust Google and not Samsung?  Google is in business of sharing your data.   I guess it comes down to researching which company you trust most and purchase from them.  Get an Apple TV or Fire Tv or Roku and buy a dumb TV.  Then you've cut out the middle-person.
Comment: I've not had any experience with these devices but Roku looks pretty good. I definitely appreciate a smooth, responsive interface (which my TV certainly does not deliver). My TV has broken but still under warranty so out for repair / replacement. So I probably won't be buying a new one yet but would just like a better understanding of which companies are best and worst for advertising, privacy etc.
Comment: Nothing on any hardware device I know of [AppleTV *might* be the best of the bunch for this, due to their elevated privacy functions] will block Google's insidious tracking. Better off using a computer with a serious ad-blocker for YouTube. Netflix, they can track whatever you do - you're paying them to track you, you can't watch it without signing in. I'd give up on the 'safest TV' idea altogether, unless you want to start running everything through a corporate firewall or a PiHole device.
Comment: Yeah I have a Pi (first gen so limited in what it can do) sitting around doing nothing so might look into PiHole (will need to look into other hardware that might need like wi-fi AP etc). As I say though, I'd really just like the knowledge (if anyone has it) as to which TV brands do best and worst. They're obviously not all exactly as bad as each other.

Best TV brands for privacy


How can I keep a wireless card's radio powered off by default?

Question: My desktop's PCI wireless card is always scanning for available wireless networks, but I only rarely use it. Can I keep the radio turned off until I need it?
Answer: "Modern" solution using Network Manager: simply uncheck the Wireless Enabled option at the Network Manager applet (KDE: Network Management). The command nmcli nm wifi off is equivalent. Continue reading if you sporadically enable Wi-Fi, but would like to revert it to disabled on reboot.
The wireless state is remembered in the file /var/lib/NetworkManager/NetworkManager.state. To disable Wi-Fi at boot, ensure that the key WirelessEnabled stays at false. You could do that by editing the init script of Network Manager, or by using the /etc/rc.local trick below. The command you need is:
sed s/^WirelessEnabled=true/WirelessEnabled=false/ -i /var/lib/NetworkManager/NetworkManager.state
rfkill block wifi

Put this before exit 0 (as described below). The rfkill block wifi command is still needed due to a race with startup of Network Manager (NM). Once NM has started, changes to the state file have no effect.

(old answer that involves editing file /etc/rc.local with an explanation of the rfkill command)
A bit hacky, but it should work. A wireless card can be disabled using the rfkill command. All devices used by rfkill can be shown using rfkill list. Sample output:
0: phy0: Wireless LAN
        Soft blocked: no
        Hard blocked: no

Hard blocked is dependend on a hardware setting, e.g. a wireless switch on a notebook. Soft blocked can be controlled by the OS (Ubuntu).
How does it work? It does not have a manpage, running rfkill provides a help text in this case:
Usage:  rfkill [options] command
Options:
        --version       show version (0.4)
Commands:
        help
        event
        list [IDENTIFIER]
        block IDENTIFIER
        unblock IDENTIFIER
where IDENTIFIER is the index no. of an rfkill switch or one of:
        <idx> all wifi wlan bluetooth uwb ultrawideband wimax wwan gps fm

Ah, now we get somewhere. You need to run rfkill block wifi as root to disable the wireless device. (wlan is an alias of wifi, see the source code of rfkill).
Now, if you want to disable the wireless functions at boottime, add the command to /etc/rc.local by running sudo nano /etc/rc.local. Use the arrow keys / page up/down keys to navigate to the line before exit 0 and add rfkill block wifi, so that the file end like this:
# By default, this script does nothing

rfkill block wifi
exit 0

When finished, press Ctrl + X, then press Y to save it and press Enter to accept the filename.
If you decide to activate the device later, run: sudo rfkill unblock wifi. Do not forget to remove the line from /etc/rc.local if you decide to use the wireless card.
Comment: You do not need sudo in `/etc/rc.local` as the script is run with root permissions. It's very unlikely you can disable wifi without root permissions. If I try `rfkill block wifi` or `rfkill unblock wifi`, I get "Can't open RFKILL control device: Permission denied".
Comment: I solved it with a delay: It seems like in Ubuntu 14.04 you need to wait some seconds before you disable wifi in `/etc/rc.local`. Use this instead: `/bin/sleep 10 && rfkill block wifi`
Comment: @rubo77 Here (Arch Linux, Network Manager 0.9.8.10-3), there exists a `/var/lib/NetworkManager/NetworkManager.state` file which persists the settings, `WirelessEnabled=false` is one of those. I'll post in your question..
Comment: Thanks. I don't understand why, but it looks like I don't need to use sudo for rfkill to work.
Comment: On both my ThinkPad X60 and my desktop with a PCI wireless card, running `$ rfkill block wifi` causes the NetworkManager applet to display "Wireless is disabled" and `$ iwconfig` to display `Tx-Power=off`. `$ rfkill unblock wifi` undoes this effect.
Comment: This doesn't work on my Ubuntu 14.04. If I have Wifi enabled by hand (in the network manager applet) and then restart the system, it gets enabled although i have that line in `rc.local`. maybe it is restored by the session somehow, that comes later in the boot process?
Comment: @rubo77 That is a possibility, recent Network Manager versions allow to toggle the wifi state too. You can try to add the `rfkill list > /etc/wifi-state.txt` command to see if the command actually had any effect.
Comment: I checked and rebooted, it seems like right after the call in rc.local it is bocked: `1: phy0: Wireless LAN
 Soft blocked: yes
 Hard blocked: no
` - but it seems to be unblocked shortly after again - lets discuss it here: [Turn off the Wireless card on each boot](http://askubuntu.com/q/490429)
Comment: @rubo77 That is a horrible hack, racing against some service (Network-Manager?). What is wrong with the Network Manager option to disable Wi-Fi?
Comment: Cause then you always have to **think about turning the wifi off** yourself. Either after boot or at the end of a session where you exceptional  needed the wifi once. I would be glad about a less "hacky" solution
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/15479/discussion-between-rubo77-and-lekensteyn).
Comment: The new "modern" solution doesn't always work: every now and then my laptop still unexpectedly boots with wifi enabled. [(I am in chat again)](http://chat.stackexchange.com/rooms/15479/discussion-between-rubo77-and-lekensteyn)
Comment: I think we have to use advanced logging for the upstart sequence to find out, which service is re-enabling and unblocking wifi after the first rfkill blocks it And why the `sed` command on the NetworkManager state file dows not always work
Comment: I tested it twice: if you add both lines to your `rc.local` it works to disable wifi: `sed s/^WirelessEnabled=true/WirelessEnabled=false/ -i /var/lib/NetworkManager/NetworkManager.state` and `rfkill block wifi`
Comment: Even the "modern" solution including both `sed` and `rfkill` lines does not work for me on either of two laptops. What I see in `/var/log/syslog` hints that Network Manager starts up, reads `/var/lib/NetworkManager/NetworkManager.state` and that between that read and its configuration of the wireless device, the commands in `/etc/rc.local` are run. That's too late for the `sed` to achieve anything and too early for the `rfkill` to achieve anything. Adding a `sleep 10` prior to the `rfkill` is a crude workaround, but I've posted an alternative answer to this question that I think is better.
Answer: There are at least two Upstart jobs that affect the default wireless state:

/etc/init/rfkill-restore.conf restores the soft block state for all radios to what they were at last shutdown, as recorded in /var/lib/rfkill/saved-state.
/etc/init/network-manager.conf starts Network Manager which in turn restores its idea of wireless state from /var/lib/NetworkManager/NetworkManager.state.

If you look at those two job configurations you will find that they have no temporal relationship, which seems like a design flaw to me. I'm guessing this race condition is rarely a problem because /etc/init/rfkill-restore.conf is much simpler and has fewer start conditions.
All the solutions to enforcing a wireless-off default that I've seen try to use /etc/rc.local, including the "modern" solution that @Lekensteyn and @rubo77 came up with. Unfortunately, that solution does not work for me on either of two laptops I have tried. This is not particularly surprising because /etc/rc.local also has no temporal relationship that I can find to either of /etc/init/rfkill-restore.conf and /etc/init/network-manager.conf. Throwing in a lengthy sleep in /etc/rc.local prior to issuing an rfkill block wifi is an ugly workaround for this race condition mess, but it works if the delay is long enough.
A better solution would be for us to impose our desired states in /var/lib/rfkill/saved-state and /var/lib/NetworkManager/NetworkManager.state before those two Upstart jobs are even permitted to run. We can achieve this by creating our own Upstart job. In actual fact, we'll need two job configuration files to achieve the timing we need.
Our first job configuration does the actual file modifications we need. It will run as early as possible and will only run once. Create /etc/init/radio-silence.conf with this content:
# radio-silence - Ensure radio silence on startup
#
# Override default startup behaviour of radios to ensure they are all
# disabled until the user deliberately enables them. This job requires
# radio-silence-wait to delay start of any services that may depend on
# resources manipulated by this job.

description "Disable all radios by default"

start on local-filesystems

pre-start script
  sed -i -re "s/^(.+[[:space:]]+)[01][[:space:]]*\$/\11/" /var/lib/rfkill/saved-state
  sed -i -re "s/^(WirelessEnabled=).*\$/\1false/" /var/lib/NetworkManager/NetworkManager.state
end script

As I prefer total radio silence when my laptop starts, I soft block all radios, not just wireless, but you can modify the first sed in the above to limit this job's impact to whichever wireless devices you wish to soft block.
Our second job configuration is responsible for ensuring that neither of the rfkill-restore and network-manager jobs will start before radio-silence has completed the file modifications. Create /etc/init/radio-silence-wait.conf as follows:
# radio-silence-wait - Helper task for radio-silence
#
# Delays the start of all jobs that may depend on resources manipulated
# by radio-silence job. Avoids the need to modify job configuration of
# those other jobs.

description "Assist radio-silence by delaying jobs it affects"

start on (starting rfkill-restore or starting network-manager)
stop on (started radio-silence or stopped radio-silence)

instance $JOB
normal exit 0 2
task

script
  status radio-silence | grep -q "start/running" && exit 0
  start radio-silence || true
  sleep infinity
end script

With this solution I am no longer seeing race condition problems, although I have not addressed the theoretical race between rfkill-restore and network-manager.
For more detail on how these jobs work together to achieve our temporal goal, refer to my question and answer, "How do I create a single-execution Upstart job guaranteed to complete before two other jobs begin?"
Answer: The easiest way to disable your wireless card is to right-click on the NetworkManager indicator (small icon on top-right on the panel), and untick the Enable Wireless. This brings down (ifconfig wlan0 down) the interface and it does not perform scanning anymore.
Comment: Checking this menu item does not affect the default behavior. The next time I turn on the computer, the wireless radio is active again.
Comment: At my Ubuntu 14.04 it stays off as desired after a reboot if I turn it of in the network manager applet
Answer: Note : If you use tlp, please read the entire answer.
All answers to this question are pretty old now and don't work on newer Ubuntu releases which use systemd. froage's answer worked for me on 14.04 but doesn't work on 16.04.
Systemd uses systemd-rfkill.service to save the rfkill switch state during shutdown and restore it on every boot.
You have to pass a kernel command line parameter to restore rfkill switch state on every boot.

open /etc/default/grub with your preferred text editor.
add systemd.restore_state=1 as a parameter to GRUB_CMDLINE_LINUX. That line should now read GRUB_CMDLINE_LINUX="systemd.restore_state=1".
You can add it to GRUB_CMDLINE_LINUX_DEFAULT as well. Either of them works. See this question for more details.

This will ensure that the rfkill state gets restored on each boot. Make sure to turn bluetooth and wifi off before restarting.

Issues with tlp :

Update : These issues were present in tlp 0.8-1 which is available in 16.04 repositories. After updating to tlp 0.9-1 using linrunner ppa, ALL isssues were solved.
Original answer:

tlp masks/disables systemd-rfkill.service to "avoid conflicts and assure proper operation of TLP's radio device switching options". ( Source-1 , Source-2 )
This means passing the kernel parameter won't work for you.
Here is a little snippet (around line no. 195) from tlp's default configuration file (/etc/default/tlp).
# Restore radio device state (Bluetooth, WiFi, WWAN) from previous shutdown
# on system startup: 0=disable, 1=enable.
# Hint: the parameters DEVICES_TO_DISABLE/ENABLE_ON_STARTUP/SHUTDOWN below
#   are ignored when this is enabled!
RESTORE_DEVICE_STATE_ON_STARTUP=0

# Radio devices to disable on startup: bluetooth, wifi, wwan.
# Separate multiple devices with spaces.
#DEVICES_TO_DISABLE_ON_STARTUP="bluetooth wifi wwan"

# Radio devices to enable on startup: bluetooth, wifi, wwan.
# Separate multiple devices with spaces.
#DEVICES_TO_ENABLE_ON_STARTUP="wifi"

As you can see the option to RESTORE_DEVICE_STATE_ON_STARTUP is
disabled by default. But enabling that option does not help.
Even after you enable the option to RESTORE_DEVICE_STATE_ON_STARTUP
, disable wifi and bluetooth (using rfkill block all) and keep
restarting, somehow WiFi get enabled on every 2nd or 3rd boot. There
is no guarantee that on next boot WiFi will be disabled. Surprisingly
tlp manages to keep bluetooth disabled on every boot.
Same goes for 2nd option in the snippet, DEVICES_TO_DISABLE_ON_STARTUP , which is also disabled by default.
      Enabling it doesn't work either. Network Manger shows that WiFi is disabled but rfkill list doesn't show any soft-block on
WiFi.

Note : I have read the line "Hint: the parameters
  DEVICES_TO_DISABLE/ENABLE_ON_STARTUP/SHUTDOWN below are ignored when
  this is enabled!".
I disabled RESTORE_DEVICE_STATE_ON_STARTUP before enabling
  DEVICES_TO_DISABLE_ON_STARTUP to avoid conflicts.

Similarly the other options provided tlp-rdw don't work as expected.

Solution for tlp users :
Update : Update to tlp 0.9-1 using linrunner ppa.
Original Answer:

Enabling DEVICES_TO_DISABLE_ON_STARTUP and setting it to disable Bluetooth and WiFi may work for you.( It worked for this person )
This question here on askubuntu. It is similar to an older answer to this question. But please note that I haven't tried it myself. It may or may not work.

Other sources : systemd-rfkill , tlp-configuration
Comment: I tried your answer involving the modifications of both `GRUB_CMDLINE_LINUX` and `GRUB_CMDLINE_LINUX_DEFAULT` plus the (unmentioned) `sudo update-grub` command and I couldn't get it to work in Pop_OS! 19.10 (which is based on Ubuntu 19.10). According to `systemctl status systemd-rfkill`, this service is already being run on every boot and, since the default value is `1`, there should be no need for this effort. Based on the info at `man systemd-rfkill`, remembering the radio state "at early boot" should be the default behavior, which leads me to believe that this component is somehow broken.
Answer: Personally I use Jupiter the power management applet to disable the wireless on my laptop as it is controlled by a special button. I don't think is in the main repo I had to add a ppa from Andrew at Webupd8 to get it.
Hope this helps.
Answer: You could add
ifconfig wlan0 down
to /etc/rc.local, but first,
ensure that, if you're using NetworkManager (I have version 0.8.4~git.20110319t175609.d14809b-0ubuntu3), you go to menu "Edit Connections" -> "Wireless" tab, click on the connection, click "Edit", and make sure that, in the "Wireless" tab, the "Connect Automatically" checkbox is NOT checked.
Comment: While this will bring the interface down, it won't save as much power as disabling the radio using `rfkill`.


download binary file from database

Question: I have a binary file in database.
How i can make a download of this file in header of page when a button is clicked
Any ideas??
Comment: What do you mean? a progress bar in the page's title?
Comment: Yes the i have read the file from database, i wont just now to download the file.
Answer: Well, if you're generating the file with a script, the most common way a script says it's outputting is to send an HTTP header that looks something like this:
Content-Type: text/html; charset=utf-8

Which will make the browser display as text. In order to force [most browsers] to download the file, you need to send the following additional headers:
Content-Type: application/zip
Content-Disposition: inline; filename=test.zip
Content-Length: 8192

[FILE CONTENTS]

as an example for an 8kb ZIP file named "test.zip". If you can find some way to change the headers sent using the script that generates the file (i.e. pulls it from the database) then it will most likely force the browser to interpret as a file to download.
Then, just link to the script in your header.
Comment: Ok but the file is in a object, this is the problem
Comment: If it's in an object defined for the page that is generating the link, it's going to be a bit of a problem. Basically, the way browsers do things necessitates that it be redirected to a separate page in order to initiate a file transfer.
Answer: Alright, there's a couple of things you might be trying to do, its not completely clear but I'll take a stab at it. Let's say you've got the following basic database comprised of a file identifier and the binary file information:
CREATE TABLE Test
(
FileId int identity(1,1) PRIMARY KEY,
FileData varBinary(max)
)

What you need is a page (in this case called "GetFile.aspx") with the following in the the code-behind's Page_Load event:
    'Make sure we've got a querystring
    If String.IsNullOrEmpty(Request.QueryString("fileid")) Then
        Throw New ApplicationException("Missing querystring parameter FileId")
    End If
    Dim FileId As Integer
    'Make sure the querystring is an Int
    If Not Integer.TryParse(Request.QueryString("fileid"), FileId) Then
        Throw New ApplicationException("Malformed querystring parameter FileId")
    End If
    'Change YourDsnHere to point to your database
    Using Con As New SqlConnection("YourDsnHere")
        Con.Open()
        'Select the file
        Using Com As New SqlCommand("SELECT * FROM Test WHERE FileId=@FileId", Con)
            Com.CommandType = Data.CommandType.Text
            Com.Parameters.AddWithValue("@FileId", FileId)
            Using RDR = Com.ExecuteReader()
                If RDR.Read Then
                    'Get the data out as a byte array
                    Dim Bytes() = DirectCast(RDR.Item("FileData"), Byte())
                    'Clear all response headers
                    Response.Clear()
                    'Set the type, my sample is a RAR file, you'll need to look up MIME types for yours
                    Response.AddHeader("Content-Type", "application/x-rar-compressed")
                    'Set the name of the file, if you don't set it browsers will use the name of this page which you don't want
                    Response.AddHeader("Content-Disposition", "inline; filename=YourFilenameHere.rar")
                    'Optional but nice, set the length so users get a progress bar
                    Response.AddHeader("Content-Length", Bytes.Count.ToString())
                    'Push the bytes out raw
                    Response.BinaryWrite(Bytes)
                    'Close the response stream so no more HTML gets accidentally pushed out
                    Response.End()
                Else
                    'Error, no file was found
                End If
            End Using
        End Using
        Con.Close()
    End Using

Then in your page just when the person clicks the button direct them to this page making sure to pass a valid fileid in the querystring.
Like I said, I think this is what you're looking for, if not, clarify your description above

download binary file from database


Motion tracking no longer working on 6S

Question: I recently purchased an iPhone 6S, and the motion tracking worked for a few days, but then in the past four days it has recorded no steps or flights climbed. This morning, the phone started recording data again, but only briefly (maybe 2 hours); it currently is not collecting any motion data. I've been using Apple's Health app and the FitBit app. I tried turning motion tracking off and then back on, restarted the phone, no luck. I'm on iOS 9.0.2. Any ideas?
Answer: Try restarting your iPhone by holding the lock button and home button together until you see an Apple logo appear. If that doesn't work, I'd attribute that issue to a bug. The iPhone 6S is kind of buggy, mine won't play system or Taptic Feedback sounds sometimes. So we may just have to wait for software updates.

Motion tracking no longer working on 6S


Is a 1 hour layover in Amsterdam sufficient?

Question: I found a great price to Geneva (to visit a friend), however the itinerary I found calls for a 1 hour layover (only) in Amsterdam. 5:55am-6:55am. What is the likelihood of missing the connecting flight, since we have to go through customs and whatnot?
I fly abroad pretty frequently, so I know this is an extremely thin margin. I wonder if anyone knows about the speed and efficiency of Amsterdam, or what it will be like that time of day.
EDIT: I want to note the flight is incoming from the USA, not anywhere in the Schengen zone.
Comment: Are the two flights on one ticket, or separate tickets?
Comment: Check the history of arrival times of your flight on http://flightaware.com/. Often airlines take a large margin on arrival times, to limit paying penalties for being delayed. Last time I traveled transatlantic, my flight arrived 45 minutes early
Comment: A small point - you'll just go through immigration and not customs. You'll go through customs at Geneva.
Comment: Where are you flying from?
Answer: If this is a single ticket, and the airline is willing to sell it to you with that connection, then any financial risk is theirs. If your transAtlantic flight gets in late, or the airport is crowded, then they will have to put you on their next flight. If your luggage doesn't make the connection, then they will have to compensate you for that somehow. So if you don't mind waiting a bit in Schiphol for the next flight, and waiting a day or two for lost luggage to catch up to you, that's your only risk. I'd probably be willing to take it.
If this is two independent tickets, DON'T DO IT. All the advice in the other answers about the chance of missing the second flight is true, and you would have to pay anything up to a whole new ticket to get yourself to Geneva.
Comment: Note that they will normally put you on the next *available* flight.  It could very well be that the next few flights are sold out, in which case you'll wait longer.
Comment: YMMV, but unless there's something massive going on, it's highly unlikely that that would actually happen, and if it did the airline would be responsible for your expenses in the meantime.
Comment: 'and the airline is willing to sell it to you with that connection' - Is this also true for tickets that a travel desk booked for you (using one of the travel websites)?I am sorry but my adrenaline is on a rush right now and asking dont hurt
Comment: @jpatokal: I've had it happen that the next flight actually wasn't until the next day. The airline did pay for the hotel.
Comment: Yeah, I'm not sure this is great advice in practice. If you think there's any significant chance of delay, and you don't have status with the airline, or in some other way make them care about you, there's no guarantee you'll be leaving Amsterdam any time soon - their liability to you is very limited. Personally I wouldn't risk it if you think there's a chance of missing it (and one hour is not long, considering the long-haul nature of the incoming flight - a half-hour delay would be enough to break the connection).
Answer: 1 hour between connection flights is strongly NOT recommended for many reasons--you can not make sure the first flight will arrive in time, even if it does there are issues could arise from the airport itself.
Assuming your luggage arrives on time and you know Schiphol airport very well and you will directly go from the arrival gate to the departure gate, There are many factors that can cause a late arrival and most of these are out of your hands, some of these reasons would be:

Late departure. If you depart late, you will arrive late!
Weather. The best weather analyst still can not predict the weather!
Air traffic jams. In the busy airports (such as Amsterdam) this is common when pilots are asked to hold position due to jammed skies.
Airport Gates. Any delay or technical problem in any airplane parked at a gate will cause a chain delays for some time for all airplanes scheduled for that gate.

My advice is to make a better booking even if it will cost more but it will save you from more headache. Unless your both flights are operated by the same airlines/alliance and in one booking then if something happens and you lost your connection flight then its the airlines problem and they will provide accommodation until they provide you with another flight to your final destination.
Comment: I've never connected in AMS but I've done so in Munich many times, coming from North America, and 20 minutes would be enough on some occasions. Blanketly saying an hour is not enough is too strong.
Comment: @KateGregory It is indeed doable with this 20 minutes.. but not recommended if you want to be in the safe side. I do not mind booking a flight with 2 hours (if available) time between connection flights rather than 30 minutes and having a higher chance of missing my flight.
Comment: I do not agree with that statement, especially regarding AMS. If the flight is on time, 1 hour is plenty of time for well-organized AMS. Late flights are rare (but check stats for your flight) and company's responsibility, assuming this is the same ticket.
Comment: Any day a 6 AM flight is likely to be delayed due to mist/fog, which can happen almost any day of the year and is quite common early mornings in many months.
Answer: Amsterdam Schiphol is one of the busiest airports in Europe, and long queues in front of security checks are not unusual -- once I waited around 30 minutes just to get through passport control. Granted, it was a major holiday, but still something to consider.
Apart from that, the airport is quite efficient -- there is only one terminal, and everything is laid out around it. One hour should be enough bar unusual circumstances. For this early hour, I won't be concerned at all. See the airport map with walking distances -- you should be able to go from one end to the other in 20-25 minutes or so.
If you arrive from a Schengen country then you won't be even be subjected to additional security check when you transfer at Amsterdam.
One more thing to consider: KLM managed to delay my baggage 2 out of 3 times I flew through Amsterdam, and all of these were with layovers of similar length as yours. They do have a nice return policy and sent me some vouchers for the inconvenience.
Answer: If you have a single ticket, then the airline is liable if you miss your flight/something happens to your luggage. It's not ideal, but I think it's a risk worth taking. From my experience of flying into Schiphol at 6am from across the Atlantic, I was out of the airport within 30 mins - no lines ANYWHERE, incredibly speedy customs (at that time of the morning). The airport isn't too large - you should be able to make it as long as you pay attention to the signs since as it will likely be deserted off staff at that early hour so no one to ask for directions in case you are lost. Good luck!
Answer: I personally think that at this early time you should make it. Because Geneva is in Schengen area, you should not be subject to passport control (assuming you are flying from another Shengen country), there is only one terminal so you don't need to change terminals.
For these short layover staff often waits in front of the first plane to help you get to the second place. And even if you miss it, they will rebook you for free.
Having said that, from personal experience, there is really big chance that your luggage is not going to make it. Last time I changed in Amsterdam they lost it both ways, and I was told it happens often if the layover is less than two hours.
Comment: @mindcorrosive Interesting... I've flown via Amsterdam at least a dozen of times and they never lost my luggage.
Comment: I agree about the baggage -- KLM managed to lose mine *twice* out of three times when flying through Amsterdam.
Answer: I did note last time that customs had express lanes for transit passengers, showing outbound flights which would leave soon. If your inbound flight is late, chances are that your outbound flight will be listed there so you can skip the queue.
(Normally 50 minutes should be sufficient)
Answer: I've transited through AMS in the morning (this is usually when the international rush hour starts).
1 hour is very tight, the problem is that even if you were to leave and arrive on time, there are lots of other flights arriving at this window (due to noise abatement and other restrictions, there are very limited number of landing slots available to airlines from 2200-0600, thus the morning rush hour starts from 6 AM); and so the queue to pass immigration for intra-schengen flights can get very long easily taking up the majority of that one hour.
However, AMS staff is very helpful and will move you to a priority lane if you are at risk of missing your flight due to the queue length. Keep and eye and ear out for your flight being called.
The next problem is the weather especially fog. In the winter this can be a problem and cause a chain reaction of delays; so that even if you are at your gate on time, the incoming aircraft is delayed.  Not really an issue if you are at your final leg - after all, eventually you'll end up in Geneva the only issue is you'll arrive later.
The problem comes that now the airport is very crowded with people who have missed their connection so everything becomes an issue - finding a place to sit, for example. The bathroom queue.  Trying to get a snack will be an issue unless you like to stand and eat.


Sending data from Android client to database

Question: I have created an Android app which sends data to a web service. Right now the user sends his test results (test id, test name, test time, and user ID). The problem is that I send the data from the client to the web service in JSON; the web service can be accessed in the browser. However, an unauthorized user could use his browser to call the webservice method for inserting the score. How can I solve this problem? I need to let only the Android client send the data to the webservice.
Answer: In general, this can't be done.  If the user has access to the app, they can reverse-engineer it and undetectably emulate everything it does.
What you can do is put obstacles in their path.  Simply transmitting a salted hash alongside the data will stop 99% of would-be attackers; if you're willing to go to greater effort, using public-key cryptography to encrypt and sign the message being transmitted will stop almost everyone.
This won't stop or even significantly slow a professional attacker, but if your primary threat model is "college student majoring in art history" or "casual Facebook user", it's more than adequate.
Comment: As this is the correct answer let me put here how to do it correctly - approach the application like it is a web browser. It doesnt need direct writing access to the database like a super user, instead establish an authentification system where some people can modify things and the public cannot.
Answer: Call the sendPostRequest(String username, String pass) method with parameters. You can call this method from the UI thread, the request will be sent from a different thread (AsyncTask is embedded).


Open CDialog from Java using JNI

Question: I have an application that needs to open a CDialog written inside a DLL. The problem is that when DoModal() is called inside the DLL, the program crashes with: "Debug Assertion Failed!". Does anyone have any idea how can i get the CDialog opened? Thank you!
Answer: At this point you would probably need to do some debugging at the C++ level - there isn't really enough to go on here to tell what might be going wrong. The kind of things you need to look at are what the debug assertion was that failed, the stack trace at the time, what it was trying todo, and what caused the crash. A debug assertion by itself is not a crash, although it is a good clue, and the cause of the failed assertion may also be the cause of the crash.

Open CDialog from Java using JNI


How can I level up in Dark Souls 2?

Question: I've been to several bonfires, and I can't seem to level up.  Now I'm too afraid to go anywhere and die, because I have 2,500 souls and don't want to lose them.  I'm assuming there's some new mechanic in Dark Souls II where you can't just level up at any bonfire.  Help!
Answer: You won't be able to level up at all until you get past the Things Betwixt area -- essentially, the tutorial.  Past that and through a narrow cave is a town called Majula.
When you first enter Majula, there is an area to the right which has a tall tower overlooking the sea.  To the right of that, slightly down the cliff and by a tree, there is a green-cloaked woman known as the Emerald Herald.  Once you speak to her, she will relocate to the Majula bonfire.  Thereafter, you can speak to her and level up at that bonfire.  Unlike Dark Souls, this is the only place you can spend souls to level up.
When speaking to her at the Majula bonfire, be sure to wait for a menu to popup.  You may have to speak to her several times before the option becomes available.
As a side piece of advice, if you want to save your souls until you can first spend them, don't go down into the area with the two trolls.  This is a difficult encounter for a new character.  You can bypass that and come back to it later to fight them.
Comment: If you don't get the option right away, exhaust her dialog options.
Comment: Video link (not mine): Completed Dark Souls (1) without leveling: http://www.youtube.com/watch?v=VpH3al-kGow Should be subtitled "or how I learned to stop worrying and love Dark Souls"

How can I level up in Dark Souls 2?


Unable to receive sign request on Subsocial

Question: I'm trying to post on Subsocial but when I hit "Create Post" the sign request isn't popping up. I have tried logging in and out, changing the sign request type, disabling and enabling the extension, updating metadata, and I have been all through the extension trying to get the signature request to pop up. I do have a ping next to the extension logo saying "1" with a red square, signifying that I have a sign request but for some reason I can't access it. Thanks for any and all help!
Answer: When you first visit the site you'll be asked to 'Auth' the site, was this done?  If not please verify that the site is allowed in Manage Website Access.
Comment: Hey @paradox-paranodes-io , good to see you here! I think your answer is more of a clarifying Q/statement that should be in the comments to the post, not the answer directly
Comment: yep! If i go under "Manage Website Access" app.subsocial.network is enabled
Answer: It sounds like Subsocial is prompting for the signature, but you're having trouble seeing the pop-up.
When the Polkadot extension notifies you that there's a pending extrinsic to sign but you don't see the window that's usually because of either of these two reasons:

Your browser is blocking the pop-up window of the extension
You're using more than one screen and the pop-up is on a different screen.

I hope this helps!
Answer: Probably a dumb question, but have you disabled all ad-blockers for the site? That might be interfering with some js... (also you didn't mention which browser / OS you were having trouble with)


Life of Sajah bint Al-Harith ibn Suayd

Question: Sajah bint Al-Harith ibn Suayd declared her prophethood during the apostasy movement and even married Musaylimah. However, when Khalid bin walid crushed the movements around Sajah, she converted to Islam. Is there any account of her life after her conversion to Islam ?
Answer: she got married to a sahaba the story of her being a muslim and marrying a sahaba is  mentioned in a controversial book
  الروض العاطر في نزهة الخاطر
which is a fifteenth-century sex manual
I am not sure how authentic is the story, i am sharing what i know i may be wrong
Comment: Is that book really pretending she married one of the sahaba, could you share a quote? None of the known historians said more than that banu Taghlib -Sajah's tribe- decided later to convert to Islam (they seem more close to Christianity) and that she died around 55 a.H. as a good Muslima.

Life of Sajah bint Al-Harith ibn Suayd


Why was Boba Fett's blaster different in Episode 5?

Question: In Episode 5 Boba's EE-3 was almost entirely different to the one in Episode 6. Is there any canon explanation for this?
Comment: the old one was made by samsung... It exploded
Comment: The simplest explanation could be he has more than one blaster. Just like how many people own more than one car, or more than one pair of shoes, or more than one necktie.
Comment: Guns enthusiasts have guns.  Guitar enthusiasts have guitars.  Camera enthusiasts... Is a bounty hunter carrying a different weapon really so perplexing?
Comment: For me, a more interesting question is how his rifle changed *during* the Empire Strikes Back from [one with a short stock to one with a longer stock](http://www.bobafettmp.com/bobafett/bweapon/bweapon-old.html).
Answer: Boba Fett owns at least two of the EE-3 blasters. Wookieepedia says

The rifle consisted of a small handle attached to a long circular barrel. An alternate version of this model features a modified scope mounting and hand grips that run along the length of the barrel.

These are the descriptions of both of the weapons we see in the Empire Strikes Back and Return of the Jedi, meaning there is more than one EE-3 model and Fett owns both. Since he is a bounty hunter, and prone to losing weapons, this makes a lot of sense. The canon has no more on it that I could find.
One site I found said that George Lucas wanted the bounty hunter to appear more threatening in Return of the Jedi, and gave him a more imposing weapon to match, but this site also misspelled "shark". twice.
That is all I could find on it, although it is possible that a fanpage as in depth as this could come in handy if you have time enough to search it.
More sources:

https://starwars.fandom.com/wiki/EE-3_carbine_rifle/Legends
http://www.bobafettmp.com/bobafett/bweapon/bweapon-old.html
Answer: The (or at least one) out-of-universe reason for the change was that the original prop from The Empire Strikes Back could not have been used in this scene.

During the sarlacc battle in Return of the Jedi, Luke chops the gun in half to prevent Boba Fett from simply blasting all the rebels.  Something like this was probably necessary, from a dramatic standpoint; for while it may have seemed reasonable for the kind of guards employed by Jabba the Hutt to be sufficiently poor shots that Luke could take them all out with no casualties, it would have been harder to believe that Boba Fett—a character whose appearance in Episode V had already transformed him into a mimetic badass—would keep missing the good guys.
The original prop had a barrel of thick metal, as can be see here in this still.  So the barrel of the prop needed to be redesigned to accommodate it being cut apart.

As part of the redesign and modification, the barrel was made longer and thicker, producing the overall effect of a more powerful weapon.


Expansion in large and small limits

Question: Let $$f(x) = \frac{1}{\log(\frac{x}{c})}$$ where $c$ is some constant number. Consider the variable $x$ in the large regime where $x \gg c$ and small regime where $x \ll c$. How would $f(x)$ depend on $x$ in those regimes? Would one use some kind of Taylor expansion?
Answer: In the examination of the two limits of your function, we should probably start with the identity $\log{\frac x c} = \log x - \log c$. We can then identify the largest of these two terms, and make a series expansion in the ratio of the smaller over the larger term. Assuming for convenience that $x > 1$ and $c > 1$, this procedure yields:
$x >> c: f(x) = \frac 1 {\log x - \log c} = \frac 1 {\log x} \frac 1{1 - \log c/\log x} $
Expanding the term between square brackets gives
$f(x) = \frac 1{\log x} \left[1 + \frac{\log c}{\log x} + \left(\frac{\log c}{\log x}\right)^2 +\cdots \right]$
$x << c: f(x) = \frac 1{\log x - \log c} = -\frac 1 {\log c} \frac 1 {1 - \log x/\log c}$
Once again we expand the term between square brackets.
$f(x) = -\frac 1 {\log c} \left[ 1 + \frac{\log x}{\log c} + \left( \frac{\log x}{\log c} \right)^2 + \cdots \right] $
So it is indeed possible to expand your function in the given limits. However, I doubt the result has much practical use. In particular since the original function is very simple.

Expansion in large and small limits


Administrator waiting for processes to finish

Question: I got my first job as an administrator. Basically I have to install operating systems and software to computer. The problem is that I have sometimes nothing to do but wait for process to finish. It could take minutes or hours. What can I do during those breaks to be useful worker?
Comment: Read a book, get more stills - or ask what other tasks can be done.
Comment: Post questions on Workplace.
Answer: Look into streamlining your installs so they can be unattended, multi task your other duties. I have up to 5 OS installs going simultaneously many times, with one keyboard mouse and monitor on a switch on my side bench.
If I have nothing else to do, I surf the internet while I wait like I'm doing right now. The guy in the office next door plays his guitar. The young chap in there with him earnestly watches the blue progress lines like his life depends on it.
In other words do whatever you want, if the process waiting is billable hours don't do anything during that time that is billable or you end up with 20 hour days, which is great for the company but may become an expected 'norm' for you. But quite often there is non billable stuff you can do like organise your desk and check emails, tools, research etc,.
Comment: +1. Sysprep'd images & automated/unattended installs. Setting up a new computer should be streamlined to take almost no effort or handholding.
Answer: There are a few direct things doable for work, but sometimes they don't fill your available time. So look for indirect things to do.

Indirect can be preparation for the next tasks(I don't know anything about your admin job, but it can be making some scripts for checking everything is OK, or automating tasks).
Indirect can be team improvement, like writing documentation, training people, etc. opportunities to do that can vary a lot between shops & positions. As you seem to be new, it's probably not much(cool for you if I'm wrong).
Indirect can be self-improvement. Learning a new programmation language, checking the Workplace ;) , preparing yourself for any challenge that you could meet when you are not idle. Of course, it means you have access to a working computer during the install time. Depending on how your work is set up...

If not, try to have a small laptop for you, even an old crap, so that you can keep doing things while the install is running. If you can use it for things directly related to your job, perfect(as checking the network, the machines status, whatever).
NB : when it lasts only a few minutes, do not even try to be productive. The interrupt comes too quickly, and is highly demotivating. Otoh, when it's hours, you really have time to learn new things(like some kind of Linux scripting you haven't tried yet, or whatever).


Does the string of prime numbers contain all natural numbers?

Question: Does the string of prime numbers $$2357111317\ldots$$ contain every natural number as its sub-string?
Comment: It's unlikely to contain π as a substring, because π contains a decimal point and this string doesn't. Do you mean "every integer"?
Comment: See [Copeland-Erdos constant](http://en.wikipedia.org/wiki/Copeland-Erdos_constant).
Comment: Possible duplicate of [Proof that there are infinitely many prime numbers starting with a given digit string](https://math.stackexchange.com/questions/60825/proof-that-there-are-infinitely-many-prime-numbers-starting-with-a-given-digit-s)
Comment: No minus signs, @gnasher729. So has to refer to non-negative integers. :)
Comment: @Buddha, you didn't mention natural numbers, so change your question. And extending it to reals would be stupid because your string doesn't contain a decimal point or a minus character, so it doesn't contain the numbers 1.5 and -2, for example.
Answer: It follows from Dirichlet's Theorem.
If $d$ is the number we want to find, define $s=10d+1$.  By definition, $\gcd(s,10)=1$ and $s$ contains the digits of $d$.
Dirichlet's Theorem's implies there's a prime of the form $p:=s+k \times 10^n$ where $10^n$ is chosen so that it has as many zeroes as digits of $s$.  The digits of $d$ appear in the digits of $p$, and thus in the given string of primes.
Comment: Very nice! This is actually exercise 4.16.b in Apostol's Introduction to Analytic Number Theory.
Comment: You should add that $k\in\mathbb N_{\ge 0}$.
Answer: If you mean "every non-negative integer", then the answer is yes.
First, it contains the integer 0 as a substring because 101 is a prime.
Next, for every integer i > 0, there is a prime that starts with the integer i: Take the known results about gaps between prime numbers, for example that there is always a prime between $n^3$ and$(n+1)^3$ for large n. There is always a large n such that both $n^3$ and$(n+1)^3$ start with the digits of i, therefore the prime between $n^3$ and$(n+1)^3$ also starts with the digits of i.
(This is true because one of i, 10i and 100i is not the cube of an integer).
Comment: @tohecz Yes, it actually means there's no need to string the primes together, simply $2,3,5, \dots$ will contain all natural numbers somewhere.
Comment: I love that you have two solutions here: you show that every $k$ is a prefix of a prime written in the decimal system, the other answer shows that every $k$ is a suffix of it :)
Answer: Let $n$ be a natural number, and let $\mathcal N(n)$ be the set of natural numbers not containing $n$ as a substring when written in base 10. It is well-known that $\sum_{k\in \mathcal N(n)}\frac{1}{k}$ converges.
Let $\mathcal P$ be the set of prime numbers. It is also well-known that $\sum_{p \in\mathcal P}\frac{1}{p}$ diverges.
Therefore it cannot be the case that $ \mathcal P\subseteq\mathcal N(n)$, and $n$ is therefore a substring of some prime (of infinitely many primes, in fact).
Answer: It follows from the fact that the Copeland-Erdös constant is a normal number.
For any natural number $N$ with $k$ digits, $N$ appears in the Copeland-Erdos constant with a natural density of $10^{-k}$.


Converting PWM to DC voltage

Question: I am trying to convert an Arduino PWM signal into variable DC voltage source. This will be done by varying the duty cycle output from the Arduino. My question arises when I consider building higher order filters. Will a higher order filter (Sallen-Key, Butterworth, etc.) work better than just a basic active RC filter? I want the output signal to be as "DC" as possible even at low duty cycles.
Comment: Yes, generally a properly designed higher order filter will work better than a lower order one in the sense of ripple vs. response time for a given PWM frequency. But you need to define how much ripple is acceptable and how fast it needs to respond. "as DC as possible" is not a specification.
Comment: The arduino PWM signal appears to only be around 1kHz, so you'd need a huge inductor for your filter.  It might be better to use an analog voltage output to drive a faster PWM driver from a 555 or actual voltage regulator.
Comment: One option is to make your own PWM function with higher frequency which would reduce the need for higher order filters. See link for default Arduino PWM when using "analogWrite" function [link](https://www.arduino.cc/reference/en/language/functions/analog-io/analogwrite/)
Suggest to use some type of active buffer (depending on load), and not only passive components.
What type of output are you driving and what is the Arduino type?
Comment: 3rd order Bessel using a single opamp or two ( one bigger) or even a single transistor (with accompanying 0.6v voltage drop) can be suitable for many needs.
Comment: @Turbochad:  Look into using the [*TimerOne* library.](https://www.arduino.cc/reference/en/libraries/timerone/) You can set a much higher PWM frequency ( which makes it easier to filter down to "clean" DC) and you get 10 bits (1023 steps) for your PWM duty cycle.
Answer:
Will a higher order filter (Sallen-Key, Butterworth, etc.) work better
than just a basic active RC filter?

Yes it will. It'll be better because, as the "order" of the filter increases, the PWM switching artefacts seen on the output get progressively more reduced in amplitude: -

Picture from here.
So, in the above diagram you have a pass band where the frequency response is maximally flat - this covers DC to some cut-off frequency. Above that cut-off frequency you want to attenuate AC components of the signal and ideally, your PWM frequency will be at least ten times the cut-off frequency.
So, with a first order filter you get 20 dB attenuation but, with a 2nd order filter you get 40 dB of attenuation at ten times the cut-off frequency.
For a 3rd order filter you get 60 dB attenuation at the cut-off frequency. In case you didn't know, that's a real-number voltage attenuation of 1000:1.
Answer: Someone might have an idea how to squeeze a better PWM frequency out of an Arduino, but if it is limited as it appears to be around 1 kHz, the switching frequency is working directly against your goal of "As DC as possible".  You would need a huge output filter compared to a higher but still easy to work with switching frequency. Aside from the physical size and cost, you need a larger current surge to initially charge the output capacitor and you get a slower response to source voltage or load current changes.
You could use an analog output to control a voltage controlled voltage or current regulator as probably your best option (you could set it up so your reference voltage controlled 0-100% duty cycle or so that the converter just tried to match your reference voltage).  This will put much less processing load on the Arduino than manually controlling an output for duty cycle.  Even if all you wanted was a meager 100kHz switching frequency, unless all the Arduino is doing is controlling the voltage regulator, you wouldn't want to interrupt it 100,000 times a second from more important tasks without a good reason.  The point of those PWM outputs is to automate control of the output, and if they found that worth doing at a meager 1kHz, it will be well worth doing at 100kHz.  I'm only specifically aware of ones used for current control in LEDs, but I wouldn't be surprised if you could find PWM drivers or voltage regulators capable of much higher frequency that could be controlled over an I2C bus or something.  Might be worth investigating as you could control multiple converters over one pin.
If you're trying to build your own PWM regulator in order to learn, you could try the following, it's worked well for me although it doesn't respond nearly as well/fast as a dedicated chip, it's pretty easy to build up to around 100-200kHz.
First build a triangle wave generator at your desired frequency(op-amp integrator version is easy at low skill level, 555 timer version is easier but less linear) and a voltage reference (you can use a potentiometer if you want it adjustable, but it's getting removed anyway).  Use a comparator to turn ON the PWM switch if reference is higher than the triangle wave.  This gives you a voltage controlled duty cycle PWM drive.  It won't do a very good job with a variable load because it is blind to the output though.
Now replace the voltage reference with a capacitor to ground and set up a comparator or op-amp to compare the output voltage to a new reference voltage.  If the output voltage is lower than the reference voltage, charge the capacitor.  If the output voltage is higher than reference voltage, discharge the capacitor.  This turns the voltage controlled duty cycle drive into a voltage controlled voltage converter.  It has the advantage of taking almost all the processing load off the Arduino.  If you do the comparator version I recommend setting it up so it discharges faster than it charges to minimize overshoot on startup and load spikes.  If you do the op-amp version you have to do a bit more learning, but you can more easily set it up so it adjusts the reference capacitor in proportion to how far out the output voltage is, giving you a faster response.  As far as I could figure out, you need two op-amps, one to calculate the magnitude of the output discrepancy, and a second to relate that to the current voltage on the capacitor.
One more option if you're good at calculus and advanced math, you can figure out the actual mathematical relationship between output current, voltage and desired duty cycle and build an analog op-amp circuit to calculate that on the fly.  You get better results and faster response at the same switching frequency than with a simple auto adjuster, but the skill level required is high.
Now as a final step that uses more processing power for a better result, you can take out the op-amp or comparator and have the Arduino provide the reference voltage for duty cycle.  Give the Arduino the output voltage sense input and also a current sense input.  Now you can build a more advanced algorithm that uses current sense to predict changes in output voltage and respond to them proportionally and aggressively.  If you have the calculus/math skills mentioned above you can do this in direct math rather than having a system that responds to changes.  If you have some of those math skills, you can do a pauper's version where the algorithm interpolates between values in a multidimensional lookup table.  Doing so can decrease processing load at the expense of memory use and if the values of the lookup table are filled experimentally, you can bypass the need for calculus.  This also lets you add features like current limiting, current control or overvoltage/overcurrent cutoff.  Adding more inputs could give you temperature cutout, source brownout protection, or whatever else, but it's worth noting most features you would really want can be built by default into modern voltage regulator chips, saving your Arduino for more important tasks.
Now you've learned what you can learn more or less, so you might want to move on to dedicated voltage regulators/chips that will provide a better result with lower power consumption and the convenience of a modern (MHz) switching speed.  You can still use your Arduino to control them, so for example if the 0-100% voltage regulation was to drive a Peltier apparatus in an intelligent way, that might be an actual good application for using a microcontroller because of the multiple inputs and complex efficiency metrics (separate control of peltier and fan based on hot side temp, cold side temp and ambient if necessary).  Even for this type of application, an Arduino is probably a huge overkill, but it's an example where you can offset the overhead power loss of running the microcontroller with large efficiency gains.
Sorry I don't know much about advanced filters, but I though you might want to start by making your task easier in the first place.  If you have the skills to design efficient high order filters or active filters, you probably also have the skills to not need them for many applications.

Converting PWM to DC voltage