Plotting Average Positions of 2D Random Walk

Question: I need to generate a plot that shows the averaged position at n steps of 1000 different random walks that are each 1000 steps long. So far I am using Accumulate[RandomChoice[{{1, 0}, {-1, 0}, {0, 1}, {0, -1}}, 1000]] to generate the random walk of 1000 steps. By changing it to a function: randomWalk[n_] :=Accumulate[RandomChoice[{{1, 0}, {-1, 0}, {0, 1}, {0, -1}}, n]] I have found that I can use Table[randomWalk[1000], (anynumber)] to generate "any-number" of walks that are 1000 steps long. What I need now is to create a plot that shows the average position of the 1000 walks at each step, i.e. average position of 1000 walks for step 1, average position of 1000 walks for step 2......step 1000. I'm not sure how to go about this.
Comment: `Mean /@ Transpose[Table[randomWalk[1000], 1000]]` - that is, create a list of thousand random walks, and `Transpose` the result so that results for each step are on their own list, and then `Map` `Mean` on each of these vectors, giving a list of per-step means. I would also suggest taking a look at `guide/RandomProcesses` in documentation, which allows analytical reasoning of these sort of processes (although two-dimensional random walk has to be constructed using `TransformedProcess`).
Comment: ... and add `// ListLinePlot[#, AspectRatio -> Automatic] &` on the end to plot the result.
Answer: Here is an example:
n = 1000;
rw = RandomInteger[{-1, 1}, {n, n}];
rw = Accumulate /@ rw;
ListLinePlot[Mean /@ Transpose[rw]]

Answer: It is especially convenient to make use of built-in functions like TemporalData to facilitate the analysis.
Starting with your two-dimensional step function:
randomWalk[n_] :=
 Accumulate[RandomChoice[{{1, 0}, {-1, 0}, {0, 1}, {0, -1}}, n]]

walks = With[{anynumber = 1000},
  TemporalData[#, {1}] &@ Table[randomWalk[1000], (anynumber)]]

This generates a TemporalData entity, which we can use to extract the properties at any later time (i.e., at any "slice" of the collection of walkers:
Manipulate[
 ListPlot[walks["SliceData", t],
  PlotRange -> {{-31, 31}, {-31, 31}}, AspectRatio -> 1],
 {t, 1, 1000}]

We can extract values at any particular time slice and operate on them, e.g., to see the average value of the x and y values as a function of time
averages =  Table[ Mean@walks["SliceData", t], {t, walks["PathLength"]}];
ListPlot@Transpose@averages

As suggested in the past, perhaps you want to know the average distance from the origin as a function of time (using Norm instead of EuclideanDistance for no particular reason)
ListPlot@Table[
  Mean@Map[Norm]@walks["SliceData", t],
  {t, walks["PathLength"]}]

Answer: I believe that you may have intended average Euclidean distance from the per step and not average two-dimensional position per step.
Nonetheless, I provide both. In the position plot, running in Mathematica, when hovering over a point with your cursor, the step number will be displayed.
nSteps = 1000
nWalks = 1000
steps = {{1, 0}, {0, 1}, {-1, 0}, {0, -1}}
start = {0, 0}
places = Mean /@ FoldList[Plus, Evaluate[
      Transpose[Table[RandomChoice[steps, nSteps], nWalks]]]];
Graphics[(Tooltip @@ #1 & ) /@ Transpose[
    {Point /@ places, (StringJoin["    ", ToString[#1]] & ) /@
      Range[Length[places]]}], Frame -> True, Axes -> True,
  AxesOrigin -> {0, 0}, GridLines -> Automatic,
  PlotLabel -> "Average position"]
ListPlot[(EuclideanDistance[start, #1] & ) /@ places,
  Frame -> True, Axes -> True, GridLines -> Automatic,
  PlotLabel -> "Average distance from origin"]

Of course, your plot will be different. This is a Monte Carlo simulation, anyway.
Of course, I may have missed something.


Ratio question for GRE

Question:
I got kind of lost in this . . .
so you have
[(m liters)/(household)]/month and then i get lost
Answer: For all $n$ households taken together, the rate at which water is needed is
$$\left(m\;\frac{\text{liters}}{\text{households}\times\text{months}}\right)\times\left(n\;\;\text{households}\right)=mn\;\frac{\text{liters}}{\text{months}}.$$
Thus, if $p$ liters is needed to last $x$ months, then
$$p\;\;\text{liters}=\left(mn\;\frac{\text{liters}}{\text{months}}\right)\times(x\;\;\text{months}).$$
Solve for $x$.

This problem is an example of dimensional analysis (Wikipedia article).
Answer: Since there are $n$ households, the village requires $mn$ litres of water per month. We have $p$ litres of water and $p$ litres contain $\dfrac{p}{mn}$ number of $mn$ litres which is the number of months it will last.
Answer: Try this:
"$m$ litres will last $1$ household for $1$ month"
so
"$1$ litre will last $1$ household for $1/m$ months"
so
"$p$ litres will last $1$ household for $p/m$ months"
Can you continue this process?

Ratio question for GRE


My windows directory is huge

Question: I have always had problems with my windows directory's growing to a huge size.  It started with windows 95 then 98 then 2000 and xp.  Windows 7 however is absolutely huge.  The windows folder itself with out program files is 21.4 gigs...add the program files and other windows files and it is a whopping 32 gigs.  My question is why is it so large and what can I do about it.  At the rate this is going I will be out of space in now time.
Comment: Did you upgrade all the way along? That can be done, but if you've done that, well, that won't have helped much. I'm thinking it's massive from cached objects and system restore points? But that's just a guess.
Comment: Check out [this series](http://blog.superuser.com/2011/05/07/where-did-all-my-hard-drive-space-go/) of posts on the [SuperUser blog](http://blog.superuser.com)
Comment: @The White Phoenix System Restore data is not stored in the Windows directory.
Comment: It is so you don't have to insert the CD every time you add a piece of hardware. Plus added security isn't cheap, that's for sure.
Answer: In Windows Vista and 7, The Windows directory is not actually as large as Explorer tells you it is. In these versions of Windows, there are a large number of NTFS junctions (hard links) in the Windows/WinSxS directory which point to other files on the disk. Explorer will dereference these junction points when calculating the size of the directory, so a single file may be counted 2, 3, 4 or more times. (ref.)
Answer: Run WinDirStat and see if you can find any unusually large files. The program displays all of your files graphically:

Sometimes you can find a large corrupted file that is entirely useless.
Comment: +1 It will also show hidden files too! A great program.
Answer: Why?  Each new version of Windows includes a larger set of features, which requires more programming.  This often results in more .DLL and other program, data, and other files.  The Windows Registry also increases with size over time, not to mention the "Temp" directories that don't always get cleaned up properly and tend to accumulate more and more data over time.
Many other Operating Systems are also getting larger as the degree of complexity increases and the feature-sets grow.
Another factor is that newer user interfaces include higher resolution graphics, which usually also require more disk space for storage.
What can you do about it?  Make sure your hard drive is large enough (21.4 GBs is far smaller than the typical 250 GB and 500 GB drives for sale these days in new computers, so you probably don't need to worry too much about this for new computers anyway), unless you wish to take the time to determine which files are safe to delete first (not recommended because this can result in serious instability problems or just prevent Windows from booting entirely).
Answer: If you are using offline files check the client side cache.  c:\windows\csc [it is hidden].  If it becomes corrupt it expands at a great rate until it fills up your c: drive.
To fix it turn off offline files, reboot, empty the csc folder and turn offline files back on.  It will sync with the server.
Comment: I don't think that's the problem in this case, but useful info nonetheless (+1)

My windows directory is huge


In a more advanced society that experiences incursions by monsters, do you think weapons such as guns would cost more or less the same as our society?

Question: I'm in the middle of creating my own role-play game set in a fantasy/sci-fi setting where the society is attacked fairly regular by monsters and the like; Due to this I'm having trouble with the pricing of weapons etcetera while keeping it balanced and affordable for players so they have a good variety of weapons to try out and buy.
Comment: Vlaz, it would be the case of those with a license to hunt monsters are also made to protect the last surviving cities against said monsters
Comment: I suppose it would depend on what exactly the threat is. If it's "the village will be wiped out until we do something", the price gouging your saviours is not very wise. If it's "there are monsters *way over there* and you are going to hunt them", then there is no immediate threat and you aren't pressed into providing cheap labour.
Comment: How much do weapons such as guns cost in your society? In my society weapons such as guns need permits which are very difficult to obtain, and *after* obtaining a permit they are quite expensive. Most people of this world do not live in a society where one can easily try out and buy a good variety of firearms...
Comment: AlexP, In my world the way to be able to own firearms is to get a 'hunters license' which allows you to leave the city as well as own weapons to hunt said monsters that attack the cities in the world.
Answer: So, slightly unhelpful answer: they'll cost enough to keep your game balanced.
More practical answer: economies of scale will make things cheaper and easier to come by, though do be aware of tropes like Adam Smith Hates Your Guts or No Hero Discount. As far as killing monsters goes everyone is going to be happier when there are more people available and equipped for dealing with incursions.
Note though that even in areas with quite relaxed weapon ownership laws, you can't buy just anything you want, and all the nicest stuff is kept for the local military and possibly law enforcement.
Also note that in areas where citizens may be part of a militia as conscripts or volunteers they may be provided with suitable weapons by the government because again, it is in everyone's interests for the militia to be armed appropriately even if they're quite poor. Arms, or sometimes ammunition provided in this way may only be accessible in times of emergency, however. Real world people generally can't just go full murderhobo on a whim, and it has been observed that militia are more likely to shoot themselves or their friends, family and neighbours if they have ready access to guns and ammunition whenever they fancy it. Have a read up on how Swiss and Israeli militaries operate for more on this subject.
An advanced and stable society which needs to defend itself is more likely to want a well organised, trained and equipped army or militia than a bunch of mercenary murderhoboes accountable to no-one.
Answer: The price of the weaponry would depend on the severity of the most common, or most prevalent, threat.
What I mean by this is that if you can take out most threats with a relatively low-powered firearm, the need for more powerful firearms is reduced. There is no need to use a 50.cal sniper to kill a common goblin you could kill just as easily with a 9mm pistol, for example. However, you might need that 50.cal sniper if you regularly go up against dragons to pierce their armoured hide.
Supply and demand would tell us that, as there is less market for these more powerful weapons, there would be less of them produced and their price would be higher (as there is still some small market for them and due to the skill involved in their production). On the contrary, less powerful (thusly easier to produce and lower in cost) firearms that are "good enough" to deal with the most common threats would be in high supply and high demand, making them cheaper.
However, if your most common threat was dragons, 9mm pistols become almost worthless as no one really wants one and there aren't many being produced anyway. The 50.cal rifles would then be in high demand and high supply (as manufacturers prioritise on making these rather than anything less powerful), thusly their price would drop significantly.

Just a quick note, in terms of balancing the price for your players, consider that the price would change if the most common, or the most prevalent, threat changed. So at low levels, everything may be running as normal. However, as the campaign progresses and the players start levelling up and facing more dangerous threats, the average person would become more worried about these more dangerous threats and demand higher power weapons, thusly reducing the cost and making them more affordable to players at higher levels.
Further, consider that militaries and law enforcement would likely prevent most people from owning military-grade weaponry if there was no reason for them to. If everyone owned high-end fully-automatic rifles, it may be impossible to prevent a riot or civil war through force. However, as the need arose for more powerful weapons to deal with these highly dangerous threats, military and law enforcement might become far more lenient on who can own these weapons.
Answer: If the technological level allows, they're probably slightly cheaper, but much easier available. Most first world countries have hurdles to stop wide-spread gun control - from blacklisting like the USA to whitelisting like most of Europe.
In your world, wide-spread gun ownership will be required because everyone need to be able to defend themselves. Therefore, they'll be widely available. If your monsters have some specific weaknesses, then the standard ammo will be specialized for them. For example, if there's a hard shell on them, then they're probably Armor Piercing to a sufficient degree.
Benefits of mass production in an post-industrial revolution society will make them a bit cheaper than they are in our world.
This is of course dependent on wider society. If there's sufficient wizards around (like 1 in 10 or so), then perhaps they just have a duty to protect people, obsoleting general gun ownership because it's very possible to just always have a wizard around.
Answer: First, the obvious caveat to what I'm about to say is that the price needs to be whatever it has to be to keep the game balanced.
But from a worldbuilding perspective, the price of anti-monster guns depend on just how frequently monsters attack.  I say "anti-monster guns" because guns in general are good for more than just shooting monsters, and so their price is harder to predict.
The price of anti-monster guns will be an inverse bell-curve.  It'll start out high, as monster attacks are rare; few anti-monster guns will be made if monster attacks are rare.  As monster attacks become more common, demand for anti-monster guns will increase, and so will supply, and price will fall.  Eventually though, attacks become so common that society can't survive.  If the attacks are too common, supply chains break down, and guns have to be made one-at-a-time in small shops again, which drives the price up.
Answer: From a simple economic perspective, the laws of supply and demand would work to decrease the cost of weaponry. Since there will be a great demand for monster killing weaponry, suppliers will boost output to service the demand, other suppliers will enter the market and a host of other goods and services related to monster killing will also appear (everything from holsters to training and bodyguards for people incapable or unwilling of using the weaponry).
This actually applies regardless of what sorts of weapons are being discussed, so if killing monsters requires silver coated knives or anti tank missiles, the general trend will be for an increase in supply and a downward pressure on prices.
There are, of course, many factors which would serve to disrupt the supply/demand equation. The monster hunter's guild could pressure the government (or suborn officials) to prevent the sale of weapons to civilians, driving the price upwards (as demand is still high, but supply has artificially been choked off). The development of some sort of superior alternative technology that makes current generations of monster killing weapons obsolete will have the opposite effect. Similarly, the number of monsters available will also affect the demand for weapons, and thus the price. A massive kill off of monsters will make monster killing weapons a glut on the market, collapsing the prices.
So in general, the appearance of large numbers of monsters will create a demand for monster killing weapons, the price will depend to a large extent on how the supply of these weapons is dealt with.


Can this happen in Google Drive?

Question: This is a Google Drive that "anyone on the internet with this link can edit". I am not an owner of the Drive. Can it happen that (1) I posted a file there, (2) someone deleted it, and (3) there are no log of my posting the file and someone deleting the file?
Answer: This scenario is plausible if:

You upload a file over a shared drive
A Direct member of the Shared drive, like a Manager or Content Manager deleted it (or moved it to the trash).
Generally shared drive logs are only available or visible to the organization that is the owner of the shared drive itself, and those logs or information can only be viewed by a Super Admin.

The key thing here is the fact that shared drives are only available if you have a Google Workspace account as suggested over their help center article . By the moment you upload a file to a shared drive, the file no longer is owned by you, the organization that has the shared drive becomes the owner.
Which could explain why you might not have the logs, because those are only available for the organization as suggested in this article.
Edit:
Adding a screenshot with activity of the permissions of the folder and the upload of a file that was deleted:
Personal account activity (not showing upload or deletion of the picture):
Edi 2:
Updating sample on how the logs show the upload of a file to the shared drive
Comment: Yes it is a logical analysis. I think what happened is: (1) Organization A has the Google Workspace, (2) Mr. B has an account in Organization A and created a shared drive (shared for everyone), (3) I posted the file, someone has deleted it, and (4) Can it happen that Mr. B does not have logs of these activities?
Comment: Then this is another question. This time I have the Google Workspace. I have a shared Google Drive folder. I can go to "Audit and Investigation" and see "Drive Log Events." Then how can I tell whether this login is mine or not. I access with the same IP.
Comment: The moment the file was uploaded to the shared drive, the owner will always be the organization, not the original user who did it. I have tested it myself with my shared drive and a personal account with a folder from a shared drive with anyone with the link, if you were never a direct member of the shared drive, and only upload a file to that folder, there are no records (for the personal account activity info), and the only one that can see activity of the file I deleted it that was upload it, is a member of the shared drive.
Comment: Mr B should be able to check the activity of that folder showing the upload and deletion of the file, let me edit the response and add the example.
Comment: It should give you the details of the user who uploads it, it requires a login to be able to upload data to a shared drive, so this can be audited. At least it would show the email address (actor) of the user action over the shared drive or any changes being done over the Drive. I can add the example with one more update over the response.

Can this happen in Google Drive?


Can TDD and White-box testing be combined directly?

Question: I am getting started with software testing in Java and was wondering if test-driven dev and white-box testing could be combined directly... If not, what ways can we extend TDD to combine white box testing?
Comment: @RobertHarvey but why would you? You've already got tests for all of the *behaviour*, adding more tests is likely going to couple you more to the *implementation* and make it harder to refactor.
Comment: Nothing prevents you from supplementing your black box TDD tests with white-box tests after the code for the TDD tests has been written and all of the black box tests are green.
Comment: @RobertHarvey but that's not going to be adding new green tests on green tests, it's not outside the TDD process. That's going to be adding a new *failing* test that demonstrates the bug, then fixing it. You (almost) always have to "go into the code" to get the test passing, that doesn't make it white box. TDD is about testing behaviour through public APIs, it's far more closed then open box, even once you've drilled down from E2E to unit level.
Comment: @jonrsharpe: The most likely scenario is a discovered bug.  Yes, you can treat that like a black box test, but it's going to be white box in practice because you've got to go into the code and fix the bug, and because the test wouldn't exist absent the bug's discovery.
Answer: TDD and white-box testing are fundamentally incompatible with each other.
White-box testing means that you write the test based on knowledge of the internals of the code.
TDD means that you write the tests before the code.
It is fundamentally impossible to write tests based on knowledge of code that does not exist.
Note that you of course can write additional white-box tests after you have implemented the code. But that has nothing to do with TDD.
It is of course also possible to do some sort of "fake" TDD, where you write the code in your head, and then write a white-box test based on your mental image of what the code will be. But again, that is not TDD. In TDD, the tests drive the code. If you imagine the code in your head before you write the test, then you are not doing TDD. It doesn't matter that you didn't write the code down in your IDE, you still did write it in your head.
Comment: @DocBrown it's really not. The idea that you'd *"leave out tests which don't increase any coverage"* suggests what you're describing is **not** TDD. You write tests because you need new behaviour from the code, then you write the simplest code that provides that new behaviour and all previous requirements; high coverage is an *outcome*.
Comment: Knowing what's going on is different from white box. A product owner could be aware of a bug and how to reproduce it without knowing the internals of the code. Implementing this behaviour in a test to reproduce such bug using only the public api is black box testing. You don't need to know the internals. White box testing means you know for sure the implementation details of the code. TDD is always at least one unit higher than the code, since it's only using it. It does not know how it works, it knows what it does.
Comment: This answer is not completely wrong, but it ignores the fact that TDD is some kind of white-box technique, it only works well when you know which parts of the code are already covered by the former tests, and which not, so you can leave out tests which don't increase any coverage. And the resulting tests are usually undistiguishable from classic white-box tests which are written with the existing code in mind.
Comment: I agree with Doc Brown here. The red part of red/green/refactor will ensure that a TDD test exercises some new behaviour, thus effectively excluding behaviour that is already covered. TDD could theoretically work as a black box approach, but in practice the developer knows very well what's going on. The outcome is indistinguishable, except that TDD leads to a more testable design.
Answer: Some definitions (my emphasis):

Test-driven development (TDD) is a software development process relying on software requirements being converted to test cases before software is fully developed, […]. This is opposed to software being developed first and test cases created later.

Basically, TDD means writing the tests before changing any of the underlying code. Crucially, the TDD process includes changing the underlying code, not just writing the tests.

White-box testing […] is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing).

In other words, white-box testing verifies the internals of an application with full visibility of the code, while black-box testing verifies the externally visible functionality, with (ideally) no knowledge of the underlying code. Contrary to TDD, white-box testing is only concerned with the test writing phase, not with changing the underlying code.
TDD therefore requires white-box testing: it is not possible to write the underlying code to satisfy the tests without having knowledge of the existing underlying code. The only exception would be when the new code simply wraps the underlying system, and so requires only knowledge of the existing inputs and outputs.
Comment: TDD is *not* white-box testing; if anything, the box is *empty*! One of the advantages of TDD is that, because you're writing code prior to implementation, you *can't* couple your tests too closely to the implementation, and have to think about behaviour and functionality instead.
Comment: It's just a metaphor, but if you'd prefer think of it as writing each test for something that's *not yet in the box*, whether or not there's anything in there already. And you can absolutely write a new test without extensive knowledge of the existing code, because you're thinking at the level of the new behaviour you want; where exactly that fails tells you where to start looking. These are the advantages of writing tests that aren't coupled to the implementation details.
Comment: Your last paragraph isn't true, and doesn't really make sense. TDD does not require white-box testing and the fact that you have to write an implementation at some point, as opposed to just writing the tests, is not an argument at all; it's "test-driven development", not just "test". The point is that you're writing tests before the implementation exists, therefore *cannot* be writing them based on its internal structures. Compare this to writing tests *after* implementation, where you could (but shouldn't) choose to use the implementation details to drive your testing.
Comment: I agree with @jonrsharpe. To put it another way, you can do pair programming with someone that doesn't know anything about the insides of the code and that dev could write the next failing test just by looking at requirements and existing tests. This is purely black box testing.
Comment: @jonrsharpe The box is only empty when you write the very first test. And without extensive knowledge of the existing code you could not write a decent test which involves *any* existing functionality - you wouldn't know where to begin.
Answer: The questions, comments, and answers contain so many problems that I felt compelled to write an answer to this heavily downvoted question.
There are, at least, two different scenarios in which TDD is successfully applied.
There are, of course, infinitely many scenarios in which it was futile. But that's not a point. Not every rocket company ends up launching a rocket.
Scenario 1: (Macroscopic) Test-Driven Design (TDD Design)
In this approach, TDD is used to help the architect think through how the software's architecture and interfaces should look like, without being tied up with details such as implementation specifics.
TDD Design tests tend to be high-level, covering the API level, sometimes at the end-to-end level. TDD Design may entail creating and then throwing away a lot of tests as well as mock implementations.
To the extent that "very high level tests tend to be white box tests", TDD Design tend to be white box.
Scenario 2: (Microscopic) Incremental Test-Driven Development (ITDD)
In this approach, the software framework is already well-established, and code changes tend to be incremental, in that existing functionality are seldom affected. ITDD ensures that new code changes have high coverage to begin with, and that none of the existing functionality were adversely affected.
To the extent that "incremental tests tend to be unit tests", ITDD tend to result in additions to an existing unit test suite.
Comment: In scenario 1 you start by saying *"without being tied up with details"* and end with *""very high level tests tend to be white box tests""*. Which is it? Also *does* it entail throwing away a lot of tests? In my experience you generally remove tests only when you don't need that behaviour any more.
Comment: @jonrsharpe Test-Driven API Design's goal is to design an API that users will feel "right". They only test that the API design "appears to be logically sound", typically without checking the result of computation (because, at that early stage, few of the computations would have been implemented). If Test-Driven API Design sounds completely foreign to you, it is possible that you have not had experience with it.
Comment: Test-driven API design as you describe is *not* TDD, and more like consumer-driven prototyping. Also by definition that can't be white box, there isn't a real implementation to couple to, so the third sentence of your first scenario still doesn't make sense.
Comment: I said "details such as implementation specifics". It might be more clear if I had said "without being tied up with implementation details". What I mean is, TDD API Design is typically done when implementation details do not exist yet.
Comment: @jonrsharpe "Test-driven API design as you describe is not TDD, " to which I reply, "that's what you said."
Comment: Right, so it can't be white box. But then the last sentence says *"TDD Design tend to be white box"* - is that a typo?
Comment: I'm not sure what you're trying to say in your last comment. Quoting me then saying *"that's what you said"* is tautologous; yes, that's how quoting works.


Is the movie rendition of the Golden Compass significantly different from the book?

Question: I've seen some reviews which claim that all religious related subtext was subdued or removed from the movie. Did this result in a movie which is significantly different from the book?
Answer: Apart from the obvious (leaving out the ending because it would fit better in the sequel), the film largely followed the plot of the book. Of course, some scenes from the book weren't in the film. But the plot of the film was very much like the plot of the book (to a similar level of the first couple of Harry Potter films).
The main difference of the film from the book was the fact that a lot was explicitly in the film in dialogue and voice-overs which was only hinted at, or revealed much more slowly in the book. That's a common feature of most books which become films, however.
You're also right about the downplaying of religious themes. It does remain in the film to some small extent, but the themes are much more explicit in the book.
Overall, a fairly faithful adaptation of the book to film, which is about the best I could expect from a film-maker.
Comment: To be honest, I'd love to have more detailed breakdown. Which religious topics/details were omitted specifically etc...
Comment: Is this analysis based on some sort of sources?
Comment: @DVK It's been a year since I answered this, but re-reading the answer, no. Just common knowledge (the ending) and the fact that I've read the book and watched the film multiple times.


Is there enough overlap between these four "space ship shape" questions to close at least one as duplicate?

Question: Is there enough overlap between these four "space ship shape" questions to close at least one as duplicate?

Are there enough stray gasses in space to justify streamlining a probe?
Is the streamlining of a launch vehicle worth the additional fuel required to carry it beyond the Karman line?
Is streamline body shape essential for propagation in the vacuum of space?
What would be the most appropriate shape for a satellite going to mars?

It seems like there might be...
Answer: I think that this question should be closed: Is streamline body shape essential for propagation in the vacuum of space?
As it seems to be a duplicate of this: Are there enough stray gasses in space to justify streamlining a probe?


Is the Dlib library free?

Question: Is the Dlib library free? Because at the company we have plans to use it to recognize documents with Python, and we were thinking of using this lib without a headache.
Comment: If it is free to be so used, it will be *because* of the licence, not despite it.  Also, could you maybe amend your question to link to this library?
Comment: What _exactly_ do you mean by "free" (in particular, [gratis versus libre](https://en.wikipedia.org/wiki/Gratis_versus_libre)) and what exactly do you mean by "headache"?
Comment: My company doesn't want to have a headache in the future with a license. For this I wanted to know if the dlib library is free to be used in an institution
Comment: All open source licenses are free to be used - given some requirements for redistribution of itself or when used as library or as derivative. Open source does Not mean public domain. Read the license for permissions and obligations when distributing it under the license
Comment: It would improve your question if you could add more details. At minimum you should link to the project page of what Dlib you are talking about. The name is so generic that it's pretty easy for multiple projects to be floating around that have the same or a similar name. Also it would be nice to include details like what you want to do (e.g. use the library internally, use it in a product and sell that product, etc.).
Answer: The DLIB license seems to be this: https://github.com/davisking/dlib/blob/master/LICENSE.txt (pls correct me if I'm referring to the wrong 'dlib project).
The license appears to be permissive, so most users will likely consider it a 'no headache' license.
I cross-checked the license terms against the '4 essential freedoms' https://www.gnu.org/philosophy/free-sw.html and indeed the Boost Software License 1.0 appears to fulfill all of these.
But there are requirements you need to fulfill if you want to use code under this license. It is not in Public Domain, so you should be sure to understand the requirements and fulfill them when you use the DLIB library in your project.
Comment: Confirmation from the FSF that it is both free and GPL-compatible: https://www.gnu.org/licenses/license-list.en.html#boost

Is the Dlib library free?


Is it a good approach to design tables according to the queries I'll perform on it?

Question: Watching this video, and pretty new to dbms.
The speaker explains that in a row-oriented DB, rows are read in blocks.
So, my understanding is that if I have rows with fewer fields, more rows can fit into a single block and when I query the table it should take less IO operation, resulting in better performances.. Am I right?
Can I extract the rule that I shouldn't design tables according to the entity they represent but, instead, to the frequency I'll read or update that fields?
For example:
table employers:

ID
Name (frequently used)
Badge number (frequently used)
Birth date (rarely used)
Birthplace (rarely used)

Should I split the table into 2?
tbl1: ID | Name | Badge number
tbl2: ID | Birth date | Birthplace
Answer: In most database management systems, data is is stored as pages, not blocks.  Pages are normally  4 or 8 KB, depending on the database and how its been configured.
All else being equal, smaller row size will equate to better reuse of cached pages and less page reads on queries that require a large number of rows - so less I/O and faster read performance.
However
If you vertically partition the table (as you have in your example), there will be a slight increase to overall storage (equal to the primary key length and number of rows, plus the b-tree) and insert performance will be slightly slower as you'll need to maintain a PK-FK relationship between the two tables.
Furthermore, if most of your queries are for single-record lookups, you're still going to be reading a single page.  There's a greater chance that page will be cached, but reading 4 or 8 KB off a modern disk is really not an expensive operation.
Splitting the table would require 2 page reads (and navigating two B-trees) when you require BirthDate/BirthPlace.  Again, not really a big deal on modern hardware.
The only time I would vertically partition a table would be in certain data warehouse situations, or if BirthDate/BirthPlace were nullable and infrequently populated.
Other Considerations
If badge number is relatively small in size (say, under 20-30 bytes), the best thing you can do to increase performance would be to drop the unneeded ID column and make your primary key BadgeNumber since:

You shouldn't have duplicates in that column
Most likely you will primarily lookup on that column, so using BadgeNumber:

Saves you a column, making your table more compact
Removes the need for an index (and associated overhead) on BadgeNumber
Eliminates the need to join to your table to get the BadgeNumber when the table has a PK-FK relationship to another table.

There are other ways to reduce I/O and improve read performance.  Most commercial DBMSs will support some form of data compression. This can fit more rows on a single page without any changes to the structure of the table, at the expense of some CPU overhead to compress/decompress the data as it is written/read.  CPU is usually a cheaper operation than disk, so compression is usually a net benefit.
Comment: Quality answer here.
Answer: What you are considering is "premature optimization". You can't usually tell how much of an overhead those extra few bytes of "Birth date" and "Birth place" will bring, and whether it will have any material effect on performance. (I also hope that "Birth place" is actually a foreign key reference to a "Place" entity.)
So, you start with designing and properly normalizing your entities and their attributes and relationships. You validate your design by confirming that it can support all the queries you need to run from the semantic point of view.
If and when you do encounter serious performance problems caused by your table being too wide, with unequivocal hard evidence of that being the case, only then will you start considering possible remedies (of which vertical partitioning would be towards the end of the list).


Remove or replace exhaust fan assembly

Question: I am trying to remove old exhaust fan in my bathroom. I took the grill and motor apart but I do not know how to remove the housing assembly. The reason I want to replace it because the higher cfm fan and motor (70 cfm) I bought is larger than the space where the fan fits. I am including the pictures of the fan housing. Let me know any suggestions on how to remove or find smaller fan blade with higher cfm.
Comment: Is there an accessible attic space above?
Answer: You will most likely have to open the ceiling, remove the fan housing which was presumably nailed or screwed in place from the outside before the ceiling was fitted, install the new fan and repair the ceiling.
Drywall repair is a fact of life for getting things done, many times.
Answer: Hold on there.
The fan was designed to have the motor replaced, that's why there is an AC socket there. And only one screw was used for the motor.
You may be able to buy a replacement motor, or more likely, the same model fan with a higher CFM motor.   Take the motor out of the new fan and put it in your old box.
Answer: Do you see the ceiling joist? Most likely the fan is fastened to the joist with nails. There's probably some "ears" on the housing where it's nailed to the joist. You should be able to pry the fan housing away from the joist. If you're careful you should be able to remove it without any major damage to the ceiling. Turn off the breaker and disconnect the wiring first. If you have a bigger fan then you simply cut the hole a little larger which is less work than putting in a smaller fan and patching the ceiling.

Remove or replace exhaust fan assembly


Density of halting Turing machines

Question: If we enumerate all Turing machines, $T_1$, $T_2$, $T_3,\ldots,T_n,\ldots$,
What is
$$\lim_{m\to\infty}\frac{\#\{k\mid k\lt m \text{ and }T_k\text{ halts}\}}{m}\quad?$$
Or does this depend on how we enumerate them ?
Comment: You probably want to get on Wikipedia and look up Chaitin's constant Omega...
Comment: You probably want to specify that the enumeration is computable. Otherwise - since there are infinitely many machines that halt and infinitely many that don't - you could enumerate the machines one way so that the limit is $\frac{1}{2}$ and another way so that $\frac{1}{3}$.
Answer: This question is exactly the theme of my article, "The halting problem is decidable on a set of asymptotic probability one,"  (J. D.
Hamkins and A. Miasnikov, Notre Dame J.
Formal Logic 47, 2006, various links available at http://boolesrings.org/hamkins/haltingproblemdecidable/).
Similar questions have arisen several times on mathoverflow:

Probability that Turing
machine is
universal?
Solving NP problems in (usually) polynomial
time?
Turing machines the read the entire
tape?
Proability a random Turing machine is
DFA?
Limits of
non-halting?

The answer is that it depends on the model of computation. (This fact comes after the issue of enumeration raised in the comments and in the answer of Quinn Culver, which seems partially addressed by the fact that we have several canonical enumeration of Turing machine programs based on the fact that for any $n$ there are only finitely many programs having $n$ states.)
One of the ideas is that it may be more natural to consider the asymptotic density  of the desired set of programs, by considering the limit of the proportion of all $n$-state programs with the property, as $n$ goes to infinity. This seems to avoid the trivializing effects of arbitrary rearranged enumerations.
For example, here is one easy thing to say. Suppose that
your Turing machines have alphabet $\{0,1\}$, a set $Q$
of $n$ states, a single halt state (not counted inside
$Q$), and the ability to move the head left and right, so
that a program is a function $p:Q\times\{0,1\}\to
(Q\cup\{\text{halt}\})\times\{0,1\}\times\{\text{left},\text{right}\}$,
where $p(s,i)=(t,j,\text{left})$ means that when in state
$s$ reading symbol $i$, the program changes to state $t$,
writes symbol $j$ and moves left. In particular, for this
model the total number of programs is $(4(n+1))^{2n}$.
Consider now the collection of programs that have no
transition to the halt state. The total number of such
programs is $(4n)^{2n}$, and the interesting thing here is
that $$\lim_{n\to\infty}{(4n)^{2n}\over
(4(n+1))^{2n}}=\lim_{n\to\infty}[{n\over
n+1}]^{2n}=\frac{1}{e^2}$$ which is about 13.5%.
Thus, for this model of computation, at least 13.5% of the
programs never halt on any input.
The topic is fun, because we are in effect considering the
behavior of a random program, where each new program line
is chosen randomly from among all the legal program lines.
The main theorem of my article with Miasnikov is the
following.
Theorem. There is a set $A$ of Turing machine
programs (for machines with one-way infinite tape, single
halt state, any finite alphabet) such that:

One can easily decide whether a program is in $A$; it is polynomial time decidable.
Almost every program is in $A$; the proportion of all
$n$-state programs that are in $A$ converges to $1$ as $n$
becomes large.
The halting problem is decidable for members of $A$.

Thus, there is a decision procedure to decide almost every
instance of the halting problem. The way the proof goes is
to calculate, for any fixed input, the probability that a
Turing machine will exhibit a fatally trivializing behavior
(falling off the left end of the tape before repeating a
state), and observing that in fact this occurs with
probability $1$. Basically, the behavior of a random Turing
machine is sufficiently close to a random walk that one can
achieve the Polya recurrence phenomenon.
Corollary. For this model of computation, for any enumeration listing the programs in order of the number of states, your limit is $0$.
The reason is that the probability that an
$n$-state program is halting (in the sense of reaching the
halt state) goes to zero as $n$ becomes large, since almost
every program exhibits the trivializing behavior of falling
off the tape. Furthermore, the set of programs with that
behavior is decidable.
The theorem can be extended to other models of computation,
such as the model with two-way infinite tapes and halting
determined by specifying a subset of the states to be
halting states. In this model, as you can guess, machines
are likely to halt very quickly (since each new state has
50% chance of being halting), and in particular, the halting problem
becomes decidable for this reason.
Theorem. For the halting state model of computation, for any enumeration listing the programs by the number of states, your limit is $1$.
Answer: As I pointed out in the comments above, if the enumeration isn't required to be computable, then the value of the limit definitely depends on the enumeration. In fact, it depends upon the enumeration anyway. To wit, fix two machines $T$, which halts, and $S$, which doesn't halt. Given an enumeration (computable or not), use the padding lemma to insert $T$ frequently enough into the enumeration (thus creating a new enumeration) to ensure that the limit is 1. Or do the same with $S$ to ensure that the limit is 0.
Answer: Hamkins & Miasnikov's result answers the first part of your question: for Turing machines as we typically model them, you can approximate the halting problem. Quinn's reasoning also shows that you can easily change the way you enumerate programs to get any limit you want.
Now, can you say interesting things about interesting kinds of enumerations? There is one particularly natural class of enumerations: optimal ones. An enumeration $A_1, A_2, ...$ is optimal if for any other enumeration of programs $B_1, B_2, ...$ there exists a constant $K$ such that every program $B_i$ is equivalent to some program $A_j$ with $j<i+K$. These are the same building blocks that you use to define Kolmogorov complexity.
For those, if you define:
$$p_m = \frac{\#\{k\mid k\lt m \text{ and }A_k\text{ halts}\}}{m}$$
you get a very interesting sequence. This sequence does not converge, and all of its accumulation points are $\emptyset'$-random, which mean that their binary expansion is incompressible, even for a Turing machine with access to an oracle giving the answer to the halting problem (see this Wikipedia article for more details).
We can also say interesting things about $\limsup_m p_m$: not only is it $\emptyset'$-random, but it's also $\emptyset'$-upper-semicomputable (which means upper semicomputable by a Turing machine having access to a halting problem oracle). Oh, and both conditions are equivalent: given any $\emptyset'$-random, $\emptyset'$-upper-semicomputable number between $0$ and $1$, you can build an optimal machine having this number as the $\limsup_m p_m$ for this machine.
These results appear in a paper
I co-authored about this problem, and some related questions. It was published at ICALP 2015, and the long version was later published in LMCS.

Density of halting Turing machines


How do I add available team members to my party?

Question: I am trying to go see Green Winters, but none of my team follows me to the Autobahn. How do I get them to follow me? When I try to talk to them, they all just tell me they have nothing to say.
Answer: Well your team members will not be moving with you when you are in a city hub. You have to go to a quest area for them to follow you.
Go to the Autobahn, click on the train and select the option to go to the Hotel. At this point the team selection screens will show up.


Chaning bone parent after automatic weighting

Question: I've rigged my model and have used the automatic weighting option. After rigging I've modified some bone's parent to get better deformation results. But, should I re-parent the rig after doing this? Testing it out it seems everything is fine. Or rather the question is: does the rig hierarchy order affect the automatic weighting results?
Answer: The rig hierarchy will not affect the results, you should only need to correct the weighting if you move the bone.
The mathematics behind this effectively first transforms each each vertex by each bone that it is weighted to, and then uses those weights to combine the transforms from each bone proportional to the vertex weight.
Comment: Thank you for putting my mind at ease by confirming this. As being fairly new to rigging, I just wanted to be assured I wasn't overlooking something that might come back to bite me.


What is the difference between the emission process in hydrogen atom and the spontaneous emission?

Question: the confusion lies in the case of the emission that occurs normally as a result of excitation. But also when we study eg- a four level laser there is a term called spontaneous emission which actually alludes to the exponential decrease in the no of atoms present in the excited state. What is the difference between the two emission processes?
Comment: Do you think that they are different processes?  They are not.  They are the same.  Can you explain what is giving you trouble?
Answer: If an atom is in an excited state, it may spontaneously decay into a lower energy level as per QM, and emit a photon in a random direction. This is called spontaneous emission.
It is possible too, that the emission is stimulated by incoming photon, this is called stimulated emission. The emission in this case goes into the same direction as the incoming photon. The effect is that the incoming radiation is amplified. This is how it works to make amplified light in amplifiers and lasers.
Stimulated emission can only happen if the incoming photon has an energy level close to the energy of the laser transition. This laser gain only occurs for optical frequencies within a certain bandwidth. Based on the correct comments,  I mean here that a given laser can only amplify light that's close to the fundamental frequency (or frequencies) of that laser.
Based on the correct comments, I will use here ground state as the lower energy level (as opposed to the excited state being the higher energy level), but not the lowest available level. In this case, there might be certain conditions where the lowest energy level is lower then this designated ground state.
Since the atoms in the laser have ground states (lower energy level) and excited states (higher energy level), the atoms in excited states will amplify light, and the atoms in ground state (lower energy level) will absorb them (and get to a excited state). Now light amplification is only possible if more then 50% of the atoms is in excited state. How can they do that? By using methods that remove the atoms from the lower energy levels. usually by moving them to an even lower energy level (which is of course only possible if the electron's mentioned ground state is not the lowest available state).
Comment: "This laser gain only occurs for optical frequencies within a certain bandwidth." I assume you mean that a given laser can only amplify light that's close to the fundamental frequency (or frequencies) of that laser. But what you wrote could be interpreted to mean that only a limited range of frequencies can be produced by laser action, and only frequencies in the optical range. But of course both of those things are false.
Comment: Also, your last paragraph doesn't make much sense. How can you have an even lower energy level than the ground state?


Where do methods in class diagrams come from?

Question: In a class diagram, how do I know what methods to add? I'm not sure what the connection is between methods and other (i.e. Activity and User Case) diagrams. My guess is that methods can come from Sequence Diagrams, but then I'm not sure how sequence diagrams relate to the Activity and User Case diagrams. I'm trying to understand how all the design diagrams fit together. How do diagrams connect or flow into each other? I'm presuming they're related.
Comment: "how do I know what class methods to add" - if it's a yet-to-be created system and you're doing *modeling* (= working out what the code is going to be like), then there's no preexisting list of methods that you  can pull from. You need to *invent* them. That's the whole point; you're creating something new, and you're trying to work out how to satisfy system requirement or business needs using pictures. Sequence diagrams show a sequence of interactions between several objects, so there will be methods from different objects in them; you have to invent these objects (and their methods) first.
Comment: Hi @DocBrown, I mean the arbitrary methods inside the class box of classes in a class diagram.
Answer: The Use Case and Activity diagrams are high level things showing how the users (Actors) are going to interact with the system. At this stage there will be no classes or methods.
Class Diagrams and Sequence Diagrams are the details of what's going on in your system to implement the high level stuff.
You can't really have a sequence diagram without the methods. But drawing one might reveal that you are missing some methods that you need to create.
Which methods should you add? The ones you need! You are allowed to simplify diagrams by leaving bits out. But methods are things that you realise that you need when you start thinking how the classes will interact with each other.
Ultimately, UML doesn't do the design work for you. It gives you a way to illustrate the design in pictures, not (just) text in a design document.
Comment: *"Ultimately, **UML doesn't do the design work for you.** It gives you a way to illustrate the design in pictures"* - @ltrainpr, that sentence is your answer. UML does not design things. Humans do the design work, and then record those designs in UML.
Comment: Ok, so what I'm understanding is that UML is just the representation of the design you've invented.  It's not a tool to help you come up with a design.

So does the design come from spikes and pseudo code?  Or is it more trial and error that's passed between engineers?
Comment: @ltrainpr: there are a lot of books written on how software design can be accomplished, with several different schools of thought. Hence, it is pretty hard to give you a sufficient answer here in a comment box. You may start with this [online book "How to Design Programs"](https://htdp.org/). Spoiler: there is no UML inside. See also the [Wikipedia article](https://en.wikipedia.org/wiki/How_to_Design_Programs) about it.
Comment: @ltrainpr The design comes from one or more people sitting down and working out what the system is going to do.  Once they have a high-level view (e.g. UML Use Cases), they can break it up into software modules, and then down to classes and methods.  Nothing does it for you.  It's people thinking through the problem.  UML is a way of drawing your thoughts as pictures.
Answer:
In a class diagram, how do I know what class methods to add?

That depends entirely on why you are creating the diagrams in the first place.
If you are creating the diagram with the purpose to generate code from it, then you need to add all the methods of the class to it. Otherwise, the code generator doesn't know it needs to generate those methods.
If you are creating the diagram to explain something from your design or your code to someone else, then you should only add the details (including methods) that are needed to get the point you want to maker across.
Comment: I would add to your last paragraph that *"to explain something from your design"* usually means *"to explain something from your code"*.


Illustrator make white object visable on transparent background in pdf

Question: How can I save an illustrator file with a white logo on a transparent background  so the logo is visable in pdf so people I send the logo to don't think the pdf is blank.  Others need to use the logo in different documents layered over different backgrounds.
Comment: Possible duplicate of [Looking for a way to create a White logo on translucent background that will be viewable in preview?](http://graphicdesign.stackexchange.com/questions/74032/looking-for-a-way-to-create-a-white-logo-on-translucent-background-that-will-be)
Answer: There are a couple of different ways to handle this issue...
The first, simplest and my favoured approach is to have two separate files; one for use by people who are using the logo to build designs and artworks and one for people who just need to view the logo. The first file would have the logo set to white with a transparent background and the other would have a black (or some other colour) background added and would be marked as 'for viewing only' or similar. The only downside is that you sometimes need to send both files to people and rely on them to forward both on, but on the other hand you can choose to send just the appropriate file for the recipients needs.
The other option is to set up the white colour as a spot colour, called white (or 'white ink', 'prints white' or something like that) which is not actually white. There are various industry 'standards' for this (i.e. not really standard at all) which include using a pale pink or green colour (like 20% Magenta or 20% Yellow/Cyan). This means that the logo is visible to everyone, but when it is included in print jobs, the spot white colour is omitted (unless of course there is in fact white ink in the job), leaving a hole, which is generally that you want. The downside with this approach is that is looks 'wrong' to some people and can cause confusion. Also, it's pretty useless for digital.
I have seen one other approach, which is to put in a background (like in my first suggested option) which allows the white logo to be viewable, but place this on a separate layer (below the logo) which can be switched off in the PDF. This looks 'right' in a PDF viewer or email, but can also be used for creating artwork. As an artworker, I find this approach a little unfriendly, because I don't get a file that I can just drop in place without having to set the layer visibility to what it needs to be, but it's an option.
Comment: Would adding a second page with a dark background be a good idea? Then you can use the first page (and tech savvy people can see the logo in white, with a custom background color), and others can check out page 2.
Comment: @RadLexus: That would certainly work. Might be better to put the one with the coloured background on page 1 so that it's obvious for the non technical people. This still has the drawback of not being plug and play for people building it into a design, because you'd have to select the appropriate page, similar to having to turn off the layer in the final idea in my answer. Definitely an option though.
Answer: Your .pdf problem is, I think, no problem at all.. As long as you save your file as Pdf 1.4 or higher, you should be good to go. Pdf 1.4 is the earliest pdf version to support transparency.
I dont quite understand What you are are actually want in this question still
Since you said that " people I send the logo to don't think the pdf is blank", I'm assuming that you tried viewing the .pdf in acrobat ( or any other preview thingie you might have used ), which showed you a white background. I'd say that pretty much every .pdf capable viewing application has a default setting that shows a white background for convenience.
You can just try opening the .pdf in some editing application like Illustrator or Indesign or Photoshop to make sure, or In Acrobat settings Cmd+K > Page display > Transparency grid, you can turn on the view that shows you the transparency grid that is much like the one Photoshop has.
Hope this Helps
Comment: Setting prefs >page display >transparency grid is great on my mac, but appears to apply to every pdf I have.  Does this view  stay with that file when it is sent to another computer and opened in acrobat or preview?   I don't need the pdf object to be visible to me, but to be visible and then useable to others I send it to.


How do I search the linux kernel mailing list archives?

Question: I want to research why a Linux kernel feature appears to be missing from my Linux install by searching the Linux kernel mailing list for recent discussions on the subject.  The only site that I've found that offers a search of the mailing list archives is gmane.  However, searching for terms with quotes (e.g., "/usr/firmware") on gmane does not restrict results to include only the quoted phrase.  Is there a more advanced search of the Linux kernel mailing list available?
Comment: Mailing lists are atrocious compared to bug trackers http://programmers.stackexchange.com/questions/191961/why-do-some-big-projects-like-git-and-debian-only-use-a-mailing-list-and-not-a
Answer: You can use lkml.org to search through the archive. It's unofficial!
excerpt

In case you haven't read the titlebar of your webbrowser's window:
  this site is the (unofficial) Linux Kernel Mailing List archive. This
  mailing list is a rather high-volume list, where (technical)
  discussions on the design of, and bugs in the Linux kernel take place.
  If that scares you, please read the FAQ.

There are others as well:

Majordomo lists at VGER.KERNEL.ORG
The Linux-Kernel Archive - indiana.edu
Two digest forms of linux-kernel (a normal digest every 100KB and a once-daily digest)
Mailing list ARChives - MARC.info

Tips for searching
If you drill in enough to the lkml.org site you'll find a search box, like here for example:

https://lkml.org/lkml/2013/1/1

Additionally I would suggest leveraging the power of Google to help with this. Most of these types of sites suck in comparison as to what you can search with using Google.
For example:
Put this in your search bar if you want to find everything on lkml.org related to NFS!
<code>site:https://lkml.org/ nfs
</code>
Comment: It scares all of us!
Comment: I haven't been able to find anywhere on lkml.org to _search_ the archives.  Majordomo doesn't seem to host any search/archives itself, just links to other archives.  The search on indiana.edu seems to be broken, I can't get any results for any search term.  The Dell site doesn't have the lkml archives, just Dell-specific mailing lists.  And the marc.info search suffers the same problem as the gmane one: it doesn't work with quoted phrases.  Additionally, it doesn't work with slashes in the search term.
Comment: @Braiam - yeah I don't know why they say that...it's not that bad.
Comment: That's the one to sign up for if you want a personal message from Greg Kroah-Hartman telling you to "Go search the web or something". ;)
Comment: @drs - see my updates
Comment: thanks, @slm, the google tip and the drilling down in lkml.org do exactly what I'm looking for.
Comment: sometimes lkml.org show thread, but show empty emails in thread, like this: https://lkml.org/lkml/2014/8/12/532 .  But then such message possible to see via google groups interface: https://groups.google.com/forum/?hl=ru#!forum/linux.kernel
Comment: None of these suggestions actually work.  The drill-down only works for a subsection, and Google only returns a small fraction of the pages.
Answer: In addition to the places mentioned by slm, there's:

GMANE
The Mail Archive

Both of which are general services that include the linux kernel list.  The GMANE one has the most going for it -- click "Searching" in the side bar for an explanation of the features.
I'd love to say I knew an easy way to just download the entire archive so you could search it yourself, but I don't.  You could, however, put it together with a long series of automated requests (e.g. via curl) and some html scraping.
Comment: Thanks.  As I note in the question I find GMANE's search insufficient (on the "Searching" page you reference, they say that phrase searching is planned).  The Mail Archive worked pretty well, though the results were a bit inconsistent.  Nonetheless, this was a helpful link.
Answer: While looking to download the archive I found this: https://github.com/patjenk/lkmlsentiment. I have not tried it yet.


how can I insert a placeholder in Illustrator

Question: how can I insert a placeholder image in Illustrator, the way it can be replaced by an external real image file.
Answer: Drag and drop the placeholder image onto your Illustrator artboard.
When you want to update the placeholder you can go to the Window->Links to bring up the Links panel which shows external images that are "linked" to in your artwork. From there you can relink, update the link (if the placeholder source image was updated), and even embed the link into your Illustrator file.
Comment: @hsawires sounds like you want to be using InDesign.
Comment: Thanks alot it works nice when I have a dummy file, let's say "placeholder.jpg",  this file will be replaced with any real image having the same location and transformation properties. thank you. I hoped that there is a way to make a template with "editable region". that's would be powerful feature in Illustrator.


Is it common for malwares to use public-key cryptography to authenticate their C2 commmunications?

Question: I'm sure there are instances of this, but I ask it because malwares are often so makeshift that I could definitely see their authors purposely using weaker forms of authentication to perform communications (or no authentication at all) just for the sake of time or efficiency. I'm wondering what the general trend is in using strong authentication protocols for C2 and if breaking those protocols could be an avenue for disabling malwares.
Answer: In 2017, security researchers at Cyren discovered that over 1/3 of malware was using HTTPS for their C2 activities.
"Breaking those protocols" is never the answer, it is equivalent to when Kazakhstan forced a MITM attack on the entire country.
Answer: This could depend on per author / group behind the malware,
With the widespread adoption of Lets Encrypt and the lower barrier of entries for things such as SSL certs I would speculate that it will become more common.
It could potentially mitigate some IPS / IDS systems from viewing the traffic that is being sent


Can the Democrats eliminate the filibuster with 50 votes?

Question: It seems like the maximum number of seats the US Senate Democratic caucus will obtain is 50.  This means that they would be able to pass legislation using the VP to break ties, assuming there is no filibuster.  Does the VP also have the ability to break ties for senate rule changes, such as eliminating the filibuster?
Comment: It seems to me that either it must not be possible, or else neither party wants to. Otherwise it would never have become a thing in the first place, since whoever didn't want it to be a thing could just majority-vote it away as soon as they had power.
Comment: @KarlKnechtel In the case of the filibuster it was actually a bit of an accident and neither side realized such a thing was possible until a while after the rule was implemented.  And of course both the D's and R's have limited the scope of it for judicial nominations.  But I suppose the question I'm asking is more general and relates to rule changes in general.
Answer: Yes.  The filibuster is part of Senate rules, and not part of the Constitution. The Senate has a constitutional right to set its own rules, by a simple majority vote.
The procedure is to change the interpretation of the rules (without actually changing the rules which could be subject to a filibustered vote) Someone calls a vote to invoke cloture, and when this vote doesn't achieve the required majority, the presiding officer tells the Senate the vote has failed. Someone challenges that, and tells the presiding officer he's interpreting the rules incorrectly (!). The only way to solve this issue, is by having a simple majority vote on who's right about the rules, which then happens immediately.
The VP has the right to break ties in the Senate, including on votes on rules. (The VP doesn't get to add their vote in situations that require a 60% or 2/3 majority, as those are not ties)
Eliminating the filibuster is called the "nuclear option" which is hyperbole but indicates why it hasn't happened.  It might allow you to get legislation passed but it would also allow your opponents to pass legislation that you might want to block in the future.  No party has yet had the courage to remove the filibuster on votes that create new laws, for fear of doing long term damage to their future ability to hold the majority to account.
Comment: @JeopardyTempest This isn't a theoretical matter, the senate has already done this twice before. The trick is they don't actually vote to change the rule, but to change its interpretation. Basically, someone calls a vote to invoke cloture, and when this vote doesn't achieve the required majority, the presiding officer tells the Senate the vote has failed. Someone challenges that, and tells the presiding officer he's interpreting the rules incorrectly. The only way to solve this issue, is by having a simple majority vote on who's right about the rules, which then happens immediately.
Comment: [This senate guide](https://www.senate.gov/CRSpubs/cf80c40a-29bc-4d1b-9dec-541dff2349b6.pdf) notes: "The Senate has interpreted the constitutional arrangement to mean that it is a continuing body, since a quorum is always sworn, and that it therefore does not have to organize itself with each new Congress, as does the House of Representatives. One consequence, among others, of this interpretation is that the Senate does not adopt or re-adopt its rules when a new Congress convenes, the interpretation meaning that the rules continue in effect from one Congress to the next."
Comment: Thank you for this answer!  I wonder if there has ever been an example of a rule change interpretation being decided by the VP's vote.
Comment: "The Senate has a constitutional right to set its own rules"... it doesn't appear that this is by simple introductory votes like in the House:
Comment: And the [Senate Procedure](https://www.govinfo.gov/app/details/GPO-RIDDICK-1992/GPO-RIDDICK-1992-1/context) says "If proposals to amend the rules should be filibustered, it
would take a two-thirds vote to invoke cloture, but the vote on
adoption would still only require a majority vote."  So while it can set it rules... those processes can definitely be filibustered, meaning it's moot that only a majority is required.
Answer: Maybe?
To get an actual change of the rules, filibustering would block attempts to remove the filibuster.  There are tricks to get around that, but they appear to require a majority.
From this Brookings Institution article article giving more detail on the filibuster and how it could be changed/ended:
To directly end it, they would need to change the text of the Senate rules... but it actually takes a larger majority (two-thirds) to stop a debate on a rule change.  So the minority party would basically filibuster to prevent the rule changes on the filibuster.  Because two-thirds is 66.7, there can't be a tie, and the vice president wouldn't vote.
However there is also the perverted workaround of claiming Senate rules aren't being properly followed or unconstitutional.  Then it only takes a simple majority vote to form a new precedent to doing things.  This challenge is called a point of order.  As the Senate Procedure says:

A point of order is not debatable unless it has been submitted by the Presiding Officer to the Senate.

This is the so-called nuclear option.  The same method was used in the past to make specific changes to the filibuster rules when passing nominees (first by Democrats for non-Supreme votes, then Republicans for Supreme Court votes) and budgets.
When it comes to the rules of determining simple majority, though, the Constitution says

The Vice President of the United States shall be President of the
Senate, but shall have no Vote, unless they be equally divided.

And I could find no other place in the consistution or Senate Procedure that would exclude points of order votes (or appeals of the rulings of the senate leader) from that clause.  It may well be that somewhere else in rules of order is buried a key that would keep procedural questions to just the Senate... but otherwise he theoretically just might get a vote.
However the article lists some political reasons why it may be harmful to end the filibuster, such as the inability to place any blame on opponents or reduced ability for senators to push for support on projects important to their state.
And most of all, considering the significant split in the American population's political beliefs, any move to entirely end the filibuster might be short-sighted.  Had Democrats eliminated it under Obama when they lost the 60 votes to reach cloture, they would have been stuck without any recourse to hinder legislation in the first two years of the Trump presidency.  The filibuster in a sense personifies the significant challenge of finding widespread agreement in America as a whole on many issues, and may be the only hope to forcing parties to find middle ground.


What happens if Jupiter is ignited?

Question: I was wondering, what would happen if there was a large amount of oxygen released into Jupiter's atmosphere, than igniting it? Because Jupiter is 90% gas, therefore being a big fire waiting to happen (with air, since fire needs oxygen).
Comment: Related: https://physics.stackexchange.com/q/776/2451
Answer: Unless the amount of oxygen is very large, not much would happen.
Certainly a volume of oxygen would when ignited vigorously burn in the mostly hydrogen atmosphere, but it would be a local phenomenon. To actually set "things on fire" requires introducing a comparable mass of oxygen to the hydrogen - literally planetary masses of oxygen.
How much? If we have 0.9 Jupiter masses of hydrogen, we need 0.9*(16/2)=7.2 Jupiter masses of oxygen to make the reaction perfect (the 16/2 is the ratio of oxygen atom mass to two hydrogen atom masses). That sounds impractical.
Comment: True, but we're just imagining it did. Don't have any barriers for realism in size, please. So if it was practical, what would happen? Just imagine a future race that can do insanely unrealistic stuff now :P
Comment: Dropping a 7.2 times larger mass of oxygen into Jupiter will do much more than just create a burn - there will be so much potential energy released that the whole thing will turn into plasma, fluff up to a far larger size, and then slowly over a few million years or so coalesce into a dense waterworld.
Answer:
What happens if Jupiter is ignited?

Essentially that happened before Jupiter formed. Jupiter is a bit less than 75% hydrogen and a bit less than 24% helium. That adds up to a bit less than 99%. The rest is trace elements, most of which readily combine chemically combine with hydrogen to form compounds such as methane, ammonia, hydrogen sulfide and water.
The hydrogen in Jupiter that remains as hydrogen as opposed to the hydrogen compounds mentioned above does so because there's nothing left in Jupiter with which that hydrogen can combine chemically. Light a match inside Jupiter and nothing much will happen because the chemical reactions that can occur did occur 4.6 billion years ago. The previously mentioned hydrogen compounds in Jupiter (methane, ammonia, hydrogen sulfide and water) mostly formed before Jupiter was even Jupiter.
Comment: OK. Sorry, but Anders kind of got what I meant more.

What happens if Jupiter is ignited?


LocalReport - Render Excel returns 0-length byte array

Question: I am using the LocalReport class to generate PDF/Excel files from an RDL.
In my local development machine, everything works perfectly, PDF and Excel files are rendered with the expected content.
However, after deploying to the QA server, Excel files are blank, only PDF work as expected. No exceptions are thrown and the Warning array that is populated on the Render() method is empty.
My guess is that a specific assembly/package is required for Excel creation, and that assembly/package exists on my machine but not on the server.
Do you guys have any idea if that could be the case and what may be missing?
Thanks a lot!
Answer: Well, I know this is not the answer you're expecting and certainly not the way I wanted to solve this problem. We tried installing, on the QA server, Report Viewer Runtime, Visual Studio Office Tools and some other packages that I lost track of, and Excel files were still blank.
Then I thought, why not try generating XLSX instead? So I changed
var renderedBytes = lr.Render
                (
                    "EXCEL",

to
var renderedBytes = lr.Render
                (
                    "EXCELOPENXML",

and it just worked. Hope this helps someone.
Cheers.


Switching to a different wireless network on a headless Raspberry Pi

Question: I am running a headless Raspberry Pi and I have multiple wireless networks at home. I SSH into my Pi. Is there a command line way to switch which network the Pi is on? Yes, I realize that I will lose the connection until after I reconnect to the new network.
Comment: Reputation too low to add a comment, so I have to use this route. I already have my wpa_supplicant set up with four networks. And the priority. The question is if I need to jump to the priority 2 network, how do I do that via command line?
Answer: Run the following command:
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Which will enable you to edit the wpa_supplicant file. Add the following code to the bottom:
network={
    ssid="HomeOneSSID"
    psk="passwordOne"
    priority=1
    id_str="homeOne"
}

network={
    ssid="HomeTwoSSID"
    psk="passwordTwo"
    priority=2
    id_str="homeTwo"
}

Exit the file with saving. I suggest doing this for the different networks you intend to connect to. In the example code above the details for two different networks has been added.
Note also that the priority line determines which network is connected to in preference to another available network. This isn't necessary but might be useful in your case.
Next you need to run:
sudo nano /etc/network/interfaces

And using the example networks from above add these lines to the bottom of the file:
iface homeOne inet dhcp
iface homeTwo inet dhcp

The finished file should look something like this:
auto lo
iface lo inet loopback

iface eth0 inet manual

allow-hotplug wlan0
iface wlan0 inet manual
    wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf

allow-hotplug wlan1
iface wlan1 inet manual
    wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf

iface homeOne inet dhcp
iface homeTwo inet dhcp

Comment: This **MAY** work, but it will also break `dhcpcd`, preventing the networking icon on GUI from working (among other things). Have you actually tested it on a normal Raspbian installation?
Comment: I must confess I haven't, but this seems to be the way to do it by just using a command line interface. The fact it disables the GUI icon isn't great but the OP said in their question they only SSH into their Pi, so I wouldn't have thought that would have been an issue for them.


How does the Warhorse compares with other horses in Red Dead Redemption?

Question: When I purchased my copy of Red Dead Redemption, it came with a code that allowed me to get a deed for a unique horse...the "Warhorse".  How does it compare with the other horses in the game, specifically the better ones such as the Hungarian Half-Bred, etc?
Comment: How do you get the warhorse?
Comment: It's a freebie that you can get by buying the game through certain chains.  I believe I got the Warhorse by buying the game from Best Buy.
Answer: Comparing to the better horses only, it beats the Kentucky Saddler in nearly all aspects (can't confirm speed). However, in terms of stamina the Hungarian Half-Bred wins and in terms of speed the American Standardbred wins. So overall a great horse but NOT hands down the best for you, try the other top horses to see if they fit your play-style better.
For reference: http://reddead.wikia.com/wiki/War_Horse, wikis are great for this kind of info in case you didn't already check.
Answer: The Warhorse is THE best horse in the game.
The other 'best' horses such as:

Hungarian Half-bred
American Standardbred
Kentucky Saddler

...share good speed ratios with good health ratios.
However, the Warhorse has the best speed ratio and the best health ratio. Period!
Comment: This sounds good...but how do you know?
Answer: War horses do not scare. Unlike the rest who will kick you off when scared war horse will stick with till the bitter end.
Comment: That seems helpful. Do you have a source for this?


Should I use xz or lzma compression?

Question: I was wondering since in Ubuntu 10.10 I have the ability to create both xz and lzma archives. Should I switch to xz?
http://en.wikipedia.org/wiki/Xz
http://en.wikipedia.org/wiki/Lzma
Comment: Note that there's also a newer lz archive type that has basically been introduced "since this discussion"
 https://stackoverflow.com/a/49456056/32453 so don't confuse it with the `lzma` mentioned here :)
Comment: It's hard to tell, when you don't tell us your requirements. For instance, is long-term preservation a concern (think research data or backups)? Then you'd want to use the most compatible and long-term supported format.
Answer: I disagree with @dv3500ea's conclusion, "OK using either", as it was inappropriate.  Even at the posted date/time, .lzma files were known to be replaced completely by .xz.  The author of both Utils has said publicly, users should transition to XZ-Utils ≥ 5.00. The last LZMA-Utils release was 2008-07-30, and discontinued.

LZMA Utils are legacy data compression software with high compression ratio. LZMA Utils are no longer developed, although critical bugs may be fixed as long as fixing them doesn't require huge changes to the code.
Users of LZMA Utils should move to XZ Utils. XZ Utils support the legacy .lzma format used by LZMA Utils, and can also emulate the command line tools of LZMA Utils. This should make transition from LZMA Utils to XZ Utils relatively easy.

The determining factor might not be compression, although ironically it was for this example.  "practical reasons" are a self-evident, and significant, reason to use XZ-Utils. Additionally the .xz format allows you to specify compression algorithms, and filters, so .xz files can use either LZMA or LZMA2.  Avoid creating new .lzma files, they are considered a legacy format.
As for even keeping LZMA-Utils around: don't. XZ-Utils has support for legacy .lzma files.  It also has wrappers for scripts still using LZMA-Utils.  If all those reasons were still insufficient, Ubuntu dropped legacy lzma-utils from its repository.
Comment: "The author of both Utils has said publicly, users should transition to XZ-Utils ≥ 5.00." Citation needed. :-)
Comment: @FaheemMitha: Since this is a post, and not an educational reference or original research, the citation would only be the project page. I assumed since it was from the LZMA-Utils author, and he wrote BOTH utility sets with intention of xz-utils replacing legacy LZMA-Utils, it was self-evident. ...But For anyone who want's to get more details, http://tukaani.org/lzma/.
Answer: This benchmark provides some good information about this issue. It seems that LZMA has slightly better compression ratios and performance than XZ but XZ is generally preferred due to 'practical reasons' (I'm not sure what these are).
XZ is an implementation of the lzma2 algorithm and is better at compressing 'uncompressable' data:

lzma2 is the method of compression employed by the 7-Zip LZMA2 compressor. LZMA2 is a modified version of LZMA that offers a better compression ratio for uncompressible data (random data expands about 0.005%, compared to 1.35% with original LZMA), and optionally can compress multiple parts of large files in parallel, greatly increasing compression speed but with a possible reduction in compression ratio (see LZMANumBlockThreads). Like LZMA, it can consume a lot of memory; see the above table. If a compression level isn't specified, it defaults to max.

(http://www.jrsoftware.org/ishelp/index.php?topic=setup_compression)
You should be OK using either.
Comment: That was not an appropriate answer, in regards to "OK using either".  Even at the time when he posted this question, .lzma files were known to be replaced completely by .xz in the near future.  The author of LZMA-Utils has said publicly, they were deprecated and use xz-utils from 5.00 onward. The last LZMA-Utils release was 2008-07-30, even at the time years old.  So compression wise, the improvements might not be a huge determining factor. (ironically in case it was)  But "political" reasons should have been a significant reason to use XZ-Utils, and avoid making new .lzma archives.
Comment: Actually after a bit of testing it seems like xz will be the way forward. It also saves quite a bit of compression (10mb somtimes) on binary stuff over lzma.
Comment: @TechZilla Have you considered posting your take on this as an alternative answer?
Comment: The practical reasons for LZMA2 are multithreading support. Single threading performance and compression ratio are slightly lower but on a system with several cores used expect LZMA2 to be much faster when decoding.
Comment: Out of curiosity, did you decide which to use?
Comment: LZMA. I had most of my stuff already archived with it.
Comment: @ændrük: good point, just posted a more complete answer.
Answer: Definitely you shouldn't use lzma as others say.
Antonio Diaz Diaz, author of lzip format, says Xz format is inadequate for long-term archiving.
If you want to backup your important files, consider other file formats such as bzip2, which has corruption recovery tool.
Comment: xz is considered unsafe by some. That doesn't apply to lzma though.
Answer: xz is preferred due to it having the highest compression rate. However, it compresses slower, but it makes up for it with its compression.

Should I use xz or lzma compression?


How to export private key? (GnuPG)

Question: I have successfully created GnuPG public/private key pair using RSA and RSA algorithm. How can I export a public key and private key in the form of file with the .asc extension?
Comment: Note: recently gpg2 came out. Some distros (debian) use exclusively this, others (ubuntu) use still the first version on gpg and the 2 on gpg2. They seem compatible with the public keys but not with the private ones.
Answer: Hint 1: gpg calls private keys 'secret' because PGP dates from before people settled on the names 'private' key for the half of an asymmetric pair held by (ideally) only one party versus 'secret' key for a symmetric value usually held by two or more mutually trusting parties but nobody else.
man gpg2 | less "+/export-secret" then n (go to second match) shows:
   --export-secret-keys

   --export-secret-subkeys
          Same  as --export, but exports the secret keys instead.  This is
          normally not very useful and a security risk.  The  second  form
          of  the  command  has  the special property to render the secret
          part of the primary key useless; this  is  a  GNU  extension  to
          OpenPGP  and  other  implementations can not be expected to suc-
          cessfully import such a key.  See the option  --simple-sk-check-
          sum  if  you  want  to import such an exported key with an older
          OpenPGP implementation.

Hint 2: you can output to a file by just redirecting, or using (either of)
   --output file

   -o file
          Write output to file.

However, people usually use the .asc extension for files in 'ASCII armor(ed)' format, which is base64 with dashes-BEGIN and dashes-END lines and sometimes (including here) 822/MIME-style headers. If you want not just a file named .asc, but a file in the usual .asc format, use (either of)
   --armor

   -a     Create ASCII armored output.   The  default  is  to  create  the
          binary OpenPGP format.

TLDR:
gpg2 --export-secret-keys -a -o file.asc [keyid ...]

How to export private key? (GnuPG)


Passing parameters in visualforce pages

Question: I want to pass values from one page another page which have different controllers without using querystring. How can i pass more than 100 values?
Comment: Have you considered just using the same controller for both VF pages? It seems like they have a lot in common with that many values. Another option might be to store the values in a custom object or custom setting.
Comment: Am using different controllers..
Answer: There are a few ways to accomplish this:

If two pages are using the same controller and extensions (or a subset), you can just return a PageReference to the other page while not setting redirect to true (leave it as the default false). The context will automatically be passed along for you. You can experiment with this to see how it works best for you. We've had great success with this in our projects.
Modern browsers support parameters that are thousands or tens of thousands of characters long, and I'm not aware of any inherent limitation that you'd reach on the platform itself in terms of parameter length. Conceptually, you could just pass this data along the URL, keeping in mind that this exposes the data to the end user (so this can allow replays, it's stored in history, etc). Some proxies don't support long URLs, though, so try to keep it reasonable if you expect some users to be using proxies. We're using this in some legacy code in our projects, and it works well, even with 100+ parameters with an average name length of 10 characters and an average value length of 11 characters (i.e. an average of 23 characters per parameter value, including the separating equals and ampersands).
You can store the data in a custom record somewhere, and retrieve that data. This lets you pass just a single ID value instead of transferring a huge view state. This is much more bandwidth friendly than either of the first two solutions, but will have inherent limits on the amount of data that can be stored per field/record. We don't specifically use this technique, but it is definitely viable.
Answer: This can be possible using static variable
In first controller:
class FirstController
{
   // Call this method and set static variable for next VF page's controller
   public void setVar()
   {
       SecondController.staticVar = 'someValue';
   }
}

You will get same value when loading another visualforce page have this controller
    public class SecondController
    {
       public String staticVar;
       public String instanceVar {get; set;}

       public SecondController()
       {
           instanceVar = staticVar;
       }
    }

But note that redirection must be from FirstVFpage to SecondVFpage by commandlink, commandbutton, pagereference etc. so context remain same.
Hope this help.
Comment: does this really work??i was trying this out..and did not work?I used a commandbutton though
Comment: I have two VF VF1,VF2 page and two controllers C1,C2.C1 have Account id I want use this  id into an other VF Page.Before C1 called to VF2 then again called to C2 controller.Please share the code, thanks.

Passing parameters in visualforce pages


VPN IP address leak while establishing connection

Question: Even if I have a VPN configured to auto-connect in Linux, there is a brief moment where it is connected to the Internet, but not utilizing the VPN. Doesn't this provide a vulnerability? How can I protect myself?
Comment: Configure iptables rules that only allow traffic through the VPN tunnel interface.
Comment: What vulnerability ? and protect yourself from what exactly ?
Answer: Depending on your vpn software that's normal behaviour.
Configure a firewall that only allows connections to your vpn provider over the public network and force everything else to go over the vpn.
For instance:
iptables -P OUTPUT REJECT
iptables -A OUTPUT -d <VPN_SERVER> -j ACCEPT
iptables -A OUTPUT -s <VPN_ADDRESS> -d <VPN_NET> -j ACCEPT

Where VPN_SERVER is your VPN provider's public address, VPN_ADDRESS is your address on the tunnel interface and  VPN_NET is the VPN network in CIDR notation.
You can also do this with routes instead. When you setup your internet interface, instead of loading a default gateway you load a static route to your vpn server. No host other than your local neighbors and your vpn server will be reachable until you get a default route from the vpn provider (through the vpn).
Both approaches get the job done, but the firewall approach includes an insurance policy for free.