Using factorials to calculate # of chess combinations

Question: I recently came across a coding problem in which the solution involves writing a program that can take in the starting position and destination square of a chess piece, and then output the number of possible combinations for reaching the destination.  The piece can only move up or to the right by one square at a time; the squares are represented with numbers, where the first number represents the row and the second represents the column.  So if you were to input '(1 1)(2 2)', you are asking the program to calculate the number of given legal ways the piece can travel from the 1st row/1st column square to the 2nd row/2nd column.  The answer here is 2 because it can either move up then right, or right and then up.
The solution involved calculating the factorial of the difference between the start and end rows plus the difference between the start and end columns.  So in the above example this would be the factorial of 2, or the factorial of (2-1)+(2-1), which is 2 * 1 or 2. Then, this number is divided by the factorial of the difference between the start and end rows, multiplied by the factorial of the difference between the start and end columns, which here would equal 1, since the factorial of (2-1) times the factorial of (2-1) is simply 1 * 1 or 1, which the first calculation of 2 is divided into.
Can someone explain to me in simple layman's terms why this division step is necessary?  I understand the numerator represents all of the permutations and some of them are repeats (since the order doesn't matter, and we only need to count a specific set of moves once) and need to be cancelled out, but how does this work exactly?
I appreciate any help I can get.
Answer: If you move the $(a,b)$ to $(c,d)$ where $c \ge a$ and $d \ge b$.
In total, we know that you have to take $(c-a)+(d-b)$ steps of which $(c-a)$ of them are horizontal steps and clearly the rest are vertical moves.
That is out of $(c-a)+(d-b)$ steps, we have to select $c-a$ of them to be horizontal steps. Hence the number of possible moves are
$$\binom{(c-a)+(d-b)}{c-a}=\frac{(c-a+d-b)!}{(c-a)!(d-b)!}$$


How to Update "Created By" column through workflow?

Question: I am trying to make the Blog comments and discussion replies to be Anonymous. The only way to do this (a simple way) is through hiding the created by column in the view. Is it possible to update the created by column through workflow? When I tried to update this list item, created by column is not displayed in the options. Picture is given below. How to update created by or effectively hide the author from the comment?
Comment: Can only be done via server side code iirc.
Answer: You can change the Read Only field of this list in SharePoint Manager in your server, and use a workflow to change it.


Faraday's and Lenz's Law

Question: I have a question about the relationship between Faraday's and Lenz's law from a graph of the emf of Faraday's Law compared to the voltage in a coil. My question is whether or not this graph supports Lenz's law. Also, is it possible to determine from this graph if the direction of the induced current is going in the expected direction when the magnetic field is increasing?
Comment: How did you obtain this data?
Comment: Without a diagram showing the physical setup used to collect your data, your question probably cannot be answered.
Comment: The data was obtained using a rigid pendulum with a coil at its end swings through a horseshoe magnet. A resistive load is connected across the coil and the induced voltage is recorded using a Voltage Sensor. Faraday's Law is used to estimate the magnetic field of the magnet from the maximum induced voltage. Also, the direction of the induced voltage as the coil enters and leaves the magnetic field is examined and analyzed using Lenz' Law. Is this sufficient information to answer the question?
Comment: This question does not make sense. If a resistor is in series with the coil, and $V$ is voltage across the resistor, then current is proportional to $V$ and EMF, given by $-LdI/dt$, should behave as derivative of $V(t)$. But there is is no trace of this in the graph, the EMF seems to copy the function $V(t)$. How did you determine EMF?
Answer: You could say something about Lenz law if you had measured the current too (positive or negative). You should measure EMF and current, then cross-plot it carefully because of the signs involved.
If you only calculated V and measured EMF you can't 'prove' Lenz's law because you've used it implicitly (the minus sign at Faraday's law from Maxwell's equations)

Faraday's and Lenz's Law


SQL 2008 managing 2 Instances on single Physical server

Question: I need some calrification about managing SQL server 2008. The scenario is as follows:
I have One Windows Physical server at Primary site, I want to have Two different applications database on it, so shall I create two Instances on same server or shall have diffenrent server for another database. First Database is for management purpose while second would be used for Reporting purpose.
There is a second database at the secondary site, which will be in Passive mode and I intend to connect them through MSCS.
Now, can I have both Instances on Single server and both will work fine? The management database will be used more.
Please reply soon.
Can both Instances have dedicated reources allocated to them?
Comment: Why would you not simply put both the management and reporting databases on a single instance? It would be more straightforward and there would not be the overhead required by running two separate copies of SQL Server. If you really need more than one instance, you want to look into the "named instances" feature of SQL Server.
Comment: Hi,both instances belongs to different applications and First Instance would use the DB more then the second. If, One instance can have the dedicated server resource it would be equivalent to having two seperate physical severs. is this possible with SQL 2008?
Answer: From the description, it sounds like these could easily go in a single instance. Any given SQL Server can house many separate databases.
This is certainly preferable to trying to maintain two separate instances, whether on one or two physical servers, especially if these management and reporting functions are part of the same application/system.
This is the easiest way to configure things, so it is worth trying out. Only start thinking about a second physical server if you start to have performance problems with the reporting function.
Comment: Hi Jay, There are two different applications, one DB is for agents management data another one is Reporting DB which will get feeds from multiple applications? The usage of First DB is high, as there would be daily many agents working on it. The second one is reporting DB which will use SSRS to provide different reports. Now, ideally there should be two different physical servers handling each DB independently, but I would like to save the hardware cost hence want to explore the option of having two DB on one physical servers?
Comment: @Rajeev Well, you can create one or two virtual servers on a single machine, but honestly I think you'll get the best performance on a single machine by putting both databases in a single SQL instance. Anything else would mean a lot more overhead.
Comment: Jay, In case of Production Environment, the virtualization will not be considered. In case of having Single server and Single Instance and having Two databases can Physical resources be dedicated on database level with the help of Resource Governer or with something else?
Answer: Nothing in the original posting points to a need for two instances on the same server.  SQL Server will balance resource usage, by demand, across all databases in the same instance.  If there is a concern regarding the load generated by the SSRS usage, then this can be managed by utilizing SQL Server tools and well-defined queries.
There is little benefit to creating two SQL Server instances.  When you create two instances, you have to split hardware resources between each instance.  Two instances of SQL Server are not able to balance processor, memory, and I/O across all requests as each instances is not aware of the demand-load of the other instance.  Two instances requires two different listening ports for client communications and, overall, will require more management.  These are strong negative points.
The information you have provided does not indicate the need for two instances and you continue to push back with needing two instance.  It would be helpful to understand the underlying pre-determined need for two instances.
Help us to help you.
Comment: Sorry for my limited understanding, but my aim is to understand that if there are two different databases, which will get feed from different application, is it benefitial to have these two on seperate instances or may be on seperate hardware or there is no such need and both databases can be put under single instance; what will happen if one fails and other do not want to fail and would there be any problem with replication to secondary site?


Windows 10 IoT: Serial UART I/O missing. Solutions for RS485 serial I/O?

Question: Since the Serial UART I/O pins (P1 8 and 10) are not yet supported by Windows 10 IoT, I am looking for alternatives for simple RS485 access via the RPi. As far as I can see, none of the USB to Rs485 converters provide drivers for Windows 10 IoT yet either. The search continues. Any ideas or pointers (in C# or Javascript at best)?
Comment: Pointers in C#? This is really unsafe oO
Comment: SPI and I2C should work.  Can you get RS485 to SPI/I2C?
Comment: Thanks, yes after seeing this limitation I started looking for a SPI or I2C shield as an alternative. No luck finding one yet though.
Answer: The latest windows iot build from October 2015 has the onboard serial pins enabled now. You can also go from USb to Usb in some cases with no adapters needed. I have it semi working with a few issues. You can check out this thread here, if you want to take a look at the issues im running into


Accuracy in BLDC speed calculation

Question: What is an accurate manner to calculate the rotational speed of a bldc motor using hall sensors and microcontroller units, considering both floating point operations and length of the observed period (between rising and falling of the same sensor and then 180°, between two rising sensors and then 120°, or 1 electrical cycle and then 360°) ?
I have found this application from microchip where a constant MINPERIOD and the time between two transitions of the same hall are used. Is it a good way? I am doubtful that without averaging with the other sensors the measurement is not really accurate or that maybe the measured period is too short, but I do not have experience with that.
Answer: Depends on what you want and how you will use the reading. Stable reading or accurate reading. Readings taken over a small arc of revolution will be more representative of what is happening but only over short time spans or arc distances, but result in a less stable reading. Extrapolate that angular speed to the entire rotation can introduce inaccuracy since individual variations over that small angular span are extrapolated and applied to the entire rotation which may not be true, though inertia helps even things out.
Comment: @cyberdyne That seems reasonable. At the very least, doing so would reduce the number of calculations required since the measurement is directly applicable to the use, and it seems that time span would be a reasonable compromise between a more stable but less accurate reading over a longer interval for your purposes versus an unstable reading but more accurate reading over a shorter intervalfor your purpose.
Comment: Ok, if we apply this concept to sensorless control by zero-crossing, it would be better measuring the time between two zero-crossings (t.e. between two phases) to better know where to close the next contact and using the period of 1 electrical rotation to calculate the speed (as measurement unit), right?

Accuracy in BLDC speed calculation


Document file names automatically change during Share

Question: I'm using a Huawei Y7 Android 8.1.0. Everytime I select share for a PDF or DOCX file and select whatsapp or viber, the file gets renamed to a longer file name. In Whatsapp it shows 'NULL' in the thread. When opened within Whatsapp it shows long file name. Does anyone know how to stop this. It happens for file names less than 8 characters. My issue is I want to share the file with the name I have given but not a long meaningless code. All software including android and apps are updated. App caches cleared. Can anyone help?
Answer: Mine does the same when I share a file from the location in which it resides (like a pdf in "files"). But when I share it through WhatsApp (click on the attachment button in the chat) then it shows the correct name. Hope it helps for you too.


What could cause options premiums and futures market to point in opposite speculative directions?

Question: Its 8am EST here in the US. At around 530am the premium per share for a put option of SPY expiring on 2/28 was $0.13. Between 645 and 730 the premium rose to $0.22. This would suggest that a fall in the price of SPY is more likely than it was when the premium was $0.13. However, around this time, the futures market rose. It was down, in the negative (~-2.5%) but it became less negative. Suggesting that a fall in SPY was less likely. What could possibly cause these indicators to point in opposing directions during that time period?
Comment: "Its 8am EST here in the US. At around 530am the premium per share for a put option of SPY expiring on 2/28 was $0.13. Between 645 and 730 the premium rose to $0.22."  SPY options don't open for trading until 9:30 AM EST so on what are you basing this option price rise from 13 to 22 cents?
Comment: I realize they aren't traded at that time but the premium updated in price on the brokerage account
Comment: That can only be an estimate of the opening price based on current price at that moment in time and Friday's implied volatility. It also won't account for the spread which could expand or contract when trading opens ao it's not going to be accurate.
Answer: There are many factors that affect the price of an option. Even if the price of the underlying moves the option more out of the money (or less in the money), the price of an option can still go up if the market's expectation of the future market volatility rises enough.
It's simple when you think about it:
Lets say you own a call option that is deep out of the money, and the market doesn't move much day by day (it is not volatile). Your option isn't worth much at all.
But if the market suddenly experiences some event or change in circumstances that causes the prices in the future to fluctuate wildly, then your option might be more likely to end up in the money. Therefore your option is worth more, even if the underlying's price might in the short term move lower and therefore further from your strike.


Judaism for Outsiders

Question: I'm very interested in Judaism, and I would like to learn more about it. So I started to read this site and it is like a wall. Where can I find a gentle introduction to all this and the terms used. I'm specially interested in customs and how the Bible is read by observent followers. I hope I don't offend anybody with my question.
Comment: If you have an Orthodox Rabbi near you, then speaking to him would be a good start. Also http://www.faqs.org/faqs/by-newsgroup/soc/soc.culture.jewish.moderated.html has a good FAQ. I have not been following it lately but when I helped write the FAQ it had good information.
Comment: Welcome to Mi Yodeya mike. To read a lot is the good way, progressively you will understand more and more.
Comment: I've heard that Introducing Judaism (World Religions), by Eliezer Segal, is a great option for the uninitiated to familiarize with the origins and developments of Judaism.
Comment: Hello kouty, thanks for the welcome. Where do I start reading, do you have any suggestions?
Comment: Welcome Mike. If you feel you would like to discuss Judaism informally in a relaxed way with someone on "the inside" :) ... you can reach me at email@example.com
Comment: Hey, Mike. Your question has been marked as a duplicate -- that's not a bad thing, it just means that there are already answers over here: https://judaism.stackexchange.com/q/6988/5323! Enjoy, and good luck to you :)
Comment: Thank you @DavidKenner, that is very kind of you, I actually might take you up on that!
Comment: Your Welcome. :)
Answer: There are a variety of resources both on and off line. The best is via actual human contact, but unless you live near someone who can (and is willing and equipped to) give you a private conversation, that might be tough.
Online there are different kinds of sources:

this site deals with particular issues which distinguish Judaism from other religions  http://whatjewsbelieve.org/
This site is more encyclopedic but with a less formal tone http://www.jewfaq.org/
This is more formally an encyclopedia https://www.jewishvirtuallibrary.org/religion
sites like this give a broad range of ideas in a more user friendly format http://judaism.about.com/od/judaismbasics/u/Judaism-101-Judaism-Basics.htm
This kind has a lot of detail but can be overwhelming http://www.chabad.org/

For some print resources, try here http://www.beingjewish.com/learning/reading_list.html
So the particular resource which is best suited will depend on if you are looking for history, current practice, underlying theology, or something else.
Answer: I would recommend getting a copy of the book "Jewish Literacy" by Rabbi Joseph Telushkin.  It covers many different Jewish topics (history, theology, philosophy, culture) in brief, easy-to-read articles.  It is widely available at many libraries and makes for a good "Judaism 101" course.
Answer: I think an interesting text to read might be Pirkei Avot, which is often called 'Abot' or 'Ethics of the Fathers' in English. It is part of a larger text called the Mishnah. Here is a link to the beginning of Pirkei Avot in the English translation of the Mishnah by Herbert Danby:
http://archive.org/stream/DanbyMishnah#page/n475/mode/2up/
The reasons it might be a good reading project are that it is a classic text, it deals with themes that are very important in Judaism (mainly Torah study), and unlike other texts it is not about the details of particular laws, making it more approachable to read.

Judaism for Outsiders


Hierarchical clustering, linkage methods and dynamic time warping

Question: My goal is to cluster time series based on their DTW distance. Therefore I've calculated full distance matrices as input for several clustering algorithms. I first had a look at hierarchical methods, since the number of clusters don't have to be specified at the beginning (moreover k-means is problematic because of the problem of averaging time series under DTW and k-medoids is expensive).
Single linkage (not really useful), complete linkage and average linkage (UPGMA/WPGMA) are unproblematic methods, another criterion which seems to be often used is the Ward method (in R: ward.D2 for the hclust-function). I've seen at least one paper which uses the Ward method with DTW distances, however I am bit skeptical about the usage of Ward in this context. The ward distance of two clusters $A,B$ is defined, according to that paper, as:
$$D_{AB}=\frac{||c_A-c_B||^2}{1/|A|+1/|B|}$$
Where $c_A,c_B$ is the centroid of A, B.
My questions are:

How the centroids are calculated if only a distance matrix is given? (I'm guessing this calculation can't be applied when DTW distances are used.)
Since the application of the Ward method with DTW distances seems to be questionable, are there any other alternative (hierarchical) clustering methods, which can be used with a DTW distance matrix?
Comment: "(moreover k-means is problematic because of the problem of averaging time series under DTW and k-medoids is expensive)."
This is not really the case, see http://www.cs.ucr.edu/~eamonn/ICDM_2014_DTW_average.pdf
Answer: I think you are right to be suspicious of Ward's linkage with DTW dissimilarity based on the paper you referenced. In that paper the authors simply present the Ward's linkage criterion in terms of centroids without considering that the geometric notion of a centroid in a vector space will not coincide with the Frechet mean without a metric. To further complicate matters, consider that DTW allows for a similarity measurement between two vectors of different dimension (ie, time series of different lengths)! Long story short, the meaning of "centroid" in this context is not at all clear.
That said, Ward's linkage using a matrix of DTW dissimilarities is algorithmically possible and may give reasonable results. To answer your specific questions:

It is likely that the authors in the paper you referenced are not directly computing the centroids (for reasons previously stated). Instead, algorithmic implementations of Ward's linkage criterion can work directly from a matrix of dissimilarities. In this paper (see section 2.2), for example, a method for computing the sample variance of the merging of two clusters from a distance matrix (in the case of the L2 metric) is presented. The authors further reference Legendre and Legendre (2012) for additional examples of how to cluster with Ward's linkage without directly computing centroids.
Single, complete, average, and median linkage are all certainly reasonable, as these criteria theoretically depend on the the distance/dissimilarity measure itself rather than variance from a cluster centroid. Ward linkage also may give reasonable results in practice, although I would be cautious of relying on it exclusively because of the ambiguity surrounding the meaning of a centroid in the context of DTW similarity measures. Check out the answers and links at this question for more info on Ward's linkage in non-Euclidean spaces.
Comment: +1. One notion to add, though. Ward's method - in its original definition (irrespective its implementation "ward1" or "ward2") - requires (squared) _euclidean_ distance. It is occasionaly used with other, mostly metric, distances, but such usage is only warranted to the extent to which the distance matrix is de facto close to be spanning the euclidean space. One can check how close it is by observing the magnitude of the eigenvalues of the double-centration matrix ([see point 5](http://stats.stackexchange.com/a/69206/3277)).


Can I start PowerShell background job through PowerShell Web API endpoint?

Question: I have a PowerShell Web API endpoint running on Azure that takes more than the maximum duration of 230 seconds on an Azure Web App. Is it possible to start a background job consisting of PowerShell code from my PowerShell Web API endpoint, return the Sitecore Job ID as result from the Web API endpoint, and check with calls to a PowerShell Web API endpoint if the job is completed?
Answer: If you check out the docs here you can see that there are some commands to create script sessions and later poll for the results. Some of the commands exist both in the Sitecore context as well as in the SPE Remoting module.

Get-ScriptSession - Returns details about script sessions.
Receive-ScriptSession - Returns the results of a completed script session.
Remove-ScriptSession - Removes the script session from memory.
Start-ScriptSession - Executes a new script session.
Stop-ScriptSession - Terminates an existing script session.
Wait-ScriptSession - Waits for all the script sessions to complete before continuing.

Example: The following starts a script that changes its path to "master:" and sleeps for 4 seconds. The session will persist in memory as no -AutoDispose parameter has been provided.
Start-ScriptSession -ScriptBlock { cd master:\; Start-Sleep -Seconds 4 } -Id "Background Task"

Comment: I use these commands with remoting, and they work great! But can I use these also from a Web API call, return a job ID, and poll for the job ID? then I only have to open up restfulv2 with controlled endpoints, instead of opening up PowerShell remoting.
Comment: You should be able to return whatever you want from the web API call. The polling code in SPE remoting could be used in conjunction or can be copied to your web API scripts.


Taking courses beyond requirements in a Ph.D. program

Question: As I'm starting to look to put together my plan of study, a question popped into my mind: are there, generally speaking, rules against taking additional courses beyond what is required for a Ph.D., or is it simply frowned up by most advisers (in the interest of creating more time for research endeavors)? I ask because I've found several graduate certificates offered at my university that seem relevant to my interests (in teaching), and I'd like to take an additional 2 or 3 courses to fulfill their requirements.  I've heard from other grad students in my lab, however, that there is frequently push back from our adviser on such matters.
Comment: Does your advisor need to know your exact academic schedule?
Comment: @aparente001 - I'm not sure; I was under the impression that he would be monitoring (at least from a bird's-eye view) the courses I'm taking so that he knows when to advise me on scheduling various milestones.  But I suppose if that's the case, then no, he might not need to know the exact schedule.
Answer: As an adviser, I encourage my graduate students to take classes they find interesting and/or that broaden their knowledge and therefore are useful to later finding jobs. Ultimately, my goal in this is to help my students become well rounded and broadly educated graduates who will be successful in life. The only caveat I place on this is that they need to make sure they have enough time to do the research that will ultimately lead to their thesis and their defense.
I don't know whether I'm typical in this, but would imagine that most advisers will agree to let their students take other classes if a student asks and gives a reasonable justification. I think most will even agree to this if the student is paid from a grant.
Comment: Note that the type of classes matters. They probably should at least tangentially relate to your field of study (e.g. chemistry classes taken by a biologist), or contribute significantly to your career goals (e.g. teaching classes for the academia-bound). "Fun" classes (e.g. renaissance poetry classes taken by a physicist) would likely be viewed differently.
Comment: @R.M. - I don't think that's the case here - I'm an engineer, and the extra classes are related to teaching and/or engineering education. But I do love poetry, so your comment about Renaissance poetry makes me sad nonetheless haha
Comment: @tonysdg -- talk to the poetry prof. Maybe she lets you just audit the class. That satisfies your curiosity without the commitment to having to  do the homework.
Comment: @Wolfgang Bangerth - haha I was just kidding :) My poetry isn't good enough for a grad-level course anyway, I just write for the enjoyment of it!
Answer: Taking extra classes is a great idea, but be sure to check the financial aspects of your plan. As far as I know, teaching assistantships in my department come with a tuition waiver that covers only a certain number of classes (especially after you've passed your preliminary exams and are supposed to be working on your thesis); if you sign up for more classes, you might have to pay tuition that you weren't expecting to pay.
Comment: This was sort of a concern for me in grad school. However, nothing stopped me (or fellow grad students) from sitting in a course we were not registered for. So if you are only interested learn the material, and are not worried about earning credit units from it ... The problem is, of course, that when you know you are not gonna sit in the exams, your attitude may be less disciplined than it normally would.
Comment: @JyrkiLahtonen I agree that attending classes without registering is an excellent way to learn things. I did that a lot when I was a graduate student.  I omitted it from my answer because the OP asked about using these classes to fulfill the requirements for some certificates, and unregistered attendance at classes presumably won't fulfill any requirements.
Answer: It's quite common to take courses beyond the requirements just because the material is interesting to your or because you think it will be useful in your future career.  However, at a certain point, it will become clear that it is time to focus on finishing your dissertation which after all should be your top priority.  You should discuss this with your academic advisor.
Answer: If your have enough time and you are keen to take additional courses apart from your coursework subjects, then you can take them. I took two additional courses during my PhD. It helped me in many ways, e.g., I could sharpen my knowledge in my field, I could mention these subjects along with my coursework in my application for Post-Doc positions, etc.
Answer: I'd be in favour of taking extra classes. But be careful not to spread too widely, in the end, you'll have to select a line to specialize in. There are many, many subjects that intrigue me; most I have to let pass with regrets.
Whatever you do, check with your advisor if that is how the environment there works. Perhaps talk it over with people in the group, or some other person you trust. For me, the fact that you are troubled enough to ask here is a sign that you should think it over with some help.
Answer: It's ok to take extra courses to support your PhD or just out of personal interest. I took a graduate certificate myself which lasted a year (as a part time course) whilst doing my PhD which I found very helpful. This was more relevant for me because my research is very interdisciplinary requiring skills from multiple fields. However, a note of warning: it must not get in the way of your PhD-this is after all your primary focus. It's a good idea to do any course that will require some commitment from you in your first year of PhD because you're likely to get very busy towards the end of it. Generally speaking, you won't be able to take too many due to financial implications and in order not to get too distracted from your PhD.


FPE Limitations and Scope of FPE

Question: Do you think the format preserving encryption(FPE) schemes have any limitation with respect to other conventional block ciphers? Do FPE have a wide range of applications to be a future research area?
Answer:
Do you think the format preserving encryption(FPE) schemes have any limitation with respect to other conventional block ciphers?

Actually, one can view an FPE scheme as a generalization of a conventional block cipher.  After all, a conventional block cipher is fixed to a specific block size; with FPE, we can use any block size we find convenient (for example, the entire message).

Do FPE have a wide range of applications to be a future research area?

I believe so.  One nice thing about FPE (with a tweak; most FPE schemes allow one) is that it is easy to construct an AEAD scheme that is quite misuse tolerant:

To encrypt, you'd take the message $M$, and append $k$ zeros and $\ell$ random bits (or nonce; the decryptor will recover these bits and so they can be used as a sequence number if needed).  Encrypt that with the FPE scheme, using the AAD as the tweak.

To decrypt, you decrypt with the FPE scheme using the AAD as the tweak; parse the result, and check if the $k$ zeros appear at the end.

It should be obvious that this is a secure AAD scheme (assuming that the FPE scheme is secure).  In addition:

If the sender uses bad randomness for its $\ell$ random bits, it becomes a deterministic scheme where the adversary can determine if the same message was sent twice, but can't determine anything beyond that.

If the receiver forgets to check the $k$ zeros, then this becomes a fully garbling scheme, where the adversary can change the decrypted message into something random, but can't do anything beyond that.  This is in contrast to many AEAD schemes, where if the decryptor forgets to check the tag, then the encryption becomes malleable (and the AAD is essentially ignored).

So, if this is so wonderful, what's the drawback?  Well, one major one is performance - the current FPE schemes are quite slow, and so this straight-forward design isn't used.  A better performing (but still secure) FPE scheme would make this far more attactive.
Another obvious drawback is that FPE cannot be implemented in a "one-pass" manner (where we process the message in parts); this is convenient when we need to handle large messages; however it should be obvious that being able to do misuse-tolerant encryption (where lack of nonce entropy leaks only whether the messages were identical) is incompatible with one-pass encryption, and that being able to do misuse-tolerant decryption (where forgetting to check the integrity check allows an active attacker only to randomize the plaintext) is incompatible with one-pass decryption.
Answer: Limitations (also present in a normal block cipher like AES)

One limitation of Format Preserving Encryption is that it's deterministic: the same plaintext always leads to the same ciphertext. And by an entropy argument, that must be if all plaintexts that fit the format are possible. This in turn prevents meeting the criteria of resistance to Chosen Plaintext Attack.
Another limitation of FPE as practiced is that it's symmetric: a secret (typically, the same) is needed on the encryption side. That's necessary for security of low-entropy plaintext, which is the typical payload of FPE.

Perspectives, starting from the easy

When there are restriction to the plaintext w.r.t. the format, it's easy to somewhat remedy the first bullet point, and even get somewhat Authenticated FPE. For example, when encrypting a credit card number plus expiry date, at least the number's last digit (a checksum) and date field, and to a degree the first 6 digits (identifying the emitter) allow to squeeze some info usable for IV or authentication.
For high-enough entropy payload (say 128-bit), it is feasible to define asymmetric FPE secure within the limits of the first bullet point. It's easy to get this from a trapdoor permutation such as RSA, but this particular one is of limited practical interest for FPE, because FPE is primarily motivated by small payload. The challenge is trapdoor permutation over a smaller interval, say a few hundred bits. I wish I knew the sate of that art.
Comment: Wasn't there also a problem with the message size for specific forms of FPE where it needs to be larger than a specific number of bits?
Comment: @MaartenBodewes: I don't see what you mean in your second comment. We know how to achieve essentially perfect FPE for any interval, for the symmetric breed with identical keys (or any key deducible from the other). Perhaps enrich your comment, since you can!
Comment: The same two limitations you list would also apply to AES; the AES block cipher is also deterministic and is symmetric.

FPE Limitations and Scope of FPE


Bathtub separating from wall

Question: I recently bought an older house. It has had some work done in the master bath, and I just noticed that the tub is separating from the wall. What would be the best way to fix this? Oh I should mention we had the foundation leveled last week, so there are drywall cracks all over the house.
Comment: is the separation due to work in the bathroom or the foundation?
Comment: i think there was some small separation before the foundation work, but then after the foundation was leveled it seemed to separate more
Answer: Appearance of this crack has nothing to do with foundation; it is something that you can see in many bathrooms. First check if both wall and bathtub are vertical, if they are not this crack will most likely appear from time to time (every year or so).Now, you can do one of two things: You can use silicone to glue it together. Keep in mind that silicone (or similar material) should be replaced every 6-12 months you will know when to change because black dots will start to appear. Another thing you can do is that you can put a rubber band and just glue it to both sides. Now this solution is more permanent (you don't need to change it so often) but if you ask me it's not that pretty. Now, foundation....since you did all that work maybe (if possible) you should give your house couple of months or a year to settle before you start doing anything.
Comment: Ok, I will look into the Silicone for right now, just to prevent any more water from going into those cracks until the house settles. On the backside of this wall there is a crack from the foundation repair and the bottom of the crack has turned a slight brown, leading me to believe that water has gotten into the wall through the gap in the bathtub/tile. At least that is what I hope the brown stain is from
Comment: Even if the ceramic tiles have cracked this shouldn't cause any problem with water, because ceramic tiles aren't waterproofing layer. Some mold may appear since crack is small and it cannot be ventilated. Waterproofing layer under ceramic should stop the water from penetrating inside the wall if it is done properly, but now that I think since you had foundation issue it is possible that waterproofing layer has suffered damage especially if it is not elastic. If you want to be on a safe side you should remove ceramic tiles in that part and take a look at waterproofing.
Comment: But since it's not simple and cheap maybe you can make sure that water doesn't get there at all, but that's also not easy.

Bathtub separating from wall


What causes repeat trip of thermostat in dryer?

Question: My dryer did not produce heat. After some online research, there was a solution that included pressing a small red reset button. I did this and the dryer worked for a while, but then the heat stopped. I repeated the "repair", but the dryer stopped producing heat again after a while. What would be my next step. I'm not technical, but hoping there could be something that wont need a specialist for repair.
Comment: 90% of the time... partially or completely clogged exhaust ducting.
Comment: Isherwood that should be an answer. I have found lint and sand blocking the air flow in the past the thermal protector is doing its job so the cause needs to be identified. I agree that it is usually a plugged vent.+
Comment: You could have a problem with the blower fan in the dryer.  Does it make a lot of noise when running?  I had a dryer where the drum fan had a "D" shaped hole for the rod that spun it.  Eventually the hole would wear out to a circle and the rod would spin but not turn the fan much, leading to the dryer shutting down.
Answer: The most probable cause of your dryer over heating is caused by a blockage of air flow in the dryer exhaust venting. This has happened to me and causes the dryer to display a flashing AF (air flow) indicator.
Blockage is generally caused by build up of lint in the venting duct. It can also be due to something closing off the outside end of the vending duct such as snow buildup or or the vent flapper door getting stuck shut.
In my case there is about 22 feet of 4" ducting that unfortunately has to route through seven separate right angle joints as the duct goes up through an inside wall to between floor joints between 1st and 2nd floor before it finally makes it way through the back garage wall to the exterior. Even though the dryer has a good lint screen that is cleaned regularly some lint makes its way out with the warm damp air into the duct. The long duct cools the air and allows a good share of the lint to end up sticking to the inside of the pipe. Eventually over time the duct work needs to be opened up and cleaned....a painful and lengthy job.


determine $\int x\sqrt{1-x^2}\,dx$

Question: I have to determine $\int x\sqrt{1-x^2}\,dx$ and I have a little question about the substitution.
I tried to subsitute $t=1-x^2$. It is $dt=-2xdx$ and therefore $dx=\frac{-dt}{2x}$. But it is the following calculation allowed: $\int x\sqrt{1-x^2}\,dx=\int x\sqrt{t}\,\frac{-dt}{2x}=\int \frac{-1}{2}\sqrt{t}\,dt$ ? (I'm not sure if it is ok to write $\int x\sqrt{t}\,\frac{-dt}{2x}$). Regards
Comment: Yes, it is absolutely ok to write that, provided you remember the functional relation between $x$ and $t$. (Don't take $x$ or $t$ for a constant.)
Answer: You can do as the following :
Since $xdx=-\frac{1}{2}dt$,
$$\int x\sqrt{1-x^2}dx=\int\left(\sqrt{1-x^2}\right)\cdot \color{red}{xdx}=\int \sqrt t\cdot\color{red}{\frac{-1}{2}dt}$$
Answer: An idea:
We have
$$\int\sqrt x\;dx=\frac23 x^{3/2}+C\implies \int f'(x)\sqrt{f(x)}dx=\frac23 (f(x))^{3/2}+C$$
Now, just observe that
$$\;x=-\frac12\left(1-x^2\right)'\;$$
Answer: For complete safety, you can write
$$\int x\sqrt{1-x^2}\,dx=\int x(t)\sqrt{t}\,\frac{-dt}{2\,x(t)}=-\frac12\int \sqrt{t}\,dt.$$
Had the $x$'s not canceled each other, you should have had to substitute $x$ by $\sqrt{1-t}$ in the end.

determine $\int x\sqrt{1-x^2}\,dx$


Requesting constructive criticism for a question

Question: I wrote this question. It was down voted. I'm not upset and am not here to argue. My question here is what can I learn from this, for this or future questions? Is there something inherently wrong with the question that I should have thought about? How can I make this question better?
Comment: Relevant: [Should posts be self-contained?](//meta.stackexchange.com/q/18669)
Comment: I've already [told you](https://unix.stackexchange.com/questions/579853#comment1079908_579853) why both `pivot_root` and `chroot` are used in the example from the manpage. But I'm probably talking out of my ass, and there's something subtle and deep about that obscure system call, since so many people (or so many incarnations of the same) care so deeply about it. It has a metaphysical quality that `move_mount(2)` or `open_tree(2)` do not posess. Something that can only be approached by a humbling and penitential act, like watching a youtube video of some dude blabbering for an hour.
Answer: This is, of course, my personal opinion. I didn't vote on any of your questions.
The main issue I see is that you seem to be asking why a person is using two specific commands in a 50+ minutes YouTube video. This is likely not to be well received:

As it has been pointed out, questions should be self-contained.
That video is lengthy; it would take a non-negligible amount of time to locate the relevant part and properly understand it: what exactly is that person trying to illustrate? How do pivot_root and chroot fit into that? And answers would then likely refer to things that are not stated in your question.

Unfortunately, while the revised question shows your effort to make it more self-contained, you don't seem to be asking about specific points from the quoted text you added. Your main question still seems to be in (emphasis mine):

... I still do not understand why you would pick one over the other in those situations.

Again pointing to the video.
How you can improve your question:

Show that you are not asking others to digest video material for you: providing a link to it is fine, but you should reduce its content to a complete and concise example. If the snippet from pivot_root(8) is a suitable example to you, then simply state what you find unclear in it.
The difference between pivot_root and chroot isn't a well defined concept by itself. They are distinct tools, how different they can be said to be depends on the context in which they are evaluated. Thus,

Please explain the differences, an example could be great.

should probably be limited to a specific context: is it meant to be that of Docker containers? (If so, refer to my previous point). The startup process of some Linux distribution? Something else?
Especially in relation to your revised question: keep it simple. Include every definition (with quotes1) needed to make sure your readers will be aligned with you, but no more than that. Make it clear how the external material integrates with the main point you are asking about.

1 Since you mentioned a second question of yours in a comment to another answer: "I read that ..." sounds vague. You should quote some source, with a strong preference for a primary one.
Answer: If the one down voting your question doesn't leave a comment why (s)he did that, then, as there is nothing to be learned out of it, IMHO all you can do is ignore it.
Comment: @Uberhumus Questions that asks about _explaining concepts_ are harder to write in a way that attracts the wanted type of attention, while question that asks about _solving issues_ are easier to formulate.
Comment: That might be so, but the last two questions I posted are both in the negatives, so I assume there's something I'm doing wrong.


Multiple signin from same device using rails devise gem

Question: I am using active admin and devise gem in my project. However I am using token_authentication to sign in as a user from active admin. Whenever the sign in happens, the session for active admin is destroyed.
How can I do this without destroying active admin session? That means I want to be a user and admin at the same time.
Answer: try incognito window in chrome,
or Private Browsing in Firefox.
or in private browsing in IE.
In this way in every window you'll have different cookies and different sessions.


Why is English word accent traditionally defined as dynamic?

Question: Coud anyone explain why is English word accent traditionally defined as dynamic? Due to some of it's features,history?
Answer: That tradition (from the 19th century) first divides languages by applying "tone" to Asian languages (exemplified by Chinese -- they didn't know about Japanese) and "accent" to other things. Then it further subdivides "accent" into "pitch accent" exemplified by Vedic Sanskrit, Classical Greek or Lithuanian, and "stress" or "dynamic" accent for Russian, English, Spanish and so on. The theory (since disproven) was that stressed syllables in "dynamic accent" languages involve greater expiratory force and higher amplitude on the stressed syllable ("power"), but in "pitch accent" languages all there is is a change of fundamental frequency on that syllable.
Comment: Could you add some information about what led people to think that stressed syllables are louder (if that's what you mean) and what evidence disproved that idea?


Can't set ultawide resolution

Question: I have a 21:9 monitor. In every single linux distro 2560x1080resolution doesn't work for me. Why? My GPU is really old. I fixed it in antergos(arch-based distro!) by installing a package called linux49 from manjaro repository and by setting radeon.hdmimhz=372 in kernel options. After that I was able to select the 2560x1080 resolution. Solus isn't arch-based so maybe someone has an idea how can I fix my issue? I know for sure that adding a custom resolution in xrandr resizes the screen, but it's very blurry and unusable.
GPU: Radeon HD 6870
Comment: not so funny after hearing it for the 100th time when I'm not using arch
Comment: change the distro
Comment: The obvious way is to find out whatever was in the linux49 package, esp. wrt. radeon drivers, write down the version and/or patches they used to built these, and do the same on Solus, either by looking through the repositories, or by compiling it yourself. Yes, this is a lot of work, and you need to know how to compile stuff yourself. But it's better than guessing, installing random stuff and crossing fingers, or complaining it doesn't work. On Linux you *can* do stuff like this yourself.
Comment: @DZDomi We usually don't recommend anyone to change the distribution in here. Stop that, please.
Comment: some people dont have a sense of humor ;)
Answer: Radeon HD 6870 specification

Max Resolution Details DVI: 2560 x 1600 / DisplayPort: 2560 x 1600 / HDMI: 1920 x 1080 / VGA: 2048 x 1536

Conclusion
Neither VGA, nor HDMI < version 2.0 are not capable of delivering such a resolution.
You need to connect through DisplayPort in order to do that.
Comment: So DVI-HDMI should work?
Comment: No. I said your version of HDMI output  is limited to 1080p.

Can't set ultawide resolution


How can I stitch together many images to provide a 3D immersive experience like Photosynth?

Question: I'd like to create a Photosynth like experience that allows a user to navigate around a virtual environment from a collection of many pictures.  For this project I'm limited to using WPF / .NET to create the application.  Ideally I'd like a Photosynth component that I could drop in my application - is anything like that available?
Answer: The underlying components are all GPL'd, look up Photo Tour from Univ of Washington, and there are similar Open Source libs that do the same thing (some even better than photosynth)
Good luck getting them to work though, they are mostly research heavy, with little to no working GUI's, and most of it is done in C/C++, with TONS of math to figure out.
EDIT:
A lot of the algorithms you would need to get started are implemented in the OpenCV project.  All in C/C++ though, and like I said before, the MATH can get pretty heavy.
2nd EDIT:
"Bundler" Is the current project (born from Photo Tourism project)
You may also be interested in looking at "libmv", another similar open sourced library.
And another one worth looking into is "PixelStruct"
Answer: There isn't even an offline viewer, nor a method to "hint" photosynth stitching, so I'd be surprised to see an embeddable component for it (oh, and the site itself seems to advocate the Silverlight client, anyway)
Answer: Two packages I have used with mixed success are VisualSFM and Insight3d (followed by CloudCompare, Meshlab, and CMPMVS).
They are obviously processing intensive, and can take time.  VisualSFM can use the GPU on an NVidia card, which speeds it up significantly.
Getting a good solution depends heavily on the photos being used.  I have used VisualSFM for aerial photo stitching, and so far it has worked best with frame grabs from a video (using avconv).
Both of these use libraries for which the source is available, such as blas, graclus, opencv, cmvs, pmvs and SiftGPU.  These libraries contain the functionality you are after, but it is far from a 'drop in' solution.


How can I change the particles density and its direction in volume?

Question: I'd like to have the number of particles greater in the bottom , and they get lower towards the upper parts of the cube, it's because I don't want them to spread homogenously. How can I make it I have no idea? can you please help me? thanks
Answer: Use weight painting to create a vertex group which will control density of the particles or create new texture in the particle system settings.
Weight painting
Prepare mesh topology
Weight paint affects on vertices, so in order painting to be smoother you'll need a lot of vertices. The more of them, the smoother will be painting, and so distribution of particles.
Use loopcuts or Subdivision Surface modifier to make topology of the mesh more dense.

Use Weight painting to determine how to spread particles
Create new vertex group (or it will be added automatically), switch Object mode to Weight Paint and use brushes to paint areas where you'd like particles to be located more dense. Weight of 0 will make no particles in that area, weight of 1 will make to add maximum amount of particles (if the vertex group effect isn't inverted).
You can also use Weight tools, located in Tools panel (opened with T) to mirror, invert, normalize weights etc.

Set created vertex group to control density of particles
In Vertex Groups scroll locate Density field and choose created vertex group.

Note that you could also invert vertex group influence, to make particles being distributed in the top of the mesh instead of bottom (button just near the dropdown menu where you choose vertex group to affect).

Use texture to control particles' density
In the settings of the Particle system create new texture:

For simple setup Blend texture may be used. Moving control points of the ColorRamp will make density of the particles change.

The last thing is that by default this texture affects only time, not density. So in the Influence scroll check Density (you can leave Time if you need it).
Comment: thank you very much for this very helpful and informative reply, i'm gonna study them now :)
Comment: yes weight paint method did work very well for me too , thanks again.. I haven't tried the texture method yet tho, I kinda preferred the randomness of hand painted weight paint
Comment: Mr Zak have you experienced success with Blender Internal Render .... with a image sequence to control location/density?  Not Cycles Render.
Comment: @atomicbezierslinger If I remember correctly, that was the BI where I've done the test file presented in the answer. However, I tried that in BI now and it works for me. Still I'm not sure what do you mean saying "image sequence". I used simple texture for particles - either procedural or image file.
Comment: In the Cycles particles, then texture panel, [Type] field Image or Movie, Image Panel ... [Source] field.  There are 4 choices on the drop down menu in the Blender GUI.  Generated, Movie, [Image Sequence], Single Image.  Perhaps [Single Image] is the most common.   I selected more than one image, which is an animated texture, image sequence.  Because I want to generate particles from an image sequence, rather than a single  image.    You can try this out by selecting multiple images in an new particle texture example.   ..... Have you had any success with an image sequence?
Comment: I didn't understand because this is a question on another tricks. It didn't work exactly with images in the way as described here (and it's quite expected as all the particles're already born), and I think it's worth trying to use single image and changing its offset instead. However this is definitely not the place to discuss this topic as it's completely another question and wasn't included in the original answer. You may want to ask another question instead.


Searching in Paillier Cryptosystem

Question: I have implemented Paillier Cryptosystem. Lets say, I have an encrypted array E(x) = [2,4,5,10,0,20] and I want to find that if 0 exist in that array. Due to the limitations of Paillier cryptosystem I cannot multiply two ciphertext. Is there any other way to find it?
Comment: Anything wrong with decrypting the array and finding out? Sure that requires the private key, but then deducing anything about the plaintext (except it's length) from the ciphertext is bound to require the private key. Thus if this simple solution is not satisfactory, then something is missing in the problem statement. While we are at changing the problem statement does "I have an encrypted array" imply you are not free to change how the data is encoded before being encrypted?
Comment: I wanted to get a single encrypted value from the array E(x) which on decryption indicates the presence/ absence of zero. I cannot decrypt the array as my task is to search for 0 in encrypted data. I don't want to return the full array to the person having a Private Key.
Answer: Paillier is a CPA-secure cipher, thus anything we can deduce from ciphertexts requires the private key. Thus "I want to find…" requires that "I" has the private key, and then "I have an encrypted array" implies "I" can decipher each of the ciphertexts, which makes the question as stated moot.
We'll therefore change the  problem statement to: we have at most $k$ small integer $m_i\in[0,\ell)$ (in the problem at hand we want $k\ge6$, and $\ell>20$); and we want a public method to express these $m_i$ encrypted as $c_i$ using the Paillier Cryptosystem, and combine the $c_i$ into a single $c$ as we do in Paillier. That is making a product modulo $n$ of the $c_i$, and optionally multiply modulo $n$ by a ciphertext for $0$ to further mask the outcome. We want things so that a person holding the private key can determine from $c$ if $0$ was present in the initial data set of $m_i$. That changes two important things in the initial problem statement:

There's this notion of combining the ciphertexts $c_i$ into $c$, with the $c_i$ unavailable to the holder of the private key.
We do not "have an encrypted array". We want to have one, but are free to define how it's made, as long as that's with an encryption that uses Paillier. Without such freedom, there is no solution, because Paillier decryption will only let one obtain $m=(\sum m_i)\bmod n$ and there's no way to tell from that if an $m_i$ was $0$.

Here is something where we'll not only know if $0$ was present in the $m_i$, we'll also know how many times each integer in $[0,\ell)$ was present in the original array.
Towards this, we encode $m_i$ into $m'_i=(k+1)^{m_i}$ before applying Paillier encryption. Paillier will let us combine the ciphertexts $c_i$ then decipher the combination $c$ into $m'=(\sum m'_i)\bmod n$. From $m'$ we can deduce how many time each integer in $[0,\ell)$ was present in the $m_i$, as long as $k(k+1)^{\ell-1}<n$, by expressing $m'$ in base $k+1$ and looking at the digits/limbs. I'll skip the algebra for how, and use an example in decimal (thus $k=9$). $m_0=2$, $m_1=4$, $m_2=5$, $m_3=10$, $m_4=0$, $m_5=20$ encodes to $m'_0=100$, $m'_1=10000$, $m'_2=100000$, $m'_3=10000000000$, $m'_4=1$, $m'_5=0$, $m'_6=100000000000000000000$ in decimal. We'll get $m'=1000000000010000110101$, from which we can tell there was exactly one each of $0$, $2$, $4$, $5$, $10$, $20$ in the original array of $m_i$, by looking at the position of the non-zero digits and seeing they are $1$. If the $20$ was changed to $0$, the outcome would be $m'=10000110102$ and we'd know there was two $m_i=0$, because the rightmost digit is $2$.

The question does not tell what we don't want the holder of the private key to be able to determine from $c$. Perhaps we want to restrict that to determining if there was a $0$, with as little leakage about how many $0$ as possible.
Towards this, we can encode each $m_i\ne0$ to $m'_i=0$, and each $m_i=0$ to some integer $m'_i$ in $[1,\lfloor n/k\rfloor)$ picked at random with a distribution heavily biased towards low values. When the decryption of the combination is zero, it's known there was no zero in the $m_i$, and nothing else. Otherwise, it's known there was a zero, and it is leaked little information about how many. The distribution can be optimized towards minimizing that leakage, this is left as an exercise to the reader.
It's possible to refine things so that one with the private key and $c_i$ can find $m_i$, yet (as above) one with the private key and $c$ can get little information about the $m_i$, beyond that one is $0$. I won't detail because that's not in the problem statement.

As many homomorphic encryption things, these are nice theoretical systems with little matching real-life problems.

Searching in Paillier Cryptosystem


Can this lumen to lux calculator be trusted?

Question: According to this tool I've calculated that a modern, regular LED light bulb (LED 16W) would emit ~2767 lux at a 0.5 meter distance. Would this imply, that a persons face, exposed at such distance is subject to a ray of light that is ~2767 lux bright?
Answer: Since a half-meter radius sphere is about 3 m^2 area, the bulb would have to emit about 8300 lumens, which is around eight times what an actual 16W LED puts out. Also, a person's face is nowhere near a square meter area, so the actual flux would be much lower than 2767 lumens even if that number were correct.
A typical 16W bulb will have a flux at 0.5m of closer to 350 lux, and without calculating facial effective area, I'd guess the total lumens in a human face at that distance to be about 20.


Kotlin Could not resolve all files for configuration ':app:debugRuntimeClasspath'

Question: I upgraded my android studio version and now I am having this issue. I have tried some couple of solutions that I found(toggle offline mode) but they did not work.
Answer: I solved that problem by adding jcenter() to the repositories instead of maven(). It worked well so far but I am not quite sure if it is the correct way.


Java using much more memory than heap size (or size correctly Docker memory limit)

Question: For my application, the memory used by the Java process is much more than the heap size.
The system where the containers are running starts to have memory problem because the container is taking much more memory than the heap size.
The heap size is set to 128 MB (-Xmx128m -Xms128m) while the container takes up to 1GB of memory. Under normal condition, it needs 500MB. If the docker container has a limit below (e.g. mem_limit=mem_limit=400MB) the process gets killed by the out of memory killer of the OS.
Could you explain why the Java process is using much more memory than the heap? How to size correctly the Docker memory limit? Is there a way to reduce the off-heap memory footprint of the Java process?

I gather some details about the issue using command from Native memory tracking in JVM.
From the host system, I get the memory used by the container.
$ docker stats --no-stream 9afcb62a26c8
CONTAINER ID        NAME                                                                                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
9afcb62a26c8        xx-xxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.0acbb46bb6fe3ae1b1c99aff3a6073bb7b7ecf85   0.93%               461MiB / 9.744GiB   4.62%               286MB / 7.92MB      157MB / 2.66GB      57

From inside the container, I get the memory used by the process.
$ ps -p 71 -o pcpu,rss,size,vsize
%CPU   RSS  SIZE    VSZ
11.2 486040 580860 3814600

$ jcmd 71 VM.native_memory
71:

Native Memory Tracking:

Total: reserved=1631932KB, committed=367400KB
-                 Java Heap (reserved=131072KB, committed=131072KB)
                            (mmap: reserved=131072KB, committed=131072KB)

-                     Class (reserved=1120142KB, committed=79830KB)
                            (classes #15267)
                            (  instance classes #14230, array classes #1037)
                            (malloc=1934KB #32977)
                            (mmap: reserved=1118208KB, committed=77896KB)
                            (  Metadata:   )
                            (    reserved=69632KB, committed=68272KB)
                            (    used=66725KB)
                            (    free=1547KB)
                            (    waste=0KB =0.00%)
                            (  Class space:)
                            (    reserved=1048576KB, committed=9624KB)
                            (    used=8939KB)
                            (    free=685KB)
                            (    waste=0KB =0.00%)

-                    Thread (reserved=24786KB, committed=5294KB)
                            (thread #56)
                            (stack: reserved=24500KB, committed=5008KB)
                            (malloc=198KB #293)
                            (arena=88KB #110)

-                      Code (reserved=250635KB, committed=45907KB)
                            (malloc=2947KB #13459)
                            (mmap: reserved=247688KB, committed=42960KB)

-                        GC (reserved=48091KB, committed=48091KB)
                            (malloc=10439KB #18634)
                            (mmap: reserved=37652KB, committed=37652KB)

-                  Compiler (reserved=358KB, committed=358KB)
                            (malloc=249KB #1450)
                            (arena=109KB #5)

-                  Internal (reserved=1165KB, committed=1165KB)
                            (malloc=1125KB #3363)
                            (mmap: reserved=40KB, committed=40KB)

-                     Other (reserved=16696KB, committed=16696KB)
                            (malloc=16696KB #35)

-                    Symbol (reserved=15277KB, committed=15277KB)
                            (malloc=13543KB #180850)
                            (arena=1734KB #1)

-    Native Memory Tracking (reserved=4436KB, committed=4436KB)
                            (malloc=378KB #5359)
                            (tracking overhead=4058KB)

-        Shared class space (reserved=17144KB, committed=17144KB)
                            (mmap: reserved=17144KB, committed=17144KB)

-               Arena Chunk (reserved=1850KB, committed=1850KB)
                            (malloc=1850KB)

-                   Logging (reserved=4KB, committed=4KB)
                            (malloc=4KB #179)

-                 Arguments (reserved=19KB, committed=19KB)
                            (malloc=19KB #512)

-                    Module (reserved=258KB, committed=258KB)
                            (malloc=258KB #2356)

$ cat /proc/71/smaps | grep Rss | cut -d: -f2 | tr -d " " | cut -f1 -dk | sort -n | awk '{ sum += $1 } END { print sum }'
491080

The application is a web server using Jetty/Jersey/CDI bundled inside a fat far of 36 MB.
The following version of OS and Java are used (inside the container). The Docker image is based on openjdk:11-jre-slim.
$ java -version
openjdk version "11" 2018-09-25
OpenJDK Runtime Environment (build 11+28-Debian-1)
OpenJDK 64-Bit Server VM (build 11+28-Debian-1, mixed mode, sharing)
$ uname -a
Linux service1 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 GNU/Linux

https://gist.github.com/prasanthj/48e7063cac88eb396bc9961fb3149b58
Comment: The heap is where objects are allocated, however the JVM has many other memory regions including shared libraries, direct memory buffers, thread stacks, GUI components, metaspace. You need to look at how large the JVM can be and make the limit high enough that you would rather the process die than use any more.
Comment: It looks like the GC is using a lot of memory. You could try using the CMS collector instead. It looks like ~125 MB is used for metaspace + code, however without shrinking your code base, you are unlikely to be able to make that smaller. The committed space is close to your limit so it's not surprising it gets killed.
Comment: Note: [Java 8 u212+ has now (Apr. 2019) a better Docker support when it comes to memory](https://stackoverflow.com/a/56153528/6309)
Comment: Does you program execute many file-operations (eg creates files in gigabyte size)? If so, you should know that `cgroups` adds disk-cache to the used memory -- even if it is handled  by the kernel and it is invisible for the user program. (Mind you, commands `ps` and `docker stats` don't count disk-cache.)
Comment: where / how do you set the -Xms and -Xmx configuration?
Comment: In the Java command line `java -Xmx128m -Xms128m -jar the-bundle-package.jar`
Comment: I think it's worth mentioning this article : [Java SE support for Docker CPU and memory limits](https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits)
Comment: It writes logs but not in GB size, it is a webapp based on Jersey/CDI similar to this bundle published in [GitHub](https://github.com/nhenneaux/jersey-http2-jetty-connector/tree/master/jersey-http2-jetty-bundle)
Comment: If you don't know what exactly is `MiB` and `GiB` in response to get memory used by the container from the host system: https://askubuntu.com/questions/22102/meaning-of-i-in-mib
Answer: Virtual memory used by a Java process extends far beyond just Java Heap. You know, JVM includes many subsytems: Garbage Collector, Class Loading, JIT compilers etc., and all these subsystems require certain amount of RAM to function.
JVM is not the only consumer of RAM. Native libraries (including standard Java Class Library) may also allocate native memory. And this won't be even visible to Native Memory Tracking. Java application itself can also use off-heap memory by means of direct ByteBuffers.
So what takes memory in a Java process?
JVM parts (mostly shown by Native Memory Tracking)
1. Java Heap
The most obvious part. This is where Java objects live. Heap takes up to -Xmx amount of memory.
2. Garbage Collector
GC structures and algorithms require additional memory for heap management. These structures are Mark Bitmap, Mark Stack (for traversing object graph), Remembered Sets (for recording inter-region references) and others. Some of them are directly tunable, e.g. -XX:MarkStackSizeMax, others depend on heap layout, e.g. the larger are G1 regions (-XX:G1HeapRegionSize), the smaller are remembered sets.
GC memory overhead varies between GC algorithms. -XX:+UseSerialGC and -XX:+UseShenandoahGC have the smallest overhead. G1 or CMS may easily use around 10% of total heap size.
3. Code Cache
Contains dynamically generated code: JIT-compiled methods, interpreter and run-time stubs. Its size is limited by -XX:ReservedCodeCacheSize (240M by default). Turn off -XX:-TieredCompilation to reduce the amount of compiled code and thus the Code Cache usage.
4. Compiler
JIT compiler itself also requires memory to do its job. This can be reduced again by switching off Tiered Compilation or by reducing the number of compiler threads: -XX:CICompilerCount.
5. Class loading
Class metadata (method bytecodes, symbols, constant pools, annotations etc.) is stored in off-heap area called Metaspace. The more classes are loaded - the more metaspace is used. Total usage can be limited by -XX:MaxMetaspaceSize (unlimited by default) and  -XX:CompressedClassSpaceSize (1G by default).
6. Symbol tables
Two main hashtables of the JVM: the Symbol table contains names, signatures, identifiers etc. and the String table contains references to interned strings. If Native Memory Tracking indicates significant memory usage by a String table, it probably means the application excessively calls String.intern.
7. Threads
Thread stacks are also responsible for taking RAM. The stack size is controlled by -Xss. The default is 1M per thread, but fortunately  things are not so bad. The OS allocates memory pages lazily, i.e. on the first use, so the actual memory usage will be much lower (typically 80-200 KB per thread stack). I wrote a script to estimate how much of RSS belongs to Java thread stacks.
There are other JVM parts that allocate native memory, but they do not usually play a big role in total memory consumption.
Direct buffers
An application may explicitly request off-heap memory by calling ByteBuffer.allocateDirect. The default off-heap limit is equal to -Xmx, but it can be overridden with -XX:MaxDirectMemorySize. Direct ByteBuffers are included in Other section of NMT output (or Internal before JDK 11).
The amount of direct memory in use is visible through JMX, e.g. in JConsole or Java Mission Control:

Besides direct ByteBuffers there can be MappedByteBuffers - the files mapped to virtual memory of a process. NMT does not track them, however, MappedByteBuffers can also take physical memory. And there is no a simple way to limit how much they can take. You can just see the actual usage by looking at process memory map: pmap -x <pid>
Address           Kbytes    RSS    Dirty Mode  Mapping
...
00007f2b3e557000   39592   32956       0 r--s- some-file-17405-Index.db
00007f2b40c01000   39600   33092       0 r--s- some-file-17404-Index.db
                           ^^^^^               ^^^^^^^^^^^^^^^^^^^^^^^^

Native libraries
JNI code loaded by System.loadLibrary can allocate as much off-heap memory as it wants with no control from JVM side. This also concerns standard Java Class Library. In particular, unclosed Java resources may become a source of native memory leak. Typical examples are ZipInputStream or DirectoryStream.
JVMTI agents, in particular, jdwp debugging agent - can also cause excessive memory consumption.
This answer describes how to profile native memory allocations with async-profiler.
Allocator issues
A process typically requests native memory either directly from OS (by mmap system call) or by using malloc - standard libc allocator. In turn, malloc requests big chunks of memory from OS using mmap, and then manages these chunks according to its own allocation algorithm. The problem is - this algorithm can lead to fragmentation and excessive virtual memory usage.
jemalloc, an alternative allocator, often appears smarter than regular libc malloc, so switching to jemalloc may result in a smaller footprint for free.
Conclusion
There is no guaranteed way to estimate full memory usage of a Java process, because there are too many factors to consider.
Total memory = Heap + Code Cache + Metaspace + Symbol tables +
               Other JVM structures + Thread stacks +
               Direct buffers + Mapped files +
               Native Libraries + Malloc overhead + ...

It is possible to shrink or limit certain memory areas (like Code Cache) by JVM flags, but many others are out of JVM control at all.
One possible approach to setting Docker limits would be to watch the actual memory usage in a "normal" state of the process. There are tools and techniques for investigating issues with Java memory consumption: Native Memory Tracking, pmap, jemalloc, async-profiler.
Update
Here is a recording of my presentation Memory Footprint of a Java Process.
In this video, I discuss what may consume memory in a Java process, how to monitor and restrain the size of certain memory areas, and how to profile native memory leaks in a Java application.
Comment: @j-keck String objects are in the heap, but the hashtable (the buckets and the entries with references and hash codes) is in off-heap memory. I reworded the sentence to be more precise. Thanks for pointing out.
Comment: Are not interned Strings in the heap since jdk7? (https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6962931) - maybe i'm wrong.
Comment: to add to this, even if you use non-direct ByteBuffers, the JVM will allocate temporary direct buffers in native memory with no memory limits imposed. Cf. http://www.evanjones.ca/java-bytebuffer-leak.html
Answer: https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/:

Why is it when I specify -Xmx=1g my JVM uses up more memory than 1gb
  of memory?
Specifying -Xmx=1g is telling the JVM to allocate a 1gb heap. It's not
  telling the JVM to limit its entire memory usage to 1gb. There are
  card tables, code caches, and all sorts of other off heap data
  structures. The parameter you use to specify total memory usage is
  -XX:MaxRAM. Be aware that with -XX:MaxRam=500m your heap will be approximately 250mb.

Java sees host memory size and it is not aware of any container memory limitations. It doesn't create memory pressure, so GC also doesn't need to release used memory. I hope XX:MaxRAM will help you to reduce memory footprint. Eventually, you can tweak GC configuration (-XX:MinHeapFreeRatio,-XX:MaxHeapFreeRatio, ...)

There is many types of memory metrics. Docker seems to be reporting RSS memory size, that can be different than "committed" memory reported by jcmd (older versions of Docker report RSS+cache as memory usage).
Good discussion and links: Difference between Resident Set Size (RSS) and Java total committed memory (NMT) for a JVM running in Docker container
(RSS) memory can be eaten also by some other utilities in the container - shell, process manager, ...  We don't know what else is running in the container and how do you start processes in container.
Comment: It's indeed better with `-XX:MaxRam`. I think it is still using more than the maximum defined but it's better, thanks !
Comment: Maybe you really need more memory for this Java instance. There are 15267 classes, 56 threads.
Comment: The provided RSS is from inside the container for the Java process only `ps -p 71 -o pcpu,rss,size,vsize` with Java process having pid 71. Actually `-XX:MaxRam` was not helping but the link you have provided helps with serial GC.
Answer: TL;DR
The detail usage of the memory is provided by Native Memory Tracking (NMT) details (mainly code metadata and garbage collector). In addition to that, the Java compiler and optimizer C1/C2 consume the memory not reported in the summary.
The memory footprint can be reduced using JVM flags (but there is impacts).
The Docker container sizing must be done through testing with the expected load the application.

Detail for each components
The shared class space can be disabled inside a container since the classes won't be shared by another JVM process. The following flag can be used. It will remove the shared class space (17MB).
-Xshare:off

The garbage collector serial has a minimal memory footprint at the cost of longer pause time during garbage collect processing (see Aleksey Shipilëv comparison between GC in one picture). It can be enabled with the following flag. It can save up to the GC space used (48MB).
-XX:+UseSerialGC

The C2 compiler can be disabled with the following flag to reduce profiling data used to decide whether to optimize or not a method.
-XX:+TieredCompilation -XX:TieredStopAtLevel=1

The code space is reduced by 20MB. Moreover, the memory outside JVM is reduced by 80MB (difference between NMT space and RSS space). The optimizing compiler C2 needs 100MB.
The C1 and C2 compilers can be disabled with the following flag.
-Xint

The memory outside the JVM is now lower than the total committed space. The code space is reduced by 43MB. Beware, this has a major impact on the performance of the application. Disabling C1 and C2 compiler reduces the memory used by 170 MB.
Using Graal VM compiler (replacement of C2) leads to a bit smaller memory footprint. It increases of 20MB the code memory space and decreases of 60MB from outside JVM memory.
The article Java Memory Management for JVM provides some relevant information the different memory spaces.
Oracle provides some details in Native Memory Tracking documentation. More details about compilation level in advanced compilation policy and in disable C2 reduce code cache size by a factor 5. Some details on Why does a JVM report more committed memory than the Linux process resident set size? when both compilers are disabled.
Answer: Java needs a lot a memory. JVM itself needs a lot of memory to run. The heap is the memory which is available inside the virtual machine, available to your application. Because JVM is a big bundle packed with all goodies possible it takes a lot of memory just to load.
Starting with java 9 you have something called project Jigsaw, which might reduce the memory used when you start a java app(along with start time). Project jigsaw and a new module system were not necessarily created to reduce the necessary memory, but if it's important you can give a try.
You can take a look at this example: https://steveperkins.com/using-java-9-modularization-to-ship-zero-dependency-native-apps/. By using the module system it resulted in CLI application of 21MB(with JRE embeded). JRE takes more than 200mb. That should translate to less allocated memory when the application is up(a lot of unused JRE classes will no longer be loaded).
Here is another nice tutorial: https://www.baeldung.com/project-jigsaw-java-modularity
If you don't want to spend time with this you can simply get allocate more memory. Sometimes it's the best.
Comment: Using `jlink` is quite restrictive as it required the application to be modularised. Automatic module is not supported so there is no easy way to go there.
Answer: How to size correctly the Docker memory limit?
Check the application by monitoring it for some-time. To restrict container's memory try using -m, --memory bytes option for docker run command - or something equivalant if you are running it otherwise
like
docker run -d --name my-container --memory 500m <iamge-name>

can't answer other questions.


Most secure way to connect from Windows 10 to a CentOS8 Shared Desktop

Question: i tried to connect to a CentOS8 desktop from a Windows 10. In CentOS8 I have enabled the Sreen Sharing feature. Then i have tried to connect to the ip address of my CentOS8 from a Windows 10 with different VNC clients (VNCviewer from RealVNC, TightVNC, TigerVNC). All of them aborted the connection with an error message like this one from TightVNC:

Error in TightVNC Viewer: No security types supported. Server sent security types, but we do not support any of them.

Most suggestions in blog posts recommend to disable encryption at the vnc server and connect using a ssh tunnel. I also have read that somehow the vnc viewers of the world don't support tls on windows 10. But why?  What should be a reliable good to use setup for a secure vnc session?
If it comes to an alternative like rdp, i thought it is a closed protocol developed by Microsoft.
Thank you
Comment: This topic is getting more relevant these days: https://www.netscout.com/blog/asert/microsoft-remote-desktop-protocol-rdp-reflectionamplification
Answer: One can disregard any Microsoft proprietary protocols by using an X11 server on your Windows box and piping the X protocols through an encrypted ssh connection (ssh -X).
See this StackOverflow posting, or this link from Hofstra University.


cc slant not working in after effects

Question: I put the cc slant effect on an image just to slant (move) the clouds, but it ends up sliding the whole image. The same goes for many other images even after adding a mask. The whole image keeps sliding with the slant.
Answer: Not exactly sure what the problem is. However, try making a adjustment layer above your footage. Apply your mask to it. Then add a transform effect. It has a skew and skew axis that can be keyframed
Answer: Pre comp the masked image. Then apply the slant to the outer comp.

cc slant not working in after effects


can i use laravel Queues to delete some table records automatically?

Question: I need delete some records from my one table which old than 15 days form now. I am working with laravel 5.6 and hope to use Queues for this job. and do not hope to use Task Scheduler for this task. my job is like this.
<code>$vehicles = Vehicle::where('created_at', '<', Carbon::now()->subDays(15))->delete();
</code>
My problem is I need run above task every day. so may I use Queues to fulfill this task for every day?
Answer: The short answer is no. You place jobs into queues. A job could be created to do this task but what is going to place the job into a queue?  This would typically either happen during some kind of fired event or if it needs to be done on a schedule (like daily) it would be done by the task scheduler.
If you must accomplish this with a queue and without the task scheduler, you could write a custom console command and kick it off with a cron entry.


Within and Between Variation in fixed and random effects models

Question: I want to get a deepened understanding of random effects models and how they compare to fixed effects models. While there are some great answers already given by the community the following question remains unclear to me:
Broadly speaking one could say that before analyzing values across different units of analysis, each value in a fixed effects model gets subtracted by the mean value of the respective variable, right? Thus, all variation in a fixed effects model is relative to the unit of analysis "baseline-level". Consequently, applying fixed effects models with data with no/few variation is not useful.
I read in several publications that random-effects models are in such settings a better option. So do random effects models similar to fixed effects model consider within variation before they analyse variation between units?
Answer: RE models rely on within- and between-group variation, while FE models only rely on within-group estimation.
However, this is only true if the explanatory variable is independent from group specific effects. If this is not the case RE are biased.


How to calculate the basin length - which point to which point and how in ArcGIS 9.3?

Question:
This is the area of my research. I have calculated area using area and perimeter using ArcGIS 9.3 for doing morphometric analysis of the area.but I don't know how to calculate the length of the basin that is which start and end point I have to take to calculate using ArcGIS 9.3.
Comment: There are multiple ways for computing Basin Length, which method were you intending to use? Have a look at this [link](http://books.google.co.uk/books?id=x29Y2cb0NegC&pg=PA99&lpg=PA99&dq=basin+length&source=bl&ots=_WE4PaxCBV&sig=xIktT4Gc2Ezf_tKVLs034L_qk-I&hl=en&sa=X&ei=qeQ1UoGaCYL27Ab76oCACQ&ved=0CD8Q6AEwAA#v=onepage&q=basin%20length&f=false) to Google Books. Which method you intend to use will dictate different approaches.
Answer: Okay I'm going to propose a solution based on the functionality of the open source GIS, Whitebox GAT (http://www.uoguelph.ca/~hydrogeo/Whitebox/) because it's easy to do in this software and it's what I use. I imagine, given the cost of ArcGIS, that there are equivalent methods in Arc however.
I assume based on your image that you have a hydrologically corrected (depressions removed DEM) already. If you do not, then I'd point you to Whitebox's Fill Depressions and Breach Depressions tools.

Import your DEM into Whitebox using the 'Import ArcGIS binary grid (.flt)' tool. This will convert an Arc floating point grid into the native Whitebox file. If you have a raster basin file already, then import this file too. If you only have the basin vector file (as displayed above) then Whitebox's native vector format is a shapefile, so you probably don't need to import it. However, it will need to be converted to a raster of the same dimensions and grid resolution as the DEM so use Vector Polygon to Raster tool, specifying the DEM as the base grid.
Run the 'Downslope Flowpath Length' tool. It'll require a D8 flow pointer grid, which you can easily extract from your DEM (D8 flow pointer tool), as well as your rasterized basin (watershed) raster.
Use the 'Extract Statistics' tool specifying the flowpath length raster as the data raster and the basin file as the feature definition raster. Extract the 'maximum' statistic. This will spit out a text file that will contain the maximum flowpath length in your basin. The nice thing about this approach, is that it'll calculate this for any number of basins simultaneously.

Good luck and let me know if you have any questions.


How to remove custom extension and its entries from Database?

Question: I am developing a custom magento extension to add attributes and attribute sets when anyone extract the zip file and put that code to "/app" folder.

It makes several entries to database for atributes.
Now I am making further changes in attribute sets and want to install this module again.
So for this I am installing fresh version of magento to test the extension.
So any way to remove extension and its relavant entries from database?
I am not installing this extension from "Magento Connect Manager".
Answer: If the Connect Manager does not pick your extension up, you need to remove those entries by hand or write an upgrade script removing them for you. The files need to be removed from app folder by hand anyways.
If you remove the extention and its entries by hand you need to remove it's entry from the core_resource table in Magento, too.
For more specific help you need to provide a little more information on how your module works.


How to use drag and drop within a UIScrollView

Question: I am trying to drag and drop UIViews within a UIScrollView. I'd like the UIScrollView to scroll if I drag a UIView so that it intersects the top or bottom of the ScrollView. If it touches the top, it should scroll up. If it touches the bottom, it should scroll down. Once the UIView is moved such that it is no longer intersecting the top or bottom, the scrolling should stop.
The first approach I tried was to start an NSTimer whenever I detected that the UIView intersected with the top or bottom of the scrollview. The timer would scroll the UIScrollView bit by bit until it was invalidated. This resulted in very jerky scrolling, since there is no way to predict when the timers will fire.
My current approach is to simply scroll all the way to the top or bottom of the ScrollView, like this:
[UIView beginAnimations:nil context:NULL];
[UIView setAnimationCurve:UIViewAnimationCurveEaseInOut];
[UIView setAnimationDuration:1.0];
[self.parentScrollview setContentOffset:newOffset animated:NO];
[UIView commitAnimations];

In order to stop scrolling, I try to determine the position of the touch in the touchesMoved: delegate, and use that to calculate the contentOffset to use for the scrollView.
The problem is, when I try to get the location in the view via [[touches anyObject] locationInView:self], I get the final end location, rather than the current location. I'm guessing that's because the contentOffset on the scrollView is set immediately.
Is there a way to get the actual location of a touch in a situation like this? Or any other way to set the contentOffset when I interrupt the scrolling animation?
Answer: This may be late, but you may try interrupting the old animation with a new animation including this statement:
    [UIView setAnimationBeginsFromCurrentState:YES];

You could make the new animation very short or zero duration, and then set a delegate and setAnimationDidStopSelector to get the final contentOffset value.


My iPad unable to view subtitles

Question: My iPad unable to open or view video online with subtitles. But when I open the same website with my lappy, the video and subtitles work well. There have something wrong with the website or my iPad?
Answer: Try this:

Launch the Settings app on your iPhone or iPad.
Tap on General.
Tap on Accessibility.
Under the Hearing section, tap on Subtitles & Captioning.
Turn On the option for Closed Captions + SDH.
Tap on Style in order to customize how closed captions work if you'd like.

Also, what app/website are you viewing this in?
Source(s):

Google (googled "subtitles on iPad")
http://www.imore.com/how-enable-subtitles-and-captioning-auditory-accessibility-iphone-or-ipad

My iPad unable to view subtitles


Find all functions $f(x)$ such that $f\left(x^2+f(y)\right)$=$y+(f(x))^2$

Question: Let $\mathbb R$ denote the set of all real numbers. Find all function $f: R\to \ R$ such that   $$f\left(x^2+f(y)\right)=y+(f(x))^2$$   It is the problem. I tried to it by putting many at the place of $x$ and $y$ but I can't proceed. Please somebody help me.
Comment: $f(x)=x$ is a solution
Answer: Let me summarize the proposed ideas and give a self-contained answer: Let $f : \Bbb{R} \to \Bbb{R}$ satisfy the functional equation
$$ f(x^2 + f(y)) = y + f(x)^2 \tag{*}$$
for all $x, y \in \Bbb{R}$.

Step 1. $f(0) = 0$.

(This solution is due to @Leo163.) Let $a \in \Bbb{R}$ be such that $f(a) = 0$. For instance, plugging $y = -f(x)^2$ to $\text{(*)}$ confirms the existence of such $a$. Then plugging $(x, y) = (a, a)$ to $\text{(*)}$ gives
$$ f(a^2) = f(a^2 + f(a)) = a + f(a)^2 = a.$$
Finally, plugging $(x, y) = (0, a^2)$ to $\text{(*)}$ gives
$$ 0 = f(a) = f(f(a^2)) = a^2 + f(0)^2. $$
This shows that $f(0) = 0$.

Step 2. $f$ satisfies various identities:

$f(x^2) = f(x)^2$.
$f(f(x)) = x$. In particular, $f$ is bijective.
$f(-x) = -f(x)$.
$f(x+y) = f(x)+f(y)$.

The identity 1 (resp. 2) follow by plugging $y = 0$ (resp. $x = 0$) to $\text{(*)}$. For 3, we may assume that $x \geq 0$. Then replacing $(x, y)$ in $\text{(*)}$ by $(x^{1/2}, f(-x))$, we have
$$ 0 = f(x - x) = f(-x) + f(x^{1/2})^2 = f(-x) + f(x) $$
and 3 follows. Finally, replacing $(x, y)$ in $\text{(*)}$ by $(x^{1/2}, f(y))$ gives
$$ f(x + y) = f(x) + f(y), \qquad \forall x \geq 0, \ y \in \Bbb{R}. $$
The restriction that $x \geq 0$ can be removed by combining this with 3: if $x < 0$, then
$$ f(x+y) = -f(-x-y) = -(f(-x) + f(-y)) = f(x) + f(y). $$

Step 3. $f(x) = x$.

If $x > 0$ then by the first identity of the previous step,
$$ f(x) = f((x^{1/2})^2) = f(x^{1/2})^2 > 0. $$
(We can exclude the equality because $f$ is bijective and $f(0) = 0$.) Combining this with the fact that $f$ is an odd function, we have:
$$ f(x) \geq 0 \quad \Longleftrightarrow \quad x \geq 0.$$
Now for any $x \in \Bbb{R}$,
$$ f(x) -x  = f(x) - f(f(x)) = f(x - f(x)). $$
Therefore $f(x) - x \geq 0$ if and only if $x - f(x) \geq 0$, which implies that $f(x) = x$.

Remark. The Cauchy functional equation has pathological solutions which are also involution, and we need to use the extra structure given by the equation $\text{(*)}$ to exclude such possibilities. The first identity of Step 2 was essential in our solution.
Answer: Here is a short approach without Cauchy equation. Starting from $f(0)=0$ by @Leo163, we obtain $f(f(x))=x$ by plugging $y=0$. This shows $f$ is a bijection. Now plug $y=f^{-1}(\zeta)$, we see
$$f(x^2+\zeta)=f^{-1}(\zeta)+(f(x))^2\geq f^{-1}(\zeta)=f(\zeta)$$
Hence $f$ is monotone increasing. Now suppose for some $x_0$ we have $f(x_0)> x_0$, then $x_0=f(f(x_0))\geq f(x_0)> x_0$, a contradiction. Thus we have $f(x)\leq x$. Similarly we have $f(x)\geq x$, so $f(x)=x$.
Comment: My solution is just a verbose version of this solution. Nice! (+1)
Answer: Suppose $a$ is such that $f(a)=0$ (we know there is such $a$, it is enough to consider $y=-f(x)^2$ for some $x$), then substituting $x=a$ and $y=a$, we have $$f(a^2)=a.$$
Then, set $x=0$ and $y=a^2$, to obtain $$0=f(f(a^2))=a^2+f(0)^2.$$
Since we are only dealing with real numbers, it means that $f(0)=0$, and no other number has $0$ as image.
As consequences, we have $f(f(y))=y$ and $f(y^2)=f(y)^2$ for every $y\in\mathbb{R}$.
Answer: Observe

$f(x^2+f(0)) = [f(x)]^2$
$f(f(x)) = x+[f(0)]^2$.

Plugging $0$ into expression 1 yields
\begin{align}
f_2(0):=f(f(0)) = [f(0)]^2
\end{align}
and plugging $f(0)$ into expression 1 yields
\begin{align}
f(f(0)^2+f(0)) =[f_2(0)]^2 = [f(0)]^4.
\end{align}
Moreover, by expression 2, we have
\begin{align}
f(f(0)^2+f(0))= f([f(0)]^2)=f(f_2(0)) = f_3(0).
\end{align}
Hence we have
\begin{align}
f_3(0) = [f(0)]^4.
\end{align}
Next, using both 1 and 2, we have
\begin{align}
f(f(x^2+f(0)))=x^2+f(0)+[f(0)]^2. \ \ (\ast)
\end{align}
Plugging $0$ into $(\ast)$ yields
\begin{align}
f_3(0) = f(0)+[f(0)]^2.
\end{align}
Combining everything yields
\begin{align}
[f(0)]^4= f_3(0) = f(0)+[f(0)]^2 \ \ \Rightarrow \ \ f(0)[f(0)^3-f(0)-1] =0.
\end{align}
Solving for $f(0)$ yields that either $f(0) = 0$ or $f(0)$ is a root of the polynomial $p(x) = x^3-x-1$, which only has one real root.
Moreover, observe
\begin{align}
f_4(0) = [f(0)]^4
\end{align}
and
\begin{align}
f_4(0) = f(f(f_2(0))) = f_2(0) + [f(0)]^2 = 2[f(0)]^2
\end{align}
which means
\begin{align}
[f(0)]^4-2[f(0)]^2 = 0.
\end{align}
Thus, $f(0)$ is also a root of $x^4-2x^2 = x^2(x^2-2)$.
In conclusion, $f(0)$ has to be $0$. Thus, $f(f(x)) = x$ and $f(x^2) = [f(x)]^2$.
Answer: $\newcommand{\bbx}[1]{\,\bbox[8px,border:1px groove navy]{{#1}}\,}
 \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace}
 \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack}
 \newcommand{\dd}{\mathrm{d}}
 \newcommand{\ds}[1]{\displaystyle{#1}}
 \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,}
 \newcommand{\ic}{\mathrm{i}}
 \newcommand{\mc}[1]{\mathcal{#1}}
 \newcommand{\mrm}[1]{\mathrm{#1}}
 \newcommand{\pars}[1]{\left(\,{#1}\,\right)}
 \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}}
 \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,}
 \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}}
 \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$

\begin{equation}
\mrm{f}\pars{x^{2} + \mrm{f}\pars{y}} =y + \bracks{\mrm{f}\pars{x}}^{2}
\label{1}\tag{1}
\end{equation}

Deriving both members of \eqref{1} respect of $\ds{x}$ and $\ds{y}$ yield, respectively:
\begin{equation}
\left\{\begin{array}{rcl}
\ds{\mrm{f}'\pars{x^{2} + \mrm{f}\pars{y}}\pars{2x}} & \ds{=} &
\ds{2\mrm{f}\pars{x}\mrm{f}'\pars{x}}
\\[2mm]
\ds{\mrm{f}'\pars{x^{2} + \mrm{f}\pars{y}}\mrm{f}'\pars{y}} & \ds{=} & \ds{1}
\end{array}\right.
\label{2}\tag{2}
\end{equation}

By comparing both expressions in \eqref{2}:
\begin{equation}
{\mrm{f}\pars{x}\mrm{f}'\pars{x} \over x} = {1 \over \mrm{f}'\pars{y}} = \alpha\,,
\qquad
\pars{~\alpha\ \mbox{is independent of}\ x\ \mbox{and}\ y~}
\label{2.a}\tag{2.a}
\end{equation}

The second one is quite trivial:
\begin{equation}
{1 \over \mrm{f}'\pars{y}} = \alpha \implies
\mrm{f}\pars{y} = {1 \over \alpha}\, y + \beta\,,\qquad
\pars{~\beta\ \mbox{is a constant}~}\label{3}\tag{3}
\end{equation}

\eqref{2.a} and \eqref{3} yield:
\begin{align}
&{\mrm{f}\pars{x}\mrm{f}'\pars{x} \over x} = \alpha
\,\,\,\stackrel{\mrm{see}\ \eqref{3}}{\implies}\,\,\,
{\bracks{x/\alpha + \beta}/\alpha \over x} = \alpha \implies
{1 \over \alpha^{2}}\, x + {\beta \over \alpha} = \alpha x \implies
\left\{\begin{array}{rcl}
\ds{1 \over \alpha^{2}} & \ds{=} & \ds{\alpha}
\\[2mm]
\ds{\beta \over \alpha} & \ds{=} & \ds{0}
\end{array}\right.
\\[5mm] &\ \stackrel{\mrm{see}\ \eqref{3}}{\implies}\,\,\,
\bbx{\ds{\mrm{f}\pars{x} = {x \over \alpha}\,,\qquad \alpha^{3} = 1}}
\label{4}\tag{4}
\end{align}

By inserting \eqref{4} into \eqref{1} we conclude that $\ds{\alpha = 1}$:
$$
\bbx{\ds{\mrm{f}\pars{x} = x}}
$$
Comment: I see no assumption that $f$ is continuous, much less differentiable.
Comment: @SangchulLee Thanks for your remark. I knew that from the very beginning. However, it's a valid proof whenever we assume the conditions $\left(\mathbf{C}\ \mbox{and}\ \mathbf{D}\right)$ you mentioned above. At least, the final result is consistent with the assumed conditions. I am aware that, in general, we can find ( maybe ) some 'weird'  function without $\left(\mathbf{C}\ \mbox{and}\ \mathbf{D}\right)$. Indeed, I was expecting the down-vote because it happens to me before with this kind of questions. Also, I put this answer because I see nobody  follow this road.
Comment: I agree that solution by differentiation adds a new perspective to the problem. It seems to me, however, that regularity issue is often the hardest part for this kind of problems. It would have been much better if you somehow suggested a way of establishing the regularity of $f$. Also, I think you need not worry about downvote since every answer was downvoted for some unknown reason.