Selenium server won't start

Question: I'm getting the following error when trying to start selenium:
C:\Temp\selenium-server-1.0.3>java -jar selenium-server.jar
22:02:07.615 INFO - Java: Sun Microsystems Inc. 16.0-b13
22:02:07.617 INFO - OS: Windows 7 6.1 x86
22:02:07.625 INFO - v2.0 [a2], with Core v2.0 [a2]
22:02:07.811 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
22:02:07.813 INFO - Version Jetty/5.1.x
22:02:07.815 INFO - Started HttpContext[/selenium-server/driver,/selenium-server    /driver]
22:02:07.817 INFO - Started HttpContext[/selenium-server,/selenium-server]
22:02:07.818 INFO - Started HttpContext[/,/]
22:02:07.866 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@2bbd86

22:02:07.867 INFO - Started HttpContext[/wd,/wd]
22:02:07.870 WARN - Failed to start: firstname.lastname@example.com:4444
Exception in thread "main" org.openqa.jetty.util.MultiException[java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind]
        at org.openqa.jetty.http.HttpServer.doStart(HttpServer.java:686)
        at org.openqa.jetty.util.Container.start(Container.java:72)
        at org.openqa.selenium.server.SeleniumServer.start(SeleniumServer.java:396)
        at org.openqa.selenium.server.SeleniumServer.boot(SeleniumServer.java:234)
        at org.openqa.selenium.server.SeleniumServer.main(SeleniumServer.java:198)
java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.PlainSocketImpl.bind(Unknown Source)
        at java.net.ServerSocket.bind(Unknown Source)
        at java.net.ServerSocket.<init>(Unknown Source)
        at org.openqa.jetty.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
        at org.openqa.jetty.util.ThreadedServer.open(ThreadedServer.java:477)
        at org.openqa.jetty.util.ThreadedServer.start(ThreadedServer.java:503)
        at org.openqa.jetty.http.SocketListener.start(SocketListener.java:204)
        at org.openqa.jetty.http.HttpServer.doStart(HttpServer.java:716)
        at org.openqa.jetty.util.Container.start(Container.java:72)
        at org.openqa.selenium.server.SeleniumServer.start(SeleniumServer.java:396)
        at org.openqa.selenium.server.SeleniumServer.boot(SeleniumServer.java:234)
        at org.openqa.selenium.server.SeleniumServer.main(SeleniumServer.java:198)

Java is installed:
C:\Temp\selenium-server-1.0.3>java -version
java version "1.6.0_18"
Java(TM) SE Runtime Environment (build 1.6.0_18-b07)
Java HotSpot(TM) Client VM (build 16.0-b13, mixed mode, sharing)

Answer: Try this (netstat -a) on Command Prompt to see if this port  (4444 - Selenium Default Port) is being used by some other Service OR Listening.
OR
Start selenium using different port.
Example:
java -jar selenium-server.jar -port 5561
Thanks,
Ara
Answer: Run this in any of your browsers:

http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer
Answer: Ofcourse you can can release the port. Follow the below steps to release the port

type 'Netstat -a -n -o' on your command prompt
Note the PID corresponds to the port which you want to release
type the command 'tasklist /svc /FI "PID eq <>"'
Note the task name listed on command prompt (below Image name)
Kill that task from taskmanager
Answer: Since you're getting an unhandled exception error, you should probably bring this to the attention of the Selenium authors. They have a bugtracker should use.
The root cause seems to be the line Unrecognized Windows Sockets error: 0: JVM_Bind. A quick google search suggests this is not a unique problem.

Selenium server won't start


java api to read certificate and CRL from same pem file


Does the expression $\binom {t}{n+1} + \alpha \binom {t}{n}$ simplify?

Question: I've been working on a problem in economics that involves finding the roots of the polynomial
$$ P(i) = \sum_{n=0}^t \bigg[ i^{n+1} \cdot \bigg(\binom {t}{n+1} + \alpha \binom {t}{n} \bigg ) \bigg ]$$
where $t \in \mathbb{N}$, $\ \alpha \in \mathbb{R} < 0$, and $i > 0.\ $ I want to be able to take $t$ and $\alpha$ from the problem and quickly approximate $i$. Since the polynomial has a factor of $i$, $i=0$ is always a solution, but for the application, $i$ must be positive.
I know that there is some relation between adding two choose functions together, but I am worried that the factor of $\alpha$ would break the relationship. Can the expression for the coefficients be simplified at all?
Answer: $$ {t \choose n+1} + \alpha {t \choose n} = \left(\frac{t-n}{n+1}+\alpha\right) {t \choose n} $$


How can I bind a SecureString or char[] parameter in .NET Core without using strings

Question: When storing a string that is secret (eg a password or key) it is bad practice to use a System.String as strings are immutable and you cannot schedule garbage collection to remove them from memory. Microsoft recommends using SecureString to avoid this, where the content is also encrypted at rest. See MSDN: System.Security.SecureString
I cannot see a way of binding a SecureString parameter in a .NET Core Controller, the framework throws a JsonSerializationException for the parameter when doing the model mapping with an inner exception message of   Could not cast or convert from System.String to System.Security.SecureString. So the JSON parser has already made a string from it, and the secret resides unencrypted in memory.
The framework can't bind even a char[], which is a common workaround for dealing with secret strings (though not encrypted, char[] are mutable so can be overwritten to remove them from memory). It fails with a similar exception.
Comment: Isn't is a bit pointless in this situation as the string will already be in memory before any model binding has happened?
Comment: SecureString is not secure; it is simply an inconvenience to an attacker, not a barrier. Frankly you'd need to be very very clear about what attack vector you're trying to protect yourself from, because realisitically: SecureString doesn't actually save you much : in most scenarios where it is useful, you're already completely owned
Comment: `SecureString` can only be realistically used in a small number of scenarios, where applications can fully control their use of unencrypted secrets. This is not one of them. Guarding against an attacker who can read memory is *hard*. You're typically better off ensuring the secret remains encrypted for a long as possible, for example by not transmitting passwords in serialized JSON in the first place (but only hashes, or otherwise encrypted values). Beyond that you have to consider the trade-offs between securing a machine's memory, and securing the machine itself.
Comment: Also SecureString is tightly coupled to windows. At least I remember a GitHub issue around the .NET core release, stating that `SecureString` is implemented for linux (as in behaves as expected) but doesn't use any encryption, so it only use the encryption on windows.Dunno if that changed though. But yes, its very limited where you can use it. One usage scenario are desktop applications where the user can enter a password into a text field and keep it in there, encrypted inmemory until its used. It would return a string to pass it to the API and then clear the strings memory upon usage
Comment: There, that's the GitHub issue on [SecureString](https://github.com/dotnet/corefx/issues/1387) and current [code and implementation for Unix](https://github.com/dotnet/coreclr/blob/v2.1.0/src/mscorlib/shared/System/Security/SecureString.Unix.cs#L12-L19) proofs that encryption isn't implemented, so the usage in .NET Core is quite limited, unless you host it on Windows machine. The main point for SecureString were two: Possibility to read it from configuraiton file, encrypted and only decrypt it before using it and then free/zero out the memory.
Comment: Mostly for reasons that it doesn't appear in dumps, like it may be the case if you store the string password in some global configuration file. The ConfigurationManager isn't commonly used on .NET Core and ASP.NET Core, since it was superseded by appsettings.json. For an attacker which has direct access to your machine its useless, because a private certificate on the machine running the application was used to decrypt it and an attacker who has control of the machine would also have control of that certificate and its password too
Comment: @DavidG is correct. There's no point to doing this as the string is *already* in memory. Honestly, the cases where you could *realistically* use and actually benefit from `SecureString` are so limited as to make it almost pointless. The secret would pretty much have to only exists in your code and *only* as a `SecureString` from start to finish. The largest majority of the time, it would be far better to have your secret in some external store, where it can be encrypted, anyways.
Answer: I was hoping that someone would say "Yes, the requests are parsed in native code and you can avoid the values entering managed heap by... {some solution, maybe involving SecureString}".
From the comments it sounds like requests are handled in managed code, so one can argue that there is little point using a SecureString or char[] as the values are already in the heap.
It may still be worth encrypting the values at rest because the garbage collection will evict the strings at some point, just not in a deterministic manner.


como faço para executar um arquivo.php?

Question: Tenho um a pagina em php e gostaria de executar outro código em php mais não estou conseguindo alguém pode me dizer o que estou fazendo de errado:
if ($_GET["URL"] === 'WEEK' || $_GET["URL"] === 'week'){
    $obj = new COM("WScript.Shell");

    if (is_object($obj)){
        $obj->Run ("week.php");
    }else{
        echo "Não foi possível criar o objeto!";
    }

}

Answer: Você pode executar um comando para o sistema operacional utilizando crases junto com o comando de console:
$retorno = `php -f week.php`;

Veja um exemplo no Ideone.
É importante observar se a configuração safe_mode está desabilitada.
Porém essa abordagem que você está tomando é insegura!!
Como você vai garantir que o código que será executado não foi modificado, não é malicioso? Existem outras formas de resolver o problema. Se chamar um arquivo PHP a partir do console dentro do PHP, existe algo de errado.
Comment: não funcionou essa sujestão sua
Comment: @chapolin funciona sim https://ideone.com/iPznB3 Verifique a configuração do seu PHP, em específico o safe_mode http://php.net/manual/pt_BR/language.operators.execution.php

como faço para executar um arquivo.php?


French horn - Getting higher register

Question: Does anyone know some good techniques or practises I can do to improve my high range on the horn relatively quick (like 2 months quick)? It can be for brass and doesn't have to be specific for horn- though preferred.
I changed my embouchure at around March this year but I can only play the F above the stave somewhat comfortably without pushing. My grade requires me to play up to the C which is 4 notes above that, but right now even playing the A will do.
All recommendations/suggestions highly appreciated!!
Comment: From personal experience: borrow or rent a trumpet, trombone, or other larger bore / larger mouthpiece brass instrument. Just play long tones one it; you don't need to actually learn to play. But the greater demands on your breath, power, and embouchure will assist with your horn playing. Also [this post about trumpet range](https://music.stackexchange.com/a/104283/70803) might help.
Comment: @Aaron I have some odd personal experience that may support this. I studied trombone for my Bachelor of Music degree, and whenever I picked up a french horn I had zero difficulty belting out concert F5s and the like. I would play up to the same range on trombone, but the smaller mouthpiece of the french horn felt like a cheat code ;-)
Comment: Fortunately my sister plays trumpet and I'll have a go at that! And thank you for the post, it's very helpful.
Comment: @NReilingh I thought that a smaller mouthpiece was more difficult to get high notes and rather bigger mouthpieces such as trumpet mouth pieces were easier.
Comment: I'm not sure what metric you're using for small/large in that case. If you compare, say, tuba and trumpet, the tuba mouthpiece is much larger, and if you buzz on the mouthpiece alone, the "default" pitch on the trumpet mouthpiece will be much higher than the tuba.
Comment: Recently, you requested access to the Basic Mathematics chat-room. Did you have a mathematics question? And I would like to know, why did you hide your accounts on Music SE and Korean SE?
Comment: Hi :) Sorry about that! I was just interested in a question that was being asked. And what does it mean for an account to be hidden? I'm new here and I might've accidentally done something I didn't intend to do haha!
Answer: Lip slurs, scales and arpeggios up to the required range (and a little above it).  Also concoct some exercises with less predictable notes.   I made a breakthrough into my trombone's upper range (many years ago) with exercises like this.  Make sure air, not pressure, is getting you there!
WHAT exercises you use are less important than doing LOTS of playing up there!

French horn - Getting higher register


What is the functional role of the subtonic chord?

Question: In this chart

the subtonic major chord can come before the mediant and submedian chords. Does this imply that it has a tonic functionality? If so that seems counter-intuitive; it seems to share more in common with the dominant chord than the tonic. On the other hand, if it's a dominant function, why would you have a i-VII-III... progression? Wouldn't it make more sense for the VII to fall later, towards the end of the phrase.
(Also I'm assuming that the VII actually refers to the bVII chord, but if I'm wrong please correct me on that)
Answer: I basically agree with Michael Curtis's answer.  But I might summarize/classify the uses a bit differently:

The most common use of VII in minor is actually not as functional VII at all, but rather as the dominant of the relative major.  So the i-VII-III progression mentioned in the question is often best viewed as temporary tonicization of the relative major.  For example, if you were in A minor, but then had a G major-C major progression, that effectively feels like you're making a (perhaps very temporary) hint of a V-I toward C major.  As you may know, minor key pieces very commonly have a secondary key diversion to the relative major, and "VII" is basically the easiest route there.  This harmonic progression (i-VII-III, otherwise interpreted as i-V/III-III) is all over modal music and early tonal music, and it can happen in modal pop styles today that oscillate between major and minor.  Bottom line: in that case, it may be labeled as "VII," but it's often functionally acting like V/III.
The next most common use is probably sequences.  The charts in the question are hinting at that, i.e., a i-iv-VII-III-VI-iio-V-i circle-of-fifths sequence.  Chords in sequences aren't really "functional" either -- they are driven more by the repeated harmonic pattern rather than by predominant-dominant-tonic motion.
The third use is as an alternative harmony in descending tetrachord progressions, i.e., progressions where the bass descends from scale degree 1 down to 5 (over four notes, the "tetrachord").  These sorts of bass lines show up everywhere, and one minor key pop harmony version that sometimes gets used runs down i-VII-VI-V.  Once again, this motion isn't particularly "functional," as VII is a passing chord just used to harmonize a scale step in the bass in parallel chords.
So far, none of these uses of VII tend to actually give VII a unique function.  The only true functional uses of VII are perhaps found in some variant cadential progressions.  There are examples of modal cadences that use a VII-i progression to substitute as a sort of V-i cadence, and jazz sometimes uses the "backdoor" cadence VII7-I (in major or minor).  In these cases, VII is arguably a dominant substitute, containing two notes of the V7 chord.
In a slightly different fashion, VII can sometimes appear functionally as subdominant of subdominant.  I've mostly heard this clearly in things like gospel cadences that emphasize IV-I instead of V-I.  In that case, it's not unheard of to create a VII-IV-I progression (or variants), generally in a major key (rather than minor).  I don't know that there's a specific name for this function, other than the fact that it's sort of "IV of IV," so there's a kind of mirroring going the other way around the circle of fifths (i.e., VII-IV-I rather than ii-V-I, where the ii chord is based on the dominant of the dominant).

In sum, VII can act a bit like a dominant or predominant at times, but mostly it doesn't have a strong distinct function in a minor key.  The vast majority of its appearances (at least in classical harmony) are in sequences or sequence-like progressions, or as a pathway (as secondary dominant) to the relative major.
Comment: Your answer in point 5 is very apt.  You stated it so perfectly about a mirroring of the circle of fifths but going in the other direction. That is exactly correct. Fifths can move downwards into the sub-dominant region (as well as upwards into the dominant region). If you can go down one fifth, why not go down two fifths resulting in the sub-dominant of the sub-dominant.
Answer: Let's set the key to C minor.
When the given key uses a minor key signature you don't need to prefix a flat on the Roman numeral, it's assumed from the key signature. So, in C major C: bVII the flat shows the chromatic lowering of the root from the major key signature, but in Cm: VII the minor key signature already sets the flat on Bb so you don't put the flat prefix on the Roman numeral.
Also, let's be sure to not confuse VII a Bb triad with v6 (the minor dominant with Bb in the bass.) v6 is common with a descending bass, like Cm: i v6 iv6 V.
If you are working with common practice harmony, VII has a tendency to sound like the dominant of the relative major. In other words, the Bb major chord to Eb major, rather that sounding like Cm: VII III sounds more like Eb: V I.
On those flow charts, notice how iii and VII are both at the far left away from the primary tonal chords I, V, and IV. They are both out in the modal nether-regions and not very strong for defining the tonality. They are modal or secondary chords of a key. Functionally they are both pre-dominant. From them you would move along the flow charts to the left to get to the dominant function, and then the tonic function.
The VII can also act like a alternate dominant with a modal flavor.
A common form that uses VII is passamezzo antico where it appears both as a modal dominant and as the dominant of III.
A similar kind of harmony happens in Handel's sarabande in the suite HWV 437, but it also involves a harmonic sequence.
Dm: [ i V ] [ III VII ]
Harmonic sequence is another common way modal chords like VII and iii occur. In terms of function that can make them sort of ambiguous, because harmonic sequences are often used to change key.

...why would you have a i-VII-III... progression?

Don't treat those flow charts as a practical guide to harmony. Think of them more like very abstract representations of two basic music theory ideas: the fundamental tonal harmony progression is root movement by descending fifth, and functional harmony flows pre-dominant to dominant to tonic.
If you take the charts literally, then you can't even get something as fundamental as the Handel example Dm: [ i V ] [ III VII ].
Keep in mind the flow charts makes no reference to phrases or barlines. i is often the end rather than the start. Something with a barline hinting at phrases, like Cm: ...V7 i | VII III... gives a bit of context where i VII III could make sense.
Answer: Generally the "functional role" refers to "functional harmony" where chords have a "function" in a chord sequence. (I don't much think the word "function" is the best, but it's what people call the phenomenon.) The main idea of functional harmony is that phrases are marked with some type of cadence, mostly cadences with some type of V-I or V-i finish (as your charts show.) This cadence may be modified using a V7-I or V7-i or a IV-V-I or ii65-V7-I or ii06-V-i in a minor key. For historical reasons, the functions are called tonic, dominant, and predominant (or subdominant which describes the scale position instead of the compositional position of the chords.) The shortest piece would (in this formulation) tonic-predominant-dominant-tonic structure (this is very simplified). Phrases using I-IV-V-I are not uncommon. Then the idea is to expand parts, I-IV-V-I-IV-V-I which is just a repeat of the first progression or I-vi-ii-V-I-ii-V-I which expands the predominant chords into vi-ii in the first phrase.
Chords are functionally classified by their use(important), not their name (unimportant but convenient). (An incomplete list): The tonic chord in a major key is I (sometimes vi can act a bit like a tonic as will be illustrated below; it can have predominant function as above I-vi-ii-V or I-vi-IV-V). The dominant chords are V,V7,vii0,vii07,vii0 in general. The predominant chords are IV,ii,vi, (sometimes), I64 before some form of a V-I. (The minor mediant is rare in this style except in sequences.)  In a minor key, i is the tonic; V,V7,V9,vii0,vii06 are some of the dominants, iv,VII,VI,ii0,ii06,ii,ii6,i64, are a few of the pre-dominants.
Chords may be used in more than one context. In the pop (though going back before Vivaldi) progression I-vi-ii-V, the vi acts like a predominant by extending the ii-like feel. In a phrase like I-ii-V7-I-ii-V7-vi-ii-V7-I, vi acts like a tonic in the V7-vi pattern (a deceptive cadence) but overall may be seen as a predominant in the vi-ii-V7. Both views are correct simultaneously.
The "function" name may be applied over a region rather than just a chord. Music patterns are to some extent recursive and operate over different scales (mathematical scales, characteristic  length, not an ordered set of notes). A common pattern is I-V-I-vi-ii-V7-I. The I-V at the beginning doesn't act as a cadence; it really just extends the I. Some authors (and I agree though I'm not an author) like to call these "passing" chords. This gives a fourth function "passing." One can see this in pieces using a repeated cycle of fifths (or fourths depending on which direction one prefers) in a minor key. One often gets i-iv-VII-III-VI-ii0-v-i-iv-VII-III-IV-vii0-V-i. The second V-i is marks the end of a section and uses a cadence. The root movement of v-i in the middle doesn't sound like it ends the piece because the dominant (chord on the fifth note) chord is minor.
The common ii-V-I pattern in classical and jazz music (other genres) is a short example of functionality. This (and other patterns I presume) can be used to give a short coloristic sound to a piece. Sometimes one sees a pattern like II7-V7-I in a piece; this looks like a ii-V7-I but the II7 is the dominant of the V (V of V is the terminology and V/V the symbol). It keeps the cadential ii-V-I feel but digresses (with a sharpened fourth scale step) a bit just to provide interest.  It's not a true key change (unfortunately called modulation for historic reasons) because it doesn't last long enough.
Comment: Huh, when I learned chord progressions the I-vi-(rest of phrase) movement as a tonic extension. Is this inaccurate or is this another case of ambiguity?
Comment: I'd say ambiguity. On the other hand, sequences of chords (like vi-ii-V-I) and the like sometimes seem to act as a unit. Anything that's a chunk of the cycle of fifth often does that. The whole cycle  I-IV-vii0-iii-vi-ii-V-I  can be used as well as parts but I'm not sure that detailed functional analysis adds much. The beginning I-IV (without using I7) doesn't act like a secondary dominant (or perhaps the whole thing acts like a chain of dominants.) The ending V-I need not act like a cadence and isn't written as such in minor. Usage is more important than labels.


Empty a field when it is disabled

Question: I am using #state values in a hook_form_alter() to disable field_2 if field_1 (a checkbox) is checked. It works and all is good.
Now I want to empty field_2 as well. I cannot see a way to do that with the #state.
How can I make sure that values don't get saved for fields that are conditionally disabled, even if the user enters a value first?
Answer: Interesting question. Since the $form_state variable is passed by reference to all form validation & submit handlers, you can modify it prior to it reaching the submit handler that saves it. What about adding a new validation handler that will empty the $form_state['values']['field_2'] value if the field is disabled in $form? That should ensure that the field is empty by the time anything tries to save it.
Comment: I don't know that the information about whether it was enabled or disabled at time of submit is stored into a variable. How would I examine this in the PHP?
Comment: Whenever an AJAX event is triggered to update the form (such as when elements are enabled/disabled via the State framework), the form is rebuilt. Simply looking at the correct element in `$form` (`$form['field_2']`, perhaps), should give you the info you're after (this would be done in the validation callback).
Comment: I don't think it triggers an AJAX event, I think it's purely JS adding/removing classes on the elements. Is that wrong?
Comment: Come to think of it, I have a feeling you're right - there's no AJAX call happening (I'm going by memory, no example in front of me). So then, could you not look at the value of `field_1`? It must be checked for `field_2` to get disabled, so that means it'll have a value of 1 in `$form_state['values']['field_1']`, which will mean that `field_2` must have been disabled and can be safely erased.
Comment: If, during an edit a field's value is removed, and then the field is in a disabled state at the time of form submission, the empty value of the field is not saved. Instead, Drupal thinks it still has its value from the previous save. Removing the value from field_1 to enable field_2, and then adding a value to field_2, results in both fields submitting with values, which is a validation error on my form.
Answer: After a little digging, it seems there is no way to empty a field (except for a checkbox --- 'check' and 'uncheck' are supported by #states, but no other value changes) that is disabled. What I ended up doing was hiding the fields when other fields had value. That way, when the user removes the value and then enters a value in another field, the first field is saved as empty (instead of with its previous value) and the second field's value is saved normally.

Empty a field when it is disabled


Grad in cylindrical coordinates.

Question: I'm trying to work out the gradient of a function in cylindrical coordinates. With $x=\rho\cos\phi$, $y=\rho\sin\phi$ and $z=z$, the Euclidean metric is $$g=d\rho^2+\rho^2d\phi^2+dz^2$$
And the gradient is defined as
$$g\left(\textrm{grad}\left(f\right),V\right)=df\left(V\right)$$
This seems to contradict what is on wikipedia, which says that
$$\textrm{grad}\left(f\right)=\frac{\partial f}{\partial\rho}\partial_{\rho}+\rho^{-1}\frac{\partial f}{\partial\phi}\partial_{\phi}+\frac{\partial f}{\partial z}\partial_{z}$$
Answer: It does not contradict wikipedia. Wikipedia does not use $\partial_\rho, \partial_\phi$ and $\partial_z$ as you wrote. The issue is that in some places, the basis vectors are normalized, and in other places it is not. Let $(\partial_\rho, \partial_\phi,\partial_z)$ be the coordinate vector fields associated to cylindrical coordinates, and $(e_\rho, e_\phi,e_z)$ be the corresponding unit vectors, i.e., $$e_\rho = \partial_\rho, \quad e_\phi = \frac{1}{\rho}\partial_\phi \quad\mbox{and} \quad e_z = \partial_z.$$In general, the definition $g({\rm grad}\,f,V) = {\rm d}f(V)$ implies the coordinate expression ${\rm grad}\,f = \sum_{i,j=1}^n g^{ij}(\partial_if)\partial_j$, where $g_{ij} = g(\partial_i,\partial_j)$ are the coefficients of $g$ in the coordinate system considered, and $(g^{ij})_{i,j=1}^n$ is the inverse matrix of $(g_{ij})_{i,j=1}^n$. The coordinate vectors associated to cylindrical coordinates are orthogonal, so life is easy: $$(g_{ij})_{i,j=1}^3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \rho^2 & 0 \\ 0 & 0 & 1 \end{pmatrix} \implies (g^{ij})_{i,j=1}^3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \rho^{-2} & 0 \\ 0 & 0 & 1\end{pmatrix},$$and thus $$\begin{align} {\rm grad}\,f &= \frac{\partial f}{\partial \rho} \partial_\rho + \frac{1}{\rho^2} \frac{\partial f}{\partial \phi} \partial_\phi + \frac{\partial f}{\partial z}\partial_z \\ &= \frac{\partial f}{\partial \rho} e_\rho + \frac{1}{\rho} \frac{\partial f}{\partial \phi} e_\phi+\frac{\partial f}{\partial z}e_z. \end{align}$$A similar cause of confusion arises when trying to express ${\rm grad}\,f$ in, say, spherical coordinates. Find formulas in terms of the coordinate vectors $\partial_\rho, \partial_\phi, \partial_\theta$, and in terms of the corresponding unit vector $e_\rho, e_\phi, e_\theta$. Compare the coefficients in the different formulas you see around and understand which ones assume normalization and which ones do not.

Grad in cylindrical coordinates.


Dimensional Modeling - Common Attribute Used In Various Dimensions Compositie Keys

Question: I have a situation here that I have not faced before.
I have multiple instances of the same ERP system, differing by satellite locale.
Each locale is assigned their own ID.
Within each satellite location, the DB schema is the same as the others, same tables, same values.
When combining the tables, lets say Parts, from two or more of these locales, their Natural Operational Keys will be the same, but the additional attribute data may differ.  And as I need to be able to link to a part, based upon which satellite locale it came from, I am thinking I need a composite key here - Part ID, and Satellite ID.
Now this would be fine for this single dimension, however, this Satellite ID is used in the same manner elsewhere in many other dimensions.  It is also the prime slicer for many fact tables.
How should I treat this attribute?  Put it in its own dimension, and snowflake? Or push the value into each dimension (duplication), but then have the fact table hold the sole FK to the Satellite Dimension?
Comment: can you clarify what does the "the additional attribute data may differ" is supposed to mean. And make sure you complement with one example (that captures the essence of the problem and exceptions).
Comment: The Satellite ID will be different, but the Natural key will be the same.  Other attribute data such as description, Unit of Measure, and perhaps Commodity code can be different as each Satellite locale treats the part slightly different then the others.  Hope this clarifies.
Answer: In an ideal world the solution would be to replace during the ETL process the Natural Operational Keys with surrogate keys that will be unique for each PartID/SatelliteID (and for every dimension which is in the same situation, for example I do suspect that for the Time dimension you can skip a surrogate key).
Of course this will require to add this surrogate key not just in the dimension tables but also for the fact tables.
The Satellite ID column will also be present as a separate dimension if you need to report by Satellite.
This will be an ideal solution.
A quick and dirty solution could be as you suggested to add an additional column with the Satellite ID beside the natural keys, you need to add this column for each dimension (again except the Time dimension) and fact table. Then you need to include the Satellite ID column every time you have a join.
In this case in your reporting tool you need to include the Satellite ID as part of a composite ID, formed by the Natural Operational Key and the Satellite ID.
Also you can create a specific Satellite Dimension that you can use to select data for a specific Satellite.


Can I image-backup an exchange server?

Question: I would like to have an disk image backup of Exchange, before doing some significant upgrades and expanding a too-small C-Drive.  Can I use something like Acronis Backup & Recover 10 Server (price on 11/11/2010 $860) and not use Acronis Backup for Exchange which is like 50% more expensive?  Or is there a better alternative to get a disk image backup?
Comment: Keep in mind that much of the Exchange configuration is in AD, back that up too.
Comment: Good point.  My domain servers live on other hardware, but you certainly need AD working so no point in just having an Exchange backup.
Answer: Yes you can do an image backup with exchange,(as Mfinni said, just ensure all services are down). Remember that before you do image based backup ensure that the image you are making is from a clean exchange server (e.g. do exchange database maintenance and verify that the filesystem is clean).  I would still have a normal exchange backup on the disk as well as a precaution.
Comment: +1 Belt and braces is good.
Comment: Thank you.  but can I go with the cheap Acronis product to do the image?
Comment: @knox, yes any image based backup will work with the noted caveats
Answer: If you turn off all Exchange services before you start your backup, then an imaging backup should work fine.
Answer: Don't need to spend money or do anything special.
Download GPartedLive, build you a boot CD from it.
GParted lets you do anything with a partition you want, including copy, resize, create, destroy.
Play with it on a nonproduction workstation so you're comfortable with it.
Perform your full backup of the production server.
Shut down, install new hard drive, boot up to the GpartedLive CD.
Set up your new partitions on new hard drive.
(Reboot might be required at this point, but I doubt it)
Using Gparted, copy your old partitions to the new partitions.
Remove GParted CD.
Shut down.
Remove old hard drive.  Do not discard.  Put in protective wrap.
Boot to new hard drive.
Windows will want to run a CHKDSK on boot up.  Skip this the first time by pressing a key.  You can run a CHKDSK later.
Let Windows Server come up and spend some time reviewing logs, verifying connectivity and all services operational.
You know what to do from there.
P.S. This is supposed to be a list of steps; not sure how to format it as such.
Comment: Very interesting.  I think the extra stuff that Acronis helps avoid is the CHKDSK and other hardware specific stuff that windows wants when it suddenly finds itself on a new hard drive.

Can I image-backup an exchange server?


gcimagebundle tool fails when trying to package my image

Question: So I have been using GCE for exactly a week now and testing out the different tools and options before I start migrating some systems over. Currently, I am attempting to build my own OS custom image that GCE doesn't offer. To test, I am building my custom image with Ubuntu 12.04LTS. I have gone through the documentation entirely https://developers.google.com/compute/docs/building-image
However, when it is time to bundle my image, it fails. I have attempted to bundle the image as follows:

sudo gcimagebundle -r / -o /tmp/ --log_file=/tmp/abc.log

However, that fails with the following errors:
Starting logging in /tmp/abc.log found platform Ubuntu exclude list: (/etc/ssh/.host_key_regenerated, 0:0:0) (/dev, 0:1:0) (/proc, 0:1:0) (/run,  0:1:1) (/selinux, 0:0:0) (/tmp, 0:1:0) (/sys, 0:1:0) (/var/lib/google/per-instance, 0:1:0)  (/var/lock, 0:1:1) (/var/log, 0:1:1) (/var/run, 0:1:1) ignoring mounts /proc /sys /sys/fs/fuse/connections /sys/kernel/debug /sys/kernel/security  /dev /dev/pts /run /run/lock /run/shm overwrite list = Initializing disk file Traceback (most recent call last):   File "/usr/local/bin/gcimagebundle", line 28, in <module>
    main()   File "/usr/local/bin/gcimagebundle", line 24, in main
    imagebundle.main()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/imagebundle.py", line 186,  in main
    (fs_size, digest) = bundle.Bundleup()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/block_disk.py", line 148, in Bundleup
    with utils.LoadDiskImage(disk_file_path) as devices:   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 50, in __enter__
    output = RunCommand(kpartx_cmd)   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 352, in RunCommand
    stdout=subprocess.PIPE, stderr=subprocess.PIPE)   File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)   File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception OSError: [Errno 2] No such file or directory  root@none:/boot#

I then attempted to run the command a little different based on a different document and video and still failed:
root@none:/boot# /usr/share/imagebundle/compute-image-packages-126.96.36.199/gcimagebundle/build/scripts-2.7/gcimagebundle
-r / -o /tmp/ --log_file=/tmp/abc.log Starting logging in /tmp/abc.log found platform Ubuntu exclude list: (/etc/ssh/.host_key_regenerated, 0:0:0) (/dev, 0:1:0) (/proc, 0:1:0) (/run, 0:1:1) (/selinux, 0:0:0) (/tmp, 0:1:0) (/sys, 0:1:0) (/var/lib/google/per-instance, 0:1:0) (/var/lock, 0:1:1) (/var/log, 0:1:1) (/var/run, 0:1:1) ignoring mounts /proc /sys /sys/fs/fuse/connections /sys/kernel/debug /sys/kernel/security /dev /dev/pts /run /run/lock /run/shm overwrite list = Initializing disk file Traceback (most recent call last):   File "/usr/share/imagebundle/compute-image-packages-220.127.116.11/gcimagebundle/build/scripts-2.7/gcimagebundle", line 28, in <module>
    main()   File "/usr/share/imagebundle/compute-image-packages-22.214.171.124/gcimagebundle/build/scripts-2.7/gcimagebundle", line 24, in main
    imagebundle.main()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/imagebundle.py", line 186, in main
    (fs_size, digest) = bundle.Bundleup()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/block_disk.py", line 148, in Bundleup
    with utils.LoadDiskImage(disk_file_path) as devices:   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 50, in __enter__
    output = RunCommand(kpartx_cmd)   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 352, in RunCommand
    stdout=subprocess.PIPE, stderr=subprocess.PIPE)   File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)   File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception OSError: [Errno 2] No such file or directory

What am I missing here? What directory is the Python tool looking for that returns with "No such file or directory"?
I received help from someone in a different post telling me to install kpartx, which got rid of the error above. I was able to get a step further and now have the following error:
root@none:/usr/share# gcimagebundle -r / -o /tmp
--log_file=/tmp/imagebundle.log Starting logging in /tmp/imagebundle.log

found platform Ubuntu exclude list: (/etc/ssh/.host_key_regenerated, 0:0:0) (/dev, 0:1:0) (/proc, 0:1:0) (/run, 0:1:1) (/selinux, 0:0:0) (/tmp, 0:1:0) (/sys, 0:1:0) (/var/lib/google/per-instance, 0:1:0) (/var/lock, 0:1:1) (/var/log, 0:1:1) (/var/run, 0:1:1) ignoring mounts /proc /sys /sys/fs/fuse/connections /sys/kernel/debug /sys/kernel/security /dev /dev/pts /run /run/lock /run/shm overwrite list = Initializing disk file Making filesystem Copying contents

Traceback (most recent call last):   File "/usr/local/bin/gcimagebundle", line 28, in <module>
    main()   File "/usr/local/bin/gcimagebundle", line 24, in main
    imagebundle.main()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/imagebundle.py", line 186, in main
    (fs_size, digest) = bundle.Bundleup()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/block_disk.py", line 172, in Bundleup
    manifest_created = self._manifest.CreateIfNeeded(manifest_file_path)   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/manifest.py", line 46, in CreateIfNeeded
    self._LoadLicenses()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/manifest.py", line 55, in _LoadLicenses
    response = self._http.GetMetadata('instance/', recursive=True)   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 411, in GetMetadata
    versions = self.Get(base_url).splitlines()   File "/usr/local/lib/python2.7/dist-packages/gcimagebundlelib/utils.py", line 395, in Get
    return urllib2.urlopen(url).read()   File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)   File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)   File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)   File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)   File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)   File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err) urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

I have python-http-parser and python-httplib2 installed but I still get the same failure when trying to build.
Thanks in advance :)
Comment: Did you ever solve this issue? I'm having the same problem, although I do have kpartx installed, I'm still stuck with the same "OSError: [Errno 2] No such file or directory" error.
Answer: The correct command is
sudo gcimagebundle -d <boot-device> -o <output-directory>

Try with
sudo gcimagebundle -d /dev/sda -o ./ --log_file=/tmp/abc.log


PHP delete (.extension) files that are modified after specific time from directory and all sub-directories

Question: I need a little bit of help.  I need to write a script that will look through all directories and sub-directories and delete specific extensions that are modified after specific time.  I can get it to delete files from specific path, but I need the script to search inside sub directories. Is there any way to do that?
I have tried this for files but I have no idea on how to make it work with directories and sub directories
<code>$path = '/path/to/file/';
if ($handle = opendir($path)) {
    while (false !== ($file = readdir($handle))) {
        $filelastmodified = filemtime($path . $file);
        if( $filelastmodified > "MY TIME STAMP" )
        {
           unlink($path . $file);
        }
        echo "deleted";
    }
    closedir($handle);
}
</code>
Answer: Create a recursive function, if it finds a folder it calls itself passing the folder name.


spark structured streaming exception while writing

Question: I am getting below error while wrting spark structured streaming dataframe -
please tell me where I am doing wrong while running this code-
here df is reading from s3://abc/testing location and I am writing this dataframe to different s3 location using spark streaming-
 val q = df  .writeStream
        .trigger(Trigger.Once)
        .option("checkpointLocation", "s3://abc/checkpoint")
        .foreachBatch { (batchDF: DataFrame, batchId: Long) =>
           batchDF
            .write
            .mode(SaveMode.Append)
            .parquet("s3://abc/demo")
        }.start()
      q.processAllAvailable()
      q.stop()

while running above code I get below error -
    org.apache.spark.sql.streaming.StreamingQueryException: Job aborted.
    === Streaming Query ===
    Identifier: [id = 82cae180-6190-499a-99ae, runId = 23aa9dca-c6ef-49ff-b860]
    Current Committed Offsets: {}
    Current Available Offsets: {FileStreamSource[s3://abc/testing]: {"logOffset":0}}

    Current State: ACTIVE
    Thread State: RUNNABLE

    Logical Plan:
    FileStreamSource[s3://abc/testing]
        at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:379)
        at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:269)
Caused by: org.apache.spark.SparkException: Job aborted.
    at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:230)
    at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:178)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:116)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:114)
    at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:139)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:200)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$3(SparkPlan.scala:252)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:165)
    at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:248)
    at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:192)
    at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:158)
    at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:157)
    at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:999)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$5(SQLExecution.scala:116)
    at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:249)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$1(SQLExecution.scala:101)
    at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:845)
    at org.apache.spark.sql.execution.SQLExecution$.withCustomExecutionEnv(SQLExecution.scala:77)
    at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:199)
    at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:999)
    at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:437)
    at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:421)
    at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:294)
    at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:884)
    at line7d42fe70c8664871b443fdc5f6bbc35869.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.$anonfun$withCreateExtract$5(command-3858326:61)
    at line7d42fe70c8664871b443fdc5f6bbc35869.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.$anonfun$withCreateExtract$5$adapted(command-3858326:56)
    at org.apache.spark.sql.execution.streaming.sources.ForeachBatchSink.addBatch(ForeachBatchSink.scala:39)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runBatch$16(MicroBatchExecution.scala:593)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$5(SQLExecution.scala:116)
    at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:249)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withCustomExecutionEnv$1(SQLExecution.scala:101)
    at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:845)
    at org.apache.spark.sql.execution.SQLExecution$.withCustomExecutionEnv(SQLExecution.scala:77)
    at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:199)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runBatch$15(MicroBatchExecution.scala:591)
    at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:276)
    at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:274)
    at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:74)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.runBatch(MicroBatchExecution.scala:591)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$2(MicroBatchExecution.scala:231)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken(ProgressReporter.scala:276)
    at org.apache.spark.sql.execution.streaming.ProgressReporter.reportTimeTaken$(ProgressReporter.scala:274)
    at org.apache.spark.sql.execution.streaming.StreamExecution.reportTimeTaken(StreamExecution.scala:74)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.$anonfun$runActivatedStream$1(MicroBatchExecution.scala:199)
    at org.apache.spark.sql.execution.streaming.OneTimeExecutor.execute(TriggerExecutor.scala:39)
    at org.apache.spark.sql.execution.streaming.MicroBatchExecution.runActivatedStream(MicroBatchExecution.scala:193)
    at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:358)
    at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:269)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 31 tasks (4.0 GiB) is bigger than spark.driver.maxResultSize 4.0 GiB.
    at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2519)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2466)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2460)
    at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
    at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2460)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1152)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1152)
    at scala.Option.foreach(Option.scala:407)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1152)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2721)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2668)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2656)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:938)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2339)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2434)
    at org.apache.spark.sql.execution.collect.Collector.runSparkJobs(Collector.scala:273)
    at org.apache.spark.sql.execution.collect.Collector.collect(Collector.scala:308)
    at org.apache.spark.sql.execution.collect.Collector$.collect(Collector.scala:82)
    at org.apache.spark.sql.execution.collect.Collector$.collect(Collector.scala:88)
    at org.apache.spark.sql.execution.ResultCacheManager.getOrComputeResult(ResultCacheManager.scala:508)
    at org.apache.spark.sql.execution.ResultCacheManager.getOrComputeResult(ResultCacheManager.scala:480)
    at org.apache.spark.sql.execution.SparkPlan.executeCollectResult(SparkPlan.scala:401)
    at org.apache.spark.sql.execution.exchange.BroadcastExchangeExec.$anonfun$relationFuture$1(BroadcastExchangeExec.scala:127)
    at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:845)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withThreadLocalCaptured$4(SQLExecution.scala:308)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withThreadLocalCaptured$3(SQLExecution.scala:308)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withThreadLocalCaptured$2(SQLExecution.scala:307)
    at org.apache.spark.sql.execution.SQLExecution$.withOptimisticTransaction(SQLExecution.scala:325)
    at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withThreadLocalCaptured$1(SQLExecution.scala:306)
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
    at org.apache.spark.util.threads.SparkThreadLocalCapturingRunnable.$anonfun$run$1(SparkThreadLocalForwardingThreadPoolExecutor.scala:104)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at org.apache.spark.util.threads.SparkThreadLocalCapturingHelper.runWithCaptured(SparkThreadLocalForwardingThreadPoolExecutor.scala:68)
    at org.apache.spark.util.threads.SparkThreadLocalCapturingHelper.runWithCaptured$(SparkThreadLocalForwardingThreadPoolExecutor.scala:54)
    at org.apache.spark.util.threads.SparkThreadLocalCapturingRunnable.runWithCaptured(SparkThreadLocalForwardingThreadPoolExecutor.scala:101)
    at org.apache.spark.util.threads.SparkThreadLocalCapturingRunnable.run(SparkThreadLocalForwardingThreadPoolExecutor.scala:104)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Comment: Well considering that you're writing to S3, there could be problems with the connection, you can check that. But it seems to me like the main exception message is missing, you sure you posted the whole stack trace?
Comment: @Filip I added entire error message, if you can help me out
Comment: If you found my answer helpful please consider accepting it on the check mark next to the answer.
Answer: Total size of serialized results of 31 tasks (4.0 GiB) is bigger than spark.driver.maxResultSize 4.0 GiB. means when a executor is trying to send its result to driver, it exceeds spark.driver.maxResultSize. You can resolve it by increasing it till you get it to work, but it's not a recommendation if an executor is trying to send too much data.
Other thing that could cause this is that data is skewed, you should check how data is distributed on the worker nodes, possible scenario is that all data ends up on single node which causes huge input/output of data from single worker. In this case you can try to repartition your data to split the load between your workers which will be much better solution that increasing the limit.


Laplace Transformation

Question: I have the following expressions in the frequency domain and I want to transform them back to time domain. Are the following two correct?
\begin{equation}
\mathcal{L}^{-1}_s\left[\frac{a}{b} \left(\frac{1}{s+\frac{1}{c}}\right)\right](t)= \frac{a}{b}e^{-t/c}
\end{equation}
\begin{equation}
\mathcal{L}^{-1}_s\left[\frac{a}{b} \left(\frac{1}{\left(s+\frac{1}{c}\right)\left(s+\frac{1}{d}\right)}\right)\right](t)= \frac{a}{b}\left(e^{-t/c}+e^{-t/d}\right)
\end{equation}
Comment: I would say - WolframAlpha answers:  http://www.wolframalpha.com/input/?i=inverse+LT+%5Cfrac%7Ba%7D%7Bb%7D+%5Cleft(%5Cfrac%7B1%7D%7Bs%2B%5Cfrac%7B1%7D%7Bc%7D%7D%5Cright)
Comment: What's the difference? I thought that since a,b are constants does't really matter. So if I have $\frac{aw}{bz}$ instead of $\frac{a}{b}$ the inverse transform would be different?
Comment: The first job is not difference.
Answer:
Notice:

$$\mathcal{L}_s^{-1}\left[\text{C}\right]_{(t)}=\text{C}\cdot\mathcal{L}_s^{-1}\left[1\right]_{(t)}=\text{C}\delta(t)$$
$$\mathcal{L}_s^{-1}\left[\frac{1}{s+\text{C}}\right]_{(t)}=e^{-\text{C}t}$$

So, we get:

$$\mathcal{L}_s^{-1}\left[\frac{\text{a}}{\text{b}}\left(\frac{1}{s+\frac{1}{\text{C}}}\right)\right]_{(t)}=\frac{\text{a}}{\text{b}}\cdot\mathcal{L}_s^{-1}\left[\frac{1}{s+\frac{1}{\text{C}}}\right]_{(t)}=\frac{\text{a}}{\text{b}}\cdot e^{-\frac{t}{\text{C}}}$$
$$\mathcal{L}_s^{-1}\left[\frac{\text{a}}{\text{b}}\left(\frac{1}{\left(s+\frac{1}{\text{C}}\right)\left(s+\frac{1}{\text{d}}\right)}\right)\right]_{(t)}=\frac{\text{a}}{\text{b}}\cdot\mathcal{L}_s^{-1}\left[\frac{1}{\left(s+\frac{1}{\text{C}}\right)\left(s+\frac{1}{\text{d}}\right)}\right]_{(t)}=$$
$$\frac{\text{a}}{\text{b}}\cdot\mathcal{L}_s^{-1}\left[\frac{1}{\left(\frac{1}{\text{C}}-\frac{1}{\text{d}}\right)\left(s+\frac{1}{\text{d}}\right)}+\frac{1}{\left(\frac{1}{\text{d}}-\frac{1}{\text{C}}\right)\left(s+\frac{1}{\text{C}}\right)}\right]_{(t)}=$$
$$\frac{\text{a}}{\text{b}}\cdot\left(\mathcal{L}_s^{-1}\left[\frac{1}{\left(\frac{1}{\text{C}}-\frac{1}{\text{d}}\right)\left(s+\frac{1}{\text{d}}\right)}\right]_{(t)}+\mathcal{L}_s^{-1}\left[\frac{1}{\left(\frac{1}{\text{d}}-\frac{1}{\text{C}}\right)\left(s+\frac{1}{\text{C}}\right)}\right]_{(t)}\right)=$$
$$\frac{\text{a}}{\text{b}}\cdot\left(\frac{1}{\frac{1}{\text{C}}-\frac{1}{\text{d}}}\cdot\mathcal{L}_s^{-1}\left[\frac{1}{s+\frac{1}{\text{d}}}\right]_{(t)}+\frac{1}{\frac{1}{\text{d}}-\frac{1}{\text{C}}}\cdot\mathcal{L}_s^{-1}\left[\frac{1}{s+\frac{1}{\text{C}}}\right]_{(t)}\right)=$$
$$\frac{\text{a}}{\text{b}}\cdot\left(\frac{1}{\frac{1}{\text{C}}-\frac{1}{\text{d}}}\cdot e^{-\frac{t}{\text{d}}}+\frac{1}{\frac{1}{\text{d}}-\frac{1}{\text{C}}}\cdot e^{-\frac{t}{\text{C}}}\right)$$

Laplace Transformation


vs buildtools container 'dotnet test' unable to find tests for Dotnet Core test project

Question: I have a dotnet core test project using mstest and targeting framework 461 with runtime identifier of win7-x64 but it cannot locate the tests when the 'dotnet test' command is run.
Returning error: No test is available in . Make sure that test discoverer & executors are registered and platform & framework version settings are app
I am using the vsbuildtools2017 container from https://learn.microsoft.com/en-us/visualstudio/install/build-tools-container to perform the action.
csproj propertyGroup settings:
<PropertyGroup> <TargetFramework>net461</TargetFramework> <RuntimeIdentifier>win7-x64</RuntimeIdentifier> </PropertyGroup>

csproj packages:
<ItemGroup> <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" /> <PackageReference Include="MSTest.TestAdapter" Version="1.2.1" /> <PackageReference Include="MSTest.TestFramework" Version="1.2.1" />

I have also tried using the vstest.console.exe and I get the same error.
Answer: Executing dotnet test on the root folder will work IF you have the code project and the test project following the naming convention: <name>.csproj and <name>.Tests.csproj.
Another approach is to provide the path on the command like so:
dotnet test path/to/foo.tests.csproj

This will also work with multiple projects, just separate the arguments with a space.
Here is an Example using xUnit:

https://github.com/dotnet/samples/tree/master/core/getting-started/unit-testing-using-dotnet-test


Prove that $K(\alpha)=K(\alpha^6)$ when $[K(\alpha):K]=2011$

Question: Let $L/K$ be a finite extension and let $\alpha \in L$ so that $[K(\alpha):K]=2011$. Prove that $K(\alpha)=K(\alpha^6)$.
My idea is as follows:
$K \subset K(\alpha^6) \subset K(\alpha)$, therefore $[K(\alpha):K] = [K(\alpha):K(\alpha^6)][K(\alpha^6):K]$. We have two options:
First, $[K(\alpha):K(\alpha^6)] =2011$ and $[K(\alpha^6):K] =1$. In that case ,$K(\alpha^6)=K $ and $\alpha^6 \in K$. The polynomial $f(x)=x^6-\alpha^6 \in K[X]$ has the the root $\alpha$ which is not possible since $deg(irr(\alpha,K)=2011$. (*)
We are with $[K(\alpha):K(\alpha^6)] =1$ so $K(\alpha)=K(\alpha^6)$.
I'm not really sure about the part (*), is it correct?
Thanks.
Answer: Yes, $\alpha^6 \in K$ implies $[K(\alpha):K] \le6$, which contradicts $[K(\alpha):K]=2011$.


NTP: Should I configure NTP step-tickers in a domain which uses local NTP servers?

Question: I've bumped into something which is called "NTP step tickers" and read about it in this article, but I'm not sure if it's required in a domain which includes local NTP servers.
It seems like it replaces /etc/ntp.conf configuration file with /etc/ntp/step-tickers configuration file or is it supposed to be there in addition to the /etc/ntp.conf file?
Can anyone please shed some light on this matter?
Answer: NTPd famously, by default, won't adjust the time if it's more than 1000s out, presuming as it does that there's a systemic problem with which it shouldn't get involved.  This from man ntpd:

   -g      Normally,  ntpd  exits  with  a  message  to the system log if the offset exceeds the panic
           threshold, which is 1000 s by default. This option allows the time to be set to  any  value
           without restriction; however, this can happen only once. If the threshold is exceeded after
           that, ntpd will exit with a message to the system log. This option can be used with the  -q
           and -x options. See the tinker command for other options.

One problem is that BIOS clocks in the UK are often an hour out, because of misunderstandings about whether the BIOS is in UTC or summer time.  I imagine the problem gets worse as your longitude from Greenwich increases.
So there is also a command ntpdate, which wrenches the system clock into alignment with an NTP server, in an unsubtle manner that tends to upset a lot of running services.  I believe it predates the addition of the -g flag to ntpd.
Thus, for old-timers like myself, the full startup protocol for ntpd involves the use of ntpdate to forcibly sync the clock - on the basis that that's safest to do when the system starts up - followed by starting ntpd to keep it there.  /etc/ntp/step-tickers controls which server is used to do the ntpdate.  I usually list one of my ntp.conf servers in there.
To further confuse the matter, the ntpdate startup script on CentOS6 will look in ntp.conf for a startup ticker if one isn't listed in step-tickers.  So whilst it is certainly good practice to make sure that ntpd won't be left high and dry (trying to sync a system clock that's too far out), between modern startup scripts and the -g flag to ntpd, step-tickers really is a historical relic, with no necessary function in a modern distro.  But I'm a historical relic, too, and so I continue to use it.


Is classification only a machine learning problem?

Question: Wikipedia says that:

In machine learning, multiclass or multinomial classification is the problem of classifying instances into one of three or more classes.

It looks like the problem is strictly related to machine learning. But to me, this is just a problem to solve and machine learning is only one of the possible approaches to solve it. And I can compare different approaches using same metrics because the problem remains the same.
What are other approaches to multiclass classification than machine learning?
I implicitly assumed there are because I approached one of such problems before I even knew machine learning exists. I had to classify products into shop departments they can be located in. I labeled manually a few products and inferred labels for the remaining ones based on products taxonomy, e.g.: Jack Daniel's and Johnnie Walker's are both whiskies, so if I know the first one can be located in the Alcohols department, then I can guess the latter as well. A very simple approach that just traverses the taxonomy graph without using machine learning at all.
Answer: You are exactly right. Machine/statistical learning is one approach to classification, but not the only one. Simple rules created by humans are probably more common in computer programs than ones created by ML.
Comment: Multinomial logistic regression, also called polytomous logistic regression.   Note that _classification_ occurs only when you take the probabilities estimated by the model and force yourself to make a guess at the observation being in a certain class or not.   Classification is a [forced choice](https://fharrell.com/post/classification) and is usually unnecessary.
Answer: Actually, classification methodology has been around in classical probability and statistics for the better part of a century, well before "machine learning" was a deal.  See, e.g., the classic multivariate analysis text by Johnson and Wichern, Applied Multivariate Statistical Analysis, 6th Edition,  Section IV. CLASSIFICATION AND GROUPING TECHNIQUES (subsection "Classification with Several Populations"), for the optimal (nonparametric, even!) approach to this problem.  Machine learning algorithms are simply attempts to approximate this historically well-known (at least by statisticians) optimal solution.
Comment: I found "Classification with Several Populations" in 11.5 chapter of the book (6th edition), p. 606


What is the utility in writing pdfs in terms of their kernel?

Question: Consider the normal distribution. We know that $$p(x| \mu, \sigma^2) = \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(x-\mu)^{2}}{2 \sigma^{2}}} $$
The kernel is $$ p(x| \mu, \sigma^{2}) \propto e^{-\frac{(x-\mu)^{2}}{2 \sigma^{2}}} $$ omitting the part that isn't a function of $x$. Why write $p(x|\mu, \sigma^{2})$ like this?
Comment: The scaling factor can always be determined since the the pdf integrates out to $1$. So we don't need to keep track of the scaling constants in front of the pdf.
Answer: Presumably because one is only interested in comparing the values of $p(x)$ for different arguments $x$. For example, maximum likelihood estimation techniques require to maximize $p(x)$ over $x$, that is, to find $x^*$ such that $p(x^*)\geqslant p(x)$ for every $x$. If $p(x)=cr(x)$ with $c$ independent on $x$, it is equivalent to ask that $r(x^*)\geqslant r(x)$ for every $x$.


Is this graph a bipartite graph?

Question: Graph
This question has come up in a CS exam that I have to take, however my solution is different from the one given and I can't see why I'm incorrect. I'd love to know what someone else thinks!
Comment: what is your solution?
Comment: @DanWhite please revise your definition of bipartite graph. And revise your comment (V2 appears twice, V5 not at all).
Comment: @cineel my solution is that the graph is bipartite. With two sets, A: V1, V2, V3 and B: V2, V4, V6
Answer: Use a characterization of bipartite graphs: A graph is bipartite iff it does not contain an odd cycle.
In your graph, there is an odd cycle: $(v_2,v_4,v_5)$. So it's not bipartite.
Comment: Thank you! I see exactly where I made an error. And this is so obvious to me now- oops

Is this graph a bipartite graph?


Is there IO functionality to store trained models in kedro?

Question: In the IO section of the kedro API docs I could not find functionality w.r.t. storing trained models (e.g. .pkl, .joblib, ONNX, PMML)? Have I missed something?
Comment: They should have called it artifacts instead of datasets =) It's a bit counter-intuitive that models are saved in `data` with the `datasets` plugin. And I think if it were artifacts it would have been very obvious to do classes for saving Keras / PyTorch / :hugging-face: / Tensorflow models.
Answer: There is the pickle dataset in kedro.io, that you can use to save trained models and/or anything you want to pickle and is serialisable (models being a common object). It accepts a backend that defaults to pickle but can be set to joblib if you want to use joblib instead.
I'm just going to quickly note that Kedro is moving to kedro.extras.datasets for its datasets and moving away from having non-core datasets in kedro.io. You might want to look at kedro.extras.datasets and in Kedro 0.16 onwards pickle.PickleDataSet with joblib support.
The Kedro spaceflights tutorial in the documentation actually saves the trained linear regression model using the pickle dataset if you want to see an example of it. The relevant section is here.
Comment: Ah, totally missed `kedro.io.PickleLocalDataSet` and `kedro.io.PickleS3DataSet`. Thanks!
Answer: There is PickleDataSet in https://kedro.readthedocs.io/en/latest/kedro.extras.datasets.pickle.PickleDataSet.html and joblib support in PickleDataSet is in the next release (see https://github.com/quantumblacklabs/kedro/blob/develop/RELEASE.md)
Comment: Great to hear that `.joblib` support will be added soon! :thumbsup:
Answer: This is what it currently says in my code for the node that trains the model:
    path = "data/06_models/model.keras"
    model.save(path)
    # TODO make a class for the model in the catalog
    return path

and the catalog.yml
model:
  type: text.TextDataset
  filepath: data/06_models/dummy_until_I_write_my_own_class_for_read_and_write_model.txt

then you can read the path where you saved the model and load it again.
Okay, this is not exactly what I have - my code has more expletives


Merge multiple workbooks in one (new) master workbook using C#

Question: I need some help on integrating multiple workbooks into a single master workbook. I am trying to use the following code. I would really appreciate if someone could please help me with following problems:

saving the final (master) workbook having all the worksheets
the worksheets in different workbooks should not loose formatting while copying to master workbook.

Code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Excel = Microsoft.Office.Interop.Excel;
using Microsoft.Office.Interop.Excel;
using System.IO;

namespace IntegrationMacro
{
    class Program
    {
        static void Main(string[] args)
        {
            Excel.Application app = new Excel.Application();
            app.Visible = true;
            app.Workbooks.Add("");
            app.Workbooks.Add(@"C:\Users\ssehgal\Documents\Excel-Files-For-Macro\WorkBook1.xls");
            app.Workbooks.Add(@"C:\Users\ssehgal\Documents\Excel-Files-For-Macro\WorkBook2.xls");
            for (int i = 2; i <= app.Workbooks.Count; i++)
            {
                for (int j = 1; j <= app.Workbooks[i].Worksheets.Count; j++)
                {
                    Excel.Worksheet ws = (Excel.Worksheet)app.Workbooks[i].Worksheets[j];
                    ws.Copy(app.Workbooks[1].Worksheets[1]);
                }
            }

        }

    }
}

Answer: var app = new Application() { Visible = true };
var master = app.Workbooks.Add();

// repeat this for each workbook you need to combine...
app.Workbooks.Add(__PATH_TO_THE_WORKBOOK__)
    .Worksheets.Cast<Worksheet>()
    .ToList() // Move() will modify the source of enumeration, so we need to make a temp copy
    .ForEach(x => x.Move(master.Sheets[1]));

Comment: Thank you very much.. The code worked.. However, I am not able to save the formatting of the header like color. Is there any way to save the header color too?
Comment: I was unable to reproduce that. Both column header and page header copied fine for me on Excel2010. Can you provide a sample file to reproduce the problem?


Props in state anti pattern

Question: So i was reading the react docs which suggest using props in state is considered to be an anti pattern. I wanted to know what the write way is. I have a parent state container passing data to a child component which also has a state.
constructor(props) {
  this.state = {
    name = this.props.name
  }
}

So i was wondering is it ok if i update the state in the CDM method
constructor(props) {
  this.state = {
    name = ''
  }
}

componentDidMount() {
  this.setState({name : this.props.name})
}

Any help or discussion here would be grateful.
Comment: Call props directly in render.
Comment: But incase i have a controlled input like a text field?
Comment: Raise event to parent if value changed.
Then parent has responsibility to update (or not update!) the props. The parent has full control over this text field. (hence "controlled" comes from)
Answer: Using props in state is a big no no. Here is an example how to do deal with situations like these
class Parent extends Component {
    state = {
        name: 'John Doe'
    }

    updateName = (name) => {
        this.setState({ name });
    };

    render() {
        return (

                Your Name
                <Input value={this.state.name} updateName={this.updateName} />

        );
    }
}

class Input extends Component {
    handleUpdate = (e) => {
        this.props.updateName(e.target.value);
    };

    render() {
        return <input value={this.props.value} onChange={this.handleUpdate} />
    }
}

Props in state anti pattern


VBA - Range: "Run-time error '91'

Question: If the workbook isn't saved and closed and reopened,
I get the following error

"Run-time error '91': Object variable or With block not set"

I have the exact same code (only the name of the string is different) on other places, and sometimes it gives me the same error until I save close and reopen and re-run. Afterwards, the code runs smoothly.
Any ideas on how to avoid the bug? Have you had this problem before?
Dim fal As Excel.Worksheet
Set fal = wb.Sheets("Falancs")

Dim x As String
x = "F_1 ="

Dim cc As Integer ' The column as an integer (cc = 1,2,3...)
cc = fal.UsedRange.Find(x).Column

Error is on the last line, where the "(x)" is ...
Comment: What does `wb` refer to?
Comment: you should add check for case when value of `x` not found: 1) `Dim res as Range` 2) `set res = fal.UsedRange.Find(x)` 3) `If Not res is Nothing then cc=res.Column`
Comment: it refers to "ThisWorkbook"
Answer: .Find returns Nothing if value of x not found, so you should check it:
Dim fal As Excel.Worksheet
Set fal = wb.Sheets("Falancs")

Dim x As String
x = "F_1 ="

Dim cc As Integer ' The column as an integer (cc = 1,2,3...)
Dim res As Range

Set res = fal.UsedRange.Find(x)
If Not res Is Nothing Then
    cc = res.Column
Else
    MsgBox "Value " & x & " not found"
    Exit Sub
End If

VBA - Range: "Run-time error '91'


how to enter text in Name field if its value Keep changing after each refresh?

Question: I am relatively new to Java and Selenium.
URL : https://register.rediff.com/register/register.php?FormName=user_details
I want to send keys into the name field. when I am trying to find the element by name tag I am unable do so because name attributes value changed each time.
<code>`<input type="text" onblur="fieldTrack(this);" name="name86b72791" value="" style="width:185px;" maxlength="61">`
</code>
Answer: Well You can select this using relative xpath
<code>//input[starts-with(@name,'name')]</code>
Here starts-with only checks whether the name attribute starts-with name and ignore other randomly generated kinds of stuff.


Dictionary item not returning correct value

Question: I'm creating a function that returns a proper field name based on a number that is input. The test script is just returning the first key, not the desired value (1).
def GetFieldName(bldgcode):
    bldgcode = {1: "SFCount", 2: "MFCount", 3: "ComCount", 4: "IndCount", 5: "CityCount", 6: "ShedCount", 7: "SchCount", 8: "ChurCount"}
    for values in bldgcode:
        return values
#test for get field name
print(GetFieldName(8))

Running the script is giving me 1 when it should be ChurCount
Answer: How about directly use dict[key] to get name?
def GetFieldName(bldgcode):
    code2name = {1: "SFCount", 2: "MFCount", 3: "ComCount", 4: "IndCount", 5: "CityCount", 6: "ShedCount", 7: "SchCount", 8: "ChurCount"}
    return code2name[bldgcode]
#test for get field name
print(GetFieldName(8))


Unable to install WordPress plugins on Nginx

Question: I am trying to set up a WordPress website on a VPS using Nginx with the following specifications:

CentOS 7
  Nginx v1.12.2
  PHP 7.2.2

The permissions and SELinux context of the html (Nginx root) folders and files:

html root folder: 755 / httpd_sys_content_t
  folders in html: 755 / httpd_sys_content_t
  files in html: 644 / httpd_sys_content_t
  wp-content folder: 755 / httpd_sys_rw_content_t
  wp-content files: 644 /  httpd_sys_rw_content_t

Ownership:

everything: root:nginx

However, I cannot install any plugins from the dashboard. Specifically, I am trying to install the Thim Core plugin, and it is telling me that the plugin folder does not have the correct permissions. The next time I try to enter my wp-admin page, i get Unable to connect and the URL looks like this:

http://MY-IP/wp-admin/?thim-auto-compile-sass-to-css

I have used a day trying to get somewhere without success, so I turn to you. I really hope someone can point me in the right direction here!
Update:
I am not sure this is relevant, but my wp-config.php is in another directory with the same permission and context as the 'html root folder', and the file itself with the same permission and context as the 'files in html'. Also, I am running SELinux in permissive mode without luck.
Comment: So, what are the ownership of the files and folders?
Comment: The user is root and the group is nginx for absolutely everything.
Comment: Then that's why it doesn't work.
Comment: I'd like to curse now, but that is probably not a good idea here. Changed to nginx:nginx and everything works. God, I swear I tried that before. Apparently, I did not. A thousand thanks to you, Michael.
Comment: Please enter as detailed an answer as you can in the answer box below, then 24 hours later mark it as answered. That stops others coming into the question trying to help.
Answer: Thanks to the help I received from Michael Hampton, I tried changing the ownership for all folders and files to:

nginx:nginx

It now works perfectly.
The folder that contains the real wp-config.php file needs the same permission.


What is the *finite-population correction* for the Wilson score interval (for a binary proportion)?

Question: The Wilson score interval is an approximate confidence interval for an unknown binary proportion for IID binary data.  The interval is based on the central limit theorem like the Wald interval, but it uses a transformed pivotal quantity that ensures that the interval is concentrated on the true support of the proportion parameter.  Given data $X_1,X_2,...,X_n \sim \text{IID Bern} (\theta)$ the interval at $1-\alpha$ confidence level is given by:
$$\text{CI}_\infty(1-\alpha) = \Bigg[ \frac{n \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n + \chi_{1,\alpha}^2} \pm \frac{\chi_{1,\alpha}}{n + \chi_{1,\alpha}^2} \cdot \sqrt{n \hat{\theta}_n (1-\hat{\theta}_n) + \tfrac{1}{4} \chi_{1,\alpha}^2} \Bigg].$$
where $\hat{\theta}_n$ is the observed sample proportion and $\chi_{1,\alpha}^2$ is the critical point of the chi-squared distribution with one degree-of-freedom (using upper tail area $\alpha$). The Wilson score interval is known to have good coverage properties and to perform well against other confidence intervals for a binary proportion (see Brown, Cai and DasGupta (2001) for further discussion).  In particular, it is superior to the standard Wald interval.
The above interval form is for the parameter $\theta$ which (from the law of large numbers) is the long-run proportion of positive outcomes in an infinite superpopulation (see this related answer for technical details).  However, in many cases of interest we would instead be interested in forming a confidence interval for the proportion in a finite population of size $n \leqslant N < \infty$.

Question: What is the appropriate adjustment for this confidence interval when the sample comes from a finite population $N$ and the parameter of interest in the inference is the (finite) population proportion $\theta_N \equiv \sum_{i=1}^n X_i /N$?  (Note that this latter parameter must have a value in the range $0,\tfrac{1}{N},...,\tfrac{N-1}{N},1$.)
Answer: This answer is based on information in the paper O'Neill (2021),  which sets out mathematical properties of the Wilson score interval, including the finite population correction for inference to a finite population or the unsampled part of a finite population.  Please see the linked paper for more details on the matter discussed here.
CI for the proportion in a finite population: You can find a derivation of the standard Wilson score interval in this related answer.  The present analysis, adjusting for a finite population, is relatively simple to do using sample moment results found in O'Neill (2014).
To facilitate this analysis, define the effective sample size for the inference as:
$$n_* \equiv n \cdot \frac{N-1}{N-n}.$$
As in the linked paper, we will derive the confidence interval via standard manipulations but starting from a pivotal quantity for the finite-population inference.  Our pivotal quantity is:
$$n_* \cdot \frac{(\hat{\theta}_n-\theta_N)^2}{\theta_N (1-\theta_N)} \overset{\text{Approx}}{\sim} \text{ChiSq}(1).$$
As in the question, let $\chi_{1,\alpha}^2$ denote the critical point of the chi-squared distribution with one degree-of-freedom (with upper tail area $\alpha$).  For any confidence level $1-\alpha$ we then have the probability interval:
$$\begin{align}
1-\alpha
&\approx \mathbb{P} \Big( n_* (\hat{\theta}_n-\theta_N)^2 \leqslant \chi_{1,\alpha}^2 \theta_N (1-\theta_N) \Big) \\[6pt]
&= \mathbb{P} \Big( n_* (\hat{\theta}_n^2 - 2 \hat{\theta}_n \theta_N + \theta_N^2) \leqslant \chi_{1,\alpha}^2 (\theta_N-\theta_N^2) \Big) \\[6pt]
&= \mathbb{P} \Big( (n_* + \chi_{1,\alpha}^2) \theta_N^2 - (2 n_* \hat{\theta}_n + \chi_{1,\alpha}^2) \theta_N + n_* \hat{\theta}_n^2  \leqslant 0 \Big) \\[6pt]
&= \mathbb{P} \Bigg( \theta_N^2 - 2 \cdot\frac{n_* \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n_* + \chi_{1,\alpha}^2} \cdot \theta_N + \frac{n_* \hat{\theta}_n^2}{n_* + \chi_{1,\alpha}^2} \leqslant 0 \Bigg) \\[6pt]
&= \mathbb{P} \Bigg( \bigg( \theta_N - \frac{n_* \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n_* + \chi_{1,\alpha}^2} \bigg)^2 \leqslant \frac{\chi_{1,\alpha}^2 (n_* \hat{\theta}_n (1-\hat{\theta}_n) + \tfrac{1}{4} \chi_{1,\alpha}^2)}{(n_* + \chi_{1,\alpha}^2)^2} \Bigg) \\[6pt]
&= \mathbb{P} \Bigg( \theta_N \in \Bigg[ \frac{n_* \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n_* + \chi_{1,\alpha}^2} \pm \frac{\chi_{1,\alpha}}{n_* + \chi_{1,\alpha}^2} \cdot \sqrt{n_* \hat{\theta}_n (1-\hat{\theta}_n) + \tfrac{1}{4} \chi_{1,\alpha}^2} \Bigg] \Bigg), \\[6pt]
\end{align}$$
and substitution of the observed sample proportion (for simplicity I will use the same notation for this value) then leads to the Wilson score interval:
$$\text{CI}_N(1-\alpha) = \Bigg[ \frac{n_* \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n_* + \chi_{1,\alpha}^2} \pm \frac{\chi_{1,\alpha}}{n_* + \chi_{1,\alpha}^2} \cdot \sqrt{n_* \hat{\theta}_n (1-\hat{\theta}_n) + \tfrac{1}{4} \chi_{1,\alpha}^2} \Bigg].$$
As can be seen, the finite-population correction in the Wilson score interval consists of replacing the sample size $n$ with the effective sample size $n_*$.  This is an extremely simple adjustment, and it gives a confidence interval that is valid for any population size $N \geqslant n$ (finite or infinite).  It is trivial to confirm that this reduces down to the standard Wilson score interval when $N = \infty$ (giving $n_* = n$).  It is also useful to note that when you take a full census of the population with $N=n$ you get $n_* \rightarrow \infty$ and the confidence interval reduces to the single point $\text{CI}_n(1-\alpha) = [\hat{\theta}_n]$, just as we would expect.

CI for the unsampled proportion in a finite population: A useful variant of this problem occurs when you are interested in making an inference about the proportion of positive outcomes in the unsampled part of a finite population, which is $\theta_{n:N} \equiv \sum_{i=n+1}^N X_i/(N-n)$.  In this case, we can use the pivotal quantity:
$$n_{**} \cdot \frac{(\hat{\theta}_n-\theta_N)^2}{\theta_N (1-\theta_N)} \overset{\text{Approx}}{\sim} \text{ChiSq}(1)
\quad \quad \quad n_{**} \equiv n \cdot \frac{N-n}{N-1},$$
which uses the alternative value $n_{**}$ for the effective sample size.  The remaining calculations are the same as above, except that $n_{**}$ replaces $n_{*}$ in the formulae, giving the confidence interval:
$$\text{CI}_{n:N}(1-\alpha) = \Bigg[ \frac{n_{**} \hat{\theta}_n + \tfrac{1}{2} \chi_{1,\alpha}^2}{n_{**} + \chi_{1,\alpha}^2} \pm \frac{\chi_{1,\alpha}}{n_{**} + \chi_{1,\alpha}^2} \cdot \sqrt{n_{**} \hat{\theta}_n (1-\hat{\theta}_n) + \tfrac{1}{4} \chi_{1,\alpha}^2} \Bigg].$$
It is again trivial to confirm that this reduces down to the standard Wilson score interval when $N = \infty$ (giving $n_{**} = n$).  It is also useful to note that when you take a full census of the population with $N=n$ you get $n_{**} = 0$ and the confidence interval reduces to the vacuous interval $\text{CI}_{n:n}(1-\alpha) = [0,1]$, just as we would expect.

Implementation in R: These generalised confidence intervals are implemented in the CONF.prop function in the stat.extend package.  The function implements the Wilson score intervals in the form shown above.  By default the function uses the standard interval for the proportion parameter for an infinite population.  However, the function also allows specification of a population size N and a logical value unsampled to give the above interval forms.


How to implement repository design pattern for MongoDB on NestJS

Question: I'm trying to figuring out how to implement repository design pattern on NestJS that using MongoDB and mongoose database
Comment: I'd like to have a solution like this with a npm package: https://github.com/Thavarajan/Mangoose-Typescript-With-Repository-Pattern
Comment: I'm not sure I follow your question. Do you mean [similar to the model injection shown in the docs?](https://docs.nestjs.com/techniques/mongodb#model-injection)
Comment: same thing like repository pattern for typeorm [https://docs.nestjs.com/techniques/database#repository-pattern](typeorm repository pattern) @JayMcDoniel
Comment: That would be basically the same as creating a schema object and using the injection method shown in the [Mongo section of the docs](https://docs.nestjs.com/techniques/mongodb#model-injection)
Comment: I'm searching the same, a repository pattern, like spring repository in Java or typeorm, but in the docs only using mongoose directly in the services layer, I dont know if mongoose implement a pattern like active record. Any help?
Answer: Repository could be injected into the service and it should be included in providers array within the module.
// user.service.ts
@Injectable()
export class UserService {
  constructor(private readonly userRepository: UserRepository) {}

  async findAll(): Promise<User[]> {
    return this.userRepository.findAll();
  }
}

// user.repository.ts
@Injectable()
export class UserRepository {
  constructor(@InjectModel(User.name) private userModel: Model<UserDocument>) {}

  async findAll(): Promise<User[]> {
    return this.userModel.find().exec();
  }
}

// user.module.ts
@Module({
  imports: [
    MongooseModule.forFeature([{ name: User.name, schema: UserSchema }]),
  ],
  providers: [UserRepository, UserService],
  exports: [UserService],
})
export class UserModule {}

Comment: This is a really simple example, some methods could include some complicated logic. It is better to have a separate data access layer. If changing the database is necessary, only a new repository with an existing interface should be implemented and injected into the service, the service methods will stay the same.
Comment: What are the advantage of using a repository in service,it look like we are doing the same thing twice and we can do the same thing just by injecting the model in the service
Comment: How can I write unit test for this service?
Comment: @huykon225 this one doesn't make sense to be unit-tested, but usually, repositories and external dependencies are mocked in the services, and some use cases are covered with the tests
Comment: @mirsahib I think it useful for unit test in some case. Example in your's service do query with find().populate()....exec(). In this case, when you test with jest, you must mock find().populate()...exex() that is too complex. But with repository you only mock one function that wrap all query in model mongoose.
Answer: I myself asked the exact same question a few months back, and after some research and effort I created monguito, a lightweight and type-safe MongoDB Abstract Repository implementation for Node.js.
Furthermore, monguito is an open-source Mongoose ODM wrapper and has several cool built-in features, including easy integration in NestJS applications, polymorphic data model handling support, and optional audit data support. For further information, please visit monguito's GitHub site.


sockjs-node/info? 404 (Not Found)

Question: I've done lost of search for couple of hours now and still can't find what I'm looking for. Basically I get this error in the console when running ng serve in my Angular 6 app:
GET http://local.mywebsitesecure.com/sockjs-node/info?t=1540568455931 404 (Not Found)

Based on my research the solution could be just updating

webpack.config.js

However, I can't find webpack.config.js  in my project. They also say to do:

"ng eject"

to create a webpack, but it doesn't seem to work.
SOLUTION: I can get it to work by running this (It actually works!):

ng serve --public-host=http://localhost:4200

BUT I want to be able to set this up somewhere in my app so that I can only do

ng serve

Does anyone know how to fix this issue and gid rid of this error (sockjs-node/info). Thanks a lot in advance!
Answer: You can edit your angular.json file and add the option for public-host. Mind you all options are camelCased not kebab-cased.
...
"serve": {
  "builder": "@angular-devkit/build-angular:dev-server",
  "options": {
    "browserTarget": "congiftest:build",
    "publicHost":"http://localhost:4200"
  },
},
...

This will allow you to just run ng serve and the public-host flag will be added.
Comment: Have you tried this on .Net core - I've tried Https://localhost:5001/subfolder/ and it's still broken - I have tried many variations - any ideas?  I have tried both as a parameter and in angular.json.  I have my app being served inside a sub-folder.
Comment: Note to my future self and any googlers:
It seems like angular 11 has changed the behavior of the port when the application is proxied. Instead of the proxy-port, now the port specified on the command line (--port or the default) is used.
So anyone who develops through a reverse proxy now has to define the public host for any port different than 4200.
Comment: @progx not at the moment, I was mostly answering your second question.
Comment: Is not there a typo? congiftest:build

sockjs-node/info? 404 (Not Found)


Is there a guaranteed priority in Swing events of the same type?

Question: Quick example: I have a JButton with two ActionListener. Will they be executed in a guaranteed order? Is it the same for all kind of EventListener of the same type?
JButton b = new JButton();
b.addActionListener(new ActionListener(){
    public void actionPerformed(ActionEvent e) {
        System.out.println("event 1");
});
b.addActionListener(new ActionListener(){
    public void actionPerformed(ActionEvent e) {
        System.out.println("event 2");
});

Comment: IIRC, event listeners are called in reverse order of being added, but I don't think that's guaranteed. If you need a guarantee, create a composite listener?
Comment: I solved my specific problem by passing a `Callable` to an `ActionListener`
Answer: As @daniu mentioned in the comment section, when a button (or any component) have multiple listeners of the same type (in our example we have 2 action listeners in a JButton) they will be fired from last to first. The comment inside AbstractButton#fireActionPerformed makes it pretty clear:
protected void fireActionPerformed(ActionEvent event) {
    // Guaranteed to return a non-null array
    Object[] listeners = listenerList.getListenerList();
    ActionEvent e = null;
    // Process the listeners last to first, notifying
    // those that are interested in this event
    for (int i = listeners.length-2; i>=0; i-=2) {
        if (listeners[i]==ActionListener.class) {
            // Lazily create the event:
            if (e == null) {
                  String actionCommand = event.getActionCommand();
                  if(actionCommand == null) {
                     actionCommand = getActionCommand();
                  }
                  e = new ActionEvent(AbstractButton.this,
                                      ActionEvent.ACTION_PERFORMED,
                                      actionCommand,
                                      event.getWhen(),
                                      event.getModifiers());
            }
            ((ActionListener)listeners[i+1]).actionPerformed(e);
        }
    }
}

Of course one example to test it yourself is pretty easy to be created:
public class ListenersTest extends JFrame {

    public ListenersTest() {
        super("tests");
        setDefaultCloseOperation(EXIT_ON_CLOSE);

        getContentPane().setLayout(new FlowLayout());

        JButton button = new JButton("press me");
        for (int i = 1; i <= 5; i++) {
            button.addActionListener(new Listener(i));
        }
        add(button);

        setSize(400, 400);
        setLocationRelativeTo(null);

    }

    private static class Listener implements ActionListener {
        private int id;

        private Listener(int id) {
            this.id = id;
        }

        @Override
        public void actionPerformed(ActionEvent e) {
            System.out.println("My ID is:" + id);
        }

    }

    public static void main(String[] args) {
        SwingUtilities.invokeLater(() -> {
            new ListenersTest().setVisible(true);
        });
    }
}

If you press the button you will get:

My ID is:5 My ID is:4 My ID is:3 My ID is:2 My ID is:1
Comment: Yes, this is the current implementation but it is not guaranteed in the Java spec.


Double counting volume of revolution

Question: I want to understand why is there no double-counting when using the volume of revolution formula on a circle. Say we want to find the volume of revolution in the first and second quadrants about the $y$ axis. Say the equation of the circle is $x^2+y^2=R^2$. Using the volume of revolution formula, we get
$$\int_{-R}^{R}\pi (R^2-x^2)dx=\frac{4}{3}\pi R^3$$
which is correct. However, aren't we rotating around the $y$ axis basically twice, as we are doing it once with the quarter circle from $x=0$ to $x=R$ and then again with the quarter circle from $x=-R$ to $x=0$?
There is some error with this logic that I can't find. Can someone please explain to me why the formula holds even though we are apparently doing the operation twice?
EDIT: I just had a thought. Is it because $x^2+y^2=R^2$ is not a function (as it fails the vertical line test)?
Comment: By the way, aren't you rotating around the $x$ axis when you use the formula $V=\pi\int y^2 dx$?
Answer: $$\int_{x=-R}^R \pi (R^2 - x^2) \, dx = \pi \left[R^2 x - \frac{x^3}{3}\right]_{x=-R}^R = \pi \left(R^3 - \frac{R^3}{3} - \left(-R^3 - \frac{(-R)^3}{3}\right)\right) = \frac{\color{red}{4}}{3}\pi R^3.$$

Now that the original question was edited to correct the arithmetic error, the follow-up answer is this:
There is no double counting because the integral is being calculated with respect to $x$, not $y$.  Therefore, the axis of revolution is the $x$-axis, not the $y$-axis, and the differential volume for a representative slice corresponds to a disk of thickness $dx$, and radius equal to the distance from the point $(x,0)$ to $(x, \sqrt{R^2 - x^2})$, i.e. $\sqrt{R^2 - x^2}$.  This gives the full sphere volume as I have shown above.
If you want to calculate the volume of the hemisphere for a volume of revolution above the $x$-axis by rotating about the $y$-axis, then you would write the integral as follows:
$$\int_{y=0}^R \pi (R^2 - y^2) \, dy.$$  For a given $y \in [0,R]$, the differential volume element is a disk of thickness $dy$, with radius equal to the distance from $(0,y)$ to $(\sqrt{R^2 - y^2}, y)$.
Comment: Is there any reason that you wrote $x=-R$ as the lower limit for the integral rather than just $-R$?
Comment: @A-LevelStudent It's just my personal preference; a habit, if you will.  Some people find it redundant, but I find it convenient for keeping track of variables when writing multiple iterated integrals; e.g., $$\int_{x=0}^1 \int_{y=x}^{2-x} f(x,y) \, dy \, dx.$$  But the point is, the integral in question actually equals $\frac{4}{3} \pi R^3$--it counts the full volume of the sphere.
Comment: @heropup thank you for the answer. This does not address my question. My question is why is there no double counting of the full volume of the sphere, as aren't we basically rotating twice (coming from the rotation of the part that is $0<x<R$ which is the same volume as $-R<x<0$?
Comment: @MathsWizzard No.  See my edited answer.

Double counting volume of revolution


How do I evaluate this limit

Question: $$ \displaystyle \lim_{n \to \infty} \dfrac{1}{n} \displaystyle \sum ^n _{k=1} \dfrac{k^2}{n^2}$$
How do I evaluate this? I have never actually learned how to work with infinite series like this one, so I have no idea.
Comment: @Homework Prove it by induction, if you can't do it maybe tell us here or ask another question.
Comment: do you realize that you can factor $1/n^2$ out of the sum? Now use the formula for the sum of $n$ consecutive squares.
Comment: Yes I realized that, I however didn't realize there is a formula for the sum of n consecutive squares. Is it too hard to get from scratch?
Answer: You can recognize a Riemann sum: $$\lim\limits_{n \to + \infty} \frac{1}{n} \sum\limits_{k=1}^n \frac{k^2}{n^2} = \int_0^1 x^2dx= \frac{1}{3}$$
Comment: To generalize this answer see [this](http://math.stackexchange.com/questions/314020/integral-calculus-proof).
Answer: HINT: Use the formula for the sum of the first $n$ squares:
$$\begin{align*}
\frac1n\sum_{k=1}^n\frac{k^2}{n^2}&=\frac1{n^3}\sum_{k=1}^nk^2\\
&=\frac1{n^3}\cdot\frac{n(n+1)(2n+1)}6\;.
\end{align*}$$
Answer: Since $n$ is a constant within the summation, you can move it outside:
$$ \displaystyle \lim_{n \to \infty} \dfrac{1}{n^3} \displaystyle \sum ^n _{k=1} {k^2}$$
The sum of the first $n$ squares is well known to be $\dfrac{n^3}{3} +\dfrac{n^2}{2}+\dfrac{n}{6}$.
Dividing by $n^3$, we have $\dfrac{1}{3} +\dfrac{1}{2n}+\dfrac{1}{n^2}$. The limit as $n \to \infty$ is now obvious as $\dfrac{1}{3}$.
Answer: Let's remember a usefull result on the series: if the serie $\displaystyle \sum_n f(n)$ is divergent then we have
$$\sum_{k=1}^n f(k)\sim\int_1^nf(x)dx.$$
With this result we find the classical result
$$\sum_{k=1}^n\frac{1}{n}\sim\int_1^n\frac{dx}{x}=\log n.$$
Now, we return to the question. Since the serie $\displaystyle \sum_n n^2$ is divergent then we have
$$\sum_{k=1}^n k^2\sim \int_1^n x^2dx=\frac{1}{3}\left[x^3\right]_1^n\sim\frac{n^3}{3},$$
and hence we find
$$\lim_{\infty}\frac{1}{n^3}\sum_{k=1}^n k^2=\frac{1}{3}.$$

How do I evaluate this limit