How to display null values for a field if some other field has values in Oracle

Question: We have requirement wherein we need to display the sum of line cost for all the labor, material, service, tools associated to work order in Maximo. I have written the query however the sum of material line cost is getting doubled of there are more than one service line cost.
For example

wonum  - 1234
material line cost - 10
service line cost - 5 and 6 ( 2 service lines)
total material line cost - 20
total service line cost - 11

The total for material line cost is wrong. I have used the below query, please let me know how to fix it
select a.wonum,a.description,a.location,a.crewid,a.worktype,a.wopriority,a.supervisor,a.actstart,a.siteid,sum(d.linecost) as totalmaterialcost,
sum(b.loadedcost)as totalservicecost
from workorder a
 left outer join matusetrans d
on  a.wonum=d.refwo and a.siteid=d.siteid
left outer join servrectrans b
 on  a.wonum=b.refwo and a.siteid=b.siteid
where a.wonum='1234' and a.siteid='ABC'
 group by a.wonum,a.description,a.location,a.crewid,a.worktype,a.wopriority,a.supervisor,a.actstart,a.siteid

Answer: The problem is that the joins are all at the top level of your statement. This leads to multiple lines/records per workorder.
One solution would be to calculate the sum of matusetrans and servrectrans in two seperate sub-select-statements.
Example:
select a.wonum,
    b.sum as totalservicecost,
    d.sum as totalmaterialcost
from workorder a
left join (
    select sum(b.loadedcost) as sum, b.siteid, b.refwo
    from servrectrans b
    group by b.siteid, b.refwo
) b on a.wonum = b.refwo and a.siteid = b.siteid
left join (
    -- second sum-select goes here
) d on -- second join condition goes here

As a second approach check the workorder-table for columns already containing this data (eventually there is some de-normalization to boost performance).
Comment: Hi Dani , It works however the result takes a longer time to come so this query is affecting performance. Please note that this query is used in a report and the report has 4 input parameters already.
Comment: check for colums containing pre-aggregated data.
Answer: The WORKORDER table will calculate these values for you already. Are the following fields not providing the data you need?
ACTINTLABCOST
ACTLABCOST
ACTMATCOST
ACTOUTLABCOST
ACTSERVCOST
ACTTOOLCOST
ESTATAPPRINTLABCOST
ESTATAPPRLABCOST
ESTATAPPRMATCOST
ESTATAPPROUTLABCOST
ESTATAPPRSERVCOST
ESTATAPPRTOOLCOST
ESTINTLABCOST
ESTLABCOST
ESTMATCOST
ESTOUTLABCOST
ESTSERVCOST
ESTTOOLCOST


"Page Not Found" error when a google search form is submitted

Question: I am using Google Site Search on a web site, and the search form submits to a page containing the appropriate code. I can access this page directly just fine, but whenever I test out the search form, I get a "page not found" error.
I believe this is because both Drupal and Google search use the parameter 'q', has anyone run into this problem and found a workaround?
Comment: What is the path that you can access directly and what is the path that gives you 'page not found'?
Comment: example.com/search is the page with the google search code. page not found is the same path, example.com/search, but with the google search parameters.
Comment: I think there is an alternative parameter Google can use, and that would make it compatible with Drupal, but I don't recall which one is it, at the moment.
Answer: Instead of using q, you can use as_q. The description given from the page I linked is the following:

The as_q parameter provides search terms to check for in a document. This parameter is also commonly used to allow users to specify additional terms to search for within a set of search results.


Branch permissions bypassed on Bitbucket: Pull request requires approval, but merges anyways?

Question: We plan to have only admins able to make changes to our repo's master branch.  Developers can clone the repo and then create their own dev branch off of master to work on. When developers feel ready, they can merge their development branch onto the master branch using a pull request. However, they will not be able to merge their dev branch into master until the admins have approved the changes.
On Bitbucket, I set branch permissions as:
+ Write Access: Rachel and Jamie.
+ Merge via pull request: Rachel, Jamie, and team:developers.
+ Merge checks: check for at least 2 approvals.
Next, we had a developer (not Rachel or Jamie) make changes on their own dev branch.  They then committed and pushed the changes to their remote dev branch--worked as expected.
Last, they initiated a pull request to merge the remote dev branch into master.  A warning that two approvals was needed popped up, but the developer was still able to just click "Merge" and the dev branch merged into the master branch, despite 0 approvals.
I expected that the developer would not be able to merge with 0 approvals.  How do I set permissions so that this is the case?
Answer: Two things:

You explicitly granted "merge via pull request" access to the developers. They're therefore able to merge pull requests. If you never want them to merge PRs to that branch, then revoke that. (They'll still be able to create PRs, just not merge them.)
You didn't mention whether you've enabled "Prevent a merge with unresolved merge checks", under the premium features.
Comment: Thank you for the answer!  I don't have a premium account, so I can't enable the "prevent a merge with unresolved merge checks", but I see the issue.
Answer: We've created a plugin to prevent this situation. You can inspect it on marketplace:
https://marketplace.atlassian.com/apps/1219882/prevent-merge-when-needs-work


What are scale degrees?

Question: I heard my band teacher talking about "scale degrees" and do not understand what he means.
Comment: If you don't understand what your teacher means, the usual thing to do is ask them.
Comment: [Scale degree](https://en.wikipedia.org/wiki/Degree_%28music%29)
Comment: What research have you done, apart from posting the question here? At very least, ask teacher and peers.
Answer: There are 7 different notes in (most) scales.   (The 8th is a repeat of the first.)
C, D, E, F, G, A, B.   The seven notes of C major scale.  Or the 1st, 2nd etc.  degrees of the scale.  That is what 'degree' means.
https://en.wikipedia.org/wiki/Degree_(music)
Answer: Supposing you know that music notes named cdefg or doremifasol are ascending like a scale (meaning a ladder or a stair) so a degrees is a single step of this scale.
But the band leader might also have meant the triads of thirds that are built on each degree: e.g. domiso or ceg. (If you skip always one note you get thirds and fifths like 135. You can play the tones 1-3-5 from any degree of an instrument. On the piano the degrees are the keys in the scale of C major the white keys.
Study the pattern of triads (chords) on a piano keyboard and you will get the idea of scales and degrees. If we play the row of white keys we get the C major scale. Starting with C = I ,D =II,E=III  ...
As we've seen the scale degree can mean the single tone of each step, but also the chord  on each step of this ladder.
Comment: I've never heard 'degree' used to mean a chord.   We could talk of 'the triad on the 3rd degree of C major' (which would be E, G, B-  E minor) but the 'degree' is the note not the chord.
Comment: *In Schenkerian analysis, "scale degree" (or "scale step") translates Schenker's German Stufe, denoting "a chord having gained structural significance" (see Schenkerian analysis#Harmony).* see your posted wiki link.
Comment: Yeah.  But we're talking about music.  Getting into Schenkerian is like citing Leviticus in a discussion of morals :-)

What are scale degrees?


Laravel 5.2: How to autofill the creation date of the form

Question: I'm struggling with a question that i'm asking myself. I have a form in laravel that posts a new row with data on the index file. Right now I have a couple of set values that I insert into the database. One of those values is "date", I am asking myself: How can I from the moment that I press on the 'save' button automatically fill the created_at field in my database.
Would love to get some help with this
Comment: As far as i know laravels timestamps (created_at, updated_at) are automatically filled when updating or creating records. But if you use Model::insert() then they wont be automatically filled. Only if you use fillable eg. Model::create(data);
Comment: Add this line in your migration $table->timestamps();
Answer: just use date('Y-m-d H:i:s'); for created_at field inside your controller (or where you save your obj)
Or use laravel timestamps. It wil auto stamp the date of update and creation
Edit: like our Dutch friend mentioned. just use date
Comment: date('Y-m-d H:i:s'). No need for carbon.
Comment: This answered my question. thanks for the info. I can only accept this answer as the correct one in around 5 minutes (-:
Comment: Dutchies United


Are the iv and V chords used in the minor scale just like the major scale?

Question: In the major scale the V is the dominant chord, and the IV is the sub-dominant chord. Is it the same exact thing in the minor scale?
Also, I saw that the V and IV chord are the most played chords in the major scale (see: http://www.hooktheory.com/blog/i-analyzed-the-chords-of-1300-popular-songs-for-patterns-this-is-what-i-found/)
But are they just as popular in the minor scale? Meaning, is the iv and V in minor function the same as IV and V in major
Answer:
In the major scale the V is the dominant chord, and the IV is the sub-dominant chord. Is it the same exact thing in the minor scale?

"Dominant" in these two terms means fifth. The V chord is called the dominant because its root is a fifth above tonic, and the IV chord is called the subdominant because its root is a fifth below tonic. (It's not called subdominant because it is "below dominant.")
As such, these terms hold true no matter what key or mode you are in. If the root is a fifth above tonic, it is the dominant triad; if the root is a fifth below tonic, it is the subdominant triad. This obtains in minor keys, the dorian mode, etc.

Regarding the frequency of these chords in minor keys, HookTheory seems not to have published that information, because that's not how they went about collecting their data. They discuss it here:

As you have noted, Hooktheory currently stores data in the relative major mode. We chose this because we felt that the majority of progressions in popular music are either in the major mode, or ambiguous enough that analyzing them in the major mode is satisfactory. For songs that are properly in a different mode, the relationships between chords would be preserved (e.g., vi → IV of the ionian mode and i → ♭VI of the aeolian mode are counted as the same trend).

One of the reasons they chose not to focus specifically on minor keys is due to the size of their data set, because over 90% of their tabs are in the Ionian mode.
Answer: Most of the chords in the minor mode have the same function as in the major mode. The i, iv, and V function exactly the same way.
Exceptionally, the VII in the minor mode is common in sequences and as the V/III. The ii0 in the minor is even stronger in the ii0-V-i pattern than the corresponding ii-V-I. The ii can also be used in a ii=V-i pattern. Likewise a chromatic II-V-i can be used similarly to the II-V-I in the major. The III needs a bit of care so as not to sound like a tonic. The VI is used similarly in the minor mode; a deceptive V7-VI works here too.
Answer: V (complete with being a major chord) is just as popular in minor keys as in major keys. Both types of keys use it as a dominant-function chord.
IV (the major chord) is significantly less popular in minor keys than in major ones, especially since it doesn't tend to resolve to the tonic in minor keys (as opposed to IV in major keys). It's still used, though, and it still plays a subdominant role otherwise.
In minor keys, iv (the minor chord) tends to take over IV-in-major-keys's role, including resolving to the tonic.
Comment: oops yes, I meant IV = iv when I wrote the question
Answer: While theory supports that the iv and V work the same way for a minor Key, always let what I call "ear and flow" be your guide.  Theory must always be subordinate to practice.  In other words, if theory suggests something is correct, make sure it sounds good or flows with your song.
And so, having said that, what I've found is that while IV-V-I sounds good in a Major Key, what works for a minor Key, among perhaps other things, is VI - V (making the v a Major - borrowing from the "Harmonic minor") - to the i.
In other words, in A-Major, D - E - A sounds works, while in A-minor, F - E - Am works (like in "Mary Did You Know?").
David ;)


How to Add data to Dropdown in windows forms applications in C#

Question: I am trying to create a windows form application. To create a .Zip file with a password and sending the .zip file to the clients email address in one email and another email to send password of the .zip file to the client...
I am using a dropdown box to enter "TO:" email address... and I want the user not to write the entire email address if it is an frequently used email ID.. So want the whole email ID to display if entered starting part of it... (Like an intellisense) and if it is a new one then make a new entry.. I am not sure how to implement this.. as looks like dropdown does not do this..
Any one please help me or suggest how to achieve this functionality..
Thanks in Advance!
Answer: You can tell it to use the items in your ComboBox as a sort of "suggestion" list for the user. It'll display the nearest match as they type, which I think is exactly what you're looking for.
Set these properties on your ComboBox, either in the code-behind or at design time:
comboBox1.AutoCompleteSource = AutoCompleteSource.ListItems;
comboBox1.AutoCompleteMode = AutoCompleteMode.SuggestAppend;

Answer: You are talking about autocomplete.
See here: http://www.c-sharpcorner.com/UploadFile/mahesh/AutoCompletion02012006113508AM/AutoCompletion.aspx


Explanation of the rational root theorem

Question: I just discovered the rational root theorem and I feel like I can understand it if I can get past the notational jargon presented in Wikipedia.
Here's what I think I understood. If $r$ is a rational solution of a polynomial $f(x)$ then the function can be rewritten as $(x-r)(g(x))$ where $g(x)$ is another polynomial. Expanding thus with $r$ places it in the constant term. Assuming for $g(x) \implies a, f(x) \implies -ar \quad \text{As the constant}$. So r is divisible by the constant term.
First of all, is that correct. And is there a way to explain the proof for the theorem without scary notation?
Comment: But you could use the rational root theorem to narrow down the list of suspects.  The theorem tells us that, in this case, "if $\frac mn$ is a root (assuming the fraction is in lowest terms) then $m\,|\,1$ and $n\,|\,4$.  Thus the only candidates would be $\pm 1, \pm \frac 12,\pm \frac 14$" so you could find it that way.
Comment: As another example. consider $(x-2)(x-3)=x^2-5x+6$.  Clearly, that has roots at $2$ and $3$.  Neither of those are divisible by the constant term $6$.
Comment: What does it mean to say that $r$ is "in the constant term"?  Consider the polynomial $4x^2-4x+1$.  That has a (rational) root at $x=\frac 12$.  Is that "in the constant term"?
Comment: It's sort of divisible by the constant term. But you see that doesn't make sense to me, how you got that number.
Comment: Well, I got it by starting with $\left( x-\frac 12\right)^2$ and then clearing the denominator.
Comment: If the leading coefficient is $1$, a rational root must be even an integer root and divide the constant term (The root is a divisor of the constant term , it is not divisble in general by the constant term).
Comment: Or did I mean that 6 is divisible by either one of the numbers
Answer: The statement of the rational root theorem is that if polynomial with integer coeficients is $P(x) =a_nx^n + a_{n-1}x^{n-1} + ...... + a_1x + a_0$ has a rational root (bear in mind it may have irrational roots) the ration root, call it $r$, will but be of the form $r = \frac jk$ where $j$ is a factor of $a_0$ and $k$ is a factor of $a_n$.
For instance if  $2x^2 -9x + 4$ has any rational roots (it might not) then, as factors of $4$ are $\pm 1,\pm 2, \pm 4$ and the factors of $2$ are $\pm 1, \pm 2$, then the rational roots will be of the form $\frac jk$ where $j = \pm 1, \pm 2, \pm4$ and $k = \pm 1, \pm 2$ and, indeed, $r_1 = 4=\frac 41$ is one root, and $r_2 = \frac 12 =\frac 12$ is another.
And the proof can go like this:
Let $P(x) = a_nx^n + ..... + a_0$.  Now for any $b$ we can do synthetic division to divide $P(x)$ but $x-b$ to get some $\frac {P(x)}{x-b} = c_{n-1}x^{n-1} + c_{n-2}x^{n-2}+ .... + c_1x + c_0 + \frac s{x-b}$ and if we write $c_{n-1}x^{n-2} + c_{n-2}x^{n-2}+ .... + c_1x + c_0$ as $Q(x)$ we get $P(x) = (x-b)Q(x) + s$ where $s$ is a remainder constant.
Now if it turns out that $b$ was one of the roots of $P(x)$ we would have $P(b) = (b-b)Q(b) + s= s$.  That means $s= 0$ and $P(x) = (x-b)Q(x)$.
ANd that means that $P(x) = a_nx^n + a_{n-1}x^{n-1} + ..... + a_1x + a_0$ and $P(x) = (x- b)Q(x) = (x-b)(c_{n-1}x^{n-1} + c_{n-2}x^{n-2}+ .... + c_1x + c_0) = c_{n-1}x^n + (c_{n-2}-bc_{n-1})x^{n-1} + ....... +(c_0 -bc_1)x + c_0b$.
And that means  $c_0b = a_0$ and so $b=\frac {a_0}{c_0}$.  If $b$ is rational then $\frac {a_0}{c_0}$ when reduced to lowest terms will have a factor of $a_0$ in the numerator.
The also means $c_{n-1} = a_n$ and $c_{n-2}-bc_{n-1} = c_{n-2}-ba_n = a_{n-1}$.
This means $b = \frac {c_{n-2}-a_{n-1}}{a_n}$. And if $b$ is rational then $ \frac {c_{n-2}-a_{n-1}}{a_n}$ when reduced to lowest terms will have a factor of $a_n$ in the  denominator.
......
I've never done this before but let's see if we do an example:
Take $P(x) = 2x^2 -9x + 4$ and divide it by $x-\frac 12$.
We get $P(x) = 2x^2 - 9x + 4 = 2x(x-\frac 12) + x -9x +4 =$
$2x(x-\frac 12) - 8x + 4 = 2x(x-\frac 12) - 8(x-\frac 12) +(-4) + 4=$
$2x(x-\frac 12) - 8(x-\frac 12) = (x-\frac 12)(2x -8)$.
Now if we expand that out we will have $Ax^2 + Bx + C$ where:
$A = 2*1=2$ and $B=-8-\frac 12*2= -8-r*A$ and $C=-\frac 12*(-8)=-r*(-8)$.
So $\frac 12=r =\frac {-8-B}{A}$.  so we must have the denominator of $r$ divide $A$.  And, indeed, the denominator of $r$ is $2$ and $A$ is $2$.
And $\frac 12=r=\frac {-C}{-8}$ so we must have the numerator of $r$ divide $C$.  And indeed, then numerator of $r$ is $1$ and $C= 4$.
Comment: Took a lot of words but I appreciate the help
Answer: Take your polynomial $p(x) = a_0  + a_1 x + a_2 x^2 + \dotsb + a_n x^n  = 0$. We can take the $a_i$ as having no common factor, as we could divide it out. If $u/v$ is a rational zero of $p(x)$, written in lowest terms, substitute and multiply by $v^n$:
$\begin{align*}
  a_0
        + a_1 \frac{u}{v}
        + a_2 \frac{u^2}{v^2}
        + \dotsb
        + a_n \frac{u^n}{v^n}
         &= 0 \\
a_0 v^n
        + a_1 u v^{n - 1}
        + a_2 u^2 v^{n - 2}
        + \dotsb
        + a_n u^n
         &= 0
\end{align*}$
All terms are integers. The second to the last terms are all divisible by $u$, so the first one must be divisible by $u$ also, thus $a_0$ is divisible by $u$. In the same way, considering the first up to the next to last terms and $v$, $a_n$ is divisible by $v$.


Python - O que é enum?

Question: Eu usei a função type em um objeto e o retornado foi: <enum 'Button'>.
O que isso significa?
Answer: Enum é basicamente uma lista de valores constantes. O uso de enums torna mais fácil para o programador acessar uma informação. O Enum associa, de forma automática ou manual, um valor numérico à uma string. Veja aqui.

Python - O que é enum?


accuracy of a binary classifier vs probability of binomial distribution

Question: I got confused over a simple concept.
Imagine that I have a binary classifier with 50% accuracy. So, if there are 10 samples to be classified as "y", "n", it has predicted 5 of them correctly.
Now, Imagine that I just guess the categories for each sample randomly (50% chance of getting it right for each one).
For the second one, it should be:
<code>P(5 success) = 10!/(5!*5!) ((0.5)^5 * (0.5)^5) = 0.246
</code>
Are these two comparable?
This just came out of a comment from a friend who said "your model is just as good as a random guess", and I want to prove him wrong, but not sure how to put it together.
Answer: I agree with wrtsvkrfm in the sense that accuracy is not helpful. I disagree, though, in that I don't think sensitivity and specificity are any more helpful. I give my arguments here.
Specifically, the more useful comparison of your classifier in terms of accuracy is not to a random classifier. Instead it is to the best naive classifier, i.e., the one that yields the highest accuracy. And that is not the random one. It is the one that always classifies as the majority class. For instance, if class A occurs 60% of the time and class B 40% of the time, classify everything as A to immediately have an accuracy of 60%. If your classifier achieves only 50%, then you are worse than this. This is one reason why accuracy is particularly problematic if you have unbalanced classes, but as above, I argue that it is not the right tool even for balanced ones.
Comment: Unfortunately, there is no simple good way to assess classifiers. Accuracy is simple, but not good. [Tag:scoring-rules] are good, but not simple. I don't fully understand what you mean be "the probability of achieving 50% success". You may be talking about [the TPR](https://en.wikipedia.org/wiki/Sensitivity_and_specificity), which has the exact same problems as accuracy.
Comment: Thanks, I read all the comments. Still not sure how to look at it. Let's put it this way. Is there any way to find out the probability of my model achieving 50% success? I just think I am comparing orange and apple. 50% accuracy is not the same as probability of 50% success. how do I use a single metric to compare the two approaches?
Answer: When you have $n$ classes, and your classifier's accuracy is nearly $\frac{1}{n}$, it is customary to say that "your classifier is as good as random classifier"; because a random classifier would predict approximately $\frac{N}{n}$ instances correctly as yours, if you have $N$ samples (ignoring class imbalance situations for the simplicity of the discussion). So, he has a point.
Besides, your approach has flaws. We are concerned with the expected success, not the exact probability of having $\frac{N}{2}$ correct classifications. Imagine, your number of samples increases up to $N = 100$, then $P(50 \ \ \text{success})$ would be approximately $0.08$. Here, you're ignoring the neighbourhood.
Comment: It's not random, but it does no better than random classifier.
Comment: So, it is a random classifier if we assume balanced classes. right?
Answer: It's not the accuracy that counts, but rather the sens, and spec.  Take, for example, a sample size of $n=100$, with $n=5$ diseased and $n=95$ non-diseased. Assume further that the classifier is a shoe box which does nothing, then the classifier will spit out 100 votes for nothing (non-diseased).  In this case it's accuracy will be 95% since only 5 objects will be truly misclassified.   However, when running sens and spec, they will pick up the failure.
In other words, don't get hung up on accuracy.
Comment: agree, but let's consider a balanced situation for simplicity


Intercept increases in regression when adding explanatory variables

Question: I am conducting an analysis, where I examine the size of the intercepts of three regression models (time-series). The models look something like this:
$y_1=\alpha+\beta_1x_1+\varepsilon$
$y_2=\alpha+\beta_1x_1+\beta_2x_2+\beta_3x_3+\varepsilon$
$y_3=\alpha+\beta_1x_1+\beta_2x_2+\beta_3x_3+\beta_4x_4+\beta_5x_5+\varepsilon$
When I run the regressions, I then examine values of $\alpha$ and $t(\alpha)$ for all three models. I find that as I add factors (going form model 1 to 3), the $R^2$ increases (as expected), so it seems that the added factors add some explanation. However, I find that the $t(\alpha)$ values increase as I add factors.
Maybe this is something that I am missing, but shouldn't $t(\alpha)$ decrease as I add factors, since more of the explanatory output is now explained by the factors, and is not put on the intercept?
Edit: $t(\alpha)$ is the estimated $t-$statistic of the $\alpha$ intercept from the regression output.
Comment: What does the notation $t(\alpha)$ mean?
Comment: This could be a problem of overfitting. $R^2$ will go up regardless of whether or not the new variable improves the model. That is why we look at other measures such as adjusted $R^2$, AIC, BIC and Mallows' $C_p$.
Answer: It's not true that adding predictors should generally cause the estimate of the intercet $\alpha$ to decrease.
The intercept is the predicted $y$ value when all the $x$ predictors are equal to 0. So adding new predictors can cause the intercept to increase or decrease, by pretty much any amount, based on the mean of the $x$ predictor you're adding and the size/direction of its corresponding regression coefficient.
This becomes especially clear when we note that we can write the estimate for the intercept as
$$
\hat{\alpha} = \bar{y} - \hat{\beta_1}\bar{X_1} - \hat{\beta_2}\bar{X_2} - \dots,
$$
where $\bar{y}$ denotes the sample mean of $y$, $\hat{\beta_j}$ is the sample estimate of $\beta_j$, and $\bar{X_j}$ is the sample mean of $X_j$.
Comment: Thank you very much, Mr. @JakeWestfall! I guess that it was a hole in my regression terminology. This is a very nice explanation.


log4j2 mapping xml to properties file

Question: I have a log4j2 configuration written in XML, however I need to make them in properties format (key and value) for arrangement purposes. I cannot seem to map nested conditions in xml to properties.
Here is my XML configuration:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="DEBUG">
    <Properties>
        <Property name="log-path">./var/log/reporter/</Property>
    </Properties>
    <Appenders>

        <RollingFile name="RollingFile" fileName="${log-path}/reporter.log"
            filePattern="${log-path}/reporter-%d{yyyy-MM-dd}-%i.log">
            <PatternLayout>
                <pattern>[Reporter] [%level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] [%logger{36}] %msg%n</pattern>
            </PatternLayout>
            <Policies>
                <TimeBasedTriggeringPolicy interval="1" modulate="true" />
                <SizeBasedTriggeringPolicy size="20 MB" />
            </Policies>
            <DefaultRolloverStrategy max = "1000">
                <Delete basePath="${log-path}">
                    <IfFileName glob="reporter-*.log" />
                    <IfAny>
                      <IfLastModified age="90D" />
                      <IfAccumulatedFileSize exceeds="2 GB" />
                    </IfAny>
                </Delete>
            </DefaultRolloverStrategy>
        </RollingFile>

    </Appenders>
    <Loggers>
        <Logger name="root" level="debug" additivity="false">
            <appender-ref ref="RollingFile" level="debug" />
        </Logger>
        <Root level="debug" additivity="false">
            <AppenderRef ref="RollingFile" />
        </Root>
    </Loggers>
</Configuration>

And here is my non-working properties configuration mapping:
appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${logpath}/reporter.log
appender.rolling.filePattern = ${logpath}/reporter-%d{yyyy-MM-dd}-%i.log
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = [Reporter] [%level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] [%logger{36}] %msg%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=1KB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 1000
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${logpath}
appender.rolling.strategy.action.NestedConditions.type = IfALL
appender.rolling.strategy.action.NestedConditions.PathConditions.type = IfFileName
appender.rolling.strategy.action.NestedConditions.PathConditions.glob = reporter-*.log
appender.rolling.strategy.action.NestedConditions.NestedConditions.type = IfAny
appender.rolling.strategy.action.NestedConditions.NestedConditions.time.type = IfLastModified
appender.rolling.strategy.action.NestedConditions.NestedConditions.time.age = 90D
appender.rolling.strategy.action.NestedConditions.NestedConditions.size.type = IfAccumulatedFileSize
appender.rolling.strategy.action.NestedConditions.NestedConditions.size.age = 2GB

logger.rolling.name = root
logger.rolling.level = debug
logger.rolling.additivity = false
logger.rolling.appenderRef.rolling.ref = RollingFile

rootLogger.level = debug
rootLogger.appenderRef.stdout.ref = RollingFile

Comment: Mohamed, were you able to map your xml to properties? I am kind of in a similar situation..
Comment: Unfortunately not, we have used properties file instead eventually.
Comment: If you used properties file, then how did you manage multiple nested conditions? If there is only one nested condition, then elifyilbasi's answer would match, but if there are two or more, then the resulting property would look like a duplicate of the previous nested condition and thus it won't work.
Answer: It a little bit late but you can gain insight from the link https://github.com/elastic/elasticsearch/blob/master/distribution/src/main/resources/config/log4j2.properties
It has all the conditional property usages you needed.
Answer: the file has moved link has been updated to https://github.com/elastic/elasticsearch/blob/master/distribution/src/config/log4j2.properties

log4j2 mapping xml to properties file


Order statistic expectations and the variance of the original distribution

Question: Is there a proof or theorem out there that says that the difference between the expectations of the order distributions of $n$ draws from a distribution $X$ decreases as the variance of $X$ decreases?
Specifically, I am asking:
If X and Y are probability distributions defined over the same range and with the same mean,
does Variance(X) < Variance (Y) imply that $\mathbb{E}X_{kn} - \mathbb{E}X_{sn} < \mathbb{E}Y_{kn} - \mathbb{E}Y_{sn}$ where $X_{in}$ is the $i_{th}$ greatest of $n$ draws from $X$, and $k>s$.
In fact, I can be even more specific, as I am interested  in the difference between the expectation of the mean and the extreme values.  So, does $Var(X) < Var(Y) \to \mathbb{E}X - \mathbb{E}X_{1n} < \mathbb{E}Y - \mathbb{E}Y_{1n}$ (and the same for the absolute value of the difference between EX and EXnn)
Also, for the general case:  would it be true that for any distributions X and Y defined over the same range, that Variance(X) < Variance (Y) implies $\mathbb{E}X_{kn} - \mathbb{E}X_{sn} < \mathbb{E}Y_{kn} - \mathbb{E}Y_{sn}$
Comment: I fear this question may not be well-posed (yet). For example, some order statistics may have well-defined means, even when the mean (or variance) of the distribution is undefined. Second, the parameterization of a particular family of distributions may not be closely related to the variance. Are you, perhaps, interested in some restricted class of distributions, such as *[location-scale families](http://en.wikipedia.org/wiki/Location-scale_family)* or similar? More details would be welcome.
Comment: @cardinal:  I updated to hopefully be more clear, and now I am looking through your link.
Comment: Consider a $\mathrm{Ber}(p)$ random variable. Then $\mathbb E X - \mathrm X_{1n} = p - p^n$ and this is not monotonic in $p(1-p)$.
Answer: One shouldn't expect general relationships to hold between moments and order distributions.
As an example, consider the family of discrete distributions $f_{(\alpha,\beta,p)}$ for which $\Pr(\alpha)=p$, $\Pr(-\beta)=(1-p)/2$, and $\Pr(0)=(1-p)/2$ ($\alpha$ and $\beta$ are non-negative; $0 \le p \le 1$).  I plan to choose $\alpha$ and $\beta$ later and right now just want to investigate whether the variance $V$ and expected range of $n=2$ independent draws, $E$, are increasing or decreasing.  ($E$ is, of course, the difference between the first and second order statistics.) It is elementary to compute
$$\frac{dV}{dp} = \frac{(2\alpha+\beta)^2 - \beta^2}{4} - \frac{p}{2}(2\alpha+\beta)^2$$
and
$$\frac{dE}{dp} = 2\alpha - (4\alpha+\beta)p.$$
Consequently we can find simple formulas for where they attain their maxima.  Plotting the differences of these locations, as a function of $\alpha/\beta$, suggests looking at the case $\alpha=\beta$: the maximum of $V$ is attained when $p=4/9$ and the maximum of $E$ is attained when $p=2/5$.  Consequently, if we fix $\alpha=\beta$ and vary $p$ from $4/9$ to $2/5$, the variance is decreasing while the expected range is increasing.  The differences are small but real; for instance, with $\alpha=\beta=1$, the variance and expected range at $p=4/9$ are 0.694444 and 0.895062, respectively, whereas at $p=2/5$, they are 0.69 and 0.9, respectively: one has decreased and the other has increased.
Comment: (+1) For this answer and the other similar one. By happenstance the examples you give in both were quite close to ones I had contrived, but did not have a chance to post. It's always a nice present to see myself thinking along the same lines as you have in the answers you post. :)


AI::MXNet (Perl API) : how to fit() a zoo model

Question: I am trying to use the Perl API of MXNet (AI::MXNet) to fetch a standard model from ModelZoo (AI::MXNet::Gluon::ModelZoo::Vision) and train it.
The following code does not complain but it does not fit(). fit() just returns immediately. My dataset is a file of images in the RecordIO format (constructed using im2rec) and I am quite sure it is valid.
I have modified code from here https://codehex.hateblo.jp/entry/2017/09/12/160149  , where the original author creates the model symbolically (and not from ModelZoo).
Is this, in principle, the right way to train a model from ModelZoo?
A second question is what's best practice for creating a data loader from a set of images. Is what I am doing (using mx->io->ImageRecordIter) OK?
use strict;
use warnings;

use AI::MXNet qw/mx/;

use AI::MXNet::Gluon qw/gluon/; # needed for nn-> (which must be replaced by gluon->nn->...)
use AI::MXNet::Gluon::NN qw(nn); # for nn->
use AI::MXNet::Gluon::ModelZoo::Vision::VGG;
use AI::MXNet::Monitor;

my $ctx = mx->cpu(0);

my $vgg = AI::MXNet::Gluon::ModelZoo::Vision->get_vgg(
    11, # num layers
    (
        'classes' => 26, # latin alphabet recognition
        'root' => 'abc', # where to save the models
        'ctx' => $ctx # context
    )
);
die "get the model" unless defined $vgg;

my $batch_size = 4;
# num channels, width, height our png training images of letters:
my $img_shape = [3, 256, 256];
my $training_file = 'training.bin';
my $train_dataiter = mx->io->ImageRecordIter({
    'path_imgrec' => $training_file,
    'path_imglist' => 'training.lst',
    # num channels, width, height
    'data_shape' => $img_shape,
    'batch_size' => $batch_size,
    'label_width' => 1, # dimensionality of labels, for us is 1 (i.e. just the letter name)
});
die "mx->io->ImageRecordIter()" unless defined $train_dataiter;

$vgg->init_params(initializer => mx->init->Xavier(magnitude => 2));
$vgg->init_optimizer(optimizer => 'sgd', optimizer_params => {learning_rate => 0.1});
$vgg->initialize();

print "$0 : fitting ...\n";
$vgg->fit(
    $train_dataiter,
    'num_epoch' => 1000
);

Answer: It's not the right wait to train. The ModelZoo returns a Gluon Hybrid block.
The 'fit' is used in Module interface.
See how you can train with the Gluon:
https://metacpan.org/source/SKOLYCHEV/AI-MXNet-1.4/examples/gluon/mnist.pl
The same with Module:
https://metacpan.org/source/SKOLYCHEV/AI-MXNet-1.4/examples/mnist.pl
You can call ->export method of hybrid block like $vgg->export();
https://metacpan.org/source/SKOLYCHEV/AI-MXNet-1.4/lib/AI/MXNet/Gluon/Block.pm#L1242
This will save json structure of the network and current parameters into a file.
Then you can initialize the Module interface from these.
Comment: Hito Yume thanks! I need to use my custom data which are images of alphabet letters. I am flexible about model or if Gluon or not. I do not know how to use the 1st example you cited with my dataiter and I do not know how to use the 2nd example with some ready-made solution of optical character recognition. Do you have any more information about how to use a dataiter with a Gluon from ModelZoo? Do you have any information about exporting and importing parameters? In ```export()``` method it says to move forwartd once? How do you do that? Summary: I need lots of examples please if you have any.
Comment: The gluon example I posted will work as is I think with any iterator, because all of them have the same interface. The forward operation is simply calling gluon block as a subroutine with NDArray (converted image) as $vgg->($image_nd_array); . See dog breed classification example on this page http://blogs.perl.org/users/sergey_kolychev/2018/07/machine-learning-in-perl-kyuubi-goes-to-a-modelzoo-during-the-starry-night.html
Comment: If you are willing to post some code with custom training data, perhaps similarly to the code I posted, it would be much helpful. I have read and tried the links you posted many times, they are tied to some use-cases which are not what I have. I tried to adapt to my use-case but ... So post some skeletal code if you can please.
Comment: thank you @Hito Yume. I did not mean that you must get my code working but I would appreciate it if you already have some code working and you can share it with me. Please don't spend any time on this unless you already have some personal examples you want to share. I am sorry if I sounded a bit pressing. It's just that I tried already the links. So, thank you for your input, if you have already some working code then please share, otherwise don't spend more time on it if you have something better to do... thanks Hito Yume


Where methods (Clear, ForEach, Where, etc.) of powershell array are defined?

Question: According to the offical document about arrays in PowerShell, several methods are defined for arrays, for example Clear(), ForEach(), Where(), etc. Following code tested these methods:
$arr = 1..2
$arr.Clear()
$arr.Length

write "--------------------------------------------------"
$arr = 1..2
$arr.ForEach({$_ + 1})

write "--------------------------------------------------"
$arr = 65..90
$arr.Where({($_ % 2) -eq 0})

Output:
2
--------------------------------------------------
2
3
--------------------------------------------------
66
68
70
72
74
76
78
80
82
84
86
88
90

Fine! And, methods such as ForEach() have many overloads that are not tested here.
But where these methods are defined? I mean, What is the class that contains the definition of these methods? As far as I know, these methods are not defined in .net core. (I use PowerShell 7)
Answer: Clear() is part of the IList interface implemented by System.Array, which is the base type for collections (System.Object[]) in PowerShell.
To see all native methods available, use:
$arr.PSObject.Methods
# or
Get-Member -InputObject $arr -MemberType Methods

However: The Where() and ForEach() "magic" methods were introduced in v4 and are actually PowerShell-specific extension methods (as more performant alternatives to ForEach-Object and Where-Object), defined in System.Management.Automation.EnumerableOps. Have a look at the source:
internal static object Where(IEnumerator enumerator, ScriptBlock expressionSB, WhereOperatorSelectionMode selectionMode, int numberToReturn)

internal static object ForEach(IEnumerator enumerator, object expression, object[] arguments)

Comment: Very helpful! Thank you very much.
Comment: Yes! I accepted it. Thank you again. I just posted a new question about PowerShell, can you give me some suggestion? (https://stackoverflow.com/questions/64711177/why-host-ui-rawui-readkey-method-captures-a-not-really-pressed-enter-key)
Answer: Why do you state As far as I know, these methods are not defined in .net core. (I use PowerShell 7)? Really:
$PSVersionTable.PSVersion | Out-Default
$arr = 1..2
$arr.GetType() | Get-Member -MemberType Methods -Static

Major  Minor  Patch  PreReleaseLabel BuildLabel
-----  -----  -----  --------------- ----------
7      0      3

   TypeName: System.Object[]

Name            MemberType Definition
----            ---------- ----------
AsReadOnly      Method     static System.Collections.ObjectModel.ReadOnlyCollection[T] …
BinarySearch    Method     static int BinarySearch[T](T[] array, int index, int length,…
Clear           Method     static void Clear(array array, int index, int length)
ConstrainedCopy Method     static void ConstrainedCopy(array sourceArray, int sourceInd…
ConvertAll      Method     static TOutput[] ConvertAll[TInput, TOutput](TInput[] array,…
Copy            Method     static void Copy(array sourceArray, array destinationArray, …
CreateInstance  Method     static array CreateInstance(type elementType, int length), s…
Empty           Method     static T[] Empty[T]()
Equals          Method     static bool Equals(System.Object objA, System.Object objB)
Exists          Method     static bool Exists[T](T[] array, System.Predicate[T] match)
Fill            Method     static void Fill[T](T[] array, T value), static void Fill[T]…
Find            Method     static T Find[T](T[] array, System.Predicate[T] match)
FindAll         Method     static T[] FindAll[T](T[] array, System.Predicate[T] match)
FindIndex       Method     static int FindIndex[T](T[] array, System.Predicate[T] match…
FindLast        Method     static T FindLast[T](T[] array, System.Predicate[T] match)
FindLastIndex   Method     static int FindLastIndex[T](T[] array, System.Predicate[T] m…
ForEach         Method     static void ForEach[T](T[] array, System.Action[T] action)
IndexOf         Method     static int IndexOf(array array, System.Object value), static…
LastIndexOf     Method     static int LastIndexOf(array array, System.Object value), st…
new             Method     System.Object[] new(int )
ReferenceEquals Method     static bool ReferenceEquals(System.Object objA, System.Objec…
Resize          Method     static void Resize[T]([ref] T[] array, int newSize)
Reverse         Method     static void Reverse(array array), static void Reverse(array …
Sort            Method     static void Sort(array array), static void Sort(array keys, …
TrueForAll      Method     static bool TrueForAll[T](T[] array, System.Predicate[T] mat…


How many distinct ways are there to get out in cricket?

Question: Recently I attended an interview, in which the interviewer asked me how many types of out are there in cricket.
I counted on my fingers and said 6. He said it was wrong and told me that there are a total of 11 outs.
What are they?
Comment: Out of curiosity, interview for what?
Comment: Maybe you can already add the six that you know to the question?
Comment: @RanadheerReddy Hope he didn't hold your answer against you!
Comment: @ SigueSigueBen.... Interview for a software job. This is happened in HR(Human Resources) round. He asked me what's your habbies? I said playing cricket. So, he asked me this question.
Answer:
Bowled (a delivery hits the stumps and dislodges a bail)
Timed out (the batsman takes over 3 minutes to enter the field of play after a wicket falls)
Caught (the batsman hits the ball and a fielder catches it before it bounces)
Handled the ball (the batsman deliberately touches the ball with a hand that is not holding the bat)
Hit the ball twice (the batsman deliberately hits the ball twice)
Hit wicket (the batsman dislodges a bail with his person or bat)
Leg before wicket (the batsman's person prevents a delivery from hitting the wicket)
Obstructing the field (the batsman illegitimately obstructs the fielding side)
Run out (the batsman's wicket is put down when he is outside his crease while attempting a run)
Stumped (the striker's wicket is put down when he is outside his crease and not attempting a run)
Retired - out (the batsman retires for any reason other than "illness, injury or any other unavoidable cause")

Most of these explanations are oversimplifications - see the relevant Laws for the precise details.
Comment: If a batsman retires without a reason such as injury, he is recorded as "Retired - out" on the score sheet.  This is always the tricky one in this kind of puzzle question.  There are only 10 ways to be dismissed in cricket, but there are 11 ways to appear as "out".  Both 10 and 11 could be considered correct answers, depending on how exactly you phrase the question.
Comment: @gout: Absent hurt is not a dismissal or a way to get out.  A player who is absent by definition does not commence his innings, and therefore cannot be dismissed.  "Absent hurt" is only an entry in the score book for a player who did not bat when a side's innings was completed under Law 12.3 (b) (no further batsman available to come in).  The term "absent hurt" is not used in the laws or the ICC playing conditions.  Unlike the listed methods to get out, absent hurt and retired hurt do not count as an out in a player's career statistics, for example.
Comment: @PeterEisentraut; what about mankading.
Comment: Could you add a short description to each out? as it is this list, while answering the question, doesn't really add much to this site.
Comment: Nice answer Peter Eisentraut.
Comment: What is "Retired - out"?
Comment: @gout. What is that meant ?
Comment: @gout: Mankading is a run out.
Comment: @PeterEisentraut. Yes, it comes under run out.I think Absent hurt is also a type of dismissal.
Comment: @RanadheerReddy. If a bowler during his delivery stride intentionally whips off the bails, in order to dismiss the non striker who is out of the crease. In recent times R Ashwin has done it in IPL as well as in CB series.


label is disappear with auto layout if it does not have any string

Question: I am very new in IOS development.
I'm having trouble in auto layout, I have a label and "hello" string in it and I'm deleting characters using button click from string. If i don't use auto layout label is not being disappear(while it does not contain a single character).
The problem is using auto layout(I have given a standard constrains to top layout),now label is disappearing.
Any one faced the same problem ? have any solution ?
Thank you in advanced.
Answer: In auto layout, width of the label is decided on the length of the constraint if width constraint is not added.
Add a width constraint and the label will remain independent of the length of the text.
Comment: Thank you so much Keerthi for replying so soon. It works. :)
Comment: Welcome! :)Please accept the answer if it helped you :)


Domain points me to some malicious URL and I can't get rid of it

Question: Whenever I enter my URL into the browser, it keeps pointing me to
http://fornax.myvnc.com/dev

The URL doesn't even work and my antivirus doesn't complain about it so I am not sure what is happening. I logged into my domain manager at godaddy and it says that the nameservers are pointing to:
NS46.DOMAINCONTROL.COM

and I am not sure where this came from either because my hosting is with lunarpages whose nameserver is
NS1.LUNARMANIA.COM

I tried looking into my .htaccess and it is blank. My index.php was hijacked with some malicious code so I removed it completely. Everything is supposed to be normal now but still some kind of a redirection is taking place and am not sure where this is happening. Any suggestions?
Comment: Can you tell us what your domain is so we can troubleshoot for you?
Comment: @Scott: Issues got resolved so removed the URL from here... Thanks for the help :)
Answer: Create a new simple index.php page with basic information in it... ie. hello world.  See if you still get redirected.  This will at least narrow down what could be causing the problem.
It's probable that there is still some code hidden in the index.php page redirecting you, or, some dodgy iframe link.  Check for iframes and base64_decode entries in the current index.php.
Comment: Thanks... I created a new file and it still redirects. This puzzles me even more! And also, I checked the old index.php and removed all the base64 entries... I actually replaced it with the index.php from the original package of the web application...
Answer: It looks like you may have fixed this?

<META   HTTP-EQUIV="Refresh"
        Content = "1;
        URL=http://www.lunarpages.com"

I loaded this is an telnet session so there was no refresh, but if your IP address is 188.8.131.52, then you are no longer redirecting people to a bad site.  If your IP address is not 220.127.116.11, then your DNS has been hacked.
Running an nslookup from here, I don't see that your nameserver is domaincontrol.com.  Did you let your domain expire?  I have seen a domain get parked when it expires.
On another note, maybe these guys are at it again.  You might call GoDaddy and double check.
Comment: Just tried it from a different computer and it works. Turns out that it was just on this computer... Thanks for the help... Marked as answer because of the link you provided.
Comment: Also, for anyone else looking for a solution, Scott was right... My DNS was hacked someone though I have no clue as to how this could have happened...


Is a homotopically equivalent to a $\Delta$-complex space a $\Delta$-complex?

Question: I'm going through Hatcher's Algebraic Topology, and after reading the paragraphs on simplicial and singular homology, I was wondering about this :

Singular homology is homotopy invariant, that is, two homotopically equivalent topological spaces $X, Y$ have isomorphic homology groups.
Simplicial and singular homology are equivalent, meaning that for a $\Delta$-complex $X$, there are isomorphisms between $H_n^\Delta(X)$ and $H_n(X)$ for all $n$.

Now, would this mean that, given a $\Delta$-complex $X$, any other topological space $Y$ of the same homotopy equivalence class would inherit a $\Delta$-complex structure, and hence have the same simplicial homology groups as $X$?
Comment: That's right. For a specific example, a product of uncountably many copies of $\Bbb{R}$ is contractible (and hence homotopically equivalent to a point) but is not compactly generated (and hence cannot be the underlying space of a CW complex or $\Delta$-complex).
Comment: I don't see how you could use a homotopy equivalence to transfer a $\Delta$-complex structure from one space to another. In fact, I think the results presented in these [slides on products of CW complexes](http://www.math.helsinki.fi/logic/arctic/2017/Slides/BrookeTaylor_arctic2017.pdf), show that this is impossible, because you can construct contractible CW complexes $X$ and $Y$ such that $X \times Y$ is not a CW complex. (CW complexes generalise $\Delta$-complexes, and if $X$ and $Y$ are both contractible then $X$ and $X \times Y$ are homotopy equivalent.)
Comment: PS: I should stress "think" rather than "know for certain" in the above comment as I haven't checked all the details. The results definitely show that the natural CW structure on the product of CW complexes does not deliver the product topology in general. I see also that Hatcher's book covers the relevat result of Dowker in an appendix.
Comment: @RobArthan I'm glad to find out, however, that the doubt I'm having is legitimate. Thank you for your comment.
Comment: I am not sure exactly what it is that you doubt: if spaces $X$ and $Y$ are homotopy equivalent, and if their topologies are induced by a $\Delta$-complex structure, then the simplicial and singular homology groups will all agree.
Comment: Yes, that's clear. The doubt I'm talking about is stated in the title : If I have a topological space $Y$ homotopically equivalent to a $\Delta$-complex $X$, would that immediately imply that $Y$ is a $\Delta$-complex? And if I understand your comment correctly, you're suggesting that for CW-complexes the implication is false.
Answer: Having the same homology and "inheriting" $\Delta$-structure are very vaguely related things.
A simplest counterexample would be a contractible space $Y$ which is not a simplicial/$\Delta$/CW-complex. For example the comb space. Because it is not locally connected.
Homotopy equivalences rarely preserve topological properties.


Acquiring advanced level sight reading

Question: Beyond the basics of sight reading that everyone talks about, what are some of the skills you need to learn to sight read advanced and complex pieces?
What are the nuances and techniques that not a lot of people know, but only people of a higher level of understanding know?
I'm nowhere advanced but I think there's just more than reading ahead. I feel like you can find anchor points or something, or you can take something complex and simplify it in your brain and chunking it for easy reference, but I do not know the details of it and how to acquire the skill.
I want to learn something not just for the talented — like people who have perfect pitch and can do things that normal people cannot, or people with insane memory like photographic memory. I just want to know something that can be practice and learned.
Comment: What are these "basics" that you claim everyone talks about?  It's just like reading a book, you need to read a lot and every day.
Comment: For example everyone just says chunking and gives an example of an easy c major Alberti bass for 4 measures, of course that's easy. But how about more advanced pieces where the left hand jumps quickly and complex chords are being played? I was taught to find anchor points where my middle fingers would jump to and how my fingers prepare for what's to play in that position. I'm sure there are many more things that I haven't even thought about.
Answer: Sight reading is, at core, the process of associating written notation with physical gestures.1 Advanced sight reading relies on quickly recognizing patterns (scales, chords/arpeggios, repeated figures) and associating them with the manner of execution on one's instrument.
There are a variety of ways to develop this:

Read lots and lots and lots of simple music. A) In every piece, we encounter different patterns as well as the same patterns over and over. By encountering and re-encountering these patterns in different contexts, they become ingrained in us such that when seen in a more complex context, they are recognizable and don't get lost in the mix of all the other parts of the music. B) By playing easy music, we can cover more ground (encounter more patterns) faster and more efficiently. C) By playing easy music, we can take in all of the musical instructions (dynamics, articulations, musical form, etc.), not just the notes.

Practice scales, chords, arpeggios, and common chord progressions. This, too, will help recognize common musical elements when seen in a piece and prepare us to execute them efficiently.

Practice rhythm. One important way to see patterns and groups of notes is to recognize common rhythms. This can be practiced separately, allowing for learning much more complex rhythms that one could if also trying to play the notes at the same time. There are many books of rhythm exercises as well as apps and websites.

Study music theory. For me personally, this proved essential. Once I could look at a piece and see the harmony and the structure, my reading took a significant jump.

1 The exception is singing, where sight reading is more about the development of relative pitch. Whereas most instruments require movement in space and interaction with an external object, the physical aspect of singing is entirely internal to our bodies.
Comment: There may be some 'externality' to singing - as we do need to be able to hear what we sing, meaning the sound comes out and back in through ears, allowing the sound to be adjusted if needed. True, there is an internal sound connection through bones, but ears work more effectively.
Comment: I would like to add something to point #3: do not neglect rhythm. When I started, I was very focused on reading notes fast and I learned that without focusing on respecting rhythm very much. This issue stayed with me for decades. Don't make the same mistake.
Answer: As Aaron says, sight reading is "the process of associating written notation with physical gestures".
What that tells us is that the "physical gestures" come first, the sight reading second. So, the first step to advanced sight reading is advanced playing ability. In the classical world, international soloists generally perform without sheet music, from memory alone. Sight reading is an adjunct. Playing the instrument comes first.
At a lower level, candidates for the ABRSM sight reading tests, where the candidate is given a piece of sheet music and asked to perform it, are advised to use some of the minute or so of preparation time to "clap the rhythm" as playing each note for the right amount of time is almost as important as playing the right note. Playing the right notes is usually the easiest part.
The other two key components are 1) playing louder and softer as marked in the sheet music and 2) speeding up and slowing down as required.
Comment: Remember, you can sight-read a piece only *once* in your lifetime; after that first time you are practicing it. One ability you'll develop is looking just ahead as you're playing. When you finish a piece, go back and analyze the measures that gave you difficulty. Make exercises out of these. If you identify a technique that is hard, look for an etude that features it.

Acquiring advanced level sight reading


Is it legitimate to use music technology to make me sound more proficient than I am?

Question: Music technology lets a single musician do things that are physically impossible for one person without the technology. For instance one musician can create a multitrack recording where they play every instrument and sing every line in a song, and then create the final mix.
As a trumpet player it would be easy to process my performance to make me sound like I'm playing an octave higher than I actually can.
I'm comfortable with the first scenario, but the second feels like cheating somehow. Is this a legitimate use of technology? Where do you draw the line?
Comment: I think you have to determine where you draw your line. No one else can decide what is "legitimate" for you.
Comment: all's fair in love and war.  music is love.  therefore, all's fair in music.
Comment: It's cheating when you do it in a job interview tape - it's not cheating when you do it on an experimental album. It's iffy whether it's cheating if you do it in a live performance and don't tell people. (Fun anecdote: when Leonidas Kavakos applied to study with Josef Gingold, he sent a tape of himself playing Paganini's A minor caprice. Gingold was so suspicious that he immediately ordered Kavakos to play that piece live for the audience, thinking the speed must have been manipulated. It wasn't.)
Comment: Wait, that trumpet octave thing sounds kind of cool! It would realistically be used as an *effect*, and in the right context be absolutely fine. I think Neurosis once did a song where they altered a trumpet recording (octave down?) to make it sound like a foghorn or something...
Comment: the photoshop artists community went over that debate 2 decades ago: was it cheating to draw on top of a photo? was it cheating to draw things by hand, then scanning them and redraw on top with the computer? etc. In music, besides other musicians, no one cares how you did it. The real question is: does it sound good?
Answer: Other than plagiarism, if you're creating your own music, "cheating" really doesn't matter.
Sure.. I think effects, like pitch & rhythm correction, are all bad substitutes for being a great musician.  But there aren't any "music cops" making sure you don't use those tools.
One exception is if you're recording for an audition. Then any post-processing effects are a definite no-no.
Answer: It really comes down to what genre of music you write and who you're intended audience is. In something like electronic music your audience actually expects you to edit sound beyond the means of acoustic production, while if you were writing a jazz album most listeners would be turned off by an octave shifted trumpet. This is doubly true if you intend to perform live. You could theoretically mic and pitch shift your trumpet as you play it, which is what electric guitar and bass guitar players often do with pedals. If you write good music that utilizes such a special sound it could even be accepted in jazz, but it will probably be harder to get a large audience.
As far as it being "cheating", that only applies if it's your intent to deceive your audience into believing you have more talent in performance than you truly have. Like it or not, people are impressed by a good, technical display of virtuosity on any instrument. It shows dedication and generally sounds better than extremely simple lines. So if you build up your talent on an album as if you can play such a thing but in reality can't, many people will be disappointed.
Alternately, using electronic editing to produce a technique that is possible to play with normal playing comes off as very lazy, as if the player simply couldn't be assed to learn how to actually play. It comes off as an unnecessary crutch.
Answer: Define legitimate. Essentially you're utilizing two different skills and combining them to produce an end result.
Assuming that you're doing the work yourself, what you're doing is using your musical talent on an instrument or your voice to play out a melody, then using your talent on audio manipulation software to tweak the sound until it is what you want.
If what you're trying to describe is the situation in which you feel that your skill on your instrument or the range in, or the strength of your voice is insufficient, and as a result you're compensating by means of a device, then yes it's inauthentic and not legitimate in terms of single skill. This won't do if you're attempting to engage in a skill-based endeavor such as an audition to an ensemble.
But if your end goal is to create music, especially the kind that is distributed in a digital format for listening, however you achieve your desired result as a composer is completely fair.
Answer: Of course it's OK!  Why on earth wouldn't it be?  This isn't like not telling your husband (or wife) -to-be that you wear a wig!   The end product is all that matters.
Of course, it would be foolish to apply for a session musician job based on a doctored demo tape.
Answer: Why should there be a line?  Some stuff works bad in live contexts, like multitracking.  Queen had its greatest hit with "Bohemian Rhapsody", unfortunately a studio album.  Still in great demand for live concerts, so they reverted to a mixture of starting with a recording on an empty stage and the musicians coming in for a somewhat modified finale.
Frank Farian did not even go to the trouble of live performances but rather hired playback goons for his studio produced music, like "Boney M" and "Milli Vanilli".  What a minor and major scandal when people found out.
Cover bands don't write their own music.  Plugged bands do things unplugged bands can't.  Violins sing higher than their players.  And so on.
In the end there is a product.  Spicing up bad ingredients only works to a minor degree.  Fixing stuff after the fact using technology rarely leads to good results.  Creating stuff in a process involving technology can be quite interesting.
Answer: Having working in the recording industry, I have some strong feelings on this one.
Fist I want to say that there is plenty of creative stuff that can be accomplished with the aid of technology that cannot be accomplished without it. In this way i think of it as a creative tool and to some degrees it can be viewed as another instrument. Example of this being done well can be found in bands like Radiohead, the Beatles, as well as the Queen song "Bohemian Rhapsody" (this song was made using multi track recording techniques that I would consider "technology". Even though this song could have been recorded with a choir instead they performed a choir worth of voices by overdubbing the four voices over and over again).
In the OP's question he uses shifting the trumpet up an octave artificially and this reminds me of a guitar effect that is used all the time.
But i think at the heart of the matter is the "making myself sound more proficient than I am" and to this piece I say that you have to do what you feel comfortable with. Let it be known that every recording I worked on (and I am talking about major top 40 bands and artists) had some degree of assistance from technology to make them sound more proficient than the recording would have been otherwise. If this meant editing drums to the grid, tuning vocals, recording one note at a time (yes this happened), cutting multiple takes together, editing out breathes, and everything in between I saw it all happen. It is actually one of the reasons I left the recording industry (there were others, but this really bothered me) and I think there are still artists that don't do this (or at least claim to not do it) such as Jack White (formally of the white stripes) but if you feel comfortable editing yourself to sound better than it is fine and actually very common.
My biggest problem with this is not the aspect of cheating though, it is the fact that I feel it removes the elements of human performance that draws me to enjoy a piece of music. I feel it is the slight imperfections that make a player sound like a player, whether it is Ringo's slightly behind the beat feel or McCartney's slightly flat pitch, if you remove those elements you are removing the essence of that player. If however you are using it as a crutch to help you sound as good as you want to sound, perhaps you can use it as a tool to help you improve to the point of actually being able to perform that way. record yourself, edit it to sound the way you like, practice along with that recording and then record yourself again to see the improvement.
If you have a creative idea that needs to use technology to create, then by all means go for it.
just my opinion. just make good music.  ;)


Custom 404 error page issue

Question: I want to have a custom 404 page that displays some additional information. I need to display the requested path that was not found and some pages suggestions based on that path (this is based on some internal logic that I don't think is relevant to my question).
Here is what I did:
I defined the default 404 page to be page-not-found in admin/config/system/site-information
Then in my module I used hook_page_alter() to define my pages suggestions (if drupal_get_http_headers("status") is 404 Not Found), and the request_uri() to be shown on the 404 template page I created.
I have tried to save these values in different ways (session variable, ctools cache, everything I have tried leads to the same issue — see below).
In mymodule.module:
/**
 * Implement hook_menu
 * @return mixed
 */
function mymodule_menu() {
  $items['page-not-found'] = array(
    'title' => '',
    'page callback' => 'mymodule_page_not_found',
    'file' => 'inc/functions.inc',
    'access callback' => TRUE,
  );
  return $items;
}

/**
 * Implement hook_theme
 */
function mymodule_theme() {
  return array(
    'mymodule_page_not_found' => array(
      'template' => 'templates/404',
    ),
  );
}

In inc/functions.inc:
function mymodule_page_not_found() {

  return theme('mymodule_page_not_found');
}

Problem is, if I request a non existing URL for example mysite.com/not-found-page-1 then request mysite.com/not-found-page-2 the information displayed on the 404 page template when I query the not-found-page-2 corresponds to the values I saved when requesting not-found-page-1.
Any idea why I always get the data of the previous query?
Comment: Yes and no. But for now, no. I'd first like to understand why my code behaves this way.
Answer: The trouble is, hook_page_alter() is called after your theme function is called.  So if you're setting things there it's too late.
I don't know why you're doing it this way.  Why not generate your suggestions in your page callback: mymodule_page_not_found().  Then pass them to the theme function.
Something like this:
/**
 * Implement hook_theme
 */
function mymodule_theme() {
  return array(
    'mymodule_page_not_found' => array(
      'template' => 'templates/404', // Not relevant to your question, but you should consider renaming this to mymodule-page-not-found because of a "bug" in D7; see discussion here: https://www.drupal.org/node/342350
      'variables' => array('suggestions' => array()),
    ),
  );
}

function mymodule_page_not_found() {
  // ... generate list of $suggestions ...
  return theme('mymodule_page_not_found', array('suggestions' => $suggestions));
}

Comment: It's the more proper way to do it as it offers a form of documentation for folks who want to know what the template expects.  More practically, it provides a way for you to pass in default values in case anyone ever calls your theme function/template without providing a list of suggestions.
Comment: Yes, that's also what I thought regarding the hook_page_alter. Will try your suggestion.
Comment: Thanks a lot! That is way easier and works just fine! :) And thanks for the additional information too.
Comment: Quick question: I didn't add `'variables' => array('suggestions' => array())` to my code and it works. Why should I add that part?

Custom 404 error page issue


Update a record in custom module

Question: I've read several custom module tutorials but none seem to show how to edit a record. My module does not use node but has a custom table. So, I think I ought to be able to drupal_get_form and populate the #value with the fields. But to set up the menu to get the parameter in the URL and my page callback is not working.
Looking around I found this answer and modeled the below after it but I get a default page instead of my expected form.
$items['redashboard/project/%/edit'] = array(
  'title' => (t('Edit project')),
  'page callback' => 'drupal_get_form',
  'page arguments' => array('re_dashboard_edit_project_form', 2),
  'access callback' => TRUE,
);

function re_dashboard_edit_project_form($form, $form_state, $projectid) {
  $project = pull_project($projectid);
  $form['name'] = array(
    '#type' => 'textfield',
    '#title' => t('Research Project Name/Question'),
    '#value' => $project->projectName,
    '#required' => TRUE,
  );
  $form['submit'] = array(
    '#type' => 'submit',
    '#value' => 'Create',
  );
  return $form;
}

function pull_project($projectid) {
  $query = db_select('Projects', 'p');
  $query->fields('p', array('ProjectID', 'residentID', 'projectName', 'projectMentor', 'projectStatus', 'citation', 'lastUpdated'));
  $query->condition('ProjectID', $projectid);
  $result = $query->execute();
  return $result->fetch();
}

So, how should I set up a page to edit this record if not this way? Or to the point how should I set up the menu so that this callback gets rendered? thx, sam
Answer: The $items['redashboard/project/%/edit'] declaration needs to be made inside of re_dashboard_menu(), and there needs to be a re_dashboard_edit_project_form_submit() function to pipe $form_state['values'] into db_update().
Other than that, it looks like it will get the job done.  You could use db_query() instead of db_select(), since it's faster, and the query conditions are static.  Also, make sure you have a way to handle db_select() not returning any results, since fetch() will return FALSE in that case.
Comment: Yes, the menu() and re_dashboard_edit_project_form_submit() was left off for clarity. But you are right and will check for empty in my submit handler. But I'm still not getting to this page, even after drush cc all
Comment: Another day, a fresh restart seems to have shaken its cobwebs. Thanks for validating the page args.

Update a record in custom module


How to convert inline SQL queries to JOINS in SQL SERVER to reduce load time

Question: I need help in optimizing this SQL query.
In the main SELECT statement there are three columns which is dependent on the outer query result. This is why my query is taking a long time to return data. I have tried making left joins but this is not working properly.
Can anyone help me to resolve this issue?
SELECT
  DISTINCT ou.OrganizationUserID AS StudentID,
  ou.FirstName,
  ou.LastName,
  (
    SELECT
      STRING_AGG(
        (ug.UG_Name),
        ','
      )
    FROM
      Groups ug
      INNER JOIN ApplicantUserGroup augm ON augm.AUGM_UserGroupID = ug.UG_ID
    WHERE
      augm.AUGM_OrganizationUserID = ou.OrganizationUserID
      AND ug.UG_IsDeleted = 0
      AND augm.AUGM_IsDeleted = 0
  ) AS UserGroups,
  order1.OrderNumber AS OrderId -- UAT-2455
  ,
  (
    SELECT
      STRING_AGG(
        (CActe.CustomAttribute),
        ','
      )
    FROM
      CustomAttributeCte CActe
    WHERE
      CActe.HierarchyNodeID = dpm.DPM_ID
      AND CActe.OrganizationUserID = ps.OrganizationUserID
  ) AS CustomAttributes -- UAT-2455
  ,
  (
    SELECT
      STRING_AGG(
        (CActe.CustomAttributeID),
        ','
      )
    FROM
      CustomAttributeCte CActe
    WHERE
      CActe.HierarchyNodeID = dpm.DPM_ID
      AND CActe.OrganizationUserID = ps.OrganizationUserID
  ) AS CustomAttributeID
FROM
  ApplicantData acd WITH (NOLOCK)
  INNER JOIN ClientPackage ps WITH (NOLOCK) ON acd.ClientSubscriptionID = ps.ClientSubscriptionID
  INNER JOIN [ClientOrder] order1 WITH (NOLOCK) ON order1.OrderID = ps.OrderID
  AND order1.IsDeleted = 0
  INNER JOIN OUser ou WITH (NOLOCK) ON ou.OrganizationUserID = ps.OrganizationUserID

Comment: For performance help, you *must* include table *and* index definitions, as well as share a query plan (you can do this via https://brentozar.com/pastetheplan). This is not answerable without that information. Why do you have `NOLOCK`, do you know what it does, do you have blocking problems you are trying to solve? It is not a go-faster switch, it is a give-incorrect-results switch, and can cause more problems than it solves
Comment: "mysql" ***or*** "SQL Server"? These are entirely different databases. Update the incorrect tag/text.
Comment: @user2864740, Question is basically related to SQL SERVER
Comment: Then *remove* "mysql" from the tags.
Comment: You can do the second and third `STRING_AGG` inside a single `APPLY`, to avoid querying that table twice. Why is `DISTINCT` there, I suspect that is probably your main issue, do you have duplicates you are trying to remove? If so why are there duplicates, which join is causing them?
Comment: Charlieface mentioned this: to help you with your [tag:query-optimization] question, we need more information. Please [read this](https://stackoverflow.com/tags/query-optimization/info), then [edit] your question.
Answer: It looks like this query can be simplified, and the dependent subqueries in your SELECT clause removed, Consider your second and third dependent subqueries. You can refactor them into one nondependent subquery with a LEFT JOIN. Using nondependent subqueries is more efficient because the query planner can run them just once, rather than once for each row.
You want two STRING_AGG() results from the same table. This subquery gives those two outputs for every possible combination of HierarchyNodeID and  OrganizationUserID values. STRING_AGG() is an aggregate function like SUM() and so works nicely with GROUP BY.
      SELECT HierarchyNodeID, OrganizationUserID,
             STRING_AGG((CActe.CustomAttribute), ',') CustomAttributes -- UAT-2455,
             STRING_AGG((CActe.CustomAttributeID), ',') CustomAttributeIDs -- UAT-2455
        FROM CustomAttributeCte CActe
       GROUP BY HierarchyNodeID, OrganizationUserID

You can run this subquery itself to convince yourself it works.
Now, we can LEFT JOIN that into your query.  Like this. (For readability I took out the NOLOCKs and used JOIN: it means the same thing as INNER JOIN.)
SELECT DISTINCT
       ou.OrganizationUserID AS StudentID,
       ou.FirstName,
       ou.LastName,
       'tempvalue' AS UserGroups,  -- shortened for testing
       order1.OrderNumber AS OrderId, -- UAT-2455
       uat2455.CustomAttributes,      -- UAT-2455
       uat2455.CustomAttributeIDs     -- UAT-2455
  FROM ApplicantData acd
  JOIN ClientPackage ps
               ON acd.ClientSubscriptionID = ps.ClientSubscriptionID
  JOIN ClientOrder order1
               ON order1.OrderID = ps.OrderID
              AND order1.IsDeleted = 0
  JOIN OUser ou
               ON ou.OrganizationUserID = ps.OrganizationUserID
  LEFT JOIN (
      SELECT HierarchyNodeID, OrganizationUserID,
             STRING_AGG((CActe.CustomAttribute), ',') CustomAttributes -- UAT-2455,
             STRING_AGG((CActe.CustomAttributeID), ',') CustomAttributeIDs -- UAT-2455
        FROM CustomAttributeCte CActe
       GROUP BY HierarchyNodeID, OrganizationUserID
     ) uat2455
               ON uat2455.HierarchyNodeID = dpm.DPM_ID
              AND uat2455.OrganizationUserId = ps.OrganizationUserID

See how we collapsed your second and third dependent subqueries to just one, then used it as a virtual table with LEFT JOIN? We transformed the WHERE clauses from the dependent subqueries into an ON clause.
You can test this: run it with TOP(50) and eyeball the results.
When you're happy, the next step is to transform your first dependent subquery the same way.
Pro tip Don't use WITH (NOLOCK), ever, unless a database administration expert tells you to after looking at your specific query.  If your query's purpose is a historical report and you don't care whether the most recent transactions in your database are represented exactly right, you can precede your query with this statement. It also allows the query to run while avoiding locks.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;

Pro tip Be obsessive about formatting your queries for readability. You, your colleagues, and yourself a year from now must be able to read and reason about queries like this.


update a module in the front page after adding new content into another page

Question: I want to create a block on the home page which highlights the latest news from the 'news' page. I would like to add new content into the 'news' page frequently. Every time when I add a piece of new article (news), I would like the block to show the image and the title of that new article automatically.
What is the best way to achieve this?
Thanks
Answer: Sounds like you should be using Views.

Create a new view
Create a page display and configure it to output the content you want.
Create a block display and configure it to output the content you want.
Add the block to your homepage

Now when you add new news it will show up on both the page and the block.


Singing along to guitar chords (harmony)

Question: Recently I started writing songs and I started to wonder:
Let's say I sing middle C (while playing open C chord on guitar) for a bar and then go to singing E, third higher in pitch. Should I follow that by a chord that also sounds higher on guitar to match the E note I sing? I'm a little confused because when I play E minor chord it sounds LOWER than C chord and especially lower than E note I'm singing now.
Would it be better to match singing note with a chord that also sounds as high (or as low) as this note or maybe I should follow the structure of harmony with my singing (going higher and lower as chords sound higher and lower)? Or, third option, maybe it'd be better to just ignore the fact that E minor sounds lower than C and go along with singing E higher in pitch?
That's the problem in short.
singing 1 bar: C -> E (higher in pitch)
playing 1 bar: C -> E (lower in pitch)
Isn't it weird that even though harmony went down in pitch I sing higher in pitch at the same time?
Answer:
Should I follow that by a chord that also sounds higher on guitar to match the E note I sing?

Not necessarily, no. It might sound more interesting for the two parts to move in opposite directions.

I'm a little confused because when I play E minor chord it sounds LOWER than C chord and especially lower than E note I'm singing now

The Lowest E in the most common E minor chord shape is E2. This is lower than the lowest C in the most common C chord shape, which is C3. So this is as expected.

Would it be better to match singing note with a chord that also sounds as high (or as low) as this note or maybe I should follow the structure of harmony with my singing (going higher and lower as chords sound higher and lower)? Or, third option, maybe it'd be better to just ignore the fact that E minor sounds lower than C and go along with singing E higher in pitch?

It sounds like you're trying to play a new chord with every new note you sing. This isn't usual in most styles of music.
I would recommend learning some existing songs (learning the correct melody and the correct chord progression) in the style you are interested in, and then start to learn something about the music theory for that style of music.
For many styles of music, it's common to have a chord progression that mostly fits into one key, and a melody that also mostly fits that same key - but for the chords and the melody to move quite differently, and to change at different times.
Answer: I think it's due to the voicings you use with the guitar chords. You're probably using open C followed by open Em. A couple of notes (both C) happen to drop by a semitone going to Em, so yes, it will sound lower. If you were to play different chord shapes for each, you could feel that the opposite effect is happening.
In answer to the question, when you sing a C note, obviously, a C chord will sound good under it. But - that's not the only chord that might fit. Basically, any chord with a c note will also be a contender. F, Fm, Am, D7, Co are a few that spring to mind. There's also the idea that the note's hanged, so the chord has to too. Not true. The note may be E. So now, the best fit chords could be Em, Am, Fmaj7, D9, or even stay on that humble C, as it contains the sung E note. It will also be dependent on what else is in that second bar. That's where the clues lie as to what chord fits best. say there's another C, that narrows the choce down to maybe C or Am. If it's anything else, try other chords (starting with those diatonic) and hear which is the best fit.
Comment: Thank you for the answer. Let's say I want (for some reason) to higher in pitch in that movement from C to E in singing. Beside staying on the C in chords, what other chord would fit? Barre E minor chord on 9 fret comes to my mind and E minor chord played on four strings (high E - 3 fret, B - 5 fret, G - 4 fret and D - 5 fret).
Answer: On guitar, the simplest chord shapes for E or E minor use the open bottom string - E - as the root.   To get a C root, we can only go higher.   That is why, using the 'easiest' chord shapes, the C chord may feel higher than the E chord.
(Other shapes and positions are available.  It's perfectly possible to play a C chord in a low position then shift up the neck to play an E chord that sounds higher in all respects.)
If a song starts on the note C and the chord C major, and the next note is E, you COULD move to an Em chord.  But you could also stay on the C chord.  The notes of the C major chord are C, E, G.  E fits in!
It sounds like you're taking a 'Theory-out' approach to writing a song.  I strongly suggest you don't try to work from rules, rather that you study and play lots of existing songs to get a feel of what works.  You can formulate this knowledge into rules later if you like.
Answer: I'm not sure if you want follow the bass line or the back voices or invent a melody:
It depends of the style:
In classic, Jazz and Pop the you may keep on this rules:
bass: you can choose the pitch you like - most comfortable to your voice (priority: the counter-point to the tune).
back-voicing: keep the common tones or go to the next step up or down (preferably counter movement to the melody or bass line.
Gospel songs: parallels (even fifth parallel) are usual creating a typical effect.
In European Folk Music parallels of thirds and sixths are usual, in English and American Folk Songs and Pop Songs (Beatles) parallel fourth with the tune are also usual.
try this out:

Isn't it weird that even though harmony went down in pitch I sing higher in pitch at the same time?

summary:
in almost all styles it is even better to lead the voice in opposition to the bassline respectively the chords.


Rails Httparty JSON to params to save

Question: Rails 4.5 Ruby 2.3.1
I am getting json from an API and trying to store the following into a model CLrates
1. timestamp as unix time (date)
2. Currency_code (string)
3. quote (decimal monetary value)
I can use the following in irb after parsing the json and know how to get the elements individually using: response["quotes"]. How can I generate params to be saved in the model above when the body is as follows:
irb(main):036:0> puts response.body
{
 "success":true,
 "terms":"https:\/\/xxxxx.com\/terms",
 "privacy":"https:\/\/xxxxx.com\/privacy",
 "timestamp":1504817289,
 "source":"USD",
 "quotes":{
   "USDAED":3.672703,
   "USDAFN":68.360001,
   "USDCUC":1,
   "USDCUP":26.5,
   "USDCVE":91.699997,
   "USDCZK":21.718701,
    ............ many more lines removed for brevity
   "USDZWL":322.355011
 }

I can do this using a separate associated model but have very little idea how to create the params to save to a single table.
The following links got me to this point and well worth a read if you need info on httparty GET (client):
1. http://www.rubydoc.info/github/jnunemaker/httparty/HTTParty/
2. http://eric-price.net/blog/rails-api-wrapper/
3. https://www.driftingruby.com/episodes/playing-with-json
The class and method in lib/clayer.rb:
 class clayer
   include HTTParty
   format :json
   read_timeout 10

  def self.get_quotes
    response = HTTParty.get('http://www.nnnnnnn.net/api/live?
    access_key=nnnnnnnnnn&format=1')
  end
end

I used irb as I am still learning how to run this through rails c. This will be called in the controller and saved however need to work out how to get the params from the json
Thanks for the help
OK: after digging I think I am on the right track
I get the response["QUOTES"], loop through them and build the params required saving each at the end of the loop
rates = response["QUOTES"]
rates.each do |k,v|
  clrate = Realtimerates.new
  clrate.date = response["timestamp"]
  clrate.countrycode = "#{k}"
  clrate.price = "#{v}"
  clrate.save
end

Going to give this a whirl
In model
class Realtimerate < ActiveRecord::Base

 include HTTParty
 format :json
 read_timeout 5

def self.get_cl_rates
    response = HTTParty.get('http://www.mmmmm.net/api/live?access_key="key"&format=1')
    rates = response["quotes"]
    rates.each do |k,v|
        create!(
        date: Time.at(response["timestamp"]),
        country_code: "#{k}",
        price: "#{v}")

    end

 end

end

In the controller:
def index
  Realtimerate.get_cl_rates
  @realtimerates = Realtimerate.all
end

This is working and shows latest GET.
Answer: You already have a hash in your response.body. All you need to do now is to assign the relevant key-value to your model's attributes. e.g.
clrate = ClRate.new
res = response.body
clate.time = res["timestamp"]
...
clate.save

Comment: Thanks for the response (punny isnt it). So I need a model of the keys: USDAED, USDAFN, USDCUC, USDCUP...Then loop through each element and build the params hash. Is there a way to do it without knowing the key and adding an extra model? I understand how to access them as follows and will use your method posted to construct: response["quotes"]["USDAED"]. This does give me the price: is there a way to reference the key symbolically?
Comment: Hi your latest attempt is on the right track but will only capture the last currency code, value pair. One solution is to save each pair in one object, otherwise you need one field for each currency code.
Comment: thanks a stack for the help. I have got it working via the index action so it will show the latest GET: quite funky: lovn it. Would like to hear your comments on the solution. Especially the object approach! Probably a lot better than this. Also: moved the method into the model and called it from the controller Realtimerates. Please see edits
Comment: Glad you got it to work. But I don't think you need to include HTTParty if you only need to call its class method. In fact it's probably better to not mix it in here, because you'll end up with a messy interface.

Rails Httparty JSON to params to save


Best practice in HTML? How to delete table entry by submit?

Question: I would like to implement a table with a button in each row to delete that row.
Now I'm asking myself what is the best way to do that.
By the way I'm using JSPs with Spring tags but I think this is a common html question.
In my opinion I have two options: Using links or using buttons with forms
Link example (Delete users):

        Delete Button
        User Id
        Name

    <c:forEach var="user" items="users">

                <s:url value="/users/${user.userId}/delete" var="deleteUrl" />
                <icon:delete/>

            ${user.userId}
            ${user.name}

    </c:forEach>

Yes, this should work, but the URL to delete an entry is exposed directly in the browser. Not very save...
Button with from example (Delete users):

            Delete Button
            User Id
            Name

        <c:forEach var="user" items="users">

                    <form:form action="/users/${user.userId}/delete" method="POST">
                        <button type="submit"><icon:delete/></button>
                    </form:form>

                ${user.userId}
                ${user.name}

        </c:forEach>

This should work as well, but you have one form tag per button. And the URL to delete an entry is maybe exposed after submit in the browser url bar.
Not the best way too...
So, what do you think is the best way? Or are there any other which are maybe better?
Comment: And exposing the URL is an isuse because? It is exposed nonetheless, even if you use AJAX it is fairly simple to retrieve the URL. What you want is the form with the URL, and you might want to use a DELETE in a hidden field (s o you can use a DELETE method instead of POST). And finally you want to add a CSRF token so only authorized and authenticated users can use the delete functionality.
Answer: Never use a GET request (i.e. a link) to delete data on the server. I remember a famous story (not sure if it was a HOAX or not) that overnight all data was deleted, the reason being that Google indexed the site and followed all the links...
You want to use the form way and probably want to use a DELETE request, in spring you can can configure the HiddenHttpMethodFilter, then when adding a hidden field named _method with the value DELETE.
Next you want to add a CSRF token to your form to make sure only authenticated (and authorized!) users can use a delete. Assuming you are using Spring Security (or going to add) your final form should look something like this.
<form:form action="/users/${user.userId}" method="POST">
    <input type="hidden" name="_method" value="DELETE" />
    <input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}"/>
    <button type="submit"><icon:delete/></button>
</form:form>

In your controller you should have an @DeleteMapping for the URL.
@DeleteMapping("/users/{id}")
public void remove(@PathVariable("id") Long uid) { ... }


How can I efficiently store pseudo queries to the database inside their own table, and how to correctly store items with varying attributes?

Question: First, I wish to apologize if the title is vague, the question is hard to summarize - please feel free to edit as appropriate.
In a website I am building, I have products within categories (and the categories are being expanded constantly) that have attributes (I can't think of a better word, I apologize for the fact that this crosses with database terminology, but I mean attribute as in color, size, weight, not as in a field of a database table).  Those attributes may or may not be defined depending on the category, and sometimes specifically the product.
Now, individuals using the website will want to create a query that searches for items matching specific attributes (think "is green, weighs less than 4 lbs., rated 3 stars or higher"), and save that query in the database as a tuple of a table that they can reference.  Mind that these will be end-users, so they will not be able to actually write the query themselves.  Also note that there may be a great deal of redundancy in some types of queries, and that there will be many more of these queries than there are items.

My first thought:
Store the items all in one table (items), with an id, category id, name.  Store attributes types in attribute_types with id, name, type (which would be int, string, enum, etc).  Store attributes in a separate table with item_id, attribute_type_id, and value.
Then, create the query based on HTML selections as though I were actually querying the database, save the exact query, and store it as a string in a table with an id and query.  Check for the existence of a query before storing it.

This seems as though it would work, but I can't immagine that this is the best way to go about it.  Is there a more efficient, or frankly less ugly, solution?  Am I even storing the items in an appropriate manner?
Thank you,
James
Comment: i have done it in the past, it has its pro's and con's like everything else
Answer: If the number of specific attributes and categories is small enough, this is usually solved using Concrete Table Inheritance (or some other derivation of it), if not you need an Entity-Attribute-Value model - which is difficult to do complex queries with and, in the long run, can lead to GIGO pretty fast.

Another alternative approach is to use schemaless datastores, such as MongoDB or serialized blobs.


Why I am not able to add custom validate Handler function for UserPasswordResetForm on form_alter?

Question: I am trying to override default reset password functionality by using hook_form_FORM_ID_alter as below:
The below code is written in custom module called usr_registration.module file
<code>    /**
     * Implements hook_form_FORM_ID_alter().
     */
    function usr_registration_form_user_pass_reset_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id, AccountInterface $user = NULL, $expiration_date = NULL, $timestamp = NULL, $hash = NULL) {

            $form['account']['pass'] = [
            '#type' => 'password_confirm',
            '#size' => 25,
            '#description' => t('Provide a password for the new account in both fields.'),
            '#required' => TRUE,];

             $form['#validate'][] = 'usr_registration_validate';
             $form['actions']['submit']['#submit'][] = 'usr_registration_form_user_pass_reset_custom_submit';
    }

    // this function itself is not getting called
    function usr_registration_validate(&$form, FormStateInterface $form_state) {

      //unable to see below debug message and form is getting submitted
       echo "I am here"; exit;

       if($never_login) {
                        //some custom logic
       } else {
                       //some custom logic
       }

     }
</code>
The important thing I noticed in UserPasswordResetForm class(user core module's form) is that there is no default validate method defined.
I am unable to find out that why I cant alter the UserPasswordResetForm and custom validate handler over here.
note: My question is similar to this but not exactly same so please dont mark it as redundant.
Answer: $form['#validate'] will not work as that form
"This form works by submitting the hash and timestamp to the user.reset"
aka redirects you to the user.reset from
what you can do is something like temp replace this form with a custom one though. however i don't really see the point
<code>/**
 * Implements hook_form_alter().
 */
function yourmodule_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  if ($form_id === 'user_pass_reset') {
    $action = $form['#action'];
    // Set the original action.
    $form_state->set('original_action', $action);
    // temp replace from with custom one.
    $form = \Drupal::formBuilder()->buildForm('\Drupal\yourmodule\Form\AlterTheLoginResetFrom', $form_state);
    return $form;
  }
}
</code>
then do
<code>class AlterTheLoginResetFrom extends FormBase {

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'alter_the_login_reset_from';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state) {
    // Get the stuff you need from ...
    dump($form_state->get('original_action'));
    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    foreach ($form_state->getValues() as $key => $value) {
      // @TODO: Validate fields.
    }
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    // redirect to original as uses get
  }
}
</code>
or alter the user.reset directly
see Drupal\user\Form\UserPasswordResetForm; line 60 & 71
and see how core does it (resetPassLogin)
\Drupal\user\Controller\UserController::resetPassLogin
if all else fails perhaps just validate the passwords in js.


Fazer um filtro de negação no ng-repeat

Question: Tenho um objeto e estou lendo ele no ng-repeat, só que se a chave id for igual a 0777 não quero que seja exibido essa posição,

                {{account.name}} - {{account.accountId}}

estou fazendo o fillter normalmente, exibe somente o objeto, mas tentando usar de forma de negação, não exibe todos as posições do objeto.

                {{account.name}} - {{account.accountId}}

Se alguém já conseguiu fazer a filtragem com negação dessa forma e puder compartilhar, obrigado.
Answer: Para que o filtro seja de negação a ! precisa estar dentro da string passada para o filtro.
Ao invés de:
!'00777'

Deve ser:
'!00777'

Caso você tenha uma variável ou função e queira negar o resultado da expressão, é necessário concatenar uma ! com o item na declaração:
filter:{accountId: '!'+algumaFuncao}

Um exemplo:

angular.module('myApp', [])
.controller('TesteCtrl', ['$scope', function($scope) {
    $scope.negateAccountId = function(item) {
        return item.accountId != '00777';
    };

    $scope.accounts =
        [
            {
                name: "Teste 1",
                accountId: '00111'
            },
            {
                name: "Teste 2",
                accountId: '00777'
            },
            {
                name: "Teste 3",
                accountId: '00444'
            },

        ]
    ;
}]);

                    {{account.name}} - {{account.accountId}}

Fazer um filtro de negação no ng-repeat