PHP Custom Form Render warning

Question: I'm receiveing the warning
Strict warning: Only variables should be passed by reference in eval() (line 2 of D:\wwwroot\tixers\modules\php\php.module(80) : eval()'d code).
Everywhere I've looked tells me its a render error but I can't find anything wrong.  The form works and submits all my info to the database, but there warning still shows up.
If you need the code for the module let me know and I will reply to it.
THANKS!
I use
<?php
print drupal_render(drupal_get_form('sellTickets_form'));
?>

then in the form 'sellTickets_form
function sellTickets_form($form, &$form_state) {

$form['team'] = array(
    '#type' => 'textfield',
    '#title' => t('Team Name:'),
    '#size' => 30,
    '#required' => TRUE,
);

......break for forms.....
$form['total_points'] = array(
    '#type' => 'textfield',
    '#title' => t('Total Points Awareded'),
    '#size' => 30,
    '#required' => TRUE,
);
$form['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Submit'),
);
return $form;

}
function sellTickets_form_validate($form, &$form_state){
}

function sellTickets_form_submit($form, &$form_state) {
    global $user;
    $uid = $user->uid;
    $team = $form_state['values']['team'];
    $price = $form_state['values']['price'];
    $section = $form_state['values']['section'];
    $row = $form_state['values']['row'];
    $seat = $form_state['values']['seat'];
    $quantity = $form_state['values']['quantity'];
    $opponent = $form_state['values']['opponent'];
    $date = $form_state['values']['date'];
    $location = $form_state['values']['location'];
    $description = $form_state['values']['description'];
    $total_points = $form_state['values']['total_points'];
    $organization = $form_state['values']['organization'];

    db_query("INSERT INTO ticket (
    `TEID`, `price`, `location`, `date`, `organization`, `section`, `row`, `seat`, `quantity`, `opponent`, `description`, `points`)

    VALUES ('".$team."', '".$price."', '".$location."', '".$date."', '".$organization."','".$section."', '".$row."', '".$seat."', '".$quantity."', '".$opponent."', '".$description."', '".$total_points."');"
);
}

Comment: Putting the code in would help, but if you read and understand the answer to [this question](http://drupal.stackexchange.com/questions/6322/strict-warning-only-variables-should-be-passed-by-reference-in-include) you should be able to find the error quite easily
Comment: I've read that question and I'm not referencing a variable.  I am not using node_show or a var.  I use
Comment: If you can edit your code into the question I'm sure it'll be simple enough to spot...
Answer: drupal_render() takes it's first argument by reference. This'll fix it:
$the_form = drupal_get_form('sellTickets_form');
print drupal_render($the_form);

Comment: Works perfect.  I guess to many shortcuts.  thanks!

PHP Custom Form Render warning


Mean and Median in a Classic River Crossing Problem

Question: Consider the following classic problem:
Four people on the west side of a river wish to use their single boat to get to the east side of a river. Each boat ride can hold at most two people, and the time it takes to get across will be the time preferred by the slower occupant. The time preferences are: $1, 2, 5$ and $10$ minutes. What is the minimal amount of time in which you can get all four people across the river, where an eastbound trip must have two occupants and a westbound trip must have one occupant?
Answer: $17$ minutes. (Though many mistakenly believe it is $19$ minutes.)
Question 1: If you look at all possible ways of ferrying these four people across the river, subject to the above constraints, what would the average (mean and median) times be?
Question 2: What if you replace the time preferences with $x_1, x_2, x_3$ and $x_4$ minutes?
Question 3: What if you have time preferences $x_1, \ldots, x_n$ minutes for $n$ people, respectively?
Probably the easiest way to broach Question 1 would be to write a quick program to compute the answer, and perhaps doing this for several different time preferences would give some insight into the mean and median in the general four person case. I'm not quite sure how I would start thinking about the general $n$ person case; perhaps by solving it for $n = 1, 2, 3, 4$.
Answers (even partial ones) to any or all of my questions would be greatly appreciated!
Comment: The mean should be calculable without going through case by case. You have $108$ total "journeys" each journey involves three eastbound trips occupied by two people. This gives a total of $324$ eastbound trips. By symmetry, we expect each pair of passengers to have the same number of trips as a whole, so each pair takes $324/6 = 54$ trips together. Since the slow ones dominate, we have a total time of $3\times 10 \times 54 + 2\times 5 \times 54 + 2 \times 54$ for eastbound trips over all possible journeys. The same analysis can be done for westbound. The average comes out to be $30$ minutes.
Comment: Even though it is not the question, I want to point out [this interesting article](http://www.inf.fu-berlin.de/inst/ag-ti/uploads/tx_tipublications/Crossing_the_bridge_at_night.pdf) how to find the solution for the generalized problem.
Comment: 6 ways to choose the 1st eastbound, 2 ways the 1st westbound, 3 ways the 2nd eastbound, 3 ways the 2nd westbound; $6\times2\times3\times3=108$.
Comment: I think the article linked by @A.Schulz can be found [here](http://page.mi.fu-berlin.de/rote/Papers/pdf/Crossing+the+bridge+at+night.pdf) now.
Comment: I count 108 protocols for $n=4$, so it shouldn't be too hard to find answers for that case.
Answer: There are $N!(N-1)!^2/2^{N-1}$ different schedules. The average for the generalized problem has already been calculated in carlop's answer. I suspect that calculating the median for the generalized problem in closed form is intractable. Here's code that checks the average of $30$ and determines the median. The time taken for the schedule just below the middle is $27$, the time taken for the schedule just above the middle is $30$. According to the usual definition of the median of an even number of numbers as the mean of the two central ones, the median is therefore $28\frac12$.
Answer: Using @EuYu idea that fixing a single trip (eg. the third trip) each subset of the right cardinality have the same probability, you can write the average traversing time as:
$$E[ \text{time} ] = E[ \text{one person trip} ] * N[ \text{one person trip} ] + E[ \text{two person trip} ] * N[ \text{one person trip} ]$$
and you know that:
$$N[ \text{one person trip} ]=N-2 \\
E[ \text{one person trip} ]=\frac{\sum_i x_i}{N} \\
N[ \text{two person trip} ]=N-1 \\
E[ \text{two person trip} ]=\frac{\sum_{i,j} \min(x_i,x_j)}{N*(N-1)/2}=\frac{\sum_i(x_i*(i-1))}{N*(N-1)/2} $$
For example, for the case $N=4$ and $x_i=1,2,5,10$, you have 2 one person trip with a mean time of $(1+2+5+10)/4=18/4=4.5$, 3 two person trip with a mean of $(1*0+2*1+5*2+10*3)/(4*3/2)=42/6=7$ for a total expected time of $2*4.5+3*7=30$.
This method works for the mean, for other statistic (like the median) I think a computational approach should be used.


What is the difference between the Load and Render objects in Devel?

Question: When viewing a Node in the Devel tab, what are these different objects Load and Render? How and where would I have access to them in the theme layer?
Answer: The "Load" tab shows the output of node_load(), while the "Render" tab shows the output of node_view(). In the first case, the output is a node object; in the second case, the output is an array as accepted by drupal_render().
Comment: Note that in the render tab you have a row "#node" and if you try to open it you get a "(Recursion)". The load tab has the content of that node object.


group by two columns and use third column as value without using pivot_table

Question: I have the following dataframe.

     user  movie rating
0      1   1     3

1      1   2     4

2      2   1     2

3      2   2     5

4      3   1     3

My desired output is

movie  1   2
user
1      3   4

2      2   5

3      3   0

If a user has not rated a movie, I need to have '0' in the corresponding output column, otherwise, the rating value should be present.
Note: I was able to achieve this with pivot_table, but the catch is my dataset contains more than 100000 columns because of which I am getting "Unstacked DataFrame is too big, causing int32 overflow". I am trying groupby as an alternative to bypass this error.
I am trying the following, but it doesn't include the values from 'value' column of my dataframe.
df.groupby(['user', 'movie']).size().unstack('movie', fill_value=0)
Comment: there is no `3` in your movies but this is a simple `pivot/crosstab` with a `fillna` `pd.crosstab(df.user,df.movie,values=df['rating'],aggfunc='sum').fillna(0)`
Comment: agree. removed the column 3 in the dataframe
Answer: try  using crosstab:
pd.crosstab(df.user, df.movie, values = df.rating, aggfunc = 'first').fillna(0)
# movie    1    2
# user
# 1      3.0  4.0
# 2      2.0  5.0
# 3      3.0  0.0

to get interger values, just use .astype(int), as follows :
pd.crosstab(df.user, df.movie, values = df.rating, aggfunc = 'first').fillna(0).astype(int)
# movie  1  2
# user
# 1      3  4
# 2      2  5
# 3      3  0

Comment: This works on the sample dataframe I had provided in the question. But, I am still getting "Unstacked DataFrame is too big, causing int32 overflow" error as my input dataframe has 283228 rows (exact number)
Answer: I'm not sure why would you expect movie 3 since it doesn't exist in the original data sample but other than that this will work for you: movie_ratings.set_index(['user', 'movie']).unstack('movie', fill_value=0)
Comment: This works on the sample dataframe I had provided in the question. But, I am still getting "Unstacked DataFrame is too big, causing int32 overflow" error as my input dataframe has 283228 rows (exact number)
Comment: This worked for me with 300,000 columns.


Why does the Vuelta a España have a combination classification?

Question: Stage races in cycling are interesting because they typically have multiple competitions inside one race. You have the general classification (usually seen as the most important one), and then there are numerous other classifications such as points classification, intermediate sprints classification, mountains classification, young riders classification, and the combination classification.
The combination classification is the one that confuses me, especially in the Vuelta a España, where it has consistently been one of the prizes for many years. This classification is almost always won by the same rider who wins the general classification. Because the combination classification is (literally) made up of a combination of the other classifications, there is no way for a rider to go for this prize; in order to improve your standings in the combination classification, a rider needs to improve his standings in all other classifications, which is something nobody is interested in doing. People who are possible contenders for this prize are usually also contenders for the general classification, which typically means that they only care about the general classification.
Interesting statistic: After 2003, only one rider has won the combination classification without also winning the general classification. This was Alejandro Valverde in 2012.
So, my question: why does the Vuelta keep employing the combination classification, and why don't they switch to a different prize that riders would actively try to achieve? Are there any historical reasons behind this, or sponsor contracts or so?
Comment: Pretty much related: [Why does not Vuelta have young rider jersey?](http://sports.stackexchange.com/a/14579/11439)
Comment: Damn, @fedorqui, that answer is what I've been waiting for. I think you should post it (and give credit where credit is due) so that I can mark it as accepted. That source is as good as it gets.
Answer: Few weeks ago I did some research for the question Why does not Vuelta have young rider jersey?. For that, I found an interview with La Vuelta's general director Javier Guillén from back in 2014. It is a pity it is in Spanish, but to simplify things I translated into English the relevant parts of it.
In a specific moment he stated:

We do have the same amount of maillots as the other ones; this is, four. We do not have young classification, and I am not saying we should not have it, but nowadays we have the combination with a sponsor which is very much identified with this ranking since it has been sponsoring La Vuelta for 19 years. The combination is a distinguishing mark of La Vuelta, since it is the only race having it and we want to keep it like this. If we ever get the chance, we could include the young rider classification, but nowadays we think that the podium is full enough.

I also found you are absolutely right about this classification being quite secondary, since people winning it are normally those trying to win La Vuelta itself.
I could not find a full statistic on the winners of the combined classification compared to the overall winner. However, the Catalan article of Wikipedia on Combination classification in the Vuelta a España contains a very representative column. There, you can see which others jerseys the winner won: from 2004, just in one case (Joaquim Rodríguez in 2015) the winner just won this one. And in total just in 7 cases out of almost 30.
Answer: I think that it has to do with a historic reasons. Spanish riders were always trying to be the most complete rider. Hense their disire to win stage races and excel in those. They were lousy one-day riders. (e.g. The first Spanish World Champion was Olano in 1995). So I think that the persuit of being the most complete rider back in the days is the reason why there is still such a jersey.


Harmonic Mean equation/solutioin

Question: Here is the question:
A a number x is said to be the harmonic mean of y and z if 1/x is the average of 1/y and 1/z.
a. Write an equation for which the solution is the harmonic mean of 30 and 45.
b. Find the harmonic mean of 30 and 45.
My book doesnt give an explanation of what a harmonic mean is, how to make an equation for one or how to solve one therefore please make your explanation as detailed as possible.
Comment: Average of $a$ and $b$ is $(a+b)/2.$ Apply this to $1/y$ and $1/z$ and equate to $1/x.$
Comment: @kodlu Would you mind showing me how this would work for the two parts of the question. Like I said the book doesnt give a great explanation and I would like to know for when a test comes around :)
Comment: @kodlu If you show me a. would help a lot.
Answer: Let $a=\frac{1}{y}$ and $b=\frac{1}{z}.$ The average of $a,b$ is
$$
\frac{(a+b)}{2}=\frac{\frac{1}{y}+\frac{1}{z}}{2}=\frac{1}{2}\left(\frac{1}{y}+\frac{1}{z}\right)=\frac{(y+z)}{2yz}.
$$
The Harmonic mean $x$ is the reciprocal of this average so
$$
x=\frac{2yz}{(y+z)}.$$
So, the answer to (a) is (letting $y=30$, $z=45$),
$$
x=\frac{2yz}{(y+z)}=\frac{2\times 30\times 45}{30 + 45}.
$$
This should be enough to enable you to answer (b).

Harmonic Mean equation/solutioin


Independent Events- $P(\text{neither/exactly/at least one})$

Question: I'm working on some problems from Pitman's Probability.

Let events $A$ and $B$ be independent, with $P(A)=0.1$ and $P(B)=0.3$. Find:
$(a)$ Neither of the events occur
$(b)$ At least one of the events occur
$(c)$ Exactly one of the events occur

Attempt:
$(a)$ If neither of the events occur, this is the complement of both events occurring. Hence $1-P(A\cap B)=1-P(A)P(B)=1-\frac{1}{30}=\frac{29}{30}$
$(b)$ At least one of the events can be represented as the union of the two events. Hence $P(A\cup B)=P(A)+P(B)-P(A\cap B)=P(A)+P(B)-P(A)P(B)=\frac{4}{10}-\frac{1}{30}=\frac{11}{30}.$
$(c)$ Not sure what this would be. Maybe $P(A^c\;\cap\;B)+P(A\cap B)$?
Answer: a) The probability that $A$ does not occur is $0.9$. The probability $B$ does not occur is $0.7$. So the probability neither occurs, by independence, is $(0.9)(0.7)$.
b) This is the complement of the event of a).
c) We want the probability that $A$ occurs but $B$ does not, or that $A$ does not occur but $B$ does. So in symbols we want $\Pr(A\cap B^c)+\Pr(A^c\cap B)$. We leave the rest of the calculation to you.
Remark: Your calculation in b) was almost right. There was an arithmetical slip, since $\Pr(A)\Pr(B)=(0.1)(0.3)=0.03\ne \frac{1}{30}$.
Answer: a.$P(A'\cap B') = P((A\cup B)') = 1 - P(A\cup B) = 1 - P(A) - P(B) + P(A\cap B) = 1 - P(A) - P(B) + P(A)P(B) = 1 - 0.1 - 0.3 + 0.1\cdot 0.3 = 0.6 + 0.03 = 0.63$.
b.Same as your answer.
c. $P((A\cap B')\cup (B\cap A')) = P(A\cap B') + P(B\cap A') = P(A)P(B') + P(B)P(A') = 0.1\cdot 0.7 + 0.3\cdot 0.9 = 0.07 + 0.27 = 0.34$


Breaking a string down into words using D language

Question: I am new to using D, and I wanted to ask for help on a simple method of breaking down a string into words. My program takes text from a file and is supposed to count the number of words in that text.
Answer: splitter and walkLength can do this trivially.
void main()
{
    import std.stdio, std.algorithm, std.range, std.file;
    writeln(readText("file.txt").splitter.walkLength());
}


C++ struct reinterpret_cast

Question: Suppose there are two struct A and B. They have a common struct C.
I would like to know if it is safe to call reinterpret_cast to A or B to C.
if not, is there any way to do so without any performance impact?
struct C
{
    string m_c1;
    int32_t m_c2;
    double m_c3;
    string m_c4;
};

struct A
{
    C m_a1;
    string m_a2;
    int32_t m_a3;
};

struct B
{
    C m_b1;
    string m_b2;
    int32_t m_b3;
    double m_b4;
};
int main(int argc,char *argv[])
{
        A a;
        a.m_a1.m_c1="A";
        a.m_a1.m_c4="AA";

        B b;
        b.m_b1.m_c1="B";
        b.m_b1.m_c4="BB";

        C* pc = reinterpret_cast<C*>(&a);
        cout << pc->m_c1 << " " << pc->m_c4 << endl;

        pc = reinterpret_cast<C*>(&b);
        cout << pc->m_c1 << " " << pc->m_c4 << endl;

        return 1;
}

Comment: Even if it is safe today, can you guarantee that the structure of A and B will never change in the future? And if one of them does change, will whoever changes them know that the C struct *has* to be the first member?
Comment: Nothing in your example code indicates why you don't do `C *pc = &a.m_a1;`, which is entirely safe.
Answer: As Mike DeSimone points out the string class is not guaranteed to be a standard-layout class, and thus the class C is not standard-layout, which means that you have no guarantees of the memory layout at all. So it is not safe. Only if you change the string to a (const) char* it will be guaranteed to be safe.
Even then it will only be safe as long as the layout of the classes stays the same (you cannot change the order of members or change the access specifier on them), and the classes stays without any vtables, this is "safe" in such a way that the compiler will generate code that display the behavior you would like.
This is how ever two guarantees that a software developer seldom is able to give. Also the code is hard to understand written like this. Another developer (or the same developer a month later) might ignore the this code (or simply don't understand it), and do the changes needed, and suddenly the code is broken and you got some hard to catch errors on your hand.
A and B are classes that give access to a C (or some members of C). More readable and thus more safe solutions are:

Create an accessor for both A and B, this would probably be inlined and incur no performance penalty.
If there is any reason for inheritance use simple inheritance. Either A and B is-a ClassThatHasAC or A and B is-a C As long as there are no virtual functions you would probably not see any performance issues here either. In both cases an accessor would provide you benefits probably without any performance cost.

Create some simple and readable code at first, and measure performance. Onli if this C access is costing you too much, optimize. But if your optimization boils down to the reinterpret cast trick make sure that there are plenty of warning signs around to make sure that no one steps on this booby trap.
Answer: Why dont you inherit both A,B from C and then use static_cast instead? Should be safer/cleaner.
Indeed in your case, you shouldnt need a cast at all, you should be able to assign A or B ptrs to a C*
Comment: It is seldom a good idea to inherit from a class just for the sake of convenience. Sooner or later this will lead to problems.
Comment: @Gorpik It depends on the meaning of A,B,C but it does sound like inheritance is a fair interpretation of their relationship. plus the choice is made for cleaner,safer code, not more convenient code.
Comment: We have no idea of their relationship because we don't know what are those structs modelling. We don't even know why is OP trying to use a cast there. Public inheritance should model an IS-A relationship; otherwise, someone will sometime try to use an A where a C is needed and have their code behave nonsensically.
Comment: The way this particular example is written strongly, strongly suggests is-a, and therefore inheritance, relationships. Further, if inheritance is used, along with `static_cast`, it avoids the whole what-if-it-has-`virtual`-methods case I mentioned in the comment on icepack's answer, because `static_cast` will take care of doing the right thing.
Comment: Agree. Inheritance is the way to go. I agree with Gorpik that public inheritance should model an IS-A relationship, but private inheritance is a perfectly valid way to express IS-IMPLEMENTED-AS. The code from the question doesn't express which of the two applies.
Comment: I agree. This seems to be written in a C-style inheritance way. Also, obviously the OP is trying to use both A and B as a C here, which means A must be a C for that to make any sense at all.

C++ struct reinterpret_cast


Delete MySQL row where multiple indexes are the same, but keep the most recent

Question: Here is what my table looks like
=======================================
|| id | collection_id | product_id | price ||
=======================================
|| 1  |   1    |   1   | 2.50  ||
|| 2  |   1    |   1   | 2.60  ||
|| 3  |   1    |   1   | 2.40  ||
|| 4  |   2    |   1   | 2.50  ||
=======================================

Here is a rough outline of what I want to achieve; using some pseudo code.

What I want to do is DELETE FROM prices WHERE (collection_ id && product_ id are duplicated) AND WHERE (id < highest duplicated row)

SO what I want to do in plain English is delete rows where the id is equal to 1 & 2 but keep the row where the id is equal to 3 because this the newest.
Answer: A simple DELETE JOIN should do it; delete all rows where there exists at least one row with a matching collection_id and product_id and a greater value of id;
DELETE p1
FROM prices p1
JOIN prices p2
  ON p1.collection_id = p2.collection_id
 AND p1.product_id = p2.product_id
 AND p1.id < p2.id

An SQLfiddle to test with, and remember to always back up your data before running updates or deletes from random people on the Internet.
Comment: @gcoulby Basically, it's a regular `JOIN`. The query picks two rows from the table `prices` and call them `p1` and `p2`. If the conditions are true (ie their rows match and p1 has a lower id), we delete p1 from the table. It then basically repeats for every row combination in the table, so the end result is that all rows that have a matching row with a greater id have been deleted.
Comment: Thanks for the tip, but I am not running this until I understand it. I can see it works from the fiddle but I can't see how. Where are p1 & p2 coming from? and what happens if there is fifty values?

Is this a pseudo table?
Comment: OK so I've tested this in the fiddle with 10 lines and it works, can you explain what this is doing?
Comment: Ahh that makes a lot of sense now. I didn't use it in this project, but this came in very handy for a second project. In this project I put in code that detected duplicates on entry and automatically deleted old ones.


Reaching fixed point of "type"

Question: Calling type(x) on an object x returns the type of that object. Calling type(type(...type(x)...)) always eventually yields type itself, with type(type) = type being a fixed point. The most complex example I could come up with was
class Meta(type):
    pass

class C(metaclass=Meta):
    pass

x = C()

which requires three calls to type (type(type(type(x)))) to evaluate to type.
Is it true for any expression in Python, that at most three calls to type are needed to reach the fixed point? If not, is there a finite limit (just larger than three) or can the required stack of type calls grow indefinitely?
Comment: Note that in your example, ``isinstance(Meta, type)``, or in other words ``type(C) <: type``. Requiring the extra ``type`` layer is merely from ignoring subtyping. However, ``type(type) == type`` is a base case, from which arbitrary many further levels can be derived.
Answer: The required stack of type calls can grow indefinitely.
Here is an example:
>>> class Meta(type):
...     pass
...
>>> class C(type, metaclass=Meta):
...     pass
...
>>> class D(type, metaclass=C):
...     pass
...
>>> class E(type, metaclass=D):
...     pass
...
>>> class F(type, metaclass=E):
...     pass
...
>>> f = F("1",(),{});
>>> type(f)
<class '__main__.F'>
>>> type(type(f))
<class '__main__.E'>
>>> type(type(type(f)))
<class '__main__.D'>
>>> type(type(type(type(f))))
<class '__main__.C'>
>>> type(type(type(type(type(f)))))
<class '__main__.Meta'>
>>> type(type(type(type(type(type(f))))))
<class 'type'>
>>> type(type(type(type(type(type(type(f)))))))
<class 'type'>
>>>

Comment: correct me if I'm wrong, but I think all objects are stored in heap, while the pointer to object itself is stored in stack
Comment: heap is used when stack overflows, for local variables which get changed a lot like i in loops; stack is used for global data and function returns, a typical segmentation fault in c is a overflow in stack or accessing non-permissable locations
Answer: The case type(type) == type is merely the base case of instantiatable types. Just as the first layer of types can be created from type, further layers can be constructed. There is no fixed limit.
def descend(base=type):
    while True:
        class base(base, metaclass=base):
            pass
        yield base

def type_depth(tp):
    depth = 0
    while type(tp) != type:
          depth += 1
          tp = type(tp)
    return depth

for tp in descend():
    print(tp, type_depth(tp))

Practically, there is a limit for the usefulness of meta-layers: Most classes are direct instances of type itself or one of its subtypes.

Reaching fixed point of "type"


coldfusion cffile upload replace no file selected

Question: In Coldfusion, using cffile upload, when a file is selected with the browse button the name of the selected file shows next to the button.  So far, so good.  When I submit the file for upload, the text changes back to "no file selected".  This is doubtless because we are submitting a form.
However, I'm submitting the form to the same page, and would like to see the name of the file selected instead of "no file selected" which is likely to confuse the user.
The code:
<form
 enctype="multipart/form-data"
 method ="post"
 name   ="attupload"
 action ="">

<cfoutput>

   <input name="theupload"
          type="file"
          style = "font-family: comic sans ms;
                   color: ##679C9C">

     <input name  = "submit"
            type  = "submit"
            class = "submitbut"
            style = "font-size: 16px"
            value = "Upload File">

</cfoutput>
</form>

The words 'no file selected' seem to be part of the input name = 'upload' field, but ColdFusion evidently takes control of it, and I cannot insert anything else.

Does anyone know how I can control the 'no file selected' note, so that I can put the name of the selected file back in that spot?  I have tried with javascript, but it is simply ignored.
Answer: Once a file has been selected, you can access its properties via JavaScript with the File API. You can't affect the value displayed next to the default <input type="file"> field with JavaScript.
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/file
You should also avoid submitting forms to the same page. You should follow the process:

Form page
HTTP POST to form processing page.
after form processing, <cflocation> (301 redirect) back to a list page or the form page.

When you post to the same page, the user can hit reload and be prompted to resubmit form data. This can be problematic in many situations. As an alternate, you can post the form data using Ajax or better, via Fetch. This avoids a re-post on page reload and redirecting away from the current screen.
Also, you should avoid naming an <input type="submit"> "submit". It can conflict with the JavaScript function submit() should you need to use it.
<input name  = "submit"
        type  = "submit"
        class = "submitbut"
        style = "font-size: 16px"
        value = "Upload File">

Comment: You are quite right that I should submit it to another page.  You say that I can't affect the value displayed next to the default  field with JavaScript. I'm wondering if you know another way to affect it?  Do you know how it works?  It does seem to be an HTML property, not coldfusion.
Comment: Re the input submit button, name = "submit" is a mistake and I've removed it.  As for type = submit, in all my code either there is a submit button with type = submit, or else I  submit the form with javascript using submit().  However, there is never both.
Comment: Traditionally, file input values were read only for security reasons. Not sure if that's changed with html5 https://stackoverflow.com/questions/1696877/how-to-set-a-value-to-a-file-input-in-html


Form Select not rendering as a dropdown

Question: I'm building a form that contains a select but it's not rendering as a dropdown. It's rendering with a scrollbar and a dropdown down arrow

$form['my_select'] = [
  '#type' => 'select',
  '#empty_option' => '- None -',
  '#title' => $this->t('My Select'),
  '#options' => ['Option1', 'Option2'],
  '#size' => 6,
  '#weight' => '0',
];

What am I doing wrong?
Comment: If you switch to the default, Bartik theme, does it look how you expect?
Answer: It's rendering as a list because you have set #size to 6, which means 6 rows will be visible. Either remove the #size property completely, or set it to 1 to display a normal dropdown.
From the documentation:

#size: The number of rows in the list that should be visible at one time.
Comment: That fixed it! Thank you!

Form Select not rendering as a dropdown


Dynamically fetch a value based on drop down selection

Question: I have a form with a drop-down menu that allows users to select how low they want to book an item. I am letting stripe to handle the payment. I would like to have the length retrieved dynamically based on the selection.
<%= form_tag pages_next_url(@booking), :length => params[:length] do %>

      <%= select_tag "length", options_for_select(@garage_times.map { |obj| [obj['time'], obj['value']] }) %>

    <%= hidden_field_tag :garageid, @garage.id %>

<% end %>

In my controller, I have the values listed. When "1:00 hour" is selected, it should return length, 1 for this case, dynamically.
@garage_times = [
 {"value" => 1, "time" => "1:00 hour"},
 {"value" => 2, "time" => "2:00 hours"},
 {"value" => 3, "time" => "3:00 hours"},
 {"value" => 4, "time" => "4:00 hours"}
]

I understand that it requires AJAX/JS in order to retrieve the value dynamically. I did some research and watched Railscast on their dynamic form. However, it does not help on my case. All of the examples I have seen are retrieving values from a model. Also, I know that it can be done if I put the calculation in the controller, but the whole point is to allow users to see the total amount on the pay button when they input their payment information.
Please guide me. I have minimal knowledge of JS and AJAX. Thank you.
Edit:
I am still unable to solve this problem still.
Answer: It seems like you can avoid using AJAX and use the select list value. Then use the length value to update the button.
You can do something like this in jQuery:
$('#length').on('change', function() {
  weekday_rate = $('#weekday_rate').html();
  length = $(this).val();
  $('.stripe-button').attr('data-amount', weekday_rate * 100 * length)
})

Comment: I am not sure if it is possible to do in the way you mentioned. `garage.weekday` will return a float value from database (eg 1.50 per hour).  I think it is easier just to retrieve `length` (1, 2, 3 hours) instead of letting jQuery to do the entire calculation.
Comment: Updated answer. You can have the current weekday rate in a hidden div.
Comment: I tried your method and it did not work for my case. I changed the code to `alert` and remove `weekday_rate` to make sure that my JS works. And it did an alert with the correct multiplied value. But it refused to change the `data-amount` for the script. I will report back when I have a working answer for my case.


Titan batch graph

Question: I want to add a new Property [and some times add edges] to a selection of nodes in an existing Graph of 2 million nodes, 10+ million edges. I thought of using BatchGraph but from their WIKI looks like it does not support any retrieval queries.
For e.g. retrieve these nodes: g.V('id',1).has('prop1','text1') and update 'prop1' to 'text2'.
What is the best way to do this.
Answer: I don't think you need to use BatchGraph here.  It sounds as if you are doing a large graph mutation in which case it would probably best to just write a Gremlin script to do your changes.  You don't have a very large graph so unless you plan to do some very complex mutations (e.g a fat multi-step traversal), it shouldn't take very long to execute.  If you do think it's going to run "long" you should think of ways to parallelize the job.  If you go this route you might consider using gpars.
As your graph grows, you will find that you will need to use Faunus for most data administration.  Specifically, that means utilizing script step.
Comment: Thanks. Well, the graph will grow eventually, so was looking for better ways. I'll take a look at gpars. Also in my Graph [Cassandra-ES, has User and Place nodes in millions] noticed, when I'm adding edges between existing nodes [using Java,  about 7 threads], reading from a file,User --> Place with 'likes' property set on it, it is throwing large number of 'com.thinkaurelius.titan.diskstorage.locking.PermanentLockingException: Local lock contention' exceptions. Any better way to handle this?
Comment: Updated my answer a bit to reflect "eventual growth" of your graph.  If you have locks you should expect failure at times.  You should have some sort of retry strategy in place to re-execute your transaction in the event of failure.  Blueprints has the `TransactionRetryHelper` (https://github.com/tinkerpop/blueprints/wiki/Graph-Transactions#transaction-retry-helper) that can be useful in that regard. You might also consider preprocessing your file so you can turn `storage.batch-loading` on with confidence (which will turn locking off)
Comment: Faunus script step looks great. In the past I also tried using TransactionRetryHelper, and still saw lot `PermanentLockingException` messages in the log. Just trying to understand, is it because adding an edge updates adjacency list of both out and in vertex? Since my data set [User --> Places] is Many-To-Many, the moment multiple threads try to update the graph, these exceptions are bound to happen..correct? So given this, can I enable storage.batch-loading?
Comment: It might not be safe to enable batch-loading if you need to ensure that your uniqueness.  You would need to pre-process your data to be a unique list of users/places.  Load that followed by the edges which with batch-loading on.  If you can't preprocess for some reason, then you should try to release locks more quickly (by commit/rollback).  You don't mention how big your transaction sizes are, but the longer you leave a lock open, the greater the chance of a locking exception.  Perhaps lowering your transaction size would help.
Comment: You also want to make sure that you don't create weird race conditions with your locks.  So within a transaction you have a lock on  userId and placeId.  One transaction in one thread grabs the userId lock but fails to get the placeId lock.  It fails to get the placeId lock because a different thread has that placeId, but not the userId lock.  Neither can complete.  Sometimes having some additional randomization in the retry delay can help with those situations, but if they can be avoided all together so much the better.

Titan batch graph


Exclude duplicates from an array in Google Apps Script

Question: I've been looking for a function or method in Google Apps Script that removes all the duplicates from an array (discarding the duplicates themselves) and keep only the unique values. For instance:
const chars = ['A', 'B', 'A', 'C', 'B', 'D'];
Expected result/output:
['C', 'D']
So far, I haven't been able to achieve that. Any help would be much appreciated.
Answer: Try this:
<code>function removeDuplicates(a=['A', 'B', 'A', 'C', 'B', 'D']) {
  let oA = {pA:[]};
  //count the occurrence of each element
  a.forEach(e=>{
    if(!oA.hasOwnProperty(e)) {
      oA[e]=1;
      oA.pA.push(e);
    } else {
      oA[e]+=1;
    }
  });
  //return only the elements that only occur once
  Logger.log(oA.pA.filter(e => oA[e]==1));
}
</code>
This just counts the occurrences of each element and only returns the elements that only occur once.
Comment: That's it! Very concise and accurate. Thanks.


GCM example project error message "Could not find google-play-services_lib.apk"

Question: With the example code provided on http://code.google.com/p/gcm/source/browse/#git%2Fgcm-client, the application launch protocol contains a message about a missing APK.
<code>[2013-06-22 10:28:24 - GCMClientNewAPI] ------------------------------
[2013-06-22 10:28:24 - GCMClientNewAPI] Android Launch!
[2013-06-22 10:28:24 - GCMClientNewAPI] adb is running normally.
[2013-06-22 10:28:24 - GCMClientNewAPI] Performing com.google.android.gcm.demo.app.DemoActivity activity launch
[2013-06-22 10:28:29 - GCMClientNewAPI] Uploading GCMClientNewAPI.apk onto device '015d3e63881c020d'
[2013-06-22 10:28:30 - GCMClientNewAPI] Installing GCMClientNewAPI.apk...
[2013-06-22 10:29:07 - GCMClientNewAPI] Success!
[2013-06-22 10:29:07 - google-play-services_lib] Could not find google-play-services_lib.apk!
[2013-06-22 10:29:07 - GCMClientNewAPI] Starting activity com.google.android.gcm.demo.app.DemoActivity on device 015d3e63881c020d
[2013-06-22 10:29:07 - GCMClientNewAPI] ActivityManager: Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.google.android.gcm.demo.app/.DemoActivity }
</code>
However the application is able to connect to Play services, and get a registration id. Also sending messages seems to work fine - at least no errors are logged.
The library project is referenced by the demo project in the Android settings (as also can be seen in the project.properties file).
I am testing on a Nexus 7 device with the current Android 4.2.2 version, and the current Google Play Services version 3.1.36 (669520-30).
What could cause this Could not find google-play-services_lib.apk message?
Answer: The solution was in a comment to the answer for this question:
Using the new Google Play Services
Steps:

remove the google_play_services-lib reference from the Java build path
add the library reference in Properties | Android | Library references
Comment: THanks buddy... You are champ~~~
Comment: Thanks ! It works - although the error for me had no side effects - i had already added the library reference.
Comment: This compiles, but when I go to run the project I get an 'Unable to execute dex: GC overhead limit exceeded. GC overhead limit exceeded.'
Comment: Ok, please see this link to fix the  'GC overhead limit exceeded.' http://stackoverflow.com/questions/23080078/eclipse-crashes-with-a-gc-overhead-limit-exceeded-error


Understanding the definition of convexity in $\mathbb{R}^k$

Question: I'm taking a course in real analysis.
The formal definition of convexity is given as follows:
The subset $A \subset \mathbb R^k$ is convex if for all $x,y \in A$ if the whole line section between $x$ and $y$ also lies in $A$.
The formal definition
$$\forall x,y \in A, \lambda \in (0,1) : \lambda x+(1-\lambda)y$$
I have no familiarity with convexity other than see some drawing of the concept in a $\mathbb R^2$ plane. I'm trying to figure out why the formal definition looks like that.
I have no understanding of this...
Answer: If $x,y\in\mathbb R^k$, then$$\{\lambda x+(1-\lambda)y\mid\lambda\in[0,1]\}$$is the line segment joining $x$ to $y$. Therefore, that definition simpliy means "if $x,y\in A$, then the line segment joining them is contained in $A$".
Comment: Note that $\lambda x+(1-\lambda)y$ is $y$ when $\lambda=0$ and $x$ when $\lambda=1$. Does this help?
Comment: Yes. Another way of seeing it is:$$\lambda x+(1-\lambda)y=y+\lambda(x-y).$$So, you add to $y$ the vector that goes from $y$ to $x$ times a scalar $\lambda\in[0,1]$. It is clear that you get $y$ if $\lambda=0$ and that you get $x$ if $\lambda=1$.
Comment: I'm glad I could help.
Comment: Yes, no problem abou that.
Comment: Oh, okay. This was actually what I was thinking, but I could (still can) not figure out have the line segment is parameterised.
Comment: It actually makes some things much clear than the representation in my textbook.  But to be honest I still do not know how the line segment is parameterised. Is the line segment parameterised from $y$ to $x$, right?
Comment: Amazing explanation. I finally understand this.
Comment: One last question. Could $A$ be an improper subset, for example  $A=\mathbb{R}^k$?


How to calculte the area between a set of coordinates and a give axis?

Question: let's say I have an array of 20 values, for example: [ 0. , -2., 3., -4., etc.]
And if I plot them with seaborn lineplot with y = 0, y = 1 and y = -1, they would look like this:

The question is, if I want to calculate the area of the triangle on the left top (the one above the green line), how would I do that in Python?
Thanks in advance
Comment: @d.b greeting! I have looked into this function before,but it doesn't quite solve my problem. Because I have to find the intercept of my coordinates with line (y = 1) first.
Comment: Please provide enough code so others can better understand or reproduce the problem.
Answer: This solution is for calculating area between your curve (given x-values and y-values) and y=1, you can change the code accordingly to your needs.
Create the arrays and plot them
To do this we first have to create our arrays (x, y) and check if the line connecting two given point intersect y=1.
First of all we create the arrays:
import random

x = [i for i in range(20)]
y = [random.randint(-5, 10) for i in range(20)]

To plot these points and naming them we can use matplotlib (or whatever you want). We also add our y=1 line:
import matplotlib.pyplot as plt
import string

plt.plot(x, y)
plt.axhline(y=1)        # Straight line
alphabet = string.ascii_lowercase       # naming points on plot
for i in range(len(x)):
    plt.scatter(x[i], y[i])
    plt.annotate(alphabet[i], (x[i], y[i]))
plt.show()

Find the intersection point
Now we have to find if y=1 intercept our line and the x-value of the point:
def intersect(y1, y2):
    '''
    Find if the line connecting two points intercept y=1
    '''
    if (y1 <= 1 and y2 >= 1) or (y2 <= 1 and y1 >= 1):
        return True

def slope_intercept(x1,y1,x2,y2):
    '''
    Find the slope and intercept for the given coordinates
    and find x value if y=1
    '''
    m = (y2 - y1) / (x2 - x1)
    q = y1 - m * x1
    point = (1-q)/m
    return point

intercept = []          # x values of intercept point
for i in range(len(x)-1):
    if intersect(y[i], y[i+1]):
        print(f"{alphabet[i]}-{alphabet[i+1]} intersect")       # which segment intersect y=1
        point_x = round(slope_intercept(x[i],y[i],x[i+1],y[i+1]), 2)  #round our result to two decimal places
        print(f"Intercepted point is x={point_x}")
        intercept.append(point_x)

Find the area
Now that we have our intersection point, we can calculate area of the given polygon.
For function we would have used Integral, but here we have straight lines, so we can use the trapezoidal rule. First of all we calculate area under our curve, than we subtract the area under y=1 for the given interval in order to get the area of the polygon:
from math import ceil, floor

def area_calc(interval):
    '''
    Find area of the given interval
    using the following formula:
    ((x2 - x1) * (f(x2) + f(x1)) / 2
    '''
    tot = 0
    for i in range(len(interval)-1):      # Calculate area for every trapezium
        h = round(interval[i+1] - interval[i], 2)
        if i == 0:
            b2 = abs(y[interval[i+1]])    # We need to calculate distances, so we use absolute values
            a = (h * (1 + b2)) / 2
            print(f"[{h=} * (1 + {b2=})] / 2 = {a}")
        elif i == len(interval) - 2:
            b1 = abs(y[interval[i]])
            a = (h * (1 + b1)) / 2
            print(f"[{h=} * ({b1=} + 1)] / 2 = {a}")
        else:
            b1 = abs(y[interval[i]])
            b2 = abs(y[interval[i+1]])
            a = (h * (b1 + b2)) / 2
            print(f"[{h=} * ({b1=} + {b2=})] / 2 = {a}")
        tot += abs(a)                         # Total area will be the sum of all trapezium's areas
    tot -= 1 * (interval[-1] - interval[0])   # Subtract area under y=1 for the given interval
    return round(tot, 2)

for i in range(len(intercept)-1):
    start = intercept[i]
    finish = intercept[i+1]
    interval = [z for z in range(ceil(start), floor(finish)+1)]     # Interval used for calculate area
    interval.insert(0, start)       # Insert intercept point at the beginning
    interval.append(finish)         # Insert last point at the end
    #area = area_calc(interval)
    area = area_calc(interval)
    print(f"{start=} {finish=}: {interval=}     {area=}\n")

Finally we get what we wanted!
Conclusion
It might seems difficult at the beginning, but we have to remember some basic calculus and analytical geometry. Of course you can use whatever array you want and calculate areas between your curve and any other curve now, just change some parts of the code.
I hope my explaination was clear and the code work as expected
Have fun!
Final code:
from math import ceil, floor
import matplotlib.pyplot as plt
import string
import random

def intersect(y1, y2):
    '''
    Find if the line connecting two points intercept y=1
    '''
    if (y1 <= 1 and y2 >= 1) or (y2 <= 1 and y1 >= 1):
        return True

def slope_intercept(x1,y1,x2,y2):
    '''
    Find the slope and intercept for the given coordinates
    and find x value if y=1
    '''
    m = (y2 - y1) / (x2 - x1)
    q = y1 - m * x1
    point = (1-q)/m
    return point

def area_calc(interval):
    '''
    Find area of the given interval
    using the following formula:
    ((x2 - x1) * (f(x2) + f(x1)) / 2
    '''
    tot = 0
    for i in range(len(interval)-1):
        h = round(interval[i+1] - interval[i], 2)
        if i == 0:
            b2 = abs(y[interval[i+1]])
            a = (h * (1 + b2)) / 2
            print(f"[{h=} * (1 + {b2=})] / 2 = {a}")
        elif i == len(interval) - 2:
            b1 = abs(y[interval[i]])
            a = (h * (1 + b1)) / 2
            print(f"[{h=} * ({b1=} + 1)] / 2 = {a}")
        else:
            b1 = abs(y[interval[i]])
            b2 = abs(y[interval[i+1]])
            a = (h * (b1 + b2)) / 2
            print(f"[{h=} * ({b1=} + {b2=})] / 2 = {a}")
        tot += abs(a)
    tot -= 1 * (interval[-1] - interval[0])
    return round(tot, 2)

x = [i for i in range(20)]
y = [random.randint(-5, 10) for i in range(20)]

plt.plot(x, y)
plt.axhline(y=1)        # Straight line
alphabet = string.ascii_lowercase       # naming points on plot
intercept = []          # x values of intercept point
for i in range(len(x)):
    plt.scatter(x[i], y[i])
    plt.annotate(alphabet[i], (x[i], y[i]))
for i in range(len(x)-1):
    if intersect(y[i], y[i+1]):
        print(f"{alphabet[i]}-{alphabet[i+1]} intersect")       # which segment intersect y=1
        point_x = round(slope_intercept(x[i],y[i],x[i+1],y[i+1]), 2)
        print(f"Intercepted point is x={point_x}")
        intercept.append(point_x)

print()

for i in range(len(intercept)-1):
    start = intercept[i]
    finish = intercept[i+1]
    interval = [z for z in range(ceil(start), floor(finish)+1)]     # Interval used for calculate area
    interval.insert(0, start)       # Insert intercept point at the beginning
    interval.append(finish)         # Insert last point at the end
    #area = area_calc(interval)
    area = area_calc(interval)
    print(f"{start=} {finish=}: {interval=}     {area=}\n")

plt.show()

Simulation case
Simulation results:
a-b intersect
Intercepted point is x=0.83
c-d intersect
Intercepted point is x=2.62
d-e intersect
Intercepted point is x=3.36
e-f intersect
Intercepted point is x=4.75
g-h intersect
Intercepted point is x=6.5
i-j intersect
Intercepted point is x=8.43
j-k intersect
Intercepted point is x=9.31
l-m intersect
Intercepted point is x=11.33
m-n intersect
Intercepted point is x=12.18
n-o intersect
Intercepted point is x=13.6
o-p intersect
Intercepted point is x=14.55
q-r intersect
Intercepted point is x=16.57
r-s intersect
Intercepted point is x=17.5
s-t intersect
Intercepted point is x=18.75

[h=0.17 * (1 + b2=2)] / 2 = 0.255
[h=1 * (b1=2 + b2=9)] / 2 = 5.5
[h=0.62 * (b1=9 + 1)] / 2 = 3.1
start=0.83 finish=2.62: interval=[0.83, 1, 2, 2.62]     area=7.07

[h=0.38 * (1 + b2=4)] / 2 = 0.95
[h=0.36 * (b1=4 + 1)] / 2 = 0.8999999999999999
start=2.62 finish=3.36: interval=[2.62, 3, 3.36]     area=1.11

[h=0.64 * (1 + b2=10)] / 2 = 3.52
[h=0.75 * (b1=10 + 1)] / 2 = 4.125
start=3.36 finish=4.75: interval=[3.36, 4, 4.75]     area=6.25

[h=0.25 * (1 + b2=2)] / 2 = 0.375
[h=1 * (b1=2 + b2=2)] / 2 = 2.0
[h=0.5 * (b1=2 + 1)] / 2 = 0.75
start=4.75 finish=6.5: interval=[4.75, 5, 6, 6.5]     area=1.38

[h=0.5 * (1 + b2=4)] / 2 = 1.25
[h=1 * (b1=4 + b2=4)] / 2 = 4.0
[h=0.43 * (b1=4 + 1)] / 2 = 1.075
start=6.5 finish=8.43: interval=[6.5, 7, 8, 8.43]     area=4.4

[h=0.57 * (1 + b2=3)] / 2 = 1.14
[h=0.31 * (b1=3 + 1)] / 2 = 0.62
start=8.43 finish=9.31: interval=[8.43, 9, 9.31]     area=0.88

[h=0.69 * (1 + b2=10)] / 2 = 3.795
[h=1 * (b1=10 + b2=2)] / 2 = 6.0
[h=0.33 * (b1=2 + 1)] / 2 = 0.495
start=9.31 finish=11.33: interval=[9.31, 10, 11, 11.33]     area=8.27

[h=0.67 * (1 + b2=1)] / 2 = 0.67
[h=0.18 * (b1=1 + 1)] / 2 = 0.18
start=11.33 finish=12.18: interval=[11.33, 12, 12.18]     area=0.0

[h=0.82 * (1 + b2=10)] / 2 = 4.51
[h=0.6 * (b1=10 + 1)] / 2 = 3.3
start=12.18 finish=13.6: interval=[12.18, 13, 13.6]     area=6.39

[h=0.4 * (1 + b2=5)] / 2 = 1.2000000000000002
[h=0.55 * (b1=5 + 1)] / 2 = 1.6500000000000001
start=13.6 finish=14.55: interval=[13.6, 14, 14.55]     area=1.9

[h=0.45 * (1 + b2=6)] / 2 = 1.575
[h=1 * (b1=6 + b2=9)] / 2 = 7.5
[h=0.57 * (b1=9 + 1)] / 2 = 2.8499999999999996
start=14.55 finish=16.57: interval=[14.55, 15, 16, 16.57]     area=9.9

[h=0.43 * (1 + b2=5)] / 2 = 1.29
[h=0.5 * (b1=5 + 1)] / 2 = 1.5
start=16.57 finish=17.5: interval=[16.57, 17, 17.5]     area=1.86

[h=0.5 * (1 + b2=7)] / 2 = 2.0
[h=0.75 * (b1=7 + 1)] / 2 = 3.0
start=17.5 finish=18.75: interval=[17.5, 18, 18.75]     area=3.75


How to fix 'Cannot redeclare ctools_content_process' problem after sql-sync?

Question: I'm encountering a really weird bug with Drupal. I'm running Debian's Drupal in a multisite configuration on the same host with both the dev and live websites. I know it's not ideal but this is a constraint I cannot avoid for a variety of reasons.
So I have my sites (e.g. dev.example.com and www.example.com) and I sync them with drush sql-sync. Right after doing so, I cannot access the synced website anymore and Apache throws the following errors:
PHP Fatal error:  Cannot redeclare ctools_content_process()
(previously declared in /etc/drupal/7/sites/www.example.com/modules/ctools/includes/content.inc:18)
in /etc/drupal/7/sites/dev.example.com/modules/ctools/includes/content.inc on line 84

To make my website work again, I have to manually do a find-replace in MySQL to change any instance of dev.example.com to www.example.com in tables field_data_body and field_revision_body.
I reckon the problem is linked to CTools (and hence to views), but I have no idea why. Have any of you ever encountered such a problem?
Comment: Seems you synced too much - information about module locations should NOT be synced. Is it a bug in drush, or something with your setup or the way you synced, I cannot tell. Try clearing all caches, and if that won't help restore `{system}` from backup.
Answer: In Drupal 7; this could be caused by corruption in the code registry. The code registry

is an inventory of all classes and interfaces for all enabled modules
  and Drupal's core files. The registry stores the path to the file a
  given class or interface is defined in, and loads the file when
  necessary.

ref: Drupal Code Registry
The solution to this particular problem would be to force a rebuild of the registry; which you can do as per this post; Code Registry Rebuild.
Answer: Are you using Drupal 6?  In Drupal 6, the full path to the sites folder is written into the database for a variety of purposes.  Because of this, if you change the path to settings.php, you break Drupal.  The usual solution to this problem is to put your settings.php file in sites/default instead of sites/www.example.com; however, it sounds like that is not an option in your instance.
If you are using Drupal 7, it could be that CTools has taken on the practice that was common in Drupal 6, but for the most part this has been left behind in Drupal 7.
In any event, it sounds like you have your workaround.  Stream-editing an SQL dump is inelegant, but since "dev" and "www" are the same length, it would be possible for you to do this.  (If the site names were different lengths, this would not work, because serialized strings contain length bytes, and the path to the sites folder probably appears in serialized strings.)  I really wouldn't recommend it -- better to try to overcome some of the constraints you've mentioned -- but if you're really stuck, it beats not being able to make a dev site.  I will also advise you to only sync from www.example.com to dev.example.com to avoid overwriting user content.  Look into the features and ctools export bonus as possible alternate ways to move configuration from dev to live without using sql-sync.


Is it possible to force a computer to join a domain when connected to the network?

Question: Is it possible to force computers, who are connected to the local network, to join the domain?
Comment: define force better, there is no way to force this unless you already have the local admins pw.  If you do then you can do the netdom trick (excluding fw issues).
Comment: Do you control these computers, or at least know a consistent administrator username/password on them?
Answer: The "force" term is not enough precise to provide specific help.
You can use netdom from a central computer to join them to the domain remotely (providing both local admin account and a domain account with right to join computer).
Firewall on remote computer may block netdom from connecting and join it to the domain.
Comment: When a client is connected, it should automatically join the domain.
Comment: I see 2 ways: 1/Do a batch that scan computer on network and try to join them (assuming same local admin account all time). 2/Do your own master, so you can join domain during silent automated install of Windows. This is what companies does, for example using MDT from Microsoft which is free.
Answer: Yes.
Any computer for whom you have administrative privileges can be made part of the domain on command.
If you don't have local administrative privileges, you cannot make the machine become part of the domain.
Assumes you have certain administrative privileges within the domain.  You can be part of the admin group, or the right to do this can be assigned granularly.
Netdom.exe can be used for this purpose:
NETDOM [ ADD | COMPUTERNAME | HELP | JOIN | MOVE | QUERY | REMOVE |
         MOVENT4BDC | RENAMECOMPUTER | RESET | TRUST | VERIFY | RESETPWD ]

like:
netdom add /d:reskita mywksta /ud:mydomain\admin /pd:password


Does it make sense to create a database user for each system admin at my company, or should we share a database account?

Question: Should I be creating a Mongo user for each physical developer that needs access to MongoDB, or should we just share a "sysadmin" account?
There's three of us that need full access, and everyone needs the same permissions. We only have a few thousand users and don't have any special compliance requirements (not healthcare, finance, etc.). Everyone on my team I deem trustworthy and competent. We all have sudo access, and there are no other Linux accounts on the server (besides apache, etc.).
I'm considering this since it's a lot more work to set up multiple users and there is negligible benefit. (If a team member left, I'd just change the password). However, if there is a real reason to create separate users (if it helps post-breach auditing/investigations, etc.), I'd be more than happy to do so.
If we shared I would create a simple bash script to "wrap" the mongo commands, like this, and add it to a group only given to us three developers:
#!/bin/bash

/path/to/bin/mongo -u="username" -p="password" $@

This would prevent the password from ever leaving the server in the first place.

I was originally considering the security implications of writing a script like the one above that has a unique password for each user and having the file owned by them with 700 permissions. The assumption would be that keeping the password "hidden" from the user would be safer than having it on their clipboard/stored on their local computer in some random password.txt file on their desktop.
But, I started thinking: there's no reason to have three separate accounts if the three of us could just sudo cat to see the other password anyway.
I think making each user keep a password locally with a password manager is overkill as, since they already have sudo, it's easy enough to just disable authentication in the Mongo config file in the first place.
Answer: A shared account means it's harder (r impossible without proper system/access logging) to determine who actually logged if something goes wrong.
Also if one of those people is terminated/quits, you can't remove ONLY their access, you'd have to change the password for the shared account, as you said.
If people are really storing passwords in clear text on their systems, that a whole other security can of worms to address.
I'm not sure why you think 3 separate accounts is considered a significant amount of work, as you probably spent more time writing this question.
Comment: `If people are really storing passwords in clear text on their systems, that a whole other security can of worms to address.` We use a password manager with randomly generated passwords for all of our production API accounts. However, after talking with my team more, they say a password manager specifically for MongoDB is a pain as we each authenticate at least half a dozen times a day. One of the other developers claims we're too small of a target to worry about a MongoDB password being stolen from a disk, and if someone had local access to steal such a password, they could easily install...
Comment: ...a keylogger/copy our private keys and gain access to the server directly. I think storing the password with 700 permissions on the same server that runs the database is low risk as an attacker who could read a 700 file and use that password would already have to have root access (and it takes three minutes to disable authentication with MongoDB if you have root access). I'd personally prefer the password manager, but I respect the other developer's wishes to not lose productivity to protect against such a rare scenario, and I think keeping it on the server is a fair compromise. Am I wrong?


Cross namespace communication in minkube on ubuntu 20.04

Question: Hy guys, please I need some help; I'm using Kubernetes in minikube on Ubuntu 20.04 with ingress add-on enabled. I want my NextJS service to communicate to my Express Service connected via ingress server by making request only via Ingress controller. So when I use cross namespace communication it doesn't work as illustrated in the tutorial presented. Since ingress-nginx in minikube runs on kube-system namespace.

$ kubectl get service -n kube-system

NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
ingress-nginx-controller-admission   ClusterIP   10.103.20.47   none          443/TCP                  15d
kube-dns                             ClusterIP   10.96.0.10     none          53/UDP,53/TCP,9153/TCP   36d

The services that I am running
$ kubectl get services

NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
auth-mongo-srv   ClusterIP   10.96.2.86       <none>        27017/TCP   20m
auth-srv         ClusterIP   10.106.150.115   <none>        3000/TCP    20m
client-srv       ClusterIP   10.106.45.17     <none>        3000/TCP    20m
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP     26h

Expected request url was: http://ingress-nginx.ingress-nginx.svc.cluster.local(Communication through nginx)
But instead I have to go through http://auth-srv:3000 (Which is direct communication between services)
My Ingress Config file
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
  rules:
    - host: ticketing.dev
      http:
        paths:
          - path: /api/users/?(.*)
            pathType: Prefix
            backend:
              service:
                name: auth-srv
                port:
                  number: 3000
          - path: /?(.*)
            pathType: Prefix
            backend:
              service:
                name: client-srv
                port:
                  number: 3000

Comment: All services listed are using Cluster IP service type. I recommend taking a look at the following post for a comparision between available K8s service types, and the DNS names available for each service type:
https://stackoverflow.com/questions/41509439/whats-the-difference-between-clusterip-nodeport-and-loadbalancer-service-types
Comment: Can you update your questions with your ingress definition?
Comment: @thomas, I've updated it with the Ingress config file. Thanks.
Comment: It is still not clear to me where the NextJs service is located. Is it inside the cluster or running in the host machine? Can you provide output from the command `kubectl get pods,svc --all-namespaces --show-labels`. You mentioned some tutorial that you are using? Can you also provide some reference to it?
Answer: I see this is Stephen Grider course about microservices.
As a temporary solution, you can try to do next:
$ kubectl get ingress

Take an IP Address.
You can use it as next link in your application.
192-168-49-2.kubernetes.default.svc.cluster.local

Instead of ingress-service.default.svc.cluster.local
====
More detailed information if needed.


Can you Help me Understand this Chord Progression | Gospel

Question: I keep hearing the same little chord progression all over the place and I want to understand its function from a music theory perspective. Also I want to understand if there's some cultural reason why it keeps reappearing (like is it a staple of the genre and artists are referencing it, or something to that effect?).
Here are the places I have found it.
It appears in at 0:02 of Daniel Caesar's We Find Love. There are three chords in the first two seconds; I am interested in the second and third.

At 0:40 of Chance the Rapper's Same Drugs. It happens on the more of "same drugs no more"

. I managed to find a piano roll of someone playing this song and I transcribed (poorly) the chords.

Also I have found the chord progression at 1:04 of this performance of Love in Need of Love Today.

I think there might be slight differences in the chord tones between these examples (my ear isn't the best), but they all have that staccato chord into the longer one. It sounds so distinctly gospel; I love this progression. I just want to understand it!
Comment: Your examples have the same rhythmic motif (syncopation)  but the progressions aren't all the same.  The triad of the tonic is DoMiSo and the progression I-IV is Do - Fa  in the bass line. What you hear is the first inversion of the tonic (I6 = MiSoDo).  So the bass line you hear is Mi-Fa. http://musictheory.alcorn.edu/Version2/theory1/invert.htm
Answer: In the bass line you  hear the 3rd of the chord which is the tonic of the song. This third is resolving (leading) in the root tone of the subdominant, which is the fourth degree of the song (IV).
Your 2nd example (notation) should be in Gb major.  The natural B in the last chord is a Cb (C flat), the subdominant in Gb major.
The progression you identify is a bass line mi-fa (I6-IV), whereby the eighth note of the syncopation is the 3rd of the tonic (this note is the bass tone when we have the 1st inversion of the tonic.
In classic Roman number we write I6. This means C/E in C major,  respectively Gb/Bb in Gb major  or F#/A#.
(The last example has a similar rhythmic motif but the harmony is different: ii - I6).
Edit:
May be you have edited your question or I have overseen this last comment to your 3rd example:

I think there might be slight differences in the chord tones between
  these examples (my ear isn't the best), but they all have that
  staccato chord into the longer one. It sounds so distinctly gospel; I
  love this progression. I just want to understand it!

I think I have understood that what is fascinating you is rather the rhythmic figure than the chord progression itself:  they all have that staccato chord into the longer one
that staccato chord into the longer one is called syncopation.
You are right: this syncopation is a specific feature of black gospel music!
 The European audience was fascinated by the syncopation of the afro-american music in the second half of the last century (e.g. Mahaliah Jackson: Down by the riverside) and this is certainly a cultural feature - as well the syncopation is an element of the English language.
(I remember we sang a song that was .. .. (short long short) but it was adapted as: _.. (long short short).
Now as the other answers say it is used in passing chords,  that's correct!
(The first thing I've copied from the guitar songsters was that they filled the cadence I-IV-V-I with passing tones. May be you know: This old house (Rocky Docky) or Lean on me (Someone to lean on). These tunes are both just built on passing tones between the cadence! Here you can study the passing tones and adapt them to other songs: you just play Do Do Do Re Mi Fa FA Fa Mi Fa So So and then you syncopate the bold chords and you have what you were looking for. (Mind the inversion I6 (MiSoDo=3-5-8) of the tonic!)

I can tell you other situations when this syncopation is used in Gospel music:

F/G (this means F above G)  I think that's what is really happening in your last example.
C E7 am (E7 as dominant of the relative chord)
C E F F#dim7 C/G  (F#dim7 as secondary viio7 of G

here are some other examples of syncopations:

and here you can see how Eric Clapton sings the syncopation while Pavarotti is not so friend with it.

I always thought Elton John has been inspired by Gospel. You can discover plenty of syncopation and passing chords:
Answer: The chord you're looking for is a basic major chord, but with the third in the bass. For C it would be C/E.
The important thing to realize is that the chord's root note doesn't have to be the bass note. If you have some other note in the bass, then it will work mostly just like the original chord, but it will additionally have some properties of chords with that bass note. For example, C/G ... in C it's kind of a tonic chord (home chord), but it also partly feels like a G chord. Halfway to a dominant. A common progression to end a song is C/G - G7 - C. First you almost give the listener the final high-point, the dominant chord, before really giving it in the G7. Prolong the excitement!
How to come up with chords like C/E? Practice them in various places. In any song with chords, try to play the song with different chord tones in the bass, and that way you learn how it changes the harmonic feeling and your expectations of "what chord could come next".
How to use those chords in progressions: usually you want to have some sensible movement in your voices. The lowest voice, the bass is a very important voice. Another important voice is the melody, which is usually the highest voice. Move those two voices according to taste - though usually the melody is given to you - and set the in-between notes to taste. For more on that see my answer to How to determine passing chords
Here's an example of a movement with the melody going E-F-G, and the bass C-D-E. You have many options for what happens in between. Ask yourself: what four-note chords have a G and an E, and E as the bass note? Well, at least C/E and Em and Em7!

All of the progressions start from C major, but what are the other chord options there? How would you analyze their chord roles in C major?
Answer: We could call this a passing chord, accented through a syncopated rhythm.   Yes, a common feature of gospel music.


Creating sub-fields within a field

Question: I am using the profile2 module in Drupal 7. I have created a field using the check box/radio widget that has 2 options and both options can be checked. How would I be able to create a sub-field that will show another checkbox/radio widget with options depending on what was checked in the main field?
Answer: For Drupal 6, there was Conditional Fields.
There is no Drupal 7 version yet, but there is an issue which links to field conditional state which at this point is only a sandbox project but seems to be widely used already.
Comment: There is also a [development snapshot for Drupal 7](http://drupal.org/node/1137614) of Conditional Fields.

Creating sub-fields within a field


How to sort by condition in entityQuery

Question: I want to get a list that the items with a specific term in the top (like promote). The query would be like:
<code>$query = \Drupal::entityQuery('node')
  ->condition('status', 1)
  ->condition('type', 'news')
  ->sort('field_product_category', $tid, '<>') //Error <>
  ->range(0, 5);
</code>
Can I put calculation in sort condition? I do not want to make 2 queries.
Comment: Do you only care about having the items with a specific term at the top and then you don't care about the sorting of the rest? Because if that's the case I would just load all the nodes into an array and sort the array by that key after the fact.
Answer: I don't think you can sort that way with an entity query.  But with a select query, you have more flexibility.
I would try something like this:
<code>$database = \Drupal::service('database');

$query = $database->select('node_field_data', 'nfd')
  ->fields('nfd', ['id']);

$query->condition('nfd.status', 1);
$query->condition('nfd.type', 'news');
$query->range(0, 5);

$query->leftJoin('field_product_category', 'pc', 'pc.entity_id = nfd.nid');
$query->addExpression('pc.field_product_category_target_id <> :target_tid', 'is_featured', [':target_tid', $target_tid]);

$query->orderBy('is_featured');
$query->orderBy('nfd.changed');

$matches = $query->execute()->fetchAll();
</code>

How to sort by condition in entityQuery


Rendering for AJAX

Question: Within my module I have a button that calls an ajax request. The controller for this request is to collect some data, then render a Twig template using that data.
Using Drupal 10, how do I render the data/twig file, assign it to a variable, then send it back so the JS script can then place the code in the page?
Thanks.
Comment: See https://drupal.stackexchange.com/questions/231188/how-to-load-content-in-a-div-with-ajax for some examples. I'd personally recommend following https://drupal.stackexchange.com/a/255476/2800 rather than the accepted answer. The accepted answer isn't necessarily wrong, but using the AjaxCommand classes is a much better solution
Comment: Please edit the question to add the code you've written so far, what you expect it to do, and where it's failing specifically, plus any error messages you see. To make a good question for this site it's important we can understand what the exact problem is, otherwise any answer would just be a vague end-to-end tutorial, which doesn't suit the Q+A format here
Comment: No extra code necessary. In the past Drupal 7 used a method - render() - I need to know the new way Drupal 10 does this so I can return the marked up template back to the ajax calling the controller.
Answer: This is how I got it to work. Thanks to Angrytoast on Slack for the pointer!
<code>public function paginateListing(Request $request) {
$rc = new RemindersController();
\Drupal::service('page_cache_kill_switch')->trigger();
$activeReminders = $rc->getAllActiveReminders($request->get('numrows'), $request->get('goto'));
$tableData = $rc->getReminderStatus($activeReminders);
$build = [
  '#theme' => 'rmdrs_table_listing',
  '#activeReminders' => $tableData,
  '#pagination' => array(
    'page_now' => (int) $_SESSION['rmdrs']['pager']['page_now'],
    'next_page' => (int) $_SESSION['rmdrs']['pager']['next_page'],
    'previous_page' => (int) $_SESSION['rmdrs']['pager']['previous_page'],
  ),
];

$output = \Drupal::service('renderer')->render($build);
return new JsonResponse($output);
</code>
}
The service is what I needed, along with building a $build array. I also had to create an entry in my .module file's hook_theme method.
<code>'rmdrs_table_listing' => array(
  'variables' => array(
    'activeReminders' => null,
    'pagination' => null,
  ),
),
</code>
Comment: If the render process is final and you return the result in a response use `\Drupal::service('renderer')->renderRoot($build)`. Otherwise, the controller causes an exception if the build array contains cache data or libraries. render() is only for rendering at the theming level.

Rendering for AJAX


Selectively displaying data into the Long Text Viewer

Question: I have four levels of report filters: Month, Year, Region and Country, all of which were set as Page Level and Report Level filters.
The data is structured like so:

And basically, this is how it looks like if all slicers are set to All:

This is obviously not helpful and doesn't really give proper context to the data if viewed by anyone since it just listed down everything in the Comment column.
So is it possible to selectively display data from the Comment field into the Long Text Viewer (ie: display nothing if all Slicers were set to All, only display comments for the particular country/month, etc.)?
Answer: Depending on how you want the exact display logic to be, you may find the HASONEFILTER() and HASONEVALUE() function useful in this case.
The following is an example where a new measure is created, and it'll display the comment (concatenated by newline which will be handled later) when there are any filters applied to the column Month, Year or Country; or it'll return an empty string.
Useful comment =
IF(
    HASONEFILTER(Data[Month]) || HASONEFILTER(Data[Year]) || HASONEFILTER(Data[Country]),
    CONCATENATEX(Data, Data[Comment], "\n"),
    ""
)

The Long Text Viewer provides a format option called Newline starts with, which allows us to split the measure we created (which is a long concatenated string containing all filtered comments) back to separate lines for display purpose.

The following shows the result when no filter is applied:

The following shows the result when filter is applied:

EDIT:
Same result when slicer is a dropdown:

Another example, display nothing when year is 2016:
Useful comment 2 =
IF(
    CONTAINS(DATA, DATA[Year], "2016"),
    "",
    CONCATENATEX(DATA, DATA[Comment], "\n")
)

When Year = 2016:

When Year = 2017:
Comment: Hi Foxan! Thank you for the help! I checked this with my visualization and I think this only works with list slicers and not dropdown list slicers. Is it possible to check for the value of a slicer? Ie: if the year slicer = 2016 then do not display anything?
Comment: @RemiDarren List or dropdown slicer is just a display difference. It has nothing to do with the logic behind so the solution should work as is.
Comment: @RemiDarren I've also added example to the Year = 2016 question. It's easier to think in "table" rather than think in "slicer". When you pick a value in the slicer, you're applying filter to the particular table. So in the measure, you can write something like "If the particular column contains such value (after filtering), then..". Essentially, you're checking the value against the table but not the slicer. Please check and accept the answer if it works for you.
Comment: Okay, this one worked for me: `IF(HASONEFILTER(Productivity_Data[Country]) && HASONEFILTER(Productivity_Data[Month]), CONCATENATEX(Productivity_Data, Productivity_Data[Comment], "\n\n"), "Please select only one Country from the list")`.
Comment: @RemiDarren Can you accept the answer if it solves your problem?


What is the earliest we know wire-strung instruments to have existed?

Question: What is the earliest we know wire-strung instruments to have existed? I was reading about gut strings, nylon strings, wire strings used on guitar, and wanted to know, but couldn't find the answer where I thought I might.
Comment: One thing is certain: wire strings existed before nylon did.
Answer: From Wikipedia: Piano wire:

The oldest record of wire being made for musical instruments is from Augsburg in 1351.

This is based on Alfred Dolge's book Pianos and Their Makers: A Comprehensive History of the Development of the Piano from the Monochord to the Concert Grand Player Piano (1911), in which he writes:

Records tell us that iron wire for musical instruments was drawn from Augsburg as early as 1351. (page 123, Google books)
Comment: Any mention elsewhere of first use on guitars, lutes, violins, etc?
Answer: An early wire strung instrument that helped promote using metal strings on guitars was the X braced Martin guitar.  From Guitar .com "X-bracing was considered unsubtle by most guitar makers in Europe at the time, who favoured a fan bracing pattern, but Martin broadly stuck with it, and it would eventually prove a masterstroke. As the guitar became more popular as the 20th century dawned, so did the demand for more volume, which led to the introduction of metal strings instead of the traditional catgut."


Duvida sobre xamarin-forms

Question: Boa noite,
Estou iniciando um projeto de TCC com desenvolvimento para plataforma mobile e meu grupo estamos desenvolvendo utilizando a plataforma Xamarin forms, minha duvida é a respeito da criação do projeto, pois estamos tentando fazer uma comunicação via web-service e só estamos conseguindo adicionar as bibliotecas necessárias o Microsoft.Net.HTTP e a Newtonsoft utilizando o projeto cross-plataform que não conta com a opção de drag and drop para a criação das views, existe alguma forma de utilizar esse drag and drop para desenvolver as views com o cross plataform ou é apenas utilizando arquivos XML mesmo ?
Answer: Não é possível construir as views com drag-and-drop de componentes pois não há um editor de layout para isso.
O método de construir as views é realmente através do XAML (e é muito melhor do que parece em um primeiro contato).
Para começar, você pode utilizar ferramentas de pré-visualização instantânea de XAML como o Forms Previewer ou o Gorilla Player (Este segundo é o mais popular).
Neste artigo, Adam Padley dá umas dicas de como o Gorilla Player funciona.
Comment: Ah, ok, obrigado pela informação

Duvida sobre xamarin-forms


Why exception when only the last two strings are empty?

Question: Can someone explain the below behavior?
In [23]: l = ['', 'Camino Cielo', '', '']
In [24]: ll = ['', 'Maricopa', 'Highway', '']

In [26]: ' '.join(e for e in l if e)
Out[26]: 'Camino Cielo'

In [27]: ' '.join(e for e in ll if e)
Out[27]: 'Maricopa Highway'

In [29]: glue = ' '
In [30]: '%s'.join(e for e in ll if e) % glue
Out[30]: 'Maricopa Highway'

In [31]: '%s'.join(e for e in l if e) % glue
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
31-7ea8ccb65d69> in <module>()
----> 1 '%s'.join(e for e in l if e) % glue

TypeError: not all arguments converted during string formatting

Answer: The % operator is applied to the result of the str.join() call:
>>> '%s'.join(e for e in ll if e)
'Maricopa%sHighway'
>>> '%s'.join(e for e in ll if e) % glue
'Maricopa Highway'
>>> '%s'.join(e for e in l if e)
'Camino Cielo'

Note that there is no %s in the last result there; there is only one string in the output of the e for e in l if e generator expression, the rest are all empty and don't pass the if e filter.
With no placeholder, you cannot interpolate glue either:
>>> 'Camino Cielo' % glue
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting

You'd also have a problem if you had more than two strings to join:
>>> '%s'.join(['foo', 'bar', 'spam'])
'foo%sbar%sspam'
>>> '%s'.join(['foo', 'bar', 'spam']) % glue
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not enough arguments for format string

In this case, just use glue directly:
>>> glue.join(e for e in ll if e)
'Maricopa Highway'
>>> glue.join(e for e in l if e)
'Camino Cielo'
>>> glue.join(['foo', 'bar', 'spam'])
'foo bar spam'

There is little use for interpolating when all your template consists of is '%s'.


compare and show significant differnce between arrival times (poisson distribution)

Question: I have the number of arrivals per hour to a center before and after the implementation of a scheduling software that allows clients schedule their own appointments.
Both distributions are Poisson with high arrivals at the beginning of the day that reduce as the day proceeds. The goal of the program however is to the reduced the variation and have people show up more evenly through the day. Because these aren't normal, I can't just compare standard deviations or any of the normal stats tools of significance.
Is there a way to normalize the curve? (using the mean arrival per hour / log??). Or are they any other tools I can use to show the significant reduction in variability of arrival times. thank you.
Answer: (per request)
You are trying to look at the difference between two things.  When I compare two things I like to use a box-plot, or a violin plot.
Your data IS going to be different than what I simulate.  The distributions are the weakest examples of how to compare two things.  I think you are looking to compare the overall distribution, and that you want to move the peak from "opening" to later in the day.  You also want to move the "whiskers" in - aka spread out the bulk of the arrivals.
So this suggests wait time can be approximated using negative binomial distribution.  I'm going to use it to describe arrival times.
Here is some toy code:
<code>set.seed(1)
library(vioplot)

N = 1000 #how many samples

#arrival time before
before <- sort(rnbinom(n=1000, mu = 5, size = 24))

#arrival time after
after <- sort(c(runif(n=N/2,min = 0, max = 12.25),
        rnbinom(n=N/2, mu = 5, size = 24)))

#comparison

vioplot(before,after,names=c("before","after"))
</code>
Here is a toy plot:

This gives you a very initial way to explore your problem.  Look at the center of the plot, the shoulders, the whiskers, and the "bout" of the violin.
Questions you should ask yourself:

How many people are in each different contributing function of the "after"?  It should change over time.  Sample size drives detection of change.  Nature of the different mode also drives detection of change.  Things change over time.  In the winter in Wisconsin a clinic is going to get online scheduled visits much more reliably than in summer - because 20-below is really really cold.
What is the new "mean" or defining parameters in the "after" function?  What does it have to be in order to see a "statistically significant difference"?  If everyone does it but the parameter change is small, then it drives detection.
What does it have to be to see a "technically significant
difference"?  Technical and statistical aren't always the same.  A 1% change that math likes isn't going to change your bottom line by 50%.
Comment: It is difficult to see how these boxplots could help evaluate *variation* in arrival times *over the course of each day*.  Could you explain how that would work?
Comment: @AntoniParellada - I am time constrained at the moment.
Comment: +1, but I was looking forward to a generalized linear model being simulated. I wonder if you opted not to spell it out, since it is already in the reference you provide (either Poisson or NB).


Semi-supervised classification objective from Kingma et al

Question: In this 2014 paper, Kingma et al. develop different methods to do semi-supervised learning with VAEs. In one of their proposed solutions ("M2"), they approach this problem by incorporating the labels (y) as a causal factor in the generative model, s.t. images depend (through a generative model parameterized by a neural network; the decoder of the VAE) on both these labels (e.g. MNIST digit classes) and other latent variables (z). The labels (y) are partially observed: for some items in the data set, we have labels, while for others we don't. The other latents are never observed - discovering them is part of the learning process.
They then derive the ELBO loss for a VAE with this generative model. They observe that, for labeled images, this loss does not end up training the encoder network to produce good class labels (since the class labels are given for these cases, the ELBO does depend on the decoder's ability to use these class labels to reconstruct good images, but not on the encoder's ability to generate good class labels). This is undesirable, as it means that the classification part of the network effectively only learns from unlabeled examples. So, they augment their loss function with a classification loss (equation 9), which is basically just the cross-entropy between the class probabilities produced by the encoder, and the true labels, but scaled by a certain factor.
They then say the following:

While we have obtained this objective function by motivating the need for all model components to learn at all times, the objective 9 can also be derived directly using the variational principle by instead performing inference over the parameters $\pi$ of the categorical distribution, using a symmetric Dirichlet prior over these parameters.

And this is where they lose me. I would really like to understand how such a derivation would work, as I assume this also produces the scale factor that they apply to this term in the loss (which otherwise seems like a hyperparameter that would need to be tuned empirically). However, I just don't understand what they mean with this sentence.
Specifically, $q_\phi(y|x)$ is a categorical distribution produced by the encoder. Its parameters are just the category probabilities, which they denote by the vector $\pi_\phi(x)$. Thus, the entries in $\pi$ simply are the probabilities in $q_\phi(y|x)$. So "performing inference over the parameters $\pi$", presumably just means "estimating those numbers", or perhaps "optimizing variational beliefs over those numbers". I just don't see how you get from that idea to a loss term $E_{\tilde{p_l}(\mathbf{x},y)}\left[ -\log q_\phi(y|\mathbf{x}) \right]$.
Can anyone explain this?
Answer: I think they're describing a VAE with $\pi$ and $z$ as latent variables, and $x$ and $y$ as observed variables. The prior on $z$ is normal, as usual, and the prior on $\pi$ is $\text{Dirichlet}(\alpha)$.
(As a note, I don't think they mean literally that the objective in eq. 9 will exactly fall out of this model. I think they're just suggesting that something similar looking can be derived, as is described below.)
The generative process remains the same, as described by eq. 2. The evidence lower bound becomes:
$$
\begin{align}
\log p_\theta(x,y) &\geq \mathbb{E}_{q_\phi(z,\pi|x,y)} \left[ \log p_\theta(x,y|z,\pi) \right] - \mathcal{D}_{KL}\left[ q_\phi(z,\pi|x,y) \vert \vert p(z,\pi)\right] \\
&\geq \mathbb{E}_{q_\phi(z,\pi|x,y)} \left[ \log p_\theta(x,y|z,\pi) + \log p(z,\pi) - \log q_\phi(z,\pi|x,y) \right] \\
&\geq \mathbb{E}_{q_\phi(z,\pi|x,y)} \left[ \log p_\theta(x|y,z)+\log p(y|\pi) + \log p(z) + \log p(\pi) \\- \log q_\phi(z|x,y) - \log q_\phi(\pi|x) \right]
\end{align}
$$
This looks very similar to eq. 6 -- in fact, we only have three extra terms*:
$\log p(y|\pi) + \log p(\pi) - \log q_\phi(\pi|x)$. The last two terms are the KL divergence between the posterior and prior distributions over $\pi$, which has a tractable form in the case that they're both dirichlet, or just an irrelevant constant if $q_\phi(\pi|x)$ is a degenerate point distribution.
The first term $\log p(y|\pi)$ is essentially the classification loss. In eq. 9, it's written as $\log q_\phi(y|x)$, but really it's the same thing, just written differently because the authors have $y$ as a latent, and infer it from $x$, whereas in the proposed model, $\pi$ is the latent which is inferred from $x$, and $\pi_k$ is the probability which the model assigns to the label $y$ being from the k-th class. Hence, we have the typical classification log-loss.
*There's one additional term in eq. 6, $p_\theta(y)$ which is missing from my derivation -- the authors don't seem to define this anywhere, so I'm not sure where it came from, but I think it's just a red herring.
As for $\alpha$ in eq. 9, this is probably a random hyperparameter they threw in, and not at all related to the $\alpha$ parameter of the dirichlet prior.
Comment: Well, if you're lucky and there are only a small number of classes $y$, this should only be a small amount of extra computation. Alternatively, it can be lower bounded by $E_y[\log p(x|y,z)]$, and you could just use a one-sample estimate for this.
Comment: Thanks! This was essentially my thinking too, but I was thrown off by the wording which suggests that the precise objective of their eq. 9 may be derived this way. From that perspective, I also couldn't explain the expectation under $\tilde{p}_l$, rather than under a belief $q$, but I think that falls out of having $q(\pi)$ be a point mass as you suggested. The expectation under that $q$ then disappears, and the other expectation is just because we sum over items in the dataset (and not directly related to the ELBO).
Comment: Actually one thing is still not clear to me: how do you adapt this to the case where y is sometimes unobserved, e.g. in semi-supervised training (which is what they do in the paper)? Because you now get $q(\mathbf{\pi})$ out of your inference network - not $q(y)$. So you now need to evaluate $\log p(\mathbf{x}|\mathbf{\pi},\mathbf{z}) =\log\int p(\mathbf{x}|y,\mathbf{z})p(y|\mathbf{\pi}) $ which seems intractable.


How to fetch database table column names in java

Question: I need to display all the columns from the table in java. So far I have tried resultSet java object but it is not working for me
public class dbtest {

    public static Session session = null;

    public ArrayList < String > ColValue() throws IOException {
        // TODO Auto-generated method stub

        FileReader reader = new FileReader("Configuration\\Config.properties");

        Properties p = new Properties();
        p.load(reader);

        String serverIp = p.getProperty("ServerIp");
        String keyspace = p.getProperty("ServerPwd");

        Cluster cluster = Cluster.builder().addContactPoints(serverIp).withCredentials("user", "password").build();
        session = cluster.connect(server);
        String query = null;
        String cqlStatement = "select * from table_name";

        ArrayList < String > casVal1 = new ArrayList < >();
        for (Row row: session.execute(cqlStatement)) {
            if (!casVal1.contains(query)) {
                casVal1.add(query);
            }
        }
        System.out.println(casVal1.size());
        System.out.println(casVal1);

        return (casVal1);
    }
}

Need all the columns in results as outcomes.
Comment: are you sure you have provided correct code as you mentioned in question ? I dont see usage of resultSet
Comment: Yes but it was not working
Comment: @Dhanraj what he is using is not JDBC, it is DataStax Cassandra Driver
Comment: Oh yes.. sorry .. I misundertood
Comment: If this is not [JDBC](https://en.wikipedia.org/wiki/Java_Database_Connectivity), make that clear in the body and title of your Question. The Question and comments make this quite unclear.
Answer: You can get the column via the ResultSet.getColumnDefinitions Method
com.datastax.driver.core.ResultSet result = xxxxx;

for (Definition definition : result.getColumnDefinitions()) {
    String name = definition.getName();
}


Como preencher um xml (request de um WS SOAP) com Javascript

Question: Tenho uma função Ajax a fazer um POST para um webservice SOAP e preciso de inserir valores nos elementos do XML de request com base no que o utilizador da página inserir numa input text. Como posso fazer isso?
O meu script é parecido com este:
var soapMessage = '<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://teste.transaction.service">\
                <SOAP-ENV:Header>\
                <ns1:testeHeader>\
                <channel>Portal</channel>\
                </ns1:testeHeader>\
                </SOAP-ENV:Header>\
                <SOAP-ENV:Body>\
                <teste:VC>\
                <teste:VCRequestHeader>\
                <PIN>1234</PIN>\
                </teste:VCRequestHeader >\
                </teste:VC_WSLOGIN>\
                </SOAP-ENV:Body>\
                </SOAP-ENV:Envelope>';
            $.ajax({
                type: "POST",
                dataType: "xml",
                url: wsUrl,
                data: soapMessage,
                crossDomain: true,
                success: function(data) {
                    console.log(data);
                },
                error: function(error) {
                    console.log(error);
                }
            });

Obrigado desde já pela atenção.
Comment: já tentou fazer um parser? `var parser = new DOMParser();
var soapMsg = parser.parseFromString(soapMessage, "text/xml");`
Answer: Para fazer isso primeiro você vai precisar:
1 - Primeiro: Declarar o campo HTML
<input type="text" id="cidade"/>

2 - Pegar o schema do xml utilizado no webservice
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:tns="http://www.webserviceX.NET" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" >
    <SOAP-ENV:Body>
        <tns:GetWeather xmlns:tns="http://www.webserviceX.NET">
            <tns:CityName></tns:CityName>
            <tns:CityName></tns:CityName>
            <tns:CountryName></tns:CountryName>
            <tns:CountryName></tns:CountryName>
        </tns:GetWeather>
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

3 - Chamar o método Js
var cidade = $("#cidade").val();
var soapMessage =
    "<?xml version='1.0' encoding='UTF-8' standalone='no'?>" +
    "<SOAP-ENV:Envelope xmlns:SOAP-ENV='http://www.w3.org/2003/05/soap-envelope' xmlns:tm='http://microsoft.com/wsdl/mime/textMatching/' xmlns:soapenc='http://schemas.xmlsoap.org/soap/encoding/' xmlns:mime='http://schemas.xmlsoap.org/wsdl/mime/' xmlns:tns='http://www.webserviceX.NET' xmlns:soap='http://schemas.xmlsoap.org/wsdl/soap/' xmlns:s='http://www.w3.org/2001/XMLSchema' xmlns:soap12='http://schemas.xmlsoap.org/wsdl/soap12/' xmlns:http='http://schemas.xmlsoap.org/wsdl/http/' xmlns:wsdl='http://schemas.xmlsoap.org/wsdl/' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:xsd='http://www.w3.org/2001/XMLSchema'>" +
        "<SOAP-ENV:Body>" +
            "<tns:GetWeather xmlns:tns='http://www.webserviceX.NET'>" +
                "<tns:CityName>" + cidade + "</tns:CityName>" +
                "<tns:CityName>" + cidade + "</tns:CityName>" +
                "<tns:CountryName></tns:CountryName>" +
                "<tns:CountryName></tns:CountryName>" +
            "</tns:GetWeather>" +
        "</SOAP-ENV:Body>" +
    "</SOAP-ENV:Envelope>";

var wsUrl = "http://www.webservicex.com/globalweather.asmx?wsdl";
$.ajax({
    type: "POST",
    dataType: "xml",
    url: wsUrl,
    data: soapMessage,
    crossDomain: true,
    success: function(data) {
        console.log(data);
    },
    error: function(error) {
        console.log(error);
    }
});

Comment: Muito obrigado, Emir! Era mesmo que isto que procurava.