Log transformation of input features for Random Forests and XGBoost

Question: I have read articles mentioning that Random Forests and XGBoost does not require scaling or normalisation of features as decision trees identify thresholds or cut-points in the features. Is there any benefit to log transforming data in the event that my features comprise outliers, or does this not effect the thresholds or cut-points selected by Random Forests and XGBoost?
Are the preparation of continuous features for Random Forests and XGBoost the same for classification and regression models?
Answer: Basically, all 1-1 transformations (e.g. log-, sqrt-, exp-, square-transformation of positive numbers, standardization, adding constants, multiplying by a constant etc.) have no effect for random forest, XGBoost, LightGBM and similar algorithms that pick cut-points in variables. Non-1-1-transformations like taking the square of numbers that could be negative (in that case suddenly -1 and +1 get both mapped to 1 and treated as the same) on the other hand of course make a big difference.
That's very different from regression models, where things are easiest for you if you transform your feature so that it's relationship with the outcome is linear (after accounting for the other features). E.g. when you do a Poisson regression with a log-link-function for number of phone calls received in a day in a call center, then I'd probably first try log(phone calls last week) as a predictor rather than the untransformed number. Standardization tends to be useful for two reasons: 1) with extremely different scales between some features you can get numerical issues with regression, and 2) if you want regularization techniques to treat different variables similarly, it's easiest if they are on the same scale.
This doesn't mean, at all, that variable transformations are unimportant for XGBoost, RF and friends, it's just that it's usually more (possibly non-linear) transformations across multiple variables (and maybe records) that you may want. E.g. ratios such as when you have miles driven and fuel used for a car, then L per 100 km (or miles per gallon) could be a useful feature, which a human can understand as potentially important and provide so the model does not need to figure this out.


How to check if object with no names exists in array?

Question: I am building a javascript function where I want to add an object to an array IF it is not already present, if it is I just want to update value.
My objects look like this:
[{31652237148248: 12}, {4365124714824: 4}]

How can I check the Array if there is an object with the ID of 31652237148248?
I tried this but it did not work:
var index = cartItems.findIndex((obj => obj[id] == id));

Comment: Your code is looking to see if the id is equal to the value of the object....
Comment: It is used for Shopify cart
Answer: Use the in operator to check if the id is a property of the object:

const cartItems = [{31652237148248: 12}, {4365124714824: 4}];

const id = '4365124714824';

const index = cartItems.findIndex((obj => id in obj));

console.log(index);
Comment: Worked perfectly fine!
Answer: You are looking to see if the value of objects property is equal to the id. 31652237148248 is never going to equal 12
so you can just do a type of
var index = cartItems.findIndex(obj => typeof obj[id] !== undefined);

you can do a truthy check - will fail if it is a falsey value.
var index = cartItems.findIndex(obj => obj[id]);

you can use object keys and includes or if first is equal
var index = cartItems.findIndex(obj => Object.keys(obj).includes(id));
var index = cartItems.findIndex(obj => Object.keys(obj)[0] === id);

A bunch of ways to do it
Personally a better way is just to use an object and not an array for the data.
var items = {31652237148248: 12, 4365124714824: 4}

const addItem = (id, count) => {
  cartItems[id] = (cartItems[id] || 0) + count
}

const getArray = () =>
  Object.entries(items).map(([key, count]) => ({ [key]: count }))

Answer: Try
let add = (obj,arr,k=Object.keys(obj)[0]) =>
           arr.some( o=> (k in o)&&(o[k]=obj[k]) ) || arr.push(obj)

let data = [{31652237148248: 12}, {4365124714824: 4}]

let add = (obj,arr,k=Object.keys(obj)[0]) =>
           arr.some( o=> (k in o)&&(o[k]=obj[k]) ) || arr.push(obj)

// TEST
add({31652237148248: 15}, data);
console.log('update',data);

add({666: 15}, data);
console.log('add',data);


Creating a Custom Content Listing Page based on specific content type or taxonomy terms selected in the node

Question: I am trying to create a content type that generates a list of content/nodes based on other content type or taxonomy terms selected within that node in the content type. I have a general content type for Articles and those are tagged with a taxonomy vocabulary for "Category".
This is within Drupal 9.
What I've done so far:
Create a new content type named "Custom List"
Add a field for an entity reference for Taxonomy Terms. Set the field to unlimited and selected the taxonomy vocabulary for Category. Then set the field to display as radio/checkboxes.
Created a View for Custom Lists as a Block. Set the block to display only on Custom List content types.
Set the View to list 10 items, with a contextual filter for Taxonomy Term ID, set to display all results.
What I want to happen:
When I create a new node for this Custom List content type, I will see the Categories taxonomy field I added as a list of checkboxes. I can check any number of taxonomy terms from this Categories vocabulary and the page will then display a content list of content that are only tagged with the terms I selected within this node. For example if I selected "News" and "Blog" as my terms, the list will only show content tagged with Category > News and Category > Blog.
Eventually I will be able to select Content Types as a filter on top of the taxonomy terms, but for now Im just trying to get the taxonomy filter working first.
What is actually happening:
The list generated still displays every content regardless of the term I selected when I created or edit the node. I have tried many things such as adding relationships from the Taxonomy Entity Reference field (Category) as well as the taxonomy term itself but to no avail.
Can someone help me with this issue or suggest a better or easier solution? Most likely my View isnt setup correctly as Im very inexperienced with contextual filters. I'm not even sure if I can achieve what Im trying to do with Contextual Filters. If there are any modules out there that can do what Im trying to do please advise or suggest them as well. Thank you all for your time.
Answer: For the Content Type side, you've already done what needs to be done. In summary, a Content Type with an Entity Reference field pointing to Taxonomy Terms.
The Views side is where it gets tricky, but here's what you need to do:

You need a View of Nodes filtered by your "Custom List" content type.

You need to add a relationship from your Custom List nodes to the Category terms. To do this, add the relationship named:
Taxonomy term referenced from field_category

In the relationship window, update the Administrative Title to "Categories" so that it's less confusing. It will help in the next step.

Here's the tricky part: You need to add a relationship to all the Nodes that are using your Category taxonomy. To do this, add a relationship called:
Content using field_category (Relate each Content with a field_category set to the taxonomy term.)

In the relationship window, update the Relationship field to "Categories" (the name we set in Step 2). Also update the Administrative Title of this relationship to "Viewed Node". It will help in the next step.

To round things off, you need a contextual filter to filter "Viewed Node" to the current node's ID. To do this:

Add a contextual filter. Filter by "Content", select the field named "ID".
Under Relationship, choose "Viewed Node" (the name we set in Step 3).
Under When the filter value is NOT available, select "Provide default value" and set the Type to "Content ID from URL".

Place the view block on your content type's page.

In summary, what the above did was link nodes to the terms, then terms back to nodes. Then, we just filtered the linked nodes to the current node. In SQL, it's something like the following pseudocode:
SELECT *
FROM nodes AS custom_list
INNER JOIN terms AS categories ON categories.id = custom_list.categories
INNER JOIN nodes AS viewed_nodes ON categories.id = viewed_nodes.categories
WHERE viewed_nodes.id = current_node.id

Or visually:
Nodes that appear on the view block
                v
            List nodes <- Category terms <- Nodes using the terms
                                                    ^
                                   Filter these by the current node's ID

Comment: Thank you so much! These steps were really easy to follow and you even explained the reasoning behind each which was very helpful. One thing I had to edit was Step #1, I had to leave the filter blank as content being pulled are different content types than "Custom List" but once that was removed it works like a charm. Thanks again you rock!


Active directory HyperV VM to VMware

Question: we just copied an active directory VM from an Hyper-V VM to a VMware one.
This Hyper-v VM is our second active directory server (the first one is a physical server). We used Symantec Backup Exec to restore our VM to VMware. Those 2 VMs are normally almost identical and the HyperV one is still the one in production.
We want now to switch off the Hyper-V VM and power on the VMware one, we intend to :

switch off the HyperV VM
switch on the VMware one and modify its IP to take the one from the inactive HyperV VM
reboot the vmware VM

Is there another thing to check before (i'm sure there are but i did not find it for this use case) ?
Answer: You can proceed as described provided that

you have performed a non-authoritative restore of the DC from a system state backup on the ESXi version of the DC
your backup is more recent than the tombstone lifetime of your forest (default is 180 days for DCs running Server 2003 SP2 or newer and 60 days for previous versions)

The non-authoritative restore part is crucial as otherwise your directory would see a USN rollback condition. Just restoring from an image (or reverting to a snapshot) won't do.
I would recommend watching this video regarding AD operations in virtualized environments as the snapshot-revert virtualization scenario is very similar to a simple image restore. VMWare also has a nice paper online which is (among others) covering the revert-to-snapshot topic for AD DCs.
In general, it is a safer and simpler approach to switch off the to-be-converted DC, convert the virtual disk to the destination format and switch it on on the new platform. It would incur additional downtime for the duration of the conversion procedure, though.
Answer: If i were you i would prefere to power down the Hyper-V VM, and then make a convert of the powered down VM via VMWare Converter Standalone editon. And then only power on the new VMware-VM without powering on the old hyper-V VM anymore.
I would feel more safe this way, but it's up to you.
There are also other tools like StarWind V2V Image converter.
The problem i see: if you used Backup Exec i think you made an online backup, right? I would't do that with a dc. i think you can get many problems with timestamps and synchronisation. But its only a guess...
Answer: I think everyone is overthinking this one.
Just install a new server and promote it on the VMware infrastructure. Then transfer any FSMO roles and decomission the old DC.
Comment: But our payroll software runs on this domain controller!!
Comment: you have a virtual environment and didn't made a new VM for the payroll software? Sorry but that isn't a very good concept i think. Can't you separate them? - If the software wasn't installed on DC i think MDMarra is right...

Active directory HyperV VM to VMware


dnssec remove keys from zone

Question: is there a way to remove all dnssec related stuff from a zone on a running  bind server?
I configured bind like like here described.
If i use rndc signing -clear all domain.tld nothng happens to the zone.
If i delete the dnssec signed zone via rndc delzone domain.tld
and recreate it via rndc addzone domain.tld ... , the domain.tld.jnl file with the related dnssec data will be automaticly recreated and the zone is dnssec signed again.
How can i remove all dnssec related data from a zone with rndc ?
Answer: (i use the inline-signing option to auto sign the zones in another file called {Zonename}.signed)
To remove all DNSSEC related Data of the Zone u have to remove the {ZoneName}.signed and {ZoneName}.jnl file of the Zone.
Remove also all the Key Files of the Zone (which should be in the keys - Directory) - otherwhise bind9 will autoresign the Zone.
After all the deletion it would be the clear way to rndc delzone {ZoneName} and reinititate the unsigned Zone with rndc addzone {Options without DNSSEC}

dnssec remove keys from zone


Python pandas apply function if a column value is not NULL

Question: I have a dataframe (in Python 2.7, pandas 0.15.0):
df=
       A    B               C
0    NaN   11             NaN
1    two  NaN  ['foo', 'bar']
2  three   33             NaN

I want to apply a simple function for rows that does not contain NULL values in a specific column. My function is as simple as possible:
def my_func(row):
    print row

And my apply code is the following:
df[['A','B']].apply(lambda x: my_func(x) if(pd.notnull(x[0])) else x, axis = 1)

It works perfectly. If I want to check column 'B' for NULL values the pd.notnull() works perfectly as well. But if I select column 'C' that contains list objects:
df[['A','C']].apply(lambda x: my_func(x) if(pd.notnull(x[1])) else x, axis = 1)

then I get the following error message: ValueError: ('The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()', u'occurred at index 1')
Does anybody know why pd.notnull() works only for integer and string columns but not for 'list columns'?
And is there a nicer way to check for NULL values in column 'C' instead of this:
df[['A','C']].apply(lambda x: my_func(x) if(str(x[1]) != 'nan') else x, axis = 1)

Thank you!
Answer: The problem is that pd.notnull(['foo', 'bar']) operates elementwise and returns array([ True,  True], dtype=bool). Your if condition trys to convert that to a boolean, and that's when you get the exception.
To fix it, you could simply wrap the isnull statement with np.all:
df[['A','C']].apply(lambda x: my_func(x) if(np.all(pd.notnull(x[1]))) else x, axis = 1)

Now you'll see that np.all(pd.notnull(['foo', 'bar'])) is indeed True.
Comment: note that `np.all([True, True])` will return `True`. I think that'll work for the OP.
Comment: @PaulH Thanks. I tried with `all` but neglected `np.all`. The problem is not with `[True, True]`, but rather with `False`. While `all(pd.notnull(None))` raises an error, `np.all(pd.notnull(None))` does not.
Comment: Thank you Korem, it works! What I wonder is that a single `pd.notnull(df['C'])` returns with `False, True, False` but not with `False, [True, True], False`.
Comment: @ragesz I don't know. You could ask a new question if you'd like.
Answer: I had a column contained lists and NaNs. So, the next one worked for me.
df.C.map(lambda x: my_func(x) if type(x) == list else x)

Answer: Also another way is to just use row.notnull().all() (without numpy), here is an example:
df.apply(lambda row: func1(row) if row.notnull().all() else func2(row), axis=1)

Here is a complete example on your df:
>>> d = {'A': [None, 2, 3, 4], 'B': [11, None, 33, 4], 'C': [None, ['a','b'], None, 4]}
>>> df = pd.DataFrame(d)
>>> df
     A     B       C
0  NaN  11.0    None
1  2.0   NaN  [a, b]
2  3.0  33.0    None
3  4.0   4.0       4
>>> def func1(r):
...     return 'No'
...
>>> def func2(r):
...     return 'Yes'
...
>>> df.apply(lambda row: func1(row) if row.notnull().all() else func2(row), axis=1)
0    Yes
1    Yes
2    Yes
3     No

And a friendlier screenshot :-)
Answer: Try...
df['a'] = df['a'].apply(lambda x: x.replace(',','\,') if x != None else x)

this example just adds an escape character to a comma if the value is not None
Comment: This code will return an error as NaN != None
Answer: Add the Following IF condition which returns NONE when it's TRUE
def funtion_name(input):
    if (pd.isnull(input)==False)
        return np.NAN
     //Rest funtion code//

Answer: The below will work for different data types.
df=

   col_1  col_2
0    1     NaN
1  three  seven
2   NaN    NaN
3  [4,5]    2

It can be done with map, for eg to replace NON-NULL values in col_1:
def my_func(n):
    return 'func'

df.loc[df['col_1'].notnull(), 'col_1'] = df['col_1'].map(my_func)
df =

    col_1  col_2
0    func   NaN
1    func   seven
2    NaN    NaN
3    func   func

Answer: If you have a string and want to apply function like this example:
'September 25, 2021'
df['Year'] = df['date_added'].apply(lambda x : re.split(' |,', x)[-1] if isinstance(x, str) else np.nan)
df['Month'] = df['date_added'].apply(lambda x : re.split(' |,', x)[0] if isinstance(x, str) else np.nan )

You can apply this way and use isinstance(x, str) to avoid NaN or any other type, you can also use type() like this.
df['Year'] = df['date_added'].apply(lambda x : re.split(' |,', x)[-1] if type(x)==str else np.nan )


Understanding the Central Limit Theorem (CLT)

Question: My question revolves around the Central Limit Theorem. To read from the same page, I quickly state it here:
Let $X_{1}, ..., X_{n}$ be IID with mean $\mu$ and variance $\sigma^2$. Let $\overline{X_{n}} = n^{-1} \sum_{i=1}^{n} X_{i}$. Then lets define
$Z_{n} := {\frac{\sqrt{n}(\overline{X_{n}}-\mu)}{\sigma}} \rightarrow Z$ where $\rightarrow$ means convergence in distribution, $Z \sim N(0,1)$.
The central limit theorem therefore does the following: We take the random variable $\overline{X_{n}}$ which as one can show: $\mathbf{E}(\overline{X_{n}}) = \mu$ and $Var(\overline{X_{n}}) = \frac{\sigma^2}{n}$. Then we rescale it, as if we would assume it is a normal distributed random variable, because for any normal distributed random variable $Y \sim N(\mu_n, \sigma_n^2)$ one can show that $\frac{Y-\mu_n}{\sigma_n} \sim N(0,1)$. And then, after we rescaled n (the rescaled version is exactly the statement of the CLT as written above), we notice that "by suprise" this scaled version of $\overline{X_{n}}$ is $\sim N(0,1)$. In other words, our assumption that $\overline{X_{n}}$ is normal distributed was right. Why does this suprise us? Because $\overline{X_{n}}$, or better to say the underlying $X_{1}, ..., X_{n}$ come from any distribution. Apparently the normal distribution is for this scaled version of $\overline{X_{n}}$ some kind of a "convergence goal", not considering where it started.
Fair enough. Until here we should be on track. Now here is my issue:
I frequently here from people, especially those who have not deeply studied statistics and use it for example in an application in business, that not just this very specific term $\overline{X_{n}}$ converges in distribution against a standard normal distribution, but also "any sequence" of random variables, as long as you draw enough of them. Let me firstly describe it a little bit intuitively (because that's how I here it from them), and than try to formalize it and understand what this would imply). So what they say is: you draw sufficiently many random variables $A_1, ..., A_n$ from any distribution (so n should be large). Then you plot the histogram of these n random variables that you have just drawn. Then, this histogram looks like a normal distribution (this is what they claim).
So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, ..., A_n$ would "form" a normal distribution. However, this would mean that at least from a certain value $n_0 > 0$ on, all $A_j$ with $j \geq n_0$ would be normal distributed. But does this make any sense? We know that they follow a different distribution, right? So their theroy is wrong, right?
Other questions related to this are: What can we imply from this very specific statement of the CLT about the average of random variables? Can we imply anything on the random variables themselves? Or some other statements? Is there any theorem that the above statement can be implied from, using CLT?
Thank you guys so much for your help! I would also appreciate any good literature that could help me understand.
Comment: Note that the distribution of the scaled version of $\bar X_n$ is *never* Normal unless the common underlying distribution of each $X_i$ already is itself Normal.  Furthermore, *no* assumptions whatsoever are made about the distribution of $\bar X_n$ in the CLT, nor does the CLT make any assertions about any particular $\bar X_n$.
Comment: Because I am not sure whether you are referring to this or not, I post this as a comment.
AFAIK the central limit theorem applies to the distribution of the means drawn from your data. This would mean (no pun intended) that if your sample $n$ is large enough, the distribution of **means** you calculate within different 'draws'/samples tends be normally distributed irregardless of the underlying population. E.g. 1000 times calculating the mean within a sample of 100 numbers drawn out of a uniform distribution (==not normal) will give a distribution of means which is approximately normal.
Comment: Check http://stats.stackexchange.com/questions/3734/what-intuitive-explanation-is-there-for-the-central-limit-theorem
Comment: @whuber Good reply! I just want to comment: By the berry esseen theorem, under a further assumption on the third moment, we also have a rate of convergence for the CLT. (This is fortunate since statisticians often use the CLT for finite sample inference.)
Answer:
So here is what I imply from their statement: If that would be true, than any sufficiently large sequence $A_1, \cdots,A_n$ would "form" a normal distribution.

It's hard to interpret this.  If you want to get a single random variable, then you need some operation to reduce this sequence of many random variables down to one.  The classical central limit theorem says that, roughly, if you use the operation "sum", then the result will tend to get more normal in distribution as you add more summands.
Another possible interpretation is that they are saying the joint distribution tends to become a normal random vector, but this is clearly absurd.  For example, if we take a sequence of independent coin flips, the joint distribution of all the flips is uniform over all the $2^n$ possibilities.  Somehow I doubt this is what they were going for though.
Finally there is the thought that in any sufficiently long sequence of random variables, the ones at the end must eventually be normal.  This would mean that there are a finite number of non-normal variables, that we must somehow "use up", and we would only be left with the normal ones.  But I built a robot to flip a coin as long as I tell it to.  Look, it's there doing it right now.  Flip, flip, flip...
As stated, your friends belief is "not even wrong" (to quote Fermi), it just makes no sense.
Comment: The conjecture as described by op must be wrong, there are many well known sequences of random variables which do have limit distribution that is normal
Answer:
"I frequently here from people, especially those who have not deeply
  studied statistics and use it for example in an application in business, that
  not just this very specific term $\bar X_n$ converges in distribution
  against a standard normal distribution, but also "any sequence" of
  random variables, as long as you draw enough of them."

The Central Limit Theorem applies to sums of random variables properly centered and scaled. Of (almost) any distribution indeed. In its basic form, if we have a collection $\{X_1,...,X_n\}$ of random variables, define $S_n = \sum_{i=1}^n X_i$. What usually converges to a Standard Normal is the quantity
$$\frac {S_n -E(S_n)}{\sqrt {\text{Var}(S_n)}} \to_{d}\mathbf N(0,1),\;\; n\to \infty$$
Any "description of an assembly" of random variables that can be mapped to this quantity is a candidate for convergence in distribution to a Standard Normal.
So "any sequence", if with it it is meant "the leading term of the sequence" -it does not satisfy the CLT. If it is meant "any sequence -and then take the sum and then center and scale", it may satisfy the CLT.


How to reset Drupal's content cache / deleted content?

Question: I'm working with Drupal 7.44. I created some content, but then deleted those articles. However, when I go to add new content, Drupal still recognizes the title of my articles that I deleted (as I'm typing it appears as an autocomplete below the field). I'm not sure if this is still completely stored in the database or if it's just cached somewhere. How do I go about deleting this (preferably without completely having to re-install everything)?
Administration -> Configuration -> Development -> Performance and "Clear all caches" did not seem to work.
Thanks!!
Comment: This is not in Drupal at all.  It is in your browser cache.  Clear that and it will be fine or look in a new web browser.
Comment: @heart.cooks.mind Yes - that was the issue! Thank you.
Answer: Unless you are using some module that I havent heard of (certainly possible) to give autocomplete style info as you are entering a node title, then I think the functionality you are describing is actually something being done by your browser.
You can alter your form to add automcomplete="off" to your title field (see https://stackoverflow.com/questions/2530/how-do-you-disable-browser-autocomplete-on-web-form-field-input-tag)
Comment: yes, you're correct. Checked it in another browser and it does not appear. Thanks!


Feed Import Products ( Commerce ) and Products Category ( Node Page )

Question: I need to create a CSV feed importer to import both Commerce products and products category node using single table and I am using Commerce Feed Module... when I create it using two different tables I can easily achieve it using Feeds Tamper Module (using explode plug in)
But when I implement single table structure as mentioned in the image... since the title is not unique it creates multiple nodes for each product... Is there any other table structure to implement this..
But what I actually need is:
Display1- A, SKU1|SKU2|SKU3
Display2- B, SKU4|SKU5|SKU6
Display3- C, SKU7|SKU8|SKU9

Is there no one familiar with this or done this before.. ??  or else I have to refine the question.. ??
Answer: The following CSV template and Feed Import settings will import 1 Product Node with multiple Product Entities being referenced.
Step 1 - Creating The CSV
The following example provides the title and product_variation headers...
title: Mapped to the Node Title field in Feed Import
product_variation: Mapped to the Commerce Product Referenced field. Field limit should be greater then 1, so multiple entities can be imported.
<code>title,product_variation

Product Example,"product_sku-1|product_sku-2"
</code>
Step 2 -Import Node
The "|" is used to create a unique entry using the Feeds Import Explode plugin.
The Feed Tamper product_variation field should have a single Plugin called Explode with the String separator set to "|".
Essentially, the Feeds Tamper Module will prepare the column values into an array, so Drupal can handle the request all awesome like :)
Answer: Using Feed to import product into Drupal Commerce is not the best approach.
Instead you should use Commerce Migrate which add Commerce Product support for Migrate.
Many helpers are available to support for example taxonomy terms, files ...
Many tutorials explain how to use Migrate which is a wonderfull module, but a quite huge.
But here is a sample workflow :

Import Category into Taxonomy
Import Product Ref
Import Product Display and link them to previous imported elements (category and product ref)

To do the last thing, you have to specify the previous source used. Like that, Migrate will be able to "convert" SKU and categories codes to the drupal corresponding elements.
Modules to use :

Migrate
Commerce Migrate
Migrate Extras
Comment: Everything works fine for me using feeds. What I am stuck with is just implementing the single table structure. Have you created this scenario using migrate module
Comment: I made a such work with Migrate. You have create 9 product ref into with commerce product entity, but only 3 product displays using nodes. And with your field product_ref you have to reference the 3 skus corresponding to your mapping.
Comment: ... with Migrate I made 2 migrate scripts. The first one which create SKUs into Commerce Product content type, and a second one which create product display and link it with previously created SKUs. Based on your file, you will have to parse two times your csv. The first time you will create a static mapping which store an array : $display['A'] = array('101', '102', '103); and when you'll import your product display, create the Display A and link it to SKUs and skip new line for the same Display product
Comment: ... but if you can, keep in separated files the content of Product Display from Product Ref. Otherwise it's quite hard to maintain and import. Hope it helps


What's the difference between timbres built from sine and square waveforms?

Question: Once again I've come across some ambiguous concepts,which I'm not sure if I've got their meanings right.
It says that a sine wave has a pale timbre whereas,for example,timbres built from a square wave are penetrating.Is it talking about their penetration depth or there's some other meaning behind this?
Answer: A sine wave (at least an ideal sine wave) is truly only a single frequency, and has no harmonic content beyond the fundamental. This gives it an extremely simple timbre that is indeed rather dull or pale. Square waves still have the fundamental frequency, but they also have many harmonic partials above it—specifically the odd partials, such as an octave and a fifth above, 2 octaves and a major third above, 2 octaves and a very flat minor 7th above, 3 octaves and a major second above, etc. It isn't infinite of course, but it a very rich timbre, especially compared to a sine wave. A completely unfiltered square wave is fairly harsh, though with a hollow component. With lopass filtering you can get a bunch of woody sounds and sounds reminiscent of a clarinet (which is an instrument built around mostly the odd partials).
A sawtooth wave is even harmonically richer than a square wave, and massively richer than a sine wave. It has all partials—odd and even—above the fundamental (octave, octave+P5, 2 octaves, 2 octaves+M3, 2 octaves+P5, 2 octaves+m7, 3 octaves, 3 octaves+M2, etc.). This makes for an especially harsh, penetrating sound when left unfiltered, even more intense than a square wave. But the timbre is fuller and not as hollow or potentially cold as the square wave timbre.
And with that, you have the three basic waves of additive and subtractive synthesis (a triangle wave is just a square wave with quieter upper partials). You can look up the harmonic overtone series on this site if you need more information about what it is and its sonic effects.
Comment: @ToddWilcox Well, how it sounds may be subjective, but I'm just talking about the actual mathematical construct. In a square wave, the amplitude of the (odd only!) partials is inversely proportional to their frequency—i.e., their amplitude is 1/f. In a triangle wave, the amplitude of the (still odd only) partials drops off much faster—inversely proportional to the *square* of their frequency, 1/f^2. The upshot is: if you put a lopass filter on a square wave you can create a triangle wave. No amount of filtering can turn a sine wave into a triangle. But agreed, the triangle sound is quite pure.
Comment: "triangle wave is just a square wave with quieter upper partials" - or couldn't it be viewed as a sine wave with a little bit of upper partials added in? To me a triangle sounds closer to sine than square, but every ear is different.
Comment: @ToddWilcox every sound can be described as a (possibly infinite) sum of sines, and every periodic sound by a *discrete* sum of sines. Saying that a signal can be described as a sine wave with a *little bit* of upper partials added in isn't saying much, at least until you define a *little bit* precisely.
Comment: All I was trying to say was that in some of the synthesizers I have (DSI Tempest being an example), you don't always have a sine wave available, and instead of a sine wave you use a triangle wave, because *sound-wise* they are almost the same thing. So from a perspective of building sounds on an actual analog subtractive synth, it helps to think of triangles as almost sine waves, as opposed to thinking them as more like square waves.
Answer: Don't put too much meaning into the specific words used to describe the sounds, they are, necessarily, ambiguous metaphors. "Talking about music is like dancing about architecture".
In this case, the difference in timbre is due to the much greater harmonic content of the square wave signal.  A sine signal has just a single harmonic component.  A square wave has multiple overtones at odd integer multiples (3x, 5x, 7x...) of the fundamental, with their amplitude decreasing like 1/f.  If you care about
the numerical/mathematical description of sounds, the analysis of the the spread, and intensity of the harmonics is important (probably the most important) aspect of the sound that affects timbre.
Answer: Looking at diagrams or reading written descriptions of sounds will be of little help unless you listen to examples of the waveforms with your own ears.
A sine wave is more like the sound of a flute. A square wave is more like the sound of a kazoo. But don't take my word for it. Find examples and listen to them.
A pure sine wave is like the atomic building block of all sounds. Acoustic theory says that all the other (complex) single waveforms actually consist of a combination of a number of pure sine waves (of different frequencies, amplitudes, and phase relationships). Since all acoustic sounds with a periodic waveform can be "deconstructed" into a number of constituent sine waves, this means, according to theory, that the sound of any pitched or tuned acoustic musical instrument can be approximated by combining a number of sine waves together in the proper proportions (which are varied over time). This theory is the basis of a technique used by some electronic music synthesizers called additive synthesis.
Answer: words such as "pale" and "penetrating" as descriptives for waveforms is specious at best. I could put support to words such as "simple/complex," "thin/rich."
Pale and penetrating seem to describe more about the sensibilities of the person describing the sound, than the sound itself.
A loud 4k sine wave can be extremely "penetrating" while a particular square wave might actually be "round and fuzzy."  So, while the words do evoke certain timbres, they don't really serve well to objectively describe waveforms.
Answer: A sine wave doesn't have a "timbre", similar to a single point on a paper not having a contour.  Sine waves are the "Eigenfunctions" of linear time-invariant systems which are the overwhelming number of components in any sound transmission.
If you put a sine wave into a non-distorting amplifier, you'll get a sine wave out.  When you fiddle with the tonal controls, you'll still get a sine wave out.  There is no way to distinguish fiddling with the tonal controls with fiddling with the volume control: either way you'll only affect the amplitude of what remains a single sine wave.
If you put a sine wave generator in your throat and form vowels, again there will be nothing except a sine wave of varying volume.  There will be no way to actually distinguish the vowels.
Flutes are rather sine-heavy, not having many sound components above the fundamental note being played.  The human voice box rather works with a pulse train which has a high content of overtones (for good vocal closure, higher even than a square-shaped wave), so you can use your mouth to form vowels and the shape of the mouth leaves revognizable shapes in the overtone spectrum of the voice rather than just influencing loudness at a single frequency.  Reed instruments of various kinds mimic this kind of sound production, making it possible to shape the overtone spectrum into very characteristic timbres by shaping the airways appropriately.
Bowed string instruments start with a saw-tooth kind of tone generator.
With electronic instruments, square-shaped tones can easily be generated (and they are approximated even when starting with clean sounds by overdriving amplifier stages like distortion pedals do) and have even higher overtone content than triangular wave forms but less so than pulse trains (a perfect pulse train has all harmonics with the same strength as the fundamental whereas harmonics of square waveforms taper off inversely with the frequency, and those of triangular waveforms taper off inversely with the square of the frequency).
Square waves (and more so pulse trains) have "too much" of a timbre to be usefully employed, like the color white (which contains actually colors of all the spectrum).  You use them more as raw material and shape them with electronic filters (technically related to tone controls on an amplifier or graphical equalizers) into more characteristic timbres.
For example, high-class accordions (which produce their tones by free reeds like a harmonica) may feature a tone chamber for some reed banks, a "cassotto", making for a characteristic mellower sound.  It only makes sense with comparatively high-quality reeds since those have the smallest air gaps and consequently the sharpest overtone spectrum.  Since those are much more expensive as some additional wood, nowadays you can get cassotto registers even in comparatively cheap instruments where the overall impression is "somewhat more muffled" rather than the sort of vowelizing effect that this was originally built for.
I digress.
At any rate, starting electronically, squares waves are the simplest reasonably rich timbre building blocks (not a useful finished timbre, mind you, but a good building block).  You can turn up the overall harmonic content at the cost of sound energy by playing with the duty-cycle and making the "square waves" asymmetrical (note that your tweeters in particular will not share the impression of a decrease in sound energy, and blowing the same energy through them as through a large bass woofer is not likely going to make them happy once you try for some more solid volume).


Navigating using XPath stored as a string

Question: I am trying to store XPath of an XML attribute as a string in a separate file so that if XPath changes, I can easily modify the navigation to the attribute without changing code.
For example, in following XML:
<Result>
  <Server = "main">
    <Client id="1"></Client>
  </Server>
</Result>

if I want to navigate to id attribute of Client element, I can have following string:

Result->Server->Client->id

I am not sure how in C# I can navigate using this string form of XPath and then, read the attribute value from the target XML.
Please help.
Harit
Comment: that is not valid xml ``
Answer: Well, firstly, your XML is a bit strange, with
<Server = "main">

Do you mean
<Server id="main">

But, regardless of that, you could just store the XPath directly instead of your string version. Like:
/Result/Server/Client[0]/@id

then you read the string from the file and pass it into something like:
public string GetClientIdString(string xPathString)
{
    var doc = new XmlDocument();
    doc.Load("SomeXml.xml");
    return doc.DocumentElement.SelectSingleNode(xPathString).Value;
}

The issue becomes that you can't really store the XPath exactly how you would like if you plan on having more than one Client under Server. If you need that functionality, though, you could parse out your version of the XPath and do something like:
public IEnumerable<string> GetClientIdStrings(string elementXPath, string attribute)
{
    var doc = new XmlDocument();
    doc.Load(SomeXml.xml);
    var clientIdStrings = new List<string>();

    foreach(var node in doc.DocumentElement.SelectNodes(elementXPath))
    {
        clientIdStrings.Add(node.Attributes[attribute].Value);
    }

    return clientIdStrings;
}


NestJS exception filter for MongoError returns UnhandledPromiseRejectionWarning

Question: I'm trying to create a custom exception filter for Mongo errors through Mongoose. When I use the filter, an UnhandledPromiseRejectionWarning occures and it gives no result. Any idea how I can fix this?
Error: UnhandledPromiseRejectionWarning: TypeError: Right-hand side of 'instanceof' is not an object
users.controller.ts
import { Body, Controller, Post, UseFilters } from '@nestjs/common';
import { UsersService } from './users.service';
import { RegisterUserDto } from './dto/register-user.dto';
import { MongooseExceptionFilter } from './filters/mongoose-exception.filter';

@Controller('users')
export class UsersController {
  constructor(private readonly usersService: UsersService) {}

  @UseFilters(MongooseExceptionFilter)
  @Post('register')
  register(@Body() registerDto: RegisterUserDto) {
    return this.usersService.create(registerDto);
  }
}

mongoose-exception.filter.ts
import { ExceptionFilter, Catch, ArgumentsHost } from '@nestjs/common';
import { Request, Response } from 'express';
import { MongoError } from 'mongoose';

@Catch(MongoError)
export class MongooseExceptionFilter implements ExceptionFilter {
  catch(exception: MongoError, host: ArgumentsHost) {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const request = ctx.getRequest<Request>();
    // const status = exception.getStatus();
    console.log('Exception', exception);

    response
      .status(418)
      .json({
        statusCode: 418,
        timestamp: new Date().toISOString(),
        path: request.url,
      });
  }
}

Answer: The problem was that "MongoError" was not available in the mongoose package directly and that MongoError from the mongodb package wasn't the same as the one mongoose was using. The solution was very simple, in the mongoose-exception.filter.ts file, I changed the import to:
import { MongoError } from 'mongoose/node_modules/mongodb';

Answer: import {
  ArgumentsHost,
  Catch,
  ConflictException,
  ExceptionFilter,
} from '@nestjs/common';
import { MongoServerError } from 'mongodb';
import * as mongoose from 'mongoose';

@Catch(mongoose.mongo.MongoServerError)
export class MongoExceptionFilter implements ExceptionFilter {
  catch(exception: MongoServerError, host: ArgumentsHost) {
    switch (exception.code) {
      case 11000:
        throw new ConflictException(
          `Duplicate unique key '${Object.keys(exception.keyValue)}'`,
        );
    }
  }
}

Answer: Try to do this.
import { Error } from 'mongoose';

@Catch(Error.ValidationError)
export class MongooseExceptionFilter implements ExceptionFilter {

  catch(exception: Error.ValidationError, host: ArgumentsHost) {

  }
}


A computionally-easy neccessary and sufficient condition for the inverse of line graph to exist

Question: Here I have a conjecture:

Let $G$ be a simple undirected graph. The neccessary and sufficient condition for a graph $H, L(H) = G$ to exist is to have no vertex $u \in V(G)$ so at least three adjacent vertices of $u$ appear in an independent set.

It surely is a necessary condition:
Let's assume that there is a vertex $u \in V(G)$ with three adjacent vertices in an independent set, and let $xy$ be its corresponding edge in $H$. According to the pigeonhole principle, there are at least two edges $e_1, e_2$ that their corresponding vertices in $G$ are independent, and are adjacent with $xy$ in $H$, where their common vertex is either $x$ or $y$. In this case, $e_1, e_2$ are adjacent, that contradicts with the independence of their corresponding vertices in $G$.
Answer: Your condition is equivalent to forbidding $K_{1, 3}$ as an induced subgraph (the 3 branches star graph). However, this is not sufficient to characterize fully the line graphs. However, a characterization exists by excluding induced subgraphs, which requires eight more forbidden subgraphs. These eight graphs provide the minimal counterexamples to your conjecture.
Another (easy) characterization is that a line graph needs to admit a clique partition where each vertex belongs to exactly two cliques.


F-test in Regression and Crossvalidation

Question: I am wondering what is the better measure of predictive power in a multivariate regression setting.
So an F-test for regression tests "the utility of the model", or exactly, if any of the $\beta$ coefficients are non-zero.  When the null is rejected, the model is considered "useful"; so, useful for prediction?  Crossvalidation also can help tell you if your model is useful for prediction.  Can you talk about them using the same phrasing?
What about the $R^{2}$ statistic?  Say $n$ is large, and the degrees of freedom in your model is small.  It is pretty unlikely overfitting has occurred.  What can you say about the predictive power of your model in this case when $R^{2}$ is high? medium? zero?
Examining the predictive power of some model is a huge subject, but do F-tests play into that?  Has crossvalidation become so popular because of machine learning methods where statistical tests don't exist?  Was it popularized to fill that void in algorithmic models, or do they really address different issues?
Answer:
So an F-test for regression tests "the utility of the model",

Well, no, it really doesn't. A rejection doesn't imply the model is actually useful at all.

or some such carefully worded statement.

If you mean that 'there's a carefully worded statement that applies to the F-test', this is a trivially true, but essentially useless statement.

Crossvalidation basically does the same thing.

No, it doesn't do the same thing as the F-test, not even approximately.

What is the best way to express what an F-test tells you about the predictive power of your model?

Perhaps something along the lines of "Unless you have an unusual definition of predictive power, the F-test may not directly tell you much about it."
Specifically, the summary of the result of the test would be "$H_0$ was rejected" or "$H_0$ was not rejected".
But let's take a step back and consider something slightly more informative, like the $p-value$. That doesn't really tell you about predictive value (which to me wouldn't tend to change much with $n$, once the sample size was large enough to have reasonable parameter estimates, utterly unlike the p-value.).
If you define predictive power as say $R^2$ (which to me would seem bizarre), then the $F$ statistic is related to it (for fixed model and sample size, at least), but we're now getting some distance from the test and still not all that close to predictive power yet. And if we start doing variable selection or something, $R^2$ is getting further and further from carrying any information about actual predictive power.

Would it be weird to see an analysis of data where someone used crossvalidation and quoted the p-value for the F-test to asses its predictive ability?

Well, weird because the F-test doesn't really address that question.
It seems to me a better question for you might be "What does an F-test actually do?" and a similar question about cross-validation, and maybe resolve exactly what 'predictive power' might usefully mean.
Once you define more carefully what particular thing you're doing cross-validation on, and exactly what the circumstances and assumptions are, it would be possible to make more useful/interesting statements.
--
You might find this a good starting point for basic discussion of cross-validation from a statistical point of view:
http://robjhyndman.com/hyndsight/crossvalidation/
You might notice there's hardly a mention of hypothesis tests at all, except a couple of sentences of precaution about circumstances in which they don't make much sense.
Comment: Thanks for the great link! That was a great overview.  I should have worded my question better though.  Perhaps $R^{2}$ would have been a better example instead of the F-stat.  I'll edit my post for that.  The second paragraph of your link gets to the point much better though.  What if there's only a small number of degrees of freedom and one is confident that $R^{2}$ isn't being inflated?  Or the adjusted $R^{2}$ almost matches?  Is $R^{2}$ then a good measure of predictive power?
Comment: Also, thank you for the detailed answer!  You clarified most of my other questions.


How to proper delete plugin in wordpress?

Question: I am working with WordPress plugin wp-latest-posts I deactivated this
plugin then deleted this plugin using FTP.After that I again upload fresh same
plugin but it's configuration setting has a previous value which I was added like
css etc. I checked database but this plugin is not generating any table where it
stores the css data and why previous data is coming fresh plguin setting.
Answer: Normally good plugins always provide options to uninstall them completely(files and their database modifications) from the admin panel itself.
First of all please note that "It is a good practice to delete plugins from WordPress admin in spite of deleting their files from FTP".
Now for a case when you delete a plugin from WordPress admin and still it automatically fill your previous settings as before on their next install. It means that this plugin is not coded well.
So we need to check below mentioned things:

Sometimes plugin create tables in database at the time of their installation, so check for them and delete them.
Some plugin save their values in "wp_options" table in WordPress database. First check manually for them and then try few plugins that help you in doing this like: https://wordpress.org/plugins/plugins-garbage-collector and https://wordpress.org/plugins/clean-options
Some plugins create few files or folders generally in "wp-content" directory. Check for them.

Hope this will solve your problem.
Answer: This is because some plugins store data in the wp_options table in your database, the correct way to uninstall a Plugin, is doing it from the WordPress Plugin Tab:
1.- Go To WordPress Dashboard
2.- Click on Plugins > Installed Plugins
3.- Select the Plugin that you want to delete and click on deactivate (if it's activated)
4.- Then click Delete
That way even the data that the plugin saves in the wp_options will be removed. Otherwise if you want to delete that data manually, you will need to have MySql Knowledge to find the data in that database.


Webgl 1 bit per pixel texture

Question: Is there any way to use 1 bit per pixel texture data in WebGL (for example texture 16×16px in 32 bytes of data) or is it necessary to unpack data from 1bpp to 8bpp first?
I have found similar OpenGL related question and answer https://stackoverflow.com/a/15706596/4540236, but it seems to me, that WebGL does not have GL_BITMAP data type constant.
Comment: You can store 8 bitmaps in a single 8-bit texture, but then you have to interpolate manually.
Comment: Thanks for the suggestion, but there will be only one 1bpp texture most of the time (it is for medical images overlay)
Answer: WebGL itself has no one bit per pixel format. You'll have to unpack the data yourself.
You can unpack it to 1 byte per pixel with formats gl.ALPHA or gl.LUMINANCE
you could try creating a fragment shader to unpack, no idea if the precision issues would kill you. Something like
precision mediump float;

varying vec2 v_texcoord;
uniform vec2 u_textureSize;
uniform sampler2D u_texture;

void main() {
  float texelCoord = floor(v_texcoord.x * u_textureSize.x);
  float bit  = mod(texelCoord, 8.0);
  float byte = texelCoord / 8.0;
  vec2 uv = vec2(byte / u_textureSize.x, v_texcoord.y);
  float eightPixels = texture2D(u_texture, uv).r * 255.0;
  float pixel = mod(floor(eightPixels / pow(2.0, bit)), 2.0);
  gl_FragColor = vec4(pixel, pixel, pixel, 1.0);
}

Hmmm I guess we should test ...

// Using TWGL.js because it's easier and I'm lazy
var gl = twgl.getWebGLContext(document.getElementById("c"));
var programInfo = twgl.createProgramInfo(gl, ["vs", "fs"]);

var arrays = {
  position: [-1, -1, 0, 1, -1, 0, -1, 1, 0, -1, 1, 0, 1, -1, 0, 1, 1, 0],
};
var bufferInfo = twgl.createBufferInfoFromArrays(gl, arrays);

function render(time) {
  twgl.resizeCanvasToDisplaySize(gl.canvas);
  gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);

  var uniforms = {
    u_texture: twgl.createTexture(gl, {
      format: gl.LUMINANCE,
      min: gl.NEAREST,
      mag: gl.NEAREST,
      width: 1,
      src: [ 0x3C, 0x42, 0xBD, 0x81, 0xA5, 0x81, 0x42, 0x3C, ],
    }),
    u_textureSize: [8, 8],
  };

  gl.useProgram(programInfo.program);
  twgl.setBuffersAndAttributes(gl, programInfo, bufferInfo);
  twgl.setUniforms(programInfo, uniforms);
  console.log("foo");
  twgl.drawBufferInfo(gl, gl.TRIANGLES, bufferInfo);
}
render();
canvas { border: 1px solid red; }

<canvas id="c"></canvas>

Comment: Thank you for your solution. I have already solved it by unpacking data to 1 byte per pixel. Fragment shader solution sounds good but it would work only for NEAREST filtering, not for LINEAR, which is what I need.
Comment: Well, you can do bilinear filtering in the shader. Pull out 4 pixels. But you won't get mips :(
Answer: As gman answered (https://stackoverflow.com/a/30529136/4540236): WebGL does not support one bit per pixel format.
So I used LUMINANCE format and unpack data manually with:
int off = 0;
for (int i=0;i<packedData.length;i++,off+=8) {
  int val = packedData[i];
  for (int j=0;j<8;j++) {
    unpackedData[off+j] = ((val&(1<<j))!=0)?255:0;
  }
}

Webgl 1 bit per pixel texture


Fisher test interpretation for left or right sided p-values

Question: Let say I have a frequency table of two variables $x$ and $y$ having or not having some property.
$$ \begin{array}{lcr}
\mbox{} & x & y \\
\mbox{has property} & 20 & 2 \\
\mbox{does not have property} & 61 & 79 \end{array}\
$$
What does it really mean to have positive (right hand side p-value) or negative (left-hand side p-value) association. Can I think of it as of correlation between variables $x$ and $y$? Why then for table
$$ \begin{array}{lcr}
\mbox{} & x & x \\
\mbox{has property} & 20 & 20 \\
\mbox{does not have property} & 61 & 61 \end{array}\
$$
I get large p-value not rejecting the null hypothesis?
EDIT: Null hypothesis states that variables are independent, that is the proportions do not differ among $x$,$y$ vs. has/does not have property. The right / left p-value regards associations on the diagonals of the tables like here
Comment: Please explain on what you mean by "right hand side" and "left-hand side" p-values.  In your second question, tell us specifically what the "null hypothesis" says.
Comment: @whuber Post was edited.
Answer: The error is to believe that you can analyze two 'variables' having only one observation of each.
What you have is two levels for a qualitative variable and two levels for the other. You cannot observe variation on that situation.
In this case you can try a Chi square test and after that, try to explain the level of correlation (if exists) with Cramer or any other measure of relationship.


Como pegar o total de registro de uma tabela

Question: Preciso saber quantos registros existem dentro de minha tabela e adicionar o total achado em uma variável. Estou fazendo da seguinte maneira mas não dá certo:
   <?php
        $w_querybusca = "SELECT * FROM Elem_matr;";
        $w_queryresultado=f_class_conecta_bd($w_querybusca);

        $result = pg_num_rows($w_queryresultado);
        echo $result;
    ?>

Deveria fazer de qual maneira?
O problema é: Com esse exemplo acima ele só mostra que a valores dentro da tabela. O que eu preciso é do total contido dentro dessa tabela!
Comment: Já resolvi o problema, era uma determinada função do próprio `PostgreSQL`.
Comment: Estou utilizando o PostgreSQL
Comment: Pode descrever melhor o que não funciona? da erro? de um `print_r` em `$w_queryresultado`
Comment: Bem, o que não funciona é que o valor retornado é 1. E o total de registros dentro da tabela passa de 500.
Comment: Poderia colocar o código de `f_class_conecta_bd`
Comment: @perdeu eu não tenho acesso a conexão do banco, por isso não poderei lhe mostrar esse trecho.
Comment: Vamos [continuar esta discussão no bate-papo](http://chat.stackexchange.com/rooms/18992/discussion-between-perdeu-and-alexandre).
Comment: Restrição de proxy -.- ! E sim, ser peixe pequeno na empresa é complicado!
Comment: O `print_r` retorna 1 ou um array com mais algumas chaves?
Comment: O `print_r` retorna 1
Comment: Veja se esse código muda o retorno:
`$w_querybusca = "SELECT COUNT (seq_elem) as total FROM \"Elem_matr\"";
        $w_queryresultado=f_class_conecta_bd($w_querybusca);
echo $w_queryresultado['total'];`
Comment: Esse trecho que você passou "anteriormente", não retorna nada, fica em branco!
Comment: Adicione essas duas linhas no inicio do arquivo e veja se aparece algum erro: `ini_set('display_errors', true); error_reporting(E_ALL);`
Comment: Não precisa colocar `resolvido` no título, pode marcar a resposta como aceita com `V`. [porque e como aceitar um resposta](http://meta.pt.stackoverflow.com/q/1078/91)
Comment: É por causa que se a própria pessoa que perguntou respondeu. Só da para aceitar daqui 2 dias! Ai para não haver equivoco com quem visualizar essa pergunta, eu coloquei o resolvido. Mas daqui 2 dias eu edito!
Answer: A resposta para o meu caso era a falta de utilização de um comando do próprio PostgreSQL, que no caso ficaria assim:
   <?php
        $w_querybusca = "SELECT * FROM Elem_matr;";
        $w_queryresultado=f_class_conecta_bd($w_querybusca);

        $result = pg_num_rows($w_queryresultado);
        echo $result;
    ?>


Executar comando no cmd com CD

Question: Estou querendo executar um comando específico do composer update pelo cmd. Porém, sempre preciso acessar a pasta da minha api com cd para executar a tal. Exemplo:
C:\Users\Luiz>cd api
C:\Users\Luiz\api>composer update

Existe uma maneira de fazer isso em um comando só? Tipo:
C:\Users\Luiz>cd api composer update

Answer: vc pode usar o & para executar comandos em uma só linha ex:
cd Links & dir

vc também pode usar o && neste caso ele só executa o segundo comando se o primeiro for executado sem erros:
cd Links && dir

tente utilizando os seus comandos e seja feliz:
cd api & composer update

Answer: Considerando que o path para a api já foi setada na variavél %path%, já tentou apenas executar call composer update? Veja essa resposta Running "composer update" within a Windows .bat file
Use o comando cd == check directory, usando com a flag /d, faz ir até o drive, opção útil quando usando comandos entre pastas e drives diferentes.
O seu comando pode ser em uma linha usando operadores:
• Observações sobre os operadores: &, |, && e ||

execute & execute & execute

execute | recebe _saída_do_comando_anterior

execute | recebe _saída_do_comando_anterior & execute

executou_sem_erro && então_execute_também

executou_com_erro || execute_também_porque_deu_erro

executou_com_erro || executou_com_erro || executou_com_erro

executou_sem_erro && executou_sem_erro && executou_sem_erro

executou_sem_erro && então_execute || execute_esse_no_primeiro_deu_erro

rem :: entre na pasta/check directory
cd /d "C:\Users\Luiz\api"
rem :: daí execute o update do compuser ::
composer update

rem :: ----------

rem :: para fazer em uma linha ::
cd /d "c:\Users\Luiz\api" && composer update

rem :: usando variável %userprofile% do ambiente ::
rem :: usando operador && (execute (se executou sem erro) && também execute)
cd "%userprofile%\api" && composer update

Para a maioria das apis, o envio de comandos para execução se dá nessa sintaxe:
cmd /c cd /d "C:\Users\Luiz\api" && composer update

Já no compuser é diferente, nele você usa script por evento pré-definidos, ou algo do tipo.
Mas acredito que seria melhor dar uma olha na documentação que aborda as formas de adicionar scripts/comandos no compuser:
Uma resposta tratando o uso de script no compuser
Como fazer um Calling Composer commands
Resposta como rodar um cmd/bat para fazer o update do composer
5 macetes  do compuser
Answer: set composerup=cd api ^& composer update

execute o comadno:

%composerup%

Executar comando no cmd com CD


How to draw a colored square on event change using canvas

Question: I want a blue square next to it to pop up. I want this to be a continuous process
let's take button clicks as the event which creates a new boxes (such as a red, blue and yellow button) where on click it spawns one, then another next to it and so on.
I have already attempted to do this utilizing both the DOM and CSS, but to no avail as my programming knowledge is not good. I have attempted to perhaps, append childs in the form of divs (I am not sure how that works).
I expect the output to be small differently colored squares which are pasted on the screen using canvas when it recognizes an event (such as a button click). This I will later attempt to adapt to the face recognition.
What must I do to achieve this result?
Theoretical results:
https://i.stack.imgur.com/auS7w.png
If possible, I would like it to loop from the start when it reaches the edge screen, so that it eventually fills the whole canvas. Thank you.
Answer: You can put a simple canvas in html so you can draw inside with javascript.
A good tutorial you can found here https://www.w3schools.com/graphics/canvas_intro.asp

const lengthSquare = 50, marginSquare = 10, colors = ["red", "blue", ""];
var x = marginSquare, y = marginSquare;
var canvas = document.getElementById("myCanvas");
var width = canvas.width, height = canvas.height;
var ctx = canvas.getContext("2d");

var addSquare = function(color){
  if(y + lengthSquare > height){
   console.error("Too much squares");
    return;
  }
  ctx.beginPath();
  ctx.strokeStyle = color;
  ctx.lineWidth = 3;
  ctx.strokeRect(x,y,lengthSquare,lengthSquare);
  ctx.stroke();
  x += marginSquare + lengthSquare;
  if(x + lengthSquare > width){
   x = marginSquare;
    y += marginSquare + lengthSquare;
  }
}
<!DOCTYPE html>

<input type="button" onclick="addSquare('red')" value="AddSquare RED"/>
<input type="button" onclick="addSquare('yellow')" value="AddSquare YELLOW" />
<input type="button" onclick="addSquare('blue')" value="AddSquare BLUE" />

<canvas id="myCanvas" width="500" height="200" style="border:2px solid #d3d3d3;">
Your browser does not support the HTML5 canvas tag.</canvas>


Identify kiosk app version via Directory API

Question: Is there a way to access information, specifially the version, of the installed kiosk app on a managed Chrome device? The chromeosdevices API has the OS version, but no mention of the kiosk app.
The lack of information itself should be an answer to this, but I wanted to be sure.
Answer: There's no way to determine the extension version running on a given device remotely though assuming you control the extension, you could have it report back to your servers the current version.
Comment: You can file an FR at crbug.com. If you've purchased licenses you can also submit a support ticket asking for the feature to be added.
Comment: Thanks Jay, is there a feature request system so this might be added?
Comment: Thanks, I'll go via the Chrome for Work dashboard


SQL - Como filtrar apenas dominios que contém "hotmail.com";

Question: Escreva um select que atenda os critérios abaixo:
1° Filtre apenas dominios que contém "hotmail.com";
2° Considere os contratos abaixo; 683679 705126 715651
TABELA
SELECT * FROM FDADOS_RE170(3,3) limit 12
carteira; tn; contrato; cpf; tipo_contato; contato; classificacao; origem; score;

3; 1; 710195; 02714662196; Email firstname.lastname@example.com; 1 - Excelente; M; 0

3; 1; 710195; 02714662196; Email email@example.com ;1 - Excelente M; 0

3; 1; 710195; 02714662196; Email firstname.lastname@example.com; 1 - Excelente; M; 0

3; 1; 710195; 02714662196; Email firstname.lastname@example.com 1 - Excelente; M; 0

3; 1; 710195; 02714662196; Email email@example.com ;9 - Indefinido O; 0

sql

Comment: Não saber fazer não é dúvida. As dúvidas só surgem após se tentar fazer ou estudar como fazer.
Comment: Por favor não duplique perguntas. Se a [pergunta anterior](https://pt.stackoverflow.com/q/400378/112052) não foi bem recebida, concentre seus esforços lá. Edite-a e siga as instruções que foram dadas no comentários, e se ela estiver de acordo será reaberta
Comment: Qual a dúvida então? Qual parte do filtro você não entendeu? Conhece a cláusula `LIKE` de validação?
Comment: Seja bem vindo! Use o operador `like`, ficará assim ... `contato LIKE '%hotmail.com'` Leia esta resposta: [Qual real diferença entre o operador '=' e LIKE?](https://pt.stackoverflow.com/a/212228/110782)
Comment: Amigo você poderia deixa o código melhor para visualização?
Comment: A duvida é fazer um select que me traga 1° Filtre apenas dominios que contém "hotmail.com";

2° Considere os contratos abaixo; 683679 705126 715651
Comment: Sem saber o nome da tabela onde iremos trabalhar ficará dificil, os numeros dos contratos  serão inseridos manualmente? Faça: `select * from SUA_TABELA where (contrato = (VARIAVEL_COM_NR_DOS_CONTRATOS)) and contato like '%hotmail.com'`
Comment: Obrigado Luiz Augusto
Answer: Ficará assim:
SELECT * FROM 'TABELA_NOME'
where
(contrato = 683679
OR contrato = 705126
OR contrato = 715651)
LIKE '%@hotmail.com%'

em TABELA_NOME é o nome de sua tabela.
Comment: Aceita `firstname.lastname@example.com`, falso positivo para os propósitos da questão
Comment: @JeffersonQuesado verifique novamente o enunciado. Filtre apenas domínios que contêm "hotmail.com". shotmail.com contém hotmail.com, está de acordo com o enunciado. No entanto, email@example.com também contém "hotmail.com" e não seria exibido com essa resposta.
Comment: então coloca o '@' e '%' no final, então tudo que tiver @hotmail.com será exibido :'%@hotmail.com%'


How to change the values in `stat_n_text` function?

Question: I have the following issue with labeling the smaple size in my dataset:
library(ggplot2)
library(EnvStats)
library(tidyverse)

cars_pre= mtcars %>% mutate(time="Pre")
cars_post= mtcars %>% mutate(time="Post")

df= rbind(cars_pre,cars_post)
p <- ggplot(df,             aes(x = factor(cyl), y = mpg, fill = time) +
  theme(legend.position = "none"))

ggplot(df,  aes( factor(cyl),  mpg)) + geom_boxplot(aes(fill = time) ) +
  stat_n_text()

The issue I have is that I have pre/post groups: but those are the same subjects, so when I plot stat_n_text it doubles my sample size, cause it adds pre and post samples together ( treats them as separate subjects). Is there a way to update the n, so it will be half of its value (in a picture I want n =11, 7 and 14 for cyl 4,5, and 6 respectively)?
Answer: I figured the following way how to modify it: by changing figure data:
    library(splitstackshape)
    p<-ggplot(df,  aes( factor(cyl),  mpg)) +
geom_boxplot(aes(fill = time) ) +
          stat_n_text( )

Then extract data from the ggplot object:
    q <- ggplot_build(p)

Modify label  of the sample size data
    q$data[[2]]= q$data[[2]] %>% cSplit(., 'label', '=') %>% mutate(label=paste0(label_1, "=", label_2/2))

Plot the object back
q <- ggplot_gtable(q)
plot(q)
Answer: According to packaged developers: a bit of a hack, but a direct manipulation of N is not implemented:
ggplot(df, aes( factor(cyl), mpg)) + geom_boxplot(aes(fill = time) ) + stat_n_text(data=cars_pre)


How can I use a FOR loop to create circles in a circle (in Processing)

Question: I need to create a loop which will space circles equally around a circle in Processing.
I know I can somehow implement a FOR loop.
I need to be able to increase or decrease the number of circles around this circle (with button presses) but keep them equally spaced.
I know the formula's I need to include in the FOR loop to get the X and Y axis. The formulas:
being
   X = R*cos(angle-90)+Y0
   Y = R*sin(angle-90)+X0

I understand the three parameters of the FOR loop; when does it start, when does it finish, what changes when it runs.
What I can't see is how to implement the formulas into the FOR loop.
Many thanks
Here is the code I do have
void setup () {
  size (600, 600);
  background (255, 255, 255);
  smooth ();
  ellipse (width/2, height/2, 200, 200); // the guide circle. Not needed in final code.
}

void draw() {

  for (int i = 0; i < 20; i ++) {
    for (int j = 0; j < 20; j ++) {

      ellipse (i *20, j * 20, 20, 20);
    }
  }
}

Comment: What exactly are you trying to do? Are you trying to just put X number of circles inside some other circle? If so, post what you've tried in the form of an [MCVE](http://stackoverflow.com/help/mcve). Or are you trying to fit as many circles in as possible? If so, check out [circle packing](http://en.wikipedia.org/wiki/Circle_packing_theorem).
Comment: @KevinWorkman

I need to write a sketch where n circles (between 2 and 10) are equally spaced around a circle. I need n to be adjustable by the user of the sketch using left and right mouse buttons or '+' and '-' or similar.

I want to use a FOR loop for equally spacing n circles using the formulas in my question.

Whilst I can use FOR loops for basic things like grids and rows of shapes, I can't see how to put them in a circle and equally spaced.

The bit most confusing is how the formulas are added to the FOR loop parameters.

Thanks for taking the time to read / reply.
Comment: It's not homework, it's an end of chapter exercise.

I'm trying to put circles around a circle. I don't think circle packing is what I'm after.

The minimum number of circles is 2. The max number is 10. They need to remain equally spaced as more circles are added through key presses.

What I'm most stuck on is how to get the formulas for x and y co-ords into the FOR loop.
Comment: @Jdoh Okay, I see. I agree circle packing isn't what you're after. Here's a hint: You've got a formula (involving sin and cos). You know the R value (it's the radius of the circle). X0 and Y0 are just the center of the circle. Now all you need is the angle. That's what the for loop is for. If you want to add 10 circles, you create a for loop that starts the angle at 0, then increments it by 36 each iteration. When the angle reaches 360, you've gone the whole way around the circle.
Comment: Thank you Kevin.

Will I need multiple loops. One for n=10, n=9, n=8...n=2?

As you said, A is incremented by 36 for n=10, do I write a similar loop for n=9 but it is incremented by 40? (360/9).

Thanks again. Your last comment made everything much clearer. Not clear yet, but I'm getting there :)
Comment: @Jdoh (btw, I don't get a notification if you don't tag me in your comment) No, you won't need separate loops for each N value. You can parameterize that in a variable. Just try it out with a hardcoded N (like 10) and see what patterns you notice, then try to generalize to any N value.
Answer: This code should do the trick:
    float incrementalAngle = 0.0;

void setup(){
  size(600, 600);
  smooth();
  background(0);

  ellipse(width/2, height/2, 200, 200);
  drawCircles(20, 200);
}

void draw(){

}

void drawCircles(int circlesNumber, int bigCircleNumber){
  float angle = incrementalAngle;

  for(int i = 0; i < circlesNumber; i++){
    ellipse(bigCircleNumber * cos(incrementalAngle) + height/2,
            bigCircleNumber * sin(incrementalAngle) + width/2,
            circlesNumber, circlesNumber);
    incrementalAngle += TWO_PI / circlesNumber;
  }
}

So the second loop wasn't needed, and the formula you were trying to introduce would go in the X and Y position of your ellipse, there by playing whit the angle and the cos and sin you can get the result you were looking for.
What's left now is for you to get the number of circles you want by the clicking inside a mousePressed() method and drawing that amount.
Hope this comes useful and call me if you need more help
Regards
Jose.
Comment: It's great that you're trying to help, but you really should consider walking the OP through the process instead of just dumping syntax here. Spoon-feeding isn't helpful.
Comment: Don't be so hard... I'll take the advice for the next time.
Answer: Thank you to everyone who helped.
I managed to do it (slightly differently to you @Jose Gonzalez
   int nbr_circles = 2;
void setup() {
  size(600, 600);

  smooth();
  background(255);
}

void draw() {
  background(255);
  float cx = width/2.0;
  float cy = height/2.0;
  fill(0);
  //float x, y; //
  for (int i = 0; i < nbr_circles; i++)
  {
    float angle = i * TWO_PI / nbr_circles;
    float x = cx + 110.0 * cos(angle);
    float y = cy + 110.0 * sin(angle);
    ellipse(x, y, 20, 20);
  }
}

void mousePressed() {

  if (mouseButton == LEFT) {
    if (nbr_circles < 20)
    nbr_circles = nbr_circles + 1;

  } else if (mouseButton == RIGHT) {
    if (nbr_circles > 2)
      nbr_circles = nbr_circles - 1;

  }
}


Comparando Datas em string, utilizando SQLite e NHibernate

Question: Eu estou usando SQLite e NHIbernate, e estou guardando no meu banco as Datas como String no SQLite, pois não é possível armazenar datas no mesmo. Tudo tava indo ok, mas eu precisei comparar datas para imprimir um relatório. Agora, não consigo fazer essa comparação, pois está em string, e não consigo converter para Date no LINQ/Lambda.
Tentei o seguinte:
var dataInicial = DateTime.Parse(_DataInicial);
var dataFinal = DateTime.Parse(_DataFinal);
return session.QueryOver<Locacoes>()
     .Where(c => DateTime.Parse(c.DataInicial) >= dataInicial )
     .Where(c => DateTime.Parse(c.DataFinal) <= dataFinal).List();

Mas da uma Exception no primeiro Where:

"A variavel 'c' do tipe 'Locacoes' é referenciada no escopo '', mas não foi definida".

Não sei se esse é o problema, mas eu gostaria mesmo como eu posso comparar duas datas no NHibernate, sendo que elas estão em string pois gravadas em SQLite.
Answer: Obrigado pela ajuda de todos, mas eu tive que modificar o método da pesquisa.
Vou ter que alterar o formato da minha data, pois estava em "31/07/2016" para "20160731".
e ficou dessa maneira:
            string dataInicial = "'20100604'";
            var query = "SELECT * from Clientes as c where c.Data >= " + dataInicial;
            var lista = session.CreateSQLQuery(query).List();


Python serial to bytes fails when passing array

Question: First, I am a beginner... My code is:
<code>data = bytearray(b'\x01\x04\x00\x16\x00\x02\x00\xf9')

# 1 - does not work as expected:
ser.write(serial.to_bytes(data))

# 2 - works as expected:
ser.write(serial.to_bytes([0x01,0x04,0x00,0x16,0x00,0x02,0x90,0x0f]))
</code>
So, what am I missing? I need to have the same results as in 2 but loading the data from a bytearray.
Comment: Although I have fixed the bytearray, still does not work. So the question still there...
Answer: Reading my own post I found the answer: The last two bytes are flipped.


Is there any possible method to calculate effect size in mixed models?

Question: I run MIXED command for mixed model analysis of repeated measured data.
However, there is no option or menu for estimate power(like partial eta in GLM) in mixed analysis.
Is there any method to calculate effect size in mixed model?
Answer: The estimates are the effect sizes as for simple regression. For example in following output in R:
<code>Linear mixed model fit by REML ['lmerMod']
Formula: Reaction ~ Days + (Days | Subject)
   Data: sleepstudy

REML criterion at convergence: 1743.6

Scaled residuals:
    Min      1Q  Median      3Q     Max
-3.9536 -0.4634  0.0231  0.4634  5.1793

Random effects:
 Groups   Name        Variance Std.Dev. Corr
 Subject  (Intercept) 612.09   24.740
          Days         35.07    5.922   0.07
 Residual             654.94   25.592
Number of obs: 180, groups:  Subject, 18

Fixed effects:
            Estimate Std. Error t value
(Intercept)  251.405      6.825   36.84
Days          10.467      1.546    6.77

Correlation of Fixed Effects:
     (Intr)
Days -0.138
</code>
In above output, the effect size for Days can be taken as 10.5. If multiple predictors are present, their estimates will reflect effect sizes for each.
Comment: I think they are after some sort of standardized effect, something like $R^2 $  (which is all $\eta^2 $ is in glm).
Answer: I found myself two options.
One is calculation based on correlation, based on the book of Rosenthal, Rosnow, & Rubin.
"Contrasts and Effect Sizes in Behavioral Research A CORRELATIONAL APPROACH"
Another option is use SAS and calculate f squared.


How to pronounce $\mathcal{E}$?

Question: How to say $\mathcal{E}$ when we are speaking about mathematics? Can I call it "curly E"?
And also how to pronounce $\mathcal{A}$ and $\mathcal{B}$ and $\mathcal{P}$ and so on?
Answer: It depends on your audience and the context. In physics, $\mathcal E$ usually denotes emf, so you would say that. In set theory, $\mathcal P$ usually denotes a power set, so you would say that. Otherwise, just do your best to describe it in a manner that your audience would understand. "Curly" or "calligraphic" works fine in my opinion.
Side note: You actually can type $\mathcal E$ into Google if you use the Unicode character ℰ (U+2130).
Answer: A useful resource for "speaking" mathematics is The Handbook for Spoken Mathematics by Lawrence A. Chang.  This book suggests that we should use a description of the font or script, plus the name of the letter (see pages 3–5).  Hence $\mathcal{E}$ might be read as "calligraphic capital $E$".  In most contexts, this would likely be overkill and require too much talking—I gather that much of the intent of the Handbook is to give a guide for instructors of blind students.  Hence an abbreviated "calligraphic E" might also be appropriate, as we can rely on students to read / copy what is on the board in most circumstances.
Other thoughts (including those above):

Follow the handbook:  "calligraphic capital [letter]" or "calligraphic [letter]".
Describe the letter more loosely: "script [letter]", "curly [letter]", or "cursive [letter]".
Use TeX:  "mathcal [letter]" or "cal [letter]".[1]
Don't Sweat It: "[letter]".[2]
Name the Symbol from Context: as suggested in this answer, the symbols may have specific meaning in a given context.  For example, $\mathcal{H}$ is "the Hilbert space H", and the symbols $\mathcal{X}$ and $\mathcal{Y}$ are "the Banach spaces $X$ and $Y$".

[1] This is what I find myself doing most often.  The symbol $\mathscr{H}$ is "math script H", while $\mathfrak A$ is "mathfrak A".  This sometimes causes me problems in classrooms where I am dealing with students who don't know TeX, but has never been an issue when talking with my peers.
[2] This option will often be the most appropriate.  For example, the space of distributions $\mathcal{E}'$ is just "E prime".  There may be potential for ambiguity, but in many, many contexts, it just won't be a problem.  I mean, do we often make a big deal about an element $x$ in a space $X$?  Sometimes we carefully speak the letters out loud, but I find myself saying "Let eks be an element of eks..." more often than I care to admit.

How to pronounce $\mathcal{E}$?


Pegar o objeto da lista ou fazer um novo select no banco?

Question: Tenho um Controller que a primeira coisa que ele faz, é trazer do banco uma lista de objetos. Após clicar num dos botões, o cliente vai usar um dos itens da lista.
A minha dúvida é a seguinte: Qual seria a melhor forma de pegar esse objeto? Dá lista que já tenho, ou faço um novo select no banco?
@PostConstruct
    public void init() {
        int IdPaciente = Integer.parseInt(Str.NullToStrDef(FaceUtil.getRequest().getParameter("idpaciente"), "0"));
        paciente = pacienteDao.find(IdPaciente);
        listProgramas = programaDao.findByAplicador(loginController.getUsuarioLogado().getIdusuario(), IdPaciente);

        int IdPrograma = Integer.parseInt(Str.NullToStrDef(FaceUtil.getRequest().getParameter("programa"), "0"));
        if(IdPrograma > 0) {

            programaAplicando = programaDao.find(IdPrograma); //Assim?

            programaAplicando = listProgramas.stream()
                    .filter(t -> t.getIdprograma() == IdPrograma)
                    .collect(Collectors.toList()).get(0); //ou assim?
        }
}

Answer: @Alexandre Biancuzzi,
É um pouco difícil sugerir algo, pois não tenho todo o conhecimento do seu código e de suas implementações e nem sua real necessidade.
Segue abaixo minhas contribuições, talvez ajude em algo:
1 Validações antes de consumir o banco de dados
O custo do Banco de dados é altíssimo ( Integridade da base, valor dos dados, valor da concorrencias das conexões, valor em R$ kkkk, comparação de escalanomento de aplicação x escalonamento do banco e etc ).
O máximo que for possível fazer validações de negócios, validações dos parametros do requests, preparações de variáveis antes de consumir o banco será melhor.
2 Trazer os dados atualizados ao usuário
Não entendi o que o seu controller faz. Mais é inevitável o ato de sempre tentar trazer o dado mais atualizado ao usuário.
Ou seja será necessário sempre ir ao banco trazer a informação mais atualizada ( ou utilizar outras tecnicas como caches e proxies ), mais creio eu que não é uma boa prática buscar uma lista em algum momento sem novamente ir no banco e atualizá-la conforme o que está no banco. Teoricamente na maioria dos casos o usuário irá querer esta informação atualizada em cada request ( dependendo da regra de negócio é claro ).
3 Otimize suas buscas nos bancos.
O máximo que for possível fazer o menor número de buscas no banco e o máximo de buscas otimizadas melhor será para o trafego das informações, otimizações no processamento do banco e diminuicão da concorrência de conexões ao banco.
Referente ao seu código, seria melhor se vc tivesse algo em seu DAO que não precisasse trazer toda a lista de programas para a aplicação sendo que irá usar apenas o primeiro item ex:
    programaAplicando = programaDao.findOnlyOneObjectByIdPacienteAndIdPrograma(
                             Integer.parseInt(Str.NullToStrDef(FaceUtil.getRequest().getParameter("idpaciente"), "0"),
                             Str.NullToStrDef(FaceUtil.getRequest().getParameter("programa"), "0")
                        );

Comment: Legal Dilermando! Era +- o que tinha em mente mesmo, mas precisava saber se fazia "sentido". Uma das coisas que me preocupo sempre, é não fazer requisições ao banco desnecessariamente. Referente à sua sugestão, realmente preciso trazer todos os programas, já que o usuário irá "pegar" um deles para executar. E como uso um ViewScoped, não preciso dar outro select. No caso, o programa não é alterado com frequência, portanto, não preciso me preocupar com concorrência de dados. ;o)


Bayes factor from posterior odds

Question: I tried to answer my own question Comparing two Bayesian models under disjoint prior supports using MCMC. Here is my intent. I am not confident in what I  wrote so prefer to post it as a question : Is this a correct way to compute Bayes factor ?
I would like to compute the Bayes factor:
 $$  K = \frac{P(x|H_1)}{P(x|H_2)} =\frac{P(H_1|x)}{P(H_2|x)} \frac{P(H_2)}{P(H_1)}  $$
I have a model consisting of three parameters $\theta_1$, $\theta_2$ and $\theta_3$.  $H_1$ is $\{(\theta_1,\theta_2,\theta_3) \in [m,M]^3  \mbox{ such that } \theta_1<\theta_2<\theta_3\}$, $H_2$ is it complementary.
Then suppose that I have a model to infer $P(\theta_1,\theta_2,\theta_3 | x)$ under the complete parameter space $H_1 \cup H_2 $ then $P(H_1|x)$ can simply
be computed as:
$$
P(H_1|x)=\int_m^M P(\theta_1,\theta_2,\theta_3|x) 1_{\theta_1>\theta_2>\theta_3}(\theta_1,\theta_2,\theta_3) d\theta_1 d\theta_2 d\theta_3
$$
by simply counting the posterior MCMC samples satisfying the condition and $P(H_2|x)=1-P(H_1|x)$.
So it remains to extract the associated $\frac{P(H_2)}{P(H_1)}$ by simply considering the subpart of $H_1$ over the overall parameter space:
$$
P(H_1)=\int p(\theta_1,\theta_2,\theta_3) \cdot 1_{\theta_1>\theta_2>\theta_3}(\theta_1,\theta_2,\theta_3) d\theta_1 d\theta_2 d\theta_3
$$
which can be computed analytically for the prior $p(\theta_1,\theta_2,\theta_3)$ associated to my posterior and again $
P(H_2)=1-p(H_1)$.
Is this a correct way to compute Bayes factor ? If yes, is this method has a name?
Answer: This is a correct approach indeed, since you compare two subsets of the parameter space after selected a prior for the entire parameter set. This prior indeed defines single-handedly prior weights on both subsets $H_0$ and $H_1$, which can be computed by integration. (If the three parameters are exchangeable, the prior weight of $H_0$ is $1/3!$.)
Comment: Thanks again. Have you a reference in mind for this approach ?
Comment: This is standard practice when considering null hypotheses that have a non zero weight under the original prior. There is no specific name or reference for this.

Bayes factor from posterior odds


How to find the derivative of $(3x-1)^2(2x+3)^2$

Question: I used the power rule and the chain rule and ended up with this:
$$y'= (3x-1)^2 \times 2(2x+3) \times 2 + (2x+3)^2 \times 2(3x-1)\times 3$$
The next step, which I do not understand how it is combined or created is this:
$$y'= 2(3x-1)(2x+3)\left[2\cdot(3x-1)+3(2x+3)\right]$$
Where did the exponents go? What is combined? How is it combined?
The Final Answer should be this, according to my teacher:
$$y'= 2(3x-1)(2x+3)(12x+18)$$
Comment: @Nick Assuming the OP copied the expected answer correctly, your edit is fine; it's just that the teacher's answer is incorrect. (The $18$ should be a $7$.)  Three of us + Maple (I checked) have come up with the correct answer. :-)
Comment: No it's all correct.
Answer: I think that you forgot that the derivative of $u^n(x)=n u^{n-1}(x)u'(x)$.
I suppose that you want the derivative of $$y=(3x-1)^2 (2x+3)^2$$ So, let us set $u=(3x-1)^2$ and $v=(2x+3)^2$. So $y=u\times v$ and $y'=u'\times v+u\times v'$.
Now $$u'=2\times(3x-1)'\times(3x-1)=6(3x-1)$$ $$v'=2\times(2x+3)'\times(2x+3)=4(2x+3)$$ $$u'\times v+u\times v'=6(3x-1)(2x+3)^2+4(2x+3)(3x-1)^2$$ Factor $(3x-1)(2x+3)$; so $$u'\times v+u\times v'=(3x-1)(2x+3)\Big(6(2x+3)+4(3x-1)\Big)$$ Expand what is inside the last bracket and simplify to arrive to $$y'=2 (2 x+3) (3 x-1) (12 x+7)$$
With reference to the exponents, look at the first line and think about $n=2$ and remember that we never use exponent $1$ in such context of polynomials.
Comment: Oh! Thank you so much, I didn't realize she factored it out. And yes, my math teacher makes a lot of mistakes so it wouldn't be unusual for the answer to be different.
Comment: You are very welcome. When you have powers such as in $y=u^n \times v^m$, the common factor will be $u^{n-1}\times v^{m-1}$. Just for fun, prove it. Cheers :-)
Answer: $$(f^2g^2)'=((fg)^2)'=(h^2)'=2h'h=2(f'g+fg')fg$$
$$\Big((3x-1)^2(2x+3)^2\Big)'=2\Big(3(2x+3)+(3x-1)2\Big)(3x-1)(2x+3)=2(12x+7)(3x-1)(2x+3)$$


Submit php não funciona

Question: Meu formulário de contato não está enviando, alguém poderia ver qual erro estou cometendo?
HTML:
 <form id="contactForm" novalidate class="s-form wow zoomInUp" data-wow-delay="0.5s">

        <select name="user-topic" id="user-topic" class="m-select">
                <option value="Not select">O que você precisa?</option>
                <option value="Topic 1">Solicitar contato de um vendedor</option>
                <option value="Topic 2">Agendar Revisão</option>
                <option value="Topic 3">Cotação de Peças</option>
                   <option value="Topic 3">Crítica/Sugestão</option>
            </select>

    <input type="text" placeholder="NOME" value="" name="user-name" id="user-name" />
    <input type="text" placeholder="E-MAIL" value="" name="user-email" id="user-email" />
    <input type="text" placeholder="TELEFONE" value="" name="user-phone" id="user-phone" data-mask="(00) 0000-0000" />
    <textarea id="user-message" name="user-message" placeholder="DEIXE SUA MENSAGEM"></textarea>
    <button type="submit" class="btn m-btn">Enviar mensagem</button>
 </form>

PHP:
  <?php
  // Check for empty fields
  if(empty($_POST['name']) || empty($_POST['email']) || empty($_POST['message']) || !filter_var($_POST['email'],FILTER_VALIDATE_EMAIL)){
    echo json_encode(array('error'=>'true'));
    return false;
  }

  $name = $_POST['name'];
  $email_address = $_POST['email'];
  $message = $_POST['message'];
  $lastname = $_POST['lastname'];
  $phone = $_POST['phone'];
  $subject = ($_POST['subject'] ? $_POST['subject'] : "Website Contact Form:  $name");

  // Create the email and send the message
  $to = 'firstname.lastname@example.com';// Add your email address inbetween the '' replacing firstname.lastname@example.com - This is where the form will send a message to.
  $email_subject = $subject;
  $email_body = "You have received a new message from your website contact form.\n\n"."Here are the details:\n\nName: $name\n\nLast Name: $lastname\n\nEmail: $email_address\n\nPhone: $phone\n\nMessage:\n$message";
  $headers = "From: firstname.lastname@example.com\n"; // This is the email address the generated message will be from. We recommend using something like firstname.lastname@example.com.
  $headers .= "Reply-To: $email_address";
  mail($to,$email_subject,$email_body,$headers);
  echo json_encode(array('success'=>'true'));
  return true;
  ?>

JS:
$(function() {
$("input,textarea").jqBootstrapValidation({
    preventSubmit: true,
    submitError: function($form, event, errors) {
        // additional error messages or events
    },
    submitSuccess: function($form, event) {
        event.preventDefault(); // prevent default submit behaviour
        // get values from FORM
        var topic = $("select#user-topic").val();
        var name = $("input#user-name").val();
        var email = $("input#user-email").val();
        var phone = $("input#user-phone").val();
        var message = $("textarea#user-message").val();
        var firstName = name; // For Success/Failure Message
        // Check for white space in name for Success/Fail message
        if (firstName.indexOf(' ') >= 0) {
            firstName = name.split(' ').slice(0, -1).join(' ');
        }
        $.ajax({
            url: "././mail/contact_me.php",
            type: "POST",
            dataType: 'json',
            data: {
                topic: topic,
                name: name,
                email: email,
                phone: phone,
                message: message
            },
            cache: false,
            success: function(data) {
                if(data.error){
                    // Fail message
                    $('#success').html("
");
                    $('#success > .alert-danger').html("<button type='button' class='close' data-dismiss='alert' aria-hidden='true'>×").append("</button>");
                    $('#success > .alert-danger').append("Sorry " + firstName + ", it seems that my mail server is not responding. Please try again!");
                    $('#success > .alert-danger').append('
');
                    //clear all fields
                    $('#contactForm').trigger("reset");
                }
                else if(data.success){
                    // Success message
                    $('#success').html("
");
                    $('#success > .alert-success').html("<button type='button' class='close' data-dismiss='alert' aria-hidden='true'>×").append("</button>");
                    $('#success > .alert-success').append("Your message has been sent. ");
                    $('#success > .alert-success').append('
');
                    //clear all fields
                    $('#contactForm').trigger("reset");
                }
            }
        })
    },
    filter: function() {
        return $(this).is(":visible");
    },
});
});

/*When clicking on Full hide fail/success boxes */
$('#name').focus(function() {
    $('#success').html('');
});

A página é esta aqui.
Será que pode ser alguma coisa do site ser em um subdomínio?
Comment: parece que seu formulário está enviando via GET, coloca um method="post" na tag do formulário pra ver se funciona.
Comment: coloquei o method="post", mas infelizmente nao funcionou
Answer: Boa tarde Pedro, Bom vamos lá...
HTML:
<form id="contactForm" name="form1" action="<?php echo $_SERVER['PHP_SELF']; ?>" method="POST" class="s-form wow zoomInUp" data-wow-delay="0.5s">

    <select name="user-topic" id="user-topic" class="m-select">
            <option value="Not select">O que você precisa?</option>
            <option value="Topic 1">Solicitar contato de um vendedor</option>
            <option value="Topic 2">Agendar Revisão</option>
            <option value="Topic 3">Cotação de Peças</option>
               <option value="Topic 3">Crítica/Sugestão</option>
        </select>

<input type="text" placeholder="NOME" value="" name="user-name" id="user-name" />
<input type="text" placeholder="E-MAIL" value="" name="user-email" id="user-email" />
<input type="text" placeholder="TELEFONE" value="" name="user-phone" id="user-phone" data-mask="(00) 0000-0000" />
<textarea id="user-message" name="user-message" placeholder="DEIXE SUA MENSAGEM"></textarea>
<button type="submit" name="enviar" class="btn m-btn">Enviar mensagem</button>

PHP:
  <?php
  // Check for empty fields
  if(isset($_POST['enviar'])){
  if(empty($_POST['name']) || empty($_POST['email']) || empty($_POST['message']) || !filter_var($_POST['email'],FILTER_VALIDATE_EMAIL)){
    echo json_encode(array('error'=>'true'));
    return false;
  }

  $name = $_POST['name'];
  $email_address = $_POST['email'];
  $message = $_POST['message'];
  $lastname = $_POST['lastname'];
  $phone = $_POST['phone'];
  $subject = ($_POST['subject'] ? $_POST['subject'] : "Website Contact Form:  $name");

  // Create the email and send the message
  $to = 'email@example.com';// Add your email address inbetween the '' replacing firstname.lastname@example.com - This is where the form will send a message to.
  $email_subject = $subject;
  $email_body = "You have received a new message from your website contact form.\n\n"."Here are the details:\n\nName: $name\n\nLast Name: $lastname\n\nEmail: $email_address\n\nPhone: $phone\n\nMessage:\n$message";
  $headers = "From: firstname.lastname@example.com\n"; // This is the email address the generated message will be from. We recommend using something like email@example.com.
  $headers .= "Reply-To: $email_address";
  mail($to,$email_subject,$email_body,$headers);
  echo json_encode(array('success'=>'true'));
  return true;  }

  ?>

JS:
$(function() {
$("input,textarea").jqBootstrapValidation({
    preventSubmit: true,
    submitError: function($form, event, errors) {
        // additional error messages or events
    },
    submitSuccess: function($form, event) {
        event.preventDefault(); // prevent default submit behaviour
        // get values from FORM
        var topic = $("select#user-topic").val();
        var name = $("input#user-name").val();
        var email = $("input#user-email").val();
        var phone = $("input#user-phone").val();
        var message = $("textarea#user-message").val();
        var firstName = name; // For Success/Failure Message
        // Check for white space in name for Success/Fail message
        if (firstName.indexOf(' ') >= 0) {
            firstName = name.split(' ').slice(0, -1).join(' ');
        }
        $.ajax({
            url: "././mail/contact_me.php",
            type: "POST",
            dataType: 'json',
            data: {
                topic: topic,
                name: name,
                email: email,
                phone: phone,
                message: message
            },
            cache: false,
            success: function(data) {
                if(data.error){
                    // Fail message
                    $('#success').html("
");
                    $('#success > .alert-danger').html("<button type='button' class='close' data-dismiss='alert' aria-hidden='true'>×").append("</button>");
                    $('#success > .alert-danger').append("Sorry " + firstName + ", it seems that my mail server is not responding. Please try again!");
                    $('#success > .alert-danger').append('
');
                    //clear all fields
                    $('#contactForm').trigger("reset");
                }
                else if(data.success){
                    // Success message
                    $('#success').html("
");
                    $('#success > .alert-success').html("<button type='button' class='close' data-dismiss='alert' aria-hidden='true'>×").append("</button>");
                    $('#success > .alert-success').append("Your message has been sent. ");
                    $('#success > .alert-success').append('
');
                    //clear all fields
                    $('#contactForm').trigger("reset");
                }
            }
        })
    },
    filter: function() {
        return $(this).is(":visible");
    },
});
});

/*When clicking on Full hide fail/success boxes */
$('#name').focus(function() {
    $('#success').html('');
});

Testado e funcionando ! Abs
Comment: Fiz as alterações, infelizmente ainda não está dando certo.
Comment: @pedro está tudo na mesma página ? o **PHP, HTML e JAVASCRIPT** ?
Comment: não, ta assim, o HTML está no /site, o PHP está no /site/mail, e o Javascript está no /site/assets/contact. seria melhor colocar tudo junto?
Comment: @PedroHenriqueKuzminskas me adiciona no skype tenho um código aqui : **programacao.ciclum**

Submit php não funciona