similarity measures with missing values

Question: How to handle missing values when computing similarity (or distances)?
(I have binary feature values and do use the simple matching coefficient, but I feel that the answer to this question may be more general)
I can think of two options:

Remove missing values
Count missing values as error

But removing null values has the problem that a high score can be achieved with only one/a few values (see example A). And counting missing as error has the problem that real miss matches should be counted higher than missing values (see example B).
Is there a technique that has none of these shortcomings?
Example A
<code>           Instance1 Instance2
  Feature1 missing   missing
  Feature2 1         missing
  Feature3 0         0
  Feature4 missing   1
  Feature5 missing   missing
  Feature6 missing   missing

  Simple-matching-similarity-REMOVE = 1/1 (twice as high as B)
  Simple-matching-similarity-COUNT-AS-ERROR = 1/6
</code>
Example B
<code>           Instance1 Instance2
  Feature1 missing   1
  Feature2 1         0
  Feature3 0         0
  Feature4 1         1
  Feature5 1         0
  Feature6 0         missing

  Simple-matching-similarity-REMOVE = 2/4
  Simple-matching-similarity-COUNT-AS-ERROR = 2/6 (twice as high as A)
</code>
Comment: As far as missing is an unknown _existing_ value (which might be really 0 or 1, but you don't know it), the approach that you call "error" is not justified. I think you ought either to completeky remove rows with missing data (= _listwise_ deletion) or to do an _imputation_ (such as, for example, hot-deck imputation). You may also do _pairwise_ deletion, but it is generally considered statistically not a very good choice.
Comment: Completely removing rows with missing values is not an option, because in our example, we have 80% missing values, and the features are our target variables (multi-label-classification). We have used imputation for the modelling.
But now, I need the similarity measure to evaluate clusters of instances, and this should be done on the raw, unimputated data.
Comment: 80% of missing data is too much. Imputation is recommended to do with no more than 20% of missing data.
Answer: As far as I know, there isn't any formal theoretical framework that describes how to do this.  A heuristic technique that I've seen used in the past for similar types of missing-data problems is to replace missing values (but for purposes of performing the relative distance calculation only, not for the rest of your analysis!) with a sensibly chosen default value.  In your case, a sensible default might be to choose the mean or median of each feature value, tabulated over all instances which do have the feature present.  Alternatively, if you wanted to penalize instances with missing values a little bit (i.e., causing them to be treated a little more "conservatively", or more likely dissimilar, since you just don't know for sure) you might substitute missing feature values with the mean value plus some margin, say one sigma or something like that.  As I said, it's a heuristic technique, so there's no well-defined "correct answer", you'd have to use your own judgment in deciding precisely which values to substitute for the missing instances.
Comment: Thanks for your answer, I considered your suggestions. I feel, that for my problem, just weighting the score (w/o missing values) with the number of actual matches works quite well..
Answer: There is a method to calculate Jaccard Similarity with missing values. Please refer this paper.
This method uses matrix factorization technique. To use this method, you first need to calculate the approximate pair-wise Jaccard similarities for your entire dataset. Then you have to use matrix factorization to calculate the exact Jaccard values. However, this method can be used only for small datasets.

similarity measures with missing values


What could cause random files being uploaded without permission?

Question: I have been having issues lately with a certain directory.  It seems someone is placing files into it, or something of that sort, and any attempt to delete them is successful, HOWEVER they reappear over time (maybe not the exact same ones, but random files).
I will provide you the information I can and several pictures of my problem:

sandbox.mys4l.com/visual/files/b1.jpg
Files like this have been appearing in my /visual/ folder, and I have no clue where they are coming from.
sandbox.mys4l.com/visual/files/b2.jpg
This is what is inside on of those weird files, it appears to be nothing problematic.
sandbox.mys4l.com/visual/files/b4.jpg
As you can see, in the time it took me to take the first picture, more odd files showed up.  These log files are also being uploaded to this directory, and I know I didn't put them there.
sandbox.mys4l.com/visual/files/b7.jpg
This inside one of these mysterious .log files, I'm not sure what it's all about.

These files only appear to be going into this specific area, and I'm not sure of their origin, only that they will not go away.  I have done a full system scan at least twice with an up-to-date virus scan, and have looked for an unknown script which may be writing them there.  Nothing has come up, so I come to you guys as I hear this is the best place to find answers.
Hope this problem has a solution!
Comment: @Dustin - what platform? What programs are involved? What languages?
Comment: @warren -Only problem is that these files are appearing.  Not sure what they are from or how to stop them.  Not sure if my FTP program is to blame or not, but I'm using CuteFTP 8.0 Professional.  As for language and platform, I don't understand what you mean.
Comment: by language and platform, I am referring to what platform is the host that this is occurring on running? Windows, Linux, FreeBSD, etc. What application is running on that host? Do you know what language it is written in? Do you own the server that is running the app?
Comment: @warren -I'm not sure what platform the server may be running, I believe it is Windows, but I am not sure.  I do not own the server, I am renting out the domain through Bravenet(.com).  I have looked into the odd files and noticed an even more weird line, it seems they are throwing a copyright with my page link on it.  These files seem to be articles...
Answer: you most probably got your password to ftp stolen, and some bot uploads files that are later used to infect / spam visitors. or those files are referred by other infected pages.
or even worse - you have a backdoor / some guests on your server.
Comment: @pQd -I have deleted my ftp login, and remade it several times using different names and passwords.  Do you have any suggestions on how to get rid of this, and prevent it in the future?
Comment: @Dustin - what software do you host there? can it be some popular script that has security holes allowing upload of such files? did you tried to check what http requests were done when files were created?
Comment: Sorry my response took so long.  I currently do not host any software, I do however allow people to host music that they own, and images.  I'm a newbie, so I'm not sure how to do a http request or anything like that.  Sorry for my ignorance.
Comment: @Dustin - just ask your hosting provider to send you logs from around time those suspicious files were created.


Ler registro de histórico de SMS no iPhone usando Xamarin

Question: Preciso ler o registo de histórico de SMS enviados / recebidos no iPhone, e o seu tempo gasto em cada envio. Não sei como fazer e não encontrei material em português sobre o assunto. Alguém pode me ajudar nessa questão? Abri uma outra pergunta relacionada sobre o registro de chamadas no iPhone.
Comment: acho que tu poderia juntar todas tuas perguntas em uma coisa só, elas são bastante parecidas.
Comment: Para registrar: eu não acho que esta pergunta seja duplicata, votei para fechar como "ampla demais".
Comment: Realmente não tem nada de duplicata. O problema é outro.
Comment: Coloquei esta pergunta (e as irmãs dela) em discussão no meta: http://meta.br.stackoverflow.com/questions/521/tres-perguntas-amplas-demais-o-que-fazer
Comment: Editei a pertunta para deixá-la mais clara.
Comment: @Bacco, técnicamente não é ampla demais. Ele cita a plataforma (iOS), cita a liguagem Xamarin/C# e exatamente o que ele precisa: A lista de mensagens de SMS enviadas e recebidas. A pergunta pode até ter outros problemas (eu acho que não tem, mas é minha opinião pessoal), mas não é ampla não.
Comment: @Alexandre Marcondes talvez seja mais o caso de downvote, do que fechar. O que mais me preocupa é o fato dela estar como duplicata, isso realmente está incorreto.
Comment: @Bacco isso me incomoda também. Não sei o que podemos fazer para acelerar. Os três tickets dele já estão prontos para ficarem abertos e serem respondidos e nenhum é duplicata do outro.
Answer: A Apple não tem nenhuma API no SDK para acessar o registro de SMS por motivos de segurança e privacidade: para impedir que hajam apps que coletem informações privadas do usuário e usem ou compartilhem com terceiros.
No Android há como fazer isso e se o aparelho tiver Jailbreak você vai poder também fazer isso, mas não é uma boa prática exigir que seu usuário tenha jailbreak, pois esse tipo de app não é aprovado na AppStore oficial da Apple, apenas na Cidia (App Store do muindo do jailbreak).
Infelizmente não dá para fazer isso que você precisa.


Module Block destroys head

Question: I'm struggling with blocks and a custom module. What I've got so far is a module which generates a block with that code:
function commerce_bulk_invoice_block_info() {
    $blocks = array();
    $blocks['showinvoiceoptions'] = array(
        'info' => t('Define Invoice Options'),
    );
    return $blocks;
}
function commerce_bulk_invoice_block_view($block_name = '') {
    if ($block_name == 'showinvoiceoptions') {
        echo '
';
        if(!isset($_SESSION['selected_entities'])) {
            echo 'select a product first';
        } else {
            echo '
SESSION Variables:
';
            echo '
';
            print_r($_SESSION['selected_entities']);
            echo '
';
        }
        echo '
';
    }
}

My problem is, the content generated by hook_block_view is not shown in the selected region but directly after the opening body tag. And it pulls the whole head-content into the body. Anyone knows a solution for that?
Answer: By the documentation, you should return content you want in block, not echo it.
$block['subject'] = t('Subject'); // String here
$block['content'] = t('Content'); // Array or string here
return $block;

Module Block destroys head


Clients connected to "vlan hub" can't access internet on dedicated, natted vlan

Question: I'm trying to get my network up and running with my new provider's vdsl router, which can't operate in "modem-mode" or "bridge-mode".
My setup so far is as follows:
I have two managed switches, both with several 192.168.x.0/24 vlans (vlanX is mapped to network 192.168.X.0/24). All these vlans are connected over a trunk port to my openbsd firewall (ip forwarding enabled). Each computer on the network has the address of the virtual nic on the firewall set as default gateway.
For example, the network for vlan20 is 192.168.20.0/24, the firewall has the address 192.168.20.1 and this address is set as the default gateway for each client on this vlan.
With this setup, I can ping between all clients on the same or different vlans/switches.
Here comes the problem:
The VDSL-Router is connected to the firewall via another dedicated 192.168.x.0/24 vlan, say 192.168.100.0/24, with the firewall ip 192.168.100.2 and the vdsl-router ip 192.168.100.1. I've setup 192.168.100.2 as the default gateway on the firewall and can now sucessfully reach the internet from the firewall, but not from my other machines. Traceroute shows that the packets are stuck at the gateway of the client-vlan, for example, if I try to reach something outside from client 192.168.20.100, packets are stuck at the client gateway 192.168.20.1.
I have disabled all firewall rules (for testing purposes) and also tried enabling nat on the firewall for the vdsl-vlan (the vdsl-router of course does its own nat on the internet port as well). Am I missing something obvious?
The vdsl-router is connected to an "access" port of the switch and thus not aware of the vlan. Also, due to physical restrictions, I can't remove one of the switches in between.
Thanks for your help!
Comment: It sounds like the config for the OpenBSD firewall might be the culprit, more info on that would be helpful.  IP forwarding should be configured correctly.  VLAN 20, for example, should have a default route of 192.168.100.2 and know how to route to that VLAN.
Comment: I've set net.inet.ip.forwarding=1 via sysctl. Unless I'm getting something wrong, this should have the effect that if a packet arrives on any of the interfaces with a destination not on the corresponding subnet, it is forwarded according to the routing table. Here's an excerpt of the table (mac addresses partially greyed out), I've only added the default route manually: http://pastebin.com/KDnjSSyB
Comment: In that case it looks fine from a routing perspective.  Sorry I'm no further help
Answer: Maybe I'm missing something from your setup but here's what I think is the issue.
If the Modem is connected to one of the switch as pictured in your design via an access port, then that modem needs to have routes for all of your VLAN pointing back to your OpenBSD firewall.
That would explain why the Modem seems to only "see" the Firewall since it's layer 2 all across.
But it would fail to reach one of the PC because the Modem does not know about any other IP routes (execept for the directly connected ones).
Comment: Good point, but then if I do nat on the vlan-port of the firewall which is connected to my vdsl-router, the problem should disappear, right? Unfortunately, it doesn't... Just to be clear: The vdsl-router also does nat towards the internet and runs its own dhcp server. I have simply connected a vlan interface of the firewall to one of the internal ports of the vdsl-router (that there's a switch in between shouldn't make a difference).
Comment: And what does the NAT rule look like? Do you NAT behind the external interface of the firewall or behind another IP?
Comment: I NAT behind the external interface of the firewall. Strangely enough, after inserting the routes to my vlans on my vdsl-router as you suggested, things are working now (with NAT on the firewall). Apparently, I haven't understood something about IP-networking... again, in my opinion these routes shouldn't be necessary as the vdsl-router just needs to know how to reach the NAT-Interface on the firewall. I'll look into it further when I have more time (i.e. on the weekend). Anyway, thanks for your help!


How to avoid bias/avoid overfitting when choosing a machine learning model?

Question: My typical workflow in the past, when creating machine learning models, has been to do the following:

Decide on some candidate model families for the task at hand.
Divide dataset into train and test sets. Set aside the test set, and pretend it doesn't exist.
Do some EDA on the training set.
For each model candidate, (depending on the results of the EDA), tune hyperparameters and estimate generalization error via nested cross validation.
Choose whichever model family has the best performance in step 4.
Check performance of the model chosen from step 5 on the holdout test set.

I feel like this still introduces bias. In step 5, we're picking a model family based on performance in the nested cross validation. Is this safe to do?
Also, how do we decide on the hyperparameters to try? I feel like one's choice is to get an idea for the ideal range and then do a grid search, or else do a wide grid search over hyperparameters which might leave some performance on the table.
Finally, how do you actually choose a final model?
Comment: There is a possible step between 5 and 6, which is once you have your selected model family and tuned its hyperparameters, you can retrain on the full training set.
Comment: The cross-validation step 4 is there to reduce excess overfitting, including choosing hyperparmeters which tend to achieve this.  It cannot eliminate the risk, and the testing step 6 is designed to indicate how well you turn out to have done.
Answer:
If you are doing EDA then you will get to know about if there is a bias or not from data side. On the other hand, if you ask me about the model being biased then I can say that if you can reduce the overfitting then it's highly likely that you will reduce some of the bias. Or you can choose another model to train.
Second, if you think that nested cross validation is introducing some bias, then you can try with other cross validation techniques. But cross validation techniques are there to deal with these cases.
Third, about which hyperparameters to tune, there is a framework called optuna: https://optuna.org/. It uses random search over the range of parameters. Now why I am suggesting you is because it can plot the hyperparameters that are actually influencing the model. So you can see the hyperparameters that are actually useful to tune. So do check it out.
Lastly, choosing a final model could be tricky. Why? Because different domains have different use cases. For e.g. banking, domain needs the models that can be explained easily. Other domain might require you to create a model that gives the metric score as good as possible. So, it depends on your domain and use case. But in general case, you need model to be less biased, highly explainable (except you are using black box model like Neural Network). RandomForest is a good start as it has low bias and low variance.
Hope I answered your questions.


Hooking in to the email verificatin process

Question: In Drupal 7 (7.22), we'd like to hook into the email verification process for new users.  Perhaps the one time password node.  We need to do a test on the email address and set a value in a custom user field based on what we find.  This will allow us to give software activation terms to users with email matching certain business partner domains, for example.
Extra credit: we'd also like to enable a particular role for these users.  The first step, though, is to figure out where to hook in our php to do a regex test of $user->mail.
Answer: It would make more sense to create a rule to activate on After saving new user account. You could do a Text comparison on Account:mail using your regular expression, and if the condition was TRUE, Assign a role for the  new user. It would be waiting for them when they verified their account.
If your requirements change for the site, it is easier to make an alteration through the UI than it is to track down the code that has to be changed.
The same rule can Set a data value in the custom user field if you wish.
EDIT:
Install and activate the Rules module. Then go to /admin/config/workflow/rules and create a new rule,  calling it, lets say 'Upgrade new user' and set it to react on After saving a new user account.
When the configuration screen comes up Add condition and select Text comparison (wait for the AJAX request to complete) then set the Data selector to account:mail. Under Matching Text, click the Switch to direct input button, enter your regular expression in the  Value box, and at the bottom use Regular expression for the Comparison operation. Then save the condition.
Then add two actions; First add Add user role (wait for the ajax to complete) and you should see account as your data selector. Then choose the role(s) to add and save the action.
Then add an action to Set a data value. In the Data selector, find your field, and click Continue. In the Value box using <?php ... ?> delimiters, enter the code you need to return your activation key.
Save everything and try it out.
Comment: Jeez...  hurry UP!!!   :)
Comment: That sounds like an excellent idea.  Now I just need to figure out how to create a rule. :)  I'm a little green in the Drupal 7 world.
Comment: First things first, I guess. Do I need both the Rules module and the Entity_API module for this?
Comment: Yes. Wait for me to finish :-)
Comment: I appreciate your help, btw.
Comment: You will notice the _wait for the ajax to complete_ comments. Rules does not play well if you are impatient and click the button to hurry things up. Good luck.
Comment: I got the custom field working with rules straight away.  The regex evaluation works correctly and we're pushing a value into the custom field correctly.  Thank you!  I'm still trying to get adding the role to work.  I'm miles ahead of where I was though.  Thanks a bunch!
Comment: The role is the easiest part. There are two input modes: Data selector and Direct input. Make sure that you are using a Data selector for the action, and it is `account`.
Comment: I have it working when changing an existing user to have a matching email address.  On a new user I get this failure: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '4048-6' for key 'PRIMARY': INSERT INTO {users_roles} (uid, rid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1); Array ( [:db_insert_placeholder_0] => 4048 [:db_insert_placeholder_1] => 6 ) in user_save() (line 603 of /home/.sites/111/site30/web/modules/user/user.module).
Comment: My guess is timing - perhaps the user isn't completely created when the rule fires.
Comment: Might be timing. Try a second rule that fires on the event _After updating an existing user account_ or try _User has logged in_. Use the same email condition, and make sure they have the role.
Comment: The _After updating an existing user account_ version is the one that works correctly... making me think it might be timing. I've added a 'sleep(10)' php action prior to setting the custom field and adding the role, but that didn't hep (or seemingly delay anything).
Comment: Then work with two rules. When the user first logs in and sets their password, the account is updated. Probably a good idea to set a second condition to test if the role is not there so the user object is not reloaded unneccessarily.


How do you decide whether a ball is in or out

Question: In volleyball, how do you decide if a ball is in or out?
Is it like in tennis, where you would look at the ball from the top perspective at the moment it touches the ground and see if it crosses the line or not? Because that can, in some circumstances, give a different result than if you looked at the ball from the perspective of a player inside the court. From the perspective of the player, the ball would not touch the line, but from the top perspective, it would.
Comment: Welcome to Sports SE! I found your English to be reasonably comprehensible outside of minor grammar improvements.
Comment: Please someone with proper english knowledge edit my question to make it clear and concise. I'm not very skilled at english with regards to sports speech. Thank you
Answer: First of all, your premise is wrong: In tennis, the exact Rule 12 of ITF Rules of Tennis is:

If a ball touches a line, it is regarded as touching the court bounded by that line.

Note that the mark of the ball touching the ground is not even a circle; it is not like there is a moment at which we should take a projection of the ball to the ground and decide. A part of the ball might touch the ground e.g. because of compression caused by the impact, and in that case, the ball is in.

(On the other hand, football (soccer) is a sport with such a rule: The ball is out if and only if its orthogonal projection is outside the lines.)
Now, to the volleyball:
The respective rule (FIVB Official Volleyball Rules 2013–2016, Rule 8.4.1) states:

[The ball is "out" when] the part of the ball which contacts the floor is completely outside the boundary lines

with the FIVB Refereeing Guidelines and Instructions stressing "It is essential to realise the importance of the word 'completely'".
So: There is no "perspective" involved at all. You need to determine the part of the ball which contacted the floor, and the decision is based on whether this part was outside the line. Or, to use your words, if any player could see the ball not touching the line from his/her perspective, the ball was out. (However, the fact that someone (including referee) saw the ball "above" the line does not necessarily prove anything.)


Better understanding of Drupal behaviors

Question: I read about Drupal behaviors today, and I tried writing the following code.
(function ($) {

  Drupal.behaviors.mymodule = {
    attach: function (context, settings) {
      $('#mymodule_id', context).change(function () {
        alert('Handler for .change() called.');
      });
    }
  };

}(jQuery));

Is Drupal.behaviors.mymodule the namespace?
What are the context and settings parameters passed to the Drupal behavior?
Is this the equivalent of document.ready()?
Can I attach any number of functions?
Can I define JavaScript functions which will be called somewhere?
Comment: This blog post from Lullabot does a great job of explaining the concepts around JS behaviors: https://www.lullabot.com/articles/understanding-javascript-behaviors-in-drupal
Comment: Good article here about behaviors for all the versions (D6, D7 & D8): https://www.benmarshall.me/drupal-behaviors/
Answer: In short, the advantage of Behaviors over document.ready() is that they are automatically re-applied to any content loaded through AJAX. mymodule is your namespace, which needs to be unique. context is the part of the page for which this applies, for example a part of a form that has been updated with AJAX. You can attach multiple behaviors, but I think you need to use a unique name (mymodule) for each of those.
Have a look at the following resources for more information:

Drupal behaviors
Drupal Javascript API
Changes in the Drupal JS API for D7
Answer: In easy words, Drupal.behaviors is a better way to implement jQuery.ready
Unlike jQuery.ready which only runs once when the DOM is ready, Drupal.behaviors can be ran multiple times during page execution.
For example, in views infinite scroll, more elements will be loaded when user clicks load more button, hence , DOM will change after initial load.
What if we want to add classes to the newly added elements? Here Drupal Behaviors come handy.

The behaviors will be executed on every request including AJAX
  requests.

Drupal will call attached behaviors when the DOM is loaded and also when it is changed by Ajax, passing in two arguments
context and settings
The first time Drupal.attachBehaviors() is called, the context variable contains the document object representing the DOM, but for the rest of the calls context will hold the affected piece of HTML.
settings contains information passed on to JavaScript via PHP, it is similar to accessing it via Drupal.settings.
Furthermore, modules may call Drupal.attachBehaviors() as well.


How does the equation $y'=\frac{y^2}{1-xy}$ with $y(0)= 1$ satisfy the relation $y=e^{xy}$?

Question: My book says that the equation satisfies this relation, but I don't understand why.
Answer: Hint
Consider $x'=\frac {dx}{dy}$
to get the classical form:
$$x'+ \frac x y= \frac 1 {y^2}$$
Multiply by y the equation :
$$(xy)'=\frac 1 y$$
Integrate:
$$xy=\int \frac {dy} y$$
$$xy=\ln(y)+K$$
$$y=Ke^{xy}$$
Use $y(0)=1$ to get the value of K
Comment: Thank you for answering. I finaly understand.
Comment: yw Chen ... ...........
Answer: You can verify it easily by differentiating $$y=e^{xy}$$
So that $$y'=e^{xy}(y+xy')=y(y+xy')$$ and the result follows immediately
Comment: @MohammadRiazi-Kermani You may not, but I do.
Comment: You are correct. Now I got it right.


500 Action does not contain method named

Question: Estou efetuando uma chamada de uma action mas recebo a exception de que o método não existe na Action:
500 Action[/historicoAction] does not contain method named listar

Alguém saberia dizer o que está acontecendo?
Segue arquivos:
struts-config
<form-bean name="historicoForm" type="br.com.liquigas.portal.form.HistoricoForm"/>

<forward name="/historicoAction" path="/historicoAction.do"></forward>

<action path="/historicoAction" type="br.com.liquigas.portal.action.HistoricoAction" name="historicoForm" scope="request" parameter="method">
    <forward name="success" path="/jsp/historico/historicoPedido.jsp"></forward>
    <forward name="failure" path="/jsp/historico/erro.jsp"></forward>
    <forward name="historicoPedido" path="/jsp/historico/historicoPedido.jsp"></forward>
    <forward name="historicoProduto" path="/jsp/historico/historicoProduto.jsp"></forward>
</action>

JSP
Avançar

Javascript
function avancar(){
    $("#method").val("listar");
    $('#historicoForm').submit();
    alert("Avancou");
    return false;
}

FORM
public class HistoricoForm extends GenericForm {

    private static final long serialVersionUID = 7434349782026204632L;

    /**
     * Filtro da tela com o numero do pedido
     */
    private String numeroPedido;

    /**
     * Filtro da tela com o codigo do cliente
     */
    private String codigoCliente;

    /**
     * Filtro da tela com a Dta Inicial da pesquisa
     */
    private String dataInicial;

    /**
     * Filtro da tela com a Dta Final da pesquisa
     */
    private String dataFinal;

    /**
     * Lista com as informações do historico
     */
    private List historicoList;

    /**
     * Lista com as informações do Histórico do produto, a tela de detalhe do historico
     */
    private List historicoProdutoList;

    /**
     * parametro da jsp
     */
    private String method;

    /**
     * @return o numeroPedido
     */
    public String getNumeroPedido() {
        return numeroPedido;
    }

    /**
     * @param numeroPedido o numeroPedido a ser configurado
     */
    public void setNumeroPedido(String numeroPedido) {
        this.numeroPedido = numeroPedido;
    }

    /**
     * @return o codigoCliente
     */
    public String getCodigoCliente() {
        return codigoCliente;
    }

    /**
     * @param codigoCliente o codigoCliente a ser configurado
     */
    public void setCodigoCliente(String codigoCliente) {
        this.codigoCliente = codigoCliente;
    }

    /**
     * @return o dataInicial
     */
    public String getDataInicial() {
        return dataInicial;
    }

    /**
     * @param dataInicial o dataInicial a ser configurado
     */
    public void setDataInicial(String dataInicial) {
        this.dataInicial = dataInicial;
    }

    /**
     * @return o dataFinal
     */
    public String getDataFinal() {
        return dataFinal;
    }

    /**
     * @param dataFinal o dataFinal a ser configurado
     */
    public void setDataFinal(String dataFinal) {
        this.dataFinal = dataFinal;
    }

    /**
     * @return o historicoList
     */
    public List getHistoricoList() {
        return historicoList;
    }

    /**
     * @param historicoList o historicoList a ser configurado
     */
    public void setHistoricoList(List historicoList) {
        this.historicoList = historicoList;
    }

    /**
     * @return o historicoProdutoList
     */
    public List getHistoricoProdutoList() {
        return historicoProdutoList;
    }

    /**
     * @param historicoProdutoList o historicoProdutoList a ser configurado
     */
    public void setHistoricoProdutoList(List historicoProdutoList) {
        this.historicoProdutoList = historicoProdutoList;
    }

    /**
     * @return o method
     */
    public String getMethod() {
        return method;
    }

    /**
     * @param method o method a ser configurado
     */
    public void setMethod(String method) {
        this.method = method;
    }

ACTION
public ActionForward listar(ActionMapping mapping, ActionForm form,
        HttpServletRequest request, HttpServletResponse response) throws Exception {
    ...
}

Answer: Galera, consegui resolver meu problema com uma simples alteração nos parâmetros do método ActionForward. O que apresentava problema era o método:
public ActionForward detalhe(ActionMapping mapping, ActionForm form,
            HttpActionRequest request, ActionResponse response) throws Exception {
}
Tentei também com o:
public ActionForward detalhe(ActionMapping mapping, ActionForm form,
            HttpRequest request, ActionResponse response) throws Exception {
}
e não funcionou....
Corrigido com o
public ActionForward detalhe(ActionMapping mapping, ActionForm form,
            ActionRequest request, ActionResponse response) throws Exception {
}


Armazenar credencias de um repositório Git no Windows sem SSH

Question: Como configurar o Git em um repositório local para que não peça o login e a senha toda vez que for fazer um Push ou Pull?
Meu servidor Git não suporta SSH. Apenas HTTP.
Tenho somente o username e a password de usuário no Git.
Comment: http://oraculum.blog.br/blogoraculum/index.php/salvar-senha-do-git-no-windows-com-git-credential-winstore/ da uma olhada nesse  link
Comment: Qual S.O.? Windows ou Linux?
Comment: Windows @MarcosRegis
Answer: Existem duas possibilidades.
Acessando o repositório via SSH
Configure eu projeto para acessar o repositório usando um par de chave pública e chave privada.
Primeiro você deve alterar o origin para acessar o repositório remoto por meio do protocolo SSH ao invés do protocolo HTTPS (esse caminho você vê na página do repositório):
git remote set-url origin git://<caminho do seu repositório>

Depois você deve criar um par de chave pública e chave privada, e configurá-los no Github (ou no serviço de Git que você usa). Nessa pergunta eu explico direitinho como fazê-lo.
Usando o credencial helper
O Git ainda dá a possibilidade de armazenar seu usuário e senha em um keychain próprio, se modo que você possa usar HTTPS e não precise informá-los em todo git pull e git push.
A única coisa que você precisa fazer é informar ao Git para usar o keychain do sistema operacional. Abaixo os procedimentos para cada um deles.
Linux:
git config --global credential.helper cache

OSX:
git config --global credential.helper osxkeychain

Windows:
git config --global credential.helper wincred

Depois disso é só clonar o repositório via HTTPS (ou fazer qualquer operação que exija conexão, como git pull ou git push), informar usuário e senha, e o Git nunca mais os perguntará novamente.
PS: o credencial helper só está disponível na versão 1.7.10 para cima; verifique a versão do Git instalada com git -v.
Comment: Esqueci de colocar na pergunta meu servidor Git não suporta SSH. Somente tenho o "username" e a "senha" para usar o servidor do Git.
Comment: Entendi. Alterei a resposta para suas necessidades.
Comment: Digitei o comando do *credential helper*, mas não deu nenhuma opção a mais para digitar as credenciais.
Comment: A única coisa que você precisa fazer é informar ao Git para usar o keychain do sistema operacional. No caso do Linux esse comando é `git config --global credential.helper cache`; no OSX é `git config --global credential.helper osxkeychain`; e no Windows é `git config --global credential.helper wincred`. Depois disso é só clonar o repositório via HTTPS, informar usuário e senha, e o Git nunca mais os perguntará novamente.
Comment: Legal. Funcionou! Creio que é interessante colocar esses comandos na sua resposta, pois a solução vai estar imediatamente disponível para outros que aqui chegarem. Vlw!
Comment: Legal, valeu! Complementei a resposta conforme sua sugestão. :)
Answer: Se for msysgit em Windows 7 e não se preocupar em ter sua senha de forma aberta em um arquivo use este método.
Abra um prompt de comando e execute:
setx HOME %USERPROFILE%

A variável %HOME% será criada com o valor 'C:\Users\SEU USUARIO'
Agora abra esse diretório e crie um arquivo com o nome _netrc com o seguinte conteúdo
machine <hostname>
login <login>
password <password>

Comment: Fiz o que escreveu mas não foi.
Comment: como ficou seu arquivo _netrc?
Comment: Ficou assim: machine https://gitlab...
login meuUsername
password minhaSenha


Question about existance of global minimum in a strictly convex function

Question: Is it true that a strictly convex function must have a global minimum ?
I know that every strictly convex function has at most one global minimum and each local minimum is a global minimum but I can not prove or disprove the statement.
Strictly convex function: The real-value function that for every $x$ , $y$ $\in$ $R$ and $t$ $\in$ $(0,1)$ we have:
$$f(tx + (1-t)y) < tf(x) + (1-t)f(y)$$
Comment: Try to find a counterexample.
Comment: Hint for counter example is: the most famous convex function
Comment: That is strictly convex. (Note: it must be $t \in (0,1)$, not $t\in [0,1]$.)
Comment: I have a counter example for convex function : $e^x$ but I have no idea about strictly convex function
Comment: but $x^2$ is not a counter example
Answer: No, take the function $f: (1, 2) \to \mathbb{R}$, $x \mapsto x^2$. Alternatively, you can consider $\exp: \mathbb{R} \to \mathbb{R}$.
This statement can even fail when the domain is compact, take for example the function $f: [0, 1] \to \mathbb{R}$ that is defined by $f(x) = \begin{cases} 1 & x = 0 \\ x^2 & x > 0\end{cases}$.
If the domain is compact and $f$ is continuous, the statement holds. The existence of a minimum follows directly from the compactness of the domain and the continuity of the function. If $f$ assumes its minimum at two different points $x \ne y$, then by definition of strict convexity $f\left(\frac{x + y}{2}\right) < \frac{1}{2}(f(x) + f(y)) = f(x)$, in contradiction to the choice of $x$.
Comment: That's the example of the exponential function.
Comment: Yes it is. The exponential function is strictly convex and doesn't have a minimum in the reals. Note that a minimum is not the same as an infimum. An infimum is only called a minimum if its value is actually attained by the function.
Comment: thank you .what if domain of $f$ is $R$
Comment: yes you are right I mixed up strongly convex functions with strictly convex functions. what can we say about strongly convex functions
Comment: I have neither a proof, nor a counterexample in this case. Since you've initially asked about strictly convex functions, I suggest you open a new question asking about strongly convex functions and link to this thread.


How to pass parameters to a Local report that has object data source in asp.net web application?

Question: I want to pass parameters to local reports that have Object Data source(gets list of data from c#method that needs parameters) , how can I do that ?
    #region Parameterized Report
            ReportViewer1.ProcessingMode = ProcessingMode.Local;
            string FolderPath = Server.MapPath("~/LocalReports");
            string ReportName = "ParameterizedReportWizard.rdlc";
            string DataSetName = "DataSet1Parameter";
            string SelectedMethod = "GetSomeEmployees";
            string Parameter = "ID";

            ReportViewer1.LocalReport.ReportPath = Path.Combine(FolderPath, ReportName);
            ObjectDataSource objDataSource = new ObjectDataSource() { ID = DataSetName, TypeName = "BussinessLogic.Custom", SelectMethod = SelectedMethod};
            ReportParameter p1 = new ReportParameter("ReportParameter1", "1"); //I pass the same parameter name tha exist in report definition and it's value is on 1 as  static value for now

            this.ReportViewer1.LocalReport.SetParameters(new ReportParameter[] { p1});

            ReportDataSource datasource = new ReportDataSource(DataSetName, objDataSource);
            ReportViewer1.LocalReport.DataSources.Clear();
            ReportViewer1.LocalReport.DataSources.Add(datasource);

            #endregion

Comment: Here's a related post. http://stackoverflow.com/questions/8802707/setting-the-datasource-for-a-local-report-net-report-viewer
Comment: But My case is that the method -that return my list of objects= has parameters , how can i do that, plz help?
Answer: All data access for local reports should be performed before binding it to your report. If possible, filter at the source.
You need to add to the ObjectDataSource.SelectParameter collection.
string DataSetName = "DataSet1Parameter";
string SelectedMethod = "GetSomeEmployees";
string Parameter = "ID";

ObjectDataSource objDataSource = new ObjectDataSource() { ID = DataSetName, TypeName = "BussinessLogic.Custom", SelectMethod = SelectedMethod};
    objDataSource.SelectParameters.Add(new Parameter("id"));

then subscribe to the ObjectDataSource.Selecting event to set the parameter.
protected void objDataSource_Selecting
    (object sender, ObjectDataSourceSelectingEventArgs e)
{
    e.InputParameters["id"] = "1";
}

Comment: What about displaying the parameters in the report? should i add the parametes controls mannally or it can be automatically generated as it has been happened in case of the report server???????
Comment: your parameters have to be in your report definition to use them in your report, so add them in normally.
Comment: okay ,Actually i did that in the report definition. but SetParameters() still cannot help me?!
Comment: can you update your post to show how you are setting your parameters?
Comment: I have updated it and  I have get thhis error after run  :"An error has occurred during report processing.
ObjectDataSource 'DataSet1Parameter' could not find a non-generic method 'GetSomeEmployees' that has no parameters."
Comment: It looks like the error is happening when the `ObjectDataSource` is trying to call the GetSomeEmployees method and you are not passing a parameter. See my updated answer.
Comment: Dear Sir Cam Bruce, I want to thank u , and i'll mark you answer as correct one But for now u now that  we have passed hard coded parameter value what if i want to make it user defined???? Many Thanks in advance :)
Comment: That parameter does not need to be hard coded. It can be anything (drop down value on your aspx page, query string from page, etc)
Comment: I got your  point but do you know if there is any way to make it auto generated as The same case when I deploy  my ssrs reports on the report server?
Comment: That is the limitation of local reports. You have to perform the data access for your parameters yourself. i.e binding the data to dropdowns.


Curva não se ajusta no lines

Question: Estou tentado incluir a curva predita no meu gráfico, mas ela não se ajusta. Estou usando o seguinte argumento:
plot(df$n.kill, df$body)
lines(df$n.kill,predict(fit,df$n.kill))

Dados:
n.kill<-c(79,4,86,9,10,49,45,260,10,8,182,16,824,2,11,112)
body<-c(160,1.5,23,40,4.5,4.5,0.8,3,1.2,60,70,0.5,35,30.5,4,47)
df<-data.frame(n.kill, body)
ca=max(df$n.kill)

eps <- .Machine$double.eps^0.5
fit <- nls(n.kill ~ a*(1 - exp(-b*body))^c, data = df,
           start = list(a = ca, b = 0.05, c = 1),
           algorithm = 'port',
           control = nls.control(maxiter = 100, tol = eps))

Comment: Tente ao contrário, `n.kill` é a resposta, deverá estar no eixo dos y tal como o resultado de `predict`. No eixo dos x deverá estar `body`.
Comment: O ajuste da curva ainda continua com "erro", são plotadas várias linhas emaranhadas, ao invés de somente uma com a curva predita.
Comment: Veja a terceira resposta aqui, a que usa a função `curve` https://pt.stackoverflow.com/questions/6267/como-colocar-a-linha-de-regress%c3%a3o-em-um-gr%c3%a1fico
Answer: Com os dados da pergunta aqui estão duas maneiras de traçar o gráfico com a curva prevista.
R base
O problema das linhas emaranhadas resolve-se ordenando a variável do eixo dos x e, depois, usar essa variável ordenada para calcular os valores de y.
b <- sort(df$body)
new <- data.frame(body = b)
with(df, plot(body, n.kill))
with(df, lines(b, predict(fit, newdata = new)))

Em vez de ordenar body, outra maneira, talvez melhor, é criar um vetor para os valores do eixo dos x com seq.
b <- seq(min(df$body), max(df$body), length.out = 500)

O resto do código é exatamente igual.
Pacote ggplot2
Com o pacote ggplot2, não é necessário ajustar o modelo antes de traçar o gráfico, a geom_smooth faz isso desde que também se passem os argumentos usados no fit.
Mas cuidado, a geom_smooth só usa variáveis com os nomes x e y, as mesmas de aes(). É portanto necessário modificar a fórmula para usar esses nomes.
library(ggplot2)

ggplot(df, aes(x = body, y = n.kill)) +
  geom_point() +
  geom_smooth(method = 'nls',
              formula = y ~ a*(1 - exp(-b*x))^c,
              se = FALSE,
              method.args = list(start = list(a = ca, b = 0.05, c = 1),
                                 algorithm = 'port',
                                 control = nls.control(maxiter = 100, tol = eps)))

Comment: Muito obrigado novamente,  Rui.
Comment: @MiltondePaula O que fazer quando [alguém responde](https://stackoverflow.com/help/someone-answers) às suas perguntas.

Curva não se ajusta no lines


Bessel Functions of Half-Integer Order

Question: I recently came across the general form of Bessel Functions of half-integer order given by:
$$
J_{n+\frac{1}{2}}(x)=(-1)^n\sqrt{\frac{2}{\pi}}x^{n+\frac{1}{2}}\left(x^{-1}\frac{d}{dx}\right)^n\frac{\sin{x}}{x}.
$$
I am required to prove this using the recurrence relation:
$$
J_{s\pm 1}(x)=\frac{s}{x}J_s(x)\mp J'_s(x).
$$
I tried to prove by induction, but I think my main issue is that I am not sure what $\left(x^{-1}\frac{d}{dx}\right)^n\frac{\sin{x}}{x}$ actually means. I was thinking that the power $n$ is distributed across to $x^{-1}$ and $\frac{d}{dx}$ but that gives $x^{-n}$ which allows me to simplify the general form to be
$$
J_{n+\frac{1}{2}}(x)=(-1)^n\sqrt{\frac{2}{\pi}}x^{\frac{1}{2}}\frac{d^n}{dx^n}\frac{\sin{x}}{x}.
$$
If that is indeed true, textbooks should have presented this formula instead. I have used MATLAB to help me find the zeros using 2 different methods (the formula above, and by besselj) but gave me different answers. That is how I know that my simplification is wrong.
Could anyone kindly enlighten me what $\left(x^{-1}\frac{d}{dx}\right)^n\frac{\sin{x}}{x}$ means and how I can go about proving the general form using the recurrence relations?
Any form of help is deeply appreciated!
Thank you!
Answer: It means that you differentiate with regard to x, and then divide by x, and then repeat the entire process $($by repeatedly applying these two operations, in this exact order$)$, a total number of n times.

Bessel Functions of Half-Integer Order


probability of slot machine

Question: A person is doing 3 trials in 2 slot machines A and B.
The chance of winning in machine A is 0.4 and in B is 0.2.
In the first trial, this person choose the machine at random.
In each stage, if he wins, he would again choose the same machine, else he chooses the other machine.
1) what is the probability that the 2nd trial is machine A?
2) same question, if we know that the 3rd trial is done on machine A?
solution:
1) $\mathsf P(A_2)~=~\mathsf P(A_1,A_2)+\mathsf P(B_1,A_2)~=~1/2\cdot 0.4+1/2\cdot (1-0.2)~=~0.6$
2) $\mathsf P(A_2\mid A_3)~=~[\mathsf P(A_3\mid A_2)\cdot P(A_2)]/\mathsf P(A_3)~=~[0.4\cdot 0.6]/\mathsf P(A_3)$
I need help with calculating $\mathsf P(A_3)$ pls.
tnx.
Comment: Welcome to Math StackExchange. Check out this post to learn about how to format your posts using MathJax. http://meta.math.stackexchange.com/q/5020/74357
Answer:

1) what is the probability that the 2nd trial is machine A?

1) $\mathsf P(A_2)$ $=\mathsf P(A_1,A_2)+\mathsf P(B_1,A_2)\\=\tfrac 1 2\cdot 0.4+\tfrac 1 2\cdot (1-0.2)\\=0.6$

Yes, indeed it is.
Note: You have: $\mathsf P(A_2) =\mathsf P(A_2\mid A_1)\cdot \mathsf P(A_1)+\mathsf P(A_2\mid B_1)\cdot \mathsf P(B_1)$

2) same question, if we know that the 3rd trial is done on machine A?

2) $\mathsf P(A_2\mid A_3)$ $=\dfrac{\mathsf P(A_3\mid A_2)\cdot \mathsf P(A_2)]}{\mathsf P(A_3)}\\=\dfrac{[0.4*0.6]}{\mathsf P(A_3)}$

So far, so good.
By the Law of Total Probability:$$\begin{align}\mathsf P(A_3)
~ = & ~\mathsf P(A_3\mid A_2)\cdot \mathsf P(A_2)+\mathsf P(A_3\mid B_2)\cdot \mathsf P(B_2)
\end{align}$$
You can complete this now.

I got 14/25, am I right?

Short answer: Yes.
Longer answer: If you did this:-

 $$\begin{align}\mathsf P(A_3) ~ = & ~\mathsf P(A_3\mid A_2)\cdot \mathsf P(A_2)+\mathsf P(A_3\mid B_2)\cdot \mathsf P(B_2) \\ ~ = & ~ (0.4)\cdot (0.6)+(1-0.2)\cdot (1-0.6) \\ ~ = & ~ 0.56 \end{align}$$
Comment: I got 14/25, am I right? @Graham Kemp
Comment: @TalorT Yes. (Added text)

probability of slot machine


Query KTable in the same Application where it is created

Question: I have an Kafka streams application in which I read from a topic, do aggregation and materialize in a KTable. I then create a Stream and run some logic on the stream. Now in the stream processing, I want to use some data from the aforementioned KTable. Once I start the stream app, how do I get access to the KTable stream again? I don't want to push the KTable to a new Topic.
KStream<String, MyClass> source = builder.stream("my-topic");
KTable<Windowed<String>, Long> kTable =
            source.groupBy((key, value) -> value.getKey(),
                    Grouped.<String, MyClass >as("repartition-1")
                            .withKeySerde(new Serdes.String())
                            .withValueSerde(new MyClassSerDes()))
                    .windowedBy(TimeWindows.of(Duration.ofSeconds(5)))
                    .count(Materialized.<String, Long, WindowStore<Bytes, byte[]>>as("test-store")
                            .withKeySerde(new Serdes.String())
                            .withValueSerde(Serdes.Long()));

Here I want to use data from the kTable.
inputstream.groupByKey()
    .windowedBy(TimeWindows.of(Duration.ofMinutes(1)))
    .count(Materialized.<myKey, Long, WindowStore<Bytes,   byte[]>>as("str")
    .withRetention(Duration.ofMinutes(30)))
    .toStream()
    .filter((k, v) -> {
        // Here get the count for the previous Window.
        // Use that count for some computation here.
    }

Comment: If depends what you exactly want to do... Mabye you can use a `transform()` and connect the `KTable` store to the transformer?
Comment: Thanks for the response. In my Streams application, by doing some aggregation on a Stream, I am creating KTable of Windowed count by a key. In the same Kafka streams application, I want to access the count for previous TimeWindow. Is this doable without requiring an additional intermediate topic?
Comment: Should be doable -- what exact access to you need? As said before, you could connect the `KTable` state to a `transform()` to gain read access to it.
Comment: @MatthiasJ.Sax Updated the question. Please check the section marked with "Here I want to use data from the kTable."
Comment: You can replace `filter()` by `flatTransformValues(..., "my-key-count-one-minute")` and implement the filter in your transformer and can access the state via the context: https://docs.confluent.io/current/streams/developer-guide/processor-api.html#accessing-processor-context
Comment: I'll give it a shot. Thank you @MatthiasJ.Sax
Answer: You can add the KTable store to a processor/transformer. For you case, you can replace the filter with flatTransform (or any sibling like transform etc depending if you need access to the key) and connect the store to the operator:
inputstream.groupByKey()
    .windowedBy(TimeWindows.of(Duration.ofMinutes(1)))
    .count(Materialized.<myKey, Long, WindowStore<Bytes, byte[]>>as("str")
        .withRetention(Duration.ofMinutes(30))
    )
    .toStream()
    // requires v2.2; otherwise use `transform()`
    // if you don't need access to the key, consider to use `flatTransformValues` (v2.3)
    .flatTransform(
        () -> new Transformer<Windowed<myKey>,
                              Long,
                              List<KeyValue<Windowed<myKey>, Long>>() {

            private ReadOnlyWindowStore<myKey, Long> store;

            public void init(final ProcessorContext context) {
                // get a handle on the store by its name
                // as specified via `Materialized` above;
                // should be read-only
                store = (ReadOnlyWindowStore<myKey, Long>)context.getStateStore("str");
            }

            public List<KeyValue<Windowed<myKey>, Long>> transform(Windowed<myKey> key,
                                                                   Long value) {

              // access `store` as you wish to make a filtering decision

              if ( ... ) {
                  // record passes
                  return Collection.singletonList(KeyValue.pair(key, value));
              } else {
                  // drop record
                  return Collection.emptyList();
              }
            }

            public void close() {} // nothing to do
        },
        "str" // connect the KTable store to the transformer using its name
              // as specified via `Materialized` above
    );


Chromedriver no MAC - Eclipse/Intellij

Question: Estou com o seguinte problema:
Estou agora com MAC e não consigo mais executar minha automação.
Já fiz o download do Chrome para MAC, mas mesmo assim recebo o seguinte erro:
java.lang.IllegalStateException: The driver executable does not exist: /Users/estevaomarcos/Documents/Projetos/DBServer/\Users\chromedriver
No meu código está o seguinte:
System.setProperty("webdriver.chrome.driver", "\Users\chromedriver");
Alguém poderia me auxiliar?
Desde já agradeço.
Answer: Fala carinha, blz? olha de cara eu falaria que o caminho do driver esta incorreto. já verificou isso? tenta colocar ele em uma pasta dentro do projeto por exemplo:
<code>System.setProperty("webdriver.chrome.driver", ".//src//test//resources//chromedriver");
</code>
Comment: Bom dia Marcos Costa! Funcionou! Muito obrigado viu!

Chromedriver no MAC - Eclipse/Intellij


How to get lines in UITextView after word wrapping?

Question: I can't just split the string with "\n" and count the lines that way since word wrapping occurs dynamically.
What I need is a function that returns the lines (strings) from UITextView at that point in time.
Edit: this is not a duplicate of Counting the number of lines in a UITextView, lines wrapped by frame size , since I need contents of lines, not just line count.
Comment: Yes, it _is_ a duplicate. If you look at that answer and think about the technique used, you will see that you also get the line fragments that give you the range information for each wrapped line.
Answer: Inspired by this answer.
It appears, that you need to calculate word wrapping by yourself.
Here is a simple example:
<code>@interface UITextView (Lines)

- (NSArray*) textLines;

@end

@implementation UITextView (Lines)

- (NSArray*) textLines{
    NSMutableArray *result = @[].mutableCopy;

    NSArray *input = [self.text componentsSeparatedByString:@" "];

    NSDictionary *attributes = @{NSFontAttributeName:self.font};
    CGFloat maxWidth = self.frame.size.width - self.textContainer.lineFragmentPadding*2;
    NSMutableString *currentLine = @"".mutableCopy;

    for (NSString *component in input) {
        NSString *componentCheck = [NSString stringWithFormat:@" %@",component];
        CGSize currentSize = [currentLine sizeWithAttributes:attributes];
        CGSize componentSize = [componentCheck sizeWithAttributes:attributes];
        if (currentSize.width + componentSize.width < maxWidth) {
            [currentLine appendString:componentCheck];
        }else{
            [result addObject:currentLine];
            currentLine = component.mutableCopy;
        }
    }

    return result;
}

@end
</code>


Criar uma coluna com case_when com intervalos aparece erro

Question:
Tenho esta base de dados e pretendo criar uma nova coluna com esta
  estruturação (ver imagem). Por exemplo, se a linha na coluna Calendar_Week tiver um valor entre 36 e 45 é suposto a nova coluna Quartile ficar com o número "1".

datanew1$Quartile <- case_when(
  datanew1$Calendar_Week == seq(36,45) ~ "1",
  datanew1$Calendar_Week == seq(46,52)+seq(1,5) ~ "2",
  datanew1$Calendar_Week == seq(6,16) ~ "3",
  datanew1$Calendar_Week == seq(17,27) ~ "4"
)

O problema é que o código me está a dar um erro :

"1: In datanew1$Calendar_Week == seq(36, 45) :
  longer object length is not a multiple of shorter object length
2: In seq(46, 52) + seq(1, 5) :
  longer object length is not a multiple of shorter object length
3: In datanew1$Calendar_Week == seq(46, 52) + seq(1, 5) :
  longer object length is not a multiple of shorter object length
4: In datanew1$Calendar_Week == seq(6, 16) :
  longer object length is not a multiple of shorter object length
5: In datanew1$Calendar_Week == seq(17, 27) :
  longer object length is not a multiple of shorter object length

Alguma ideia em como resolver isto? Obrigada!
Answer:
Na verdade, consegui resolver o problema desta forma:

datanew1$Calendar_Week <- as.numeric(datanew1$Calendar_Week)
datanew1$Quartile <- case_when(
  datanew1$Calendar_Week %in% seq(36,45) ~ "1",
  datanew1$Calendar_Week %in% seq(46,52) ~ "2",
  datanew1$Calendar_Week %in% seq(6,16) ~ "3",
  datanew1$Calendar_Week %in% seq(17,27) ~ "4"
)


Datatables cell value updating only when refresh the page

Question: I have star rating in last column through java script,

When i update my rating through on click its updating in database but not in data table.
Because of this ,i am not able search or sort in last column with updated data.

I need to get updated data in data tables  when api calls for updating start
Any solution? Here is code: http://debug.datatables.net/uyidet

$(document).ready(function() {

 performanceRatingStar();

      $.ajaxSetup({
       async : false
      });

      $('#table_id2').DataTable({
       "order" : [ [ 2, "asc" ] ],
       "aoColumnDefs" : [ {
        "bSortable" : false,
        "aTargets" : [2]
       } ],
       "drawCallback": function ( settings ) {
                 var api = this.api();
                 var rows = api.rows( {page:'current'} ).nodes();
                 var last=null;

                 api.column(2, {page:'current'} ).data().each( function ( group, i ) {
                     if ( last !== group ) {
                         $(rows).eq( i ).before(
                             '
'+group+'
'
                         );

                         last = group;
                     }
                 } );
       }
      });
      // Setup - add a text input to each footer cell
      $('#table_id2 tfoot th')
        .each(
          function() {
           var title = $('#table_id2 thead th')
             .eq($(this).index()).text();
           if (title == 'First'
               || title == 'Last'
               || title=='GroupName'
               || title == 'Position'
               || title == 'Market'
            || title == 'Performance Rating'
             ){

            $(this)
              .html(
                '<input type="text" placeholder="Search '+title+'" />');
           }

          });

      // DataTable
      var table = $('#table_id2').DataTable();

      // Apply the search
      table.columns().eq(0).each(function(colIdx) {
           $('input',table.column(colIdx).footer()).on(
               'keyup',function() {
                table.column(colIdx).search(this.value).draw();
               });
          });

      // Order by the grouping
         $('#table_id2').on( 'click', 'tr.group', function () {
             var currentOrder = table.order()[0];
             if ( currentOrder[0] === 2 && currentOrder[1] === 'asc' ) {
                 table.order( [ 2, 'desc' ] ).draw();
             }
             else {
                 table.order( [ 2, 'asc' ] ).draw();
             }
         } );

});

function performanceRatingStar(){
   $(".rateYo").each(function(e)
      {
      var rateYoDivId = this.id;
      var rosterId_userId_fields = rateYoDivId.split(/_/);
      var rosterId = rosterId_userId_fields[0];
   var replaceRosterId = rosterId.replace("rateYo", "");
   var userId = rosterId_userId_fields[1];
   var rateFromHiddenField = $('#hiddenRating'+replaceRosterId+"_"+userId).val();
   var  pubId = $(this).attr('data-showPubPosiId');
   var   uId = $(this).attr('data-userId');

   $(this).rateYo({
         onSet: function(rating, rateYoInstance)
      {
          getRate = rateFromHiddenField;
      },

          onChange: function (rating, rateYoInstance) {

       if(rating == 0){
       $(this).next().text("Not Rated")
    }else if (rating == 1){
    $(this).next().text("Poor Performance");
    }else if (rating == 2){
    $(this).next().text("Needs Improvement");
    }else if (rating == 3){
    $(this).next().text("Solid Performance");
    }else if (rating == 4){
    $(this).next().text("Exceeded Expectations");
    }else if (rating == 5){
    $(this).next().text("Best In Class");
    }
    },
       starWidth: "20px",
       numStars: 5,
       fullStar: true
       });

    if(getRate == 0){
    $("#ratingStatus"+replaceRosterId).text("Not Rated")
    }else if (getRate == 1){
    $("#ratingStatus"+replaceRosterId).text("Poor Performance");
    }else if (getRate == 2){
    $("#ratingStatus"+replaceRosterId).text("Needs Improvement");
    }else if (getRate == 3){
    $("#ratingStatus"+replaceRosterId).text("Solid Performance");
    }else if (getRate == 4){
    $("#ratingStatus"+replaceRosterId).text("Exceeded Expectations");
    }else if (getRate == 5){
    $("#ratingStatus"+replaceRosterId).text("Best In Class");
    }

    $("#"+rateYoDivId).rateYo("rating", rateFromHiddenField);
     $("#"+rateYoDivId ).mouseover(function() {
    var maxValue = $("#"+rateYoDivId).rateYo("option", "rating");
    });
      });

  $(".rateYo").rateYo().on("rateyo.set", function (e, data)
    {
   var  pubId1 = $(this).attr('data-showPubPosiId');
   var   uId1 = $(this).attr('data-userId');
   $.ajax({
     url : "api/users/saveOrUpdateRating/"+data.rating+"/"+pubId1+"/"+uId1,
     contentType : "application/json",
     success : function(response)
     {
     count++;
     }
     });
  });
}

           First
           Last
           Position
           Market
           Performance Rating

           First
           Last
           Position
           Market
           Performance Rating

Supervisor :

<input type="hidden"      th:attr="id='hiddenRating'+${roster.showPubPosiId}+'_'+${roster.userId},data-showPubPosiId=${roster.showPubPosiId},data-userId=${roster.userId}" onclick="ratingData(this)" th:value="${roster.perforRating}" />

Comment: You might want to format your code snippets and reduce/correct intendation for better readability.
Comment: Btw, this seems to be related to JavaScript only. Where's the connection to Java?
Answer: It looks like you're trying to update the column with this statement:
$(this).next().text("Not Rated");

To update the data so that the DataTables plugin will know it has changed, you have to use its cell().data() function, then do a draw() for this to show on the browser.
I would also strongly recommend applying a variable to the dataTable to making working with it easier:
var myDataTable = $('#table_id2').DataTable({ /* ... skipped ... */ });

myDataTable.cell($(this).next()).data("Not Rated").draw();


Using an alternate way to write a Taylor series of $f(t+\tau)$ to derive numerical integration and differentiation formulas

Question: In chapter 9 section 3 of An Introduction to Mathematical Modeling, the author Edward A. Bender points out that: $$f(t+\tau)=\sum_{n=0}^\infty \frac{(\tau D)^n}{n!}$$ so that $$f(t+\tau)=e^{\tau D}f(t)$$ Where $D=\frac{d}{dt}$. This is easy enough to see once $f(t+\tau)$ has been expanded in its Taylor Series.
However, Bender then makes the comment: "I can't resist the side remark that (21) can also be used to derive numerical integration and differentiation formulas. For example, see L.P Ford (1955, ch.8)". I think he meant L.R Ford but I can't get a hand on that book so can anyone explain what type of numerical integration and differentiation formulas can be derived from this and how they can be derived? Bender seems fairly excited about the prospect so I'm really interested in how this works.
Answer: The Euler Method and the Three Term Taylor Method spring to mind immediately but to be honest I don't see how much more useful this formula really is once you know
$$f(x+h)\approx \sum_k\frac{f^{(k)}(x)}{k!}h^k,$$
which is exactly the same as your formula with $\tau=h$, $n=k$ and well $\displaystyle D=\frac{d}{dx}$.


How to calculate remainder value of a fraction

Question: Question:
Four brothers split a sum of money between them. The first brother received 50% of the total, the second received 25% of the total, the third received 20% of the total, and the fourth received the remaining $4. How many dollars did the four brothers split?
What I understand:
15 percent of the value is $4
The total value should be the remaining 85% plus the value of the 15% (4).
Confused about:
How do I calculate this?
Answer: You made a mistake. $4$ dollars corresponds to $5$ per cent of the total, so the total is $\frac{100}{5}\times 4$ dollars.


Chance $U_1$is bigger than all other random variables

Question: Please, help with the following would be highly appreciated. Again, I have an idea and a solution, but would like to see what other people think.
Let $X_1, X_2,\dots, X_N$ be iid random variables drawn from $F(x)=e^{-e^{-x}}$. Define $U_i=X_i+\delta_i$ for all $i\in\{1,2,\dots N\}$, where $\delta_i$'s are constants. What is the probability that $U_1=max_i\{U_i\}$?
Answer: $\int_{-\infty}^{\infty}{\prod_{i=2}^{N}{F(x+\delta_1-\delta_i)}\mathrm{d}F(x)}$
=$\int_{-\infty}^{\infty}{\prod_{i=2}^{N}{e^{-e^{-(x+\delta_1-\delta_i)}}}e^{-x}e^{-e^{-x}}\mathrm{d}x}$
=$\int_{-\infty}^{\infty}{e^{-e^{-x}\sum_{i=1}^{N}{e^{-(\delta_1-\delta_i)}}}e^{-x}\mathrm{d}x}$
$=\frac{1}{\sum_{i=1}^{N}{e^{-(\delta_1-\delta_i)}}}\int_{-\infty}^{\infty}{\mathrm{d}e^{-e^{-x}\sum_{i=1}^{N}{e^{-(\delta_1-\delta_i)}}}}$
$=\frac{e^{\delta_1}}{\sum_{i=1}^{N}{e^{\delta_i}}}$


Drupal 7 - Error on Checkout, e-Commerce

Question: I installed a e-commerece module for my website. And enalbed all the supporting files under 'modules'. I'm able to add products and view them and can also add to the shopping cart. however when i checkout, it sends me some here..like this
<code>PDOException: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column
'exid' at row 1: INSERT INTO {ec_customer} (type, exid, uid, name) VALUES
(:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2,
:db_insert_placeholder_3); Array ( [:db_insert_placeholder_0] => anonymous
[:db_insert_placeholder_1] => 4mePOKXg5_hhSCYgVSumyWMhsej6EZ5LRLKafRfPBTA
[:db_insert_placeholder_2] => 0 [:db_insert_placeholder_3] => ) in drupal_write_record()
(line 6975 of C:\xampp\htdocs\cart\includes\common.inc).
</code>
Does anyone know why is it throwing an error.
Comment: Did you install Drupal commerce by yourself or are you using the commerce-kickstart distribution?
Comment: I installed the e-commerece module all by myself! Didn't used the commerece kickstart.
Comment: This is not Commerce at all, it is the e-Commerce module.
Answer: If you're new to Drupal commerce, try installing Commerce Kickstart.  There are detailed installation instructions available.
Drupal Commerce is fairly complex and there are lots of different types of errors that can result.  Commerce Kickstart is a package (known in Drupal as an installation profile) that sets up most of the commerce stuff for you.  If you're not experienced in working with Drupal, this is an easy way to get started without having to troubleshoot error messages.
If you do try Commerce Kickstart, do so from a new site.  If you try to install over Drupal Commerce it could get messy.
Comment: Thanks Patrick! Will try the commerece kickstart. Are Drupal e-Commerece module and commerece kickstart different? Or is commerece kickstart is just a way to install the e-Commerece module.

Thanks again!
Comment: First, some terms clarification: the name of the e-commerce module for Drupal 7 is "Drupal Commerce."  In past versions of Drupal, there were modules like "Drupal E-commerce," so to avoid confusion please call the module "Drupal Commerce."
Comment: To answer your question, yes, Drupal Commerce and Commerce Kickstart are the same project.  Commerce Kickstart simply includes basic configuration and helps you set things up your first time.  (You can always do advanced customizations later if necessary.)
Comment: Drupal Commerce or Ubercart? what's your pick. Considering that Ubercart has more payment options than Drupal commerce, i'm little confused here.


Does a melody ending on the 6th degree mean that the section of a song is in aeolian mode?

Question: I am learning a song that is in C major and most of the phrases end on one of the stable tones (C,E,G) however, there is a part of the song where the melody repeatedly ends on the A note (6th degree) which is supposed to be a note that resolves somewhere else (usually down to the 5th). Does this mean that for that section the chord progression is in A aeolian since if the melody feels at rest here then it won't be in C major anymore right?
Here is the whole chord progression of the verse
C,G,Am,E,F,G,C,Am,G
The chords each get half a bar except for the last two chords Am, G which get a beat each to return back to the C chord
The meldoy ends on A over the Am and over the F chord but then ends on G on the last C chord. So perhaps I am looking at this wrong and should only consider that the end of the melody is in fact the G over the C chord instead of those brief stops on Am and F on the A note.
FYI, the song is by oasis called "Dont look back in anger"
Comment: You should update your question with the chords involved in the phrases.
Comment: This question would be much easier to answer if we were looking at a notated melody with chord symbols.
Comment: Take any song with a melody that has 6th degree notes. When any 6th degree note comes, stop the playback. Did your pressing STOP change the mode of the song? What if it's not a modal song, does it become modal now?
Answer: Don't Look Back in Anger is made up of three phrases:
Verse / Chorus
The progression here is a variation on the canon, there is no modulation at all, it really wants to resolve to a C (like the end of the song).
The pre-chorus (So I start a revolution form my bed...)
F / Fm / C
This is a cliché line that you find in many songs. Fm has dominant function, and creates tension that resolves over the C.
The "Stand up beside the fireplace" part
G / G#dim / Am / G / F
Yes, this little section is more properly in Am, because of the G#dim (or E7) passing chord. It is a brief dramatic detour, before jumping back into the chorus.
The same idea is used in Leonard Cohen's Hallelujah - (G) The baffled king (E7) composing Hallelujah (Am).
Comment: You can't really separate a melody and harmony like that @armani (unless if the composer made them really separate)  If compared to a drawing: harmony would be the dark and light patches, broad and thick strokes, then the melody would be the small intricate detail in thin strokes residing in those patches. When talking about a song's progression you're almost always talking about the whole. (But exceptions can be found of course)
Comment: Thanks. I was actually asking about the melody, not harmony
Comment: I can take a melody and change the chords to get multiple options for harmony... the melody is still the same so in that sense it is very independent IMO. Yes the meldoy will sound different depending on what chords are used but the notes still function as they do in the scale without the chords. If you look at bands like Coldplay (old albums) they purposefuly make sure the melody and chords don't work together which gives them their unique sound.
Comment: Yeah sure, you can change the harmony as much as you want, but then it becomes a different progression. That's the point. You're asking about a specific melody in a specific rendering of a song, i.e. with a specific harmony.
Comment: I know it becomes a different progression which is why I was even reluctant to even put the progression int he question. I asked about the melody in my OP then added the chords later.
Comment: You probably ought to identify "the canon."
Answer: It is very common for melodies to have one or more contrasting phrases that cadence on a note other than the tonic.  This contrast is often realized in the harmony by moving temporarily to another tonal area, for example the dominant.  (In fact, the term "dominant" arose from melodic theory before current harmonic practice ever developed.)
In the big picture, that doesn't mean that the key of the piece has changed or should be called into question, but it is common to say that a section or phrase is "in" whatever related key when analyzing a piece at a smaller-scale level of detail.
Contrasting harmonic areas are most commonly the dominant and the relative minor, and A minor is the relative minor of C major, so that's clearly what's going on here.

So perhaps I am looking at this wrong and should only consider that the end of the melody is in fact the G over the C chord instead of those brief stops on Am and F on the A note.

Consider for what purpose?  To determine the overall key of the song?  If so, yes.  But to identify a phrase that moves to A minor in some way or another to set up a return to the tonic, you obviously don't want to ignore the cadence on A.
I don't see any reason to prefer "A Aeolian" here instead of "A minor"; on the contrary, A minor makes more sense as it is a tonal shift.  (Consider: if it moved to the dominant, we would say "G major" rather than "G Mixolydian.")
Answer: A Aeolian and C major are alike in many ways, the main one being a complete set of shared notes. The only difference in reality is where 'home' is perceived to be. Usually, in key C major, it's C, in A Aeolian, it's A.
It could be that it's slipped into A minor for a short while - not so much a key change as a modulation. It will depend on what the harmony is at that point. If there are Cs and Es along, then it would seem to be , at the end of a phrase, an interrupted cadence.
It may also be accompanied by Cs and Fs. In which case, there's an F chord. Giving, I believe, another interrupted cadence.' sort of an unfinished plagal cadence. Another term is deceptive cadence - a lovely term which encapsulates what has happened. 'You didn't really expect that, did you?'
So, it hasn't officially left key C major, just gone for a small detour - obvious, as it returns shortly after anyway.
Comment: The progression is C,G,Am,E,F,G,C,G,C (Don't look back in anger by Oasis). It is rather long but the melody repeats ending firs on the Am chord (on the A note) and again on the F chord ( again on the A note). At the end of the whole progression on the C chord it does end on the G but the first two phrases end rather comfortable on A which is supposed to be an unstable tone in C major.
Comment: If C, E and G are stable tones in C major and A,C,E are stable tones in A minor then doesn't it make more sense to add A to the list of stable tones in C major since it is seems a common tone to end a melody in C major?
Comment: Is it official that A is an unstable note in key C? I've always considered it pretty stable. So much so that some numbers actually end with a C6 chord, containing that A.
Comment: @Tim: I considered the 1st phrase to be "Slip inside the eye of your mind" with the word "mind" ending on A. Can I understand why you did not consider the same? Yes the whole section ends on G but I consider where a melody stops firmly to be the end note of the phrase. Am I incorrect?
Comment: @Tim, yes my apologies, Am and G at the end each for a beat before returning to C
Comment: As I listen, that Am isn't an end chord - merely a transition before the G, which is there to return to C for the next verse. There is no 'rule' saying this or that *must* be done, anyway.
Comment: Listened to it, and 1st verse ends on G melody note, then on C, before next section, which ends on G note. Now the question is asked with odd premises. And verse chords are C G Am E F G Am G.
Comment: I agee, all endings are in C or in a "legato" that goes from C (quaver), to A and finally G (semiquavers), so I believe that the question is not valid. (To be precise, the final bar also has a C chord, before A- and G). And that song is also another Pachelbel's spoof ;-)
Comment: Why "Aeolian" rather than "minor"?
Comment: @phoog - trying to keep within the vocabulary used by OP. Either would do, but Aeolian uses only the notes from the mother key. I guess relative minor is usable, too.


Jssor Thumbnail Navigation Doesn't Slide/Drag

Question: We are implementing a Jssor slider on our website. Everything was working good, but while we were adjusting for design and images the thumbnail navigation stopped moving.
The page is here: http://visionsunltd.com/CMS/advertising-gallery

Any ideas on why the thumbnail navigation doesn't move and we can't drag it to get to the other thumbnails?
Answer: I checked your slider, there are totally 9 slides.
In the $ThumbnailNavigatorOptions, you set $DisplayPieces: 10. That's to say, the total 9 thumbnails are all displayed already, there is no chance to drag to display more.
Please set $DisplayPieces: 8, thus you can get 8 displayed and another one hidden, then you can drag to loop thumbnails.


ionic gravar audio não tras para view

Question: Fiz um app, para gravar audio, ele  abre a gravador nativo, mas não retorna o audio para view.
<code>.controller('MyCtrl', function ($scope, $cordovaCapture) {
$scope.captureAudio = function () {
    var captureSuccess = function (mediaFiles) {
        var i, path, len;
        for (i = 0, len = mediaFiles.length; i < len; i += 1) {
            path = mediaFiles[i].fullPath;
            // do something interesting with the file
        }
    };

    // capture error callback
    var captureError = function (error) {
        navigator.notification.alert('Error code: ' + error.code, null, 'Capture Error');
    };

    // start audio capture
    navigator.device.capture.captureAudio(captureSuccess, captureError, {
        limit: 2
    });
}
</code>
});
Answer: Rodrigo, Boa Noite,
Recomendo utilizar o seguinte plugin pra gravação de áudio: cordova-plugin-media-with-compression(https://github.com/remoorejr/cordova-plugin-media-with-compression). Eu já utilizei e não tive problema nenhum.
Comment: Obrigado @JeordaneBatista.

ionic gravar audio não tras para view


Creating Code Snippets

Question: I'm attempting to use Drupal as creating a pretty versatile code snippet site for storing all of my  and others goodness. Yeah I know, sounds a bit familiar, but stick with me. What I'm having a hard time grasping is the best way to handle this in terms of both functionality and security.
At its simplest form, I'm thinking I'd create a Code content type with some taxonomy attributes, language type, etc. and then the actual 'body' of the node will be the code snippet itself. What I'm struggling on is, do I allow unfiltered text through here so anything and I mean anything can be put in and then on the node display just wrap the body in a code or pre block? I think then I could just  wrap that with Google Prettify and bam, code snippets.
Am I way off with this? Is there something I'm missing with how that could conceptually work? In terms of security, it seems a single code or pre block from any code actually rendering sounds a bit scary - is that how normal code snippet sites work?
I've been fumbling around with ideas in my head, so any help to either lead me in the right direction or slap some sense into me would be greatly appreciated. Thanks!
Comment: I might not be following, but if you just use a plain text input filter/formatter, then all text entered will be properly sanitized, and then you just have to theme it the way you want it.
Comment: Ha, I was just thinking that. Makes logical sense and guess I needed a "don't over think it stupid" kind of nudge ;) Thanks!
Comment: And of course you can do what drupal.org does and just provide [Codefilter module](http://drupal.org/project/codefilter) and add it to the filtered_html format and go with that.
Comment: Ah - very cool. Looks like this is only for PHP though - but maybe could be used for commenting on code snippets. Thanks!
Answer: I would definitely encourage you to look at the http://drupal.org/project/geshifilter input filter
From the page:
At the time of this writing the GeSHi library supports highlighting for more than 200 programming languages and configuration syntaxes, including: ActionScript, Ada, Apache Log, AppleScript, ASM, ASP, Bash, C, ColdFusion, C++, QT, C#, CSS, D, Delphi, Diff, DOS, dot, Eiffel, Fortran, Groovy, Haskell, HTML, INI, Inno, Java, Javascript, LaTeX, Lisp, Lua, Motorola 68000 Assembler, Matlab, mIRC Scripting, MySQL, Objective C, OCaml, OpenOffice.org Basic, Oracle 8 SQL, Pascal, Perl, PHP, PL/SQL, Python, QBasic/QuickBASIC, Rails, Microsoft Registry, robots.txt, Ruby, SAS, Scheme, Smalltalk, Smarty, SQL, TCL, Text, Visual Basic, vb.net, VHDL, XML, X++.
There is a stable D7 and D6 version.

Creating Code Snippets


Specify a blocking factor in H2O

Question: In the R version of H2O, is it possible to specify a blocking factor when splitting data in training/validation/test sets and/or when doing cross-validation?
I'm working on a clinical dataset with multiple observations from the same patient that should be kept together during these operations.
If this is not possible to do within the H2O framework then suggestions on how to achieve this in R and integrate with H2O functions would be great.
Thanks!
Comment: We have a ticket open for this already: https://0xdata.atlassian.net/browse/PUBDEV-4442  I just added this to the roadmap for the next major release -- hopefully it will get implemented soon, but I can't make any guarantees. (I've personally wanted this for a while).
Comment: This questions seems to address how to do this in base R: https://stackoverflow.com/questions/22518982/randomly-split-data-by-criterion-into-training-and-testing-data-set-using-r
Comment: Thanks @ErinLeDell. Do you have a working example of how to encode the fold_column given a column of IDs? I.e., an example of this: `The user-specified fold_column method requires the user to code the stratification-by-ID themselves, which is a pain.`
Answer: When using H2O-3 with cross validation, you can tell the training algorithm which fold number an observation belongs to with the fold_column parameter.  See:

http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/fold_column.html

The code example below (copied from the link above) shows folds being assigned randomly.  But you could alternately write a piece of code to assign them specifically yourself.
library(h2o)
h2o.init()

# import the cars dataset:
# this dataset is used to classify whether or not a car is economical based on
# the car's displacement, power, weight, and acceleration, and the year it was made
cars <- h2o.importFile("https://s3.amazonaws.com/h2o-public-test-data/smalldata/junit/cars_20mpg.csv")

# convert response column to a factor
cars["economy_20mpg"] <- as.factor(cars["economy_20mpg"])

# set the predictor names and the response column name
predictors <- c("displacement","power","weight","acceleration","year")
response <- "economy_20mpg"

# create a fold column with 5 folds
# randomly assign fold numbers 0 through 4 for each row in the column
fold_numbers <- h2o.kfold_column(cars, nfolds=5)

# rename the column "fold_numbers"
names(fold_numbers) <- "fold_numbers"

# print the fold_assignment column
print(fold_numbers)

# append the fold_numbers column to the cars dataset
cars <- h2o.cbind(cars,fold_numbers)

# try using the fold_column parameter:
cars_gbm <- h2o.gbm(x = predictors, y = response, training_frame = cars,
                    fold_column="fold_numbers", seed = 1234)

# print the auc for your model
print(h2o.auc(cars_gbm, xval = TRUE))

Comment: That's very helpful, thanks. I wish there was a more general way to do it, regardless of whether cross-validation was chosen (I.e.: something that could be specified in `h2o.splitFrame()`, which would cover the simplest case with one training and one validation set).

Specify a blocking factor in H2O


ERROR 500 on certain machines, client side or still server side?

Question: Hope someone can explain if we have to search for a solution on the clients or server side script. This certain link can be opened by many workstations except at one company. The site they have to visit is writen in PHP and requires an SSL/https connection.
The company has 40-50 Windows 7 x64 Enterprise stations with IE11, Chrome & Firefox which can't go to the url, it results in a error 500 message. Tried with 2 internet connections, one with proxy and one without (directly). Within a domain and one within a workgroup. Tried to reset the IE settings to default, cleared the group policy's on the workgroup workstation, without success.
They have 1 workstation on Windows 10 Pro. This machine is able to visit the url without problems on the same network connection mentioned above, with IE11, Edge & Chrome.
And have another workstation on Windows 10 Pro, this one is only able to visit the url in IE11. Edge browser crashes only when visiting the url and chrome results in error 500.
Is there anything browser related what this problem can cause or is it server side?
Comment: Maybe related: http://serverfault.com/questions/819121/500-error-only-in-chrome-works-in-firefox
Answer: The HTTP error code 500 ("internal server error") is reserved for serverside errors and you'll need to check the servers (error) logs for the cause, possibly after increasing the verbosity/debug level to get sufficient detail. Debugging from the client side alone  will be nigh impossible.
Answer: The 500 error message will be coming from the server. You should look in the logs on the server to get more information about what the problem is.
Answer: 500 means it a server-side issue, Please check the error log on the server.


Problems exporting HTML table to Excel

Question: I have a table in which one td element has an anchor tag inside it for a number.
213123

I am exporting my table data elements to Excel. When the table is exported it is also printing the anchor tag along with the number. Is there a way that I could avoid the anchor tag in Excel sheet and just display the number in the column?
I am using the following code:
function export_to_excel( ptablebody) {

    str="";
    var ExcelApp = new ActiveXObject("Excel.Application");
    var ExcelSheet = new ActiveXObject("Excel.Sheet");
    ExcelSheet.Application.Visible = true;

    var myTableHead = document.getElementById(ptablebody);
    var rowCount = myTableHead.rows.length;

    var colCount = myTableHead.getElementsByTagName("tr")[0].getElementsByTagName("td").length;

    for(var i=0; i<rowCount; i++) {
        for(var j=0; j<colCount; j++) {
            str= myTableHead.getElementsByTagName("tr")[i].getElementsByTagName("td")[j].innerHTML;
            ExcelSheet.ActiveSheet.Cells(i+1,j+1).Value = str;
        }
    }
}

Comment: How are you _exporting_?
Comment: @nivas I modified my post with the code i am using.
Answer: I think you want innerText (or on firefox textContent) instead of innerHtml

Problems exporting HTML table to Excel


PHP function to get data from web service

Question: How can I get data from this web service using PHP?
I need a simple PHP function to list the countries.
Web service Data
{
   "valid":true,
   "id":"0",
   "data":{
      "@type":"genericObjectArray",
      "item":[
         {
            "id":"DE",
            "description":"Deutschland"
         },
         {
            "id":"ES",
            "description":"España"
         },
         {
            "id":"FR",
            "description":"France"
         },
         {
            "id":"PT",
            "description":"Portugal"
         },
         {
            "id":"UK",
            "description":"United Kingdom"
         },
         {
            "id":"US",
            "description":"United States"
         }
      ]
   }
}

Answer: Assuming allow_url_fopen is on in php.ini (if not, use cURL library).
$json = file_get_contents('http://onleague.stormrise.pt:8031/OnLeagueRest/resources/onleague/Utils/Countries ');

$data = json_decode($json, TRUE);

$countries = array();

foreach($data['data']['item'] as $item) {
    $countries[] = $item['description'];
}

CodePad.
Of course, handle if $json is FALSE (error with request).
Alternatively, if using >= PHP 5.3.
$countries = array_map(function($item) {
    return $item['description'];
}, $data['data']['item']);

CodePad.
Comment: @Carlos You must be doing something wrong, because it definitely works.
Comment: Something is not right?! I'm getting wrong values on my array
print_r($countries);

Array ( [0] => g [1] => )


add is null condtion in db_update

Question: I want to use db_update with condition that a specify field is null ,how can I do it?
something like
<code>->condition('used_date',  'isnull','=')
</code>
is this correct ,if not,what is the correct condition ?
Answer: As per documentation, there is a method just for that: <code>->isNull()</code>
<code>$query->isNull('myfield');
// Results in (myfield IS NULL)
$query->isNotNull('myfield');
// Results in (myfield IS NOT NULL)
</code>

add is null condtion in db_update


Names features importance plot after preprocessing

Question: Before building a model I make scaling like this
X = StandardScaler(with_mean = 0, with_std = 1).fit_transform(X)

and after build a features importance plot
xgb.plot_importance(bst, color='red')
plt.title('importance', fontsize = 20)
plt.yticks(fontsize = 10)
plt.ylabel('features', fontsize = 20)

The problem is that instead of feature's names we get f0, f1, f2, f3 etc.....
How to return feature's names?
thanks
Answer: first we get list of feature names before preprocessing
dtrain = xgb.DMatrix( X, label=y)
dtrain.feature_names

Then
bst.get_fscore()
mapper = {'f{0}'.format(i): v for i, v in enumerate(dtrain.feature_names)}
mapped = {mapper[k]: v for k, v in bst.get_fscore().items()}
mapped
xgb.plot_importance(mapped, color='red')

that's all
Comment: I needed to use `bst.booster().get_score().items()` instead of
 `bst.get_fscore().items()`, when the `bst` is instance of `XGBClassifier`.
Comment: I needed to use bst.get_booster().get_score().items()
Answer: For xgboost 0.82, the answer is quite simple, just overwrite the feature names attribute with the list of feature name strings.
trained_xgbmodel.feature_names = feature_name_list
xgboost.plot_importance(trained_xgbmodel)

Comment: FYI, the previous answer didnt work for me with the latest version of xgboost.
Answer: You can retrieve the importance of Xgboost model (trained with scikit-learn like API) with:
xgb.feature_importances_

To check what type of importance it is: xgb.importance_type. The importance type can be set in the Xgboost constructor. You can read about ways to compute feature importance in Xgboost in this post.


Convert SAS Logistic Regression to R

Question: Here's my issue of the day:
At the moment I'm teaching myself Econometrics and making use of logistic regression. I have some SAS code and I want to be sure I understand it well first before trying to convert it to R. (I don't have and I don't know SAS). In this code, I want to model the probability for one person to be an 'unemployed employee'. By this I mean "age" between 15 and 64, and "tact" = "jobless". I want to try to predict this outcome with the following variables: sex, age and idnat (nationality number). (Other things being equal).
SAS code :
/* Unemployment rate : number of unemployment amongst the workforce */
proc logistic data=census;
class sex(ref="Man") age idnat(ref="spanish") / param=glm;
class tact (ref=first);
model tact = sex age idnat / link=logit;
where 15<=age<=64 and tact in ("Employee" "Jobless");
weight weight;
format age ageC. tact $activity. idnat $nat_dom. inat $nationalty. sex $M_W.;

lsmeans sex / obsmargins ilink;
lsmeans idnat / obsmargins ilink;
lsmeans age / obsmargins ilink;
run;

This is a sample of what the database should looks like :
      idnat     sex     age  tact
 [1,] "english" "Woman" "42" "Employee"
 [2,] "french"  "Woman" "31" "Jobless"
 [3,] "spanish" "Woman" "19" "Employee"
 [4,] "english" "Man"   "45" "Jobless"
 [5,] "english" "Man"   "34" "Employee"
 [6,] "spanish" "Woman" "25" "Employee"
 [7,] "spanish" "Man"   "39" "Jobless"
 [8,] "spanish" "Woman" "44" "Jobless"
 [9,] "spanish" "Man"   "29" "Employee"
[10,] "spanish" "Man"   "62" "Retired"
[11,] "spanish" "Man"   "64" "Retired"
[12,] "english" "Woman" "53" "Jobless"
[13,] "english" "Man"   "43" "Jobless"
[14,] "french"  "Man"   "61" "Retired"
[15,] "french"  "Man"   "50" "Employee"

This is the kind of result I wish to get :
Variable    Modality    Value   ChiSq   Indicator
Sex         Women       56.6%   0.00001 -8.9%
            Men         65.5%
Nationality
            1:Spanish   62.6%
            2:French    51.2%   0.00001 -11.4%
            3:English   48.0%   0.00001 -14.6%
Age
            <25yo       33.1%   0.00001 -44.9%
        Ref:26<x<54yo   78.0%
            55yo=<      48.7%   0.00001 -29.3%

Indicator is P(category)-P(ref)
(I interpret the above as follows: other things being equal, women have -8.9% chance of being employed vs men and those aged less than 25 have a -44.9% chance of being employed than those aged between 26 and 54).
So if I understand well, the best approach would be to use a binary logistic regression (link=logit). This uses references "male vs female"(sex), "employee vs jobless"(from 'tact' variable)... I presume 'tact' is automatically converted to a binary (0-1) variable by SAS.
Here is my 1st attempt in R. I haven't check it yet (need my own PC) :
### before using glm function
### change all predictors to factors and relevel reference
recens$sex <- relevel(factor(recens$sex), ref = "Man")
recens$idnat <- relevel(factor(recens$idnat), ref = "spanish")
recens$tact <- relevel(factor(recens$tact), ref = "Employee")
recens$ageC <- relevel(factor(recens$ageC), ref = "Ref : De 26 a 54 ans")

### Calculations of the probabilities with function glm,
### formatted variables, and conditions with subset restriction to "from 15yo to 64"
### and "employee" and "jobless" only.
glm1 <- glm(activite ~ sex + ageC + idnat, data=recens, weights = weight,
            subset= recens$age[(15<= recens$age | recens$age <= 64)]
        & recens$tact %in% c("Employee","Jobless"),
            family=quasibinomial("logit"))

My questions :
For the moment, it seems there are many functions to carry out a logistic regression in R like glm which seems to fit.
However after visiting many forums it seems a lot of people recommend not trying to exactly reproduce SAS PROC LOGISTIC, particularly the function LSMEANS. Dr Franck Harrel, (author of package:rms) for one.
That said, I guess my big issue is LSMEANS and its options Obsmargins and ILINK. Even after reading over its description repeatedly I can hardly understand how it works.
So far, what I understand of Obsmargin is that it respects the structure of the total population of the database (i.e. calculations are done with proportions of the total population). ILINK appears to be used to obtain the predicted probability value (jobless rate, employment rate) for each of the predictors (e.g. female then male) rather than the value found by the (exponential) model?
In short, how could this be done through R, with lrm from rms or lsmeans?
I'm really lost in all of this. If someone could explain it to me better and tell me if I'm on the right track it would make my day.
Comment: Questions about programming are better suited for stack overflow.
Comment: This is an example where users are too quick to close or suggest a Q is OT due to programming. Let me be clear; there *is no programming* involved here. The OP is trying to replicate the behaviour of one stats package in another. That will require understanding of both the software and its implementation, and the statistical aspects of the outputs/analysis the OP wants to replicate. Using R != Programming.
Comment: @Binh please [merge](http://stats.stackexchange.com/help/merging-accounts) your two accounts. If your question is not about programming, please rephrase it to more clearly reflect the statistical/modelling aspects of your question.
Comment: Questions about programming are better suited for stack overflow.
Comment: Thank you for your answer but to me it's not about programming but mostly about understanding exactly if I'm right on my problem. That said, if SAS lsmeans really will be of use so I could create the same process later in R. On Stack Overflow I've already been told to post here instead. Perhaps I need to be more precise on what I want?
Comment: @Gavin I agree with your conclusion (the Q is on topic) but I might disagree with the some of the reasoning. The question overtly requests a way to refactor SAS code into R code ("convert it into R"). As such it seems likely to have fairly little statistical interest. However--as you argue well--that port, to be done well and correctly, demands knowledge not only of the source and target platforms, but also of nuances of the statistical procedure itself. That I believe is the strongest reason to entertain such multi-platform questions on this site and not to migrate them.
Answer: The least squares means are, in a sense, weighted averages.  You can obtain similar results by using predict.glm with a type="response" option and then summarising over the variable of interest.  E.g. predict the umeployment rates for all the data, and then summarise for all Males / Females and calculate the difference.
SAS's ilink option is doing the same thing by inverting the link (logit) function and turning the estimates from log odds back into probabilities - converting the estimates to the scale of the response variable (i.e. probability).
Be careful reporting differences in probability derived this way.  As an extreme example, if there was a sub-population with a Male unemployment rate of zero, would you expect the unemployment rate for Females in that same sub-group to to be zero, or -8.9 percent?

Convert SAS Logistic Regression to R


Orthogonal matrix is unitary?

Question: If a matrix lies in $\text{GL}(n,\mathbb{R})$ why does it also ly in $\text{GL}(n,\mathbb{C})$?
If a real matrix $A$ hat determinant $\neq 0$ then its determinant as a complex matrix is also non zero since the determinant depends not on the field.
That's it, isn't it?
Comment: Your post is different than your title
Comment: @user455297 I agree with you on this point.
Comment: Yes but this is the main point. If I have $A^t=A^{-1}$ then of course I do have $A^*=A^{-1}$ for real $A$.
Answer: Yes that's it.
You can also write  $  AB = BA = I_n  $ , and as real matrices are also complex ones, if $ A $ is invertible as a real matrix, it is also invertible as a complex one.
Comment: And in the same way, if $A$ is an $n\times n$ matrix with real entries and satisfies $AA^T=A^TA=I_n$, then seen as a complex matrix that is $AA^*=A^*A=I_n$ where asterisk denotes the conjugate transpose (a.k.a. Hermitian adjoint) of a matrix. This shows that if a real matrix is orthogonal, then when seen as a complex matrix it is unitary (the question title).
Answer: It is common to say that $\Bbb R \subset \Bbb C$.  In particular, we identify the real number $x \in \Bbb R$ with the complex number $x + 0i \in \Bbb C$.

Orthogonal matrix is unitary?