Exponential Family: Observed vs. Expected Sufficient Statistics

Question: My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a Dirichlet distribution based on observations of random vectors:

As always with the exponential family, when the gradient is zero, the expected sufficient statistics are equal to the observed sufficient statistics.

I haven't seen maximum likelihood estimation in the exponential family presented in this way, nor have I found any suitable explanations in my search.  Can someone offer insight into the relationship between observed and expected sufficient statistics, and perhaps help to understand maximum likelihood estimation as minimizing their difference?
Answer: This is a usual assertion about the exponential family, but in my opinion, most of the times it is stated in a way that may confuse the less experienced reader. Because, taken at face value, it could be interpreted as saying "if our random variable follows a distribution in the exponential family, then if we take a sample and insert it into the sufficient statistic, we will obtain the true expected value of the statistic". If only it were so...  More over it does not take into account the size of the sample, which may cause further confusion.
The exponential density function  is
$$f_X(x) = h(x)e^{\eta(\theta) T(x)}e^{-A(\theta)} \tag{1}$$
where $T(x)$ is the sufficient statistic.
Since this is a density, it has to integrate to unity, so ($S_x$ is the support of $X$)
$$\int_{S_x} h(x)e^{\eta(\theta) T(x)}e^{-A(\theta)}dx =1 \tag{2}$$
Eq. $(2)$ holds for all $\theta$ so we can differentiate both sides with respect to it:
$$\frac {\partial}{\partial \theta} \int_{S_x} h(x)e^{\eta(\theta) T(x)}e^{-A(\theta)}dx =\frac {\partial (1)}{\partial \theta} =0 \tag{3}$$
Interchanging the order of differentiation and integration, we obtain
$$\int_{S_x} \frac {\partial}{\partial \theta} \left(h(x)e^{\eta(\theta) T(x)}e^{-A(\theta)}\right)dx  =0 \tag{4}$$
Carrying out the differentiation we have
$$\frac {\partial}{\partial \theta} \left(h(x)e^{\eta(\theta) T(x)}e^{-A(\theta)}\right) = f_X(x)\big[T(x)\eta'(\theta) - A'(\theta)\big] \tag{5}$$
Inserting $(5)$ into $(4)$ we get
$$\int_{S_x} f_X(x)\big[T(x)\eta'(\theta) - A'(\theta)\big]dx  =0 $$
$$\Rightarrow \eta'(\theta)E[T(X)] - A'(\theta) = 0  \Rightarrow E[T(X)] =  \frac {A'(\theta)}{\eta'(\theta)} \tag{6}$$
Now we ask: the left-hand-side of $(6)$ is a real number. So, the right-hand-side must also be a real number, and not a function. Therefore it must be evaluated at a specific $\theta$, and it should be the "true" $\theta$, otherwise in the left-hand-side we would not have the true expected value of $T(X)$. To emphasize this we denote the true value by $\theta_0$, and we re-write $(6)$ as
$$E_{\theta_0}[T(X)] =  \frac {A'(\theta)}{\eta'(\theta)}\Big |_{\theta =\theta_0}  \tag{6a}$$
We turn now to maximum likelihood estimation. The log-likelihood for a sample of size $n$ is
$$L(\theta \mid \mathbf x) = \sum_{i=1}^n\ln h(x_i) +\eta(\theta)\sum_{i=1}^nT(x_i) -nA(\theta)$$
Setting its derivative with respect to $\theta$ equal to $0$ we obtain the MLE
$$\hat \theta(x) : \frac 1n\sum_{i=1}^nT(x_i) = \frac {A'(\theta)}{\eta'(\theta)}\Big |_{\theta =\hat \theta(x)} \tag {7}$$
Compare $(7)$ with $(6a)$. The right-hand sides are not equal, since we cannot argue that the MLE estimator hit upon the true value. So neither are the left hand-sides. But remember that eq. $2$ holds for all $
\theta$ and so for $\hat \theta$ also. So the steps in eq. $3,4,5,6$ can be taken with respect to $\hat \theta$ and so we can write eq. $6a$ for $\hat \theta$:
$$E_{\hat\theta(x)}[T(X)] =  \frac {A'(\theta)}{\eta'(\theta)}\Big |_{\theta =\hat\theta(x)}  \tag{6b}$$
which, combined with $(7)$, leads us to the valid relation
$$ E_{\hat\theta(x)}[T(X)] = \frac 1n\sum_{i=1}^nT(x_i)$$
which is what the assertion under examination really says: the expected value of the sufficient statistic under the MLE for the unknown parameters (in other words, the value of the first raw moment of the distribution that we will obtain if we use $\hat \theta(x)$ in place of $\theta$), equals (and it is not just approximated by) the average of the sufficient statistic as calculated from the sample $\mathbf x$.
Moreover, only if the sample size is $n=1$ then we could accurately say, "the expected value of the sufficient statistic under the MLE equals the sufficient statistic".
Comment: @Theoden In between eq. $(2)$ and $(3)$ I write "eq. $(2)$ holds _for all_ $\theta$" - and therefore for $\hat \theta$ also. So all the steps in eq. $3,4,5,6$ can be taken with respect to $\hat \theta$. I repeated this remark in the text for clarity.
Comment: Could you explain why we can interchange the order of differentiation and integration in eq. (3) please?
Comment: Could you further elaborate why the transition from 6a to 6b is valid please?
Comment: @Markus777  See here, https://alecospapadopoulos.files.wordpress.com/2014/05/the-information-matrix-equality1.pdf, for a set of sufficient conditions.
Comment: @AlecosPapadopoulos your proof below seems to suggest that what you say at the start - "if our random variable follows a distribution in the exponential family, then if we take a sample and insert it into the sufficient statistic, we will obtain the true expected value of the statistic" is true. I mean I can just always do that for (2), replacing it with observed sufficient stat and get the result. What am I missing here? I don't quite get it.
Comment: @user136266 The _true_ expected value of the statistic is $6a$, and in order to be calculated, one needs to know the, by design unknown, parameter $\theta$. So what we can actually calculate is $6b$ which is the expected value of the statistic _under the assumption that our point estimate has hit the true value_.


How to handle different output for a same block (Caching Problem)?

Question: I have a theme which has two different appearance for main menu block, so I wrote a theme suggestion like this:
function MYTHEME_theme_suggestions_menu_alter(array &$suggestions, array $variables) {
  if ($variables['menu_name'] == 'main' and Drupal::service('path.matcher')
  ->isFrontPage()) {
    array_splice($suggestions, 1, 0, 'menu__main__front');
  }
}

so that I can theme main menu for front page in its own manner. The problem raises is about how Drupal caching system behaves about blocks so that when user navigates to other pages it uses the same block output which is produced for the first page. At first I thought that might be theme suggestion problem but then I realized that when I clear the caches it works fine.
Any suggestion for the case when we want a block be output in different manner based on some condition?
Comment: You need to add that `context` that changes the appeance to the `#cache` array, in your case `url.path`. See [What is the correct way to set cache contexts on custom blocks?](https://drupal.stackexchange.com/questions/223376/what-is-the-correct-way-to-set-cache-contexts-on-custom-blocks)
Comment: See https://drupal.stackexchange.com/questions/182005/is-front-is-not-available-in-all-templates
Answer:
Any suggestion for the case when we want a block be output in different manner based on some condition?

You want Cache Contexts: https://www.drupal.org/docs/drupal-apis/cache-api/cache-contexts

Cache contexts provide a declarative way to create context-dependent
variations of something that needs to be cached.

As hudri has stated, you need a cache context on url.path.
Comment: In Drupal 8.3.x `url.path.is_front` is available - saves a few database entries. @see https://www.drupal.org/docs/drupal-apis/cache-api/cache-contexts


Summary and frequency tables in JMP

Question:
Does anyone know how to create a nice looking summary table in JMP for several variables? I use Tables>Summary and the table I get has one row that includes for example mean(var1) mean(var2) mean(var3) median(var1) median(var2) median(var3) and so on. I would like each variable to have row of its own, with columns representing the statistics.
Does anyone know how to create a frequency table when creating a histogram in JMP? I get a histogram by using Analyze>Distributions but I don't see any option of getting the frequency table.

Thank you,
Naama Wolf
Comment: I assume you have found the answers yourself in the meantime?
Answer: for your first question:
this should be possible with Table/ Create Table
(I'm using the german version of JMP7, so I'm not sure how the menu commands are called in the english version)

Summary and frequency tables in JMP


Given the advancements in statistical testing, can estimating correlations be an end in itself?

Question: Background Context:
I am amazed at the complex statistical questions people ask on this site and I have learnt a lot from them. Thank you all!
However, I have focused on correlation because it is the easist method to understand for a person like me who has no formal training in statistics but is required to do market research as part of my job. A lot of my projects seek to test the relationship between two variables, so Pearson's r is the one I use.
I know each statistical method has a specific purpose and requires certain basis assumptions to be met to function well.
Questions:

Can correlation (in particular, Pearson's r) be an end in itself?
Given all the advancements in statistical testing, is it still a robust method for data analysis?
Answer: For me the answer to your question is no.
I do not think that a given method or technique can be an end in itself. If you have some data at hand or even before you start collecting data, you should ask yourself what are the problems you want to tackle, or what are the questions you would like to answer. When you have the data and the question you can start thinking about techniques.
If you are focussing on one particular technique you are likely to hide a lot from yourself. In that case, the questions you will answer or would like to answer are very much conditioned by the technique you intend to use. Think a lot about the following proverb:

If all you have is a hammer, everything looks like a nail

In addition I would like to note the following. There are indeed a lot of fancy techniques out there. However, the mere fact of using a fancy and sophisticated technique does not validate a statistical analysis by itself. Or to phrase it differently, a fancy technique is no subsitute for a convincing empirical strategy.
Comment: +1 Every bit of this excellent advice applies to the sequence of questions you have been asking on this site, @Adhesh.  I would prefer, though, that one remark had been made with slightly different emphasis: the time to start thinking about analysis techniques is *before* you collect data, not afterwards (which for many people is too late).
Comment: "Hiring a statistician after the data have been collected is like hiring a physician when your patient is in the morgue. He may be able to say what went wrong, but he is unlikely to be able to fix it" = George Box (I believe)
Comment: Thanks lejohn. It makes sense! Let's see what others have to say...
Answer: It is true, as @lejohn said, that if all you have is a hammer, everything looks like a nail.
It is also true, though, that if all you have is a nail, then you might only need a hammer!
The thing to do is to define your substantive question, whether it be from market research, psychology, physics or whatever. Then investigate methods, probably not on your own. The method to solve your problem MIGHT be correlation. It might be something else that is very simple. But it might not.
Answer: Plotting you data should never be overlooked. In this example, the correlation between X and Y is 0, but surely the two variable are related.

 |  x                         x
 |   x                      x
 |    x                    x
 |     x                  x
Y|      x               x
 |        x           x
 |          x       x
 |            x x x
 |_______________________________
               X
Comment: Indeed.  See also the other parts of the Anscombe quartet http://en.wikipedia.org/wiki/Anscombe%27s_quartet
Answer: I agree with the others who have posted in this thread but have one point to add: Simple methods -- like correlation -- are more justifiable when you can be more sure that there aren't more complicated things going on.
In experimental work where you have used randomization to take into account potentially complicating observed and unobserved variables, you're get much more mileage out of a simple correlation than you will out of observational data where you've got all kind of complicated interplay between the variables you've measured and the variables you haven't. I'm not sure what kind of market research data you have. If it's observational, I'd be even more worried about relying only on correlations.
That said, simple methods like correlation play a very important role. What I try to do in most of my work is start out with simple relationships like correlations, t-tests, and chi-squared tests that establish the relationship in terms that people are more likely to understand intuitively. Only after I've built a substantive understanding, I'll present more complicated models. At that point, it's about addressing threats and creating better estimates than it is about making the strong substantive point.
Comment: Welcome to our site, Benjamin!  Thanks for weighing in with your thoughts.


Views 3 remove exposed filter programmatically

Question: I've got several node types each of them referenced to different taxonomy term vocabulary.
For category showing i use taxonomy_display module.
For example, we've got two categories: cues and billiard balls.
Cues has price from, price to and wood type filters.
But balls has no wood type filter.
So i need to remove wood type filter if taxonomy_term tid referenced to billiard balls vocabulary.
Thus, I need to remove one of many exposed filters programmatically.
function modulename_views_pre_view(&$view, &$display_id, &$args) {
  // Some custom logic wich field_info_instances checking ...
  $filter_field = 'filter_id';
  // Removes from everywhere where i can find filter or filters properties
  unset($view->display[$view->current_display]->display_options['filters'][$filter_field]);
  unset($view->display[$view->current_display]->handler->options['filters'][$filter_field]);
  unset($view->display_handler->display->display_options['filters'][$filter_field]);
  unset($view->display_handler->options['filters'][$filter_field]);
}

Filter field removes successfully, but I've got php notice:
  Notice: Undefined index: field_wood_reference_tid in function views_handler_filter_term_node_tid->exposed_validate()

Also i try to remove field in hook_pre_execute(), but with the same result:
function modulename_views_pre_execute(&$view) {
  $filter_field = 'filter_id';
  unset($view->display_handler->handlers['filter'][$filter_field]);
  unset($view->filter[$filter_field]);
}

Also try with override_option() method like here - http://groups.drupal.org/node/82219, but no result at all.
Any suggestions?
Pls help =)

Thanks to everybody for reply but I still has no answer.
Maybe something is not clear? =(
Comment: Have you tried [hook_views_pre_render()](http://api.drupal.org/api/views/views.api.php/function/hook_views_pre_render/7)? You could also show/hide relevant filters depending on selection with jQuery.
Comment: Hi, I understand that your solution works okay, but there is **not right way**. So i can hide it using even CSS-conditions...

Trying to explain.

Each kind of code should place where it should be. For this issue if i hide exposed filter pre or after render, views still processes it. And if i try adding query string, smt like `?filter_id=val`, views returns empty display or illegal choice error...
Comment: Have you tried my answer?
Comment: I don`t need to altering form (I know that your variant works), I wanna modify views object to has similar to yours result, but earlier than form builds.
Answer: I would use hook_form_alter using unset on the form object as you are in your example for the element that you want to remove.
Answer: Here is the tested snippet which will allow you to remove the filters from the views exposed form using form alter as mentioned by Scott Thomas,
/**
 *Implements hook_form_FORM_ID_alter().
 */
function hook_form_views_exposed_form_alter(&$form, &$form_state) {
  $filter_field = 'filter_id';
  // Get the filters list for the current view. Here page_1 is the display ID.
  $filters = $form_state['view']->get_items('filter', 'page_1');
  if (isset($filters[$filter_field])) {
    $info_key = 'filter-' . $filter_field;
    unset($form[$filter_field], $form['#info'][$info_key]);
  }
}

Apart from removing the form field you also need to remove the information for particular filter which is available in $form['#info'], So that it will remove the label also. If you remove the form element alone then the filter label will be  displayed even though the field is removed, So always make sure that you're removing this information also.
Modify this according to your need. If you want to perform this one for particular view then add a condition for that too like ,
if ($form_state['view']->name == 'view_name') {
  // DO your operation.
}

Here $form_state['view'] will give you the current view object which is being processed.
Hope this helps.
Comment: This was the only solution that hid the labels as well as field widgets for me, and it worked in an exposed filter form in a block.
Answer: Follow the steps below

Firstly write a hook_form_alter in a custom module
Next try the following piece of code
unset($form['#info']['your_field_name']);
$form['your_field_name']['#access'] = 'FALSE';

Hope that helps.
Answer: You can remove exposed filter using template file as well.
Use this command:
cp sites/all/modules/contrib/views/theme/views-view.tpl.php sites/all/themes/costa/templates/views/views-view--<machine_name>.tpl.php

In the template file find this code:
  <?php if ($exposed): ?>

      <?php print $exposed; ?>

  <?php endif; ?>

And remove it. Your exposed filter will be removed from display, but will keep working of you pass proper arguments in URL.
Comment: Thx for reply, but I'm waiting D8 that uses core form api for building exposed filters :)
Answer: Correct me if I'm wrong because I'm not sure to fully understand your issue.
So you would like to hide an exposed filter in a view depending on an other exposed filter's value? If so you can try Views Dependent Filters module. I used it several times and it does the job.
You can check our fiend Lullabot's review about this module.
If you really need to do it in code the exposed option should work:
filter[$filter_id]->options['exposed'] = FALSE;
Comment: I would like to hide exposed filter if current content type has no this (filtered) field.

Trying your snippet `$view->display_handler->handlers['filter'][$filter_field]->options['exposed'] = FALSE;` i have fatal error - `PHP Fatal error:  Call to undefined method stdClass::access() in .../view.inc on line 766`. I think its correct behavior because if filter is not exposed, it also required default value. Any ideas?
Comment: http://pastebin.com/f1FKgUde here is my code, maybe it will be more clear than my english
Answer: The following did it for me inside hook_form_alter:
$info_key = 'filter-' . $fieldName;
unset($form[$fieldName], $form['#info'][$info_key], $form_state['view']->display_handler->options['filters'][$fieldName], $form_state['view']->display_handler->handlers['filter'][$fieldName], $form_state['view']->filter[$fieldName]);

Answer: Method 1
By using hook_views_query_alter().
See the following example:
<?php
/**
 * Implements hook_views_query_alter().
 */
function foo_views_query_alter(&$view, &$query) {

  if ($view->name == 'foo_view') {

    // Allow any distance when the postcode it is not specified.
    if (empty($_GET['postcode']['postal_code']) || $_GET['postcode']['postal_code'] === 'All') {
      // Scan through the query.
      foreach ($query->where as $condition_group_key => &$condition_group) {
        foreach ($condition_group['conditions'] as $condition_key => &$condition) {
          $search_name = '(COALESCE(ACOS(';
          if (is_string($condition['field']) && strstr($condition['field'], $search_name) !== FALSE) {
            // Remove filter from the query.
            unset($query->where[$condition_group_key]['conditions'][$condition_key]);
          }
        } // end: foreach
      } // end: foreach
    } // end: if

    /*
     * Change the field conditions.
     * Possible field values: 1, 2, 3
     * Logic: When 3 is selected, then display 1, 2 and 3.
     */
    switch (@$view->display_handler->handlers['filter']['field_123_value']->value[0]) {

      case 3:
        foreach ($query->where as $condition_group_key => &$condition_group) {
          foreach ($condition_group['conditions'] as $condition_key => &$condition) {
            if($condition['field'] == 'field_data_field_123.field_123_value') {
              unset($query->where[$condition_group_key]['conditions'][$condition_key]);
              $query->where[] = array(
                  'conditions' => array(
                      array(
                          'field' => 'field_data_field_123.field_123_value',
                          'value' => 1,
                          'operator' => "=",
                      ),
                      array(
                          'field' => 'field_data_field_123.field_123_value',
                          'value' => 2,
                          'operator' => "=",
                      ),
                      array(
                          'field' => 'field_data_field_123.field_123_value',
                          'value' => 3,
                          'operator' => "=",
                      ),
                  ),
                  'args' => array(),
                  'type' => 'OR',
              );
            }
          }
        } // end: foreach
        break;

    } // end: switch

  } // end: if
}

Method 2
See the example by using hook_views_pre_execute and custom function which trying to find the right field condition and return the reference to it:
/**
 * Implements hook_views_pre_execute().
 */
function foo_views_pre_execute(&$view) {

  if ($view->name == 'foo_view') {

    foo_get_view_filter_recursively(
      $view,
      $view->build_info['query']->conditions(),
      'field_data_field_123.field_123_value',
      $filter
    );

    // We want our filter to work as a bit mask.
    $filter[0]['operator'] = '&';

    unset ($filter);

    // Example of finding Proximity filter condition
    $search_name = '(COALESCE(ACOS(';

    foo_get_view_filter_recursively(
      $view,
      $view->build_info['query']->conditions(),
      $search_name,
      $filter
    );

    if (empty($_GET['postcode']['postal_code']) || $_GET['postcode']['postal_code'] === 'All') {
      // Allowing any distance.
      $filter[0]['value'][':distance'] = 10000000;
    }
    else {
      $filter[0]['value'][':distance'] = 80000;
    }

    unset ($filter);

    // Fetching single record?

    foo_get_view_filter_recursively(
      $view,
      $view->build_info['query']->conditions(),
      'node.nid',
      $filter
    );

    if (!empty($_GET['nid'])) {
      $filter[0]['value'] = (int) $_GET['nid'];
    }
    else {
      $filter[0]['operator'] = '<>';
    }

    unset ($filter);

    // echo '
'; print_r($view->build_info['query']); exit;
  }
}

/**
 * Custom function to find the field condition within the view
 */
function foo_get_view_filter_recursively($view, &$conditions, $field_name, &$filter) {

  if (!empty($conditions)) {

    foreach ($conditions as &$condition) {
      if ($condition instanceof DatabaseCondition) {
        if (foo_get_view_filter_recursively($view, $condition->conditions(), $field_name, $filter)) {
          return TRUE;
        }
      } else if ($condition['field'] instanceof DatabaseCondition) {
        if (foo_get_view_filter_recursively($view, $condition['field']->conditions(), $field_name, $filter)) {
          return TRUE;
        }
      } elseif (is_string($condition['field']) && strstr($condition['field'], $field_name) !== FALSE) {
        @$filter = array(&$condition);
        return TRUE;
      }
    } // end: foreach

  } // end: if

  return FALSE;
}

Answer: I'm removing it from template files. Override the views-view.tpl.php and remove the following code from it:
  <?php if ($exposed): ?>

      <?php print $exposed; ?>

  <?php endif; ?>

Answer: I think the op was on the right track. I had a similar situation without exposed filters so I could not use the hook_form_alter method. Here is some example code for anyone else coming across this:
$view->set_item($view->current_display, 'filter', $filter_id, null);

The view object does not have a remove_item function but simply states in the code that you can set it to null in order to remove an item including filters:
views/includes/view.inc on line 2526.
Here is a full example for anyone that targets a specific view and display:
/**
 * Implements HOOK_views_pre_view().
 */
function HOOK_views_pre_view(&$view) {
  if($view->name == 'VIEW_MACHINE_NAME') {
    switch($view->current_display) {
      case 'VIEW_DISPLAY_MACHINE_NAME':
        $view_filters = $view->display_handler->get_option('filters');
        foreach ($view_filters as $filter_id => $filter) {
          if ($filter_id == 'my_filter') {
            $view->set_item($view->current_display, 'filter', $filter_id, null);
          }
        }
      break;
    }
  }
}


T/F: The interval of width $n$ containing the most amount of primes is $[2,n+2]$?

Question: Given $n\in\mathbb{N},$ the interval of width $n$ containing the most amount of primes is $[2,n+2]$ ( rather than $[2+x, n+2+x]$ ).
This sounds like it should be true since the primes spread out more in general as $x$ increases. But general trends don't prove specific statements.
This is obviously true for $n=1$ as there is no even prime greater than $2,$ and so for any $x\geq 1,\ [x+2,x+3]$ contains at most one prime whereas $[2,3]$ contains two primes.
I think simple arguments like this can be made for small $n$.
For fixed even $n\geq 2, n$ even, and for all $x\geq 1:$ (number of primes in $[2, n+2]$ ) $\geq$ (number of primes in $[x+2, n+2+x]$ ) implies that (number of primes in $[2, (n+1)+2]$ ) $\geq$ (number of primes in $[x+2, (n+1)+2+x]$ ) because $\underset{x\geq 1}{\max}$(number of primes in $[x+2, n+3+x]$ ) $=\underset{x \text{ odd}}{\max}$ (number of primes in $[x+2, n+2+x]$ ) . Therefore, we only need to check even $n'$s going forward (because if it's true for even $n$ then it's true for $n+1$.)
$n=2:\ $ for any $x\geq 1,\ [x+2,x+4]$ contains at most two primes, which can only happen if $x$ is odd, which is not more than the number of primes in $[2,3].$
$n=4$ and $n=6$ are easy to check.
$n=8:\ [2,10]$ contains four primes. In order for $[x,x+8]$ to contain five primes, $x$ must be odd and $x, x+2, x+4, x+6, x+8$ must all be prime. But one of $x, x+2, x+4$ is divisible by three and so this is not possible.
Comment: Asymptotically this is true, since if $x\to\infty$, then the number of primes in $[2+x,n+2+x]$ will be approximately $\frac{2+x+n}{\ln(2+x+n)} - \frac{2+x}{\ln(2+x)} \stackrel{x\to\infty}{\rightarrow} 0$.
However it is technically possible that there is a really dense interval of say length $100$, with more primes than the interval $[2,102]$ contains (which is $26$).
I just wrote a quick Python program to check all primes up to $n=1,000,000$, and all interval sizes up to $x=500$, and in every case, the interval $[2+x,n+2+x]$ contained the most primes. But of course, this is far from a proof.
Comment: I also watched the video you linked, and Tao didn't talk about this specifically.
Comment: Well, the disjoint union of $[2,n+2]$ and $[2+x,n+2+x]$ is $[2,2+A] \cup [N,N+A]$   where $N=(n+3)$ and $A=(x-1)$. I don't know of a proof off-hand but I am about positive that there are at least as many primes in $[2,2+A]$ as there are in $[N,N+A]$ for general positive integers $N$ and $A$.
Comment: @Daniel P- you mean the interval $[2, n+2]$ contains the most primes?
Comment: @AdamRubinson Ah, sorry. I meant $[2,n+2]$, not $[2+x,n+2+x]$.
Comment: For all $n\ge 2$, let $m_n\ge 2$ be an integer such that $[m_n,m_n+n]$ is optimal for $n$. If you can show that $m_n\le n+3$, you are done, because (abusing notation), we have $\pi([2,n+2])-\pi([m_n,m_n+n])=\pi([2,m_n-1])-\pi([n+3,m_n+n])$, and we can use induction on $n$ to show that this is non-negative.
Answer: I am surprised that nobody has mentioned this yet, but this is essentially the Second Hardy-Littlewood conjecture and it is most likely false, since it contradicts the theoretically much more sound First Hardy-Littlewood conjecture. There might not be a counter-example below $10^{100}$ or even $10^{1000}$, however.
Answer: For $n\ge 1$ a positive integer, let $m_n$ be the smallest positive integer for which $\pi(n+m_n)-\pi(m_n-1)$ is optimal.
Lemma For $n\ge 59$, we have $m_n\le n+1$.
Proof: For $n\ge 59$, we have the inequalities
$$
\frac{n}{\log n - 1/2}\le \pi(n)\le \frac{n}{\log n-9/8}.
$$
Therefore, if $m_n\ge n+2$, we have
$$
\log(m_n+n)-9/8\ge \log(2)+\log(n+1)-9/8> \log(n+1)-1/2\quad\text{and}\quad \log(m_n-1)-1/2\ge \log(n+1)-1/2,
$$
whence
$$
\begin{align*}
\pi(n+m_n)-\pi(m_n-1) &\le \frac{n+m_n}{\log(n+m_n)-9/8} - \frac{m_n-1}{\log(m_n-1)-1/2}\\
&\le \frac{n+m_n}{\log(n+1)-1/2} - \frac{m_n-1}{\log(n+1)-1/2}\\
&=\frac{n+1}{\log(n+1)-1/2}\le \pi(n+1).
\end{align*}
$$
So the interval $[2,n+2]$ contains more primes than $[m_n,m_n+n]$. This contradicts the definition of $m_n$. We conclude that $m_n\le n+1$. $\square$
Theorem If $m_n=2$ for all $n\le 58$, then $m_n=2$ for all $n\ge 1$.
Proof: By induction on $n$. The base case $n=1$ is clear. Let $n\ge 2$ and assume that $m_k=2$ for all $k=1,\ldots,n-1$. If $n\le 58$, we are done. Otherwise, $n\ge 59$ and by the lemma, $m_n\le n+1$. Now,
$$
\begin{align*}
\left(\pi(n+2)-\pi(1)\right) - \left(\pi(n+m_n)-\pi(m_n-1)\right)
=& \left(\pi(m_n-1)-\pi(1)\right) - \left(\pi(n+m_n)-\pi(n+2)\right)\ge 0,
\end{align*}
$$
because $m_{m_n-2}=2$, by the induction hypothesis. $\square$
Comment: (1) We have $\frac{n}{\log n-1} \le \pi(n) \le \frac{n}{\log n - 9/8}$ for all $n \ge 4$ (as I mentioned [here](https://math.stackexchange.com/a/4260895/383078); it's enough to check that this holds for $4 \le n \le 60183$). That should reduce the number of cases to check by hand.
Comment: @MishaLavrov Ooh, that's super useful. I'll edit my answer.
Comment: (2) I don't see how the second inequality on $\pi(n+m_n)-\pi(m_n-1)$ works - what lets you replace $\log(m_n - 1)$ by $\log(n+1)$? Or maybe we assume $m_n \ge n+2$, but then we get a slightly weaker result.
Comment: @MishaLavrov Good catch. The result will be slightly weaker, but still strong enough, since ultimately, we need $m_{m_n-2}=2$
Comment: Sorry, I misquoted myself. Though we get $\pi(n) \le \frac{n}{\log n - 9/8}$ at $n \ge 4$, we don't have $\pi(n) \ge \frac{n}{\log n - 1}$ until much later. My observation is much less useful now! We do get $\frac{n}{\log n - 1/2} < \pi(n) < \frac{n}{\log n - 9/8}$ for $n \ge 59$.
Comment: @MishaLavrov I've edited my answer.
Answer: This is far from a proof, I just wanted to share some code to find potential counterexamples. Though so far, the statement in the question seems to be true.

I wrote a Python program to check all primes up to $n=1,000,000$, and all interval sizes up to $x=500$, and in every case, the interval $[2,n+2]$ contained the most primes.
The way it works is it lists all the consecutive gaps between primes (dPrimes), then for every possible interval length I = 2...500 it adds up the size of all I consecutive  gaps. In each case, this sum is clearly the difference between the first and last prime in the interval, and if this difference is smaller than the sum of the first I gaps, then we have found an interval more dense than $[2,n+2]$.
def main(N = 1000000, X = 500):

    import math

    primes = [i for i in range(2,N+1) if all(i % j != 0 for j in range(2,int(math.sqrt(i))+1))]
    foundDenserInterval = False

    def diff(L):
        return [L[i+1]-L[i] for i in range(len(L)-1)]

    dPrimes = diff(primes)

    for I in range(2,X+1):
        print(I) # not necessary, just shows progress
        orig_gapsums = sum(dPrimes[:I])
        all_gapsums = [sum(dPrimes[j:j+I]) for j in range(1,len(dPrimes)-I)]
        for g in range(len(all_gapsums)):
            if all_gapsums[g] <= orig_gapsums:
                print("FOUND DENSER INTERNVAL:", [p for p in primes[g+1:] if p <= primes[g+1]+I])
                foundDenserInterval = True

    if not foundDenserInterval:
        print("DID NOT FIND A DENSER INTERVAL THAN [2,N+2]")

if __name__ == "__main__":
    main()

It took around $4$ minutes to run this on my PC (with a dual-core $2 \times 2.8$ GHz processor), so if anyone has a beefier computer, feel free to run it for even larger $N$ and $X$! Or feel free to improve on my code. :)
Comment: Here is a c++ code that does the same thing: https://pastebin.com/XVq9cwhV Compile with something like `g++ -O3 prime-intervals.cpp -o a && ./a`. It uses a sorted queue (maybe priority queue is quicker) to process the events. For a fixed interval size $x$, it takes $O(\pi(n) \log\pi(n))$ where $\pi(n)$ is the number of primes $\leq n$. With constraints 1e6 and 500 (same as yours), it takes 2 seconds, while with 1e8 and 1000, it's still running but expected to take a few minutes.
Answer: This is a strategy on how (I think) this can be proved not a complete answer.
The wikipedia page on the prime counting function has an explicit bound on the difference between the prime counting function and the logarithmic integral function.
This can be used to get an explicit interval bound for the number of primes in any interval $[a,b]$. As the error term is decaying faster than the number of primes, this should suffice to prove that there can be no counter example for $a,b$ sufficiently big with explicit bounds on how big $a,b$ need to be. One can then in principle brute force the smaller cases numerically.


Is it possible to add box shadowing to inputs in forms css?

Question: I have been trying to add some snazzy effects to my input boxes and one I found was box shadowing, I looked at the w3schools tutorial for box shadowing and tried multiple things to add it to my input box:
.l_input {
  box-shadow: 10px;
}

That didn't seem to work for my code. I also tried to use it using a hover pseudo effect and it again didn't work for that, the pseudo code I tried doing:
.l_input {
  box-shadow: 5px;
}
.l_input:hover {
  box-shadow: 10px;
  transition: 2s;
}

And again that didn't seem to work, if this is the right syntax for how the code should work or I am doing a small mistake thanks in advance, I have thought that maybe it isn't supported for inputs, but that sounded weird.
Answer:

.l_input  {
  box-shadow: 0 1px 2px rgba(0,0,0,0.15);
  transition: 0.3s ease-in-out;
}

/* hover effect  */
.l_input:hover {
  box-shadow: 0 5px 15px rgba(0,0,0,0.3);
}

 Content

Answer: You can give box shadow like following way:
.l_input {
    box-shadow: 5px 5px 5px #000;
}

You should give at least first two value.
First two describe horizontal shadow and vertical shadow. Third one is blur and color.
box-shadow: [horizontal offset] [vertical offset] [blur radius] [optional spread radius] [color];

Reference link:
Working Fiddle
And you can give effect to shadow following way.
.l_input {
 box-shadow: 5px 5px 5px #000;
}
.l_input:hover {
  box-shadow: 10px 10px 10px #000;
  transition: 2s;
}

Fiddle
Answer: It should be worked with below code.If your class is .l_input then it will be worked.Its depends on your mark.
.l_input {
box-shadow: 5px 5px 5px #000;
}

If not then try below code:
You can see in JSFIDDLE .

body {
  background: #ccc
}
.box h3 {
  text-align: center;
  position: relative;
  top: 80px;
}
.box {
  width: 70%;
  height: 200px;
  background: #FFF;
  margin: 40px auto;
}
/*==================================================
 * Effect 1
 * ===============================================*/

.effect1 {
  -webkit-box-shadow: 0 10px 6px -6px #777;
  -moz-box-shadow: 0 10px 6px -6px #777;
  box-shadow: 0 10px 6px -6px #777;
}
/*==================================================
 * Effect 2
 * ===============================================*/

.effect2 {
  position: relative;
}
.effect2:before,
.effect2:after {
  z-index: -1;
  position: absolute;
  content: "";
  bottom: 15px;
  left: 10px;
  width: 50%;
  top: 80%;
  max-width: 300px;
  background: #777;
  -webkit-box-shadow: 0 15px 10px #777;
  -moz-box-shadow: 0 15px 10px #777;
  box-shadow: 0 15px 10px #777;
  -webkit-transform: rotate(-3deg);
  -moz-transform: rotate(-3deg);
  -o-transform: rotate(-3deg);
  -ms-transform: rotate(-3deg);
  transform: rotate(-3deg);
}
.effect2:after {
  -webkit-transform: rotate(3deg);
  -moz-transform: rotate(3deg);
  -o-transform: rotate(3deg);
  -ms-transform: rotate(3deg);
  transform: rotate(3deg);
  right: 10px;
  left: auto;
}
/*==================================================
 * Effect 3
 * ===============================================*/

.effect3 {
  position: relative;
}
.effect3:before {
  z-index: -1;
  position: absolute;
  content: "";
  bottom: 15px;
  left: 10px;
  width: 50%;
  top: 80%;
  max-width: 300px;
  background: #777;
  -webkit-box-shadow: 0 15px 10px #777;
  -moz-box-shadow: 0 15px 10px #777;
  box-shadow: 0 15px 10px #777;
  -webkit-transform: rotate(-3deg);
  -moz-transform: rotate(-3deg);
  -o-transform: rotate(-3deg);
  -ms-transform: rotate(-3deg);
  transform: rotate(-3deg);
}
/*==================================================
 * Effect 4
 * ===============================================*/

.effect4 {
  position: relative;
}
.effect4:after {
  z-index: -1;
  position: absolute;
  content: "";
  bottom: 15px;
  right: 10px;
  left: auto;
  width: 50%;
  top: 80%;
  max-width: 300px;
  background: #777;
  -webkit-box-shadow: 0 15px 10px #777;
  -moz-box-shadow: 0 15px 10px #777;
  box-shadow: 0 15px 10px #777;
  -webkit-transform: rotate(3deg);
  -moz-transform: rotate(3deg);
  -o-transform: rotate(3deg);
  -ms-transform: rotate(3deg);
  transform: rotate(3deg);
}
/*==================================================
 * Effect 5
 * ===============================================*/

.effect5 {
  position: relative;
}
.effect5:before,
.effect5:after {
  z-index: -1;
  position: absolute;
  content: "";
  bottom: 25px;
  left: 10px;
  width: 50%;
  top: 80%;
  max-width: 300px;
  background: #777;
  -webkit-box-shadow: 0 35px 20px #777;
  -moz-box-shadow: 0 35px 20px #777;
  box-shadow: 0 35px 20px #777;
  -webkit-transform: rotate(-8deg);
  -moz-transform: rotate(-8deg);
  -o-transform: rotate(-8deg);
  -ms-transform: rotate(-8deg);
  transform: rotate(-8deg);
}
.effect5:after {
  -webkit-transform: rotate(8deg);
  -moz-transform: rotate(8deg);
  -o-transform: rotate(8deg);
  -ms-transform: rotate(8deg);
  transform: rotate(8deg);
  right: 10px;
  left: auto;
}
/*==================================================
 * Effect 6
 * ===============================================*/

.effect6 {
  position: relative;
  -webkit-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  -moz-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
}
.effect6:before,
.effect6:after {
  content: "";
  position: absolute;
  z-index: -1;
  -webkit-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  -moz-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  top: 50%;
  bottom: 0;
  left: 10px;
  right: 10px;
  -moz-border-radius: 100px / 10px;
  border-radius: 100px / 10px;
}
.effect6:after {
  right: 10px;
  left: auto;
  -webkit-transform: skew(8deg) rotate(3deg);
  -moz-transform: skew(8deg) rotate(3deg);
  -ms-transform: skew(8deg) rotate(3deg);
  -o-transform: skew(8deg) rotate(3deg);
  transform: skew(8deg) rotate(3deg);
}
/*==================================================
 * Effect 7
 * ===============================================*/

.effect7 {
  position: relative;
  -webkit-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  -moz-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
}
.effect7:before,
.effect7:after {
  content: "";
  position: absolute;
  z-index: -1;
  -webkit-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  -moz-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  top: 0;
  bottom: 0;
  left: 10px;
  right: 10px;
  -moz-border-radius: 100px / 10px;
  border-radius: 100px / 10px;
}
.effect7:after {
  right: 10px;
  left: auto;
  -webkit-transform: skew(8deg) rotate(3deg);
  -moz-transform: skew(8deg) rotate(3deg);
  -ms-transform: skew(8deg) rotate(3deg);
  -o-transform: skew(8deg) rotate(3deg);
  transform: skew(8deg) rotate(3deg);
}
/*==================================================
 * Effect 8
 * ===============================================*/

.effect8 {
  position: relative;
  -webkit-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  -moz-box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
  box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3), 0 0 40px rgba(0, 0, 0, 0.1) inset;
}
.effect8:before,
.effect8:after {
  content: "";
  position: absolute;
  z-index: -1;
  -webkit-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  -moz-box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  box-shadow: 0 0 20px rgba(0, 0, 0, 0.8);
  top: 10px;
  bottom: 10px;
  left: 0;
  right: 0;
  -moz-border-radius: 100px / 10px;
  border-radius: 100px / 10px;
}
.effect8:after {
  right: 10px;
  left: auto;
  -webkit-transform: skew(8deg) rotate(3deg);
  -moz-transform: skew(8deg) rotate(3deg);
  -ms-transform: skew(8deg) rotate(3deg);
  -o-transform: skew(8deg) rotate(3deg);
  transform: skew(8deg) rotate(3deg);
}

Effect 1

Effect 2

Effect 3

Effect 4

Effect 5

Effect 6

Effect 7

Effect 8

Answer: Try This

.l_input  {
 box-shadow: 3px 3px 2px #666;
  transition: 0.2s all linear;
  border:1px solid #999;
}

.l_input:hover {
 box-shadow: 0 5px 15px #666;
 transition: 0.2s all linear;
}
  <input type="text" class="l_input">


taxonomy terms view with custom alphabar (for Spanish)

Question: I have a lot of views displaying taxonomy vocabularies with alphabar, the problem is that all the alphabar are by default showing the order for English alphabet, Spanish has ñ and accented letters, at both the alphabar as in the views, words starting with this letters are sent to the back, for instance, alphabar is like this:

......w | x | y | z | ñ | á | é | í | ó | ú

the ñ should be placed after n and those letters with accent shouldn't appear, they should be included within the corresponding unaccented letter
same for order in the view, for instance words are ordered like this (fictional words for the example)

awto
axma
aypa
azor
años
aísi
aóko
etc.

when correct order should be like this:

aísi
años
aóko
awto
axma
aypa
azor

Any idea on how to solve this? Thanks in advance!
Answer: I've been reading some documentation about
character settings in mysql, and I have found a solution, I don't know if it's the right way for drupal, and it would not interfere with the taxonomy modules at update time in some way, I hope somebody could comment on this....
Anyway I just changed at the taxonomy_term_data table the collation of the "name" field to utf8_spanish_ci


SQL query not working when used with select tag

Question: Before I Used The select tag the code was working perfectly...but now it isn't....
Is there anything special about select tag that has to be handled separately....?
I know the code is too long to read...but i can assure that the code worked perfectly when the select was replaced by an  input type="text"
Note : The page slides down to the results section after i click the link, but appends nothing to the html page....
Here is the Form's Code :
<form name="queryForm" id="outputform">
<input id="date2" type="date" class="text" name="date2" value="2014-07-30" />
<select form="outputform" name="mode2">
                        <option value="">Select Travel Mode</option>
                        <option value="Train">Train</option>
                        <option value="Flight">Flight</option>
</select>
<input id="time2" type="time" class="text" name="time2" value="12:00:00" />
<input id="tfno2" type="text" class="text" name="tfno2" placeholder="Train/Flight Number" />

Query It
</form>

Here's The Ajax Function :
function ajaxFunction2() {
    var ajaxRequest;  // The variable that makes Ajax possible!

    try {
        // Opera 8.0+, Firefox, Safari
        ajaxRequest = new XMLHttpRequest();
    } catch (e) {
        // Internet Explorer Browsers
        try {
            ajaxRequest = new ActiveXObject("Msxml2.XMLHTTP");
        } catch (e) {
            try {
                ajaxRequest = new ActiveXObject("Microsoft.XMLHTTP");
            } catch (e) {
                // Something went wrong
                alert("Your browser broke!");
                return false;
            }
        }
    }
    // Create a function that will receive data
    // sent from the server and will update
    // div section in the same page.
    ajaxRequest.onreadystatechange = function () {
        if (ajaxRequest.readyState == 4) {
            var ajaxDisplay = document.getElementById('results');
            ajaxDisplay.innerHTML = ajaxRequest.responseText;
        }
    }
    // Now get the value from user and pass it to
    // server script.
    var date = document.getElementById('date2').value;
    var mode = document.getElementById('mode2').value;
    var time = document.getElementById('time2').value;
    var tfno = document.getElementById('tfno2').value;
    if (date == "" || date == null || date == '') {

        alert("Not Sure When You Are Going..?? \n Search Later....!! ");

        exit();
    }
    var queryString = "?date=" + date;
    queryString += "&mode=" + mode + "&time=" + time + "&tfno=" + tfno;
    ajaxRequest.open("GET", "insert2.php" +queryString, true);
    ajaxRequest.send(null);
}

Here's The PHP Code :
    <?php
    $dbhost = "localhost";
    $dbuser = "root";
    $dbpass = "plsdonthack";
    $dbname = "cab";
        //Connect to MySQL Server
    mysql_connect($dbhost, $dbuser, $dbpass);
        //Select Database
    mysql_select_db($dbname) or die(mysql_error());
        // Retrieve data from Query String
    $date = $_GET['date'];
    $time = $_GET['time'];
    $mode = $_GET['mode'];
    $tfno = $_GET['tfno'];

        // Escape User Input to help prevent SQL Injection
    $date = mysql_real_escape_string($date);
    $time = mysql_real_escape_string($time);
    $mode = mysql_real_escape_string($mode);
    $tfno = mysql_real_escape_string($tfno);
        //build query
    $query = "SELECT * FROM cabs WHERE DATE='$date' ";
    if ($mode!=='' || $mode!=="")
          $query .="AND MODE='$mode' ";

    if ($tfno!=='')
          $query .="AND TFNO='$tfno' ";

    $qry_result = mysql_query($query) or die(mysql_error());

        //Build Result String
    $display_string = "

Here Are The    Results...!!

ID	Name	Description	Contact  No.

";

    // Insert a new row in the table for each person returned
    while($row = mysql_fetch_array($qry_result)){
    $display_string .="
" . $row[IDNO] . "	" . $row[NAME] . "

 Date : " . $row[DATE] . "

 Time : " . $row[TIME] . "

Train/Flight No. " . $row[TFNO] .   "

	" . $row[CONTACT] . "
";

    }
    $display_string .= "
";
    echo $display_string;
    ?>

Comment: `name` and `id` are different things. Using getelementbyid on a name won't do anything
Comment: yup..got it was a miss on my part..thank you..!
Answer: var mode = document.getElementById('mode2').value;

You don't have id on your select so your javascript get no value for mode
<select id="mode2" name="mode2">

Comment: OHH MAN..HOW COULD I MISS THAT....THANks FOR THE PATIENCE FOR READING MY LONG CODE......answer will be accepted...
Answer: Simply adding a get element by document.getElementById("id") in the code can solve and appending the select box will work just fine.
http://www.w3schools.com/jsref/met_doc_getelementbyid.asp
Comment: never ever ever use w3schools....its tutorials dont keep you updated with compatibility standards plus lots of tutorials are written without security concerns!! :)


What's so special about the discriminant in the form $b^2 - 4ac$?

Question: The following is the quadratic formula derived by completing the square:
$$x = -\frac{b}{2a}  \pm \sqrt{ \frac{b^2}{4a^2} - \frac{c}{a}} \tag{1}$$
This equation is equivalent:
$$x = \frac{-b \pm \sqrt{b^2 -4ac}}{2a} \tag{2}$$
We can obtain $(2)$ from $(1)$ by trying to simplify the term under the $\sqrt{\dots}$ as much as possible.
I've always felt there is no such thing as "the quadratic formula", and that both $(1)$ and $(2)$ are equally arbitrary.

Everything the discriminant $b^2 - 4ac$ can do, so can $\frac{b^2}{4a^2} - \frac{c}{a}$ (literally because  it's just a scalar multiple of the other).
What makes the form $b^2 - 4ac$ so special? It does not seem to be "arbitrary"? As stated in the Wikipedia article for it:

The discriminant is widely used in polynomial factoring, number theory, and algebraic geometry

I could try to "formalize" the notion of:

"Maximally simplifying the term under the $\sqrt{\dots}$"
"The discriminant in $(1)$ normalized is the discriminant in $(2)$"

However this does not seem intuitive nor correct.

Why is the discriminant $b^2 - 4ac$ not arbitrary/special?
Does it appear elsewhere more "naturally"?
Comment: I upvoted all answers.
I want to point out it also follows from matrix theory and others.
And there is also the lesser known formula with the square root in the denom ofcourse.
Comment: If $a=0$ the regular definition still works :)
Comment: I can also write it as $x=\frac{-b}{2a} \pm \frac{\sqrt{4a^{2}b^{2}-16a^{3}c}}{4a^{2}}$, so we have $4a^{2}b^{2}-16a^{3}c$, which works for $a = 0$.
Comment: Does it? I thought $0/0$ is a bit of a pickle
Comment: Ah sorry I see what you mean, but no, it won't work the same. It'll give you $0$ instead of $b$ for $a=0$
Comment: It is a bit funny applying the definition to first degree polynomials either way, admittedly.
Comment: In the case of quadratics, the sign of the discriminant tells you how many real solutions there are. If positive, you get 2 real roots, if 0, 1 rea read, and if negative, there are only complex roots. In this sense, it is arbitrary up to a multiplicative factor, however this definition is more consistent with higher order discriminants.
Answer: The discriminant generalizes to polynomials of any degree (not just quadratics). Given a polynomial $$p(x) =a_{n}\prod^{i}_{1\dots n} (x-r_{i}) $$
the discriminant is the formula for:
$$\Delta = a_{n}^{2n-2}\prod_{i<j}(r_{j}-r_{i})^2$$

For a second degree polynomial:
$$p_2(x)  = ax^2 + bx + c = \\  = a(x-r_{1})(x-r_{2})$$
the second degree discriminant formula gives:
$$\Delta_{2} = b^2 - 4ac =\\ =  a^2 \;(r_{2}-r_{1})^2 $$

For a third degree polynomial:
$$p_3(x) =  ax^3 + bx^2 + cx + d = \\ a(x-r_{1})(x-r_{2})(x-r_{3}) $$
the third degree discriminant formula is:
$$\Delta_{3} = 18abcd – 4b^3d + b^2c^2 – 4ac^3 – 27a^2d^2 = \\ = a^4 \;(r_{2}-r_{1})^2(r_{3}-r_{1})^2(r_{3}-r_{2})^2 $$

The discriminant gives the (squared) product of the differences of distinct roots of a $n$'th degree polynomial, times the top coefficient to the power $2n-2$.
There exists formulas for higher degrees. The general formula for the discriminant is the determinant of the Sylvester matrix of $p(x)$ and $p'(x)$, divided by the top coefficient of $p$.

The discriminant formula can be characterized by the following properties:

It's an algebraic  formula in the coefficients of our polynomial. That is, it uses only sums and products of the coefficients, no fractions.

It is equal to $0$ if and only if the polynomial has repeated roots. That is, it discriminates the polynomials which don't have $n$ distinct complex roots. This can be seen by looking at its expression in terms of the polynomial roots.

It is the simplest formula with the above properties. Specifically, it is a divisor for any formula satisfying them. This makes it unique up to a constant factor.

More info on the generalized discriminant formula can be found here: https://en.wikipedia.org/wiki/Discriminant
Comment: A small typo: you want $\Delta = a_{n}^{2n-2}\prod_{i<j}(r_{j}-r_{i})^2$ (rather than $r_1$)
Comment: Fixed. Thank you.
Answer: The definition of the quadratic discriminant as $\,b^2 - 4ac\,$ makes it consistent with the definition of the discriminant for higher degree polynomials.
In general, for a polynomial $\,p(x)=a_nx^n+a_{n-1}x^{n-1}+\ldots+a_1x +a_0\,$ of degree $n$ with roots $\,x_1, x_2, \ldots.x_n\,$ the discriminant can be defined as $\,\Delta = a_n^{2n-2} \prod_{1 \le i \lt j \le n}(x_i-x_j)^2\,$. For the quadratic $\,ax^2+bx+c\,$ this reduces to $\,\Delta = a^2(x_1-x_2)^2 = b^2 - 4ac\,$.
Answer: Geometrically for a parabola with equation $y=ax^2+bx+c$ the coordinates of the vertex are given by $V=(x_v,y_v)=\left(-\frac b{2a},-\frac {\Delta}{4a}\right)$ such that the condition on the sign of $\Delta$ with the sign of $a$ determine the number of solutions for the quadratic equation $ax^2+bx+c=0$.
Moreover $\Delta$ appears in all the key parameter for the parabola as the focus $F=(x_f,y_f)=\left(-\frac b{2a},-\frac {1-\Delta}{4a}\right)$ and the related diretrix with equation $y=-\frac {\Delta}{4a}-\frac {1}{4a}$.

In multivariable calculus, in the simpler case of $2$ variables, the study of the nature of stationary points can be done with reference to the quadratic form
$$f(x,y)=ax^2+bxy+cy^2= \begin{bmatrix}x&y\end{bmatrix}\begin{bmatrix}a&\frac b2\\\frac b2&c\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}$$
and of the Hessian determinant $$H=\begin{bmatrix}a&\frac b2\\\frac b2&c\end{bmatrix}=ac-\frac{b^2}4=-\frac{\Delta}4$$


Java RSA Input encrypted Text and decrypt it.(BigINteger)

Question: I spent a day to check out a lot of combinations of this last lines of my code but I'absolutely not going to understand how i can get an encrypted Input as BigInteger and decrypt it to a String...
 So this is the last block of the Code:
<code> package rsa;
import java.io.IOException;
import static java.lang.System.exit;
import java.math.BigInteger;
import java.security.SecureRandom;
import java.util.Scanner;
import java.util.concurrent.TimeUnit;

public class RSA {
  private BigInteger n, d, e;

  private int bitlen = 1024;

  /** Create an instance that can encrypt using someone elses public key. */
  public RSA(BigInteger newn, BigInteger newe) {
    n = newn;
    e = newe;
  }

  /** Create an instance that can both encrypt and decrypt. */
  public RSA(int bits) {
    bitlen = bits;
    SecureRandom r = new SecureRandom();
    BigInteger p = new BigInteger(bitlen / 2, 100, r);
    BigInteger q = new BigInteger(bitlen / 2, 100, r);
    n = p.multiply(q);
    BigInteger m = (p.subtract(BigInteger.ONE)).multiply(q
        .subtract(BigInteger.ONE));
    e = new BigInteger("3");
    while (m.gcd(e).intValue() > 1) {
      e = e.add(new BigInteger("2"));
    }
    d = e.modInverse(m);
  }

  /** Encrypt the given plaintext message. */
  public synchronized String encrypt(String message) {
    return (new BigInteger(message.getBytes())).modPow(e, n).toString();
  }

  /** Encrypt the given plaintext message. */
  public synchronized BigInteger encrypt(BigInteger message) {
    return message.modPow(e, n);
  }

  /** Decrypt the given ciphertext message. */
  public synchronized String decrypt(String message) {
    return new String((new BigInteger(message)).modPow(d, n).toByteArray());
  }

  /** Decrypt the given ciphertext message. */
  public synchronized BigInteger decrypt(BigInteger message) {
    return message.modPow(d, n);
  }

  /** Generate a new public and private key set. */
  public synchronized void generateKeys() {
    SecureRandom r = new SecureRandom();
    BigInteger p = new BigInteger(bitlen / 2, 100, r);
    BigInteger q = new BigInteger(bitlen / 2, 100, r);
    n = p.multiply(q);
    BigInteger m = (p.subtract(BigInteger.ONE)).multiply(q
        .subtract(BigInteger.ONE));
    e = new BigInteger("3");
    while (m.gcd(e).intValue() > 1) {
      e = e.add(new BigInteger("2"));
    }
    d = e.modInverse(m);
  }

  /** Return the modulus. */
  public synchronized BigInteger getN() {
    return n;
  }

  /** Return the public key. */
  public synchronized BigInteger getE() {
    return e;
  }

  /** Trivial test program. */
  public static void main(String[] args) throws InterruptedException, IOException {
    RSA rsa = new RSA(1024);
    loop:

    /** Input  Scanner switch*/
    Scanner first = new Scanner(System.in);
    int a = first.nextInt();

    switch (a) {
        case 1:
       /** Output Text zum verschlüsseln */
        System.out.println();
        System.out.println();
        System.out.println();
        System.out.println("### Text zum verschlüsseln eingeben: ###");

       /** Input Text zum verschlüsseln*/
        Scanner scanner = new Scanner(System.in);
        String plaintext = scanner.nextLine();

        /** Output eingegebenen Text*/
        System.out.println("Eingegebener Text: ");
        System.out.println("------------------------");
        System.out.println(plaintext);
        BigInteger bigplaintext = new BigInteger(plaintext.getBytes());

        /** Output */
        System.out.println();
        System.out.println();
        System.out.println();

    /** Output verschlüsselten Text*/
    BigInteger vbigplaintext = rsa.encrypt(bigplaintext);
    System.out.println("Veschlüsselter Text: ");
    System.out.println("------------------------");
    System.out.println(vbigplaintext);
     /** ---------------------------------------------------- */
            break;
        case 2:
            /** Output Text zum Entschlüsseln */
        System.out.println();
        System.out.println();
        System.out.println();
        System.out.println("### Verschlüsselten Text eingeben: ###");

         /** Input verschlüsselter Text*/
        Scanner scanner3 = new Scanner(System.in);
        String vertext = scanner3.nextLine();
        System.out.println(vertext);

        /**----Here is the Problem------*/

        BigInteger decbigtext = rsa.decrypt(vertext);
        BigInteger entbigtext = new BigInteger(decbigtext.toByteArray());

    System.out.println("Entschlüsselter Text:");
    System.out.println("------------------------");
    System.out.println("Plaintext: " + entbigtext);

            break;
        default:
            System.exit(0);
    }
  }
}
</code>
Would be nice if somebody could give me anwer how to solve this Problem.
If this post already exist please move or delete and sorry for my bad english.
Comment: @xTRX16 Have you tried to use `new BigInteger(1, messageBytes)`? Please don't use this code for anything critical. This is an implementation of textbook RSA without padding which is not that secure.
Comment: Are you required to implement this yourself? if you are just trying to do RSA encryption/decryption there are numerous libraries that implement it already
Answer: You are generating a new key pair each time you run your application. Obviously you cannot decrypt with a brand new private key.


Is it possible (how?) to Route incoming UDP packets with one target IP to a destination IP that is resolved through a DNS query for a static URL?

Question: I have a set of mobile devices that are configured to send UDP messages to an IP-address that is set in their SRAM. They don't have the capability to either store a URL, or resolve a URL to an IP address. The traffic going to the destination IP that the devices are sending to must pass through a gateway router under my control. The gateway router is running OpenWRT linux. I would like to route these UDP packets to Amazon Elastic Load Balancing (a AWS service). One constraint of Elastic Load Balancing is that (as far as I have been able to determine) DNS must be used to resolve the IP of the load-balancer (because at any time there may be more than one loadbalancer in use by AWS, and the choice of which to use is set at the DNS under Amazons control).
That's the long story.
The short story is "How can I direct UDP packets routed through an OpenWRT router towards an address resolved through DNS at that router?"
I'm looking for a solution through IP chains/tables, firewall rules etc (or at least "plugins" with some history) rather than custom generated code.
Thank you all in advance for your help!
Answer:
I'm looking for a solution through IP chains/tables, firewall rules etc (or at least "plugins" with some history) rather than custom generated code.

Not going to happen, sorry. Firewall and routing engines will typically resolve the DNS record to an IP address when the policy is activated, and Linux is no exception to this. The kernel will not perform additional DNS lookups to look for the DNS record changing.
The manpage for iptables makes this pretty clear, emphasis mine:

Address can be either a network name, a hostname, a network IP address (with /mask), or a plain IP address.  Hostnames will be resolved once only, before the rule is submitted to the kernel.  Please note that specifying any name to be resolved with a remote query such as DNS is a really bad idea.

DNS traditionally does not have a role in these kinds of stacks because they are trying to keep the operation as deterministic as possible. Ignoring the complexity that implementing this code would add to the overall picture, it's also a security risk because you're essentially delegating control over your security policy to whoever controls the DNS record. Even if you're 100% certain that you trust the remote server operator, UDP based DNS is inherently spoofable protocol (DNSSEC protects against upstream forgery, but provides no protections between client and recursor aka "the last mile"), making this a terrible idea across the board. If you find a product that supports this, you probably shouldn't be supporting them!
You need to remove DNS from this equation. It's the wrong tool for the job at the layer you're functioning at.
Comment: Additionally, ELB only supports TCP, not UDP.  It can't load balance UDP traffic at all, even if you could easily route it there.


Safely normalizing data via SQL query

Question: Suppose I have a table of customers:
CREATE TABLE customers (
    customer_number  INTEGER,
    customer_name    VARCHAR(...),
    customer_address VARCHAR(...)
)

This table does not have a primary key. However, customer_name and customer_address should be unique for any given customer_number.
It is not uncommon for this table to contain many duplicate customers. To get around this duplication, the following query is used to isolate only the unique customers:
SELECT
  DISTINCT customer_number, customer_name, customer_address
FROM customers

Fortunately, the table has traditionally contained accurate data. That is, there has never been a conflicting customer_name or customer_address for any customer_number. However, suppose conflicting data did make it into the table. I wish to write a query that will fail, rather than returning multiple rows for the customer_number in question.
For example, I tried this query with no success:
SELECT
  customer_number, DISTINCT(customer_name, customer_address)
FROM customers
GROUP BY customer_number

Is there a way to write such a query using standard SQL? If not, is there a solution in Oracle-specific SQL?
EDIT: The rationale behind the bizarre query:
Truth be told, this customers table does not actually exist (thank goodness). I created it hoping that it would be clear enough to demonstrate the needs of the query. However, people are (fortunately) catching on that the need for such a query is the least of my worries, based on that example. Therefore, I must now peel away some of the abstraction and hopefully restore my reputation for suggesting such an abomination of a table...
I receive a flat file containing invoices (one per line) from an external system. I read this file, line-by-line, inserting its fields into this table:
CREATE TABLE unprocessed_invoices (
    invoice_number   INTEGER,
    invoice_date     DATE,
    ...
    // other invoice columns
    ...
    customer_number  INTEGER,
    customer_name    VARCHAR(...),
    customer_address VARCHAR(...)
)

As you can see, the data arriving from the external system is denormalized. That is, the external system includes both the invoice data and its associated customer data on the same line. It is possible that multiple invoices will share the same customer, therefore it is possible to have duplicate customer data.
The system cannot begin processing the invoices until all customers are guaranteed to be registered with the system. Therefore, the system must identify the unique customers and register them as necessary. This is why I wanted the query: because I was working with denormalized data I had no control over.
SELECT
  customer_number, DISTINCT(customer_name, customer_address)
FROM unprocessed_invoices
GROUP BY customer_number

Hopefully this helps clarify the original intent of the question.
EDIT: Examples of good/bad data
To clarify: customer_name and customer_address only have to be unique for a particular customer_number.
 customer_number | customer_name | customer_address
----------------------------------------------------
 1               | 'Bob'         | '123 Street'
 1               | 'Bob'         | '123 Street'
 2               | 'Bob'         | '123 Street'
 2               | 'Bob'         | '123 Street'
 3               | 'Fred'        | '456 Avenue'
 3               | 'Fred'        | '789 Crescent'

The first two rows are fine because it is the same customer_name and customer_address for customer_number 1.
The middle two rows are fine because it is the same customer_name and customer_address for customer_number 2 (even though another customer_number has the same customer_name and customer_address).
The last two rows are not okay because there are two different customer_addresses for customer_number 3.
The query I am looking for would fail if run against all six of these rows. However, if only the first four rows actually existed, the view should return:
 customer_number | customer_name | customer_address
----------------------------------------------------
 1               | 'Bob'         | '123 Street'
 2               | 'Bob'         | '123 Street'

I hope this clarifies what I meant by "conflicting customer_name and customer_address". They have to be unique per customer_number.
I appreciate those that are explaining how to properly import data from external systems. In fact, I am already doing most of that already. I purposely hid all the details of what I'm doing so that it would be easier to focus on the question at hand. This query is not meant to be the only form of validation. I just thought it would make a nice finishing touch (a last defense, so to speak). This question was simply designed to investigate just what was possible with SQL. :)
Comment: What do you mean by "fail, rather than returning multiple rows"?
Usually when I think of a sql query failing, that mean I got no rows, or a product of tables that I'm joining.  I though you were looking for a
select * from (select count(*) as cnt, customer_number, customer_name, customer_address From customers group by customer_number, customer_name, customer_address) where cnt > 1
type of query.
Comment: By "fail", I meant the DBMS should return an error rather than results (like when you query a non-existent table). I understand that I can use the "SELECT COUNT(*) ... GROUP BY ... HAVING ..." query to identify the presence of conflicting data, however I was simply curious to see if a query similar to the one I described actually existed. Thanks for the comment, though! :)
Answer: Your approach is flawed. You do not want data that was successfully able to be stored to then throw an error on a select - that is a land mine waiting to happen and means you never know when a select could fail.
What I recommend is that you add a unique key to the table, and slowly start modifying your application to use this key rather than relying on any combination of meaningful data.
You can then stop caring about duplicate data, which is not really duplicate in the first place. It is entirely possible for two people with the same name to share the same address.
You will also gain performance improvements from this approach.
As an aside, I highly recommend you normalize your data, that is break up the name into FirstName and LastName (optionally MiddleName too), and break up the address field into separate fields for each component (Address1, Address2, City, State, Country, Zip, or whatever)
Update: If I understand your situation correctly (which I am not sure I do), you want to prevent duplicate combinations of name and address from ever occurring in the table (even though that is a possible occurrence in real life). This is best done by a unique constraint or index on these two fields to prevent the data from being inserted. That is, catch the error before you insert it. That will tell you the import file or your resulting app logic is bad and you can choose to take the appropriate measures then.
I still maintain that throwing the error when you query is too late in the game to do anything about it.
Comment: +1 any serious data table should definitely have a primary key to uniquely identify every single data row - database design 101!
Comment: You both are absolutely correct. I was hoping the question would be answered without people questioning its design. ;) I hope my latest edit can help shed some light on the question.
Answer: A scalar sub-query must only return one row (per result set row...) so you could do something like:

select distinct
       customer_number,
       (
       select distinct
              customer_address
         from customers c2
        where c2.customer_number = c.customer_number
       ) as customer_address
  from customers c
Comment: +1. This is actually what I was originally trying. However, I was hoping for a way that would avoid having to write a subquery for each distinct column. Thanks, though!
Answer: Let's put the data into a temp table or table variable with your distinct query
select distinct customer_number, customer_name, customer_address,
  IDENTITY(int, 1,1) AS ID_Num
into #temp
from unprocessed_invoices

Personally I would add an indetity to unporcessed invoices if possible as well. I never do an import without creating a staging table that has an identity column just because it is easier to delete duplicate records.
Now let's query the table to find your problem records. I assume you would want to see what is causing the problem not just fail them.
Select t1.* from #temp t1
join #temp t2
  on t1.customer_name = t2.customer_name and t1.customer_address = t2.customer_address
where t1.customer_number <> t2.customer_number

select t1.* from #temp t1
join
(select customer_number from #temp group by customer_number having count(*) >1) t2
  on t1.customer_number = t2.customer_number

You can use a variation on these queries to delete the problem records from #temp (depends on if you choose to keep one or delete all possible problems) and then insert from #temp to your production table. You can also porvide the problem records back to whoever is providing you data to be fixed at their end.
Answer: Making the query fail may be tricky...
This will show you if there are any duplicate records in the table:
select customer_number, customer_name, customer_address
from customers
group by customer_number, customer_name, customer_address
having count(*) > 1

If you just add a unique index for all the three fields, noone can create a duplicate record in the table.
Comment: Adding to Guffa's post, try adding an if statement, check the count and use RAISEERROR if the count is > 1
Answer: The defacto key is Name+Address, so that's what you need to group by.
SELECT
  Customer_Name,
  Customer_Address,
  CASE WHEN Count(DISTINCT Customer_Number) > 1
    THEN 1/0 ELSE 0 END as LandMine
FROM Customers
GROUP BY Customer_Name, Customer_Address

If you want to do it from the point of view of a Customer_Number, then this is good too.
SELECT *,
CASE WHEN Exists((
  SELECT top 1 1
  FROM Customers c2
  WHERE c1.Customer_Number != c2.Customer_Number
    AND c1.Customer_Name = c2.Customer_Name
    AND c1.Customer_Address = c2.Customer_Address
)) THEN 1/0 ELSE 0 END as LandMine
FROM Customers c1
WHERE Customer_Number = @Number

Answer: If you want it to fail you're going to need to have an index.  If you don't want to have an index, then you can just create a temp table to do this all in.
CREATE TABLE #temp_customers
    (customer_number int,
    customer_name varchar(50),
    customer_address varchar(50),
    PRIMARY KEY (customer_number),
     UNIQUE(customr_name, customer_address))

)
INSERT INTO #temp_customers
SELECT DISTINCT customer_number, customer_name, customer_address
FROM customers

SELECT customer_number, customer_name, customer_address
FROM #temp_customers

DROP TABLE #temp_customers

This will fail if there are issues but will keep your duplicate records from causing issues.
Comment: If I have (1, Bob, 1st Street) and (2, Bob, 1st Street)... this returns both.
Comment: Sorry, I didn't think of that scenario.  I have now added a UNIQUE constraint.  Now that should work.
Answer: If you have dirty data, I would clean it up first.
Use this to find the duplicate customer records...
Select * From customers
Where customer_number in
  (Select Customer_number from customers
  Group by customer_number Having count(*) > 1)

Answer: Select t1.* from #temp t1
join #temp t2
  on t1.customer_name = t2.customer_name and t1.customer_address = t2.customer_address
where t1.customer_number <> t2.customer_number

select t1.* from #temp t1
join
(select customer_number from #temp group by customer_number having count(*) >1) t2
  on t1.customer_number = t2.customer_number

Safely normalizing data via SQL query


The correct way to copy a field's data to another field in the same table with groovy

Question: I'm writing my first groovy script to update  the whole database table. Basically I want the number is the ID field to be put into ANOTHER_ID field
 import groovy.sql.Sql

def url = project.getProperty("database.url")
def driver = project.getProperty("database.driver")
def user = project.getProperty("database.user")
def pass = project.getProperty("database.pass")
def output = properties['output']
def sql = Sql.newInstance(url, user, pass, driver)
def outwriter = new PrintWriter(output)

sql.eachRow("select * from THE_TABLE", { row ->
    def id = row.id
    outwriter.println "Update THE_TABLE set (ANOTHER_ID)=(${id});"
} );

outwriter.close()
sql.close()

I'm posting to make sure I'm doing this correctly or if there is a better way of updating my table.
Cheers
Answer: There's a problem with the SQL you're generating, it should be:
outwriter.println "UPDATE THE_TABLE SET ANOTHER_ID = ${row.id} WHERE id = ${row.id} ;"

However, you can replace this whole loop and everything with
outwriter.println "UPDATE THE_TABLE SET ANOTHER_ID = id ;"

Which will set every row in a single query instead of needing an UPDATE per row


No response from url when requesting from javascript

Question: I am trying to use ipinfo api in my django app.
my js code -
var button = document.getElementById('login_button');
var user_location = document.getElementById('user_location');
button.onclick=function()
{
    get_location();
    form.submit();
}
function get_location()
{
    alert('inside');
    $.get('http://ipinfo.io', function(response)
        {
            alert('response');
            user_location.value = response.loc;
            console.log(response.loc);
            alert(response.loc);
        }
    ,'jsonp');
}

my html code(relevant)-

            <form id="form" method="post" class="form">
                {% csrf_token %}
                <input type="hidden" id="user_location" name="user_location"/>
            </form>

            <button type="button" id="login_button" name="confirm_login" class="tooltip-test btn btn-sm btn-success" title="Click to Log In">
                Login
            </button>

and my views.py relevant code -
 o = online_status(username = u.username, location = post['user_location'])
                devices_no = 1
                o.save()

My problem starts with the js file. It is alerting 'inside' but niether it alerts 'response' nor there is any thing in my browser console and also there is no alert of response.loc.
Where I am wrong..??
Help me please.
Note: My internet is behind a proxy.
Thanks in advance.
Comment: That site won't let you access its content from your page. The browser will prevent such attempts.
Comment: And why is that..I followed the syntax given on the site itself...
Comment: If you check the response headers from loading that URL, you won't see any "Access-Control-Allow-Origin" header. Without that, the browser won't let you see it from a different domain.
Comment: http://ipinfo.io supports CORS, so that's unlikely to be the issue. You might be hitting a rate limit though. Try alerting or logging response, rather than response.loc
Comment: I don't think rate limit is an issue in this case...and about response, the function upon success is not running. There is no alert 'response'. Any other idea??
Comment: Why don't you think a rate limit is the issue? If you're calling this from localhost it almost certainly will be (lots of people call the API from localhost!). You should see an 429 error in the console.
Answer: Most likely, you are hitting the "cross-origin problem".  See here for more discussion.
Comment: Does not Chromium show this error ?? because I am using it but there is no cors-error in the console.
Comment: A lot of browsers are distressingly silent on this issue.  If you can open a URL from the Location bar but not from AJAX, it is almost certainly a cross-origin problem.
Comment: Even using jsonp this isn't resolved, any other suggestions please.
Comment: It's working for me.  You don't see any errors in the console?
Comment: Do you think its because my internet is behind proxy??
Comment: No, if you can get to it from the location bar, you can get to it from Ajax.


Android submission: I get a "CSRF validation failed" error while using a cookie

Question: The Drupal Version is 7 and the Services version is 3.4.
So, My login into Drupal goes like this:

Fetch a token. (POST)
Fetch a Cookie using the token. (POST)
Make GET Requests using DefaultHttpClient class to download data.
(GET). This uses a token and cookie

This all works fine. But when I try to post my submissions to drupal server, look at the code below
<code> DefaultHttpClient httpClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost(submissionURL);
  SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(SurveyReviewActivity.this);
        String id = prefs.getString("sessid", "FAIL");
        Log.e("SRA", id);
        String name = prefs.getString("sessname", "FAIL");
         Log.e("SRA", name);
         //String token_test = prefs.getString("token", "FAIL");
        //Log.e("SRA", token_test);

          httpPost.setHeader("Content-Type", "application/x-www-form-urlencoded");
         httpPost.setHeader("X-CSRF-Token", X_CSRF_Token);
         // httpPost.setHeader("X-CSRF-Token", token_test);
         httpPost.setHeader("Cookie",name+"="+id);  try {
             httpPost.setEntity(new StringEntity(message, "UTF8"));
         } catch (UnsupportedEncodingException e) {
             Log.d("Encoding Exception",e.toString());

        }
          try {
            HttpResponse httpResponse = httpClient.execute(httpPost);
             if(httpResponse.getStatusLine() != null && httpResponse.getStatusLine().getStatusCode() == 401) {
                 Log.d("Submission failed ", httpResponse.getStatusLine().toString());

             }

            HttpEntity httpEntity = httpResponse.getEntity();
            is = httpEntity.getContent();
            try {
                BufferedReader reader = new BufferedReader(new InputStreamReader(
                        is, "iso-8859-1"), 8);
                StringBuilder sb = new StringBuilder();
                String line = null;
                while ((line = reader.readLine()) != null) {
                    sb.append(line + "\n");
                }
                is.close();
                String json = sb.toString();
                Log.d("Response", json);
            } catch (Exception e) {
                Log.e("Buffer Error", "Error converting result " + e.toString());
            }
        } catch (ClientProtocolException e) {

            Log.d("Protocol Exception",e.toString());
        } catch (IOException e) {

            Log.d("Protocol Exception",e.toString());
        }
</code>
When I set the last header (the cookie header), I get a token invalid or CSRF Validation Failed Error to be specific. If I don not use a token the submission goes through, but submissions are shown as anonymous on the drupal CMS side. I thought that my shared preferences are crapping up my stored cookie, but they are not thats why I was verifying them by printing them to logcat. I can get a new cookie and even tried that, but its still the same error. Whats going on here? Here are my references to the previous posts I checked on various sites.
...so if you plan on giving a solution which says your shared preferences are giving you the wrong cookie back or your token is coming down with a new line....tried that all...not working. But its working great on the iOS side, the exact same logic and actions.Please also note that, my cookie was working in fetching the data (bolded that part on step 3), but not while uploading data, its giving me a CSRF validation failed or token failed error . Please help me out on this issue.....and yes for testing the current cookie lifetime is 2 years..
Comment: I think you're doing that slightly the wrong way round...you need to get the cookie before fetching the token, and pass it along in the request. There's a working php implementation [here](http://drupal.stackexchange.com/questions/79497/new-user-registration-sequence-for-services-3-4/80004#80004) that describes the steps, should be easy enough to convert to java
Comment: but when i get a cookie, i parse it from json....I have to manually set a cookie unlike the browser....so i think it wont depend on the order...because the dowloads are working fine...only the uploads are giving me this trouble...thoughts?
Comment: Hi, I have your same problem. I tried with HttpPost and HttpURLConnection, but I have solved nothing. Have you? If yes, could you kindly help me?
Answer: CSRF token error comes when user is allready login to drupal site just logout then send the request you wont give this error again..hope this will help.
Answer: For what it's worth, this is only possible when using a HttpURLConnection in Android.
A code sample in this regard can be found at How to automate login a website – Java example.
<code>    URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();

// Acts like a browser
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "accounts.google.com");
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept",
    "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
for (String cookie : this.cookies) {
    conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Referer", "https://accounts.google.com/ServiceLoginAuth");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));

conn.setDoOutput(true);
conn.setDoInput(true);
</code>


How is splitting done on numerical predictors in randomForest package in R?

Question: I understand that for the individual trees, a least squares measure is used to measure node impurity, given candidate splits of the data at that split, and the best split is selected.
What I don't understand yet (since I couldn't find an answer in the documentation) is how candidate splits are found in the first place, i.e., given numerical predictors (not nominal or ordinal), how are the split points found for those numerical predictors in the randomForest package?
Aside: I am also wondering whether ordinal predictors and dependent variables are supported in randomForest now?
Answer: It is the same as with ordinal variables -- the algorithm goes from the minimal to a maximal value present in the attributes as a candidate for a threshold and selects best. This can be elegantly speed-up to linear complexity using presorting.
Because of that randomForest simply converts ordered factors to numerical values for predictors and to categorical data in case of decision.


Expected number of turns

Question: A person has $n$ keys and one lock. After every failed attempt to unlock, the person may pick a new key and discard the old one. What is the expected number of attempts needed to open the lock?
Comment: what is the probability that it takes $k$ tries, for $1 \le k \le n$?
Comment: Welcome to MSE, it will be pretty good if you post what you've tried to get better answers. Have a nice day.
Comment: 'after every attempt the key replaced it' - what does this mean?
Comment: This question has been edited very significantly. Suraj, please confirm that it now means what you intended it to mean -- my own interpretation of your original formulation would have been a different one.
Comment: A similar question answered here- https://math.stackexchange.com/questions/246855/probability-problem-with-n-keys
Answer: The experiment is equivalent to arranging the $n$ keys in order and trying them one by one till the lock opens. Assuming that all $n!$ orderings are equally likely, the right key is equally likely to be in any of the $n$ positions, that is,
$$P\{\text{lock opens on}~i\text{-th try}\} = \frac{1}{n}, ~ i = 1,2,\ldots, n$$ and
the average number of tries is
$$\frac{1+2+\cdots + n}{n} = \frac{n(n+1)/2}{n} = \frac{n+1}{2}.$$
Answer: The probability of succeeding at the first try is obviously $\frac{1}{n}$. If that fails, there are only $n-1$ keys left so the probability of suceeding at the second try is $\frac{1}{n-1}$. This goes on until the $n$-th trial, at which point there's only one key left, and the probability of success is $\frac{1}{1}$.
Note that the probability of success at the $k$-th trial above is the conditional probability under the assumption that all previous trails failed. So to get the unconditional probability of success at the $k$-th trial, you have to multiply with the probability that the first $(k-1)$ trials failed.
Once you have worked out the probabilities of succeeding exactly at the $k$-th trial, you can compute the expected values of $k$, i.e. the average of all $k$s weighted by their respective probability.
Answer: It is marginally neater to first find the expectation of $Y$, the number of unsuccessful trials. Then our desired expectation is $E(Y)+1$.
Define the random variables $X_1,X_2,\dots,X_{n-1}$ by $X_i=1$ if all trials up to the $i$-th are unsuccessful, and $X_i=0$ otherwise. Then the number $Y$ of unsuccessful trials is $X_1+X_2+\cdots+X_{n-1}$.
By the linearity of expectation, we have $E(Y)=E(X_1)+E(X_2)+\cdots+E(X_{n-1})$.
The probability that $X_i=1$ is $\frac{n-i}{n}$. Add up, $i=1$ to $n-1$. On top, we have (backwards) the sum of the integers from $1$ to $n-1$. So the expectation of $Y$ is $\frac{n(n-1)}{2}\cdot \frac{1}{n}=\frac{n-1}{2}$.
It follows that the expected number of trials is $\frac{n+1}{2}$.

Expected number of turns