Why do we differentiate the RSS with respect to minimizers?

Question: I am having a hard time understanding simple linear regression. I got to a point, in this website to which I can see the closest answer to my question :
To minimize our error function, S, we must find where the first derivative of S is equal to 0 concerning a and b. The closer a and b are to 0, the less total error for each point is. Let's find the partial derivative of a first... and then they proceed with that calculations.
My logical point which I can not link is, I can understand that the derivative of a quadratic function gives us the slope of the tangent line at that point. I can partially understand why RSS is squared, the only reason I being that is only this mathematical reason of deriving this function afterwards. But I can not understand anything more. All I see is that I have an error function, then I square it only to differentiate it afterwards with respect to minimizers. Why?
Comment: Succinctly, we use the squared error in traditional regression, simply because the sum across all $Y_i - \hat{Y_i}=0$ (and the mathematics is a bit easier than using something like the absolute value of the difference).  If we square $Y_i - \hat{Y_i}$ and then add these up, across all $i$, the result will not be zero (it would be without the square).  Then we want to find where this function is at a minimum, as this would be minimizing the errors.  From calculus, this can be found by setting the derivative of the function equal to zero and solving for $a$ and $b$.
Comment: Do you get why we take derivatives of the error function, whatever it is, but not get why we choose to square the individual errors? That is a completely legitimate inquiry but quite different from what it appears others have addressed and what I first thought when I saw this.  // [Related?](https://stats.stackexchange.com/a/596994/247274)
Comment: I would suggest you to turn this as an answer @StatsStudent. Perhaps it could be a complement to the existing answer of mine. Of course, if you wish.
Comment: thank you for your answers. i am still trying to process the new informations before formulating a more intuitive and concise approach of asking what I don't understand
Answer: Let me sketch a brief picture of the underlying theme of linear regression:
When $\mathbf y=\mathbf X\boldsymbol\beta+\boldsymbol\varepsilon,~\mathbf y\ne \mathbf X\boldsymbol\beta^\star$ for some $\boldsymbol\beta^\star\in\mathbb R^p$ that is, $\mathbf y\notin \mathcal C(\mathbf X). $ This means the system $\mathbf y= \mathbf X\boldsymbol\beta^\star$ is not solvable.
The problem of linear regression ultimately boils down to the more intrinsic question: how to solve a non-solvable linear system?
One can resort to a useful result in functional analysis, known as Closest Point Principle, which states that

Let $U$ be a non-empty closed convex subset of a Hilbert space $\mathcal H;$ let $y\in U^\complement.$ Then there exists a unique $\hat a\in A$ such that $$\Vert y-\hat a\Vert=\inf\{\Vert y-a\Vert:a\in A\}.$$

Simplifying in our perspective, the result assures us that we can find $\hat{\mathbf y}\in\mathcal C(\mathbf X) $ such that it has the closest square distance to $\mathbf y\notin \mathcal C(\mathbf X)$ i.e. $\Vert \mathbf y-\hat{\mathbf y}\Vert^2\leq \Vert\mathbf y-\mathbf y^\star\Vert^2~\forall ~\mathbf y^\star\in \mathcal C(\mathbf X). \tag 1\label 1$ And one can find that $\hat{\mathbf y}=\mathbf P_\mathbf X\mathbf y, $ where $\mathbf P_\mathbf X:=\mathbf X(\mathbf X^\top\mathbf X) ^-\mathbf X^\top$ is the projection operator onto $\mathcal C(\mathbf X)$ and thus $\hat{\mathbf y}=\mathbf X\underbrace{(\mathbf X^\top\mathbf X) ^-\mathbf X^\top\mathbf y}_{:=\hat{\boldsymbol\beta}}.$ When the norm is $\Vert \cdot\Vert_2, ~\eqref 1$ becomes
$$\sum_i (\mathbf y-\mathbf X\hat{\boldsymbol\beta})^2_i\leq \sum_i(\mathbf y-\mathbf X\boldsymbol\beta^\star)^2_i~\forall~\boldsymbol\beta^\star\in\mathbb R^p.\tag 2$$
So far, so good. But what is the working principle? The same old calculus to the rescue:
$$\frac{\partial}{\partial\boldsymbol\beta^\star}\Vert \mathbf y-\mathbf X\boldsymbol\beta^\star\Vert^2=2\mathbf X^\top\mathbf X\boldsymbol\beta^\star-2\mathbf X^\top\mathbf y=\mathbf 0.\tag 3$$
Meanwhile there have been handful of discussions on or around this topic here, which you can look into. One of the more general and expository posts to start with would be this CV post: Why do we usually choose to minimize the sum of square errors (SSE) when fitting a model? and links therein.

References:
$\rm [I]$ Linear Regression, Jürgen Groß, Springer-Varleg, $2003, $ sec. $2.2.1, $ pp. $37-39.$
$\rm [II]$ An Introduction to Functional Analysis, James C. Robinson, Cambridge University Press, $2020, $ sec. $10.1, $ pp. $126-127.$
Comment: While this is a correct answer, I'm guessing an answer involving Hilbert spaces and your notation will not prove to be very useful to OP since it seems she is struggling with basic concepts in regression.  If the OP isn't quite grasping these concepts, I'm guessing an explanation involving functional analysis, Hilbert spaces, and more complex notation will not be useful.  I'd suggest modifying your response to be more aligned with the experience/educational level of the OP.
Comment: As for complex notation, I have to say I have always embraced usage of the conventional notations and symbols most books have adapted into.
Comment: "under the hood."  But that's just my perspective, and may not be that of others.  Of course, feel free to do as you wish, but I do think a brief plain English explanation **would** go a long way for the OP, and would be more likely to earn you a checkmark.
Comment: Thanks for the feedback. The principle has been stated for completion. How that works from the pov of OP has also been stated subsequently. If one wants to present the fundamental underlying principle, one can not shed all the mathematical jargons, imo. Still if you wish, I would water down the already what I deem the simplest way I could have provided a crux of.
Comment: I probably didn't convey that well -- I didn't mean to imply that the notation isn't conventional.  I simply meant to convey that the notation is probably difficult for the OP to understand given she seems to be struggling with some basic concepts in mathematics/statistics/calculus.  I'm assuming that she's likely not to benefit greatly  from what the OP would probably consider a more heavy notational explanation and a  rigorous mathematical explanation.  Instead, I'd argue that a plain English explanation might be more suited to allow the OP to comprehend what's really going on... (con't)
Comment: Sure @StatsStudent. Currently I would go off for few hours. Meanwhile if someone adds any other answer, then that's welcome. Otherwise I would edit mine (I need to think how to make it _wordy_ and articulate the same I want to).
Comment: as i said earlier,
thank you for your answers. i am still trying to process the new informations before formulating a more intuitive and concise approach of asking what I don't understand
Comment: Hello @Perja. I would be happy to help further but you need to pinpoint your problem first. What exactly is bothering you? David linked a post, I provided you a gist of the underlying theory and also a link to a general discussion. After going through all these, please then specify what you need to know more.


Validate a file extension using getMimeType

Question: I want to create a hook that will validate a file upload and only allow jpg extension or a multiple array jpg, gif. I am not sure how to do this. If this  below can validate the length of the file title.
if (strlen($file
 ->getFilename()) > 100) {
    $errors[] = t("The file's name exceeds the 100 characters limit.");
 }

Then can something simular to this work?
use Drupal\Core\Form\FormStateInterface;
 function blank_moda_file_validate(Drupal\file\FileInterface $file) {
   $errors = array();

if ($file
 ->getMimeType()) !== jpg, gif) {
   $errors[] = t("The file is not a jpg or gif.");
}

Answer: You can use file_validate_extensions() for this
Comment: I saw the example for  hook_validate_extensions() inside: /core/modules/file/file.module, line 336. And here:https://api.drupal.org/api/drupal/core%21modules%21file%21file.module/function/file_validate_extensions/8.3.x  But I do not know how to alter the function to specifically allow only jpg and gif.
Comment: The second argument to file_validate_extensions() is an array of file extensions that are allowed.


navigate from ListView item to fragment

Question: I am trying to navigate from my ListView Item to a fragment. Fragment and ListView are both part of a TabsAdapter.
I ve tried this:
    @Override
public void onListItemClick(ListView l, View v, int position, long id)
{

    // Set up different intents based on the item clicked:
    switch (position)
    {
    case 0:
        Intent newActivity = new Intent(l.getContext(), Fragment_2.class);
        startActivityForResult(newActivity, 0);
        break;

    }

}

and this
    case 0:
    Intent newActivity = new Intent(getActivity(), Fragment_2.class);
    startActivityForResult(newActivity, 0);
    break;

Both result in a shutdown of my app in the emulator after choosing the item on position 0.
Thank you,
Christian
Comment: Fragment is not an Activity. For displaying frgament yuo should use FrgmentManager
Comment: You should use a Fragment manager example: `getFragmentManager().beginTransaction().replace(R.id.container, newFragment).addToBackStack(null).commit();` Is your list view part of a fragment or an Activity?
Comment: you can't startactivity on a fragment.
Comment: My listview is part of a fragment, I also asumed that my goal-fragment is not to handle as a activity but I also received the same error aiming at the main activity...
Answer: If you want to make your new fragment appear like your other fragment I recommend this method :
Fragment newFragment = new MyNewFragmentClass();
FragmentTransaction transaction = getFragmentManager().beginTransaction();

// Replace whatever is in the fragment_container view with this fragment,
// and add the transaction to the back stack
transaction.replace(R.id.my_current_layout, newFragment);
transaction.setTransition(FragmentTransaction.TRANSIT_FRAGMENT_FADE);
transaction.addToBackStack(null);

// Commit the transaction
transaction.commit();

I had the same issue myself : New fragment in actionBar
Answer: As the list view is also a part of your fragment, you can do in following way:
1. Listen to Item Click Listener of List in Fragment (as I think, you have already done) . But instead of simply creating an Activity, pass an Bundle message to pass the Tab position.
@Override
public void onItemClick(AdapterView<?> arg0, View arg1, int position, long arg3) {
     switch (position) {
         case 0:
           Intent newActivityIntent = new Intent(getActivity(), Fragment_2.class);
           newActivityIntent.setFlags(Intent.FLAG_ACTIVITY_NO_HISTORY);
           newActivityIntent.putExtra("tabToStart", "2"); //Pass the desired tab no.
           startActivity(fragmentIntent);
           break;
      }
}

2. In  Oncreate() method of the Parent Activity (which holds the Fragment 1 and Fragment2) , you have to get the bundle message using getIntent(), and change the Tab position accordingly.
Intent intLoc = getIntent();
String loc = intLoc.getStringExtra("tabToStart");
if (loc.equals("2")) {
    mViewPager.setCurrentItem(2);
}

Kindly let me know if further details are required.

navigate from ListView item to fragment


Change GarageBand key pitch/scale

Question: I am very new to GarageBand and music in general, so please excuse me if this is a noob question. I can display the virtual keyboard and play it. There is also an option to do pitch bend the keyboard, but if I press it, the pitch bend value can just be -20 or +20.
Say for example I am playing the C major chord on the keyboard, can I change the pitch/scale of the digital keyboard to changed to F major chord? I can change the key signature after I have recorded. But can I change while playing the keyboard?
Answer: You are able to change the pitch and scale real time. It's possible when you change the key signature it may effect any other MIDI instruments. Such as  your drum tracks if created on a MIDI keyboard, it may throw them off. Or possibly not.
If you are new to Garage Band, you could turn on the "Quick Help Button", that way you could hover your cursor over whatever is in GarageBand, and get contextual help right there.
If you are using Garage Band 10 or above, there is a 'Musical Typing' feature to assist you in real time changes.
Example: Pitch Bend- you could use the 1+2 keys to adjust the pitch of whatever notes you are playing in real time.
Octave Use the Z and X keys to shift octaves for lower or higher pitched notes. HTH. ~catz
Answer: In the Inspector window (shortcut key: i) under MIDI defaults enter "5" in the transpose parameter.

Change GarageBand key pitch/scale


Draggable  content using jQuery

Question: I would like to use jQuery and its drag-and-drop function, such as:
http://jqueryui.com/demos/draggable/
In my case, I would like to enable drag-and-droping on  content of an HTML table.  For instance with a table of 20x20, in order to move a  tag from a cell to an other cell.
Is that hard to do? I don't see any example after searches.  If you're got one or if you can show me how to process, I will be cool!
Comment: Have you tried implementing the examples from the link you posted? It's right on there! Give it a shot before posting here, man! Maybe you didn't see this part, the [Droppable](http://jqueryui.com/demos/droppable/)
Comment: Sorry Jason, but it's not as simple as you could think.  jQuery drag-and-drop don't really work for table content.  Even if it's possible to drag-and-drop TD content from a cell to an other cell, this moved content is not center in its new cell.  And after some tests it's seems to be much harder then you said.  But it's fun (and it's not a homework ;)
Comment: have you tried putting your `` content into a div? not get rid of your table, but do `Content` and use the `` s as your draggables?
Comment: Yes (div, span, ...) but it steal not work, and I think I will begin to search an other solution in order to finish my project.  Thanks anyway.
Comment: It should work if you do it right. It doesn't care what container you put it in. Your selectors are probably incorrect...
Answer: Try using jQuery.ui.sortable.  That should allow for drag and drop within the table.
Answer: I think you can use a div inside the .so that yo can drag the div and drop it where you want.

Draggable  content using jQuery


Stationary points of general function

Question: When studying the stationary point(s) of the following
$$
Q=\frac{K(x)}{x}
$$
I find
$$
\frac{dQ}{dx}=\frac{x\frac{dK(x)}{dx}-K(x)}{x^2}=0
$$
and hence
$$
\frac{K(x)}{x}=\frac{dK(x)}{dx}
$$
I'm sure this is elementary, but I'm looking for an intuitive explanation why the stationary point occurs when the original function is equal to the differential of the numerator. Thanks for your help.
Answer: A slight recasting gives
$$x\cdot Q(x)=K(x).$$
Differentiating this gives another way of reaching the same conclusion as you without much more illumination.
Instead let us go softly and induce a change at a stationary point $x_0\rightarrow x_0+\Delta x$. If $Q$ is stationary then $Q_0\rightarrow Q_0+\Delta Q\approx Q_0$. Finally $K_0\rightarrow K_0+\Delta K$.
Now we have
$$\begin{align}(x_0+\Delta x)\cdot Q_0&=K_0+\Delta K
\\\Rightarrow x_0Q_0+\Delta x\cdot Q_0&=K_0+\Delta K
\\ \underset{x_0Q_0=K_0}{\Rightarrow} \Delta K&=Q_0\cdot\Delta x,
\end{align}$$
and so if $Q$ is stationary at $x_0$ then near $x_0$, $\Delta K$ is proportional to $\Delta x$, and we have for points near $x_0$
$$Q(x)=\frac{K(x)}{x}\approx \frac{K_0+Q_0\Delta x}{x_0+\Delta x}=\frac{K_0}{x_0}=Q_0,$$
where we appealed to the lemma
$$\frac{a+x}{b+y}=\frac{a}{b}\Leftrightarrow \frac{a}{b}=\frac{x}{y}.$$
On the other hand (in hindsight), we could say that for $Q$ to be stationary near $x_0$ we need $K(x)$ to grow in proportion to $x$ so $K(x)\approx mx$:
$$Q(x)=\frac{K(x)}{x}\approx \frac{mx}{x}=m\approx \frac{dK}{dx},$$
and for stationarity we need $m=Q_0$.
Comment: Thanks, that's a very clear explanation.
Answer: I think a very intuitive, but not rigorous explanation could be that the function (or better the limit of the function for $x\rightarrow 0$ is a (limit) of a difference quotient of the function $K$.

Stationary points of general function


How do I programmatically add fields to Paragraph types?

Question: I have one paragraph type of Product. and I have to add one field to paragraph product programmatically, how can I achive this.
function my_module_field_widget_paragraphs_form_alter(&$element, FormStateInterface &$form_state, $context){
  if ($element['#paragraph_type'] == 'products') {
    \Drupal::logger('my_module')->info('debug');

  }
}

and other answer I found is
use Drupal\paragraphs\Entity\Paragraph;

  $paragraph = Paragraph::load($paragraph_id);
  $paragraph->field_image[] = [
    'target_id' => $fid,
    'alt' => $alt,
    'title' => $title,
  ];
  $paragraph->save();

But in which function or preprocess I need to add above code?
Comment: check this out https://www.flocondetoile.fr/blog/automatically-provide-options-list-field-type-drupal-8
Comment: Normally you go to the UI edit a paragraph type, add fields, save, export config, done. What are you trying to do here? And why don't you use the UI and export config as usual?
Comment: @leymannx I need to create one dropdown field in paragraph and options of that dropdown will be come from config file.
Comment: Then add the field from the UI and define an allowed values callback in a custom module.
Comment: @leymannx can you share link
Answer: We cannot create a field to existing paragraph type using hook_field_widget_paragraphs_form_alter(), because it will not create the field.  So the best way to add a custom field is using hook_entity_base_field_info()
In my case I need to create a text field in all my existing paragraph types and into the new paragraph which will be created in future so I used below code.
function hook_entity_base_field_info(EntityTypeInterface $entity_type) {
  if ($entity_type->id() === 'paragraph') {
    $fields['field_parent_section_class'] = BaseFieldDefinition::create('string')
      ->setLabel(t('Parent section class'))
      ->setDescription(t('Class for the parent section element( Add more than one class separated by space )'))
      ->setTranslatable(TRUE)
      ->setComputed(TRUE)
      ->setDisplayConfigurable('form', array(
        'settings' => array (
        'display_label' => TRUE
      )))
      ->setDisplayConfigurable('view', TRUE)
      ->setSettings(array(
        'default_value' => '',
        'max_length' => 255,
        'persist_with_no_fields' => TRUE
      ));
    return $fields;
  }
}


Choice for priors for exponential spatial covariance

Question: I doing spatial bayesian data analysis, I am assuming a no-nugget exponential covariance. I have tried a variety of priors for the sills and range parameters (gamma, inverse gamma etc.) , unfortunately the convergence diagonstics are typically horrible.
I am wondering how to figure out the poor mixing I observe, is there something I can do to make the MCMC chain behave better?
Answer: Diggle and Ribeiro discuss this in their book ("Model-based Geostatistics"): see section 5.4.2.  They quote some research suggesting that re-parameterization might help a little.  For an exponential model (a Matern model with kappa = 1/2) this research suggests using the equivalent of log(sill/range) and log(range).  Diggle and Ribeiro themselves recommend a profile likelihood method to investigate the log-likelihood surface.  Their software is implemented in the R package geoRglm.
Have you looked at an experimental variogram to check that a zero nugget and an exponential shape are appropriate?
Comment: Thanks. I fitted a no nugget model to the likelihood using proc mixed, I guess I should compare the AICC with that of a model with nugget.
Comment: I would agree with whuber though that the visualization of the variogram cloud would be really helpful in determining if you choice of model seems appropriate.


Comparing two histograms

Question: I would like to know what are the  common techniques to compare two histograms?
I have histogram of two images and I want to see are they similar or not meaning that is there any correlation between them or not. The histograms are for two different parts of tissue.
Comment: I think this answer applies to your question as well: http://stats.stackexchange.com/questions/99777/calculating-the-variance-of-the-histogram-of-a-grayscale-image/99929#99929
Answer: Some common measures of histogram discrepancy or agreement include
Earth mover's distance
Bhattacharyya distance
Chi-square distance, $d(x,y) = \frac{1}{2}\sum_i \frac{(x_i-y_i)^2}{x_i+y_i}$
Kullback-Leibler divergence
There are many others.
Comment: Very good point indeed. These can be used to quantify the similarity (if you need numerical ''answer'' to that question).
Comment: @Tamas Once you have a statistic/measure that gives an ordering (*histogram B is more like histogram A than histogram C is*), you can have a test of equality based off it (e.g. a permutation/randomization test).
Answer: If you understand a graphical technique under ''comparison'' you should probably try a QQ-plot (qqplot under R).
If you are thinking of an analytical way (i.e. statistical test), the two-sample Kolmogorov-Smirnov test is the most classical way (ks.test under R). There are other, more modern goodness-of-fit tests (this is how your task called statistically, this case in a two-sample sense) available, such as Anderson-Darling test (ad.test from the package kSamples).
You might also consider binning the data (with cut) and then performing a $\chi^2$-squared test (chisq.test).
(If you don't want to compare the whole distribution, just some characteristics, such as the mean or variance, you get back to elementary statistical tests. Also note that the above was a non-parametrical approach; if you're willing to assume some functional form for the distributions, you can also use classical tests.)
Note that for very large sample sizes (which might be the case judging from your histogram) you will likely obtain significant results from every test. (They'll have extremely high power, detecting even minuscule deviations from the null.) In that case, graphical techniques (QQ-plot) might be still useful to decide how large is the deviation.
Comment: I have one question should I plot qqplot for horizontal value or vertical value meaning x value or y value ?
Comment: The two parameter of `qqplot` is the two sample you have, i.e. a vector of the values (ranging from 0 to 1 if I see your histogram correctly).

Whether you use `qqplot(x,y)` or `qqplot(y,x)` doesn't really matter, the two plot will be mirrored.
Comment: I thought qqplot(x1,x2) means x1 is from one sample and x2 is from the other and not x and y of the same histogram.is that right?
Comment: @user59419 : Yes, x1 is the data behind the red histogram on the image you posted, x2 is the data behind the blue histogram (or the other way around).
Answer: you may use Hassanat distance, http://arxiv.org/ftp/arxiv/papers/1409/1409.0923.pdf
it is invariant to the scale of the histograms, i.e. if you have 2 histograms came from different image sizes, the final distance is not affected much, also it is not affected by outliers, as the distance of each bin is bounded between [0,1].
Dhi(Ai,Bi)=1-(1+min(Ai,Bi))/(1+max(Ai,Bi))
Dh=sum(Dhi)

Comparing two histograms


Orthogonal decomposition in a Hilbert Space

Question: I'm trying to prove this:
Let $X,Y$ Hilbert spaces and $A\in L(X,Y)$ such that $R(A)=Y$. Also $V=\ker(A)$. Given the operator of orthogonal projection $P:X\rightarrow V$ we define $ B: Y \rightarrow V^\perp$, such that  $B(y)=x-P(x)$ for all $y\in Y$ with $x\in X$ where $A(x)=y$. Prove that $B$ is surjective and bounded..
I have proved that the Operator $B$ is well defined, linear and injective, but i can't prove that $B$ is surjective
Thanks :)
Comment: You haven't defined $P$.
Comment: $P : X \rightarrow N(A)$ , is the operator of orthogonal projection :)
Comment: I think the codomain of $B$ should be $V^\perp$, otherwise it cannot be surjective.
Comment: Yes !!! I had been wrong, sorry :D
Answer: That $B: Y \to V^\perp$ is surjective just means that for every $z\in V^\perp$, there exists $y\in Y$ such that $B(y)=x$.  To show that such a value of $y$ exists, just let $y = A(x).$ Then $B(y) = x - P(x).$ If we can show $P(x)=0$, then that says $B(y)=x$ and we're done.  Since $x\in V^\perp,$ we have $P(x)=0$ by the definitions of orthogonal projection and of $V\mapsto V^\perp.$


Find the Moment of Inertia (Need Help With a Single Step)

Question: Find the moment of inertia around the z axis of the region bounded by x=0, y=0, z=0, x+y+z = 1.
My attempt at a solution:
I believe that this should require a triple integral of the form $\int_0^a\int_0^{y(x)}\int_0^{z(x,y)}z^2dzdydx$.
I can get the upper limit of z by solving as z = 1-x-y, giving the integral $\int_0^{1-x-y}dz$. However, for the life of me, I can't figure out how to get y in terms of x, as I've already used up my only equation which involves all 3 variables, or even two of them. What's my next step here? I can solve the final integral, I just need a hand setting it up.
Answer: First off, you need to integrate the squared distance of any point in the region to the $z$ axis, which is given by $(x^2+y^2)$. Next, for any fixed $z$ coordinate, note that the cross-section of the region is a right-angled triangle bounded by $x=0$, $y=0$ and $x+y=1-z$. Finally fix a $y$ too, then $x$ varies between $0$ and $1-y-z$, which results in the integral
$$
\int_0^1 \int_0^{1-z} \int_0^{1-y-z} (x^2+y^2)dx dy dz
$$
which is easy to solve.
Comment: Thank you for the reply. I'm a bit confused though: how does the fact that $x$ varies between $0$ and $1-y-z$ result in $y$ varying from $0$ to $1-z$?
Comment: The way you should interpret this integral is starting from the outer one. You first pick $z$ to be a free variable, which varies between 0 and 1, then determine the range of $y$ based on an arbitrary $z$ within this range, and then determine the range of $x$ based on an arbitrary pair $(y,z)$ within their respective ranges.


Help with speeding up dictionary sort loop in Python?

Question: I have a really ugly loop in my code which is really slowing down my program. The loop basically performs a dictionary comparison where, if a specific key in dict_A is the same as in dict_B, then for all matches a sort is performed which is written to a file.
for k, v in A_dict.items():
    for i, value in B_dict.items():
        if k == value[0]:
            sorted_B = [list(value) for key, value in groupby(sorted(B_dict.values()), key=itemgetter(1,2))]
            outfile.write('{0}\t{1}\t{2}\t{3}\t{4}\t{5}\n'.format (i, k, v, value[1], value[2], value[3])

Unfortunately, the dictionaries both contain over a million items. Other than putting this data into a database then sorting, does anyone have any suggestions on how to speed up this loop? Thanks for the help.
Comment: Your code compares a key and a value, but your text describes comparing only values. So what is a "match"? Values are the same? Also, when you use the `items()` method a whole new list is created. You can just fetch directly from B_dict using the key from A_dict.
Comment: @Keith Thanks for the catch. Hopefully my question is clear now - I am matching a key from one dictionary (dict_A) to a value from another dictionary (dict_B)
Comment: So, what do you want to write to the file when you have a match?
Comment: When I match `k == value[0]` I am writing to file some of the other values from the dictionary item that matched.
Comment: Ok.. so why are you doing the `groupby` / sort / list comprehension at all?
Comment: You're not sorting the data that you're writing, your sorting the whole dictionary (not in place -- a copy of the dictionary's data) and saving it into a list, that you're not doing anything with.
Comment: Anyway, either explain the purpose of the sort, or try the loop given in my answer -- it should be much faster. It looks like you just need to check if `value[0]` is a key in `A_dict`, which is very fast -- I'm not overwriting `k`, since `k` isn't used in my version of the loop.
Comment: The purpose of the sort is that I need the data sorted in the output file for another application. Would it be better to use your code and then sort after writing to file?
Comment: You're saying you want the lines in the ouput file sorted? You're not doing that now. Either sort `B` before the loop, or instead of writing in the loop, save the lines in a list, sort them after the loop, then write them.
Comment: This would be much clearer with a sample of input and the expected output.
Comment: I think if we had a better picture of your overall goal and dataset we could come up with something faster.
Answer: Your example code may be inaccurate, but as written,
sorted_B = [list(value) for key, value in
                groupby(sorted(B_dict.values()), key=itemgetter(2,3))]

will be the same every time... why is it in a loop at all?
Also
for k, v in A_dict.items():
    for i, value in B_dict.items():
        if k == value[0]:
            outfile.write('{0}\t{1}\t{2}\t{3}\t{4}\t{5}\n'.format(
                i, k, v, value[1], value[2], value[3])

Looks like it could just be written as
for i, value in B_dict.items():
    k = value[0]
    if k in A_dict:
        outfile.write('{0}\t{1}\t{2}\t{3}\t{4}\t{5}\n'.format(
            i, k, A_dict[k], value[1], value[2], value[3])

Which should be faster -- it's linear time rather than quadratic time.
Comment: as always, thank you for the help. i was attempted to put sorted_B in the loop so that only the items where `if k == value[0]:` would need to be sorted.
Comment: You're sorting the whole dictionary though, not just the items that match. Is what you want to get a list of all the matching items then sort that?
Comment: I not sure I understand how you have rewritten the dictionary matching loop - in particular, `k = value[0]`.`k` is the key to the dictionary.  Won't this rebind k to value[0] from the other dictionary. Sorry if I am being unclear.
Comment: You need to better explain what you want to write to the file.
Comment: For each item in dict_B there will be always be a match from dict_A, so I will need to go through the entire dictionary I think. However, the reverse is not necessarily true - there are items in dict_A that may not be matched in dict_B.
Comment: You don't need to iterate over `A`, just over `B`. Iterating `B` for every item in `A` then comparing is `O(len(A)*len(B))` while iterating over just `B` and checking for each item in `A` is `O(len(B))` because dictionary lookups are constant time.
Comment: Ok, I now understand! I can't think you enough agf. Your solution speed up my code dramatically. I do this sort of thing quite a bit so your explanations were really helpful. I ultimately converted the dictionary to a list, then sorted, then performed the match on the sorted list.


Property of Zero element in factor ring

Question: I'm trying to learn proof for "Let R be a commutative ring with unity and let A be an ideal of R.Then R/A is an integral domain if and only if A is prime".
In first part of proof in text book I see this statement:
MY PROBLEM IS: 1:Why ab is the zero element of the ring R/A?
2:I'm confused with properties of "general" R/A ring, what is similar and not similar to the factor groups?
IMPORTANT NOTE: I have read lots of similar related topics but there wasn't any helpful or explanation about my questions.
Answer:
As a set, $R/A = \{a + A \big| \ a\in R \}$, the set of equivalence classes modulo $A$. Now, the zero of the ring $R/A$ is the equivalence class of $0_R$ ie $0_R + A$, but  $0_R + A = A$. That's why the author of this proof says that "$A$ is the zero element of the ring $R/A$.
Quotient groups and quotient rings work alike. The difference is that while you quotient groups by normal subgroups, you don't quotient rings by rings but by ideals instead.
Answer: Because $ab \in A$ we have $ab + A = 0 + A$ in $R/A$, and $0 + A$ is the zero element.
The ring $R/A$ is exactly a factor group.  The ring $R$ under addition is an abelian group called the additive group of $R$.  The ideal $A$ is a subgroup of the additive group of $R$.  Since the additive group is abelian all subgroups are normal, so the factor group $R/A$ exists.  This factor group is the additive group of the ring $R/A$, in fact the construction of the ring $R/A$ is done by defining a multiplication operation on this factor group.

Property of Zero element in factor ring


passport deserializeUser method never called

Question: I have got 200 response for login request, but 401 for any futher auth check requests, because deserializeUser never called. I dived into passport source and noticed that passport checks whether req._passport.session.user exists, and if no it doesn't call deserializeUser.
I have searched through other questions on stackoverflow, it seems i have specific case.
There is single local strategy auth type, i use Ajax request to make login requests, CORS settings configured, http://localhost:8080 - frontend, http://localhost:3000 backend)
I use bodyParse, cookieParser, express session, passport initialize and passport sessions. Express session secure:false configured as i run auth requests through http.
You can find my project here (backend package.json is good to go, so you can use it, it has no missing dependencies, as for frontend not sure), at least you can check the code there.
Backend https://github.com/rantiev/template-api
Frontend https://github.com/rantiev/template-angular
Express session configuration and CORS is here https://github.com/rantiev/template-api/blob/master/modules/appConfigure.js
var path = require('path');
var bodyParser = require('body-parser');
var session = require('express-session');
var cookieParser = require('cookie-parser');
var MongoStore = require('connect-mongo')(session);

module.exports = function (app, express, config, mongoose) {

    app.use(cookieParser());
    app.use(bodyParser.urlencoded({
        extended: true
    }));
    app.use(bodyParser.json());

    app.use(function (req, res, next) {

        // Website you wish to allow to connect
        res.setHeader('Access-Control-Allow-Origin', 'http://localhost:8080');

        // Request methods you wish to allow
        res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, PATCH, DELETE');

        // Request headers you wish to allow
        res.setHeader('Access-Control-Allow-Headers', 'X-Requested-With, X-AUTHENTICATION, X-IP, Content-Type, Origin, Accept, Cookie');

        // Set to true if you need the website to include cookies in the requests sent
        // to the API (e.g. in case you use sessions)
        res.setHeader('Access-Control-Allow-Credentials', true);

        // Pass to next layer of middleware
        next();
    });

    /*app.use(function (req, res, next) {
        console.log('coockie is:', req.cookies);
    });*/

    app.use(session({
        saveUninitialized: false,
        resave: false,
        secret: config.sessionsSecretToken,
        cookie: {
            secure: false
        },
        store: new MongoStore({ mongooseConnection: mongoose.connection })
    }));

    app.use(express.static(path.join(__dirname, '..' , 'public')));

};

Passport configuration is here https://github.com/rantiev/template-api/blob/master/api/authentication/authenticationR.js
var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var rememberMe = require('../../modules/rememberMe');
var createAccessToken = require('../../modules/createAccessToken');

var bcrypt = require('bcrypt-nodejs');

var UserM = require('../users/userM');

module.exports = function (app, mainRouter, role) {

    passport.use(new LocalStrategy({
        usernameField: 'email',
        passwordField: 'password'
    }, function (username, password, done) {

        UserM.findOneQ({email: username})
            .then(function(user){

                if (user && bcrypt.compareSync(password, user.password)) {
                    done(null, user);
                } else {
                    done(null, false);
                }

            })
            .catch(function(err){
                done(err);
            });

    }));

    passport.serializeUser(function (user, done) {

        console.log('serialize');

        if (user) {
            createAccessToken(user, done);
        } else {
            done(null, false);
        }
    });

    passport.deserializeUser(function (token, done) {

        console.log('deserialize');

        UserM.findOneQ({accessToken: token})
            .then(function(user){

                if (user) {
                    done(null, user);
                } else {
                    done(null, false);
                }

            })
            .catch(function(err){
                done(err);
            });

    });

    app.use(passport.initialize());
    app.use(passport.session());

    mainRouter.post('/me', passport.authenticate('local'), function (req, res) {
        res.status(200).send();
    });

    mainRouter.get('/logout', function (req, res) {
        req.logout();
        res.redirect('/');
    });

    mainRouter.get('/me', function (req, res) {

        if (!req.user) {
            res.status(401).send('Please Login!');
            return;
        }

        var currentUser = {
            id: req.user._id,
            role: req.user.role
        };

        res.status(200).json(currentUser);
    });

};

Comment: That's correct. But express session or passport session is never restored, and that's why it doesn't call deserialize method. I don't know why. I see that cookie set properly and it's being passed to backend in auth check request.
Comment: I'm having the same issue, have you solved this?
Comment: Nope, will answer here when i have solution. Gave it a timeout. By far i did all i can with that, it doesn't work )
Comment: I assume login request is setting the cookie which is then passed back in subsequent request and that cookie has the access token which you issued.
Comment: @Rantiev, you may want to make life easier for yourself and use the `cookie-session` middleware, instead of the `express-session` middleware, it will be less setup and less time and less headache.
Answer: If you look in your call stack and find that deserializeUser is not being called because req._passport.session.user is not set, then your problem is as follows. The offending lines are in the express-session module:
if (!req.sessionID) {
  debug('no SID sent, generating session');
  generate();
  next();
  return;
}

If sessionID is set, generate is never called:
store.generate = function(req){
 req.sessionID = generateId(req);
 req.session = new Session(req); // THIS
 req.session.cookie = new Cookie(cookieOptions);

 if (cookieOptions.secure === 'auto') {
  req.session.cookie.secure = issecure(req, trustProxy);
 }
};

But it is possible to have req.sessionID set, while req.session is null, which explains req._passport.session.user being null—req.session is never set.
I continued to trace back even further to when req.sessionID is set, which, with new cookies, was sometimes being set and sometimes not.
Why? I don't know, and would love for someone to investigate further, but, basically, the lesson is try using the cookie-session module instead.
Answer: Have you tried maxAge?
app.use(express.session({   store: sessionStore,
                            cookie: { maxAge : 3600000 } //1 Hour
                            }));

Comment: yep, i did.
i actually forgot what is the status of the problem. Whether i solved it or not, it seems i have updated all the modules past 1 year and everything worked.


How to set circle at the right end of the last bar / item using D3

Question: right now i am displaying the live data using d3 and here my requirement is how can i display a small circle at the end of last bar at the right edge and what ever the data may based on last bar i should display on top of it  below is snap shot

below is my js fiddle :- https://jsfiddle.net/madpop143/5atLwk6h/10/
below is my code
var margin = {
  top: 20,
  left: 20,
  right: 20,
  bottom: 20
};

var width = 750 - margin.left - margin.right;
var height = 750 - margin.top - margin.bottom;

var randomDate = function(start, end) {
  return new Date(start.getTime() + Math.random() * (end.getTime() - start.getTime()));
};

var data = d3.range(1000).map(function(d) {
  return {
    x: randomDate(new Date(2015, 0, 1), new Date(2015, 0, 20)),
    y: Math.random() * 10
  };
}).sort(function(a, b) {
  return a.x.getTime() - b.x.getTime();
});

var svg = d3.select('body').append('svg')
  .attr('width', width + margin.left + margin.right)
  .attr('height', height + margin.top + margin.bottom);

var g = svg.append('g')
  .attr('transform', 'translate(' + margin.left + ',' + margin.top + ')');

var x = d3.time.scale()
  .range([0, width]);
var xAxis = d3.svg.axis()
  .scale(x)
  .orient('bottom');

var y = d3.scale.linear()
  .range([height, 0]);
var yAxis = d3.svg.axis()
  .scale(y)
  .orient('left');

var line = d3.svg.area()
  .x(function(d) {
    return x(d.x);
  })
  .y1(function(d) {
    return y(d.y);
  })
  .y0(y(0))
  .interpolate("step-before");

g.append('path')
  .attr('class', 'line')
  .attr('d', line(data));

g.append('g')
  .attr('class', 'x axis')
  .attr('transform', 'translate(0,' + height + ')')
  .call(xAxis);

g.append('g')
  .attr('class', 'y axis')
  .call(yAxis);

var tick = function() {
  var minMax = d3.extent(data, function(d) {
    return d.x;
  });
  var duration = 250;
  // 1 day offset
  var offset = 24 * 60 * 60 * 1000;
  var from = minMax[0].getTime();
  // 2-days window
  var timeWindow = [from, from + 2 * offset];

  // Recompute x,y domains
  x.domain(timeWindow);
  y.domain([0, d3.max(data, function(d) {
    return d.y;
  })]);

  // Redraw the line
  g.select('.line')
    .attr('d', line(data))
    .attr('transform', null);

  // Update x axis
  g.select('.x.axis')
    .transition()
    .duration(duration)
    .ease('linear')
    .call(xAxis);

  // Update y axis
  g.select('.y.axis')
    .transition()
    .duration(duration)
    .call(yAxis);

  //console.log(x(from))
  //console.log(x(from + offset));
  //console.log(x(from + 2*offset));

  // Slide the line to the left
  g.select('.line')
    .transition()
    .duration(duration)
    .ease('linear')
    .attr('transform', 'translate(' + x(from - duration) + ',0)')
    .each('end', function() {
      tick();
    });

  // Remove first point
  data.shift();

}

tick();

Answer: I've created a new demo here.
This is what it looks like:

The first change I made was just to create the circle right before the tick function is declared:
g.append("circle")
  .attr('class', 'last-circle')
  .style("fill", "red")
  .attr("r", 10)
  .attr("cy", 0)
  .attr("cx", 0);

Then, in the tick function, right below where you recompute the new axes-domains, I created a new variable that holds only the data that will be displayed in the current frame. It filters the data by everything that has a smaller x value than the maximum value in the timeWindow:
var newdata = data.filter(function(d) {
  return d.x < timeWindow[1];
});

Then I changed the part where you redraw the line to use the newdata value so that the line won't be drawn outside of the graph's boundaries/domain:
g.select('.line')
  .attr('d', line(newdata))

Then lastly, I got the last value from the newdata and updated the position of the circle to be at the y and x value of the last value in the graph:
var last = newdata[newdata.length - 1]

g.select('.last-circle')
  .transition()
  .duration(0)
  .attr("cy", y(last.y))
  .attr("cx", x(last.x));


Template Suggestions Causing Undefined Variable

Question: I am developing a Drupal theme from scratch, and I have run into some trouble with template suggestions. Using the solutions discussed on Drupal support, I created a template.php file and included this code:
function stronger_preprocess_page(&$vars) {

  if (isset($vars['node'])) {

    $vars['theme_hook_suggestion'] = 'page__'.$vars['node']->type; //
  }
}

This has allowed me to successfully use page--nodetype.tpl.php template files, but I am also randomly seeing errors such as:

Notice: Undefined variable: node in include() (line 24 of /Applications/MAMP/htdocs/strong-local-final/sites/all/themes/stronger/page.tpl.php).

I have read elsewhere that the node variable needs to be re-declared, but I am unsure of how or where to do this. Any help or references would be greatly appreciated!
Answer: hook_preprocess_page() is run for any page, even non-node pages. If you're getting that error it means your trying to access $node in the page.tpl.php file when you're on a non-node page. As no node is available on such a page, the variable won't have been set.
Because of the delay that can happen when messages are set in template files, you might only see this message when you've left a non-node page, and navigated to a node page. So it will look like it's that particular page complaining, when in fact it was the previous one.
You should be able to fix the problem by checking the $node variable is available in your template file as well as in the preprocess function. e.g.
<?php if (isset($node)) : print $node->something; <?php endif; ?>

Comment: Thanks @Clive! Bear with me as I work to understand this. Do I need to be checking for the `$node` variable in the template.php file or the page--nodetype.tpl.php file?
Comment: From the error it looks like the page.tpl.php file is the culprit (/Applications/MAMP/htdocs/strong-local-final/sites/all/themes/stronger/page.tpl.php), but yes it's the template file you need to add the check in, not the preprocess function


Must a secure cryptographic signature reside outside of the file it refers to?

Question: I'm programming a pet project in Python, and it involves users A & B interacting over network, attempting to insure that each has a local copy of the same file from user C.
The idea is that C gives each a file that has been digitally signed. A & B trade the digital signatures they have, and check it out on their own copy. If the signature fails, then one of them has an incorrect/corrupt/modified version of the file.
The question is, therefore, can C distribute a single file that somehow includes it's own signature? Or does C need to supply the file and signature separately?
Comment: "then one of them has an incorrect/corrupt/modified version of the file." - or one of them is lying about the checksum, or both are, or C is lying, or someone is messing with messages: http://en.wikipedia.org/wiki/Byzantine_fault_tolerance
Comment: thanks for the wikipedia link. :)
Answer: The digital signature from C alone should be enough for both A and B to confirm that their file is not corrupted, without ever communicating with eachother.  If A and B did not receive a signature from C, they could each create a cryptographic hash of the file and compare the hash, but that does not require any digital signing on C's part.
If you want C to sign the file, either send the signature and the file seperately, or wrap them both in some sort of container, such as a zip file or home grown solution (e.g., the first line in the file represents the signature, the rest is the payload).
To answer your question literally, the signature doesn't have to be outside the file per se, but the part that is being signed cannot include the signature itself.
Comment: That answers my question, I believe. There's not really a clean solution, as the file format in question is already a .zip file. Creating a .zip file with a custom header probably isn't a good idea, but neither is nested .zips, so perhaps separation is the way to go. Thank you.
Comment: The zip format supports arbitrary leading and trailing data. You could simply append or prepend the file's hash to the zipfile, and it'll still be a valid zip.
Answer: If you have control over the file format, yes. Include the signature in a header before the content proper, and make the signature cover only the content section of the file, not the entire file. Something like:
<code>SIGNATURE=72ba51288199b829a4b9ca2ac911e60c
BEGIN_CONTENTS
... real file contents here ...
</code>


trying to post data into table with php & sql - getting "Notice: Undefined index:..."

Question: Here's the html:
<form action="php/buzz_data.php" method="get">

Image: <input type="text" name="buzz_img" size="30" />

Link: <input type="text" name="buzz_link" size="30" />

Description:  <textarea rows="5" cols="30" name="buzz_desc"></textarea>

Title: <input type="text" name="buzz_title" size="30" />

    <input type="submit" value="Submit" />
    <input type="submit" value="Clear" />

</form>

Here's the php:
<?php
$con = mysql_connect("localhost","***","***");
if (!$con)
  {
  die('Could not connect: ' . mysql_error());
  }

mysql_select_db("ubook247", $con);

$sql="INSERT INTO buzz_data (buzz_page_title, buzz_img, buzz_link, buzz_desc, buzz_title)
VALUES
('$_POST[buzz_page_title]','$_POST[buzz_img]','$_POST[buzz_link]','$_POST[buzz_desc]','$_POST[buzz_title]')";

if (!mysql_query($sql,$con))
  {
  die('Error: ' . mysql_error());
  }
echo "1 record added";

mysql_close($con)
?>

and here's the result:
Notice: Undefined index: buzz_img in C:\wamp\www\ubook247\php\buzz_data.php on line 12
Notice: Undefined index: buzz_link in C:\wamp\www\ubook247\php\buzz_data.php on line 12
Notice: Undefined index: buzz_desc in C:\wamp\www\ubook247\php\buzz_data.php on line 12
Notice: Undefined index: buzz_title in C:\wamp\www\ubook247\php\buzz_data.php on line 12
1 record added
The db and table are both correctly set up as a new record was created, it just didn't add the data from the inputs.
Answer: <form action="php/buzz_data.php" method="get">

You are using GET, so access the values from the $_GET[] variable, not from $_POST[]. Or you can change the method to "post".
Also, say hello to Bobby Tables!
Comment: That makes all kinds of sense. Oh, and Bobby Tables says, "Hi," and wonders when he can have his lawn mower back. Thanks, cularis.
Comment: He can have his lawn mower when I can have my Students table!
Answer: Maybe you should also try
    $sql=sprintf("INSERT INTO buzz_data (buzz_page_title, buzz_img, buzz_link, buzz_desc, buzz_title)
VALUES (%s,%s,%s,%s)",
$_POST["buzz_img"],$_POST["buzz_link"],$_POST["buzz_desc"],$_POST["buzz_title"]);

Answer: Your array indices also should be in quotes, try something like
VALUES('{$_GET['buzz_page_title']}', '{$_GET['buzz_img']}', ...
in the VALUE clause.


Can't find valid JDBC Driver (implementation class) for Hana in Websphere Liberty

Question: I'm trying to connect a Websphere Liberty v126.96.36.199 to a HANA datasource, but I'm unable to find the right driver.
Currently in my server.xml I have the following:
<xml version="1.0" encoding="UTF-8">
<server description="o01-cls-pega-static-prpc02-appsrv03">

    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>localConnector-1.0</feature>
        <feature>jdbc-4.1</feature>
        <feature>ssl-1.0</feature>
        <feature>servlet-3.1</feature>
        <feature>ejb-3.2</feature>
        <feature>ejbLite-3.2</feature>
        <feature>ejbRemote-3.2</feature>
        <feature>jndi-1.0</feature>
        <feature>jms-2.0</feature>
        <feature>jaxws-2.2</feature>
        <feature>jaxb-2.2</feature>
        <feature>restConnector-2.0</feature>
        <feature>wmqJmsClient-2.0</feature>
        <feature>jmsMdb-3.2</feature>
    </featureManager>
    <httpEndpoint id="defaultHttpEndpoint"
                  host="localhost"
                  httpPort="28740"
                  httpsPort="28741" />

    <jdbcDriver id="myHanaDriver" javax.sql.DataSource="com.ibm.ws.jdbc.jdbcDriver">
        <library>
            <fileset dir="${wlp.user.dir}/shared/resources/hana" includes="*.jar" id="HANALib"/>
        </library>
    </jdbcDriver>

    <dataSource
              type="javax.sql.DataSource"
              containerAuthDataRef="PegaDBUserSAPHANA"
              id="hanaDataSource"
              jdbcDriverRef="myHanaDriver"
              jndiName="jdbc/hanaDataSource">
    <properties serverName="localhost" port="35315"/>
    </dataSource>
    <applicationManager autoExpand="true"/>
</server>

This is the error I get:
[5/6/19 14:24:24:735 CEST] 00000038 LogService-156-com.ibm.ws.jdbc
E CWWKE0701E: FrameworkEvent ERROR Bundle:com.ibm.ws.jdbc(id=156) org.osgi.framework.ServiceException:
Exception in com.ibm.ws.resource.internal.ResourceFactoryTrackerData$1.getService()
Caused by: java.lang.RuntimeException: java.sql.SQLNonTransientException:
DSRA4000E: A valid JDBC driver implementation class was not found for the jdbcDriver jdbcDriver[myHanaDriver] using the library com.ibm.ws.classloading.sharedlibrary_587. [/opt/wlp/usr/shared/resources/hana/ngdbc.jar]
Caused by: java.sql.SQLNonTransientException:
DSRA4000E: A valid JDBC driver implementation class was not found for the jdbcDriver jdbcDriver[myHanaDriver] using the library com.ibm.ws.classloading.sharedlibrary_587. [/opt/wlp/usr/shared/resources/hana/ngdbc.jar]
Caused by: java.lang.ClassNotFoundException: com.ibm.ws.jdbc.jdbcDriver

Any one that can help me, get the right implementation class that I need?
Answer: You are correct that javax.sql.DataSource="com.ibm.ws.jdbc.jdbcDriver" is misconfigured.  This should either point to the JDBC driver's implementation of javax.sql.DataSource, for example, javax.sql.DataSource="com.sap.dbtech.jdbcext.DataSourceSapDB"
or, for many JDBC drivers, newer versions of Liberty (but not the older 18.104.22.168 level that this question asks about) are often able to infer the correct implementation class if you omit this configuration attribute altogether.
Comment: I'm using version 22.214.171.124 because a newer version isn't supported by for the application. Also if we use version 22.214.171.124 the last one I tried, the application isn't working with Chrome.
Question remains what do I need to put here: javax.sql.DataSource="......" com.sap.dbtech.jdbcext.DataSourceSapDB is not working.
Comment: for more details, check out the official doc here: https://www.ibm.com/support/knowledgecenter/en/SSEQTP_liberty/com.ibm.websphere.wlp.doc/ae/twlp_dep_configuring_ds.html
specifically, the "For a JDBC driver that is not known to Liberty" section
Comment: also, Liberty has a zero-migration policy, which means that in theory you should be able to upgrade to a newer version of Liberty, and as long as you don't modify the app or config, it should work the same way (except more Liberty bugs will be fixed)
Comment: Regarding "com.sap.dbtech.jdbcext.DataSourceSapDB is not working", what error do you get?  Is it the same error about the implementation class not being found, or something different?  If the class isn't found, you should supply additional information such as what the full JAR name for your JDBC driver and what version it is.
Answer: I have it working now, I changed this:
javax.sql.DataSource="com.ibm.ws.jdbc.jdbcDriver"
into
javax.sql.ConnectionPoolDataSource="com.sap.db.jdbcext.ConnectionPoolDataSourceSAP"
And type="javax.sql.DataSource" into this type="javax.sql.ConnectionPoolDataSource".


Why values can not be replaced with their extensionally equal values in an intensional system?

Question: Thomas Streicher states in Investigations into Intensional Type Theory(§Introduction p.5) that:

Although in Intensional constructive set theory (Intensional Type Theory) one can do most of the things one wants to do... certain theorems simply do not hold due to the lack of extensionality. A typical example is that from $t\in B a$ and $p\in Id A ab$ one is not allowed t conclude that $t\in Bb$ where $A$ is a type and $B$ is a family of types indexed over $A$.(But of course one is allowed to infer $t\in Bb$ from $t\in Ba$ and $a=b\in A$ !)

An almost similar thin is mentioned in Definition of extensional and propositional equality in Martin-Lof extensional type theory
:

The (Id-DefEq) means that extensional equality is baked into the type system: if you have a type constructor :((:)→)→ then you can use a value of type   in a context expecting   if  and  map equal inputs to equal outputs. Again this is not true in an intensional system, where   and  might be incompatible if they're syntactically different.

Why is that? Isn't it that two functions that are producing exactly same output for their inputs, equal? So why can't one be replaced with another in a context? What makes definitionally equal functions eligible to be replaced with each other, but not the extensionally equal ones?
Answer: Extensionally equal objects can be replaced by each other freely in all extensional contexts.  That is, if you say something about a function that depends only on its values, then if that statement is true about some function $f$ then it is also true about any other function $g$ that takes the same values as $f$ (i.e. is extensionally equal to it).
The point is that not all contexts are extensional.  In particular, in Streicher's example, the property "$t\in B a$" of a term $t$ is not extensional.  One might say that a term in type theory represents an "intension" or a "sense", which has an "extension" or "reference" but is not identical to it.
The situation of different algorithms for computing the same function is a good example, as Alexis mentioned.  Another manifestation in computation is that you might have two implementations of the same data structure, e.g. lists as implemented with arrays or pointers.  The two representations are extensionally identical, and hence can be substituted for each other in any "extensional" context (which in this case means any function that uses only the "interface" of lists); but some functions can also "peek beneath the hood" and use the representation, thereby no longer being extensional.
For a philosophical (though not uncontroversial) example, the Fregean notions of "sense" and "reference" may be helpful.  For instance, "the morning star" and "the evening star" are extensionally equal (both refer to the planet Venus), but there are contexts in which it doesn't make sense to replace one by the other: "the morning star is visible at dawn" wouldn't make sense as "the evening star is visible at dawn".
For an example in mathematics, recall that in dependent type theory, propositions are represented by particular types, and proofs by their inhabitants.  For instance, Fermat's Last Theorem is a type, as is $1+1=2$.  Since both of these are true, they have the same extension, namely the truth value "true".  But a proof of $1+1=2$ still doesn't count as a proof of FLT, i.e. $t \in (1+1=2)$ doesn't imply $t\in \rm FLT$.
Answer: From a computational point of view, two functions that are equal extensionally can be very different intensionally. Sorting functions are a good example: two different algorithms can produce the same ordered set from the same input, yet differ in terms of things like computational complexity and time/space tradeoffs.


Como criar dropdownlist razor em .net mvc via ajax?

Question: Gostaria de carregar dinamicamente minha dropdownlist sem submitar a pagina!
meu projeto é na arquitetura .net mvc !
 @Html.Label("Categoria do Veiculo: ")
    @Html.DropDownListFor(v => v.CodigoCategoriaVeiculo,Model.TipoModeloEquipamentosList)

gostaria que ele carregasse via ajax!
Comment: Teria como explicar melhor o que quis dizer com *"sem submitar a pagina"*?
Comment: cara na mesma pagina eu tenho um formulario que cadastra categoria. quando cadastrar a categoria eu queria que automaticamente populasse o dropdownlist. sem fazer o postback,refresh ,submit via ajax
Comment: Seria uma opção fazer isso por partial ? Se for, talvez funcione e você não precise dar refresh  na pagina porque isso vai ficar a cargo da partial.
Comment: @Hans, um HttpHelper, nada mais é que um utilitário que escreve uma string, então você pode apenas limpar o select com Javascript e adicionar novas opções.
Comment: @Erik Poderia mi mostrar um exemplo do tipo? preciso de só ver como funciona a coisa
Comment: @HansMiller dê uma olhada [neste link do dev media](http://www.devmedia.com.br/asp-net-mvc-e-ajax-atualizando-o-conteudo-de-partial-views-via-jquery/26232) que deve te ajudar ! Mas resumindo, partial pages são partes de páginas que você pode integrar a uma página principal sem a necessidade de dar refresh na pagina toda, tipo um iframe. Enfim, dê uma olhada que usa ajax também. Espero ter ajudado.
Answer: como sugerido pelo jHonathan você pode fazer uma requisição AJAX, segue abaixo os trechos importantes para se fazer isto:
Controller
[HttpPost]
public JsonResult GetSubValor(Int32 valor)
{
    return Json(this.GetSubValores(valor));
}

private List<SubValorModel> GetSubValores(Int32 valor)
{
    var subValores = new List<SubValorModel>();
    if (valor == default(Int32))
        return subValores;

    valor = valor - 1;
    var startIndex = (valor * 10) + 1;
    for (int indice = startIndex; indice < startIndex + 10; indice++)
    {
        var tmp = indice.ToString("000");
        subValores.Add(new SubValorModel {
            SubValorID = indice,
            Descricao = "Sub Valor " + tmp
        });
    }
    return subValores;
}

JavaScript
var valor = $("#Valor");
var subValor = $("#SubValor");
valor.change(function () {
    var request = { "valor": valor.val() };
    $.post("/Home/GetSubValor/", request, function(subValores) {
        subValor.empty();
        var option = $("<option>").text("Selecione um SubValor");
        subValor.append(option);
        $.each(subValores, function (indice, item) {
            option = $("<option>", { "value": item.SubValorID }).text(item.Descricao);
            subValor.append(option);
        });
    }, "json");
});

Caso precise, pode estudar o exemplo completo abaixo.
Model
using System;
using System.ComponentModel.DataAnnotations;

namespace MvcApp
{
    public class Model
    {
        public Int32 Valor { get; set; }
        public Int32 SubValor { get; set; }
    }

    public class ValorModel
    {
        public Int32 ValorID { get; set; }
        public String Descricao { get; set; }
    }

    public class SubValorModel
    {
        public Int32 SubValorID { get; set; }

        public String Descricao { get; set; }
    }
}

Controle
using System;
using System.Web.Mvc;
using System.Linq;
using System.Collections.Generic;

namespace MvcApp
{
    public class HomeController : Controller
    {
        [HttpGet]
        public ActionResult Index()
        {
            var model = new Model();
            this.CreateLists(model);
            return View(model);
        }

        [HttpPost]
        public ActionResult Index(Model model)
        {
            this.CreateLists(model);
            return View(model);
        }

        [HttpPost]
        public JsonResult GetSubValor(Int32 valor)
        {
            return Json(this.GetSubValores(valor));
        }

        private void CreateLists(Model model)
        {
            ViewBag.Valor = new SelectList(this.GetValores(), "ValorID", "Descricao", model.Valor);
            ViewBag.SubValor = new SelectList(this.GetSubValores(model.Valor), "SubValorID", "Descricao", model.SubValor);
        }

        private List<ValorModel> GetValores()
        {
            var valores = new List<ValorModel>();
            for (int indice = 1; indice <= 10; indice++)
            {
                var tmp = indice.ToString("000");
                valores.Add(new ValorModel {
                    ValorID = indice,
                    Descricao = "Valor " + tmp
                });
            }
            return valores;
        }

        private List<SubValorModel> GetSubValores(Int32 valor)
        {
            var subValores = new List<SubValorModel>();
            if (valor == default(Int32))
                return subValores;

            valor = valor - 1;
            var startIndex = (valor * 10) + 1;
            for (int indice = startIndex; indice < startIndex + 10; indice++)
            {
                var tmp = indice.ToString("000");
                subValores.Add(new SubValorModel {
                    SubValorID = indice,
                    Descricao = "Sub Valor " + tmp
                });
            }
            return subValores;
        }
    }
}

View
@model MvcApp.Model
@{
    Layout = null;
}

<!DOCTYPE html>

        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta name="viewport" content="width=device-width, initial-scale=1">

        @using (Html.BeginForm())
        {

                @Html.LabelFor(model => model.Valor)
                @Html.DropDownList("Valor", ViewBag.Valor as SelectList, "Selecione um Valor")

                @Html.LabelFor(model => model.SubValor)
                @Html.DropDownList("SubValor", ViewBag.SubValor as SelectList, "Selecione um SubValor")

            if (Model.Valor != 0) {

                Valor Enviado:
                <input type="text" readonly="readonly" disabled="disabled" value="@Model.Valor" />
                <input type="text" readonly="readonly" disabled="disabled" value="@Model.SubValor" />

            }
            <input type="submit" value="Enviar" />
        }

Answer: Basta fazer uma requisição ajax e retornar por Json por exemplo.
Veja esta pergunta: https://stackoverflow.com/questions/22928756/how-to-bind-json-data-to-dropdownlist-in-asp-net-mvc-using-jquery
Answer: Como Jhonathan sugeriu deve funcionar, porém você pode fazer assim também:
PartialView: DropDown.cshtml
@model IEnumerable<SelecListItem>

Drop Down
@Html.DropDownList("NomeCampo", Model, "Selecione um item")

Controller
public PartialViewResult GetDropDown(int id)
{
    var itens = new List<SelectListItem>();

    itens.Add(new SelectListItem() {Text = "Ajuda", Value = 1});

    return PartialView("DropDown", itens);
}

E fazer uma requisição ajax e colocar o retorno dentro de um container.


Função Focus TextBox e Verificar se está preenchido

Question: Preciso de uma função que ao clicar no textbox se ele tiver vazio fica de uma cor, e se preenchido mude a cor, mas somente quando clicar nele, como posso fazer?
Se não tiver preenchido vou chamar esta função:

function (element) {
            $(element).closest('.control-group').removeClass('success').addClass('error');

E se tiver, vou chamar esta:

function (element) {
            element.text('OK!').addClass('valid')
                .closest('.control-group').removeClass('error').addClass('success');

Edição:
Tenho este campo no modal, por exemplo:

  <asp:TextBox ID="txtHora" runat="server" class="form-control" onblur="Verifica_Hora(this);"></asp:TextBox>

Já tenho a função que funciona se eu colocar required:

$(document).ready(function() {
  $('.input-group input[required], .input-group textarea[required], .input-group select[required]').on('keyup change', function() {
    var $form = $(this).closest('form'),
      $group = $(this).closest('.input-group'),
      $addon = $group.find('.input-group-addon'),
      $icon = $addon.find('span'),
      state = false;

    if (!$group.data('validate')) {
      state = $(this).val() ? true : false;
    } else if ($group.data('validate') == "email") {
      state = /^([a-zA-Z0-9_\.\-])+\@(([a-zA-Z0-9\-])+\.)+([a-zA-Z0-9]{2,4})+$/.test($(this).val())
    } else if ($group.data('validate') == 'phone') {
      state = /^[(]{0,1}[0-9]{3}[)]{0,1}[-\s\.]{0,1}[0-9]{3}[-\s\.]{0,1}[0-9]{4}$/.test($(this).val())
    } else if ($group.data('validate') == "length") {
      state = $(this).val().length >= $group.data('length') ? true : false;
    } else if ($group.data('validate') == "number") {
      state = !isNaN(parseFloat($(this).val())) && isFinite($(this).val());
    }
    if (state) {
      $addon.removeClass('danger');
      $addon.addClass('success');
      $icon.attr('class', 'glyphicon glyphicon-ok');
    } else {
      $addon.removeClass('success');
      $addon.addClass('danger');
      $icon.attr('class', 'glyphicon glyphicon-asterisk');
    }

  });

  $('.input-group input[required], .input-group textarea[required],
.input-group select[required]').trigger('change');

});

Se for preenchido, ele muda a imagem do asterisco para o ok.
E este, muda o textbox de vermelho para normal, caso seja digitado algo:

$(document).ready(function() {

  $('#contact-form').validate({
    rules: {
      txtNome: {
        minlength: 2,
        required: true
      }
    },
    highlight: function(element) {
      $(element).closest('.control-group').removeClass('success').addClass('error');
    },
    success: function(element) {
      element.text('OK!').addClass('valid')
        .closest('.control-group').removeClass('error').addClass('success');
    }
  });

});

Só que estes dois só funcionam fora do modal. Então no modal preciso verificar os campos, ao ser clicado, e ao ser digitado algo. Não consegui fazer o modal funcionar com required, pois se coloco required, eu não consigo fazer abrir o modal com o click do button.
Answer: Tente isso:

function valid(element) {
  $(element)
    .closest('.control-group')
    .removeClass('error')
    .addClass('success');
}

function notValid(element) {
  $(element)
    .closest('.control-group')
    .removeClass('success')
    .addClass('error');
}

$('#textbox').on('focus change', function() {
  if (! $(this).val() || $(this).val().length === 0) {
    return notValid($(this));
  }

  valid($(this));
});
.success {
  border: solid 1px green;
}

.error {
  border: solid 1px  red;
}

  <textarea id="textbox">Teste</textarea>

Comment: Não deu certo Luiz Felipe, os campos que vou precisar fazer esta verificação é textbox, eu tenho a função pronta pro required, porém por estar no modal eu não consigo fazer abrir o modal se colocar o required nos campos dentro. Vou precisar chamar essa função em vários textboxs.
Comment: Editei a pergunta, para uma melhor esclarecida.
Answer: Basta capturar o evento click para tratar quando o campo for clicado
Se quiser também remover o efeito quando sair, basta usar o evento blur:

$('#i').click(function() {
 if ($(this).val() == '') {
      $(this).addClass('vazio');
           $(this).removeClass('preeenchido');
        } else {
      $(this).removeClass('vazio');
           $(this).addClass('preeenchido');
        }
});

$('#i').blur(function() {
  $(this).removeClass('preeenchido');
  $(this).removeClass('vazio');

});
input {
  background-color: #fff;
  width: 200px
}

.vazio {
  background-color: cyan
}

.preeenchido {
  background-color: yellow
}

<input type="text" id='i' />

Se quiser também fazer isso quando o elemento estiver com focus (clicado ou movido com tab) pode usar o evento focus no lugar do click
Comment: Não funcionou Ricardo, editei a pergunta, para maiores esclarecimentos.
Comment: Vamos lá, o que não funcionou? Chegou a colocar o evento `onclick`  no TextBox apenas para teste, pra ver se exibe pelo menos um `alert`? Algo assim: `$('#txtHora').click(function() { alert('clicou'); });`
Comment: Lembre que um `TextBox` com `runat="server"` não aceita um evento `onclick` no javascript, se colocar `OnClick` será executado um evento server, por isso é importante associar o evento click através do próprio javascript, como no exemplo do comentário acima
Comment: Na verdade Ricardo eu não posso tirar o runat=server, pois utilizo o campo para várias coisas, e teria que mudar toda a lógica, tem alguma forma que consigo contornar este problema ? no caso o click não ira funcionar no javascript mesmo então.
Comment: Funciona se você associar o evento no próprio javascript, assim: `$('#txtHora').click(function() { .... seu código });`  se fosse um `asp:Buttom`, teria o evento `OnClientClick` que resolveria, mas o `TextBox` não tem
Comment: Coloquei a função do txthora  com click, porém ele não retorna o alert.
Comment: Tentei desta forma, porém o modal nem abre, tanto com onchange ou como blur.   $(function () {
            $("#txtHora").live("keyup change", function () {
                if ($(this).val() == "") {
                    alert('campo obrigatorio')
                }

            });
        });


convert an opencv affine matrix to CGAffineTransform

Question: I'd like to take an affine matrix that I have from OpenCV:
Mat T = getAffineTransform(src_pt, dst_pt);

and then convert it into a CGAffineTransform for use in Core Graphics/Objective-C/iOS. (CGAffineTransform docs)
I've tried:
CGAffineTransform t = CGAffineTransformIdentity;
t.a = T.at<float>(0,0);
t.b = T.at<float>(0,1);
t.c = T.at<float>(1,0);
t.d = T.at<float>(1,1);
t.tx = T.at<float>(0,2);
t.ty = T.at<float>(1,2);

This works fine for x and y translations, but NOT if there is any rotation in the matrix. Something seems to be missing since the resulting image seems to be skewed strangely. I've tried multiplying .b and .c by -1 and switching .b and .c, but neither of those seemed to work. The image still appeared incorrectly skewed.
Edit: I should mention that it's almost rotated correctly. When I switch b and c it's at least rotated in the right direction. It just seems a bit off, as in rotated a little too far.
Comment: I'm surprised switching `b` and `c` didn't work, since that would result in the correct rotation. Your posted code currently constructs an incorrect rotation.
Comment: Yeah I tried `t.b = T.at(1,0)` and `t.c = T.at(0,1)` but no luck. I think it may have something to do with the origin.
Answer: Your problem is that opencv is row major and CGAffineTransform is column major you want
t.a = T.at<float>(0,0);
t.b = T.at<float>(1,0);
t.c = T.at<float>(0,1);
t.d = T.at<float>(1,1);

you can tell because in the documentation CGAffineTransform takes the form
[a  b  0
 c  d  0
 tx ty 1]

note that tx and ty are in the bottom row.  In standard row major matrices the translation components go in the rightmost column
[a c tx
 b d ty
 0 0 1]

If the problem persists after making this change (which your question suggests you have already tried) then you need to post more information.  As you suggest the problem could be in the origin of your coordinate system but without any information about your origin nobody will be able to help you.
Comment: Alright thanks, I'll try to update in further detail with images, etc.
Comment: Got it! It was a combination of swapping b and c, needing to set the layer's anchorPoint to 0,0 and then translating by -1/2 the image width, -1/2 the image height. Thanks.
Comment: @taber: The first part—b and c being swapped—is why you were getting skew behavior. You can see this for yourself by manually swapping the numbers around in [my affine transform test app](https://github.com/boredzo/AffineTransformTest).


find all core patches

Question: I work on site that is drupal 7.20, I am sure previous team that worked on this project patch core in some places,I want to find all of patched they did, is there any way to find all patched they did?
one of my idea is find a clean drupal 7.20 core and compare it but I am not sure this is good solution
Comment: I don't think there is any way straight way to do this you need to go through manual comparison, that's why hooks are provided so you need not to touch core.
Comment: As per my opinion you are on the right track comparison will work for you.You can use tool like [WinMerge](http://winmerge.org/) to find out patched files and then you can do whatever you want.
Answer: You can use the Hacked! module

This module scans the currently installed Drupal, contributed modules and themes, re-downloads them and determines if they have been changed. Changes are marked clearly and if the diff module is installed then Hacked! will allow you to see the exact lines that have changed.
Hacked! also provides drush integration so that you can see what files have changed from the command line.
Answer: The Hacked! module will analyze the code base of the core and all contributed modules, compare each to a freshly downloaded version, and if the diff module is installed it will also generate a easy to read report showing all "hacks" to the code base.

This module scans the currently installed Drupal, contributed modules and themes, re-downloads them and determines if they have been changed. Changes are marked clearly and if the diff module is installed then Hacked! will allow you to see the exact lines that have changed.
Hacked! also provides drush integration so that you can see what files have changed from the command line.

find all core patches


Package name is corrupted while copying the class

Question: I defined Point class like below.
package com.sample.app.model;

public class Point {
    public int x;
    public int y;

}

When I try to construct other class with Point class data, I seen package name is corrupted.
CtClass pointClass = classPool.getAndRename("com.sample.app.model.Point", "com.sample.app.model.MyPoint");

Generated MyPoint.class code looks like below.
package model;

public class MyPoint {
  public int x;

  public int y;
}

As you see above snippet, package name is given as model, instead of "com.sample.app.model".
Surprisingly, When I print the new class name, it prints the package name correctly.
Class<?> myPointClass = pointClass.toClass();

System.out.println(myPointClass.getName());

But When I decompile the class file using Java decompiler, I see package name corruption.
Comment: Sounds more like a potential defect entry on the bug tracker of javaassist. Beyond that: probably the "in memory" representation of that class is correct ... but what gets written to disc is wrong.
Comment: is this your goal: How  changing a class name for defining a new class?
Comment: I want to construct new class from existing class content
Answer: Maybe getAndRename method is not for your requirement.
From official documentation: http://www.javassist.org/tutorial/tutorial.html
Changing a class name for defining a new class
A new class can be defined as a copy of an existing class. The program below does that:
ClassPool pool = ClassPool.getDefault();
CtClass cc = pool.get("Point");
cc.setName("Line");

Comment: setName rename the existing class, but I want other class with the same content of existing class.
Comment: Could you save the renamed class as another class?


How to open up the Battery usage Activity?

Question: I have been looking around for a way to launch the Battery use (Setting-->Battery&performance-->Power usage) activity through my android app, but cannot find it.I want know how to start this Activity though an Action or Intent.
What is the action that I should pass into an Intent in order to get the Battery use(which show every apps usage of battery) screen to open up ?
I try to read the guide which Google provide, but I failed.
Intent intent =  new Intent(?????);
                startActivity(intent);
I only want to start this Activity.
Answer: use
Intent intent = new Intent(Intent.ACTION_POWER_USAGE_SUMMARY);
startActivity(intent );

Comment: Ok,It work,thanks very much!
Answer: You can use this -
IntentFilter ifilter = new
IntentFilter(Intent.ACTION_BATTERY_CHANGED);
Intent batteryStatus = context.registerReceiver(null, ifilter);


Server System drive on a USB stick? Good? Bad?

Question: This is a follow up to this question.
I am assembling a storage server with 14 SATA drives.
I don't want to "waste" a drive for the OS system, so I thought I could isntall and run the OS from a USB stick. The system will be very skinny (just a webserver and ssh deamon and some custom scripts).
Is there any reason I should not do this? Like USB stick durability, etc? Do you have any experiences with running server systems from usb sticks?
I noticed that FreeNAS recommends running it from a USB stick.
Answer: The main consideration is making sure the OS doesn't swap, and/or doesn't write much to the flash memory. The USB stick, like a compact flash or SD card (and to a far lesser extent, SSD) has a limited number of write cycles so if you've got something thrashing it constantly for swap you may find it wears out prematurely.
The Sun X4540 systems we've got here use a compact flash card for the OS, and as the other poster said, VMWare ESXi has a embedded version which runs off a stick, in both cases the idea is to liberate your physical disks for actually storing your data, rather than the OS. In the case of the Sun configuration, it is officially supported and they make some suggestions in the install guide for Solaris and Linux here (page 79) .
Comment: Might want to log to a different drive / remote syslog as well.
Comment: yes, i plan to log all logs to nfs anyway because my application calculates important stuff and billing from the logs, and i don't want to do that on the storage slaves.
Answer: Well, ESXi Embedded runs off of a USB Stick - and I have had zero issues with performance or anything for that matter. As long as it's Unix based I see no issues.
Comment: We do the same thing here without any problems.  It works great.
Answer: No - it's a good idea - unlike CDROM booting, maintaining patches customer config is a lot simpler. Just make sure you do as the others said and keep all the logs / swap space elsewhere.
But wouldn't it be just as easy to boot it across a network?
Comment: network is a good thing but my server will be spread across several datacenters for data durability and availability. therefore this could get complicated.


Why is "the probability that a continuous random variable equals some value always zero"?

Question: I found lots of references that say, "the probability that a continuous random variable equals some single value is always zero". Why is that?
Here is a counterexample I thought of: supposing $X\sim N(0,1)$, define $Y=min(X,0)$. Then Y is a continuous random variable but the probability of $Y$ at a single point $0$ should be $0.5$, not zero.
Also, I think any CDF would be left continuous if "the probability that a continuous random variable equals some single value is always zero".
What is wrong with my thoughts?
P.S. Examples of the references are:

http://www.henry.k12.ga.us/ugh/apstat/chapternotes/7supplement.html
http://mathinsight.org/probability_distribution_idea
Comment: see definition 1.32 of the link http://math.arizona.edu/~jwatkins/probnotes.pdf for the definition of a continuous random variable
Comment: By the way, the distribution of your $Y$ is not continuous, it is mixed continuous-discrete :)
Comment: thanks for the editing @gung, and what is your opinion?
Comment: @fcop, which part of that definition is helpful? I have looked at it, and I do not immediately see.
Comment: @Richard Hardy: part 2, the definition of continuous random variable. I think JohnK makes a similar remark
Comment: Asking about the $Y$ counterexample may be sufficiently unique to merit this Q remaining open.
Comment: According to the definition of a continuos variable given in the link in my comment supra, $Y$ is not a continuous variable.
Comment: @fcop, thanks for the link, I agree with you now. It seems I missed the rigorous definition of continuous random variable.
Answer: The thing is $Y$ is not that continuous to begin with. To be continuous, the distribution function of $Y$ must be absolutely continuous (see definition 1.32, page 10 of link http://math.arizona.edu/~jwatkins/probnotes.pdf by @fcop). You see the distribution of Y has a half impulse (Dirac delta) function at zero. When you approach zero on the negative side there is a jump in distribution function value. So the distribution function of $Y$ is not continuous.
If $f(x)$ is continuous, $g(f(x))$ is not necessarily continuous.
Comment: Hi, I believe Y is a continuous random variable. And I think you mean the density function of Y has a half impulse. Am I correct?
Comment: (+1) $Y$ does not have a density function and therefore is not continuous.
Answer: Let's just go with a simple intuitive explanation. But it can get really mathy really fast (if you prefer).
A continuous distribution is a line between two points A and B. On this line there are infinitely many points, no matter if the distance between the points (A, B) is extremely small. If all of those infinite points had a probability larger than 0, then the sum of probabilities would be infinity.
But if that where the case, then the said probability distribution would violate the (Kolomogorov) axioms of probability. So it would not be measure of probability in the modern understanding.
Edit, the derivative of the function $\text{min}(x,y)$ is not defined for $x=y$. So your example is not continuous.
Comment: OK, let me rephrase: you say that if *all* points had positive probabilities then the sum would be infinity. However, if only a fixed number of points had positive probabilities, the sum would be finite. So your argument does not go through for the case of the OP. Also, mind the spelling of *continuous*.
Comment: OP does not ask for *all* the points to have non-zero probability. The question is about *at least one* point ("some single value") with positive probability.
Comment: @Richardhardy then it is not a contiounes distribution


Running a Python script saved on a Windows 7 server... on a Mac?

Question: We have a server running Windows 7 Pro. I have several Python script I'd like to save to the server and have it so that client computers can run them by simply double-clicking. The client computers are all running OSX. This is proving to be... problematic.
First I tried to simply make the Python scripts executable, but this doesn't seem to be possible on a Windows server -- since you can't set the 'executable' flag, double-clicking on a file will always open it in an editor (unless I were to go to every single computer and make .py files open with Python). Trying to create a shell script has the same problem -- there's no way to make them executable from the server.
My solution was to just make a simple AppleScript app that sends a command to launch the script. Unfortunately, as soon as I copy the app to the server, it stops working. It seems that OSX apps refuse to execute properly when saved to the server -- if you run the file, nothing happens at all.
Is there a simple solution I'm overlooking?
Answer: This is probably what you're looking for: http://oreilly.com/catalog/samba/chapter/book/ch05_03.html says that Samba clients (that OS X uses to connect to Windows shares) can map archive/hidden/system file attributes to owner/group/world executable bits respectively.
Try setting those attributes on the script file and make sure its first line is <code>#!/usr/bin/python</code>. If this mapping is enabled by default, the script will run by double-click.
Comment: I'm guessing if the Windows server is using some proprietary system to facilitate mounting, and the system doesn't support permission mapping, I'm out of luck?
Comment: If OS X clients connect via smb://, it's only matter of their SMB client configuration.
Comment: They don't :(  Drives are mounted via proprietary software on each of the client computers.
Comment: Maybe creating an application bundle will work. AFAIK the file in /Contents/MacOS does not have to be executable, however I can't check right now whether it can be a script.
Comment: I'm pretty sure it needs to be executable. If I copy a simple application made with the AppleScript Editor to the server, the 'applet' file in /Contents/MacOS/ loses its execute bit and the program refuses to run.
Answer: actually the issue is that windows has no equivalent of the execute bit for files.
the solution is to change the mount options on the share so that all the files have their execute bit set.


Does the neck length of a guitar affect its sound?

Question: I'm a bass player but I dabble in guitar.If I put a 6 stringed bass neck on a guitar would it affect the sound at all ?
Comment: Bass necks are generally longer, so yes.  But it depends on how you change the strings too.
Comment: Longer necks generally aid sustain at lower frequencies.
Comment: I would keep the same type of strings just longer. Would than mess up my guitars sound ?
Comment: I don't think you would be able to use the same type of strings.
Comment: @DrMayhem - I was wondering that. Does OP mean same gauge, or 2 plain /4  wound? Will it be tuned lower? Will the strings be long enough? So many unknowns!
Comment: I know. I think your answer was spot on covering the real issues: it is unlikely to work for various reasons:-)
Answer: The spacing of bass guitar strings, and their thickness , means that the neck, aspeciallly a 6 string bass neck, will be a lot wider than that of the original guitar where it joins the body. It will probably mean the strings are not exactly aligned with the pup poles. There will be major surgery to do. That in itself will slightly affect the sound.
Having a longer neck will add more wood, so may give a little more sustain, but it will hardly be noticeable. With different strings, or the same strings tuned differently - otherwise the originals would be too tight - will affect the sound, too.
A bigger problem is going to be intonation, as the bridge will most likely be in the wrong position - 12th fret needs to be half way between bridge and nut.After that, the machine heads won't be suitable for the thin strings. It just goes on - and on...    but, by all means try it, you'll only end up with two wrecked guitar/basses at worst.
Seems  like you are trying to make a baritone guitar - maybe try one out before embarking on this venture.


Why is the sigmoid activation function better than the sign function?

Question: I would like to know why we don't just use the sign function if it caps it off at 0 or 1 anyway.
Comment: Do you mean the sign function that only gives values of 0 and 1? And do you mean as an activation function?
Comment: Perhaps because choice is binary but certainty is not.
Answer: Backprop requires differentiable functions with nonzero gradients in a meaningful portion of their domain. The derivative of the sign function is zero almost everywhere so weights will never update.
Comment: great, thank you. Would you possibly be able to point me to some mathematical proof? I understand what you mean, but not definitively.
Comment: furthermore, why is a sigmoid more powerful in general?
Comment: To prove this, just write down the backprop for two networks, one using sigmoid and one using sign. Because the derivative of the sign function is 0 almost everywhere, the weights will never move. The sigmoid derivative is never 0 anywhere (though it does become small), so it does not exhibit this behavior.


For the Sake of Argument

Question: So someone sent me this xkcd as a reply to something I said online:
https://xkcd.com/1432/
I frankly felt assaulted by the pop culture monster. Perhaps as a mathematician this is not uncommon.
My question is, in math we often prove things by contradiction. For example perhaps the best known way to prove that $\sqrt{2} \notin \Bbb{Q}$. This proof is, what I consider to be, well known. In that many people may have seen it in high school.
My question is: are there other well known examples of proofs via contradiction (so far) that are commonly known? I guess, sort of as a rebuttal to the idea that only liars assume things for the sake of arguing about them.
If you consider this question too broad, I could use a reference for such results or a nice book about them, etc.
Comment: You don't need contradiction to show that $x \in \mathbb{Q} \setminus \mathbb{Z}$ satisfies $x^2 \not\in \mathbb{N}$. This shows that the only natural numbers whose square roots are rational are the squares.
Comment: Are you asking for proofs of things that are commonly known in mathematics, or that would be commonly known by people who would send you that xkcd?  (By the way, I found that xkcd funny, but in a oh-that-hurts kind of way.)
Comment: @BrianTung I guess I'm looking for proofs that are the MOST commonly known by people who have gone through college. Not only mathematicians. I guess this is a short list. That's why I'd like a reference if anyone has one.
Answer: The first question is, what do you mean you MUST prove things by contradiction? The only thing you can possibly mean, is that there are things which can be proved without proof-by-contradiction, i.e. without some of the very rules of logic that we usually take for granted. What exactly do we mean by this?
In classical logic, it is a theorem that proof by contradiction holds: for any statement $P$, if $\lnot P$ implies $Q$ and $\lnot Q$, then $P$.
This theorem relies on three facts:

The law of the excluded middle (LEM): for any $P$, either $P$ or $\lnot P$.
The law of non-contradiction (LNC): for any $P$, if $P$ and $\lnot P$ then $\bot$. (This symbol can means the statement that is always false, read as just "false".)
The principle of explosion (PE): for any $P$, if $\bot$ then $P$.

The proof of the theorem, then, goes like this: Assume that $\lnot P$ implies $Q$ and not $Q$. By LEM, either $P$ or $\lnot P$. In the former case, $P$ and we are done. In the latter case, because $\lnot P$, by our assumption, $Q$ and $\lnot Q$. Therefore by LNC, $\bot$, and by PE, $P$. So we are done in this case as well.

So, what can you get done without proof by contradiction? Well, as the above theorem shows, you can get anything done as long as you accept LEM, LNC, and PE. But what if you reject these? A prominent alternative to classical logic is intuitionist logic, which rejects LEM. In intuitionistic thinking, you are not allowed to prove something by showing the opposite cannot be true.
In particular, if you show that $P$ results in a contradiction, that means that $\lnot P$, because that's what $\lnot P$ means. But if you show that $\lnot P$ results in a contradiction, you have only shown that $\lnot \lnot P$--you haven't given a direct proof of $P$.
So, this has been a bit roundabout, but to answer your question, what are some things that you "need" proof by contradiction to prove? Here are some examples (I wish I had more), in the sense that intuitionist logic is not enough to prove them:

There exists a function which is not continuous.
There exists an incomputable function.

In fact (!), proving $\sqrt{2}$ is irrational does NOT require proof by contradiction, as you have claimed; you can do it by finding the continued fraction of $\sqrt{2}$.

See also:

How can some statements be consistent with intuitionistic logic but not classical logic, when intuitionistic logic proves not not LEM?
Comment: I appreciate the answer but this is really not what I'm looking for. Maybe I should just close the question.
Comment: @amcalde I like the question. But it's not really well-defined the way you meant it.
Answer: If by a proof we mean a formal deduction in a Hilbert System that obeys classical propositional logic, then we never have to proof anything by contradiction. The reason for this is, that in such a system

$\phi \rightarrow \psi$
$\neg \psi \rightarrow \neg \phi$

are provably equivalent and thus any proof for $\neg \psi \rightarrow \neg \phi$ may be extended to a proof for $\phi \rightarrow \psi$ and vice versa.
Most mathematicians rarely (never) think about proofs in a formal way, but they agree (intuitively) on a common logical calculus which allows the above equivalence. However, this is not true of all mathematicians and there are alternative logical systems (e.g. intuitionistic logic) where 1. and 2. may not be provably equivalent. Thus, if your definition of a proof differs from mine in a substantial way, you would have to provide us with your logical system and notion of proof, before we could try to answer your question.
Comment: Some proofs by contradiction are of the form $\neg \phi \Rightarrow (\psi \wedge \neg \psi)$. I'm not sure whether this can be converted directly into contrapositive form.
Comment: @Ian $\neg \phi \rightarrow (\psi \wedge \neg \psi) \equiv  (\psi \wedge \neg \psi) \vee \neg \neg \phi \equiv (\psi \wedge \neg \psi) \vee \phi \equiv (\phi \vee \psi) \wedge (\phi \vee \neg \psi)$
Comment: Isn't this a little misleading? Of course proofs by contradiction can be converted into direct proofs in a Hilbert system *where the axioms already have all the components to make proof by contradiction valid*. However you prove something in a Hilbert system, if proof by contradiction is a valid rule, then you are basically relying on rules equivalent to proof by contradiction.
Comment: If you want to make a distinction between whether or not a proof *uses* proof by contradiction, you will have to use as a base a logical system where proof by contradiction is *not valid*, so that you can then say which other theorems rely on it based on whether or not they follow from the base set of axioms, or whether they only follow from the base set of axioms + proof by contradiction.
Comment: All of the subtlety is hidden in these three rules: $A \land \lnot A \vDash \bot$ (LNC), $\bot \vDash A$ (explosion), $A \lor \lnot A$ (LEM).
Comment: @6005 I agree and the point of my answer is that while most mathematicians work with "proofs" that render the notion of "proofs by contradiction" pointless, this is "by design". If amcalde prefers to work with a formal system that doesn't have this feature, his question isn't pointless anymore and if he provides us with a suitable system, we then could try and deliver interesting statements for which there are only "proofs by contradiction".
Comment: @Stefan Sure, okay. Understood.
Answer: I think that the main problem with contradiction proofs is that they belong to a binary world where thing are radically true or radically false. This works great for our form of mathematics, but in a real world where things are sort of always mixed-up doesn't work that well.
I think that's why didn't really catch up with other lines of thought from the common sense. I guess that whatever example you will present at that stage it will appear as another "for the sake of argument".

For the Sake of Argument


Deriving the standard error of the mean formula

Question: I don't understand , here $X$ is data like for ex. height in a class :
$X_1=186cm$
$X_2=186.9cm$
$X_3=188cm$
and so on ... or they are independent random vars like one var for height other for
 weight and so on ?
The first step is confusing for me , please explain the following :
The variance of  $T=(X_1+X_2+X_3 + ... X_n)$ is $n\sigma^2$
Why ?
Comment: https://en.wikipedia.org/wiki/Variance#Basic_properties
Answer: The variance of the sum of independent random variables is the sum of their variances. So if $X_1$, $X_2$... all have variance $\sigma^2$ then the sum is $n\sigma^2$.
Comment: so the $X_i$ here are all random vars, it confused me because the article says observations .... so I thought that they meant observations for a particular random var.
Comment: Each observation is a different random variable. Think about surveying people and measuring their height. Your first measurement is 5' 10". But you could have picked somebody different and got a different height.
Comment: thanks, I coined the wrong term in my head about a random var.