scatterplot smoothing in r with big dataset: different methods

Question: I have a large dataset (>300,000 rows) with two variables. y is binary and x is continuous & numeric. I'd like to plot y and add smooth curve against x. I understand that loess(y~x) is a solution, but since I have such a big dataset, it takes too long to run, even if I set the 'cell' parameter to 500.
Using scatter.smooth, it runs much faster and I think it also uses loess. but I have trouble understanding the parameter 'evaluation = 50'. Does this mean that it only uses 1/50 of data to produce the smooth curve?
I also tried using geom_smooth, it would automatically switch to 'method=gam' since I have more than 1000 data points. but the curve looks different from the one I got using scatter.smooth (I guess that's normal as they are different models).
My goal was just to see the pattern of the data. Which smoothing method should I use? Can I trust scatter.smooth? what's the difference between using loess and gam?
below is the plot from scatter.smooth. It looks good, but it runs so much faster than the regular loess(). I'm not sure how it works...

Using the method whuber provided:

any help would be highly appreciated!
Thanks
Answer: It's actually efficient and accurate to smooth the response with a moving-window mean: this can be done on the entire dataset with a fast Fourier transform in a fraction of a second.  For plotting purposes, consider subsampling both the raw data and the smooth.  You can further smooth the subsampled smooth.  This will be more reliable than just smoothing the subsampled data.
Control over the strength of smoothing is achieved in several ways, adding flexibility to this approach:

A larger window increases the smooth.
Values in the window can be weighted to create a continuous smooth.
The lowess parameters for smoothing the subsampled smooth can be adjusted.

Example
First let's generate some interesting data.  They are stored in two parallel arrays, times and x (the binary response).
set.seed(17)
n <- 300000
times <- cumsum(sort(rgamma(n, 2)))
times <- times/max(times) * 25
x <- 1/(1 + exp(-seq(-1,1,length.out=n)^2/2 - rnorm(n, -1/2, 1))) > 1/2

Here is the running mean applied to the full dataset.  A fairly sizable window half-width (of $1172$) is used; this can be increased for stronger smoothing.  The kernel has a Gaussian shape to make the smooth reasonably continuous.  The algorithm is fully exposed: here you see the kernel explicitly constructed and convolved with the data to produce the smoothed array y.
k <- min(ceiling(n/256), n/2)  # Window size
kernel <- c(dnorm(seq(0, 3, length.out=k)))
kernel <- c(kernel, rep(0, n - 2*length(kernel) + 1), rev(kernel[-1]))
kernel <- kernel / sum(kernel)
y <- Re(convolve(x, kernel))

Let's subsample the data at intervals of a fraction of the kernel half-width to assure nothing gets overlooked:
j <- floor(seq(1, n, k/3)) # Indexes to subsample

In the example j has only $768$ elements representing all $300,000$ original values.
The rest of the code plots the subsampled raw data, the subsampled smooth (in gray), a lowess smooth of the subsampled smooth (in red), and a lowess smooth of the subsampled data (in blue).  The last, although very easy to compute, will be much more variable than the recommended approach because it is based on a tiny fraction of the data.
plot(times[j], x[j], col="#00000040", xlab="x", ylab="y")
a <- times[j]; b <- y[j]   # Subsampled data
lines(a, b, col="Gray")
f <- 1/6                   # Strength of the lowess smooths
lines(lowess(a, f=f)$y, lowess(b, f=f)$y, col="Red", lwd=2)
lines(lowess(times[j], f=f)$y, lowess(x[j], f=f)$y, col="Blue")

The red line (lowess smooth of the subsampled windowed mean) is a very accurate representation of the function used to generate the data.  The blue line (lowess smooth of the subsampled data) exhibits spurious variability.
Comment: Thanks whuber for such a comprehensive answer! I'm still trying to get my head around this method. I tried applying the code to my data. Could you please have a look at the plot in the OP?
Comment: I'd need to investigate the code further to fully understand it. Thanks again!
Comment: I used part of this.  The kernel stuff in R is still something that I am not yet familiar with.
Answer: I found the locfit function from the locfit package to be a good solution - it is quick and made it easy to plot by group (https://cran.r-project.org/web/packages/locfit/locfit.pdf). The scale, alpha, deg, kern, kt, acri and basis parameters control the amount of smoothing. I used it with geom_smooth: geom_smooth(method='locfit', method.args = list(deg=1, alpha=0.3))


How do I add JavaScript specific for a view?

Question: I have a view in my site that has a specific class name. I want to know, inside the template.php file of a theme, how I can know if a view with specific class name is available in the requested page.
This is very important for me because I need to include specific JavaScript and CSS, when a view with a specific class name (such as image-gallery) is used in a page.
Answer: You could use the template_preprocess_views_view() hook to do this:
function THEME_preprocess_views_view(&$vars) {
  $view = &$vars['view'];
  // Make sure it's the correct view
  if ($view->name === 'your-view-name') {
    // add needed javascript
    drupal_add_js(drupal_get_path('theme', 'your-theme') . '/your-js.js');
    // add needed stylesheet
    drupal_add_css(drupal_get_path('theme', 'your-theme') . '/your-css.css');
  }
}

Note that you could also check the specific display of the view using:
if ($view->name == 'your-view-name' && $view->current_display === 'your-display-id') {
  // include javascript & css here
}

You should be able to check for a custom css class like this:
if ($view->name == 'your-view-name' && $view->display[$view->current_display]->display_options['css_class'] === 'your-css-class') {
   // include javascript & css here
}

Comment: Thanks a lot, but this function recognize the view by name, I want to recognize the class name, because There are several views in one page that has same class name . for example 2 views has 'gallery-image' class name & 3 views has 'light-box' class name. I want to add js for example if a view with 'light-box' css class name is available in the page.
Comment: Can't you just check the display? `if($view->name == 'your-view-name' && $view->display_id == 'your-display-id')`
Comment: no, because it is not clear that how many views are available in the page. maybe there ate 10 views that all of them have a same css class name.
Comment: I implemented this hook and flushed all caches - but it didn't get called.  Not sure why.  Yes I double-checked the name.
Comment: That seems very strange? You have replaced THEME with the name of your actual theme? This should work just fine as long as you implement it correctly.
Answer: You can use views_get_page_view function to find out which view is being served on the page. It return views object. Note that in the code snippet below you should compare with views machine readable name, but of course you can write your own comparaison basing on the view object returned by views_get_page_view
function YOURTHEMENAME_preprocess_page(&$variables) {
    $view = views_get_page_view();
    if($view->name == 'YOURVIEW') {
    //Do what ever you want
    }

Answer: Just in case this is useful to anyone else stumbling across this question, searching like I did for attach JavaScript to a Drupal View. In terms of D7 & Views 3.7, the following worked best for me:
function HOOK_views_pre_render ( &$view ) {
  /// check to make sure the view has a classname
  if ( $view->display_handler && !empty($view->display_handler->options['css_class']) ) {
    $cln = $view->display_handler->options['css_class'];
    $cls = 'CLASS GOES HERE';
    /// test that the classname contains our class
    if ( preg_match('/(^|\s+)' . preg_quote($cls) . '(\s+|$)/i', $cln) ) {
      /// build the path to the js, which is local to my module, js/view.js
      $sep = DIRECTORY_SEPARATOR;
      $dir = rtrim(drupal_get_path('module', 'HOOK'), $sep);
      $pth = "{$dir}{$sep}js{$sep}view.js";
      drupal_add_js($pth);
    }
  }
}

This was beneficial as I wanted to keep the code within my module, rather than the theme — because the enhancements brought by the JavaScript had nothing to do with visual appearance.

NOTE: Obviously HOOK should be replaced with your module name, in both locations, and CLASS GOES HERE should also be replaced with the class you are searching for.
Comment: For clarity, why was pre_render better than pre_process?
Comment: @Nick ~ There isn't a `HOOK_views_pre_process` (https://api.drupal.org/api/views/views.api.php/7) unless you are meaning the reason why to use this over `THEME_preprocess_views_view` -- in which case it's only out of preference of where you wish to add your code modifications. `HOOK_views_pre_render` can be hooked into from a module, whereas `THEME_preprocess_views_view` should go in your theme/template file. You could also use `HOOK_views_post_render` if you prefer.
Comment: Sorry, I did mean `THEME_preprocess_views_view`, yes... I'm sure I have done `hook_preprocess_views_view` in modules before.
Answer: If you're on drupal 6 or 7 you can also include javascript assets with the module context add assets. Which does require context but you'll rest assure that it'll always be included when the view is being rendered.
https://drupal.org/project/context_addassets
From the module's project page:

Have you ever wanted to include javascript or css when a particular view or block is being rendered? Or have you ever wanted to include javascript or css for just the front page without having to write code?
Context add assets allows you to do this. It has an easy to use UI to allow you to do this all without writing any code. Because it uses ctools all of this is also exportable.
Answer: Write JavaScript code that searches for specific class in the view, using jQuery's hasClass(), and include your JavaScript file when the condition is satisfied.
Another way is to put a template for your view, and add JavaScript code through that template using drupal_add_js().
Comment: Can you tell me, how can I do this. Indeed how can include css and js files after a conditional expression in java script is true ??!
Comment: For Drupal 8, this answer outlines how to make a custom view template for the view output to do this: https://stackoverflow.com/a/43131240/227926


Javascript Regex - Get All string that has square brackets []

Question: I have string data below:
var data = "somestring[a=0]what[b-c=twelve]----[def=one-2]test"

I need to get all strings that contain square brackets []. This is the result that I want.
["[a=0]", "[b-c=twelve]", "[def=one-2]"]

I've tried using regex /\[(.*?)\]/, but what I've got is an only the first array element is correct, the next elements are basically the same value but without the square brackets.
data.match(/\[(.*?)\]/);
// result => ["[a=0]", "a=0"]

What regexp should I use to achieve the result that I want? Thank you in advance.
Answer: You want to use the g (global) modifier to find all matches. Since the brackets are included in the match result you don't need to use a capturing group and I used negation instead to eliminate the amount of backtracking.
someVar.match(/\[[^\]]*]/g);

Answer:
In /\[(.*?)\]/,  *? means lazy, matching as few content as possible.
What you actually want is all the matches in content. Try modifier g
Try this one, http://regex101.com/r/aD6cM8/1. Any match starts with [, ends with ], but doesn't allow[ or ] inbetween.

someVar.match(/\[([^\[\]]*)\]/g)
Answer: You should just add the g switch to your regex :
someVar.match(/\[(.*?)\]/); // result => ["[a=0]", "a=0"]

results in
[ "[a=0]", "[b-c=twelve]", "[def=one-2]" ]

Answer: Your regex is correct,  just suffix g to it to make it global:
someVar.match(/\[(.*?)\]/g);
Here's more info on it: http://www.w3schools.com/jsref/jsref_regexp_g.asp


Masking with transparent PNG in AS3

Question: i want to mask objects with dynamic transparent PNG files in Flash Action Script 3.0
if you have any solution regrading that please forward..
Thank you,
Hardik
Comment: By dynamic, do you mean that the tranparent PNG is going to change like an animation, or is it dynamic in that it could be one of multiple PNGs that are used?
Comment: I know one not easy way , but in animations it will kill cpu . You can draw mask target to bitmap and then check from png which pixels are transparent to erase it from target .
Answer: I take it that by dynamic PNG you just mean one that has been loaded by code at run time. There is nothing special to that, the same answer applies as to any library object imported as a Bitmap and then set to use lossless compression.
In order to do this the png to be used as a mask must be embedded in a DisplayObject (typically a Sprite, a MovieClip or even a Loader). The DisplayObject class has a property cacheAsBitmap and you need to set that property to true for both the DisplayObject holding the mask and the one holding the content to be masked. Then assign the mask by code to let the mask reveal the content.
Example:
<code>//myClip is the MovieClip or Sprite or even Loader holding the content
myClip.cacheAsBitmap=true;
//myMask is a MovieClip or Sprite or even Loader containing the transparent PNG
myMask.cacheAsBitmap=true;

//set up the mask to reveal the content where the PNG is not transparent
myClip.mask=maskClip;
</code>
Answer: PhotonStorm has an implementation of this that you could dig through and figure out how they do it.
Comment: To whoever downvoted 3 years later, please add a comment as to what's wrong with my answer so I can improve it. Or better yet; you could edit it to be more correct, or give a better answer.
Comment: Even three years ago this should not have been an answer, it should have been a comment. Therefore another downvote.

Masking with transparent PNG in AS3


Isomorphic group to the multiplicative group of a field.

Question: Suppose that I have an abelian group $G$ and an epimorphism from $G$ to $(K^*)\times (K^*), $ where $K$ is an algebraically closed field. Is it true that $G$ cannot be isomorpic to $K^*.$?
Comment: The squaring map from $K^*$ to $(K^*)^2$ is onto (since $K$ is algebraically closed), and is a group homomorphism since $K^*$ is abelian. So you could take $G=K^*$ and get an epimorphism from $G$ to $(K^*)^2$.
Comment: So your question is "for $K$ an algebraically closed field, can there be an epimorphism $K^*\to(K^*)^2$?". Right?
Comment: @ArturoMagidin I see that i was not clear. By $(K^*)^2$ i meant $K^∗\times K^∗$.
Comment: @DaanMichiels, yeah, i want to know if there exists an epimorphism $K^*\to K^*\times K^*,$ where $K$ is an algebraically closed field and $K^*=K\setminus\{ 0\}.$
Comment: @Hector: Then **please** edit the question and write it correctly. $(K^*)^2$, in the context of fields, is naturally interpreted to be $\{a^2\mid a\in K^*\}$. If you meant $K^*\times K^*$, then you need to write it as such.
Comment: @ArturoMagidin, Done.
Answer: The multiplicative group $\mathbb{C}^*$ is isomorphic to the product of additive groups $\mathbb{R}/\mathbb{Z} \times \mathbb{R}$, the first component being the argument over $2 \pi$ and the second being the log of the modulus. As additive groups, $\mathbb{R}$ is isomorphic to $\mathbb{R}^4$, since they're both continuum-dimensional rational vector spaces. There's clearly an epimorphism from $\mathbb{R}^4$ onto $(\mathbb{R}/\mathbb{Z} \times \mathbb{R})\times(\mathbb{R}/\mathbb{Z} \times \mathbb{R})$, and hence an epimorphism from $\mathbb{C}^*$ to $\mathbb{C}^* \times \mathbb{C}^*$.
Comment: Ok  everything is correct. Thank you.
Comment: @Hector:  Please accept the answer to show that the question has been answered to your satisfaction.
Comment: @TaraB, i am not sure how to do that. Sorry i am new here.
Comment: @Hector: You have done it. It is the green tick to the left of the answer.
Comment: @Hector:  Yes, you've done it now.  You hadn't yet when I wrote my comment.


Wordpress Legacy Code Takeover

Question: So,
I've been tasked with taking over a word press project from 2012. I've been given all the source code, in addition to a database dump, and have installed MAMP.
First, I was getting the common "Error establishing a database connection". When I looked at localhost:8888 I saw that the database was not found. Hence, I created a database from the name given in my wp-config.php and all was well; or so I thought.
I'm stuck now between two situations:
1) I only create a database with the desired name and perform no import, at which point I receive a prompt to install WordPress giving me a barebones installation using none of the legacy code.
2) I import the the database dump into the appropriately named database, and nothing works.localhost:8888 cannot be found and wp-admin redirects to the live sites admin login.
Hopefully someone can help to figure out how to run this site locally so I can make the desired changes for the client.
Lastly, I should also note that I've attempted to update the wp_options table to have the appropriate url of http://localhost:8888 and still no luck. I've also tried visiting localhost and http://localhost:8888 and none of these work either.
Thanks!
Comment: What if you set the url in database to just localhost? Without the port.
Comment: Sadly no luck with this approach. It seems like for some reason when dumping in the database WordPress does not know how to serve the new files..
Comment: If you can actually connect to the database it should just be a url issue.
Comment: I wonder what the url should be, as the `wp-admin` redirects to the live page's admin login page.
Comment: That sounds like you haven't replaced all references to the old url. This must be your problem after all.
Comment: Could you advise on how I could accomplish this, or point me to an article in the right direction? (PS thank you for you continued support and quick responses!)
Comment: Check out the docs: https://codex.wordpress.org/Changing_The_Site_URL
Comment: So I have tried editing the lines within my `wp-config.php` with no luck and have even tried issuing the sql command from php admin:

`UPDATE wp_options SET option_value = replace(option_value, 'http://localhost/test-site', 'http://localhost:8888') WHERE option_name = 'home' OR option_name = 'siteurl';

UPDATE wp_posts SET post_content = replace(post_content, 'http://localhost/test-site', 'http://localhost:8888');

UPDATE wp_postmeta SET meta_value = replace(meta_value,'http://localhost/test-site','http://localhost:8888');`

Where `test-site` was the ip address of hosting. Still nothin
Comment: what are the errors you're getting? make sure all error display is turned on
Comment: So with `WP_DEBUG` set to `True` I'm only getting notices, no errors. Furthermore, these only appear when I attempt to visit `wp-admin` and not when I visit the default `localhost` url.
Comment: Is your localhost definitely pointing to the proper folder and index file? Can you troubleshoot by just adding an index.html and trying to visit that? Troubleshoot your setup.
Comment: So if I add an `index.html` to the folder being pointed at by MAMP nothing loads.. perhaps this is indicative of a deeper issue?
Comment: Maybe. Move your wp stuff out for a second, have just an empty folder with index.html. Until that works I wouldn't worry about WP.
Comment: Created new folder, simple `index.html` restarted MAMP pointed it to the new directory.. nothing

simple `index.php` MAMP pointed as this new directory.. nothing..
Comment: Well, now we know where the problem lies.
Comment: Ready? So fresh install and with `index.html` doesn't load with `http://localhost:8888` but does load with `http://localhost:8888//`.

Then, (just to add some interesting) when I remove the `index.html` I get nothing. Blank page. Even trying `http://localhost:8888/wp-admin/` nothing.

So back to square 1 ?
Comment: Well if you remove index then nothing should be loading....Unless I misunderstand.
Comment: I've been placing `index.html` in the same folder as the old webpage with this behaviour.
Answer: For posterity, as discussed in comments, this is an issue with MAMP setup and not WP configuration.

Wordpress Legacy Code Takeover


Do we have $\mathbb{P}(\Vert A \Vert >\lambda) \le \mathbb{P}(\langle Ax,y\rangle > \lambda)$?

Question: Given that $\Vert A \Vert = \sup\left\{\langle Ax,y \rangle: x,y \in\mathbb{R}^n, \Vert x\Vert_2 = \Vert y\Vert_2 =1\right\}.$ I wonder whether following statements holds:
$$\forall \lambda>0: \mathbb{P}(\Vert A \Vert >\lambda) \le \mathbb{P}(\langle Ax,y\rangle > \lambda).$$
Comment: What's being chosen at random here?
Comment: And $x$ and $y$ are fixed? Are you asking if the statement holds for all $x$ and $y$?
Comment: @MishaLavrov Sorry for the misunderstanding. $A$ in this problem is a matrix with entries that are random variables.
Comment: @MishaLavrov I am asking for all $x,y$. Thank you a lot and sorry if I am too naive.
Comment: This question needs to be clarified. What are $x,y$ on the right hand side above?
Comment: @copper.hat I don't know I understand exactly the problem in the following link. $A$ in the link is a matrix containing random variables and $x,y$ seems to be picked arbitrarily. https://math.stackexchange.com/questions/4862702/prove-that-mathbbp-left-vert-a-vert-ge-ck-sqrtn-sqrtk-t-right
Answer: The inequality should go the other way. (I am assuming that $x,y \in \mathbb R^n$ with $\|x\|_2 = \|y_2\| = 1$, as in the definition of $\|A\|$.)
If $\langle Ax,y\rangle > \lambda$ for some fixed choice of $x$ and $y$, then this implies that $\|A\| > \lambda$: $\|A\|$ is defined to be equal to $\langle Ax,y\rangle$ for the best choice of $x$ and $y$ depending on $A$. So the event "$\|A\| > \lambda$" contains the event "$\langle Ax,y\rangle > \lambda$", and therefore we must have $$\Pr[\|A\| > \lambda] \ge \Pr[\langle Ax,y\rangle > \lambda].$$
Comment: I would like to ask this question since I found a very interesting question in the following link. However, I guess it cannot use this approach.
https://math.stackexchange.com/questions/4862702/prove-that-mathbbp-left-vert-a-vert-ge-ck-sqrtn-sqrtk-t-right


Appending questions and answers into an array

Question: I am new to javascript and I have 2 strings
lets say
const questions = "1.What is love? 2.1+1 = ? 3. 2+2 = ?"
const answers = "1.Yes 2.2 3.4"

I want to append it into sth like :
const package = [{question1: answer1},{question2: answer2}]

I tried to split and append it through regex and split but it didn't work
Is there a way to individually append into a {} and then into the array of the package?
Comment: Have a look at `split()` and `map()` functions and to append it, you can have an array of objects
Comment: Just out of curiosity... **why** you use those two string format inputs to start with?? What if you create an answer that has the same format `N.` you're pretty much in problems regarding a proper parsing.
Comment: this is bad format. any number with dot in an answer or a question wll ruin it. choose another one or better use JSON
Comment: The answer to "What is love" can't possibly be "Yes.".
Answer: You can use split and filter and map to get the result as:

const questions = "1.What is love? 2.1+1 = ? 3. 2+2 = ?"
const answers = "1.Yes 2.2 3.4"

function splitArr(str) {
  return str.split(/\d+\./g).filter(Boolean).map(s => s.trim())
}

const ques = splitArr(questions);
const ans = splitArr(answers);

const result = ques.map((question, index) => ({ [question]: ans[index] }));
console.log(result);


Decompressing Files on an NTFS Volume from Linux

Question: I recently did something stupid on my dual-boot laptop, where I compressed the entire volume to make room for a Linux partition. For some reason, Windows let me compress C:\ntldr. Now I need to get it uncompressed in order for Windows to boot.
Here are some of the operating restrictions I have:

I do not have access to the BIOS.
I cannot boot from CD/USB/floppy. (I installed Linux through PXE)
It does not have network access.

Is there were some way to specify that the ntfs-3g driver shouldn't compress files even if it thinks it should (if the directory is compressed) when mounting the volume?
Or, is there a way to modify the attributes of a directory using ntfsprogs?
Answer: Okay, I think I've solved it. The problem lies in the ntfs-3g driver; it tries too hard to predict what the user wants.
I solved the problem by mounting the volume on /mnt/windows using the ntfs-3g driver, then copying the ntldr file out of the volume.
# mount -t ntfs-3g /dev/sda1 /mnt/windows
# cp -p /mnt/windows/ntldr ~/ntldr
# mv /mnt/windows/ntldr{,.bak}
# umount /mnt/windows

Then, using ntfsprogs' ntfscp, I re-copied the file back into the file system:
# ntfscp -f /dev/sda1 ~/ntldr ntldr

Then, when I did an ntfsinfo on it it no longer had the compressed attribute present.
Comment: What is the advantage of copying back with `ntfscp` rather than just `cp -p`?

Perhaps check out the `nocompression` mount option.
Answer: You can't boot from an external drive so USB/CD repair is out...can't get network access...um...I don't know of anything that can outrightly fix it from Linux. NTFS support for anything other than reading and writing (and relatively speaking even write support was dodgy until fairly recently) is scant, as the attitude from the Linux community is that NTFS is for native Windows.
I was going to suggest using VMWare or VirtualBox to boot Windows in a VM and give it access to the partition; that could fix it, but you said you didn't have network access to install a VM hypervisor.
If you absolutely need the data, the only thing I could suggest is to physically remove the drive and put it into an adapter to allow it to work as an external USB drive to a machine running Windows and decompress it/run a repair from there. Otherwise, restore from backup...
Answer:
The problem lies in the ntfs-3g driver; it tries too hard to predict what the user wants.

As far as I can see, ntfs-3g did what the user wanted. It compresses the file because the user set the directory compressed.
The problem actually lies in ntfcp which doesn't follow the NTFS specification.
Answer:
(I installed Linux through PXE)
It does not have network access.

I'm confused, but ok..
The key to getting linux installed on any machine is

that most machines have some form of boot selection
the boot process is not tied to the OS (so that you can boot something other than windows)

Given that all of the "normal" boot vectors don't appear to be available to you, I'd say, pull the drive at this point and attach it to another machine.

Follow-up based on comment(s):
So let's review:

External booting is out.  This kills any chance of using a USB stick, CD repair, live boot CD, etc.
Removal of the HDD is possible but highly undesirable
PXE booting is possible under certain circumstances

So it looks like PXE (the same method you used to get the dual-boot up and running) is going to be your easiest bet.  Can you find out what the address is for the PXE server you were using?  If so, you might be able to arrange a bit of a hack...you'll need a computer that can act as a PXE server and a wired hub that isn't attached to any other network.

Set up your PXE server environment
configure the PXE server to have the same address as the one expected by the BIOS
boot the laptop via PXE
supply the laptop with the environment of your choosing

I'm not sure if you can live-boot a windows CD in this manner (I have only done old-style DHCP/BOOTP/TFTP boots, not PXE), but if you could, you might be able to access the repair console, which would be your best bet.  Your second option would be to get a linux live-boot CD into the laptop, and try to repair it from there, although I suspect you'll have limited success.  As a last-ditch effort, you can always migrate the data onto another device (now that the laptop is functional again).
Comment: I meant to say it doesn't have Internet access; I managed to get it hooked up to a wired hub and that's how I used PXE, but through some wonky BIOS settings, it won't allow gateways that aren't the PXE server, nor will it allow the wireless if the system acknowledges the existence of the wired NIC. It's a laptop, and I don't own it. If I open it up, I could get fined upwards of $800. They don't care about the software portion, though.


In composition of two mappings, can the outer mapping access the arguments of the inner mapping?

Question: In composition of two mappings, can the outer mapping access the arguments of the inner mapping?

Here is an example to illustrate my question and my thought.
E.g. $f: \cup_{n \in \mathbb N} \mathbb R^n \to \mathbb R$ is taking
sum, i.e.  $f(x_1, \dots, x_n) =  \sum_i x_i$. Note here $n$ can
vary in $\mathbb N$.
$g: \mathbb R \to \mathbb R$ is to further divide the sum by the
sample size, to get the sample mean , i.e. $g(f(x_1, \dots, x_n)) =
\sum_i x_i/n$.
Is $g(f(x_1, \dots, x_n))$ a composition of the two mappings $f$ and
$g$?
I think that in a  composition of two mappings, the outer mapping
only acts on the codomain of the inner mapping, and thus shouldn't
know the input to the inner mapping. In the example, $g$ should not
know  $n$. So it doesn't make sense to  compose $f$ and $g$.
A revision would be $f: \cup_{n \in \mathbb N} \mathbb R^n \to
\mathbb  R \times \mathbb N$ with $f(x_1, \dots, x_n) =  [\sum_i
x_i, n]^T$, and $g: \mathbb  R \times \mathbb N \to \mathbb R$ with
$g(x,n) = x/n$. Only then the composition of $f$ and $g$ would make
sense to me. What would you think?
The question comes from statistics, where $f$ is a "statistic" which
takes in a sample of size $n$, and $n$ can vary in $\mathbb N$, and
$g$ is a transform on the statistic. To see more of my specific
question, please refer to
https://stats.stackexchange.com/questions/114240/can-the-measurable-mapping-in-the-definition-of-complete-statistics-depend-on-sa.
Here at math.se, I ask a more general question for general math. In
the link to stats.se, I asked a specific question in
statistics. I understand that the answers to the two may be related and different.

Thanks.
Comment: I agree with your reasoning. You cannot compose the original $g$ and $f$ because $g$ doesn't know $n$. Your revision solves the problem.
Answer: There are a couple of problems with your definitions. First, your definition of the domain of $f$ is missing. If you mean the domain to be the set of $\infty$-tuples in which only finitely many entries are nonzero, you should say so. In the way you wrote your domain, I would have asked you what your inclusion of $\mathbb R^n$ into $\mathbb R^{n+1}$ was. I did assume that your union was not a disjoint union, but involved inclusions of smaller spaces into largers.
As you describe $g$, it is not a function, since its value depends on the particular $\mathbb R^n$ you want to consider the argument to lie in. You could make a well-defined $g$ that counted up the number of nonzero entries in your $\infty$-tuple, and divided by that number, and this would certainly be well-defined, but neither linear nor continuous.
I'm having trouble understanding your revision, too, again for the reason that you don't know which $\mathbb R^n$ an element of your "union" lies in.
Comment: Thanks. The domain of $f$ is the union of all finite-length real sequences. Does my notation $f: \cup_{i \in \mathbb N} \mathbb R^n \to \mathbb R$ in the post make sense now? $\cup_{i \in \mathbb N} \mathbb R^n$ is a disjoint union, because $\mathbb R^n$ is the set of all length-$n$ real sequences.
Comment: I don't understand the objection. As long as $f$ assigns a well-defined value to each element of its domain, then it is a valid function. The domain is $D = \cup_{n \in \mathbb{N}}\mathbb{R}^n$, which is a disjoint union. A given $v \in D$ lies in exactly one $\mathbb{R}^n$, so there is nothing wrong with a function with domain $D$ which uses the value of $n$. Analogously, a function with domain $\mathbb{R}$ is free to assign a value that depends on whether the argument is rational or irrational.
Comment: Ah, I see that, being an algebraist, I assumed that you wanted the domain of $f$ to be a group, in which two arguments could be added. So I looked at your (disjoint) union and misinterpreted it as what we call a direct limit of the $\mathbb R^n$'s. As a disjoint union, your domain supports very well the function you described in your revision. I'll leave my "answer" up to show how preconceptions can lead one astray.
Answer: The form of expression ("access the arguments of the inner mapping") suggests to me that you're bringing a programmer's perspective to this subject.  Your formulation in #1 is the correct idea, but you should probably introduce two auxiliary functions s and d as follows: for any x in the disjoint union, d(x) is the dimension of the unique Cartesian space containing x and s(x) is the sum of the components of x.  Then your f(x) is simply (s(x),d(x)) and your g computes the desired ratio from any pair (r,n) of a real r and a natural number n.  That's too tediously pedantic for normal mathematical discourse, but comfy for functional programmers ;-)


Drag nested li doesn't work

Question: I am using jquery-ui for sort and drag https://johnny.github.io/jquery-sortable/.. But it doesn't append in li..
The code is like

$("#example3 ul").sortable({
    placeholder: "ui-state-highlight",
    connectWith: '#example3 ul'
});

Item 1

Item 1 1

Item 1 2

Item 1 3

Item 2

Item 3

Item 4

If I drag item 3 to under item 2 then it doesn't work..How can I solve this?? and also I have to detect that which li goes under which li.. please Help..
Comment: Mh? You say that, it does not work, "if I drag item 3 to under item 2", but 3 is alyways under 2.
Comment: @TobiasK. item 3 under item 2 as a child
Comment: Ah okay, solution is given below ;)
Answer: Problem is, that the empty 'ul'-Tags does not have any space and are hidden (size is equals zero). Just add a small padding to each element, then you can drag your element there. Here, have a look:

$("#example3 ul").sortable({
    placeholder: "ui-state-highlight",
    connectWith: '#example3 ul'
});
#example3 ul{
  padding-bottom: 0.3em;
}

Item 1

Item 1 1

Item 1 2

Item 1 3

Item 2

Item 3

Item 4

Comment: I thought same so but didn't think about padding.. cheers.. thankss :)
Comment: can u tell me how to detect which li goes under which li??
Comment: That is done by the lib's :) I do not detect anything.
Comment: No..I meant that item 3 goes under item 2..now I have to detect that which index goes under which index..

Drag nested li doesn't work


Is this a Pythonic way of structuring modules?

Question: Suppose that I was writing an application that used a number of GUI controls that I'm writing Python code for. Each control is described by a class, and I want to aggregate all of those classes into a module, called gui. As a seasoned C/C++ developer, it makes most sense to me to separate the class implementations by file, like this:
gui/MainWindow.py
gui/Widget1.py
gui/Widget2.py

Where, in the above, the specification of the MainWindow class would be in MainWindow.py. If I lay the files out in this way, however, then the syntax to get at those classes looks like:
import gui
w = gui.MainWindow.MainWindow()

which seems redundant. A way around this limitation is to edit gui/__init__.py to say:
from gui.MainWindow import *
from gui.Widget1 import *
from gui.Widget2 import *

which brings the classes into the gui module namespace. I can then access them as follows:
w = gui.MainWindow()

Is this typically done? Does it have sufficient Pythonicity to be considered appropriate in the community? One drawback that I can see is that I need to be sure to keep gui/__init__.py as I add new submodules to the gui module; I don't like manual steps like that.
Thoughts and/or suggestions of how to better address this would be great.
Comment: You're asking for opinions and recommendations -- that's not considered a constructive question here. Basically, other than being explicit rather than using `*` in `__init__.py`, you've already figured out the usual way.
Comment: Agreed that this butts up against the guidelines. I only thought it appropriate because there seems to be general objective consensus in the community on programming techniques that are considered Pythonic and those that aren't.
Comment: probably you mean packages instead of modules *nitpick*
Answer: Don't import * (unless you're certain there can't be a conflict; this rule can be bent for package authors), but certainly this is otherwise an acceptable solution.
Comment: Additionally, setting `__all__` in each submodule can restrict it further.
Comment: Assuming I have control of the entire `gui` module, I should be able to verify that there won't be any conflicts. the `import *` only brings the symbols into the module-level namespace (unless there's something I don't see).
Answer: Probably you have noted these PEPs and documentation sites before:

PEP8 -- Style Guide for Python Code
PEP 20 -- The Zen of Python
Packages

So the only thing I can add to this is not to use import *. One example:
>>> from pylab import *
>>> any(False for i in range(5))
True

While you validly argue that you know what you are doing. You cannot assume that all of the potential re-users that import * from your package know what they're shadowing while they tear down the final namespace barrier between their and your code.
This gives nasty errors. See also:

Now what happens when the user writes from sound.effects import *? Ideally, one would hope that this somehow goes out to the filesystem, finds which submodules are present in the package, and imports them all. This could take a long time and importing sub-modules might have unwanted side-effects that should only happen when the sub-module is explicitly imported. Docs

and

(import * is not an option ;-) PEP328

and

Namespaces are one honking great idea -- let's do more of those! PEP20

But these are only guidelines/recommendations/personal preferences, and there certainly  exist edge cases when these do not apply.


Why do I have such a small vocal range?

Question: I've been working on my vocal range for a long time now, but it seems to always stay extremely small. The lowest note I can hit is the G below middle C, and the highest note I can hit (in full chest voice, not head voice/falsetto) is the C one octave above middle C.
I am female and the same age as Billie Eilish, so I tend to compare my range to her, and I get really discouraged because she can hit both lower and higher notes than I can with relative ease.
I've taken singing lessons, and my teacher seems to think there's nothing really wrong with my voice, but it's quite frustrating to me because I can't sing the songs I want to.
Is there something wrong with my voice, and is there anything I can do to fix it or work with it?
Comment: People are different.   Everyone has a different "natural" range, and you can't do much about that  (at least until we start applying CRISPR to vocal cords).   Trust your teacher.
Comment: 1.5 octaves in chest voice is not bad at all, work with your voice teacher to try to develop your head voice and mix to extend your range upwards. :)
Comment: *"I am female and the same age as Billie Eilish, so I tend to compare my range to her"* Comparing yourself to one of the biggest vocal artists of the current generation is being a bit tough on yourself, don't you think? I mean, unless you yourself are a professional who is trying to make it in the same industry, it's a bit like if I were to say "Gee, I'm the same height and age as Michael Phelps - why can't I swim as fast as he can?".
Comment: Beats me how we are supposed to know the answer to this!
Comment: My point is, it might be more helpful and healthy to examine one's own progress rather than one's abilities relative to someone who makes their living by being good at singing. Especially in singing, where everyone is physically different and will have unique voices. And especially because range isn't always a good indicator of singing ability.
Answer: Nope, nothing wrong with your voice. An octave and a half chest range isn't bad. The quality of the sound in each register matters more than the range. No use for a 2 octave chest if the bottom is grunty and the top is squeaky.
To work with it, two thoughts:

You should focus on the transition to mix around B4 up to E5. It's quite hard, but if you can navigate the transition smoothly and naturally you can hit higher notes without pure head voice.
Key change songs you sing to fall in the nicest parts of your register. No singer can sing every song they want in the same way as other singers; too much human variation. Lewis Capaldi has a powerful belt note at Someone You Loved's climax - a Bb4. No way I can belt that. But I can belt G4. His voice is higher than mine so I lower the song a minor third to match where the notes land in his and my registers.
Answer: Couple of points:
1) Not everyone naturally has a wide vocal range. It depends on your background (e.g. how has your speaking technique been since you were a child) and biology.
2) You unfortunately cannot affect the biology part: your body is biologically what it is and that restricts how easy it is for you to do things. Of course you can train your body and your muscles, go to the gym etc., but it is still your body and its restrictions, not someone else's. Some people simply might have a (biological!) body construction that does not allow a wide vocal range, even though you practice singing for 20 years (I have met male singers that are of the same voice type as me and they struggle with high notes, whereas I can easily sing half an octave higher than them, and that is a lot)! So, some people might have a favorable body construction and they develop a wide range and a good technique in just a few years and just a little advice. That means they have a biologically favourable construction, are more able to control their muscles (note that singing is about muscles and muscle control), and most importantly, that does not mean that you are less hard-working than them: you are just different.
3) Be patient. It usually takes at least three years to automate the control of one muscle group. After that, you must automate another muscle group, and that is how you make progress with your singing. Same goes with vocal range: you need correct instruction and a good voice teacher to train the muscles for healthy singing. Gradually you will develop a wider range but it will most likely not happen in a year. It is like learning to dance: if you have danced and watched good dancers dance or maybe danced with your parents who are awesome dancers since you were a child, you will have a better basis for dancing. Or if you are like me, who has a stiff back and shoulders and never danced until I was 24 years old, it would take 2-3 years to learn the basics of posture and body muscle control. That is what I went through when learning to dance. My singing has gradually gotten better and my vocal range developed for at least five years when I started singing lessons and I'm still learning (of course) new things about singing.
Answer: Your range is small because you aren't properly accessing your head voice and mix and are just using a pushy chest in its place (which is unhealthy). Your head voice (if properly developed) is part of your full range and will probably give you at least an octave up top if not more (considering that your lowest note is a G3 indicating that you're probably a soprano). And by the way, Billie Eilish always uses head voice or even falsetto to sing her higher notes, otherwise she'd be yelling which she doesn't do, she sings them softly and beautifully - using head voice.
Answer: If a song is too high for my range I set key "up" two notes. It's like the previous comment said it's more about quality. You can sing any key in a lower octave, but you will sound like you are doing a silly lower octave. So set the song on a higher key and your low octave will rise with it till it sounds comfortable.
You won't even notice you changed the key till you hear it again and once again see that it is a difficult range. Try 2 steps up for a while then 1 for a while then try the original key. The brain can trick itself if given enough time to adjust incrementally.
Comment: This seems like bad advice if a song truly is *out of one's range*. If the problem is psychological and singing up high is in need of practice, then maybe that would work similar to wearing weighted clothes while training for a marathon. However, singing is very physical. If a song is truly out of someone's range, no amount of mental trickery is going to somehow make their vocal cords thinner. It also seems like an easy way to injure your voice by attempting things that are far beyond your capabilities - if you can't hit a note, you ought to work on developing that part of your range instead.

Why do I have such a small vocal range?


Is it possible to get different result of bootstrap statistics using different software?

Question: Is it possible to get different result of bootstrap sample statistics using different software ?
Comment: Absolutely, you can get different results from the same software. It's all about the randomness (and implementation).
Comment: Will the difference will still exist for large number of bootstrap sample? like e.g more than 5000 samples?
Comment: Depends a lot on the data, how variable it is. 5000 is a good number, but you would really have to try to know. If you run it twice with 5000 and get very similar results then you are probably good.
Comment: [Searching our site](https://stats.stackexchange.com/search?q=bootstrap+random+differ*+score%3A5+is%3Aanswer) produces several good answers.
Answer: You can get different results from the same software! Run the boot package in R with set.seed(1) before your bootstrap code and then with set.seed(2). Your results should differ at least a little.
If you go run a bootstrap in Python (SAS, Stata, etc), you will be taking yet a third set of bootstrap samples, giving a third result.
Answer: Suppose I have a sample x of size $n = 900$ with the
summary statistics below:
summary(x);  sd(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
  31.68   45.30   50.08   49.96   54.42   70.49
[1] 6.858865   # sample SD

Then the (estimated) standard error is $S/\sqrt{n} = 6.859/30 = 0.229.$ Assuming that data are normal, we would have
the 95% t confidence interval for $\bar X \pm 1.968 S/\sqrt{n},$ where $1.968$ cuts probability $0.025$ from
the upper tail of (symmetrical) Student's t distribution with 899 degrees of freedom.
qt(.975, 899)
[1] 1.962606

By computation or from the t.test procedure in R, we get the CI $(49.51, 50.41).$
The margin of error of this CI is about $1.963(0.229)=0.4495.$
t.test(x)$conf.int
[1] 49.51063 50.40805
attr(,"conf.level")
[1] 0.95

If the sample truly is from a normal distribution, then this
is a valid 95% CI for the population mean $\mu$ and we are done.
However, if we have reason to doubt that the data are normal
we might find a 95% nonparametric bootstrap CI for $\mu.$
There are many possible styles of bootstrap CIs. I will
illustrate one.
The observed sample mean is $49.96.$
a.obs = mean(x);  a.obs
[1] 49.95934

We take many (3000) re-samples of size 900 with replacement from x in order to get an idea of the sampling error of the sample mean. For each re-sample we find of its mean from the observed mean of the original sample.
set.seed(2021)
d = replicate(3000, mean(sample(x,900,rep=T))-a.obs)

The deviations $d$ are mainly between $\pm 0.452.$
which is not much different from the margin of error of
the 95% t confidence interval above. The 95% nonparametric
bootstrap CI is $(59.51, 50.41),$ which id agreement
with the 95% t CI above.
UL = quantile(d, c(.975,.025))
UL
     97.5%       2.5%
 0.4520234 -0.4522723

 a.obs - UL
   97.5%     2.5%
49.50732 50.41161

If I run this same bootstrap procedure twice again (unknown seeds)
I get bootstrap CIs $(49.49425,\, 50.42611)$ and $(49.49007,\, 50.41792),$ which are about the same as the first one above, for practical purposes.
Why not all bootstrap CIs from a sample are exactly the same:

Because bootstrapping depends on random re-sampling you
can't expect exactly the same result every time. Experience has shown that 2000 or 3000 re-samples are
enough to get nearly reproducible results.

If you try
bootstrapping with very small samples, you might get
a larger variety of bootstrap CIs.

Also, if you use a
different style of nonparametric bootstrap (there are several possibilities) you may get
get a somewhat different result.

Finally, if the data are not normal, you can't expect
a bootstrap CI to give the same results as a t CI. In this
case the bootstrap CI is usually preferred, because the t CI
assumes normal data.

Notes: (1) The fictitious data for my bootstrap CIs above were
normal, sampled in R as follows:
set.seed(407)
x = rnorm(900, 50, 7)

(2) It is important to understand that re-samples used in
bootstrapping do not provide additional information. They are part of the analysis, not part of the experiment. In the example above, the re-samples took the place of (a) the formula for the estimated standard error of the sample mean and (b) looking in a printed t table to find the constant 1.963.


Proof of claim on orthonormal elements in an inner product space

Question:
Let $X$ be an inner product space and $\{e_{n}\}_{n=1}^{\infty} \subset X$ be an orthonormal set. Show that
  $$ \sum_{n=1}^{\infty}|\langle x,e_{n}\rangle\langle y, e_{n}\rangle| \leqslant \|x\|\|y\| $$
  for all $x,y \in X$.

I am seeking a hint on how to get started. I imagine this involves some manipulation of the $RHS$ to get down to inequality on the $LHS$. Any insight is appreciated.
Using the hints below:
$$ \sum_{n=1}^{\infty}|\langle x,e_{n}\rangle\langle y, e_{n}\rangle| \leqslant \sum_{n=1}^{\infty}\|x\|\|y\|$$
I do not quite see the connection to Bessel's inequality from here.
Comment: Hint: Bessel inequality.
Comment: If for all $n$ you have $a_n\ge0$ and for all $N$ you have $\displaystyle \sum_{n=1}^N a_n \le M$ then you also have $\displaystyle \sum_{n=1}^\infty a_n \le M$. $\qquad$
Answer: Cauchy Schwarz says that
$$
\left(\sum_{n=1}^\infty\langle x,e_n\rangle\langle y,e_n\rangle\right)^2
\le\sum_{n=1}^\infty\langle x,e_n\rangle^2\sum_{n=1}^\infty\langle y,e_n\rangle^2\tag{1}
$$
If we are in a Hilbert Space, we can apply Bessel's Inequality. If not, we might want to reprove it for an inner product space.
For any $1\le k\le n$,
$$
\left\langle x-\sum_{j=1}^n\langle x,e_j\rangle e_j,e_k\right\rangle
=0\tag{2}
$$
If we set
$$
u=\sum_{j=1}^n\langle x,e_j\rangle e_j\tag{3}
$$
$(2)$ and $(3)$ show that $\langle u,x-u\rangle=0$. Therefore,
$$
\begin{align}
\|x\|^2
&=\langle x,x\rangle\\
&=\langle u+(x-u),u+(x-u)\rangle\\
&=\|u\|^2+\|x-u\|^2\\
&\ge\|u\|^2\\
&=\sum_{j=1}^n\langle x,e_j\rangle^2\tag{4}
\end{align}
$$
Letting $n\to\infty$, $(1)$ and $(4)$ show that
$$
\left(\sum_{k=1}^\infty\langle x,e_k\rangle\langle y,e_k\rangle\right)^2
\le\|x\|^2\|y\|^2
$$
Answer: Hint: Use the Cauchy-Schwarz inequality on the left hand side, and then apply Bessel's inequality.


Why my code is sending mail two times?

Question: I have written this code using node-mailer to send mail. When I start the server in cmd and go to localhost:8080, it sends to mail two times. Why this is so?
var http = require('http');
var edge = require('edge');
var port = process.env.PORT || 8080;

var params = {
    connectionString: "Data Source=localhost;Initial Catalog=node-test;Integrated Security=True",
    source: "SELECT * FROM emailtable"
};
var nodemailer = require("nodemailer");

var smtpTransport = nodemailer.createTransport("SMTP",{
   service: "Gmail",
   auth: {
       user: email,
       pass: password
   }
});
var getTopUsers = edge.func('sql', params);

function logError(err, res) {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.write("Error: " + err);
    res.end("");
}
var minutes = 1, the_interval = minutes * 60 * 1000;
var emailText = "";
http.createServer(function (req, res) {
    res.writeHead(200, { 'Content-Type': 'text/html' });
    //setInterval(function() {
    getTopUsers(null, function (error, result) {
        emailText=""
        if (error) { logError(error, res); return; }
        if (result) {
            res.write("
");
            result.forEach(function(user) {
                res.write("
" + user.emails + "
");
                emailText = emailText + user.emails + " ";
                //console.log(emailText);
            });

            res.end("
");

        }
        else {
        }
        console.log("just mailed")
smtpTransport.sendMail({
   from: email, // sender address
   to: email, // comma separated list of receivers
   subject: "Hello", // Subject line
   text: emailText // plaintext body
}, function(error, response){
   if(error){
       console.log(error);
   }else{
       console.log("Message sent: " + response.message);
   }
});
    });
    //}, the_interval);

}).listen(port);

    console.log("Node server listening on port " + port);

Answer: When you want to open a page in a browser, it usually sends two requests to the server, one for fetching the favicon.ico and another one for the page it self.

Why my code is sending mail two times?


Probably $x_1 \leq x_3 \leq x_2$

Question: From set  ${1,2...n}$ we choose three random different numbers, $x_1, x_2,x_3$. If $x_1 \le x_2$, find probability $x_1 \le x_3 \le x_2$.
I set A is event  $x_1 \leq x_3 \leq x_2$, and B $x_1 \leq x_2$, and find P(AB)=$\frac{1}{6}$, P(B)=$\frac{1}{2}$. Is it okay?
Comment: The probabilities depend on the value of $n$. Take an extreme case $n=1$. Then $\Pr(A) = \Pr(B) = 1$. You are correct that in the limit (as $n\to \infty$) the probabilities are $\Pr(A) = 1/6$, $\Pr(B) = 1/2$.
Comment: You are correct without replacement (I guess this is what the "different" means).
Comment: If the 3 numbers are *different*, you can't have,say, $x_1 \leq x_2$
Answer: The problem has two different solutions, depending on whether you take them with replacement or without it (your question is quite ambiguous in that sense: the word 'different' suggests that they are taken without replacement, and the sign '$\leq$' - that they are taken with replacement):
Solution for numbers taken without replacement:
$$P(x_1 < x_2) = \frac{1}{2}$$
$$P(x_1 < x_3 < x_2) = \frac{1}{6}$$
$$P(x_1 < x_3 < x_2| x_1 < x_2) = \frac{1}{3}$$
Solution for numbers taken with replacement:
$$P(x_1 \leq x_2) = P(x_1 = x_2) + P(x_1 < x_2) = P(x_1 = x_2) + \frac{1 - P(x_1 = x_2)}{2} = \frac{1}{n} + \frac{n-1}{2n} = \frac{n+1}{2n}$$
$$P(x_1 \leq x_3 \leq x_2) = P(x_1 < x_3 < x_2) + P(x_1 = x_3 < x_2) + P(x_1 < x_3 = x_2) - P(x_1 = x_3 = x_2) = \frac{1 - P(x_1 = x_3 < x_2) - P(x_1 < x_3 = x_2) + P(x_1 = x_3 = x_2) }{6} + P(x_1 = x_3 < x_2) + P(x_1 < x_3 = x_2) - P(x_1 = x_3 = x_2) = \frac{1}{6} + \frac{5}{6}P(x_1 = x_3 < x_2) + \frac{5}{6}P(x_1 < x_3 = x_2) - \frac{5}{6}P(x_1 = x_3 = x_2) = \frac{1}{6} + \frac{5(n+1)}{12n^2} + \frac{5(n+1)}{12n^2} - \frac{5}{6n^2} = \frac{n+5}{6n}$$
$$P(x_1 \leq x_3 \leq x_2| x_1 \leq x_2) = \frac{n+5}{6n}\frac{2n}{n+1} = \frac{n+5}{3(n+1)}$$

Probably $x_1 \leq x_3 \leq x_2$


Python PULP How To Get Block of Values Same

Question: I am trying to find a way in PULP to code such that all the values within a block specified by the IF statement at end (i.e. if j >= 7 and j <= 21) are the same.
I am not sure where to start as I am more familiar with coding in Xpress Mosel and not PULP language.
model += pulp.lpSum([YALL[(i,j)] for i in range(YPER) for j in range(HE) if j >= 23]) == pulp.lpSum([YALL[(i,j)] for i in range(YPER) for j in range(HE) if j <= 24])

The code above is a very rudimentary way I have to set the periods of 23 and 24 equal to each other.  I am trying to have it such that I can specify the start period (i.e. 7) and the end period (i.e. 22) and have all values between those periods equal to one another.  Code I have so far is below but doesn't do what I need.
model += pulp.lpSum([YALL[(i,j)] for i in range(YPER) for j in range(HE) if j >= 7]) == pulp.lpSum([YALL[(i,j)] for i in range(YPER) for j in range(HE) if j <= 21])

Comment: Are you able to share a [mcve]? It would make it a lot easier to suggest the best way to do what you're after.
Answer: I think you just need to add constraints YALL[(i,j)] == YALL[(i,j+1)] for j in 7, ..., 20:
for i in range(YPER):
    for j in range(7, 21):
        model += YALL[(i,j)] == YALL[(i,j+1)]

I might be getting the PuLP syntax slightly wrong but hopefully you get the idea.
Comment: Works perfectly! I knew it had to be something along these lines but wasn't sure.  Thanks!!


Can i use MAX function for each tuple in the retrieved data set

Question: My table result contains fields:
id    count
____________
1     3
2     2
3     2

From this table i have to form another table score which should look as follows
id    my_score
_____________
1     1.0000
2     0.6667
3     0.6667

That is my_score=count/MAX(count) but if i give the query as
create TEMPORARY TABLE(select id,(count/MAX(count)) AS my_score from result);

only 1 st row is retrieved.
Can any one suggest the query so that my_score is calculated for all tuples.
Thanks in advance.
Answer: SELECT
    a.ID,
    a.count / b.total
FROM result as A
   CROSS JOIN (SELECT MAX(Count) AS Total From Result) AS B

B only returns one row so you want to take the Cartesian product of the table against its own aggregate to get your end value.
Answer: Not sure if this works in mysql, but try:
select id, count / (select max(count) from result) as my_score
from result

Comment: +1 It works in MySQL with regular tables, with temporary tables it does not (see http://dev.mysql.com/doc/refman/5.0/en/temporary-table-problems.html).
@kshama, is result temporary too? if yes what's the source of it?
Answer: I don't think you can apply an aggregation function to each row in one step.  Use a stored procedure to do the calculation in two steps -- calculate the max and store it in a variable, then do your selection and divide the count by the variable.  Alternatively, you could use a subquery, but I don't really see that as an improvement.
create procedure calculate_score
begin
    declare maxcount decimal(6,4);

    set maxcount := select max(count) from result;

    select id, count / maxcount as score from result;
end

Note: I'm not sure if MySQL will implicitly handle the data conversions from int to decimal or what types your columns are.  If the data conversions need to be handled manually, you'll have to adjust the above.
Answer: I would be hesitating because of probable performance issues but semantically this should work:
select
    a.id
    , a.count / b.count
from
    result a cross join result b
where
    b.count = (select max(count) from result)

Edit: @eftpotrm has a much more elegant solution!


IntelliJ: display folder structure like NetBeans

Question: I prefer IntelliJ over NetBeans for Maven Java development, but I'm wondering how to get the displayed project structure that NetBeans has into my IntelliJ project. Please see the image below.

Edit: some more information about what I want.
IntelliJ displays the actual folder layout: the folder webapp is under root/main/webapp. However, NetBeans recognizes that this is an important folder and pulls it up to project level, so virtually root/webapp. This is just a visual thing: in reality the folders remain the same.
In this example, the folder is only 1 tier lower in IntelliJ than in NetBeans. But sometimes, the webapp folder is in root/./././webapp (example). Having to open 5 folders before finally ariving at the webapp folder is annoying and by pulling it up it is clearer and it saves time.
Comment: Could you please explain what do you mean in the image? What it is that you have and what do you expect?
Answer: Go to settings(as shows in image) of Project Structure.

Then select Flatten Packages and Hide Entry Middle Packages. You will get structure like Netbeans in the IntelliJ
For more details please visit: https://www.jetbrains.com/help/idea/2017.1/project-tool-window.html


Given $A,B$ are $n \times n$ matrices, $(A+B)^2=A+B,r(A+B)=r(A)+r(B)$. Prove: $A^2=A,B^2=B$

Question: Given $A,B$ are $n \times n$ matrices, $(A+B)^2=A+B,r(A+B)=r(A)+r(B)$.
Prove: $A^2=A,B^2=B$
I don't know how to use $r(A+B)=r(A)+r(B)$
Comment: To begin with, we have $r(A+B)\le r(A)+r(B)-\dim\left(im(A)\cap im(B)\right)$ because $im(A+B)\subseteq im(A)+im(B)$. The rank condition therefore implies that $im(A)\cap im(B)=0$.
Comment: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Answer: Following what user1551 said in the comments, we have: $\operatorname{Im}(A)\cap\operatorname{Im}(B) = \{0\}$.
(You did not specify the base field, so I'll just take any field $\mathbb{K}$.)
Denote the transpose of a matrix $M$ as $M^T$.
Since $r(M) = r(M^T)$ for all matrices $M \in \mathcal{M}_n(\mathbb{K})$ (one way to envision it is that the rank of a matrix $M$ is the maximum $k$ such that there exists an invertible minor $N$ of $M$ of size $k \times k$, and then see that for all minors $N$, $N^T$ is a minor of $M^T$, as well as recall that $\det(N) = \det(N^T)$), we also have:
$$r\left(A^T + B^T\right) = r\left((A+B)^T\right) = r\left(A^T\right) + r\left(B^T\right)$$
Therefore, from the same arguments as before: $\operatorname{Im}\left(A^T\right)\cap\operatorname{Im}\left(B^T\right) = \{0\}$.
Now, let $x \in \mathbb{K}^n$. We get:
$$Ax + Bx = \left(A+B\right)^2x = \left(A^2 + AB + BA + B^2\right)x = A\big((A + B)x\big) + B\big((A+B)x\big)$$
Thus, from $\operatorname{Im}(A)\cap\operatorname{Im}(B) = \{0\}$, we obtain:
$$\begin{cases}Ax = A\big((A + B)x\big)\\ Bx = B\big((A+B)x\big)\end{cases} \quad\Rightarrow\quad \begin{cases}A^2x = (A - AB)x\\ B^2x = (B - BA)x\end{cases}$$
Since this holds for any $x \in \mathbb{K}^n$, we can conclude that:
$$\begin{cases}A^2 = (A - AB)\\ B^2 = (B - BA)\end{cases}$$
By transposing everything, this grants:
$$\begin{cases}(A^2)^T = (A - AB)^T\\ (B^2)^T = (B - BA)^T\end{cases} \quad\Rightarrow\quad \begin{cases}(A^T)^2 = A^T - B^TA^T\\ (B^T)^2 = B^T - A^TB^T\end{cases}$$
Again, let $y \in \mathbb{K}^n$. We have:
$$\begin{cases}A^T(A^Ty) = A^Ty + B^T(-A^Ty)\\ B^T(B^Ty) = B^Ty + A^T(-B^Ty)\end{cases}$$
Hence, this time with $\operatorname{Im}\left(A^T\right)\cap\operatorname{Im}\left(B^T\right) = \{0\}$:
$$\begin{cases}A^T(A^Ty) &= A^Ty \\ 0 &= B^T(-A^Ty)\\ B^T(B^Ty) &= B^Ty \\ 0 &= A^T(-B^Ty)\end{cases}$$
Due to this holding for all $y \in \mathbb{K}^n$, the first and third lines give us $(A^T)^2 = A^T$ and $(B^T)^2 = B^T$, therefore: $A^2 = A$ and $B^2 = B$, and we have finished. (The second and fourth lines give more precision to the result: the image of $A$ is a subspace of the kernel of $B$ and vice-versa).
Comment: It looks correct, but it can be shortened significantly. From $(A+B)^2=(A+B)$, we obtain $A(A+B-I)=-B(A+B-I)$. Hence both sides are zero, because $\operatorname{im}(A)\cap\operatorname{im}(B)=0$. That $A(A+B-I)=0$ means that $AB=A-A^2$ and in turn,
$$
A^T(I-A^T)=B^TA^T
$$
However, as row rank equals column rank, the rank condition also implies that $\operatorname{im}(A^T)\cap\operatorname{im}(B^T)=0$. Thus from the displayed equality above we infer that $B^TA^T=0$, i.e., $AB=0$. Hence from $A(A+B-I)=0$ we may deduce that $A^2=A$.
Comment: Thank you so much for catching that!
Comment: @EwanDelanoy this should be correct now?
Comment: @BrunoB I agree with user1551 : I think it's correct now, +1
Answer: It seems like you are working on your linear algebra worksheet(very similar to mine).@Burno B 's answer is beautiful,but there's also a tricky method just using the properties of the rank of matrix.
We know that the elementary row (or column) transformation doesn't change the rank of a matrix,so we have
$$
\begin{pmatrix}
A&0&0\\
0&B&0\\
0&0&I-A-B
\end{pmatrix}
\sim
\begin{pmatrix}
A&0&0\\
0&B&0\\
A&B&I-A-B
\end{pmatrix}
\sim
\begin{pmatrix}
A&0&A\\
0&B&B\\
A&B&I
\end{pmatrix}
$$
$$
\sim
\begin{pmatrix}
A-A^2&-AB&0\\
-BA&B-B^2&0\\
A&B&I
\end{pmatrix}
\sim
\begin{pmatrix}
A-A^2&-AB&0\\
-BA&B-B^2&0\\
0&0&I
\end{pmatrix}
$$
and
$$
r(A)+r(B)+r(I-A-B)=r\begin{pmatrix}
A-A^2&-AB\\
-BA&B-B^2
\end{pmatrix}+r(I)=r\begin{pmatrix}
A-A^2&-AB\\
-BA&B-B^2
\end{pmatrix}+n
$$
Also,
$$
(A+B)^2=(A+B)\implies r(A+B)+r(I-A-B)=n
$$
$$
r(A+B)=r(A)+r(B)\implies r(A)+r(B)+r(I-A-B)=n
$$
which implies
$$
r\begin{pmatrix}
A-A^2&-AB\\
-BA&B-B^2
\end{pmatrix}=0
$$
so
$$
A^2=A,B^2=B,AB=BA=0
$$
Comment: Honestly, this is just as neat! (+1)
Comment: This is really interesting, but how did you come up with it?


Retornar a linha de uma tabela quando o valor do count(campo) for 0 no MySQL

Question: Preciso que mesmo quando a busca não gere resultados, seja exibido o valor 0, tentei com a seguinte busca SQL, mas sem êxito:
    SELECT c.categoria as categoria,
           CASE WHEN COUNT(c.categoria) IS NOT NULL THEN count(c.categoria) ELSE 0 END AS quantidade
    FROM anuncio a
        RIGHT JOIN categoria c ON a.categoria = c.categoria WHERE autorizado = '1'
    GROUP BY c.categoria;

Comment: e você quer contar o que afinal ? os anuncios ou as categorias ?
Comment: https://forum.imasters.com.br/topic/546600-union-com-com-o-mesmos-dados-e-campo-diferentes/ veja se ajuda
Comment: Quero a quantidade anúncios por categoria.
Answer: select codigo,count(id) as QUANTIDADE from(
select id,nome from (
select id from categoria
where ...
group by ID) as a
left join tabela on a.id=id
group by id,nome) as b
left join tabela on b.id=id
group by código, id

Penso que isto o possa ajudar
Answer: use o coalesce como um subselect:
select coalesce(
 (SELECT c.categoria as categoria,
        COUNT(c.categoria)AS quantidade
    FROM anuncio a
        RIGHT JOIN categoria c ON a.categoria = c.categoria WHERE autorizado = '1'
    GROUP BY c.categoria
), 0 )

Answer: Você pode estruturar a sua query de uma forma diferente, olha este exemplo do próprio stack:
SELECT
       IFNULL(SUM(TOTAL), 0) AS total,
       IFNULL(SUM(5STAR), 0) AS FiveStar,
       STORE,
       DATE
FROM `table`
WHERE DATE >= '2012-02-24'
GROUP BY TOTAL

Desta forma buscará o total e resultará 0 caso não encontre resultados.


How to determine the convergence of $\sum \frac 1 {n!}\left(\frac n e\right)^n$ using Raabe's test

Question: The series is given by :

$$ \sum_{n \geq 1} \frac  1 {n!}\left(\frac n e\right)^n$$

I tried to test its convergence by using a variant of Raabe's test as shown in Bartle's "Introduction to Real Analysis" book.
However, the limit that I got is $-\infty$, which I think fails Raabe's test (?)
I also tried using comparison test, limit comparison test, ratio test, which all failed.
Could anyone help me determine whether this series converges or not?
Comment: If it's a series, then write it as a series, not  a sequence.
Comment: So essentially there is a $n^n$ in the numerator?
Comment: Is it an infinite sum ? Jonelle ?
Comment: yes, this is an inifinite sum, and yes there is an n^n in the numerator
Comment: This is easy with Stirling.
Answer: The term test is also inconclusive since $(n/e)^n/n! \to 0$ as $n \to \infty$.  However, you can quickly establish divergence of the series since Stirling's approximation gives $(n/e)^n/n!  \sim C/\sqrt{n}$.
If you are interested, Raabe's test states that given a series $\sum a_n$ with positive terms, if we obtain
$$\tag{*}\lim_{n\to \infty}\left(n \frac{a_n}{a_{n+1}} - n - 1 \right) = r,$$
then the series converges if $r > 0$ and diverges if $r < 0$. The case where $r = 0$ is inconclusive.
In this case, $a_n = (n/e)^n/n!$, so
$$\frac{a_n}{a_{n+1}} = \frac{(n+1)!e^{n+1}}{(n+1)^{n+1}}\frac{n^n}{n!e^n} = \frac{e}{(1 + 1/n)^n},$$
and
$$n \frac{a_n}{a_{n+1}} - n - 1 = n\left(\frac{e}{(1+1/n)^n}- 1 \right) - 1.$$
Since $(1 + 1/n)^n \to e$, the term in parentheses converges to $0$. Also, this shows why the ratio and root test are inconclusive -- since $a_{n+1}/a_n \to 1$.
It can be shown that
$$\frac{e}{2n +2} < e - (1 + 1/n)^n < \frac{e}{2n+1}.$$
(A proof of this inequality is given in Problems and Theorems in Analysis I by Polya and Szego.)
Hence,
$$\frac{e}{(1+1/n)^n} \frac{n}{2n+2} - 1< n\left(\frac{e}{(1+1/n)^n}- 1 \right) - 1 < \frac{e}{(1+1/n)^n} \frac{n}{2n+1} - 1.$$
Since the limits of the left-hand and right-hand sides are both $-1/2$, it follows by the squeeze theorem that the limit in (*) is $r = -1/2$ and we can conclude that the series diverges.
Comment: wow, this is smart, especially the inequality you introduced to come up with the limit of the expression
Comment: @JonelleYu: You're welcome.  I'll edit your post to add  $\sum$ to show this is a series question, This seems to have caused confusion although you state explicitly series.


Passing Variables Between Components In React.js

Question: Background
I am trying to pass a css class name in a variable called styling to my className on the tr element. I think my problem here is that I have yet to grasp the concept of props.
Basically I have a component written in ES6 and I want to change the css class of a tr based on the value of the child td.
Problem
With my current code I get

styling not defined

But this does console.log all of the appropriate css class out so I know everything is good minus the actual class being added to className of the tr tag. For some reason it is not passing the variable inside from componentDidMount to render.
Question
Please explain to me how to pass variables throughout my React.js component / components?
Example
    class TableDisplay extends React.Component{
                    constructor(props) {
                    super(props);
                }
                componentDidMount() {
                    var styling = " ";
                var tCells = document.getElementsByTagName("td");
                for (var i = 0; i < tCells.length; i++) {
                        if(tCells[i].innerHTML == "Approved") {
                        console.log("bg-success");
                        styling = "bg-success";
                    } else if (tCells[i].innerHTML == "Denied") {
                        console.log("bg-danger");
                        styling = "bg-danger";
                    } else {
                        console.log("bg-plain");
                        styling = "bg-plain";
                    }
                }
                }
            render() {
                return

Title	Status	Created	Updated	Delete

                      {Object.keys(requests).map(function(key) {
// Problem appears to be here
                           return

                                {requests[key].title}
                                {requests[key].status}
                                {requests[key].created_at.slice(0, 10)}
                                {requests[key].updated_at.slice(0, 10)}
                                Delete

;

                                        })}

;
           }
    }

What I Have Tried
I did think this could be a scope or closure issue. So I have tried.
Global
Making styling a global variable. That did not work.
This
Tried to use this.styling. That did not work either.
Moved Inside Render
I moved the loop inside render. When I did that it added the styling variable to the className but only the first initial declaration of styling = " ";. So basically it just made every tr have a space for the className.
Comment: Does your "Approved" and "Denied" text comes from requests[key].status or requests[key].title? If so have you tried moving it inside the render function but comparing with those variables instead of innerHTML?
Comment: I have tried moving it. At the bottom of my question I explain the outcome. Basically if I remove styling from the className it runs the app and it outputs all the correct data to the console. But when I move the loop inside render it just adds a space in the className spot for every tr tag
Answer: Assuming your "Approved" and "Denied" text is from requests[key].status. can't you just do something like this:
{Object.keys(requests).map(function(key) {
    let styling = "bg-plain";

    if (requests[key].status === "Approved") {
        styling = "bg-success";
    } else if (requests[key].status === "Denied") {
        styling = "bg-danger";
    }

    return

        {requests[key].title}
        {requests[key].status}
        {requests[key].created_at.slice(0, 10)}
        {requests[key].updated_at.slice(0, 10)}
        Delete

;

})}

Comment: Thanks lol. Jesus man. I'm spending so much time trying to figure out how react's scope works I glazed right over the obvious.
Comment: I must say though. I am still worried why I cant get scope or props to work. If you have any input into that let me know.
Comment: @wuno From your "Example Based On Answer" edit. I can see why `this.state.styling` isn't defined. Since you are not using the ES6 arrow function as callback, your `this` variable isn't in the outer scope. From ES5, people like to have `var self = this` outside then use `self.state.foo` in the callback. Maybe you should take a look at this: http://jsrocks.org/2014/10/arrow-functions-and-their-scope/ for this beautiful arrow functions.
Comment: Ok awesome thanks a lot man! So you mean I should of had an arrow function inside componentDidMount?? WOuld you mind showing me what you mean?
Answer: You'll want to use state in your component so that you can access these variables that are modified at different times during the component's lifecycle.
When you create TableDisplay set the initial state in the constructor like so:
class TableDisplay extends React.Component{
  constructor(props) {
    super(props);
    this.state = {styling: ""};
  }

and based on the conditional in your componentDidMount function above change this state by using the setState function like so:
this.setState({ styling: "what ever you want to change it to"});

and access this state by using this.state.styling in your render function like so:

The reason your render function couldn't use the styling variable because it's not in the same scope. Once componentDidMount was done running, the variable styling disappeared. So React's state allows you to access variables throughout your component.
Comment: I updated my question using your explanation. Please look at the bottom of my question and tell me if I am doing what you suggested. I am gettting state is not defined


How to find the number of downloads for an app in the itunes store programmatically?

Question: NOTE: No third party websites like Appfigures etc please.
I know how to check number of download of an app like this

Just go to iTunes Connect, and hit "Sales and Trends". It should give
  you the information you want.

But I would like to know,
Is there any programmatic way through which I just pass app-id of my app and get its number of download programmatically?
Answer: Yes, you can download Apple's Autoingestion tools. For more info check the official documentation. However, AFAIK, you cannot filter by a specific Apple-ID, you get the entire report with all your apps.
The docs go through all the details for requesting and parsing the report but a basic example to get all the daily sales since 2015-05-01 and get you started trims down to:

Fill username and password in the autoingestion.properties file
In your terminal run
<code>$ java Autoingestion autoingestion.properties VENDOR_ID Sales Daily Summary 20150501</code>

The VENDOR_ID can be retrieved from the iTunesConnect under Sales & Trends > Reports (should be a 8-digit number beginning with 8).
You'll get a .tar.gz which contains a tab separated .txt with the report.
There are plenty wrappers around the autoingest tool which might come handy in your case. One in Ruby which also contains a parser is itunes_ingestion
Comment: i read you doc...seems it will work but for fixed user info added in plist.
Is any useful dock from where we can get multiple apps detail just using apple id etc ?
Answer: iTunes Connect doesn't have any official API, but there are few options for you. For sales info you may find useful itcreport tool that mention some iTunes Connect Reporting API, but I couldn't find any info about it. It's opensource, so if that doesn't fit your needs - you can write your own implementation. For non-sales info you may find this SO answer helpful.


Check if an arraylist has values from a predefined range of values only

Question: I have an arraylist which has String values which may be duplicated too, no issues there.
Also, I have a pre-defined range/options of those values, which are allowed.An arraylist having allowed values, but I can use any datastructure if I get my probem solved.
I just need a way to make sure my arraylist has no alien values, if so, it POINTS OUT which index number if faulty or are faulty if multiple.
I have tried .contains() method and many others, but I can't get the Faulty index position.
I am ready to use any datastructure or any method, but need that FAULTY INDEX NUMBER.
FYI, I am using java and the arraylist has values from a ResultSet.
Comment: Use a for loop. If the current element doesn't match the range, return the current index. https://docs.oracle.com/javase/tutorial/java/nutsandbolts/for.html
Comment: for loop is ok for infinite list given that your sistem can accomodate the resources :)
Comment: How else would you do that without looping through the elements? Note that indices start at 0, not 1.
Comment: If you have nested for loops, you're not splitting the problem into simpler pieces. Start by writing a method which checks if **one** value is in the range. Then use a loop and call that method for each element of the list.
Comment: Could you provide an example as well?
Comment: What is faulty for you?
Comment: current_arraylist : a,d,b,c
allowed values : a,b,c
result: index number 2 is faulty !
Comment: is for loop okay if I have to check 40k+ values ??
Comment: @JBNizet I tried using a nested for loop,but aint successful, which arraylist should be on the outer for loop ?
Comment: There is no way to check each element without, well... checking each element. The best you can do is a for loop, and check each element's validity with something like `allowed.contains(value)`. You may want to consider making `allowed` a `Set`.
Comment: But I have to iterate both arraylists, I am not getting the way to do that, viz the implementation part !
Comment: That's what contains and indexOf will do behind the scenes. Since you want each position where the dupe occurs, you need to use a for loop.
Comment: I'll try, thanks, and I have been trying before posting the ques, thanks
Answer: I agree that there is always a better approach to things, but I somehow solved this (please ignore approach as answer is exactly what I wanted).
I had 2 arraylists A and B and I wanted to know which element of A(Allowed+faulty values) is not in B(allowed values) and most importantly at which position.
What I did was :
--> Made an arraylist A_temp viz a copy of A.(same elements as A).
--> Used A.removeAll(B).
 This made my A contain only error values(which is what I wanted to point to).
--> Finally, I printed out A_temp.indexOf(A.get(index)).
Answer: You can probably do a this task using set (based this on your comment example with string):
List <String> all = ...
Set <String> allowed = ..

for(String current : all){
   if(!allowed.contains(current){
      // out of range
   }
}

This approach is faster w.r.t the one which iterates both lists linearly taking advantage of optimized search in a set data structure.
Comment: yupp, as I said, any datastructure is OK, will try, thanks everyone!
Comment: for(String current : o_branch_id){
      if(!setta.contains(current)){
  System.out.println("faulty index is : index "+o_branch_id.indexOf(current));
      }
   }

I used this, it is returning wrong indexes, that too multiple times,viz, the values are fine and inside the allowed set !!
Answer: This might be what you want. It might not be the optimized way to do this, but you can get some idea form this
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

public class A {

    public static void main(String[] args) {
        List<String> valuesList = Arrays.asList("aa", "bb", "bb", "bb", "cc", "dd", "ee", "ff", "gg");
        List<String> correctValues = Arrays.asList("bb", "dd", "ff");

        List wrongPositions = getPositions(valuesList, correctValues);
    }

    static List getPositions(List<String> values, List<String> correctValues) {

        List<Integer> numbers = new ArrayList<>();
        for (int i = 0; i < values.size(); i++) {
            numbers.add(i);
        }

        List<Integer> correctPositions = new ArrayList<>();

        for (String correctValue : correctValues) {
            while (values.indexOf(correctValue) > -1) {
                correctPositions.add(values.indexOf(correctValue));
                values.set(values.indexOf(correctValue), "SomeValueNotContainingInList");
            }
        }

        numbers.removeAll(correctPositions);
        return numbers;
    }
}


foreach javascript array

Question: Possuo o seguinte script:
var data = JSON.parse( '<?php echo json_encode($lista_tipo_indicadores) ?>' );
  var cores = JSON.parse('<?php echo json_encode($cores) ?>');
  // Quantidade de Indicadores por Tipo
  var doughnutData = [
      {
          value: data[0].total,
          color:cores[0],
          highlight: "#FF5A5E",
         label: data[0].nome
      },
      {
          value: data[1].total,
          color: cores[1],
          highlight: "#5AD3D1",
          label: data[1].nome
      },
      {
          value: data[2].total,
          color: cores[2],
          highlight: "#FFC870",
          label: data[2].nome
      },
      {
          value: data[3].total,
          color: cores[3],
          highlight: "#5AD3D1",
          label: data[3].nome
      },
      {
          value: data[4].total,
          color: cores[4],
          highlight: "#FFC870",
          label: data[4].nome
      },
      {
          value: data[5].total,
          color: cores[5],
          highlight: "#5AD3D1",
          label: data[5].nome
      },
      {
          value: data[6].total,
          color: cores[6],
          highlight: "#FFC870",
          label: data[6].nome
      },
      {
          value: data[7].total,
          color: cores[7],
          highlight: "#FFC870",
          label: data[7].nome
      }
  ];

Como faço para que a variável  seja gerada de forma dinâmica, com algum foreach?
Comment: Confirme se eu entendi corretamente, você quer preencher a variável doughnutData por foreach em vez de ter de fazer manualmente?
Comment: Qual é a relação entre data e cores e highlight? As cores vêm da array cores imagino, mas têm ambas o mesmo numero de elementos?
Answer: Caso ambas as arrays tenham o mesmo tamanho, ie length podes iterar uma delas e mapear para incluir dados da outra. Parece-me mais prático iterar data pois têm mais conteúdo a ser usado.
Assim podes fazer:
// Quantidade de Indicadores por Tipo
var doughnutData = data.map(function(_data, i){
      return {
          value: _data.total,
          color: cores[i],
          label: _data.nome
      };
});

Não está bem claro a relação com o highlight mas se isso fôr uma terceira array podes integrar da mesma maneira. Um exemplo usando highlight como ponto de partida seria:
var highlight = ["#FF5A5E", "#5AD3D1", "#FFC870", "#5AD3D1", "#FFC870", "#5AD3D1", "#FFC870", "#FFC870"];
var doughnutData = highlight.map(function(_highlight, i){
      return {
          value: data[i].total,
          color: cores[i],
          highlight: _highlight
          label: data[i].nome
      };
});

Comment: Show de bola sua resposta sérgio. A ultima opção funcionou perfeitamente. :D

foreach javascript array


what would be the process that you debug a webserver for issues of performance where you are getting downtime?

Question: Unfortunatly this is somewhat of a vague question.  But one i face every now and then.
Right now my server is a fully managed CentOS 5 - Plesk 9.2.1.
In recent days the server keeps choking where apache (or) mysql or both seem to slow down significantly and pages aren't returned or are returned incredibly slowly, restarting mysql helps, sometimes restarting apache helps, and others i need to restart the whole server (SSH starts to really work slow).
So my question is, what would be the process that you debug a webserver for issues of performance where you are getting downtime?
I don't think the server is getting too much traffic (although it is getting consistant traffic) just something seems to kill it...
Where do i start!
Help!
Comment: Presumably this is a dedicated hardware host, not a VM on a shared machine?
Answer: Have you tried some form of query profiling on MySQL? If you run very intensive queries eg SELECT * FROM tbl1 INNER JOIN tbl2 ... INNER JOIN tbl3... LEFT JOIN tbl 4... as a really basic but obvious example, and you're returning lots of rows, or your query isn't indexed properly, this can slow down your system massively - I've learnt from my inexperienced days where I didn't optimise ;-)
Tools such as mysqlreport and the tuning-primer.sh script are good starting points for this.  mysqlreport is at http://hackmysql.com/mysqlreport, and I can only post 1 link so Google for "tuning-primer.sh" instead :-)
Answer: Bernhard response is good, but i suggest you skip sysstat and go straight to munin. with graphical representation of load it'll be much easier to figure out if you are swapping [ too many apache proceses?], trashing disk [ badly optimized sql queries, too big temporary tables? ], running out of cpu power etc.
for mysql turn on slow queries log.
Answer: You should start by installing the sysstat package. It will create statistics about CPU, Memory and I/O load every 10 minutes. When the server starts being slow again, you can look into the output of the "sar" utility (part of the package) to identify what resource is the actual bottleneck.


Diagnostic accuracy meta-analysis using MADA in R

Question: After using the bivariate model I got a pooled sensitivity and false positive rate. How do I calculate the likelihood ratios, specificity etc with confidence intervals from here on? Can it be done using MADA on R or do I need to use something else?
Answer: You can indeed use the mada R package, by means of the madauni command for univariate meta-analysis and the SummaryPts command for bivariate meta-analysis (the latter being recommended for instance by Zwinderman and Bossuyt). A suitable example follows:
install.packages("mada")
library("mada")
data(AuditC)

# univariate meta-analysis
negLR.DSL <- madauni(AuditC, type = "negLR", method = "DSL")
summary(negLR.DSL)
forest(negLR.DSL)
negLR.MH <- madauni(AuditC, type = "negLR", method = "MH")
summary(negLR.MH)
forest(negLR.MH)
posLR.DSL <- madauni(AuditC, type = "posLR", method = "DSL")
forest(negLR.MH)
summary(posLR.DSL)
forest(posLR.DSL)
posLR.MH <- madauni(AuditC, type = "posLR", method = "MH")
summary(posLR.MH)
forest(posLR.MH)

# bivariate meta-analysis
fit <- reitsma(AuditC)
mcmc_sum <- SummaryPts(fit, n.iter = 10^6)
summary(mcmc_sum)

Comment: The OP did say s/he was using a bivariate model so I suspect SummaryPts is the function s/he needs.
Comment: No I meant SummaryPts not summary.
Comment: I think your edit had not made its way through when I made my comment, I may have been seeing a cached version.
Comment: @mdewey Thanks, I had misunderstood his request, I have amended the code and now supply a more comprehensive code
Comment: @mdewey I don't follow now. Isn't the updated code correct?
Comment: @mdewey Very well


Indexed Family of Sets

Question: Most books write a family of sets $A_i$ with index set $I$ as $\{ A_i \}_{i \in I}$. However, I've read other books that have criticized this notation; they insist that one should write $(A_i )_{i \in I}$ for the family of sets $A_i$ indexed by $I$.
Is there a difference between $\{ A_i \}_{i \in I}$ and $(A_i )_{i \in I}$? If so, could you please give a precise definition of each?
Answer: I don't like either notation: I would write $\{A_i:i\in I\}$ or $\langle A_i:i\in I\rangle$. Technically there is no difference: each implies the existence of a function $i\mapsto A_i$ whose domain is $I$. The difference is one of emphasis: when I write $\{A_i:i\in I\}$, I'm thinking of this simply as a collection of sets, whereas when I write $\langle A_i:i\in I\rangle$, I'm emphasizing the existence of the function whose domain is $I$ and whose range is that collection of sets. I might let $\mathscr{A}=\{A_i:i\in I\}$ and simply talk about the collection $\mathscr{A}$ of sets, without any reference to the specific indexing, but when I write $\langle A_i:i\in I\rangle$, the specific indexing is very much on my mind: $\langle A_i:i\in I\rangle$ is an abbreviation for a function $I\to\mathscr{A}:i\mapsto A_i$.
For a more familiar example of the distinction, compare $\{x_n:n\in\Bbb N\}$ and $\langle x_n:n\in\Bbb N\rangle$, where each $x_n\in\Bbb R$. In each case $x_n$ is just a handier notation for $\varphi(n)$, for some function $\varphi:\Bbb N\to\Bbb R$. However, when I write $\{x_n:n\in\Bbb N\}$ I'm not thinking of that function; I'm thinking of its range, the set of values that it assumes. When I write $\langle x_n:n\in\Bbb N\rangle$, however, I'm thinking of the function: this is a real-valued sequence, i.e., a function from $\Bbb N$ to $\Bbb R$, not just a countable set of real numbers.
(Note: Many people use parentheses for my angle brackets; I prefer the angle brackets for this specific notational purpose, since parentheses already have more than enough meanings.)
Comment: @BrianM.Scott Beautiful explanation!!, just one thing: when you write $\{x_n:n\in\Bbb N\}$ and you think of that as the range of its associated function, you are of course not thinking of that as a set right?, because in that case if $A=\{a,b,c\}$ and $\forall n\in \Bbb N(x_n = A)$ then $\{x_n:n\in\Bbb N\}=\{A\}$. But for example $\prod\{x_n:n\in \Bbb N\}\neq\prod\{A\}$. So, here my confusion is then because we are talking about  $\{X_n:n\in \Bbb N\}$ both as a set and also as function. Does this make sense?
Comment: @Daniela: (Sorry to be so slow.) No, any time I use curly braces (i.e., $\{\ldots\}$) I definitely **am** thinking of it as a set: the range of a function is a set of points in the codomain of the function. When you form $\prod_{n\in\Bbb N}X_n$, you're actually forming the set of all functions $x:\Bbb N\to\bigcup_{n\in\Bbb N}X_n$ with the property that $x_n\in X_n$ for each $n\in\Bbb N$. In other words, the indices are actually part of the construction. If each $X_n=A$, you get the same thing when you take ${}^{\Bbb N}A$, the set of functions from $\Bbb N$ to $A$ (which you may write ...
Comment: ... $A^{\Bbb N}$). Again, you have to have some set of indices to use as a domain for the functions that are the points of the product.
Comment: @BrianM.Scott :Thank you Brian, it's quite interesting!!, though I'm still confused. Please help me. Let's see for example accordingly $\prod\{A_i:i\in \{1,2,3\}\}=\prod\{A_1,A_2,A_3\}=A_1\times A_2 \times A_3=A^3$ if $A_1=A_2=A_3=A$, for one side. But for the other $\prod\{A_i:i\in \{1,2,3\}\}=\prod\{A_1,A_2,A_3\}=\prod \{A\}=A$ if $A_1=A_2=A_3=A$. Where am I wrong or what am I missing?
Comment: @Daniela: You're going wrong when you write $\prod\big\{A_i:i\in\{1,2,3\}\big\}=\prod\{A\}$: the index set is an essential part of the product notation. $\prod\{A\}$ is essentially just $\prod_{i\in\{1\}}A_1$, where $A_1=A$. In order to take the product of three copies of $A$, you have to use a $3$-element index set to 'separate' them.
Comment: @BrianM.Scott: ah, Ok. I think I got it. So basically an indexed set is different than a set that is not indexed, right? It's like in an indexed set repetitions are allowed and the elements are treated as different if they have different index, whereas in a "standard" set repetitions simplify to just having one copy of the repeated elements, meaning they are treated as the same object? Something like $\{A,A,A\}\neq \{A_1,A_2,A_3\}$ even if $A_1=A_2=A_3=A$?
Comment: @Daniela: Now you're getting into an area in which the use of notation is a bit inconsistent. Despite the different subscripts, if $A_1=A_2=A_3=A$, one normally **would** say that $$\big\{A_k:k\in\{1,2,3\}\big\}=\{A_1,A_2,A_3\}=\{A\}\;.$$ It's specifically in products that the indexing becomes critical, because elements of products are functions whose domain is the index set. This is technically true even when you write $\prod\mathscr{A}$ for some family $\mathscr{A}$ of sets: the implied index set is $\mathscr{A}$ itself, and a point in the product is a function ...
Comment: ... $x:\mathscr{A}\to\bigcup\mathscr{A}$ such that $x(A)\in A$ for each $A\in\mathscr{A}$. $\mathscr{A}$ here is an ordinary set, so its members are distinct; if you want to take the product of identical sets, you're going to need an external index set, so that you can have $A_i=A_j$ for distinct indices $i$ and $j$.
Comment: @Daniela: Although the notation $\{A_i:i\in I\}$ implies the existence of a function with domain $I$, it is nevertheless almost always treated as a standard set, as in the displayed formula in the first half of my previous comment. This is what I meant when I said that in this area notation is a bit inconsistent. And in my notation $\{A_n:n\in\Bbb N\}$ is **not** a sequence: it is merely the family of **distinct** sets $A_n$. That is, if $A_k=A_\ell$, they are the **same** element of $\{A_n:n\in\Bbb N\}$. The **sequence** is $\langle A_n:n\in\Bbb N\rangle$; if $k\ne\ell$, $A_k$ and ...
Comment: ... $A_\ell$ are distinct terms of the sequence even if they happen to be the same set. But as I said, they are **not** distinct elements of the set $\{A_n:n\in\Bbb N\}$.
Comment: @Daniela: You're welcome. (And don't worry about the stubbornness: the important thing is that you understand it!)
Comment: Thanks for the wonderful clarification, Brian.
Comment: @Jake: You're welcome.
Comment: @BrianM.Scott. Well, then I think in this case the notation $\{A_i:i\in I\}$ denotes rather a function and not a 'standard' set like for example when talking about a sequence $\{A_n:n\in \mathbb{N}\}$ where the repetition of elements matters while in the standard notation it doesn't matter.
Comment: @BrianM.Scott   Perfect!! I think everything makes sense now. Thank you so much for your help and patience, Brian, and sorry for my stubbornness.
Answer: There is no universally valid notation in mathematics. There are so many different mathematical concepts that it is just not possible to find a distinct notation for every one. Thus, a particular notation can have different meanings in different contexts. This means, that a notation means whatever the author defines it to be. If an author defines $\{ A_i \}_{i \in I}$ to be the indexed family with elements $A_i$ for $i \in I$, then that is how $\{ A_i \}_{i \in I}$ is supposed to be interpreted. The same can be said for $( A_i )_{i \in I}$. There are, however, certain types of notation that are more commonly used than others, and deviating from standard notation can cause confusion.
We usually distinguish between sets and tuples. A set can contain a specific element only once, while a tuple can contain the same value in multiple locations, i.e.,
$$ \{ 3, 3, 3 \} = \{ 3 \}
   \quad\text{while}\quad
   ( 3, 3, 3 ) \not= (3) \,. $$
Here, I have used curly braces for denoting sets and parenthesis for denoting tuples, which is a widely used convention.
An indexed family is a function $I \to X$, where $I$ and $X$ are appropriate sets. This means, that an indexed family can have the same value in multiple locations, and is often thought of as a generalized tuple (in the sense that a tuple has indices $1, \dots, n$, while indexed families have arbitrary index sets). Therefore, using a similar notation, i.e.,
$$ ( A_i )_{i \in I}\,, $$
would make sense.
Curly brackets are usually used for sets, thus $\{ A_i \}_{i \in I}$ can be interpreted as the set that contains the elements $A_i$ for $i \in I$, and thus, if $A_1 = A_2 = A_3 = 3$
$$ \{ A_i \}_{i \in \{1, 2, 3\}} = \{ 3, 3, 3 \} = \{ 3 \} \,. $$
This equivalence is, however, not true for indexed families, by definition.
As said in the beginning, defining $\{ A_i \}_{i \in I}$ to be the index set containing the elements $A_i$ is not wrong. It can, however, lead to confusion when the reader mistakes this notation as the set containing the elements $A_i$. Hence, would I assume that this is the reason that some books recommend to use parenthesis to denote indexed families.
In the end, it is all about communication. Parenthesis, angle brackets and square brackets have various other meanings too, that could lead to confusion, and using curly brackets to denote indexed families might be the least confusing notation in certain situations. You just need to communicated your use of notation.

Indexed Family of Sets


Hinge or universal joint?

Question: I am reading the book Convex , Polyhedra,  2005, Springer Verlag , written by A. D.  Alexandrov. In chapter 10, he introduces the term hinge mechanism consisting of the edges and vertices of a convex polyhedron,( probably is what we call framework).  What I do not understand is:

Why hinges and not universal joints ? (hinges allow only planar rotation)

Is it a bad translation or here hinge means something else?

Thanks.
Comment: Why does the name matter?  If you don't understand the definition associated to the name that's potentially a good question (and you should edit your question to clarify that) but as it stands this doesn't seem to be about maths at all.  Also, what do any of the tags you've chosen have to do with the name of an object?
Answer: These are two different joint types.
(1) When two planes intersect they make a dihedral angle  D between them and and the intersection line is called the hinge when rotation about this hinge line /axis/ joint is possible and it is said to have a single degree of motion freedom: $ 0< D < 2 \pi.$
(2) When a line/rod attached to a steel ball for example at the origin moves all around in two or more planes with no change in the position of center of the ball. We have two degrees of  spherical coordinate freedom with $ 0< \theta < 2 \pi $ and $ -\pi < \phi < + \pi. $
These latter movement freedoms exists for a universal (or Hooke's ) joint in an automobile propeller shaft application for example and is kinematically equivalent to a ball joint with two degrees of freedom.
Comment: But then the book should say: hinge mechanism consisting of edges and rigid plates, ( not vertices). ...or not?
Comment: Indeed, that is what must be written. There is no place for a ball joint in a polyhedral face/plate configuration. Maybe it is lost in translation from the Russian original.

Hinge or universal joint?