Stationarity, ACF and PACF

Question: I can't understand this sentence: "The distribution theory underlying the use of the sample ACF and PACF as approximations of those of the true DGP assumes that the $y_t$ sequence is stationary" (from: Enders, Applied Econometric Time Series"
Comment: What in particular you did not understand?
Comment: Just using them wouldn't imply stationarity but, depending on the patterns you see from them, they could suggest a particular form of the model that may or may not be stationary.
Comment: zar, is the problem clearer now?
Comment: @RichardHardy Yes, the answer of Winkelried is very clear to me. In my response to the comment of mpiktas I've added a doubt that I've
Comment: @mpiktas The fact that in order to use for inference the sample ACF and PACF we have to assume that $y_t$
is stationary (but now I've understand this). I also can't understand how can I check stationarity from sample ACF and PACF while I assume ex ante that yt is stationary, it's seems circular to me
Comment: @zar, if the series is not stationary, then ACF and PACF do not make sense. If the series is stationary then the ACF should go to zero as the number of lags is increasing. So if it does not that it is the first clue that series are probably not stationary.
Answer: If your time series is nonstationary, then generally (!) there are no population counterparts of the sample ACF and PACF. Thus the sample ACF and PACF cannot converge to the population ACF and PACF. While you normally use the sample to make inference about the population or forecast a new sample from the population, nonstationarity does not allow to do that, because in general (!) there is no well-defined population to speak of (the population is changing, possibly unpredictably, with each new time point).
Answer: You have your time series $(y_t)$. That time series has certain ACF and PACF. You don't know how the random variables that make up your time series look like, so you can't calculate the ACF and PACF from them. You do know however, some data sampled from those random variables. From that sample you can calculate the sample ACF and sample PACF. There are results from distribution theory that tell you that the sample ACF and sample PACF will be an approximation for the ACF and PACF of the time series. These results are valid under the assumption that the time series has the property of being stationary.
Be aware that there are varying definitions of stationarity. You will have to figure out which one the author is referring to.
Comment: In this case weak stationarity
Answer: You look at ACF and PACF to get an idea of the lag structure of the process. For instance take a look at this plot:

You see how ACF is declining in amplitude exponentially, while PACF cuts off after lag 1. This may suggest that you're dealing with AR(1) process.
How do I know this? If you derive the ACF and PACF assuming that your process is AR(1), which includes constant variance of errors, then you'd come up with a similar shaped curves. Hence, if your underlying series are not stationary, you're breaking the assumptions that are base for the heuristics that I mentioned about ACF/PACF. It's pointless to apply these on non-stationary series, since you can't make any conclusions about the lag structure anymore.
Comment: I think the essential argument is missing from your answer. You start out nicely by giving an example under stationarity, and then conclude that things break under nonstationarity without showing or explaining why.

Stationarity, ACF and PACF


$P(A∩B'∩C) $ given $P(A∩B∩C), P(A∩C) $ and $P(B∩C)$

Question: Having

$P(A∩B∩C) = \frac {1}{10}$
$P(A∩C) = \frac 15$
$P(B∩C) = \frac 25$

What is the probability of $A∩B'∩C$ , where B' is the complement of B?
I tried to write $A∩B'∩C $ as $(A∩C) - (A∩B∩C)$ and then I calculated the probability doing
$ P(A∩C) - P(A∩B∩C) = \frac 15 - \frac {1}{10} = \frac {1}{10}$
I am not sure if this is possible because I don't know if the probability of the subtraction of two events is equal to the subtraction of probabilities.
Comment: Yes you are right. The probability of the subtraction is equal to the subtraction of the probabilities if the event you are subtracting is contained in the other.
Answer: If events $X$ and $Y$ are disjoint, then the probability of $X\cup Y$ is the sum of their probabilities. Your subtraction law is just a rearrangement of this, so it's good!
Answer: You are allowed to proceed as you did because of the following:

$C = (B\cap C) \cup  (B'\cap C)$ as disjoint union
$\Rightarrow A\cap C = (A \cap B\cap C) \cup  (A \cap B'\cap C)$ as disjoint union

Now, you can use the additivity of probability for disjoint events:
$$P(A\cap C) = P(A \cap B\cap C) + P(A \cap B'\cap C)\Rightarrow \boxed{P(A \cap B'\cap C) = P(A\cap C) - P(A \cap B\cap C)}$$


D7: Add a class to a list item when the created date is less than 14 days old

Question: Using Drupal 7 and Views, I currently have a view that produces a list of 5 teasers of the content type "News." 5 teasers is a hard rule. I'd like to conditionally format the rows so that when their "created date" is less than 14 days old so that I may styles them as "New" with CSS styles. So if there are 3 "New" rows, the balance are 2 unmarked rows.
How to do so?
Update
I had the idea of using the "time ago" date format to populate the class name. However, the format for "time ago" changes. Unfortunately, I can't set the granularity to "days" which would have allowed me to add the text as a class, i.e., date1 through date24 for the past 2 weeks.
Update #2
My genesis subtheme is ignoring any preprocessor routines, see more here. However, I can add the routine to my base template.php.
Update #3
Thanks to Елин Й. for the working code:
function genesis_preprocess_views_view_list(&$variables) {
  $view = $variables['view'];
  if ($view->name == 'recent_news_widget') {
    foreach ($view->result as $key => $result) {
      if (time() - strtotime($result->node_created_day) < 14 * 24 * 60 * 60) {
        $variables['classes'][$key][] = 'new';
        $variables['classes_array'][$key] = isset($variables['classes'][$key]) ? implode(' ', $variables['classes'][$key]) : '';
      }
    }
  }
}

In an offline discussion, Елин Й. explains why this works; "I took a look in the views theme.inc and found out that, the classes array is flattened to another variable called 'classes_array' you need to do it as well in order to get your new class also added in this array.
Comment: in *UPDATE4* you're overriding `row_class` completely. You want to append your own class -- not nuke everything to new.
Comment: was about to say the same thing, pop the value to the `row_class` array instead of overwriting it.
Answer: You can do that using a preprocessor function for the view you created. You need to preprocess the view and check the creation date of nodes for each row and give them a relating class.
Given that your view's machine name is recent_news_widget, your diplay format is HTML lists, and your theme name is genesis. You need to use template_preprocess_views_view_list() function in /sites/all/themes/genesis/template.php file
genesis_preprocess_views_view_list(&$variables) {
  $view = $variables['view'];
  if ($view->name == 'recent_news_widget') {
    // dpm($view);
    // Here comes your logic
  }
}

In the above code you see 2 lines commented out. First one is a helpful function for developers to print out the given array/object/string etc in a browsable style. This function comes with Devel module, which I would strongly recommend to install it.
After browsing the variables, you will probably find where the results are listed and where the classes for the rows are assigned. You need to browse the variables until you find all values you need to build your logic to assign new classes, for example the 'created date' of the node. After you have all the variables you need, you can loop through the results, check the creation date and add a relating class.
UPDATE:
The preprocess functions for other display types than HTML List:
Grid: template_preprocess_views_view_grid()
Table: template_preprocess_views_view_table()
Unformatted: template_preprocess_views_view_unformatted()
UPDATE #2 (According the discussion on chat and your updates on the question):
I find the code in your Update #3 was in the right direction, but with a few mistakes. The corrected version should look like that:
function genesis_preprocess_views_view_list(&$variables) {
  $view = $variables['view'];
  if ($view->name == 'recent_news_widget') {
    foreach ($view->result as $key => $result) {
      if (time() - strtotime($result->node_created_day) < 14 * 24 * 60 * 60) {
        $variables['classes'][$key][] = 'new';
        $variables['classes_array'][$key] = isset($variables['classes'][$key]) ? implode(' ', $variables['classes'][$key]) : '';
      }
    }
  }
}

The 2 mistakes in your code were:

You passed the argument as &$variables to the function, but used $vars to assign classes, $vars is not defined in the function so the class was not added.
You wanted to use the key number of the result but in foreach you loop only through the values, and $result variable is an object. In the corrected version of your code above, I used $key => $result by defining the loop so that I can access the key number as well.

UPDATE #3 I added a new line that flattens the classes into classes_array.
Comment: You need to clear cache, at least the theme registry. Yes, it should render it to the messages region.
Comment: Oh, and the preprocess function was an example for a view that has a display type of HTML List. If your display type is different, then you need to use its preprocess function. I'll update my answer and list the preprocess functions for each display type.
Comment: To debug i'd remove the first if condition and try to dpm the view without checking the view name. However if you have too many views on the same page, you could get memory errors.
Comment: Thanks, this looks like the correct way to go. However, the code (adapted for my theme's name and view name) isn't working. `function ubercompany_preprocess_views_view_list(&$variables) {
  $view = $variables['view'];
  if ($view->name == 'recent_news_widget') {
    dpm($view);
    // logic
   }
}` isn't rendering the variable data? I presume it renders to the message box, but it isn't?
Comment: I have set Devel's `Rebuild the theme registry on every page load` _(and routinely use `drush cc all`)_ and I presume the view _is_ an HTML List type, i.e., `$handler->display->display_options['style_plugin'] = 'list';` but to no avail.
Comment: Tried that, didn't show, i.e., inserted print($view); just after the variable is set, which suggests that `ubercompany_preprocess_views_view_list(&$variables)`isn't being triggered?
Comment: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/9251/discussion-between-kyle-skrinak-and--)
Answer: Hope this will help you!!
As you need to add the class to the row, you can achieve by overriding the default view  template file
You can achieve this by the following steps

Open your view and go to Advance tab + other category you can see the theme -  information click Information link.

As per your requirement you need to override the row style. you can see Row style output, the default template file is views-view-fields.tpl.php which is highlighted. Means this template file is active now. And also you see list of  tpls will be there, so choose any one of the tpl name (eg.., views-view-fields--comments-recent.tpl.php or views-view-fields--comments-recent--page.tpl.php) for your new template file.

You need to override this template file so navigate to VIEWS module + THEME folder + get the tpl file which you want to override(here views-view-fields.tpl.php)

Copy and paste template file to your theme template folder(sites/themes/your_theme/templates/) and rename it to the choose template name
Click Rescan template file to active new template file.

No you can modify the template file and add the class.
Comment: Inizio, that's a stellar effort at documentation, and ought to serve someone looking for how to do so well. One could also add a class to the row via the Views UI if that's all that is needed. Your answer is broader than my question however, though much appreciated.
Answer: function genesis_preprocess_views_view_list(&$vars) {
  $view = $vars['view'];
  $rows = $vars['rows'];

  if ($view->name == 'recent_news_widget') {
    foreach ($rows as $id => $row) {
      $data = $view->result[$id];
      // Assumption this if condition works and node_created_day exist in row
      if (time() - strtotime($data->node_created_day) < 14 * 24 * 60 * 60) {
       // Add custom class if condition satisfies
       $vars['classes'][$id][] = 'custom-new';
      }
    }
  }
}

Also can you try replacing the classes line with this one and tell me does it work.
$vars['row_classes'][$id][] = 'custom-new';
Comment: @KyleSkrinak can you check if it is entering the if condition where the time check is made ?
Comment: @KyleSkrinak can you try this and tell me if it works. $vars['row_classes'][$id][] = 'custom-new'; Just replace the old line and clear cache.
Comment: I was hoping this was it. However, while the dpm function dutifully fired no classes added. I disabled every module except for views and devel, eliminating (?) the potential for conflict. However, classes not added in the rendered HTML output.
Comment: Running `dpm($data);` within the if block confirms the condition is correctly met.
Comment: Replaced `$vars['classes'][$id][] = 'custom-new';` with `$vars['row_classes'][$id][] = 'custom-new';` cleared server and browser cache, reloaded page. No classed applied to output, i.e., ` ` (First list item matches the condition)


covariate significant

Question: I'm a bit lost at the results part of my thesis. I have conducted an generalized maximum likelihood (GML) univariate test with Y variable: dependent variable
one X variable: independent (consists of 2 conditions)
I have a lot of covariates to control for. So i performed different ANOVA's with just one covariate at a time.
Some covariates turn out to be significant AND the X variable is also significant.
Some covariates turn out to be significant but the X variable is not significant.
The problem:
I previously wrote in the results section of my thesis, that when a covariate turns out to be significant and the X was less significant than in the analysis without the covariates, this means I found a main effect but it did not change the interpretation of the results.
But what is the case if the some covariates are not significant, but the x variable is significant and even more significant than in the analysis without covariates?
What can I conclude from this finding? I tried to google it but i couldn't find it. I hope you can help me.
A lot of thanks in advance.
Answer: Some idea. Hopefully you'll get a few responses and all together there will be something useful.
(1) I usually use the term "main effects" to refer to a multivariate regression model before I've addressed interactions, or that part of a multivariate model that includes the variables but not the interactions. Thus in y ~ x + z + x*z, x and z are the main effects.
(2) It sounds like you're doing bivariate analysis to look for confounders that should be included in a final multivariate regression model. This isn't the best way to build a regression model. I'm not sure there is a best way, and it does depend on what you're trying to achieve with your model. If you're just trying to find the best estimate of your main variable of interest then include all/most variables, if you interested in a group of factors that are all together associated with the outcome some model building is needed.
(3) Some covariates turn out to be significant AND the X variable is also significant = the covariate is associated with the outcome, but this does not say anything about whether the covariate is associated with x. If it is associated with outcome and x it could be a confounder
(4) But what is the case if the some covariates are not significant, but the x variable is significant and even more significant than in the analysis without covariates? = is this the classic "suppressor variable" ?
(http://www.uvm.edu/~dhowell/gradstat/psych341/lectures/MultipleRegression/multreg3.html) Here I'm assuming you mean statistically significant.

covariate significant


Why have all Asian Games that were held in Bangkok been on Decembers?

Question: Four Asian Games have been held in Bangkok overall:

1966 Asian Games: December 9 - 20
1970 Asian Games: December 9 - 20
1978 Asian Games: December 9 - 20
1998 Asian Games: December 6 - 20

All other Asian Games have varying dates; but these four have almost exact dates.
Are the mentioned opening and closing dates related to Thailand's history and culture or there is another specific reason?
Answer: King Bhumibol Adulyadej, the king of Thailand during all four of those Asian Games, celebrated his birthday on December 5. I therefore strongly suspect that those Games were timed to begin shortly after his birthday - while I haven't found any corroborating evidence so far, it seems unlikely to have been a coincidence.


Comando DELETE em VBA não funciona

Question: Tenho um projeto que adiciona e elimina datas, para adicionar tenho o seguinte código:
<code>   Set dbs = CurrentDb
   dbs.Execute " INSERT INTO TMP " _
   & "(diaMes) VALUES( " _
    & "'" & tmp7 & "') "
</code>
onde o tmp7 é a data em dd-mm-yyyy
e funciona perfeitamente
e para retirar uso o seguinte código:
<code>    Set dbs = CurrentDb
   dbs.Execute " DELETE * FROM TMP " _
   & "WHERE diaMes=#" & tmp7 & "#;"
</code>
O problema é que ao eliminar apenas elimina se o dia for superior a 12, ou seja eu entendo que ele, caso o dia seja inferior ou igual a 12 esteja a interpretar como mm-dd-yyyy
o tmp7 é uma string com concatenação de strings.
Como posso forçar a passagem do SQL como dd-mm-yyyy ?
A campo na tabela está assim:
Comment: Valeu amigo, eu tbm tava com esse problema. Obrigado pela ajuda!!
Answer: Descobri como fazer,
<code>    Set dbs = CurrentDb
    dbs.Execute "DELETE * FROM TMP WHERE diaMes = " & _
    Format(tmp7, "\#mm\/dd\/yyyy\#")
</code>

Comando DELETE em VBA não funciona


Height of a paralelogramm

Question: I have the coordinates of the 4 vertexes of a parallelogram.
If i calculate the length of two opposing sides, how do I get the perpendicular distance between them?
Is it just the distance between the 2 side's midpoint?
Answer: No the distance between two midpoints is the same as a side, which is not what you want.
The easiest is probably to start by calculating the area of the parallelogram (by taking the length of the cross product of two adjacent sides in 3D, or as a determinant in 2D) and then divide by the length of the side you want to have as base.
Comment: No. Just, as I said, divide the area by the known base length. This will give you the height.
Comment: @JohnFire: Who says you have to do it that way? It will be rather more difficult to find the height _without_ getting the area as an intermediate result somewhere. Much easier to call it a day at that point and use the area you've already got, I think.
Comment: I got the base length already. Should I try to calculate a diagonal?
Comment: But I don't have the area, I have to find the area, by the base and the height that's my problem :/
Comment: Well my teacher was insisting to do it that way, but then if its that hard I guess there is no choice, thank you :)
Answer: If by opposing sides you mean parallel sides, then no, it isn't. That will be the length of the side adjacent to the one you are referring to.
One way would be to find the area of the parallelogram using the coordinates and equate the area to the product of the base and the height. The length of the base can again be found using the coordinates of two adjacent points.
Comment: Sorry for the duplicate! Posted just after the other reply
Comment: @JohnFire To get the answer without getting the area, you can find the slope of the base corresponding to the height you need. Then, the slope of the height will be the negative of the reciprocal of the slope of the base. Find the equation of the line passing through a point of the parallelogram which is not a point on the base with this slope and find its point of intersection with the base. The distance between these two points is the height.

Height of a paralelogramm


How to do rowwise matching in data.table between list-column and external vector?

Question: I'm trying to do a rowwise matching for vector elements nested inside a data.table list-column.
library(tibble)
library(data.table)

my_dt <-
  tibble::tribble(
    ~col_x,   ~col_y,
    "a",      c(1, 2, 3),
    "b",      c(4, 5, 6),
    "c",      c(7, 8, 9)
  ) %>%
  as.data.table()

external_vec <- 1:9

I want to match each element from col_y against external_vec such that I will get a new column in my_dt that is equivalent to external_vec %in% col_y, such as:
# desired output
##    col_x col_y                                 new_col
## 1:     a 1,2,3  TRUE, TRUE, TRUE,FALSE,FALSE,FALSE,...
## 2:     b 4,5,6 FALSE,FALSE,FALSE, TRUE, TRUE, TRUE,...
## 3:     c 7,8,9 FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,...

EDIT

I know how to implement a solution based on tidyverse's purrr, like the following. However, I'm looking for a data.table native functionality.
# this works but relies on purrr and dplyr, which I hope to avoid in this problem
my_dt %>%
  tibble::add_column(ext_vec_as_col = list(external_vec)) %>%
  mutate(new_col = purrr::map2(.x = ext_vec_as_col, .y = col_y, .f = ~.x %in% .y) )
#>    col_x col_y  ext_vec_as_col                                 new_col
#> 1:     a 1,2,3 1,2,3,4,5,6,...  TRUE, TRUE, TRUE,FALSE,FALSE,FALSE,...
#> 2:     b 4,5,6 1,2,3,4,5,6,... FALSE,FALSE,FALSE, TRUE, TRUE, TRUE,...
#> 3:     c 7,8,9 1,2,3,4,5,6,... FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,...

Answer: I've got the tidyverse-solution:
library(tibble)
library(dplyr)
library(tidyr)
library(purrr)
library(magrittr)

my_dt <-
  tibble::tribble(
    ~col_x,   ~col_y,
    "a",      c(1, 2, 3),
    "b",      c(4, 5, 6),
    "c",      c(7, 8, 9)
  )
external_vec <- 1:9

my_dt %>%
  mutate(new_col = col_y %>%
           map(~external_vec %in% .x)) %>%
  unnest_wider(col_y, names_sep = "_") %>%  # this is just to see the result..
  unnest_wider(new_col, names_sep = "_") # this is just to see the result..

Usually, library(tidyverse) includes a lot of these, but I just thought that it would be nicer to know exactly what you need to make this work.
data.table solution
I think this is the equivalent:
my_dt[, new_col := lapply(col_y, \(x) external_vec %in% x)]
my_dt

Performance
Since that is important, here is some performance benchmark

bench::mark(
  tidy =
    my_dt %>%
    mutate(new_col = col_y %>%
             map(~external_vec %in% .x)),
  dt = my_dt[, new_col := lapply(col_y, \(x) external_vec %in% x)],
  op_tidy = my_dt %>%
    tibble::add_column(ext_vec_as_col = list(external_vec)) %>%
    mutate(new_col = purrr::map2(.x = ext_vec_as_col, .y = col_y, .f = ~.x %in% .y)),
  check = FALSE
) %>%
  plot()

Comment: Thanks! And oops. I just added an edit that counters your answer :-/
I know tidyverse tools but for my current problem I need `data.table` because of performance
Comment: The `data.table` solution seems sweet! Is `\(x)` depends on `R` version > 4.1.0?
Comment: Added to my answer. Also the way I did it is twice as fast as the way you did it;
Comment: Yes, otherwise you have to do it the "verbose" way with `function(x)` instead of `\(x)`.


Best Network Tape Backup tool

Question: We have a medium sized network with both Windows and Linux servers (about 50 of each), and I'm just wondering what the best multi-OS backup solution for this situation is.
Right now we are using bacula with a 23 tape autochanger, which is fabulous, but a bit hard to configure, and lacking some good gui tools.
What other tape backup solutions are people using, what do they cost, and how easy are they to use?
Answer: I'd recommend Symantec's NetBackup if you can afford it.  BackupExec from Symantec is pretty good until you add the Unix clients.  It doesn't do that nearly as well.
NetBackup has an excellent command line interface, as well as a GUI that is good.  I will say it has a few pitfalls in the setup phase, but if you follow the documentation and best practices guides, rather than just jumping in blind it's great.
Answer: I use BRU and Zmanda.
BRU has a GUI console that works on most *nix machines as well as OS X. Zmanda has a web based gui. BRU abstracts the idea of tape management away from you, instead you define a pool of tapes you want to backup to for a particular job or jobs and it will put data where it can while following your retention rules. Zmanda is just as configurable.
As far as cost is concerned you can get zmanda for free, but BRU will cost you at least $299 for 30 hosts. Both are simple if you've done backups before.
Answer: In the Windows world, a big player is Backup Exec by Symantec. Licensing is a pain, however, as they want you to buy a license for each server that it touches.
I would have LOVED to have found a replacement for Backup Exec.
Comment: In fact, we switched FROM Backup Exec TO bacula - and have found both pros and cons doing so.  The biggest con being that Backup Exec's GUI was excellent.
Answer: HP Dataprotector scales well in a multi-OS environment.  Just check the flavor of Linux you use,  they certify some flavors but not all.  We use it for HP-UX, and Windows.
Answer: I second the NetBackup recommendation, with a caveat:
While NetBackup is infinitely configurable, (with Multiplexing, Multistreaming, Storage Units, Storage Unit Groups, etc) for an easier life resist the urge to configure it infinitely!
Additionally, good, actually... perfect DNS and reverse DNS and the lack there of, are at the root of a good NetBackup deployment.
JB

Best Network Tape Backup tool


mysql_query() não executa

Question: Estou a tentar testar se a minha base de dados se conectou corretamente apartir duma query, mas aparentemente o mysql_query() não executa e não altera a minha tabela. O meu código é este:
<?php
//conecta a base de dados
$dbc = mysql_connect('localhost', 'dbweb', 'notime');
if(!$dbc)
{
    die("Not connected : " . mysql_error());
}

//seleciona a base de dados
$db_select = mysql_select_db('goaheadmakemyday', $dbc);
if($db_select)
{
    die("Can't connect : " . mysql_error());
}

//teste
$query = "UPDATE Colaborador SET Nome='Ola' WHERE ColNum='0'";
$resultado = mysql_query($query);

//Pequena parte da minha tabela:
//ColNum  Nome
//0       Simeão Lopes
//1       Olavo Bettencourt
//2       Sandro Vidal

Answer: No seu código o problema é quando a conexão é feita com sucesso $db_select recebe true, entra no if e fatalmente cai no die() e mata seu script. A solução para isso é negar a condição ou fazer uma verificação inline.
As funções mysql já foram removidas do php7, o mais indicado é utilizar o MySQLi ou PDO para fazer consultas em banco de dados.
$db_select = mysql_select_db('goaheadmakemyday', $dbc);
if(!$db_select)
{
    die("Can't connect : " . mysql_error());
}

Inline
mysql_select_db('goaheadmakemyday', $dbc) or die(mysql_error());

Comment: Muito obrigado!

mysql_query() não executa


How to get Avatar from discord API?

Question: I am asking how can I get an avatar of a user who has authorized with oAuth2.
https://cdn.discordapp.com/avatars/781264468998488095/ XXXXXXXXXXXXXXXXXXX ?size=128
The first one is client ID, could someone explain what could be behind "/"?
Or how can I download from api
Text
Text
Answer: The information that is provided to you by the Discord OAuth2 response includes an avatar id:

{id: '537355342313422849', username: 'Spectacle', avatar: '1ee4375beb131ae43a4e4b81e267a265', discriminator: '1738', public_flags: 64, …}

That's what goes in the URL after the client ID.
Example:

https://cdn.discordapp.com/avatars/537355342313422849/1ee4375beb131ae43a4e4b81e267a265

And of course, you can add specifications for the size.
Answer: const { get, post } = require("snekfetch");
const express = require("express");
const btoa = require("btoa");
const app = express();

const cfg = {
    id: "****",
    secret: "*****",
};

app.get("/", (req, res) => {
    console.log(req.query.code);
    res.status(200).json({ status: "ok" });
    get(`https://discord.com/users/{id}`)
        .then((response) => {
            console.log(response);
            res.send(response);
        })
        .catch(console.error);
});

app.get("/dc", (req, res) => {
    res.redirect(
        [
            "https://discordapp.com/oauth2/authorize",
            `?client_id=${cfg.id}`,
            "&scope=identify+guilds",
            "&response_type=code",
            `&callback_uri=http://localhost:8080/authorize`,
        ].join("")
    );
});

app.get("/authorize", (req, res) => {
    const code = req.query.code;
    const cred = btoa(`${cfg.id}:${cfg.secret}`);
    post(
        `https://discordapp.com/api/oauth2/token?grant_type=authorization_code&code=${code}`
    )
        .set("Authorization", `Basic ${cred}`)
        .then((response) =>
            res.redirect(`/guilds?token=${response.body.access_token}`)
        )
        .catch(console.error);
});

app.get("/guilds", (req, res) => {
    get("https://discordapp.com/api/v6/users/@me/guilds")
        .set("Authorization", `Bearer ${req.query.token}`)
        .then((response) => res.json(response.body))
        .catch(console.error);
});

app.listen(8080, () => console.log("Ready"));

How to get Avatar from discord API?


How to utilize 100% of the network bandwidth with sockets?

Question: I have a server and a client. The are working in different servers. Both of the servers have two 1000M network adapters.
I am using tcp blocking socket both in server and client.
Server
Once a socket is accepted, a new thread will be started to process the request. It works like:
while(1) {
    recv();  /* receive a char */
    send();  /* send a line */
}

The client just send a char to the server, server will send a line of text to the client. The length of the text is about 200.
The line has beed loaded into memory in advance.
Client
The client use different threads to connect to the server. Once connected, It will work like:
while(1) {
    send();  /* send a char */
    recv();  /* receive a line and  */
}

Bandwidth usage
When I use 100 threads in client(and more the result is almost the same), I get this network traffic in Server:
tsar -l -i 1 --traffic

the result:
Time              -------------traffic------------
Time               bytin  bytout   pktin  pktout
06/09/14-23:12:56   0.00    0.00    0.00    0.00
06/09/14-23:12:57  63.4M  155.3M  954.6K  954.6K
06/09/14-23:12:58   0.00    0.00    0.00    0.00
06/09/14-23:12:59  60.1M  147.3M  905.4K  905.4K
06/09/14-23:13:00   0.00    0.00    0.00    0.00
06/09/14-23:13:01  57.5M  140.8M  866.5K  866.4K

and sar -n DEV 1:
11:20:46 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
11:20:47 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00
11:20:47 PM      eth0 478215.05 478217.20  31756.46  77744.95      0.00      0.00      0.00
11:20:47 PM      eth1 484318.28 484318.28  32162.05  78724.16      0.00      0.00      1.08
11:20:47 PM     bond0 962533.33 962535.48  63918.51 156469.11      0.00      0.00      1.08

Question:
In theory, the max value of (bytin + bytout) could be 256M. How can I archive that?
Any help will be great, thank in advance.
Comment: Probably a Linux question. If it is, please tag it as such. And show the exact code...
Answer: In practice there are some overhead in several layers. 1Gbits/sec ethernet does not mean that much on the application side (but I guess at most 90% of that). A rule of thumb is to send or recv quite large data sizes (e.g. several kilobytes at least). Sending or recieving a few hundred bytes is inefficient. And the question is surely OS specific (I am thinking of Linux).
Recall that by definition TCP is not a transmission of packets, but of a stream of bytes. Read TCP wikipage. You should avoid send-ing or recv-ing a few bytes, or even a hundred of them. Try to send thousands of bytes at each time. Of course, a single recv on the recieving side is not (in general) corresponding to a single send on the emitter side and vice versa (especially if you have some routers between sending and recieving computers; routers can split or coalesce network packets, so you can't be sure to have one recv on the receptor per each send in the emitter).
Gigabit Ethernet wants Jumbo Frames of nearly 9000 bytes. You probably want your data buffer for send to be a little below that (because of the various overhead for IP and TCP), so try 8Kbytes.
The send(2) man page mentions MSG_MORE flag for tcp(7). You could use it with care. See also this.
Also syscalls(2) have some overhead. I'm surprised you are able to make a million of them each second. That overhead is another reason for buffering both outgoing and incoming data in significant pieces (of e.g. 8192, 16384, or 32768 bytes each; you need to benchmark to find the best one). And I won't be surprised if the kernel prefers page-aligned data. So perhaps try to have your buffer aligned to 4096 bytes (e.g. using mmap(2) or posix_memalign(3)...)
If you care about performance, don't use send(2) with a small byte count. At least change your application to send more than a few kilobytes (e.g. 4Kbytes) at each send syscall. And for recv(2), pass a buffer of at least 4kilobytes. So sending or recv a single byte or a line of a hundred bytes is inefficient. Your application should buffer such data (and perhaps split data into "application messages"...). There are some libraries doing that (like 0MQ...), or at least terminate each message with a delimiter (newline perhaps), which would ease the splitting of a received buffer into several incoming application messages.
My feeling is that your application is inefficient and buggy (probably would work badly on other networks, e.g. if there are some routers between both computers). You need to redesign and recode some parts of your application! You need to buffer, and you need to manage application messages - splitting and joining them ....
Yous should test your application on several networks, in particular thru ADSL and wifi and if possible long-distance networking (you'll then observe that send and recv do not "match").
Comment: Yes, I want to send thousands of messages(only a char in request from client). And the overhead is larger than the message.  `MSG_MORE` seems can make multiple message be send in a package. If yes, that would be what I want.
Comment: You should buffer more by sending less messages, but each having more bytes, if you care about performance.
Comment: I want a response for each request. Can I merge the multiple tcp headers into one? But the client / server still can receive them as separated?
Comment: You have to redesign your application with buffering, and that means that you'll could get more than one `recv` per application request (it is already the case now). With other protocols (like UDP) you should deal (in the application) with lost, truncated, or duplicate packets (and that is not easy).
Answer: According to my math, you are relatively close to saturating the link.
As I understand it, this is one second of traffic.

Time              -------------traffic------------
Time               bytin  bytout   pktin  pktout
06/09/14-23:12:57  63.4M  155.3M  954.6K  954.6K

A TCP packet sent over Ethernet has 82 bytes of overhead (42 ethernet, 20 IP, 20 TCP), so the amount of data received is (954.6k * 80 + 63.4M)*8 bits, which totals 1.1G.
I would assume that with such a large number of packets, there would be additional overhead involved with negotiation of the physical medium.  Since the links have about 50% utilization, if there's an additional delay as small as (1s / 954.6k) * 50% = 500 ns (one half microsecond!) then you've accounted for the additional delay.  500 ns is the amount of time it takes for light to travel 150 meters, which isn't that far.
Comment: @srain: The fact remains that you are trying to send and receive a million packets in one second.  Send fewer packets with larger payloads if you want to saturate the link.
Comment: @srain: TCP provides no guarantees that each `send()` on your client will match a `recv()` on your server.  That's just not how TCP works: it's a stream of bytes.  You could `send()` 1 byte 5 times, then `recv()` 5 bytes once, or vice versa.  The only guarantee is that the number of bytes will match.
Comment: Based on my assumptions, the packet overhead is larger than `bytin`, so that wouldn't be possible.  However, it is possible that it counts only TCP overhead, or TCP and IP overhead, or something like that.  I'm not familiar with `tsar`.
Comment: No, you can't, as said in my answer. TCP is a stream byte, without application level packets or messages. You have to buffer!
Comment: 24 ethernet. The `bytin` does not include the header size?
Comment: Yes, the pay load is small when compared to the overhead. I want a response for each request. Can I merge multiple requests into a tcp package, and use `recv()` (or something else) in server, to receive them as separate requests?
Comment: The lockstep nature of the application described in the question makes me wonder whether OP is telling the truth about what he's doing.  Each `send` will match up with a `recv` and vice-versa.


what's Taxed line items in the ubercart tax module?

Question: does anyone know what is referring to the fields :
Taxed line items : Shipping,Empty line,Tax in the tax configuration?
i couldn't find an explanation
Answer: Line items are the items in an order. They can include products, shipping, and tax. The taxed line items are the line items you apply the tax rate to. Generally that would just be the products.


Which is the best database to use to index the internet?

Question: If you had to make provision for 80 million records (one for each page on the internet) and store the relationships between those records (which is 80 billion to the nth power), which database would be the best for this?
I've started this project thinking we will only map a portion of the internet, but unfortunately it has gone far beyond the limits of mysql. I need a better way to keep track of this data. The frontend is PHP, but I suppose the backend can be anything, as long as it can handle that amount of data?
Answer: i won't say there is the one holy database for your needs, maybe it could be better for your company to split your database in logical parts to handle the amount of data in a better way. maybe you could outsource some data into file system as you won't need anything everytime in your database.
if you scan the interwebs, you probably save the html, css or any big data you crawl for into your filesystem while you save connections and everything meta related into your database. but i really think you'd mentioned that already.
the best advice i want to give here is to make sure, your structure of your database is whatever fits your processes the best before think about switching the database. if you really need to switch (as mysql would not give you more performance), there will be mongodb and/or webscalesql. webscale seems to be used by facebook to handle the amount of their data.
a big question would be if you just can improve your performance by improve your hardware. you should check that too, AFTER you checked your structure and processes!


What does time.sleep(10) in python do during daylight savings time?

Question: If it's 5 seconds before setting the time forward an hour, will time.sleep(10) sleep for 10 seconds, or five seconds? (after five seconds, the time is more than ten seconds from the start).
If it's 5 seconds before setting the time backwards an hour, time.sleep(10) sleep for ten seconds, or for an hour and ten seconds? (it takes an extra hour for the current time to be more than ten seconds from the start time, at least by the clock)
And for completeness, what does sleep do for leap seconds, like we just had?
Are there any times that time.sleep would not be expected to do what is requested?
I'm still on python 2.7.
Comment: Its a time duration. It works immaterial of the actual time.
Comment: It basically holds its breath for 10 seconds. It doesn't teleport you to a specific time
Comment: Any sleep function that breaks during DST or leap seconds would be critically broken. For a language as popular as Python, it would be incredibly unlikely that no-one would have noticed if it had that bug.
Answer: As the time.sleep() document says:

Suspend execution of the current thread for the given number of seconds.

time.sleep() is independent of the time-zone, or your system time. It suspends execution of current thread for the amount of seconds specified while making the call to it.
Answer to your question "Are there any times that time.sleep would not be expected to do what is requested?" is also available in document:

The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal's catching routine.

But this is changed in version 3.5, and function sleeps at least secs even if the sleep is interrupted by a signal, except if the signal handler raises an exception (see PEP 475 for the rationale).
Note: The suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system
Answer: time.sleep(10) does not look at 'wall time'; it counts 10 seconds in 'CPU time'. (there is a discussion about the accuracy of this here: How accurate is python's time.sleep()?)
Answer: Most computers track time in terms of seconds, or milliseconds, since an "epoch," which is usually January 1, 1970 UTC.
Setting the time zone to something other than UTC doesn't actually change the system clock. The system clock continues to run in UTC. Instead the computer simply remembers that you, the user, would like to see the time in some particular time zone, and converts the UTC system time to your preferred time zone whenever it's needed.
So the reason the sleep function won't be affected by a daylight savings time change is that the computer calculates the wake-up time relative to the system clock, which is always UTC.


Parameters' uncertainty for small sample size

Question: Suppose we have a small set of numbers (5 to 10 observations), and we're trying to fit a distribution to this set. Also, we know that all numbers are positive. I tried to fit lognormal, but I'm not sure how good my estimates are since the sample is very small; also I'm not sure how whether it is enough to look at goodness-of-fit test due to the small sample size.
Any suggestions on how to tackle this issue (i.e., to be more confident, certain, about my estimates)?
Comment: That, at least, is possible to make statements about. :) It would help if you told us which estimators you used for the parameters.
Comment: It's not clear if you're asking about how to assess whether the distribution is lognormal or about the uncertainity of the estimates of the lognormal parameters. In the former case, graphical investigation, using for instance boxplots and histograms, can give you a _very_ rough idea about the distribution, but apart from that, there really isn't much that you can do with only 5-10 observations.
Comment: my main concern is not the distribution, but how good my estimates are assuming i know the distribution...
Answer: I would not recommend using a goodness of fit test for such small sample. For example, if you simulate $5-10$ observations from a log-normal distribution, then the Shapiro-Wilk normallity test would fail in the sense that the associated p-value would be higher than $0.05$ more than $30\% +$ of the times, failing to provide the desired power/signficance level. See the following R code.
<code>count = rep(0,10000)

for(i in 1:10000){
x = exp(rnorm(10))
if(shapiro.test(x)$p.value>0.05) count[i] = 1
}

mean(count)
</code>
You might consider Maximum Likelihood Estimation (MLE) and quantifying the accuracy of the estimation by constructing confidence-likelihood intervals for the parameters. One option consists of using the profile likelihood of the parameters $(\mu,\sigma)$.
In this case, the MLE of $(\mu,\sigma)$ for a sample $(x_1,...,x_n)$ are
$$\hat\mu= \dfrac{1}{n}\sum_{j=1}^n\log(x_j);\,\,\,\hat\sigma^2=\dfrac{1}{n}\sum_{j=1}^n(\log(x_j)-\hat\mu)^2.$$
Now, you can use the well-known result that a likelihood interval of level $0.147$ has an approximate confidence of $95\%$. The following R code shows how to calculate these intervals for $\mu$ and $\sigma$ numerically and how to plot the profile likelihoods for your sample.
<code># Your data
dat = c(0.6695,0.5968, 0.7641, 0.7252, 0.7779)
n = length(dat)

# Profile likelihood of mu
p.mu = function(mu){
muh = mean(log(dat))
return(  (sum((log(dat)-muh)^2)/sum((log(dat)-mu)^2))^(0.5*n)  )
}

# Plot of the profile
vec = seq(-0.75,0,0.01)
rmvec = lapply(vec,p.mu)

plot(vec,rmvec,type="l")

p.muint = function(mu) return(p.mu(mu)-0.147)

# Approximate 95% confidence interval of mu
c(uniroot(p.muint,c(-0.6,-0.4))$root,uniroot(p.muint,c(-0.3,-0.1))$root)

# Profile likelihood of sigma
p.sigma = function(sigma){
muh = mean(log(dat))
sigmah = sqrt(mean((log(dat)-muh)^2))
return(  (sigmah/sigma)^n*exp(0.5*n)*exp(-0.5*n*sigmah^2/sigma^2) )
}

# Plot of the profile
vec1 = seq(0.01,0.3,0.001)
rsvec = lapply(vec1,p.sigma)

plot(vec1,rsvec,type="l")

p.sigmaint = function(sigma) return(p.sigma(sigma)-0.147)

# Approximate 95% confidence interval of sigma
c(uniroot(p.sigmaint,c(0.05,0.1))$root,uniroot(p.sigmaint,c(0.15,0.3))$root)
</code>
I hope this helps.
Comment: Thanks for the help. I truly appreciate it. I'm going through it trying to understand it... I tried it on my sample, but it gave me 2 errors : Error in uniroot(p.muint, c(-1, 0)) :
  f() values at end points not of opposite sign Error in uniroot(p.sigmaint, c(0.4, 0.75)) :
  f() values at end points not of opposite sign
Comment: here is my sample: dat <- c(0.6695,0.5968, 0.7641, 0.7252, 0.7779)
Comment: @act00 Fixed for your sample! Please, have a look.
Comment: Thank you so much. Let me please go over to digest everything and will ask some questions. Thanks again!!!
Comment: Isn't it good that Shapiro-Wilk rejects > 30% of the time, given that the sample isn't in fact Normal?  And of course that number depends on the parameters of the lognormal, holding the sample size fixed, as the lognormal can be made arbitrarily close to a Normal.
Comment: @jbowman As I mentioned, the p.value is greater than 0.05 (much greater), then it is actually **not rejecting** (at the 5% level) more than 30% of the time.
Comment: Hah! Chasing you across threads here... I see your point, it's a statement about power, or lack thereof.  That "/significance level" threw me for a bit.
Comment: I've two questions, and please correct me if I'm wrong. First, now if I want to calculate $P(x<0.80)$, how this can be done? Second, what if I decided to use t distribution (log t)? what should I change in the code. Thanks again.
Comment: @jbowman Haha, nice one. Yeah, probably I did not explained properly. My Swahili is better than my English.
Comment: @act00 You can calculate this using the estimated parameters $P(X<80)=$`plnorm(0.8,muh,sigmah)`. If you want to use a log-t, then you would need to change a lot of things. For instance, you would need to obtain the MLE numerically because they do not exists in closed form as far as I know. This can be done but I think the accuracy would be worst because you have to include an additional parameter, the degrees of freedom.

Parameters’ uncertainty for small sample size


r: %*% and lapply/mapply when arguments are reversed

Question: I have been wondering the most efficient way to do matrix multiplication on lists.
Let's say I have a matrix A and a list of matrices B:
A = matrix(c(1,2,3,4), ncol=2, nrow=2)
B = list(matrix(c(1,2,3,4), ncol=2, nrow=2), matrix(c(4,3,2,1), ncol=2, nrow=2))

And I want the list of crossproducts of A and each element b of B:
A %*% b

I was trying to use lapply() but couldn't figure out how to ensure the right sequence of multiplication without extra nested lapply's to transpose each b...
lapply(B, '%*%', A) # which gives a list of b%*%A rather than A%*%b
lapply(lapply(lapply(B, t), '%*%', t(A)), t) # gives the answer but geez...

Could anyone kindly give me some pointers here?
Answer: Just define your function inside lapply(), like so
lapply(B, function(x) A%*%x)


Is the zero ring a domain?

Question: Is the zero ring usually considered a domain or not? Wikipedia says:

The zero ring is not an integral domain; this agrees with the fact that its zero ideal is not prime. Whether the zero ring is considered to be a domain at all is a matter of convention, but there are two advantages to considering it not to be a domain. First, this agrees with the definition that a domain is a ring in which $0$ is the only zero divisor (in particular, $0$ is required to be a zero divisor, which fails in the zero ring). Second, this way, for a positive integer $n$, the ring $\Bbb Z/n\Bbb Z$ is a domain if and only if n is prime.

What are the arguments for/against this convention (besides the ones listed above)? What does the literature (i.e. your favorite textbooks) say on this matter? Note that I am specifically talking about domains, not integral domains; the only difference between them aside from the non-triviality assumption mentioned here is that domains are not required to be commutative.
Comment: I can't think of any argument for it. On the other hand, excluding the $0$ ring in the definition means to not have to consider it as a special case or exception in your proofs.
Comment: People often exclude the zero ring even as a ring (i.e. By prefacing a statement with "Let $R$ be a ring with $0\ne 1$..."). It's pretty convenient for an ring to have a quotient which is a field. If someone wants to exclude/include the zero ring into integral domains, I'd like to know why they'd include it into rings.
Comment: @A.P. There's no free lunch. If you make extra assumptions in your proof, that just means that it won't be as widely applicable, and users of the theorem will need to work harder to prove the extra assumptions, even if they are not necessary. I find it decidedly unnatural to go so far as to exclude the zero ring as a ring, because it has all the properties you would expect of a ring. The issue comes in when you start dividing things and $0$ is invertible, but domains are on the border of acceptability here.
Comment: Sure, someone would need to prove the result for the $0$ ring. The point is that most results are either trivial or inapplicable for the $0$ ring, so having to mention it every time is just an annoyance.
Answer:
Is the zero ring usually considered a domain or not?

Nearly every ring theory text says it is not. I bet guessing that 90% of texts agree with this is a conservative guess.
The main thing is what the first sentence you cited tries to say (but does not do a good job with): we want to say that $R/P$ is a prime ring if and only if $P$ is a prime ideal. (For noncommutative rings, this becomes $R/P$ is a domain if and only if $P$ is a completely prime ideal, that is, one satisfying the commutative definition of "prime." This does not matter much for the following discussion.)

What does the literature (i.e. your favorite textbooks) say on this matter?

If $\{0\}$ were considered a domain, then $R/R$ would be a domain for any ring $R$, so that $R$ is a prime ideal of itself. This is (in every reference I know) explicitly ruled out by the definition of a prime ideal. You could of course object that this is just an equivalent question ("why shouldn't the whole ring be considered a prime ideal?")
A very similar discussion took place here where someone asked why the zero ring wasn't considered a field. The argument that I gave there applies pretty well in this situation too. Our intuition of primes being "indecomposable" leads us to the idea that a prime ring shouldn't decompose into other rings. Certainly decomposing a prime ring into a product of multiple prime rings would be unattractive. But if you let $R=\{0\}$ be a prime ring, then $R\cong\prod_{i\in I} R$ or any nonempty index set that you like.
In particular if we let $\{0\}$ be called a domain, then we'd have a domain that is isomorphic to infinitely many copies of itself, which certainly seems like poor behavior for a domain.
Comment: Regarding "Why shouldn't the whole ring be a prime ideal?", for some reason I am much more comfortable with this definition. Similar to maximal ideals or filters in topology ($\emptyset\notin F$), without this assumption many statements tend to trivialize without adding the condition back in anyway.
Answer: I would like domains to have fields of fractions. Fractions are equivalence classes of ordered pairs $(p,q)$ where $q \neq 0$. Since all elements of the zero ring are zero, there are no such pairs and one deduces that the "field of fractions" should be the empty set. But the empty set is not a field (does not contain $0$ or $1$.)
Comment: @MarioCarneiro What do you mean the zero ring isn't commutative? $0 \times 0 = 0 = 0 \times 0$.
Comment: @Mario: It is common in many parts of mathematics to define a **domain** to be a commutative ring without zero divisors.  In particular, many people who study commutative algebra do not like the term **integral domain**, because the word "integral" has a completely different meaning in that subject.  (David: your proof of the commutativity of the zero ring can be simplified!)
Comment: (I checked out the project.  It looks like you folks are doing very nice work.)
Comment: I don't buy this argument. Of course you aren't going to get a *field* of fractions, because the multiplication isn't commutative. But I'm not even sure you will get a well-defined skew field / division ring with the usual construction. Even the [total ring of fractions](https://en.wikipedia.org/wiki/Total_ring_of_fractions), which allows zero divisors, has to assume that the ring is commutative at the outset.
Comment: See also: [Ore condition](https://en.wikipedia.org/wiki/Ore_condition), which details the conditions under which one can expect a noncommutative ring or domain to have an associated division ring of fractions. It would seem that zero ring cannot be an Ore domain for the results on that page to hold.
Comment: I mean that domains in general aren't commutative, so that "I would like domains to have fields of fractions" is wishful thinking. What it seems you mean is "I would like *integral domains* to have fields of fractions", in which case there is no problem since the zero ring is not an integral domain.
Comment: @PeteL.Clark If a domain is an (evidently nonzero) commutative ring without zero divisors, then what is an integral domain (as that is my exact definition of integral domain)? FYI, the context of this question is in defining these notions in a formal system, so the goal is to have a definition that everyone agrees on, even across different areas of study (to the extent that is possible).
Comment: @Mario: I for instance do not use the term "integral domain" in my papers, though I may say something like "By a domain we mean a commutative nonzero ring without zero divisors".  I didn't quite understand what you meant by defining these notions in a formal system: in a formal system, everything means exactly what you say it means.
Comment: Not everyone agrees whether $A \subset B$ means containment or proper containment, so I think having a "universally understood terminology" is prohibitively difficult.  It is better to be flexible, and to assume, when possible, that people are using terminology so as to make correct, meaningful statements (as in David's answer).
Comment: @PeteL.Clark Of course, in a formal system (as in any math paper or book), everything means what you define it to mean. However, it is also desirable to make sure that one's definitions matches the "general usage", in order to ease understanding and minimize misconceptions. The constraint in a formal system, though, is that you must make one definition once and for all - you can't change the definition later in the development when you have moved on to another topic. (e.g. to me $A\subset B$ means proper containment and $A\subseteq B$ means containment, because a choice had to be made).
Comment: @Mario: No, not all terms are defined in math papers; there's some amount of cultural understanding required, most often enough to make the paper difficult to read by the greater community.  Anyway, what you say is desirable certainly is, but what I'm saying is that it turns out not to be possible in many cases.  Therefore in practice it is more important to choose some reasonable terminology that people won't complain too loudly about, and make sure to explain it clearly.  If you read math papers dogmatically expecting $A \subset B$ to mean proper containment, you'll get in trouble.
Comment: (I'm also not completely sure that I understand what you mean by "formal system".  A formal system is something like the first-order predicate calculus.  One has a defined alphabet consisting of logical symbols, variable names, and so forth.  I have never seen a formal system in which "domain" appeared: that would either be a strange name for a variable or a very confusing way to write a product of six things!)
Comment: @PeteL.Clark [This](http://us2.metamath.org:88/mpegif/isdomn.html) is the formal system I am talking about, a proof system and library of proofs called Metamath. I'm trying to develop abstract algebra in it, but if you can understand the linked statement you will know that the current definition of a domain does not include the non-zero assumption, which is why I opened this question. (What does "a very confusing way to write a product of six things" mean? What six things are you referring to?)
Comment: The six things are *d* *o* *m* *a* *i* and *n*.  My point was that a "domain" is usually not part of the alphabet of a formal system.  But I see what you mean: you have not just a formal system but also a collective enterprise to find and exhibit formal proofs of mathematical theorems within that system.  Certainly you will want to choose terminology carefully while doing such a project.  But as above, you can't choose it so carefully that everyone will know what you mean w/o reading an attached documentation/glossary of terms, so I hope that the project is putting effort into that too.

Is the zero ring a domain?


iOS : Dynamically introduce UILabel or UITextView on an image inside iPad application by tapping and dragging

Question: I need to annotate on an image.
Probably a simple solution that I am not able to see at the moment.
I need to introduce a UILabel or UITextView onto an image that is currently displaying on the iPad probably by the TouchesBegan/TouchesMoved/TouchesEnded methods (which is something I tried) and then add some text onto the created UILabel or UITextView. Consequently, I need to embed the added text onto that particular image.
Any ideas?
Comment: That sounds reasonable.  Create a label, add it as a subview to the one holding your image, change its frame (or center) when touches move....  What part of it is giving you trouble?
Comment: I have [this](http://stackoverflow.com/questions/6992830/how-to-write-text-on-image-in-objective-c-iphone) method that needs to be implemented in order to embed that text onto the image. And I think `textview.text` will take the string that is inside the UITextView. And then perhaps "destroy" that UITextView. Is this the way to go about this? Or can you think of a better way?
Answer: So you will create a custom label, move it around, and when the user is done moving it, you will use:
<code>CGSize size = image.size;
UIGraphicsBeginImageContextWithOptions( size, NO, 0);
CGContextRef context = UIGraphicsGetCurrentContext();

// UIImages and CGImageRefs are flipped
CGAffineTransform flipVertical = CGAffineTransformMake(1, 0, 0, -1, 0, size.height);
CGContextConcatCTM(context, flipVertical);

CGContextDrawImage(context, rect, [image CGImage]); // draw the image

// CGContextConcatCTM(context, flipVertical); uncomment if the text is flipped - not sure now

[label.text drawAtPoint:label.frame.origin withFont:label.font];

UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return image;
</code>
Comment: If it works for you would be super if you accepted the answer.


iOS6: How to connect UIViewController in Storyboard via Identifier?

Question: Does anyone already know how to connect UIViewControllers in Storyboard with the identifier? It does not seem to work anymore..
Comment: Perhaps you mean UIViewController?
Answer: Select the UIViewController, once u see a highlighted UIView (usually blue color) then go to the left most column on the XCODE. Select the Attribute inspector , you will see "Title" and "Identifier" label in the "View Controller" section. Give any name to it and in the program to refer to your view controller you can do something like this:
<code>UIStoryboard *us = [UIStoryboard storyboardWithName:@"MainStoryboard" bundle: nil];
FirstViewController *fvc = [storyboard instantiateViewControllerWithIdentifier:@"firstview"];
</code>
This works , let me know if u have any issues.
Comment: Ok, in a new project it works well. I think it's an xcode error.


Using Flask to iterate over an array to input into a PyMongo search

Question: I have an array, let's say this one:
[ObjectId('5feceb11b5ffb79d03df04a0'), ObjectId('600c1c83e5c06febfb7f8ab5'), ObjectId('5fe8f9227221766fa23707d0'), ObjectId('5fe995f93137252b3bb37bbe')]

From that, I would like to find each ObjectId's profile in a database.
So all I would need to do is something like this:
person = mongo.db.users.find_one({"username": **each ObjectId**})

Where each ObjectId is one of the ObjectId's in the array.
This way then:
person == user1; and
person == user9; and
person == user13; and
person == user16

From there, I can probably manage this in Jinja as a for loop.
Answer: If you want multiple documents to be returned you
can also use find() method instead but as per your use case you might need to use $in operator that might slow down if your list is large but still it would be better if executed in batches saving round trips to db and for better performance.
Then make sure each element in your list has bson ObjectId type and not python str, if not you can type cast python str to bson ObjectId by importing
ObjectId
>>> from bson import ObjectId
>>> print(type(ObjectId('5e9dc0548a4617a55665624f')))
>>> <class 'bson.objectid.ObjectId'>

So after ensuring you have a list of ObjectIds, then you can query using find and $in as follows:
>>> users = [ObjectId('5e9dc0548a4617a55665624f'),ObjectId('5e9dc0a75db97122a193aa51')]
>>> q = collection.find({'username': { "$in": users}})

Comment: Thanks! I actually was just solving that this way now too. Thank you.


Using `chan pending output` instead of writable fileevent

Question: Yo, I've written a server with a simple protocol: the client sends a line, the server sends a line back in response, repeat. To prevent a client from filling Tcl's output buffer by sending lots of lines but not accepting data back, can I just check chan pending output instead of using the writable fileevent?
proc respond {stream msg} {
    if {[chan pending output $stream] <= 1024} {
        puts $stream $msg
    } else {
        #close $stream
    }
}

Answer: For output, chan pending output will correctly describe the number of bytes waiting in the output queue. Normally, that value will be bounded by the -buffersize value that you chan configure (or fconfigure) it to have.
That value will only be exceeded when the channel is non-blocking; with a blocking channel, when the value would go over it, instead there's a blocking write to the underlying device (socket, pipe, file, serial line, whatever) so by the time you could see that it went over, it's back under the limit again.
But if you're using non-blocking channels, you really should use chan event (or fileevent). Luckily for the actual writes, Tcl will actually do this for you automatically; the single most useful thing you could want from a writable event is already there. In practice, the most common actual use of a writable event is in detecting when an async socket connection becomes ready for service.
So what you are doing will work, but you'll have to think carefully about what to do if the output buffer is "getting full"; the idea that a message can need to be delayed is a place where a simple abstraction tends to become leaky. With 8.6's coroutines, you could (probably) do a transparent suspend or something like that, but getting that sort of thing right can take a little thought. (For example, a GUI client might need to show a busy indicator and put things into a state where the user can't enter more requests.)


SSH works locally but not remotely

Question: Recently ran into a hardware issue on my CentOS machine.  After a PSU, ram, mobo and CPU replacement I think I have the hardware issue resolved.
However, I believe I have a network configuration issue causing SSH remote connection failures.
I tried regular ssh using my original account and key and I receive a connection timeout after server is expecting: debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP.
From the server itself with a new account:
$ ssh -v -o PubkeyAuthentication=no chris@localhost
Last login: ...
[chris@dev ~]$

From a remote connection on the LAN to try remote SSH:
chris::Internets|10 ~ $ ssh -v -o PubkeyAuthentication=no chris@pug
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/chris/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: Connecting to pug [192.168.1.175] port 22.
debug1: Connection established.
debug1: identity file /Users/chris/.ssh/id_rsa type 1
debug1: identity file /Users/chris/.ssh/id_rsa-cert type -1
debug1: identity file /Users/chris/.ssh/id_dsa type -1
debug1: identity file /Users/chris/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH_5*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
Read from socket failed: Operation timed out

I have verified I can:

ping remote boxes on lan and internet from server
cannot wget web pages from server
ping server from lan
can access ssh port from lan or remote connection (still receive ssh errors)

I did see a post regarding DNS resolution issues causing an issue, I have UseDNS No which should avoid DNS entirely and not cause issues.
Any ideas here as I am scratching my head for what else to look for?
Edit:
/var/log/secure has the following contents:
Nov 29 11:19:45 dev sshd[5978]: fatal: Read from socket failed: Connection reset by peer

Also, I checked and SSH is listening on 22 as it should be.
[root@dev ~]# lsof -i TCP:22 | grep LISTEN
sshd 5424 root 3u IPv4 39030 0t0 TCP *:ssh (LISTEN)
sshd 5424 root 3u IPv6 39032 0t0 TCP *:ssh (LISTEN)

To avoid complications, I flushed iptables:
[root@dev ~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Comment: Please post the output of more standard debugging commands: "/bin/netstat -rn" to insure you have a default gateway and no strange routes. "/sbin/arp -an" to insure you have a MAC address for your default gateway. "/sbin/ip addr" to insure your netmask, broadcast, etc. are set right. "/usr/bin/dig www.google.com" and "/usr/bin/dig www.google.com @22.214.171.124" to confirm DNS is working. Finally "wget -O/dev/null --debug $IP" where IP is one of the IPs you get from the DNS lookup of google.
Comment: Check the logs on the server.
Comment: Any log in particular?  The server has been running for a handful of years, I don't see anything in /var/log/messages.  I just updated question with /var/log/secure.
Comment: The fact that you can't wget anything is alarming; I'd say do a packet capture using tcpdump and see what's going on.
Answer: I suspect a change in networking configuration because your MAC address has changed. If you swap out mainboards on an existing Linux installation, the file /etc/udev/rules.d/70-persistent-net.rules will create new entries for your devices and give them new names. So, if you had eth0 and eth1 before, you will now probably have eth2 and eth3. You need to manually update that file after changing network adapters.
Additionally, can you show your firewall rules (iptables -L -n)?
Comment: With new motherboard, it did create new network devices.  I updated /etc/sysconfig/network-scripts/ifcfg-eth0 with mac address and IP.  I flushed iptables as well so they should be empty.
Comment: I did notice a duplicate eth0 in /etc/udev/rules.d/70-persistent-net.rules.  I removed the old entry and rebooted, the issue still remains.  I believe it is network configuration related because I cannot wget a webpage from the server.
Answer: Has the SSH host key changed due to the reconfiguration?  If so then perhaps the known_hosts file on the REMOTE side is not in sync with the new host key and thus is rejecting the connection.
Comment: I removed the lines from known_hosts on my client.

SSH works locally but not remotely


Should zookeeper be run on the worker machines or independent machines?

Question: We have several kinds of software that use zookeeper like Solr, Storm, Kafka, Hbase etc.
There are 2 options to install zookeeper cluster (more than 1 nodes):

Embedded cluster: Install ZK on some of the same machines as the other software are installed OR
External cluster: Have a few not very powerful but dedicated zookeeper machines (in the same region, cloud and data-center though) to run zookeeper on.

Which is a better option for cluster stability? Note that in both the cases, we always have an odd number of machines in our zookeeper cluster and not just one machine.
It appears that the embedded option is easier to setup and is a better use of the machines but the external option seems more stable because a loss of single machine means the loss of just one component (Loss of a machine in embedded zookeeper means loss of zookeeper node as well as the worker node of Solr, Storm, Kafka whatever the case maybe).
What is the industry standard to run zookeepers in production for maximum stability?
Answer: Zookeeper is a critical component for a Kafka cluster but since the implementation of the new generation of clients the load on ZK has been greatly reduced and is now only used by the cluster itself. Even though the load is usually not very high, it can be sensitive to latency and therefore the best practice is to run a Zookeeper ensemble on dedicated machines and optimally even use dedicated disks for ZK transaction logs to avoid IO contention.
By using larger Zookeeper ensembles you gain resiliency but this also increase communication within the cluster and you could lose some performance. Since Zookeeper works with simple majority voting you need an odd number of nodes for it to make sense. A 3 node ensemble allow losing 1 node without impact, a 5 node ensemble allow losing 2 nodes and so on.
In practice, I´ve seen small, low workload clusters run very well with Zookeeper installed on the same machines as the Kafka nodes but if you aim for maximum stability and have increasing traffic, separate clusters would be recommended.
Answer: You should consider yourself discouraged from using internal ZooKeeper in production.
Its good to have external zookeeper, Best if Zookeeper ensemble(two or more)
If you have one zookeeper node and it might create problems when it goes down.
if you have cluster setup of zookeeper nodes and if one zookeeper node goes down the remaining majority nodes are running will continue to work.
More details
Answer: For SolrCloud, we strongly recommend that Zookeeper is external, and that you have at least three of them.
This does NOT mean that it cannot run on the same servers as Solr, but it DOES mean that you should NOT use the zookeeper server that Solr itself can start, embedded within itself.
Here's some information related to performance and SolrCloud that touches on zookeeper:
https://wiki.apache.org/solr/SolrPerformanceProblems#SolrCloud
Whether or not you need completely separate machines, or even separate disks for the zookeeper database when running on the same machine as Solr, is VERY dependent on the characteristics of your SolrCloud install.  If your index is very small and your query load is low, it's possible that you can put zookeeper on the same machines and even the same disks.
For the other services you mentioned, I have no idea what the recommendation is.


The geometry in this probability question is unclear to me.

Question: On the circle: $x^2+(y-1)^2=1$ a point $B$ is chosen at random . Let $C(Z,0)$ be the point in which $0x$ axis and ray $AB, A(0,2)$ intersect. Find the distribution function of $Z$.
Answer:
Let abscissa of $D$ be $z$. Random variable $Z$ is less than $z$ if the point $B$ is on the arc $AE$ going counter clockwise. We have,
$$F_Z(x)=P\{Z<z\}=\frac{2 \pi - (\pi -2\arctan{\frac{z}{2}})}{2 \pi}= \frac{1}{2}+ \frac{\arctan{\frac{z}{2}}}{\pi}, z \in R$$
I don't know which angle to look at, and don't understand how they got this.
Answer: The reesult depends on the definition of "chooosing of a point on the circle"!
I suppose that choosing a random point of the circle is equivalent to choosing a random angle $\alpha$ with uniform distribution over $[0,2\pi)$. As shown in the figure below, if $\alpha$ is given then
$$Z=2\tan\left(\frac{\alpha}{2}\right).$$

So, the distribution function of $Z$ can be calculated:
$$F_Z(x)=P(Z<x)=P\left(2\tan\left(\frac{\alpha}{2}\right)<x\right)=P\left(\alpha<2\arctan\left(\frac{x}{2}\right)\right)=\frac{1}{2\pi}\left[2\arctan\left(\frac{x}{2}\right)+\pi\right]$$
Here is the graph of the distribution function:
Answer: The distribution of $Z$ is also called "Cauchy Distribution".
Let the angle between AD and Y axis be $\theta$, $\displaystyle tan\theta = \frac{z}{2}$,
so $\displaystyle d\theta=\frac{2}{4+z^2}dz$.
$\theta$ can be any angle in the range of $\displaystyle (-\frac{\pi}{2}, \frac{\pi}{2})$, so $\displaystyle \theta \sim U(-\frac{\pi}{2}, \frac{\pi}{2})$,
and the density function is: $\displaystyle f(\theta)=\frac{1}{\pi}$.
Hence, here we get the cumulative distribution:
$\displaystyle F_Z(z)=\int^{\theta}_{-\frac{\pi}{2}}f(\theta)d\theta = \int^{z}_{-\infty} \frac{1}{\pi} \frac{2}{4+z^2}dz$
$\displaystyle =\frac{1}{2}+\frac{arctan\frac{z}{2}}{\pi}, \hspace{6mm} z\in R$


Last inserted Id in Java Hibernate

Question: How do we get the last inserted value? Below is my function which inserts values. But I want to fetch last inserted Id which is auto incremented. I am using PostgreSQL
@Override
public int executeInsertSQLSelect(String sqlQuery) throws HibernateException {

    SQLQuery query = getCurrentSession().createSQLQuery(sqlQuery);

    logger.debug("SQL query to be executed is : " + query);

    int noOfRecordsUpdatedOrDeleted;

    try {
        noOfRecordsUpdatedOrDeleted = query.executeUpdate();

    } catch (HibernateException hbExp) {
        logger.error(String.format(
                "Error occured while executing query without Scalar"
                        + " mapping - %s ", query.getQueryString()), hbExp);
        throw hbExp;
    }
    return noOfRecordsUpdatedOrDeleted;
}

Comment: Hibernate session.save() return ID of inserted row.
Comment: Use `insert ... returning`. That will return the generated ID. Or use `Statement.getGeneratedKeys()` with plain JDBC. Or run a `select lastval()` after the insert.
Comment: Refer this link, this may help you. http://stackoverflow.com/questions/21829023/how-can-i-get-last-inserted-id-using-hibernate
Answer: If you are using ms sql server then you can make the use of OUTPUT
for example
String sqlQuery=insert into employee(employeeName) output values('john')
SQLQuery query = getCurrentSession().createSQLQuery(sqlQuery);
query.getSingleResult()

Comment: Hi Iam using `Postgresql`
Comment: why do you you hibernate save method? Hibernate save method will return you id inserted
Comment: not able to use it.. bcos its lot of forighn key constratnts
Comment: you may get equivalent of output in postgresql, search in google

Last inserted Id in Java Hibernate


Bipartition and coloring

Question: I am having hard time solving the following problem:

Let $k$ be a positive integer. Then for every $2^k$ colorable graph $G=(V,E)$ we have $E=E_1 \cup \ldots \cup E_k$ and $\forall 1 \leq i \leq k \ : \ (V,E_i)$ is bipartite.

I thought about looking at the $2^k$ partition of $E$ according to some $2^k$ valid coloring , and then tring to build from them the "k-bipartitioning" , but I had no success doing so.
I will be happy for some clues about what to do.
Comment: Clue: you may take inspiration from the [poisoned wine puzzle](http://www.mindcipher.com/puzzles/121).
Answer: Each vertex has a colour which is a length-$k$ binary number.  For any edge, these two binary numbers differ in at least one digit.  How can you sort these edges into $k$ different groups?
Comment: Thanks! Apparently I got confused in my first proof :(

Bipartition and coloring


Can I get advice on my nginx configuration (as a proxy in front of Jira and Confluence)?

Question: I was wondering if I could get some advice on my nginx configuration.  The config seems to be working, but I'm unsure if I'm doing everything properly.  The basic idea is to have a Jira and Confluence server (in separate Tomcat instances) running on the same machine, with nginx in front to handle SSL for both.  I want only SSL connections to be made to Jira/Confluence.  Jira is running on 127.0.0.1:9090 and Confluence on 127.0.0.1:8080.  Here is my nginx.conf, any advice or tips would be greatly appreciated.
<code>user              nginx;
worker_processes  1;

error_log         /var/log/nginx/error.log;
pid               /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {

    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] $request '
                      '"$status" $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    # Load config files from the /etc/nginx/conf.d directory
    include /etc/nginx/conf.d/*.conf;

    # Our self-signed cert
    ssl_certificate     /etc/ssl/certs/fissl.crt;
    ssl_certificate_key /etc/ssl/private/fissl.key;

    # redirect non-ssl Confluence to ssl
    server {
        listen 80;
        server_name  confluence.example.com;
        rewrite ^(.*) https://confluence.example.com$1 permanent;
    }

    # redirect non-ssl Jira to ssl
    server {
        listen 80;
        server_name  jira.example.com;
        rewrite ^(.*) https://jira.example.com$1 permanent;
    }

    #
    # The Confluence server
    #
    server {
        listen       443;
        server_name  confluence.example.com;

        ssl on;

        access_log  /var/log/nginx/confluence.access.log  main;
        error_log   /var/log/nginx/confluence.error.log;

        location / {
            proxy_pass http://127.0.0.1:8080;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_set_header Host $http_host;
        }

        error_page  404              /404.html;
        location = /404.html {
           root   /usr/share/nginx/html;
        }

        redirect server error pages to the static page /50x.html

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
           root   /usr/share/nginx/html;
        }

    }

    #
    # The Jira server
    #
    server {
        listen       443;
        server_name  jira.example.com;

        ssl on;

        access_log  /var/log/nginx/jira.access.log  main;
        error_log   /var/log/nginx/jira.error.log;

        location / {
            proxy_pass http://127.0.0.1:9090/;
            proxy_set_header X-Forwarded-Proto  https;
            proxy_set_header Host $http_host;
        }

        error_page  404              /404.html;
        location = /404.html {
            root   /usr/share/nginx/html;
        }

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }

    }
}
</code>
Answer: Looks fine, but, you'd probably get better performance if you served static files (images, css etc.) directly fron nginx rather than proxying it to tomcat.
Comment: and moved nginx to it's own hardware, thus offloading ssl to separate cpu.


How do I monitor all Terminated messages in akka?

Question: I want to create an Actor which subscribes to Terminated messages for all Actors created in the system. How do I do that?
I have tried below - which does not seem to work,
getContext().system().eventStream().subscribe(getSelf(), Terminated.class);

and I would like to avoid below - because then I will need to do that for each actor I want to monitor.,
getContext().watch(subject);

Answer: A similar (but a bit less hacky) way to solve this would be to implement a Publish/Subscribe pattern (as described here in Scala for instance)
Then you could just have your Actor subscribe to a specific event (such as "Terminated") like that:
EventStream.subscribe(Foo.onEvent, "Terminated")


Using De Moivre's law to compute $(-\sqrt3+i)^{2/3}$

Question: Question:

If $z=-\sqrt{3}+i$, then $z^{2 / 3} = ?$

My work (which is wrong but I am not sure why):
We can write $z = r(\cos\theta + i\sin\theta)$
$$r = \sqrt{(-\sqrt{3})^2 + 1^2} = \sqrt{4} = 2$$
$$\theta = \tan^{-1} \left(\frac{-1}{\sqrt{3}}\right) = \frac{5\pi}{6} \quad (*)$$
Therefore
$$z = 2\left(\cos\left(\frac{5\pi}{6}\right) + i\sin\left(\frac{5\pi}{6}\right) \right) $$
so (applying De Moivre's)
$$ z^{\frac{2}{3}} = 2^{\frac{2}{3}} \left(\cos\left(\frac{5\pi}{9}\right) + i\sin\left(\frac{5\pi}{9}\right) \right)$$
However, the answer is apparently:
$$-2^{2 / 3} \sin \left(\frac{\pi}{18}\right)+i 2^{2 / 3} \cos \left(\frac{\pi}{18}\right)$$

How does one arrive at the given answer?
Forgive my remembrance of high school trigonometry. In $(*)$ I understand that we can take the negative out of $\tan^{-1}$ resulting in
$$\theta=-\tan ^{-1}\left(\frac{1}{\sqrt{3}}\right)=-\frac{\pi}{6} = \frac{11\pi}{6}$$

which results in a different $\theta$ than I had. My intuition for why $\theta =\frac{5 \pi}{6}$ is because $\frac{-1}{2} \div \frac{\sqrt{3}}{2}$ as $\sin \frac{5 \pi}{6} = -\frac{1}{2}$ and $\cos \frac{5 \pi}{6} = \frac{\sqrt{3}}{2}$. But I am not sure if this is the correct reasoning
Comment: Please write it in the form of $cos\theta + i sin\theta$. It is not $cos\theta+ sin\theta$ and $sin\frac{5\pi}{6} = 1/2$ and $cos\frac{5\pi}{6} = -\sqrt3/2$
Comment: Both the answers are same.
Comment: $\frac {\pi} {18}= \frac {5 \pi} 9-\frac {\pi} 2$.
Answer: You forgot $i$ when writing $z = r(\cos \theta + i \sin \theta)$. Arctangent was computed correctly, so
$$
z = 2\left(\cos(\frac{5}{6}\pi) + i \sin(\frac{5}{6}\pi)\right),
$$
$$
z^{2/3} = 2^{2/3}\left(\cos(\frac{5}{9}\pi) + i \sin(\frac{5}{9}\pi)\right).
$$
Now some trigonometry:
$$
\cos(\frac{5}{9}\pi) = \cos(\frac{1}{2}\pi + \frac{1}{18}\pi) = -\sin(\frac{1}{18}\pi).
$$
Similarly,
$$
\sin(\frac{5}{9}\pi) = \cos(\frac{1}{18}\pi),
$$
and you arrive to the answer.
Comment: Ah I see. I forgot about the compound angle trig identities.


What is the proper place of the Czerny and Hanon exercises in the Piano repertoire?

Question: It seems that every piano teacher has a different view of just how much worth these exercises have.
Is there merit to having beginners start with these exercises and work through a sizeable chunk of them or is it more of a case of, begin with your training and when you start to struggle with a certain part of the technique then do the exercise that develops that part of your technique.
Comment: There are so many questions about Hanon and Czerny on here. I've marked this as a duplicate of one in particular, but it's really a duplicate of many.
Comment: The number of question abut Hanon just goes to show the power of marketing, especially to unsophisticated "customers". A good book title always beats good content!
Answer: The best place for the Hanon exercises is on a bonfire, IMHO. They have zero musical value, and negligible technical value either.
Czerny's many collections of exercises have a lot more value, so long as you realize that they are based on 200-year-old keyboard technique and written for 200-year-old pianos, which were significantly more "fragile" than modern instruments and usually had a lighter key action.
By all means use Czerny in conjunction with more modern studies - but it would very limiting to use only Czerny. A diet of Czerny and nothing else won't teach you to play most of the piano music (of any genre, not just classical!) written in the past 150 years or so.
There is a basic dilemma in music teaching for any instrument, not just piano: do you teach how to play the instrument or how to play particular pieces? Expressed like that, it seems rather obvious that the first option is better, but all students need the gratification of "playing pieces" as well as learning the necessary technical skills. The ideal balance between the two is very student-dependent, of course.
Answer: Technique doesn't come from a finger exercise.  It comes from knowledge of physics, anatomy and mechanics.  Your arm is a "machine" of levers, pulleys, rubber bands, fulcrum and there are proper and improper ways to move.  Hanon nor Czerny can give you that.  No matter how many hours and decades you spend playing them, if you are moving incorrectly, you are only hard wiring improper movement into your brain.
Learn to move properly and a facile technique will just fall out of your hands with no exercise, warmup or need to practice.  Do you need to practice riding a bike?  No, once your brain hard wires proper movement and balance, it is there forever.  The arms are no different. Once a bad habit is hard wired, like the first time we touch a piano, it can take a lifetime to eradicate it.
If you learn to use the incorrect muscles as most pianists do, or non-existent ones as Hanon and Czerny espouse, you will hard wire improper movement into your brain and forever struggle to beat your hands into submission.  That is why teachers spew "endurance and strength" because they don't know the laws of physics which result in effortlessness.
If you've ever run in a three legged race, at first it is a struggle but once you and your partner sync up, it is easy.  However, if one of you doesn't have the mental acuity to sync, there will be an imbalance and there is not much YOU can do about it.
Playing from the wrong muscles creates a myriad of imbalances and teachers think you need "strength and endurance and more practice" to fix it.  No, you need a better teacher.
Playing the piano is all in the arms.  The fingers are the conduit between the brain and piano.  Hanon can be wonderful if you have an ergonomic technique but, if you have an ergonomic technique, you don't need Hanon.
If I see Hanon at a garage sale for a nickel or quarter, I buy it so no one else does.  Then I use it to start the fire in my smoker.  Mmmmm, 16th note burgers.  Gurgle, drool  . . . . (my Homer Simpson impersonation).
If your teacher doesn't know how the abductors affect the flexors or, the importance of pronator and supinator muscles, or the dangers or ulnar and radial deviation, then you've probably already developed bad habits.
Likewise, if a teacher teaches to cross the thumb under the palm . . . RUN!!!!!
Answer: I'd go with the second option, because I think exercises can be very boring and discouraging for beginners.
I have always learned by playing pieces. Each one of them needs different techniques, and by studying those parts you develop them as well. So, why do an exercise instead of studying a part of a piece that employs the same technique? Besides, it's much more fun and gratifying, and I believe the result is not so different.
Of course, some people like to do exercises to improve their technique faster and play more difficult pieces sooner, this is personal.
I believe in general it's better to learn through pieces, not exercises.


Intuition for Autoencoders with Single Hidden Layer and Single Latent Variable

Question: I'm currently creating an autoencoder that reduces a dataset to single a single dimension. For example: If the input is 5, the autoencoder will have the following topology:

Input Layer: 5 dimensions
Hidden Layer: 1 dimension
Output Layer: 5 dimensions

What would be the intuition behind this approach since I'm not having multiple layers? Also, will there be some effect in the case of really high dimensional data?
Answer: by minimizing squared reconstruction error, you are basically looking for a latent representation that explains the most variance in the data. If the hidden neuron has a linear activation function, this will be the same as calculating the first principal component of your dataset. If you want to get more intuition into this, look for an explanation of PCA and the relationship between autoencoder and PCA.
If the hidden neuron is not linear, then the result will be something weird, since with one hidden layer and neuron we cannot learn a meaningful non-linear mapping between the hidden layer and the data. If you have multiple hidden layers with non-linear activation functions, then the non-linear function of these components will explain the most variance in the data, in contrast to principal components that are only related to the data linearly.


Alter Table Filestream runs even if IF statement is false

Question: I am writing a script to update a database to add Filestream capability. The script needs to be able to be run multiple times without erroring. This is what I currently have
IF ((select count(*) from sys.columns a
       inner join sys.objects b on a.object_id = b.object_id
       inner join sys.default_constraints c on c.parent_object_id = a.object_id
       where a.name = 'evidence_data' and b.name='evidence'
          and c.name='DF__evidence_evidence_data') = 0)

begin

 ALTER TABLE evidence SET ( FILESTREAM_ON = AnalysisFSGroup )
 ALTER TABLE evidence ALTER COLUMN id ADD ROWGUIDCOL;

end

GO

The first time I run this against the database it works fine. The second time when the if statement should be false it throws an error saying "Cannot add FILESTREAM filegroup or partition scheme since table 'evidence' has a FILESTREAM filegroup or partition scheme already." If I put a simple select into the if statement and take out the alter table filestream on line it functions correctly and does not perform the if statement. So esentially it is always running the alter table filestream on statement even if the if statement is false.
Any thoughts or suggestions would be great. Thanks.
Comment: Are you sure that you got the spelling of all of the names in the IF correct?  If any one of them is wrong then it will always execute the ALTERs.  (I am especially suspicious of the DF__ constraint name.)
Comment: yes. I am sure. If I run the select in the IF statement by itself, it returns the correct result of 1. Thanks.
Answer: It's hard to reason about your business logic without knowing the details, but I'd say if you only want to alter tables that don't have filestream data space defined, consider a condition based on the following query (if it returns NULL for a given table, it means that filestream data space (file group or partition scheme) is not defined:
select filestream_data_space_id from sys.tables
Comment: I have tried this and am still having the same issue. No matter if the sql is true or false it runs the alter statement and fails. If I put a simple select in the query it acts correctly but as soon as I put the alter statement in it fails trying to run the alter statement.
Comment: so
IF (SELECT [filestream_data_space_id] FROM [analysis].[sys].[tables] where name = 'evidence') IS NULL
begin
select * from analysis.dbo.evidence
end
This does nothing but this
IF (SELECT [filestream_data_space_id] FROM [analysis].[sys].[tables] where name = 'evidence') IS NULL
begin
ALTER TABLE evidence SET ( FILESTREAM_ON = AnalysisFSGroup )
end
errors and says the filestream is already on.


multicollinearity between confounders logistic regression

Question: Im going to investigate if a disease have an negative impact on a binary response variable. The disease is the independent variable with additionally confounders.
I  want to do a manual stepwise regression where I subtraction the confounder with highest p-value, and I wonder if i first have to check for multicollinearity between the confounders and response variable?
Can I use the glm in r with the response variable against confounders, then use the vif() function? Do I also have to check the confounder against each other and the disease variable?
I would really appreciate an answer!
Comment: This type of stepwise selection is extremely unreliable. Please look at [this page](https://stats.stackexchange.com/q/20836/28500) for reasons why. Also, note that multicollinearity usually refers to relations among predictor variables, not relations of predictor variables to outcome variables. If you provide more details about your study (number of cases, proportions in the 2 response categories, number of predictors you are considering, the purpose of your modeling) you could get useful advice on better ways to reach your goal.
Comment: Thank you for your answer! I want to test if RDS(Respiratory Distress Syndrome ) has an negative effect on the development of preterm infants, by tests made at 7 years old. The test result are the response variables and RDS with confounders like birthweight are explanatory variables. I want to find out what confounders to include in the logistic regression together with RDS.
Comment: So I first was going to check for multicollinearity between the confounders. I compared the vif of all independent variables by first doing a logistic regression with all variables and then look at the vif values (vif(glm(result ~., family = "binomial", data = data))), i took away one by one the variables with vif over 5. Is this correct?
Comment: How many cases do you have, and how many potential "confounders" are you evaluating in total? And what exactly is the binary response variable? I thought that RDS was an issue around birth, but you are examining results of tests at 7 years old. Do you have multiple response variables? If so, how many?
Comment: I have  32 preterm with RDS and  28 without RDS, so 60 in total. I have 11 confounders right now. The response variables is the result in ABC movement at 7 years old, percentile ranking between 1-5 normal, 6-15 borderline and 16-96 poor. I have instead of having a ordinal response variables made it a binary with percentile ranking between 1-5 normal and 6-96 poor motor funciton, because I dont know how to check multicollinearity in a ordinal logistic regression. It is 3 test, made into a total score. It is a multiple response variable but in this case the response variables are  the 3 tests.
Answer: Although multicollinearity gets a lot of attention, it is seldom the biggest problem facing studies like yours. My sense it that a lot of the attention to multicollinearity comes from data sets with tens to hundreds to thousands of predictors. That's not your situation.
What's most important is first to set up the model in a way that is best able to answer your question, without overfitting, in ways that deal with multicollinearity. A few thoughts on how you might proceed:
First, you seem to have a total score for the response variable that could reasonably be considered continuous (values from 1 to 96). Try modeling the response as a continuous variable instead of with binary or ordinal cutoffs. You would probably have to do some transformation of the scores (and perhaps of the confounders, too) to meet the requirements of a linear regression model, but I suspect that approach would give you a much better way to evaluate the +/- RDS differences. You could then present results as the coefficient (estimate and standard error) for +/- RDS in the regression model, with the confounders accounted for directly in the regression. That doesn't, however, deal with potential multicollinearity.
Second, you could use your understanding of the clinical issues to choose which of multiple correlated confounders to incorporate into the regression. With a continuous outcome in a standard linear regression model you can reasonably consider 1 predictor per 10 or so cases without overfitting, so you might be able to include RDS along with 4 or 5 thoughtfully chosen confounders and avoid multicollinearity issues that way.
Third, you could use methods that are designed to deal with multicollinearity. Penalized methods like ridge regression could even be considered to take advantage of multicollinearity by including all the variables into the model while lowering the magnitudes of their regression coefficients to prevent overfitting. In your situation you might consider leaving the RDS coefficient as unpenalized while penalizing the coefficients of the confounders.
Fourth, you could try using propensity scores to correct for the confounders. Although propensity scores are typically considered in the context of comparing treatment effects, they might provide a way to correct for the confounders in this +/- RDS context, for example with inverse probability weighting. One approach would use logistic regression to model the probability of having RDS as a function of the confounders, then to weight each case inversely proportional to its estimated probability of RDS given the values of its confounders.
For any of these approaches, however, I would recommend getting some local informed statistical help and using this as an opportunity to extend your knowledge about statistical analysis of clinical data.