how to proxy to my domain with port from request urls that start with api?

Question: I have a loopback 3 set up listening on the port 3000. And my front end app is built with Vue JS. ( I uploaded dist files to the server). Whenever I make an api call (https://example.com/api/xxx), I need to proxy to (https://example.com:3000/api/xxx) in order to avoid cors issues.
How do I resolve this?
FYI, the loopback and vueJS are hosted on the same web server (apache, centos8)
Answer: Use NGINX Reverse Proxy
https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/
http://localhost:5000 - points to your Vue.js app
http: //localhost:3000 - points to your REST API app
<code>server {

  listen 80;

  server_name example.org;

  location / {
    proxy_pass http://localhost:5000;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
  }

  location ^ ~/api {
    proxy_pass http: //localhost:3000;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
  }
}
</code>


Plots do not show with bokeh

Question: i am trying to export a report as HTML using Bokeh with python. Can anybody tell me, why my code is not working? I tried to print a simple Div and succeeded, but as soon as i include plots, the browser still opens, but nothing is visible. Is it not possible to inherit Figure?
I reduced it to the relevant parts:
def export_html(self):
    plots=[]
    plots.append(PlotObject(self.plot_attributes[4]))
    VisuLog(plots)

from bokeh.plotting import Figure, show, output_file

class VisuLog():
    def __init__(self,plots):
       self.plots = plots
       filename="Report_"
       output_file("visu_out\\"+ filename + ".html", title=filename)
       show(self.plots)  # open a browser

class PlotObject(Figure):
   def __init__(self,plot_attributes):
       super(PlotObject, self).__init__()

Thank you for your help.
Answer: Try adding
__subtype__ = 'PlotObject'
__view_model__ = 'Plot'

to the body of the PlotObject class.
But I would advise against extending Bokeh objects unless you want to provide some UI behavior by writing custom JavaScript/TypeScript implementation.
Comment: Thank you for the hint, but i get the following warning: Apparently the PlotObject inherits those from the Figure class

Warning: Duplicate __view_model__ or __subtype__ declaration of 'Figure' for class PlotObject.  Previous definition:
Comment: No idea - works for me on Bokeh 2.0.2. But as I said, I strongly recommend against doing that in the first place since it's an implementation detail and not a public API.
Comment: Okey, thanks updating from 2.0.0 to 2.0.2 indeed helped.

Plots do not show with bokeh


Capture json response value and http status from cpprest sdk pplx task cpp to local variables

Question: I want to write a generic function in cpp that gets JSON data using cpprestsdk and copy the http status response code and the JSON data. The calling method will use the json_resp and http_status codes. Later on, I want to further make this function more generic by passing the URL and use it to get data from different web services. Please let me know how I can accomplish this.
<code>pplx::task<void> handleWebServerRequest( web::json::value json_resp, int *http_status)
{
..
http_client client(L"http://weburl.com:8000/getjsondata");

return client.request(methods::GET).then([](http_response response) -> pplx::task<json::value> {
// Store the http status code to be returned to calling function
*http_status = response.status_code();
..
if(response.status_code() == status_codes::OK) {
   return response.extract_json();
}

return pplx::task_from_result(json::value()); }).then([](pplx::task<json::value> previousTask) {
   try {
        // capture json response to json_resp
        json_resp = previousTask.get();
    }
    catch( const http_exception& e) {
         // print error
    }
});
}
</code>
Comment: Can someone please help on this?
Answer: In my research I have found that the only difference between using  cpprest api to consume a PHP web service and a WCF web service is the function parameter. When consuming a PHP web service you can set the function parameter to an empty string.  Where as when consuming a WCF service you need to pass it a function parameter-because the protocol for receiving requests and issuing responses in a WCF service is very different, but the process of sending requests and receiving responses is asynchronous so there will always be at least three modules, functions or tasks involved. One to make the request. The other to wait and receive the response and another to parse the data which is called asynchronously by the function that receives the response. I suppose you could put all three tasks into one function and use go to statements to execute each task, perhaps use some inline assembly to capture the response, and use pointers in place of parameters - but it is still three tasks anyway you slice it. The two others run in a thread and do not have access to the application data, but the last function that parses the data (the json object) which  is called asynchronously you could make generic. I don't know which web services you want to consume, but I posted two samples on github-Example of Casablanca (cpprestsdk 2.9.1) consuming a PHP web service and  Example of Casablanca (cpprestsdk 2.9.1) consuming a WCF (.net) web service. I believe this should get you off to a good start. To capture the json values you can convert your json values to std strings (as shown below) and then you can store them respectively in a local hashmap by adding a hashmap pointer argument to all three functions and passing a reference to the local hashmap variable from which ever function you are calling it from  where they can be converted to what ever data type you need.
<code> void get_field_map_json(json::value & jvalue, unordered_map <string, string>  * hashmap)
 {

  if (!jvalue.is_null())
   {
    for (auto const & e : jvalue.as_object())
    {
        std::string key(conversions::to_utf8string(e.first));
        std::string value(conversions::to_utf8string(e.second.as_string()));
        (*hashmap)[key] = value;

    }

}
</code>


Yahoo Weather API 3 days Forecast with humidity value

Question: I am building a web app, which uses Yahoo Weather API to provide weather information, based on a ZIP code, provided by the users.
I know that in order to obtain this data for certain number of days I have to add it as a parameter in my request, like so:
http://weather.yahooapis.com/forecastrss?p=33035&u=c&d=3

Which gives this result:
<channel>
....
<yweather:location city="Homestead" region="FL" country="US"/>
<yweather:units temperature="C" distance="km" pressure="mb" speed="km/h"/>
<yweather:wind chill="19" direction="90" speed="11.27"/>
<yweather:atmosphere humidity="78" visibility="16.09" pressure="1021" rising="1"/>
<yweather:astronomy sunrise="7:12 am" sunset="7:36 pm"/>
...
<item>
...
<yweather:forecast day="Wed" date="2 Apr 2014" low="19" high="28" text="Mostly Sunny" code="34"/>
<yweather:forecast day="Thu" date="3 Apr 2014" low="21" high="29" text="Partly Cloudy" code="30"/>
<yweather:forecast day="Fri" date="4 Apr 2014" low="20" high="28" text="Partly Cloudy" code="30"/>
<guid isPermaLink="false">USFL0208_2014_04_04_7_00_EDT</guid>
</item>
</channel>

However I need to be able to get the humidity level for EVERY day in the forecast and not just the current one. I've tried to find solution here and also read the Yahoo API documentation, but it's really a short one.
I've also tried http://www.myweather2.com/ and http://developer.worldweatheronline.com/, but they have the same issue with humidity - it's shown only for the current day and stripped from the whole forecast.
I'll keep trying with other free Weather APIs, but if you can help here I'd be very grateful.
Answer: I've spent some time researching for better APIs than the Yahoo! Weather API and I found something, that works for me, so I've decided to share the info, in case anyone else bump into this some day.
Version 2 of the Forecast API looks like a nice solution, because it provides detailed information, more specifically the humidity index for each day in the forecast, which was what I was looking for in the first place.
There are also number of libraries for this API (languages include PHP, Node.js, Python, Perl and others).
It works with longitude & latitude as in this example
https://api.forecast.io/forecast/APIKEY/LATITUDE,LONGITUDE,TIME

and of course one needs to register for an API key.
The TIME parameter is optional, the temperature uses Fahrenheit metrics by default, but this can be changed with additional ?units=ca parameter.
One disadvantage is that you get only a 1,000 free calls per day, which probably won't be suitable for big applications.
If you know a better answer to the original question, I'll be very happy to hear it. Until then I'll use this solution, without going into much details:
// The default result from a Forecast API call is in JSON,
// so decode it to an object for easier access
$fcObj = json_decode($jsonResult);

if(!empty($fcObj->daily->data[0]->humidity)) {

    // get humidity value in %
    $humidty = $fcObj->daily->data[0]->humidity * 100;

}

Comment: Can you pls let me know that the wind value is in km/h and the temperature is Celsius?
I am also using yahoo weather API but just confuse in their units


Creating a linear model / regression where all data is factors / given in proportion (data included)

Question: Example data:
df <- data.frame(Proportion = c(0.1,0.15,0.22,0.20,0.29,0.37,0.41,0.36, 0.47,0.67),
                 Age  = c("15-20", "20-25", "25-30", "30-35", "35-40", "40-45",
                          "45-50", "50-55", "55-60", "60-65"))
plot(df$Proportion, ylab = "Proportion")

From the plot the Proportion seems to be increasing with Age. What is the correct way to create a to model out of this data to see how Proportion increases with Age? To somewhat also distinguish the difference of proportion between a 15 year old and a 20 year old. Age is assumed to be contionous.
If someone knows articles or books that covers this problem, please let me know as I searched, but not found anything. In my original data set I also have number of observations within each group.
Comment: You write example data and that both are factors. (1) Proportion is a factor? is doesn't seem to be (o need to be). (2) Did you group Age for the sake of brevity here, or is it clumped up like that in the data (i.e., 15-20 and not 15,16,17..)
Comment: You are correct, it is not a factor. The age is grouped is grouped in the data set. Although, I could go back to the original source and extract it non-grouped.
Answer: What I suggest is this. If indeed you can extract the age in a continuous format it might prove advantageous:

The data is more robust.
The way it is currently created makes it ambiguous when 20, 25 etc.. are, as they belong to two groups simultaneously.
caveat - If there is a logical or theoretical reason to group Age, than do so. For example, if certain Age points hold specific meaning, like 21 (drinking age), 65 (pension), 6 (entering elementary school) or otherwise, than grouping is what you want to do.

Now, if you have two interval variables (Proportion and Age), I would plot them to see the connection visually. Since these are only two variables (no other controls), There is no real need for regression (Ill go there in a sec).
--Note that the below is resting on a linear connection (which seems the case with your data)
You can measure the correlation (Pearson's $r$):
> cor.test(df$Proportion,df$Age)

Pearson's product-moment correlation

data:  df$Proportion and df$Age
t = 8.4717, df = 8, p-value = 2.884e-05
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.7917609 0.9880651
sample estimates:
      cor
0.9485315

This means that the connection ($r=0.949$) between the two variables is very very strong (and significant). With a $95%$ chance, the correlation in the actual population is between $0.791$ and $0.988$
Regressing these:
> summary(lm(Proportion~Age, data=df))

Call:
lm(formula = Proportion ~ Age, data = df)

Residuals:
      Min        1Q    Median        3Q       Max
-0.097030 -0.032030  0.009212  0.018409  0.106545

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.075091   0.050445  -1.489    0.175
Age          0.010642   0.001256   8.472 2.88e-05 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.05705 on 8 degrees of freedom
Multiple R-squared:  0.8997,    Adjusted R-squared:  0.8872
F-statistic: 71.77 on 1 and 8 DF,  p-value: 2.884e-05

This enables you to talk about the relation in actual terms: Age's coefficient ($0.01$) means that every increase in one year in Age, raises the average predicted Proportion by $0.01$. $R^2=0.899$ means that Age explains $89.9%$ of the variance in Proportion.
If you want to plot it with the regression line (using ggplot2 here):
ggplot(df, aes(x=Age, y=Proportion)) + geom_point(size=4) + geom_smooth(method = lm)

Comment: I my example I meant for example age 15-20 is above 15 and below 20, so 20 will not be in two groups. The problem with using age is:
Comment: If data is small, the age proportion within every group has a high variance. Some ages is missing.
Comment: That shouldn't matter much, Many variables are continuous but without representatives of every value
Comment: If age is grouped, you should use Spearman rank correlation. Also, proportions should not be modeled with OLS regression, in particular if some values are close to zero or one.


The out parameter does not work in C#. Why?

Question: The out parameter does not work in C#. Why?
Here is my code:
using System;
using System.Collections.Generic;

public class Program
{
    public static void Main()
    {
        var hashSet = new HashSet<int>(2);
        Console.WriteLine(hashSet.Contains(2));
    }

    public ListNode ReverseList(ListNode head)
    {
        helper(head, out var newHead);
        return newHead;
    }

    private ListNode helper(ListNode current, out ListNode newHead)
    {
        newHead = null;

        if (current == null)
        {
            return null;
        }

        var next = helper(current.next, out newHead);

        if (next == null)
        {
            newHead = current;
        }
        else
        {
            next.next = current;
        }

        return current;
    }
}

public class ListNode
{
    public int val;
    public ListNode next;

    public ListNode(int val = 0, ListNode next = null)
    {
        this.val = val;
        this.next = next;
    }
}

Here is the error I am getting:

Compilation error (line 13, col 30): ) expected
Compilation error (line 13, col 37): ; expected
Compilation error (line 13, col 37): Invalid expression term ')'

I have spent 2 hours straight trying to figure out what is wrong. Help me, please.
Here is my code online.
Comment: Declare `newHead` before using it as an `out` parameter, and remove the `var` keyword.
Comment: @MichaelMyers, does not work. It tells me that `A local variable or function named 'newHead' is already defined in this scope`.
Answer: I tried your sample at dotnet fiddle and it seems like the compiler (version) matters. I got the same exception when running your code on the .NET 4.7.2 compiler, but when you change the compiler to Roslyn 3.8 or .NET 5 the sample works.
Answer: Change line 13
helper(head, out var newHead);

to
var newHead = new ListNode();
helper(head, out newHead);

Answer: This can also happen if you're in the wrong Visual Studio version, even if the framework version is correct.  Note that double-clicking the solution (.sln) file doesn't always result in the project opening in the desired VS version.  If you have multiple VS versions installed, first launch the one you want and then open the solution using the File menu.


How to work with Google Provisioning APIs for Domain groups and users with Google Drive API?

Question: With Google Drive APIs, I am able to initalize DriveService Object using OAuth2.0, but don't know how to initialize AppsService object of Google Provisioning API to work with Domain Gorups and Users.  I tried
AppsService _appService = new AppsService("DOMAINNAME", "AuthenticationTokenFromDriveServiceObj");
But, getting "Invalide Token" exception.

It looks like Google Provisioning APIs are still not able to handle Google Drive APIs. am I correct here?
Are Google Provisioning APIs are deprecated by Google? refer link https://developers.google.com/google-apps/admin-api-deprecation-list

Edited:
I am getting Drive service Object from the code given at:
https://developers.google.com/drive/delegation#instantiate_a_drive_service_object
[Here, I tried adding provisioning API scopes also]
And then
<code>PermissionList permissions = _DriveSevice.Permissions.List("root").Fetch();
string userName = permissions.Items[0].Name;
string authenticationToken = auth.State.AccessToken;
AppsService _AppService = new AppsService("DOMAINNAME", authenticationToken);
_AppsService.RetrieveAllUsers(); //This statement errors out
</code>
Answer: You just need to use the setOauth2Credentials with the Oauth2 credentials used to instanciate your Drive instance on your apps service instance, provided of course that your token is valid for scopes of Drive and Provisioning API.
Here are the Oauth2 scopes for the provisioning API
Comment: I have tried with all scopes of Drive and Provisioning APIs separated by Space but still getting Unauthorized error.
Comment: Could you update your question with the code you use to get your token ?
Answer: The Drive API and Provisioning API are scoped differently and can't share OAuth credentials. You need to go through the OAuth flow for the Provisioning API separately, they are completely unrelated APIs.
The Provisioning API has not been deprecated. That list is for APIs subject to the deprecation policy, which is posted at https://developers.google.com/google-apps/admin-api-terms.
Comment: You are correct that they are scoped differently but there's no need to push users through 2 auth workflows. You just need to put a space between the 2 scopes when making the request.


Markov: Expected time of first visit to a state starting from that state.

Question: Question: Calculate the expected time of first visit to state 2 given we start in state 2.
Is the answer to this the mean recurrence time of 2 or simply zero?
I at first thought that the answer was zero, since we start in state 2 and therefore take zero time to get there.  But then I thought of the mean recurrence time, and was confused as to what to do.
Answer: The answer depends exactly on how the first visit to a given state has been defined. In my experience convention says that the answer is zero. If the question asked for the expected return time to state 2 then you would indeed be after the mean recurrence time. Hope this clears things up.
To further clarify the matter:
We define the hitting time on i to be
$$ H_i = \inf(n \ge 0:X_n=i), $$
and the first passage time to i to be
$$ T_i = \inf(n \ge 1 : X_n =i). $$
Note that we can have a hitting time of zero, as in your case, but the first passage time cannot be zero. The mean recurrence time is then defined as
$$ m_i = E_i (T_i) = E(T_i |X_0=i), $$
which is strictly greater than zero.
Comment: @John don't forget to vote up and accept if this is the answer you needed ^_^


How to select what bundles a certain html page will use?

Question: the project
I have a project where the website view is written using Pug, then compiled to HTML.

index.pug ---> index.html

Because I want the project to have more than one page, I have got this, as well:

about.pug ---> about.html

Now, each of these pages have entries, in webpack.config.js:
{
  entry: {
    index: ['scripts/index.ts'],
    about: ['scripts/about.ts'],
  }
}

I don't think it's good to have a single bundle for both, because there's imported modules which are unique to each page entry (that is: if I know what I'm talking about).
the problem
When the HTML is generated, html-webpack-plugin automatically injects both bundles in both pages, which is worse then what I just explained.
How can I tell it to inject just the bundle I want?
ps: I tried including each of them, manually in the *.pug files. Although, generated bundle filenames have hashes, in order to achieve cache busting.
Answer: Before writing this question, I spent three hours searching for a solution.
Although, right after submitting it, I refreshed search page and found the solution.
It's about chunks.
You pass a hash of options to html-webpack-plugin, in which there's a property named "chunks". This property can take an array of string, with the names of desired entries.

{
  plugins: [

    new HtmlWebpackPlugin({
      template: 'views/index.pug',
      scriptLoading: 'defer',
      filename: 'index.html',
      chunks: [
        "index"
      ]
    }),

    new HtmlWebpackPlugin({
      template: 'views/about.pug',
      scriptLoading: 'defer',
      filename: 'about.html',
      chunks: [
        "about"
      ]
    }),

    new HtmlWebpackPugPlugin()
  ]
}

Sorry for the trouble.


Accessing type arguments from generic types used in typebounds

Question: I have discovered the following problems with generics. Consider the generic interface
public interface A<X> {
    X get();
    void doStuff(X x);
}

Now, let's assume the following method definition:
public <T extends A<?>> void foo(T t) {
    bar(t);
}

Due to the wildcard, the type information for the return type of get() is insufficient. Therefore, I have to delegate to another method that "binds" this wildcard to a fresh type variable:
private <X> void bar(A<X> t) {
    X x = t.get();
    t.doStuff(x);
}

The invocation of bar() in foo is not allowed, the compiler outputs the following error message:

The method bar(A) in the type test is not applicable for the arguments (T)

However, if I change the method foo() to
public <T extends A<?>> void foo(T t) {
    A<?> u = t; // No explicit cast required, no "unchecked" warning!
    bar(u);
}

it works. Why? Is this a compiler error? Any comments on this would be very much appreciated.
Notes:

The reason why I don't simply declare method foo as void foo(A) is that I'm actually using the upper type bound interesection (&).
The reason why I don't declare X as a type variable in foo() is that I actually have the problem at class level and don't want to unnecessarily increase the number of type parameters of this class.
Comment: are you using the -Xlint compiler flag? If not, try compiling with that
Comment: I agree with @kan. The java compiler does, in fact compile this. I used 1.6.0_27. Eclipse does give the exact error you gave, so maybe you're using it?
Comment: Created the eclipse bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=364200 using @kan's example.
Comment: Does not compile using javac 1.7.0.
Answer: I've checked the code:
public class Test
{
    public interface A<X> {
        X get();
        void doStuff(X x);
    }
    public <T extends A<?>> void foo(T t) {
        bar(t);
    }

    private <X> void bar(A<X> t) {
        X x = t.get();
        t.doStuff(x);
    }
}

it works. javac 1.6.0_22. Where do you have the error? Or I using another code?
Comment: Thanks, I'll check it on monday on my machine at work, I don't know exactly which javac version I'm using there. I just tested it at home and it doesn't compile using javac 1.6.0_11...
Comment: Btw, the error simply is "bar(Test.A) in Test cannot be applied to (T)", with the marker pointing at the bar() call.
Answer: The code:
public class Test
{
    public interface A<X> {
        X get();
        void doStuff(X x);
    }
    public <T extends A<?>> void foo(T t) {
        bar(t);
    }

    private <X> void bar(A<X> t) {
        X x = t.get();
        t.doStuff(x);
    }
}

compiles for only select versions of Java 1.6. This bug was logged for eclipse, but was rejected with the following explanation by Srikanth Sankaran:

I believe the eclipse compiler behavior is correct  and is matched by
  JDK5 and JDK7 (latest). It would appear the JDK6 behavior is a
  regression which has since been fixed.
Basically, here is a brief explanation of what is going on here:
Given the generic method bar and the call site as given by bar(t), the
  inference algorithm has no constrains to work with to infer the type
  of the type variable X. So after considering the arguments and
  expected return type etc, X is still unresolved and per specification
  is resolved to be the published lower bound - i.e X is inferred to be
  Object and the method becomes void bar(A) t); Since the actual
  parameters cannot be converted to formal parameters, the inferred
  method has to be rejected leaving us with no applicable candidates.
  Hence the call must be rejected.
The reason the call bar( ( A )t ); succeeds is that in this case X
  is inferred to be "capture#1-of ?" and the generic method
  parameterized with this substitution becomes void bar(A) for which the parameter compatibility holds.


Solution to Linear System Containing a Minimum Operator

Question: I have a nearly linear system which I believe should have a solution (from model intuition and from numerical experimentation using a non-linear solver), but which I am having trouble studying analytically. The system has a form similar to
$$
x_j = min \left \{0; a_j + \sum_{i=1}^N b_i x_i \right \}
$$
for all $j$ in $\{1,\dots, N \}$. My first attempt was to try to study the equation ignoring the minimum operator, find a solution, then enforce the minimum after the fact. However, this looks like
$$
\vec{x} = \vec{a} + \vec{1} \otimes \vec{b}^T\vec{x} \hspace{10pt} \Leftrightarrow \hspace{10pt} \left(\mathbb{I} - \vec{1}\otimes \vec{b}^T \right)\vec{x} = \vec{a}
$$
where $\vec{1}$ is a column vector of length $N$. However, if $\sum_i b_i=1$ (which is my particular case) this matrix is exactly singular. That being said, even in this case several non-linear solvers seem to converge to the same solution.
Is this logic correct? Do I need to worry about any other cases? Am I missing something about the minimum operator that makes these equations independent? Or is there a more appropriate way to solve this problem?
Comment: What are the assumptions on the $a_i$ and $b_i$? Are the $b_i$ positive as weights? Are the $a_i$ non-negative?
Answer: You can reformulate the problem as
$$c = \sum_{i=1}^N b_i\min(0,a_i+c).$$
Now for $c\ge -\min_{i=1,..,N} a_i$ all terms on the right are zero, while for $c\le -\max_{i=1,..,N} a_i$ the right side has the value
$$
c+\sum_{i=1}^N b_i a_i
$$
Assuming the $b_i$ are non-negative weights, you get that for $c>\max(0, -\min a_i)$ the left side of the equation is larger than the right, while for $c< -\max_{i=1,..,N} a_i$ the opposite is the case, so that in-between both curves intersect exactly once. This can be solved with the bisection or a variant of the regula-falsi method.
If $\min a_i>0$, the intercept is at $c=0$ so that the only solution has all $x_i=0$.
If the $a_i$ are sorted in increasing order, one can also use the partial sums $p_n=\sum_{i=1}^nb_i$, $q_n=\sum_{i=1}^nb_ia_i$ and compute $c_n=\frac{q_n}{1-p_n}$ and test if $-a_{n+1}\le c_n\le -a_n$. Whenever that is true, then one has found the solution $c=c_n$.


Unbiased estimator in no-intercept regression model

Question: On an assignment I've been tasked with finding whether
$$\hat{\beta}_1=\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2},$$
the estimator for the slope of a no-intercept regression model $Y_i=\beta_1 X_i+\epsilon_i$, is unbiased. I'm not sure how this is done. I would like to use the traditional expected value definition $\mathbb{E}(\hat{A})=A$, but I encounter trouble at the beginning with
$$\mathbb{E}\left(\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2}\right),$$
given that (at least from what I've seen),
$$\mathbb{E}\left(\frac{\sum_{i=1}^n x_iy_i}{\sum_{i=1}^n x_i^2}\right) \neq \frac{\mathbb{E}\left(\sum_{i=1}^n x_iy_i\right)}{\mathbb{E}(\sum_{i=1}^n x_i^2)}.$$
As such, I'm stuck. We haven't learned about the Gauss-Markov theorem (so this answer couldn't be used here) or Jensen's inequality, and we are not given the distribution of $X_i$ nor told to treat $\sum_{i=1}^n x_i^2$ as constant.
Any help would be greatly appreciated.
Answer: It does not matter whether the $X_i$ are random or fixed (so long as the probability they are all zero is $0$).
$$\mathbb E \left[ \frac{\sum X_iY_i}{\sum X_i^2}\right] =  \mathbb E \left[ \frac{\sum bX_i^2 +X_i \epsilon_i}{\sum X_i^2}\right]=  \mathbb E \left[b\frac {\sum X_i^2}{\sum X_i^2}\right]  + \sum\mathbb E\left[\frac{ X_i }{\sum X_i^2}\epsilon_i\right] \\=b+0=b$$ using the independence of the $\epsilon_i$ from the $X_i$ and $\mathbb E \left[ \epsilon_i\right]=0$.
Comment: Thanks very much for the help. Clearly I faced the classic beginner mixup between $X_i$ and $x_i$...
Answer: You don't need these theorems. $x_i$ is deterministic, and we have $y_i=\beta_1x_i+\epsilon_i$.
So just substitute $y_i$ in the numerator with the above formula.
Now just take expectation. The key is to just note that $x_i$ and $y_i$ are deterministic values and only $\epsilon_i$'s are random variables.
Answer: A major point is that the $x_i$ are NOT RANDOM and the $Y_i$ are random. (Note that below, I do not confuse $Y_i$ with $y_i$. The latter is an observed value; the former is a random variable.) So
\begin{align}
\operatorname E\left( \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2} \right) & = \frac{\operatorname E\left( \sum_{i=1}^n x_i Y_i \right)}{\sum_{i=1}^n x_i^2} \text{ because the $x$s are not random} \\[10pt]
& = \frac{\sum_{i=1}^n \operatorname E(x_iY_i) }{\sum_{i=1}^n x_i^2} \text{ by linearity of expectation} \\[10pt]
& = \frac{\sum_{i=1}^n x_i\operatorname E(Y_i)}{\sum_{i=1}^n x_i^2} \text{ because the $x$s are not random} \\[10pt]
& = \frac{\sum_{i=1}^n x_i \big( \beta_1 x_i\big)}{\sum_{i=1}^n x_i^2} \\[10pt]
& = \frac{\beta_1 \sum_{i=1}^n x_i \big( x_i\big)}{\sum_{i=1}^n x_i^2} \Large{ \begin{smallmatrix} \text{because $\beta_1$ does not} \\ \text{change as $i$ goes from $1$ to $n$} \end{smallmatrix}} \\[10pt]
& = \beta_1.
\end{align}
Answer: Explanatory variables $x_i$s are assumed to be known, fixed values. The randomness emanates from $\boldsymbol\varepsilon.$ Any linear estimator would be of the form $\mathbf c^\top\mathbf Y= \sum_i c_iY_i.$
If $\mathbb E[\sum_i c_iY_i]=\beta_1,$
this means $\sum c_ix_i=1.$ Imposing additional constraints and requirements for the estimator (viz. BLUE) would lead to particular value of $\mathbf c. $


OpenLDAP proxy : caching credentials in case Active Directory master unreachable

Question: I'm setting up a Debian server in my company where users are managed in an Active Directory.
I'd like to authenticate users with AD but I'm thinking it would be better, if feasible, to have a local OpenLDAP to authenticate against in case AD server or network falls.
I've seen tutorials about setting up pass-through authentication

https://wiki.debian.org/LDAP/PAM
http://ltb-project.org/wiki/documentation/general/sasl_delegation

but it doesn't say what happens if the AD server is not reachable. AFAIU, the request fails.
Someone here suggest using OpenLDAP Proxy Cache Engine setting a high TTL.
Should I be replicating the whole directory instead ? I don't mind if new users can"t be authenticated. I'd be happy if already locally known users can be authenticated using the last accepted password. So the easiest solution is my favorite.
I searched with a lot of terms including cache/caching, replica, etc. I didn't find any "grab-my-hand-and-show-me-how-to-do-that-on-debian-jessie" step-by-step solution, so it could be that what I thought would be relatively standard is in fact a bit tricky.
Answer: I'm not sure whether you're asking about generally how to have a Debian server do authentication/authorization against Active Directory...or how to make sure the existing authentication/authorization is highly available.  I'm going to assume the latter in this answer.
The short answer is that setting up OpenLDAP as a cacheing layer for an Active Directory is silly.  AD is a multi-master replicated database.  If you need high availability, just bring up another.  If your DCs are in a different network segment that you're worried about losing connectivity to, bring up a DC (or two) in your local network segment and setup the necessary site topology in AD.
Comment: I want the credentials synced from AD but I'd be happy if the setup could run even if AD falls or is unreachable. The AD, like most MS stuff, is managed by our IT contractors and we won't have it redunded, so I guess I'll just rely on it and the AD authenticated services on my server will be down when the AD is down, which should not be often anyway. I just thought/hoped there might be an easy way to be more "independent".
Comment: I should add that "setting up OpenLDAP as a cacheing layer for an Active Directory is silly" is actually the kind of answer I'm looking for. When I think my setup and my need/wish are pretty common and I can't find any detailed explanation/tutorial, I tend to conclude that I'm not looking in the right direction at all. I'm not asking for a twisted way to accomplish what I was trying to but for a pointer to what I should be doing.
Answer: A local OpenLDAP server is probably not the best way to handle the situation described. I think you should take a good long look at sssd. It should have everything you need.
Comment: Does this imply having all the users managed as system users, as in AD domain integration? One of the services I'm thinking of is Redmine. I can authenticate to it only through LDAP (or by creating users manually...). Are you suggesting I could do something like this : AD -> sssd -> LDAP -> Redmine? Integrating the machine to the domain (with all users as system users, AFAIU) seemed overkill but there are so many config possibilities and I'm probably missing basic knowledge. If it takes too much expertise, I should probably stick to AD only authentication and cope with potential downtimes.
Comment: @Jérôme Perhaps what you're missing is that AD provides LDAP all by itself?
Comment: Yes, I know that. I managed to have Redmine use AD as LDAP. Except [it does no caching](http://www.redmine.org/issues/19773). And even if it did, other services might not. Starting from there, I thought I might as well authenticate to a local LDAP that would do the caching, but from what I understand in the answers here, this is not the best idea.


Interpreting interaction terms and main effects in logit regression with multiple dummy variables

Question: I have a logistic regression model: logit(Y)= b0+b1+b2+b3. All my variables are dummy variables.

I want to add the interaction term to the model: logit(Y)= b0+b1+b2+b3+b1*b3. Please, find the model with interaction term below. I would like to know how the interpretation of the variables var1, var2, and var3 changes when the interaction term is added to the model. For instance, is the coefficient of var2 1.24, when var1=0 and var3=0. What is the reference category for the interaction term? Is it var1=0, var2=0, var3=0? Is the constant when var1=0, var2=0, var3=0?
Answer: The interpretation of the coefficient for var2 stays the same: the difference in log odds of the outcome between being in level 0 and level 1 of var2, holding constant var1 and var3 (and, therefore, var1_var3). The interpretation of the coefficient for var1 is now the difference in log odds of the outcome between levels 0 and 1 of var1 when var3 is equal to 0 and var2 is held constant. Replace var1 and var3 in the above explanation for the interpretation of the coefficient of var3.
The coefficient for the interaction is the difference in the effect of var1 between levels 0 and 1 of var3. So, the effect of moving from level 0 to level 1 of var1 when var3 is equal to 1 is 0.357888 - 2.152846 (whereas, the effect of moving from level 0 to level 1 of var1 when var3 is equal to 0 is just 0.357888).
Comment: Note that the interpretation of the interaction term might depend on how the model is formulated. What you described is the case for treatment contrasts with the lowest level of each categorical/dummy variable taken as the reference, the default in R. Some statistical software, however, might take the highest level of a dummy variable as the reference, or use different contrasts, which might alter the details of interpretation of the interaction term.
Comment: @EdM, true, although based on OP's Stata output I think what I've provided is accurate.


Why does AWS Glue generate multiple json files?

Question: I am busy with a POC (using AWS Glue) to pull data from a RDS AWS Postgresql table and I want to generate a JSON file.
I am using the following script, but it keeps on generating multiple files with 5 rows in each file. How do I get it to generate 1 file only?
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

## @params: [JOB_NAME]
args = getResolvedOptions(sys.argv, ['JOB_NAME'])

sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)

## @type: DataSource
## @args: [database = "temp-crawlerdb-xxxxx", table_name = "taxservice__3fa3bf8633994e1a827498190adbe56a_contingencyrunningtotal", transformation_ctx = "datasource0"]
## @return: datasource0
## @inputs: []
datasource0 = glueContext.create_dynamic_frame.from_catalog(database = "temp-crawlerdb-xxxxx", table_name = "taxservice__3fa3bf8633994e1a827498190adbe56a_contingencyrunningtotal", transformation_ctx = "datasource0")

## @type: ApplyMapping
## @args: [mapping = [("stake", "decimal(18,6)", "stake", "decimal(18,6)"), ("branchid", "long", "branchid", "long"), ("winningstake", "decimal(18,6)", "winningstake", "decimal(18,6)"), ("grossrevenue", "decimal(18,6)", "grossrevenue", "decimal(18,6)"), ("vatrate", "decimal(18,6)", "vatrate", "decimal(18,6)"), ("tmstamp", "timestamp", "tmstamp", "timestamp"), ("usrid", "string", "usrid", "string"), ("contingencyexternalreference", "string", "contingencyexternalreference", "string"), ("winnings", "decimal(18,6)", "winnings", "decimal(18,6)"), ("ggrtaxrate", "decimal(18,6)", "ggrtaxrate", "decimal(18,6)"), ("taxpayable", "decimal(18,6)", "taxpayable", "decimal(18,6)"), ("vatpayable", "decimal(18,6)", "vatpayable", "decimal(18,6)")], transformation_ctx = "applymapping1"]
## @return: applymapping1
## @inputs: [frame = datasource0]

applymapping1 = ApplyMapping.apply(frame = datasource0, mappings = [("stake", "decimal(18,6)", "stake", "decimal(18,6)"), ("branchid", "long", "branchid", "long"), ("winningstake", "decimal(18,6)", "winningstake", "decimal(18,6)"), ("grossrevenue", "decimal(18,6)", "grossrevenue", "decimal(18,6)"), ("vatrate", "decimal(18,6)", "vatrate", "decimal(18,6)"), ("tmstamp", "timestamp", "tmstamp", "timestamp"), ("usrid", "string", "usrid", "string"), ("contingencyexternalreference", "string", "contingencyexternalreference", "string"), ("winnings", "decimal(18,6)", "winnings", "decimal(18,6)"), ("ggrtaxrate", "decimal(18,6)", "ggrtaxrate", "decimal(18,6)"), ("taxpayable", "decimal(18,6)", "taxpayable", "decimal(18,6)"), ("vatpayable", "decimal(18,6)", "vatpayable", "decimal(18,6)")], transformation_ctx = "applymapping1")

## @type: DataSink
## @args: [connection_type = "s3", connection_options = {"path": "s3://tax-service-xxxxx"}, format = "json", transformation_ctx = "datasink2"]
## @return: datasink2
## @inputs: [frame = applymapping1]
datasink2 = glueContext.write_dynamic_frame.from_options(frame = applymapping1, connection_type = "s3", connection_options = {"path": "s3://tax-service-xxxxx"}, format = "csv", transformation_ctx = "datasink2")

job.commit()

Answer: Before the Apply Mapping, do this:
from awsglue.dynamicframe import DynamicFrame

# Convert to a dataframe and partition based on "partition_col"
partitioned_dataframe = datasource0.toDF().repartition(1)

# Convert back to a DynamicFrame for further processing.
partitioned_dynamicframe = DynamicFrame.fromDF(partitioned_dataframe, glueContext, "partitioned_df")

Comment: and don't forget the import statement - from awsglue.dynamicframe import DynamicFrame
Comment: the author should make the import part of the answer
Answer: Just in case anyone else come across this the above does work but doesn't really explain why. This post covers it in alot more detail.
https://thedataguy.in/aws-glue-custom-output-file-size-and-fixed-number-of-files/
"Why Glue is producing more small files?
If you are processing small chunks of files in Glue, it will read then and convert them into DynamicFrames. Glue is running on top of the Spark. So the dynamic frames will be moved to Partitions in the EMT cluster. And the Glue partition the data evenly among all of the nodes for better performance. Once its processed, all the partitions will be pushing to your target. Each partition will and one file. That's why we are getting more files."
Comment: The questioner is asking for a way to generate 1 file... you answered the 'why is it happening' for a good answer can you add 'how to resolve' also


Multiple CSV Web Sources into one by using Python and Pandas

Question: I have to fetch csv-content from multiple web-sources and append it to one file. The number of web-sources is variant, it depends on an excel-file which contains all urls.
For this reason, I generated a list (named dataList) in python, each element contains one url from the excel file.
After that, to get the data from the urls, I used a for-loop:
for k in range(count_dataList):
    dataContent.insert(k, pd.read_csv(dataList[k]))

Now, a list exists where each element contains the data from the source. I guess as a DataFrame.
At last, I have to append all elements to one DataFrame. But I can't find any way to append all datas.
Comment: Please see [ask], [help/on-topic]. As an aside, why are you seemingly mixing multiple style conventions? It's probably best to just stick to `lower_case_with_underscores` for functions and variables.
Comment: Does this answer your question? [Append multiple pandas data frames at once](https://stackoverflow.com/questions/36526282/append-multiple-pandas-data-frames-at-once)
Answer: You can use pd.concat:
df = pd.concat(pd.read_csv(source) for source in dataList)

Comment: https://stackoverflow.com/questions/36526282/append-multiple-pandas-data-frames-at-once


Redirecting Blogger blog.blogspot.com to domain blog.example.com?

Question: I am trying to redirect a Blogger blog at example.blogspot.com to a domain blog.example.com
I use the following dns entry at example.com:
blog CNAME ghs.google.com.

When accessing blog.example.com the only message I get is:

Forbidden
You don't have permission to access / on this server.
Additionally, a 404 Not Found error was encountered while trying to
  use an ErrorDocument to handle the request.
Apache/2 Server at anothersubdomain.example.com Port 80

However when I look up the blog.example.com using an online dns lookup I get
N   CNAME   blog.example.com    ghs.google.com

What is wrong here?
How long do I need to wait until the new dns record is synchronized in all other dns servers?
Answer: DNS can take up to 24 hours to fully propagate everywhere...Is the DNS entry registered someplace major (Network Solutions, etc) or say through your ISP?
Comment: That is what I thought. It is registered using a local hosting provider.


Why BROADCAST receiver registration causing error. My app just crashing because of this "android:exported="true"

Question: If i remove this "android:exported="true"" or make it FALSE, then BroadCast receiver does not work. :-/
How to solve this error, while me trying to perform some action on receiving SMS !!
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.pucit.arsh_hp.multisms">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">

        <activity
            android:name=".MainActivity"
            android:label="@string/app_name"
            android:theme="@style/AppTheme.NoActionBar">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>

        </activity>

        <receiver android:name=".SmsReceiver" android:exported="true">
            <intent-filter android:priority="999">
                <action     android:name="android.provider.Telephony.SMS_RECEIVED" android:enabled="true" />
            </intent-filter>
        </receiver>

    </application>

    <uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED"/>
    <uses-permission android:name="android.permission.SEND_SMS" />
    <uses-permission android:name="android.permission.RECEIVE_SMS" />
    <uses-permission android:name="android.permission.READ_SMS" />
    <uses-permission android:name="android.permission.READ_CALL_LOG"/>

</manifest>

Answer: You absolutely need to keep android:exported="true", because this enables your broadcast receiver to listen to events triggered outside your application.

android:exported
  Whether or not the broadcast receiver can receive messages from sources outside its application — "true" if it can, and "false" if not. If "false", the only messages the broadcast receiver can receive are those sent by components of the same application or applications with the same user ID.
  The default value depends on whether the broadcast receiver contains intent filters. The absence of any filters means that it can be invoked only by Intent objects that specify its exact class name. This implies that the receiver is intended only for application-internal use (since others would not normally know the class name). So in this case, the default value is "false". On the other hand, the presence of at least one filter implies that the broadcast receiver is intended to receive intents broadcast by the system or other applications, so the default value is "true".
This attribute is not the only way to limit a broadcast receiver's external exposure. You can also use a permission to limit the external entities that can send it messages (see the permission attribute).

http://developer.android.com/guide/topics/manifest/receiver-element.html#exported
You crash must be related to your code processing the data received by your BroadcastReceiver, and not caused by the BroadcastReceiver itself.


How do I have my HTTPS server install the issuer cert in the browser's trusted authorities keystore?

Question: To illustrate what I'm trying to do:
https://www.google.com uses a cert issued by Thawte SGC CA (which in turn is issued by one of the Verisign CAs). It looks like Internet Explorer comes with that issuer cert preinstalled, but Firefox (I'm using 3.5.1) does not. Nevertheless, the browser doesn't complain that the site cert is untrusted, because the Thawte cert gets installed right then and there. So far so good.
I have a server (using Apache) where I'd like to accomplish the same thing. It also uses a cert from the same CA, and I've got it sending the complete chain of trust during the SSL handshake (using the httpd.conf SSLCertificateChainFile directive). But if the intermediate cert isn't already previously imported into Firefox' Authorities keystore, it'll complain that the site cert isn't trusted.
In short, then, my question is: what do I need to do to get the Thawte cert to auto-install, so that the browser will accept the site cert without prompting?
Answer: Something with the chain file is screwed up. Apache will send the chain file, which must be signed by a CA that is already trusted by the browser, there's no way to add it. The browser will verify the chain, and allow a cert signed by the chain if the chain is successfully verified.
Thawte should have instructions and ready made chain files for you to install on their support pages.
Comment: Finally had a chance to test this again, and it looks like you're right - my chain file was bad. Looks like it's working now (and installing the intermediate cert into the browser cert store) as it should.
Answer: I believe you need to put the CA root certificate on the server, same place as the server cert.


rsync not using the ssh-agent?

Question: I have this running on gitlab ci, and it works for regular ssh (copied from the gitlab ci docs):
eval $(ssh-agent -s)
ssh-add <(echo "$SSH_PRIVATE_KEY")
mkdir -p ~/.ssh
echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

This gives me
Identity added: /dev/fd/63 (/dev/fd/63)

However, when I try to rsync I get an error:
Warning: Permanently added '*****' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
user@server: Permission denied (publickey,password).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(235) [sender=3.1.3]

Why is rsync not able to use the identity added, when the ssh command is?
I can also do this instead, to work around the rsync issue.
eval $(ssh-agent -s)
mkdir -p ~/.ssh
echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-add

Which gives me
Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)

Which rsync is able to use, most likely because it's in the assumed place.
Does this mean rsync uses it's own ssh implementation and not the ssh-agent on the local system?
Answer: You can specify the ssh protocol with rsync using the "-e" option.
rsync -e ssh file.tmp example.com:~/

Identity could also be provided in that option.
rsync -e "ssh -i ~/.ssh/id_rsa" file.tmp exmple.com:~/

rsync not using the ssh-agent?


How to pass arguments to aapt when building android apk?

Question: I'm building an apk using ant on Linux (not using Eclipse), and I'm trying to find an easy way to switch between compressing assets and not, in order to keep my huge database uncompressed for < 2.3.  As per this article:
http://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps/
I see that it's possible to do this by specifying something like aapt -0 db, but I can't find information on how to edit the way ant interacts with aapt.  Or do I have to run aapt by itself?  What do I do?
Answer: It's Android's build.xml (you'll find this under your SDK, tools and ant) that executes AAPT when building.
Find your build.xml and replace the "-package-resources" target with the target below. I'm pretty sure (not able to test it at the moment) you'd be able to swith compression on and off with either "-Dcompression.disabled=set" when build with ANT or "compression.disabled=set" in your .properties-file.
<target name="-package-resources" depends="-crunch">

    <do-only-if-not-library elseText="Library project: do not package resources..." >
        <aapt executable="${aapt}"
                command="package"
                versioncode="${version.code}"
                versionname="${version.name}"
                debug="${build.is.packaging.debug}"
                manifest="AndroidManifest.xml"
                assets="${asset.absolute.dir}"
                androidjar="${android.jar}"
                apkfolder="${out.absolute.dir}"
                nocrunch="${build.packaging.nocrunch}"
                resourcefilename="${resource.package.file.name}"
                resourcefilter="${aapt.resource.filter}"
                projectLibrariesResName="project.libraries.res"
                projectLibrariesPackageName="project.libraries.package"
                previousBuildType="${build.last.target}"
                buildType="${build.target}">
            <res path="${out.res.absolute.dir}" />
            <res path="${resource.absolute.dir}" />
            <if>
                <condition>
                    <isset property="compression.disabled" />
                </condition>
                <then>
                    <nocompress/>
                </then>
            </if>
        </aapt>
    </do-only-if-not-library>
</target>

Comment: Okay!
You could make it per filetype too by adding an extension-attribute to the nocompress-element.

`

`
Comment: Anyone found the way without modyfing android sdk internal files?
Comment: Thank; I didn't end up going this way, since it would have made the apk huge, but glad to know the answer anyway.
Comment: You don't modify Android sdk internal files - instead you add the rules to the build.xml file within your own project
Comment: Thanks Sveinung Kval Bakken for the answer, but this was not quite enough information for those like me who are not familiar with ant.  For others: you want to copy the `...` from android_sdk_dir/tools/ant/build.xml and paste it into the build.xml in your project directory BEFORE ``, then modify the line that says ``  See the comment at the end of your project build.xml for guidance.


"approved ultra-nationalist style" of soviet symphonies during Stalin time

Question: In a "keeping score" video about 5th Symphony by Shostakovich a director Michael Tilson Thomas says the following:

Instead of writing in the approved ultra-nationalist style, Shostakovich wrote his Fifth Symphony on the model pioneered by Beethoven; he begins his symphony with a sonata, albeit with a hesitant feel.

If I understand this correctly, sonata here refers to the sonata allegro form, first part of a classic sonata where we'd get intro, 1st/2nd theme exposition, progression, repetition of 1st/2nd theme and then a coda. Indeed, this structure kinda fits the 1st movement of the 5th by Shostakovich, with perhaps a superlong coda.
What I'm interested in though: what was this approved ultra-nationalist style? Perhaps there are some other symphonies that provide an example of such style which apparently should differ from the classical sonata one.
Comment: For examples of much more nationalistic works that prioritize programmatic considerations over musical form, listen to (and read about) Shostakovich's [11th Symphony](https://en.wikipedia.org/wiki/Symphony_No._11_(Shostakovich)) and [12th Symphony.](https://en.wikipedia.org/wiki/Symphony_No._12_(Shostakovich))
Comment: @MichaelSeifert thanks! those were composed after Stalin's death, whereas Thomas was specifically talking about early Stalin times. This makes sense, given that 20s in USSR were avangardist times and 30s was the time of a (very harsh) reaction to it.
Answer: There was not a clear set of rules to define "Soviet" music. It was supposed to "glorify" the Soviet ethos, but ultimately it was up to the bureaucrat who was tasked with giving the stamp of approval.
This system (or lack of system) plagued Prokofiev, for example, who tried mightily to write "Soviet" music, but was frequently rejected for various "reasons" that ultimately amounted to a bureaucrat not liking him, his music, or both.
One consistent theme, however, was a rejection of "bourgeois" and "Western" elements: so, Bach, Mozart, Chopin, etc. The music should represent "class struggle" and glorify the proletariat. But how that was done was ultimately just the opinion of the official in charge. Beethoven was an exception, however, due to his "revolutionary" character.
Comment: Based on listening to other Communist music and how outwardly confident those pieces are, I actually think Shostakovich got burned fairly hard on the "hesitant" part.
Comment: so no particular system, you say. Btw, I thought Beethoven was the one still allowed, allegedly due to his "revolutionary" character
Comment: @SBF Yes, my error. The post is updated.
Comment: @Dekkadeci if he was, the press would not hesitate to write about it. From what I know, the press was praising the 5th, and also Stalin liked it.
Answer: In Soviet era (after the first years) all art form were expected to convey a political message: The suffering of the proletariat and the final victory of communism. One principle common to music especially since Beethoven is the per aspera ad astra or through darkness to light principle, which implies a resolution of dark movements in a heroic final movement. In Soviet music this obtained the meaning of 'the music first shows the struggle of the working class, until the final movement shows heroic triumph of communism'. Soviet reception of art was of course particularly displeased by any form of critique, satire or mockery, and especially in music coined by preference to highly blunt and pathetic music (somewhat similar to Nazi Germany). Soviet music especially rejected music that was considered 'formalistic', to the point that a bunch of composers (including Shostakovich for his 9th) were charged with formalism and banned from composing (cf. Zhdanov Doctrine).
This means that intellectual artists in Soviet era needed to be very careful with how they shaped their art in such a way that could be justified to the authorities and still allow them some amount of submerged artistic and political expression. And this means that it is very hard to interpret such works in the historic context. The symphony in question can be seen to follow the soviet demands quite well, and it was in the time perceived as a return of the young composer to patriotic music. Much later Shostakovich himself demented this in response to some criticism of his positioning during Stalin era. But this could very well also be a sort of made up justification. Who knows exactly?
But this gives a lot of space for people to inject their own readings — which is most likely what is happening here.
Comment: @SBF Note that for one thing Tchaikovsky is not a soviet composer. And yes, Tchaikovsky's 6th is rather uncommon in it's design.
Comment: Your nickname is misleading, thanks a lot for the answer! Tchaikovskys 6th has a rather unheroic last movement. Would you say that's an exception?
Answer: It might be hard to quote symphonies in the 'approved' style, because the requirement for programmatic work, replete with military and folkloric tunes, wasn't a good fit with large scale symphonic forms.  "Once you've played a folk tune, there's not much left except to play it again, louder."
But Shostakovitch's "Soviet Artist's Response to Just Criticism" while not quite as blatantly nationalistic as a 'Má vlast' or '1812 overture' does manage to depict anguish, bucolic merriment and a triumphant resolution with quite apparent assistance from the military!  I'd suggest that it IS a good example, perhaps the pre-eminent example, of a 'Soviet Symphony'.
It's always difficult to find something new to write in a programme note. Perhaps Tilson Thomas is trying a little too hard to avoid the usual opinion?
Comment: Makes total sense. Didn't get the last part. Also, Tilson Thomas quote suggests he had in mind an alternative form of the first movement, in this "ultra nationalist" style. But overall from the answers I get the opinion that at least it's not obvious what he meant, or were there indeed examples of other symphonies with such first movement.
Comment: About "Once you've played a folk tune, there's not much left except to play it again, louder." - the Russian/Slavic habit/stereotype I've heard is that they play it again, but with accelerando this time.


How to combine 2 vectors?

Question: Let's say I have 2 vectors [2, 4, 6, 8] and [1, 3, 5].
If I combine the numbers according to their indexes, I expect [3, 7, 11] as my result. If one vector has more indexes than the other, then it will be excluded in the calculation. (That's why 8 is not included here). My question is how do I combine 2 vectors while ignoring the extra 8 as shown above? I need a function.
def v_add(num1, num2):
    total = num1 + num2
    return total

Comment: `[a + b for a, b in zip(vector1, vector2)]`
Comment: What type are these vectors? NumPy ndarray objects? Lists? Tuples?
Answer: You don't have vectors, but lists.
def v_add(num1, num2):
    return [a+b for a,b in zip(num1, num2)]

Comment: @maxywb Most people find `map` unpythonic, and this solution will work as is in both Python 2 and 3.
Comment: I wouldn't go so far as to say `map()` is *unpythonic* - in general, list comprehensions/generator expressions are more readable, but in this case they are both very readable. I would say they are both completely pythonic. I'd probably go with `map()` here due to the simplicity of the case and the fact it makes the solution more generic (obviously, `[sum(numbers) for numbers in zip(num1, num2)]` would be as generic in a list comp, but it's more verbose). That being said, I wouldn't say there is much in it.
Comment: It works but imo isn't as pythonic as alfasin's answer.
Answer: a = [1, 2, 3]
b = [1, 2, 3, 4]
c = map(sum, zip(a, b))
print(c)

OUTPUT
[2, 4, 6]

Comment: @user2581724 If you have another question, ask another question - don't tack it on here - that way you will get better answers and others will be able to find it if they have the same question (in this case, do a search first - I imagine your question has been asked before). I'd also note that these are *lists*, not vectors in Python - using the correct terminology will make your question clearer and finding answers easier.
Comment: Thank You. One more question, how would I Multiply the vectors?
Answer: a = [1,2,3]
b = [4,5,6]
print [a[i] + b[i] for i in range(min(len(a),len(b)))]

Comment: -1, iterating by index in Python is simply a bad idea - it performs badly, reads badly, and there are better ways.
Comment: It's not about complexity, it's about the method of iteration. Python has loops that work over iterators (commonly 'for each' in other languages) - using them to iterate over numbers just to emulate the original behaviour is reinventing the wheel. It just makes code less readable and slower (not in complexity, just performance - unlikely to matter, but still). If nothing else, it makes your solution rely on the inputs being sequences rather than arbitrary iterables.
Comment: you anyway need to iterate through the lists,I don't think there can be algorithm better than O(n).
Comment: thank you Lattyware, I never had been comfortable with iterators
Comment: They are very core to Python - the entire standard library uses them extensively, and language features are built around them (e.g: generators, the for loop). They are actually a very simple (yet powerful) concept - worth reading up on if you are working in Python.

How to combine 2 vectors?


Intensive resources for learning R (e.g., classes, workshops)?

Question: I have read the wonderful thread entitled Free resources for learning R. The question asks for resources to learn R "on the cheap".
I have a similar question: What is the best and quickest way to learn R, regardless of cost? I am thinking of a week-long intensive course, or similar (could even be much longer). The course would involve learning R from the very beginning and be designed for people with a background in statistics.
Answer: Honestly, once you go through the basics, I'd go with a learn-as-you-go approach. When you do a study that requires a certain analysis, then go find a place to learn how to do it. This way, you don't have to worry about forgetting how to code things plus you'll be more motivated to learn about the specific topic. At least that has worked for me so far.
Answer: Online courses like Coursera.org has a good selection for statistics using R. For example, this one from Princeton uses R as their teaching software. There are also intensive courses provided by private companies like this one.
Since you mentioned you have statistical background, I believe you also know at least one other statistical software package? If that's the case, I'd recommend first dig out a few comprehensive analyses you have done before, and try to replicate that with R. Through the process, you would need to consult the materials online and get more familiar with the language, and because you have your old output, you can get to compare your results with the model answers. In the process, remember to create and maintain a "cheat sheet" like this PDF document. It'd save a lot of time.
I have been using this language for about 8 years, started with S+. I use it almost everyday but to be frank, I never really endeavored to "learn" R. R is like a giant national park or theme park that you can just take a small chunk in each visit. There are basics like import/export, assigning object, writing function, etc. but then pretty much beyond linear mixed model it's everyone's own game. I hunt for the package that fits my need, and I study the package's manual. And all those basics are generously provided by users online.


Binomial Regression "logit" vs "cloglog"

Question: I am using a binomial regression with a categorical factor with 9 levels (named 'treat.group') and sample sizes in each group from 1-8. 1 treatment group has all positive cases (i.e., 1's) - and this creates a estimation problem with the "standard" glm() function in R caused by "perfect separation" for that treatment level. So, I am using bayesglm from the arm package.
My question is that the default identity link is "logic" but i have read that "cloglog" or (Complementary Log-Log) is frequently used when the probability of an event is very small or very large. Thus since my model exhibits perfect separation in 1 treatment group the probability of the event is very large and I should use "cloglog".
Using cloglog gives me a significant result for the treatment group with perfect separation while "logit" does not.
Am I justified in using "cloglog" or is there a way to look at my results and be certain what link is best?
f1<- bayesglm(response~ treat.group,family=binomial(link="logit"), data=df)
f2 <- bayesglm(response~ treat.group,family=binomial(link="cloglog"), data=df)
(Data frame below)
{ structure(list(response = c(0L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 0L, 1L, 1L, 0L, 0L, 1L, 0L, 1L, 1L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 1L, 1L), treat.group = c("pctc", "phth", "phth", "phtl", "pltc", "pcth", "pltl", "phtc", "pctl", "phtc", "pcth", "pctl", "pctc", "phtl", "plth", "pltc", "phtc", "pcth", "phtl", "plth", "pctl", "pltc", "phtl", "pctc", "pcth", "pltc", "phtc", "phtl", "phtc", "pctl", "pctc", "pcth", "phth", "pctc", "phtl", "pcth", "phth", "phtc", "pcth", "phth")), .Names = c("response", "treat.group"), row.names = c(NA, -40L), class = c("tbl_df", "tbl", "data.frame"), na.action = structure(c(1L, 4L, 5L, 7L, 15L, 21L, 23L, 24L, 27L, 29L, 33L, 37L, 39L, 48L, 50L, 53L, 54L, 55L, 56L, 57L, 58L, 59L, 60L, 62L, 63L, 65L, 66L, 67L, 68L, 70L, 71L, 72L, 73L), .Names = c("1", "4", "5", "7", "15", "21", "23", "24", "27", "29", "33", "37", "39", "48", "50", "53", "54", "55", "56", "57", "58", "59", "60", "62", "63", "65", "66", "67", "68", "70", "71", "72", "73"), class = "omit")) }
Answer: If you fit two models, one with logit and one with cloglog, you should report the results of both, and also carry out some type of model comparison technique and report the results of that.
As for the models, this is a great situation in which to use Bayesian multilevel models (see this paper [PDF] by Gelman). We can pool information among groups to inform the estimates for groups with smaller sample sizes and for groups with seemingly extreme outcomes (as in phth and pltl).
Fitting multilevel models with the brms package is fairly straightforward. The two models here, one with a logit link and one with a cloglog link, would be fit as follows:
fit1 <- brm(
    response ~ (1 | treat.group),
    family = bernoulli,  # logit link is the default
    data = df,
    iter = 2e4,
    warmup = 2e3,
    control = list(adapt_delta = 0.999)
)

fit2 <- brm(
    response ~ (1 | treat.group),
    family = bernoulli(link = "cloglog"),
    data = df,
    iter = 2e4,
    warmup = 2e3,
    control = list(adapt_delta = 0.999)
)

These will each generate 72000 samples from the models, which can be accessed with as.data.frame(fit). After applying the inverse logit and inverse cloglog to the samples, we get the following estimates for the desired probabilities:
group    prob using logit    sd       89% probability interval
--------------------------------------------------------------
pctc     0.405               0.169    0.12 to 0.66
pcth     0.562               0.135    0.34 to 0.78
pctl     0.438               0.170    0.15 to 0.7
phtc     0.604               0.141    0.39 to 0.84
phth     0.729               0.162    0.52 to 1
phtl     0.529               0.142    0.3 to 0.76
pltc     0.532               0.157    0.28 to 0.79
plth     0.539               0.180    0.24 to 0.84
pltl     0.623               0.193    0.39 to 1

group    prob using cloglog    sd       89% probability interval
----------------------------------------------------------------
pctc     0.395                 0.162    0.12 to 0.64
pcth     0.545                 0.136    0.32 to 0.76
pctl     0.422                 0.164    0.14 to 0.67
phtc     0.589                 0.144    0.36 to 0.83
phth     0.760                 0.182    0.52 to 1
phtl     0.511                 0.142    0.28 to 0.74
pltc     0.512                 0.158    0.25 to 0.77
plth     0.517                 0.181    0.21 to 0.81
pltl     0.627                 0.213    0.37 to 1

Overall, the model using the cloglog link induced less shrinkage in the probabilities. It's worth noting that the 89% probability intervals for phth are the same in the two models.
The brms package supports model comparison using PSIS-LOO. Using the command
loo(fit1, fit2, reloo = TRUE)

takes a while but eventually returns the following result:
            LOOIC   SE
fit1        59.19 2.95
fit2        58.59 2.91
fit1 - fit2  0.60 0.46

Here the model with lower LOOIC is estimated to have better out-of-sample prediction accuracy. The model with the cloglog link has marginally lower LOOIC, and the standard error on the difference, 0.46, is small, so we would conclude that the model using the cloglog link is marginally better than the one using the logit link.
Comment: Is the 72000 the sum of the lengths of all the MCMC chains?


Find the value(s) of $k$ so that the following set of planes has a line of intersection. Question help?

Question: Here are the equations:

$2x + 7y + 2z = 3$,
$6x + y - kz = -1$,
$2x + (2k + 1)y + (k - 1)z = 4$.

Help on this question would be greatly appreciated!
Answer: First of all, we need the determinant $$\left|\begin{matrix}
2 & 7& 2\\ 6&1&-k\\ 2&2k+1&k-1 \end{matrix} \right|=0 $$
or $$2(k-1)-14k+12(2k+1)-2+2k(2k+1)-42(k-1)=0 $$
from this we have
$$4k^2-24k+36=0 $$
which has solution $k=3$.
Now replace $k=3$ to the equation of planes to see if this planes intersect in a line or not.


MySQL -- SELECT Command denied to user

Question: This is a syntax error i am positive ...  However I am banging my head as to 'why'.  I have searched all "like" questions here on SO and they all (mostly) address a user with wrong permissions .. I assure you this isn't the case.
This  is my Query:
use database;    # I am using workbench .. I use this so I don't have to database.table

SELECT firstname
FROM user, user_role
LEFT JOIN user.uid ON user_role.uid

WHERE user.firstname LIKE '%zak%';

The exact error is
Error Code: 1142. SELECT command denied to user '###'@'##.##.##.##' for table 'uid' 0.197 sec

I am not looking for a TABLE called uid ..  I am looking for the FIELD iud.
I have truncated the statement down to this portion that's causing the error .. There are other fields and other tables .. But I simply can't JOIN.
Is is a logical error with the location of the . for field identification?
Comment: The join pattern is `FROM table1 JOIN table2 ON `. You are using something like `FROM table1, table2 JOIN table1.column ON table2.column`. That's an incorrect mix of implicit and explicit joins.
Comment: I dont think so if query is wrong and u get command denied error. Also chk user privileges
Comment: @ devpro .. You couldn't resist ...  Did you read my entire post?  The denied error is because the `table` it's looking for doesn't exist.  @Lloyd Banks is correct ... I was using comma separated tables AND `JOIN`  ..
Answer: Try writing it this way:
SELECT firstname
FROM user
LEFT JOIN user_role ON user.uid = user_role.uid
WHERE firstname LIKE '%zak%';

Answer: You should be using:
SELECT firstname
FROM user, user_role
WHERE user.uid = user_role.uid

If you want all records in the user table regardless of what's in user_role, use
SELECT firstname
FROM user
    LEFT JOIN user_role
ON user.uid = user_role.uid

Using commas to separate tables and using the JOIN keyword is mutually exclusive.  It's either one or the other.  Using commas to separate tables is the same as an inner join.  If you want an outer join (with keywords LEFT or RIGHT), then you'll have to use the format with the JOIN keyword.

MySQL -- SELECT Command denied to user


What's it called when the right hand harmonizes the melody?

Question: Say we're on the piano and we're playing a melody in the right hand. I've noticed sometimes pianists will add another note to that melody that harmonizes with it and makes the melody sound fuller.
So basically two notes are being played at once on every note in the melody. The rightmost finger will play the actual melody and the thumb (on the right hand) will harmonize with that note.
Is there a name for this? I know it's not a chord. Is it just called harmony or something, or is there an official name for it?
Also, is there a certain interval that this is usually done? I assume the note needs to be in the scale so maybe it's not a static interval.
Answer: If it's just one other note, then you can really just say they're playing an interval. Or just calling it a harmony is fine.
You are correct -- a chord consists of at least three notes. Pianists can also hit chords as they're playing a melody (which is helpful especially if they need to comp themselves).
The melody probably sounds fuller when a piano harmonizes with itself because now you hear multiple voices and multiple colors (the colors coming from the fact that there is harmony now).
If you are in-fact talking about improvisation as the tags show, then there is no rule for when a pianist can't play harmonies. It is up to them as the musician and the improviser. During their improvisation, if they feel they want a certain harmonic color to be thrown in, then that's when they would play an interval/harmony.
The note doesn't necessarily need to be in any sort of scale. Sure, if you're given a chord, it's usually nicer on the ear if in the beginning, you focus on sticking to the harmony of the chord. But again, it's completely up the pianist: if they want a harmonic color that's not within the chord or scale, then it's up to them what note to play. In fact, playing harmonies that aren't related to the scale can help define the climax of a improvised solo.
Answer: Accompanying with a single note is rather common in almost all genres of (western) music. The richest sounds come from thirds and sixths; however a long string of either sounds a bit monotonous. Alternating thirds and fifths with a variable number of each allows one to a couple of choices of implied harmony on each note. A few fifths or fourths are possible but these seem a bit empty often and consecutive fifths sounds like one voice drops out. I prefer to use a string of thirds or sixths or rests or a few three voice spots. This allows for a bit of variety with the rests and chords used to emphasize certain notes.
To get started, try accompanying with lower thirds and sixths; you will quickly learn which ones sound best. It's really just two-part counterpoint, essentially first species. Note that one may form an interval of a third or sixth that is quite dissonant with respect to the underlying harmony; this of course adds a lot of emphasis.
Answer: The usual name for this is double stops. It's a common expression among players of string instruments because it's a difficult and yet comparatively common effect. For pianists, doubling a melody is way easier, which is probably why they rarely bother naming the practice.
Answer: The supporting harmony notes can be added in various ways. Sometimes the melody is doubled in thirds, sixths, or octaves where the added part moves in parallel motion. You could refer to this as the right hand playing in thirds or in octaves, etc. These are informal terms (except parallel motion, that's standard music theory lingo) but I think they are commonly used and understood. Of course, the supporting harmony doesn't have to use these intervals - others could be used - and it doesn't need to be parallel motion. But such doubling is common.
Answer: Usually this is called layering. It's used when a hand is playing multiple notes at the same time, and they are different durations. For example, in the key of C Major, a pianist could be playing the melody with their right hand while holding a suspended C with their pinky. This is used to create harmony in higher notes without using the left hand.
Answer: Harmony can be parallel, contrary or oblique (where the melody moves up/down and the harmony stays on one note). As a general practice, it's nice to mix them up a bit. Parallel would be the most common in RH piano harmony because you can run out of finger stretch pretty quickly with contrary and oblique.
Answer: It's called filling in the harmony in the RH.  There are several techniques.  It may be a structured second voice, as if an additional singer had joined the group.  Typically it will run a mixture of 3rds or 6ths below the melody, or be static.  Or it may not follow the shorter notes of the melody, just filling in chords at certain points.
Here's a short example of what a real-life pianist might play to accompany a melody.


How can you musically notate a certain type of vocal performance?

Question: I've been listening to some Roxy Music recently, and was struck by the style of singing where much of it is not actually sung - I mean there is a pitched component, but it doesn't really belong to the accompanying harmony. But it's not spoken either.
For instance, the closing words of the song "What's her name, Virginia Plain".
How would you musically notate this sort of vocal performance?
Answer: I can think of at least a few possibilities, and I'm sure more will keep coming to me:

A shows what are, more or less, the actual pitches being sung. But I agree with you that showing these pitches would give the performer the wrong idea.
B then just adds (half spoken) above; this is pretty self-explanatory.
C is in line with modern notation of the Sprechstimme (speaking voice) tradition made famous by Schoenberg. Although it's not exactly Sprechstimme—your example has more pitch—I've put the noteheads on particular pitches to show the general area of the pitch.
And D is another variant of the Sprechstimme tradition (Schoenberg notated it with an x as the note head and as the note stem), but this time I didn't notate any particular pitches.
I'm sure there are other ways than these, but no matter what you choose, I recommend always remembering the fundamental rule of notation: just make clear to your performer what it is you want. Especially in the case of "modern" music, as long as you make your intentions clear, your notation will succeed.


What are the ornaments in the western music and their corresponding ornaments in Indian music?

Question: I am learning to play the different ornaments in western music (Trill, Tremolo, Turn, Appoggiatura, Acciaccatura, Glissando, Slide). I also want to know which are the corresponding ornaments in Indian Classical music. Are the played the same way but have different names? Am I missing any important ornaments?
Comment: Sorry for my ignorance, but what *are* commonly found ornaments in Indian classical music? Every single time I've read about Indian classical music, ornaments were never mentioned.
Comment: One thing to keep in mind is that the performance of various ornaments in European classical music has changed over time.  I too am ignorant of ornamentation in Indian classical music, so a more basic question about that would be helpful.  If you are qualified to do so, I would suggest that you ask and self-answer a question about that.
Answer: Hindustani (North Indian) Ornaments1
Alankara = Ornaments
A major difference between Hindustani and Western ornaments is that Alankara are tied to raga, in the sense that how the ornament is executed and on which notes it is executed are characteristically defined by the raga. Western ornamentation is by contrast more general, the ornament being influenced by the key/scale but less strongly tied to it.
There are four types of Alankara:

Kan ~ Grace Note/Bend: Kan serves a similar purpose to a grace note, but is performed rather like a bend. A grace note is typically very quick, while kan can be prolonged. Kan also has a bend-like contour as opposed to a clearly defined (set of) pitch(es).

Meend ~ Slide/Glissando: An individual meend can vary in speed and can come to rest briefly on intermediary pitches, which is not characteristic of a slide or glissando.

Ghaseet: glide immediately after plucking a stringed instrument.
Soont: fast vocal meend.

Andolan ~ Slow Trill: Andolan is a slow oscillation between one pitch and the microtones between it and another pitch; whereas a trill oscillates between two distinct pitches.

Gamak ~ Fast Trill: Similar to andolan, but faster and with more pronounced start and arrival points.

Khatka ~ Grace Notes/Turn: "A khatka is an ornament that involves performing a single note as a cluster of notes. The main note is featured most prominently." The closest analogy in Western music could be a series of grace notes, or a short improvisation around a central note.2

There are additional ornaments used in semi-classical Indian music not mentioned here. The ornaments here and the semi-classical ornaments are well explained and demonstrated in a video by Anuja Kamat, "Alankaar (ornamentation) in Indian Music."
A series of videos by Raag Hindustani is more explicit in relating alankar to Western ornaments.
Carnatic (South Indian) Ornaments3
Gamaka = Ornaments
As with the Hindustani ornaments, the correspondences to Western ornamentation are comparative, but inexact. Also as above, the Indian ornaments are tied to raga much more tightly than Western ornaments are tied to scales.
There are three primary types of Gamaka

Jaru/Ullasita ~ slides

Irakka-jaru: descending
Etra-jaru: ascending

Gamaka ~ "deflections" (grace notes/trills/bends)

Nokku: successive upper-neighbor grace notes on non-repeated pitches
Odukkal: successive lower-neighbor grace notes on non-repeated pitches
Kampita: trill (oscillation)
Orikai: upward bend at the end of a note

Janta: stressed ornaments

Ravai: inverted turn (i.e., starting from above)
Sphurita: successive lower-neighbor grace notes on a repeated pitch
Pratyahata: successive upper-neighbor grace notes on a repeated pitch
Khandippu: dynamic accent

1 Unless otherwise noted, this information is derived from Praytush, "Analysis and Classification of Ornaments in North Indian (Hindustani) Classical Music," Masters Thesis (Universitat Pompeu Fabra, Barcelona: 2010), pp. 7 - 12.
2 https://raaghindustani.medium.com/ornamentation-in-indian-classical-music-a67442e39eb7
3 This information comes from Gordon N. Swift, "Ornamentation in South Indian Music and the Violin", first published as "South Indian Gamaka and the Violin" in Asian Music, the journal of the Society for Asian Music, Volume XXI-2, 1990.


Looking for recommendation for naming conventions for different REST resource representations

Question: When using any REST framework (jersey, resteasy, spring-mvc) most of the time the (persistent) model classes are not the same as the exposed REST resource representations.
Even most of the time the incoming json representation (for e.g. create and update) is different then the outgoing json representation (e.g. list, get, ..).
I'm looking for some good naming conventions to sort this out.
Example structure:
<code>+-- my.comp.domain.Customer
+-- my.comp.rest.resource.CustomerResource (or CustomerController)
</code>
The CustomerResource will internally use the Customer to persist and retrieve data.
But to the outside (Request + Response objects) it will use a little bit different structure.
What I did now is:
<code>+-- my.comp.rest.representation.CustomerRequest
+-- my.comp.rest.representation.SimpleCustomerResponse
+-- my.comp.rest.representation.SimpleCustomerCollectionResponse
+-- my.comp.rest.representation.ExtendedCustomerResponse
</code>
Those are basically simple POJO containing just fields. The REST framework used will just use these to convert it to json.
Anyone else using different namings? I'm open for suggestions.
Answer: Rather than having Request, Response as separate try to use common basic representation to represent most and add additional based on needs.
POST /customers/  - class could be 'CustomerRepresentation / CustomerContract / CustomerRp/ representation.CustomerModel'
GET /customers/{id} - Return the same CustomerRepresentation or the above
PUT - same as well
GET /customers/ - CustomerListRepresentation or same
Links could be attached to its corresponding objects as optional things for create.
Please let me know your thoughts.
Comment: Whenever you return list when you dont pass customer id placeholder then your url should describe list i.e. GET /customers
Comment: otherwise GET /customer/{id}
Comment: Updated the URL plurally for all cases. It is not required to differentiate them based on singular/plural, it will be enough to define all of them plural to make the api simple. It is up to each individual to make their choice rather than confusing with too many conventions.


$2\int_0^x t y(t)\; dt = x^2 + y(x)=2 \lim_{n\rightarrow 0}\sum_{k=1}^n y(\epsilon_k) \; \Delta x_k$, what is the integral -method?

Question: I can find solution on fourth page here but it is not what I want. I have earlier learnt how to solve this kind of problems with serie like below. Could someone explain the integral way of doing things? Look at the border and the integrating terms, they are different -- you cannot integrate directly but use some tricks. What are they and the theory?
This is what I learnt earlier with series:
$$2\int_0^x t y(t) \; dt = x^2 + y(x)=2 \lim_{n\rightarrow 0}\sum_{k=1}^n y(\epsilon_k) \; \Delta x_k$$
where $x\in \mathbb R$, $y(x)$ is a continuous function in $\mathbb R$ to which holds $2\int_0^x ty(t) \; dt =x^2 +y(x)$ so
$$2\int_0^x t y(t) \; dt =2\lim_{n\rightarrow 0}^n \sum_{k=1}^n y\left(\frac{kx}{n}\right)\left(\frac{x}{n}\right).$$
but the solution shows an integral method. What is it (the integral method) on the page 4 called (sorry not in English, it uses simple integral apparently)?
More on page 644, X4.4, in the foreign course book I have been reading.
Comment: What is $f$, here?
Comment: @DylanMoreland: it is err, I think it must be $y$ here to calculate the integral where we have an unknown variable in the border.
Answer: As far as I can tell, what's being done on page 4 of the link you provided, is that you are starting with the integral equation:
$$
2\int_0^ x ty(t)\,dt=x^2+y(x).
$$
Then you differentiate both sides of the above with respect to $x$:
$$\tag{1}
{d\over dx}\biggl[ 2\int_0^ x ty(t)\,dt\biggr]={d\over dx} \bigl(x^2+y(x)\bigr)
$$
To evaluate the left-hand side derivative in the above, use the Fundamental Theorem of Calculus.
Equation $(1)$ becomes:
$$
 2  xy(x) =2x+y'(x).
$$
Upon rearangement, the above equation becomes:
$$\tag{2}
{dy\over dx}-2xy =-2x.
$$
The differential equation $(2)$  is a linear first order differential equation that can be solved using an integrating factor. This is what's done on page 4 of your link (the method used is explained in the link I provided).
Comment: ...I see now my book misleadingly calls this on page 563 with the term `"basic rule of analysis"`, I think it must be the same. Thanks.


How do I not match a word in a regex?

Question: If it possible to write a regex that matches bundle- followed by anything except when anything is map, so:
bundle-hello = match
bundle-map = don't match
bundle-ma = match
bundle- = match

Answer: Use a negative lookahead ((?!...)) which will make sure bundle- is not followed by map:
bundle-(?!map)

Demo

Javascript:
var RegExp = /bundle-(?!map)/;

RegExp.test('bundle-hello'); // true
RegExp.test('bundle-map');   // false

How do I not match a word in a regex?


Views of Grandchildren

Question: I need to create a view with contextual filters that displays all nodes below the specified term.
Examle:

Colors

Blue

Cerulean
Slate

Yellow

Bright

Hi-liter
Pastel Yellow

Not so bright
Mustard

(Bold represents nodes)
So "Colors" would list Cerulean, Slate, Hi-liter, Pastel Yellow, and Mustard.
"Yellow" would list Hi-liter, Pastel Yellow, and Mustard.
"Bright" would list Hi-liter and Pastel Yellow.
etc.
How can I create a contextually-filtered view of nodes by taxonomy term that takes the entire lineage into account?
Answer: You can do this in Views as you said with Contextual Filters.
In your view click "add" next to Contextual Filters and in the modal check "Content: Has taxonomy term ID (with depth)". Then click "Add and Configure" and you will see the "Depth" option.  In the case of your color example above your depth would be 2 and when passing "Colors" to the view it would return all the nodes you bolded.
If you want the whole depth if it has great grandchildren and so on you would need to set the depth higher to account for the number of levels of your taxonomy hierarchy.  For your example and passing "Colors" as the argument, a depth of 0 would return no nodes, 1 would return "Cerulean, Slate and Mustard", and 2 would return all of the nodes as mentioned above.
Passing the other Colors as arguments would behave in the way you desire.
Comment: Note I recently got caught out trying to do this on products (ie not nodes) and it doesn't work. If anyone's interested in that you can use [Taxonomy entity index](http://drupal.org/project/taxonomy_entity_index) and a patch from [this issue](http://drupal.org/node/1353228).

Views of Grandchildren


For any linearly independent vectors $x, y$, there is such a norm that $||x||_*>||y||_*$

Question: Today I've seen in my class that:

For any linearly independent vectors $x, y$, there is such a norm that $||x||_*>||y||_*$

Our lecturer called it Benchmark theorem.
I wanted to learn more about it: some non-trivial applications and so on. Yet after googling and skimming through a couple of books, I couldn't find anything including the theorem itself. It seems to me that the theorem is known under some other name or in some other form.
Any information on identifying the theorem is much appreciated.
Answer: You can check easily that if $\|\cdot\|$ is some fixed norm and $A$ is a bijective endomorphism
$$\|v\|_* = \|Av\|$$
defines another norm. Expand $\{x,y\}$ to a basis of the space and define $A$ by
$$Ax = kx,\qquad Ay = k^{-1}y\qquad\text{for $k$ large enough};$$
$$Ab = b\qquad\text{for the other elements of the basis}.$$
Answer: Consider the idea of Mahalanobis distance:-
$$||x-y||_* = \sqrt{(x-y)^TS^{-1}(x-y)}$$
S is a positive self definite symmetric matrix, whose eigenvaluess represent scale in the direction of corresponding eigenvector.
It is an anisotropic norm. Since $x$ and $y$ are linearly independent, $x$ has a component along a direction perpendicular to $y$. By increasing the scale indefinitely along this axis, we can increase $||x||_*$ without affecting $||y||_*$. Hence we can find $S$ such that one of its eigenvector is along $y$ and increase the corresponding eigenvalue to a large number.


Create Edges by ETL in same Class

Question: 1) Class Items with ItemId and Name ready in the database.
2) CSV-file: two columns,
ItemId1,ItemId2001

ItemId1,ItemId2345

ItemId1,ItemId2381

...
ItemId2,ItemId8393

ItemId2,ItemId8743

..
etc.
Question:
How to define a ETL json-file to create Edges between ItemId1 and all the ItemId's in col#2, and between ItemId2 and its col#2-peers.
Answer: I tried to reproduce your problem.
I had this Items

and with this code I connected them
{
  "source": { "file": { "path": "myPath/item.csv" } },
  "extractor": {"row": {}},
    "transformers": [{
        "csv": {
            "separator": ","
        }
    },
    {
    "command" : {
            "command" : "create edge from (select from Item where idItem= '${input.idItem1}') to (select from Item where idItem= '${input.idItem2}')",
            "output" : "edge"
        }
    }
  ],
  "loader": {
    "orientdb": {
       "dbURL": "plocal:myPath/myDb",
       "dbType": "graph"
    }
  }
}

and I got

Hope it helps.
Comment: Can a "unresolvedLinkAction" be added here? In case yes, where?
Comment: I don't think is possible. Reading the documentation http://orientdb.com/docs/2.2/Transformer.html it seems that you can use it only with merge and edge transformer, not with the command
Comment: This procedure ran wild creating millions of edges. Adding entry in Issue tracker.
Comment: Not an issue. The key thing is duplicate keys in the select from. Need @rid to make the source vertex unique.

Create Edges by ETL in same Class