Reorganize values in 3 arrays based off a unique combination of values in 2 of the arrays in C#

Question: I have 3 arrays with the data types below:
string[] arrID = {"111", "222", "333", "444", "555", "666", "777"};

DateTime[] arrDates = new DateTime[]
{
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 21),
    new DateTime(2015, 03, 21),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20)
};

string[] arrTime = {"8:20", "8:40", "8:20", "9:10", "8:20", "9:10", "8:20"};

I have added sample elements into the 3 arrays to simulate the types of data we will be having in these arrays.
Each index number in each of these elements contain values related to one record.
Example: The values in each of the array with index 3 contains values that make up one record.
I need to put values in these 3 arrays into the following 3 arrays: arrNEWID, arrNEWDate and arrNEWTime based on the following conditions:

For each unique date and time combination, a separate element needs to be added to each of the arrays.
If a unique date and time combination has more than one ID, then the IDs need to be separated by a comma.

Expected output in the 3 NEW arrays
--------------------------------------------------
| index | arrNEWDate | arrNEWTime | arrNEWID     |
--------------------------------------------------
|  0    | 03/20/2015 | 8:20       | 111,333,777  |
|  1    | 03/20/2015 | 8:40       | 222          |
|  2    | 03/20/2015 | 9:10       | 666          |
|  3    | 03/21/2015 | 8:20       | 555          |
|  4    | 03/21/2015 | 9:10       | 444          |

Notes:

The data types on the 3 NEW arrays need to be string
The input and output need to remain as arrays. Lists may be created for processing

Here is the code I have used already:
string[] arrID = { "111", "222", "333", "444", "555", "666", "777" };

DateTime[] arrDates = new DateTime[]
{
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 21),
    new DateTime(2015, 03, 21),
    new DateTime(2015, 03, 20),
    new DateTime(2015, 03, 20)
};

string[] arrTime = { "8:20", "8:40", "8:20", "9:10", "8:20", "9:10", "8:20" };

string str_arrNEWID = "";
string str_arrNEWDate = "";
string str_arrNEWTime = "";
string[] arrNEWID = "".Split('~');
string[] arrNEWDate = "".Split('~');
string[] arrNEWTime = "".Split('~');

for (int i = 0; i <= arrID.GetUpperBound(0); i++)
{
    int intExists = 0;

    for (int j = 0; j <= arrNEWDate.GetUpperBound(0); j++)
    {
        //check if date matches for the current index being checked
        if (arrNEWDate[j].ToString() == arrDates[i].Date.ToString("MM/dd/yyyy"))
        {
            //check if time matches for the same index
            if (arrNEWTime[j].ToString() == arrTime[i].ToString())
            {
                //existing record
                intExists = 1;
                arrNEWID[j] = arrNEWID[j] + "," + arrID[i];
                str_arrNEWID = string.Join("~", arrNEWID);
            }
        }

    }

    if (intExists == 0)
    {
        //new record
        str_arrNEWDate = str_arrNEWDate + "~" + arrDates[i].Date.ToString("MM/dd/yyyy") ;
        arrNEWDate = str_arrNEWDate.Split('~');

        str_arrNEWTime = str_arrNEWTime + "~" + arrTime[i].ToString() ;
        arrNEWTime = str_arrNEWTime.Split('~');

        str_arrNEWID = str_arrNEWID + "~" + arrID[i];
        arrNEWID = str_arrNEWID.Split('~');
    }
}

The main challenge is that the compiler that needs to be used to compile this code doesn't support the lambda operator (=>)
I need to know:

If there is a way to do it so that I don't have to use the temporary string variables and split function to refresh the arrays each time
If there is any way to get rid of the multiple loops
Comment: What have you tried? What did that code do? How was that different from what you expected or wanted it to do? Please see http://stackoverflow.com/help/how-to-ask for advice on how to present your question in a clear, answerable way.
Comment: I have added these details. thanks for ur time :)
Comment: Question has been updated. Please remove or explain down votes
Answer: The kind of processing you are looking for is very easily handled by the LINQ features in .NET. In particular, LINQ includes the very handy group...by and orderby which are directly applicable to your scenario. Unfortunately, the normal LINQ syntax also relies heavily on lambda syntax, which you say you cannot use.
However, note that the lambda syntax is just a "syntactic sugar", i.e. a way of expressing to the compiler something you could write yourself just in a more verbose way. In particular, the lambda syntax as used here generates anonymous methods, which you can of course just write as named methods.
Anonymous methods also allow "capturing" of variables, but in your scenario we can avoid needing to capture any variables by doing two things:

Put the data in class fields, where the named method can access them
Use the Enumerable.Select() overload that passes an index to the selector method, so that the named method can correlated the source items with the elements in the other two arrays that relate to each one

Likewise, while anonymous types would be handy here, effective use of those relies on anonymous methods and inferring types of the method parameters for those anonymous methods. In lieu of that, the Tuple class will serve just as well.
Putting all that together, we get something that looks like this:
class Program
{
    static readonly string[] arrID = { "111", "222", "333", "444", "555", "666", "777" };

    static readonly DateTime[] arrDates = new DateTime[]
    {
        new DateTime(2015, 03, 20),
        new DateTime(2015, 03, 20),
        new DateTime(2015, 03, 20),
        new DateTime(2015, 03, 21),
        new DateTime(2015, 03, 21),
        new DateTime(2015, 03, 20),
        new DateTime(2015, 03, 20)
    };

    static readonly string[] arrTime = { "8:20", "8:40", "8:20", "9:10", "8:20", "9:10", "8:20" };

    static void Main(string[] args)
    {
        // Generate intermediate query with grouped, ordered data
        Tuple<DateTime, string>[] q = arrID.Select(CreateRecord)
            .GroupBy(GetKey)
            .OrderBy(GetGroupKey)
            .Select(FlattenGroup)
            .ToArray();

        // Final output arrays here
        string[] str_arrNEWDate = new string[q.Length],
            str_arrNEWTime = new string[q.Length],
            str_arrNEWID = new string[q.Length];

        for (int i = 0; i < q.Length; i++)
        {
            str_arrNEWDate[i] = q[i].Item1.ToString("MM/dd/yyyy");
            str_arrNEWTime[i] = q[i].Item1.ToString("HH:mm");
            str_arrNEWID[i] = q[i].Item2;

            // Diagnostic for demonstration purposes
            Console.WriteLine("{0}: {1}", q[i].Item1, q[i].Item2);
        }
    }

    private static Tuple<DateTime, string> CreateRecord(string id, int index)
    {
        return Tuple.Create(arrDates[index] + TimeSpan.Parse(arrTime[index]), id);
    }

    private static DateTime GetKey(Tuple<DateTime, string> record)
    {
        return record.Item1;
    }

    private static DateTime GetGroupKey(IGrouping<DateTime, Tuple<DateTime, string>> group)
    {
        return group.Key;
    }

    private static Tuple<DateTime, string> FlattenGroup(IGrouping<DateTime, Tuple<DateTime, string>> group)
    {
        return Tuple.Create(group.Key, string.Join(",", group.Select(GetId)));
    }

    private static string GetId(Tuple<DateTime, string> record)
    {
        return record.Item2;
    }
}

As you can see, everywhere that you would normally write a lambda expression, in the above instead we just have a regular static method to do the work. Note that if you were dealing with a non-static class and the method needed to access instance members, the methods can simply be made non-static instead.
Delegate type inference takes care of constructing the delegate instances that need to be passed to the LINQ methods.

Note: while you don't say so explicitly in your question, the nature of the specifications strongly hints that this is homework. As such, you should make very sure you take the time to actually understand the above code example. Otherwise you have learned nothing, wasting your time in the class. Please feel free to ask questions here about my answer, and of course you should feel comfortable asking your teacher to help you understand both the original assignment and this answer.
Answer: arrays make everything more difficult.
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
using System.Linq;

namespace UnitTestProject1
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            string[] arrID = { "111", "222", "333", "444", "555", "666", "777" };

            DateTime[] arrDates = new DateTime[]
            {
                new DateTime(2015, 03, 20),
                new DateTime(2015, 03, 20),
                new DateTime(2015, 03, 20),
                new DateTime(2015, 03, 21),
                new DateTime(2015, 03, 21),
                new DateTime(2015, 03, 20),
                new DateTime(2015, 03, 20)
            };

            string[] arrTime = { "8:20", "8:40", "8:20", "9:10", "8:20", "9:10", "8:20" };

            int i = -1;
            int index = 0;
            List<ewansObject> l2 = arrID.Select(o => new ewansObject() { date = arrDates[++i].ToString(), time = arrTime[i], id = arrID[i] }).GroupBy(k => k.date + k.time).Select(o => new ewansObject() { index = index++.ToString(), date = o.First().date, time = o.First().time, id = String.Join(",", o.Select(x => x.id)) }).ToList(); ;

            string[] arrNEWID = l2.Select(x => x.id).ToArray();
            string[] arrNEWDate = l2.Select(x => x.date).ToArray();
            string[] arrNEWTime = l2.Select(x => x.time).ToArray();

            Console.WriteLine(String.Join("|", arrNEWID));
            Console.WriteLine(String.Join("|", arrNEWDate));
            Console.WriteLine(String.Join("|", arrNEWTime));
        }

        public class ewansObject
        {
            public string index;
            public string date;
            public string time;
            public string id;
        }
    }
}

maybe I should change my name to homework4points
Comment: thanks. however, would it be possible to do this without using the lambda operator?


regex multiline not working on repeated patterns

Question: I am trying to get a regex expression to match multiple patterns with multilines but it keeps matching everything. For instance I want to match two times this code:
STDMETHOD(MyFunc)(
D2D1_SIZE_U size,
_In_opt_ CONST void *srcData,
) PURE;

STDMETHOD(MyFunc2)(
_In_opt_ CONST void *srcData2,
UINT32 pitch2,
) PURE;

I followed this link:
How do I match any character across multiple lines in a regular expression?
and came up with this pattern:
\bSTDMETHOD\b((.|\n|\r)*)\bPURE\b

however it does not work. The ((.|\n|\r)*) matches the whole thing. I want it to stop when it finds "PURE". In other words a proper match would have given me two matches of the code above, but instead my expression only stops at the last "PURE" key word, making just one match.
let me know if you see why it does not work.
Comment: Read "laziness instead of greediness": http://www.regular-expressions.info/repeat.html
Answer: You should use Lazy quantifiers, as they stop on the first match they find:
\bSTDMETHOD\b((.|\n|\r)*?)\bPURE\b

Tested on Regexr.com
Comment: btw, mark as a response if solved your problem and upvote answers that helped you
Answer: Instead of (.|\n|\r)*, use .*? (dot with the non-greedy modifier ?) and add the s and g flags, like this:
/\bSTDMETHOD\b(.*?)\bPURE\b/sg

The s flag means that . matches \r and \n, and the g flag lets you capture all matching strings in the subject text.
Demo
Answer: In Python, try:
txt='''\
STDMETHOD(MyFunc)(
D2D1_SIZE_U size,
_In_opt_ CONST void *srcData,
) PURE;

STDMETHOD(MyFunc2)(
_In_opt_ CONST void *srcData2,
UINT32 pitch2,
) PURE;'''

import re

for i, m in enumerate(re.finditer(r'\bSTDMETHOD\b(.*?)\bPURE\b', txt, flags=re.S | re.M)):
    print 'Match {}:\n{}\n==='.format(i, m.group(1))

Prints:
Match 0:
(MyFunc)(
D2D1_SIZE_U size,
_In_opt_ CONST void *srcData,
)
===
Match 1:
(MyFunc2)(
_In_opt_ CONST void *srcData2,
UINT32 pitch2,
)
===

Note the regex \bSTDMETHOD\b(.*?)\bPURE with the flags re.S | re.M
The re.S says that Make . match any character, including newlines
If you anchor your matches, you want re.M so that ^ and $ match the beginning and end of lines with re.S


Cannot refer the font files that were uploaded in a server in my local html file

Question: I have to refer some font files in my html pages. The problem is the font files are uploaded in an server. So i need to refer to an external font files.
In my html page through font face i am importing the font file that has been on an server.
Unfortunately i cannot paste the code
Answer: Linking to external font files is difficult, so what you could do is copy the font files from the server so you can link to them locally.
If you don't have FTP access to the server, you could use web inspector > network > fonts. Copy the URL for each font file, then open and save it from your web browser.
Once you've downloaded them, you can link to them locally.
Hope this helps!


How to vectorize sub2ind?

Question: I have a 2 dimensional array L, and I am trying to create a vector of linear indices ind for each row of this array.
L=
1 5 25 4 0 0
2 3 3 45 5 6
45 5 6 0 0 0

I am using lenr to store the number of non zero elements in each row (starting from column 1).
lenr=
4
6
3

Then I have 1x45 array RULES. Indices stored in L refer to elements in RULES. Since I want to vectorize the code, I decided to create linear indices and then run RULES(ind).
This works perfectly:
ind=sub2ind(size(L),1,lenr(1));

while this doesn't work:
ind=sub2ind(size(L),1:3,1:lenr(1:3));

Any ideas?
UPDATE:
This is what I initially tried to vectorize the code, but it did not works and that's why I checked linear indices:
rul=repmat(RULES,3);
result = rul((L(1:J,1:lenr(1:J))));

Comment: Here let me *feed* you - `allind = reshape(1:numel(L),size(L,1),[])` and then `ind = allind(L~=0)`. Not tested though.
Comment: Please provide a *complete* example and explain what you're trying to do.
Comment: @excaze: Look my updates.
Comment: Why are you using `1:lenr(1:3)` instead of `lenr(1:3)`?
Comment: @excaza: I use 1:lenr(1:3) in order to get L elements FROM 1 TO the corresponding length for each row (lenr contains lengths excluding 0s).
Comment: Create a mask with `L~=0` and use it on a list of all linear indices `1:numel(L)`'s 2D reshaped version (to match L's size) to get such selective linear indices?
Comment: @Divakar: I appreciate if you could give some example.
Comment: @KlausosKlausos If you tested what you're trying to do you'd see `1:lend(1:3)` returns `[1 2 3 4]`. I would suggest learning how to [debug your code](http://www.mathworks.com/help/matlab/matlab_prog/debugging-process-and-features.html).
Comment: @excaza: Why do you think I yell at you??! I just marked key words with capslock...
Answer: If I correctly interpret your edit, you want to create a variable result that contains the elements of RULES indicated by the non-zero elements of L. Note that in general, the best way to vectorize sub2ind is to not use sub2ind.
If you want result to be a linear array, you can simply write
%// transpose so that result is ordered by row
L_transposed = L.';
result = RULES(L_transposed(L_transposed>0));

If, instead, you want result to be an array of the same size as L, with all numbers in L replaced by the corresponding element in RULES, it's even simpler:
result = L;
result(result>0) = RULES(result>0);

How to vectorize sub2ind?


Proving the finiteness of Expectation given a geometric condition

Question: For a non-negative integer-valued random variable $X$, it is given that $P(X > kN) \le p^k$ for any
$k \ge 1$ with some fixed $0 < p < 1$ and fixed $N > 0$, then show that $E[X] < \infty$.
My approach :
We know that for an integer valued RV which is non-negative, $E[X]=\sum_{i=0}^{\infty}P[X>i]$.
My plan was to take $k=\frac{i}{N}$ and thus $E[X]$ is bounded above by a geometric series. But the only caveat is that $k$ in my case might not be $\ge 1$. I am stuck here! Anyone with other ideas?
Comment: It will be $\ge1$ for all $i$ large enough, which is enough for the convergence of the series.
Comment: https://math.stackexchange.com/q/4324158/321264
Answer: Note that $\{kN: k \in \mathbb{N}_0\} = N\mathbb{N}_0 = \{0, N, 2N, 3N, \cdots\}$. Note that
\begin{align*}
P(X > i) \le P(X > kN) \quad \text{whenever } kN \le i < (k+1)N
\end{align*}
and so
\begin{align*}
E[X] = \sum_{i=0}^{\infty} P(X > i) \le N \sum_{k=0}^{\infty}P(X > kN) \le N\sum_{k=0}^{\infty} p^k = \frac{N}{1-p} < \infty
\end{align*}
Comment: Ah thats sweet!
Answer: Let $Y=X/N$. Since $P(Y>0)\leq 1$ and $P(Y>i)\leq p^i$ for all $i\geq 1$, we have:
$$E(Y)=\sum_{i=0}^\infty P(Y>i)\leq 1+\sum_{i=1}^\infty p^i = \frac{1}{1-p} < \infty.$$
The result follows (and $E(X)\leq N/(1-p)$).


Group date field by day for chart

Question: I have a custom database table with a datetime column that I have made accessible to Views. I want to create a line charts of number of rows per day but I can't get the View right.
With aggregation on I have a COUNT on my ID field and I'm trying to group on my date field by day (it currently has year -month - day - hour - minute - second). I have set the date format to d/m/Y and aggregation to group results together but they are still separate.

Obviously this makes for a pretty bad line chart

What I really want is
<code>18/01/2017    23
19/01/2017    25
20/01/2017    19
</code>
I tried the Views Date Format SQL Field module (dev version too) but it doesn't seem to work as I don't get an option to format my date as SQL.
Is this possible in Views?
Comment: About the "Views Date Format SQL Field"-module you mentioned, do you mean https://www.drupal.org/project/views_date_format_sql ? I.e. without the "field"-part of it? Also, are you using any (contributed) charting module for such charts? If so which one? FYI: tagging a question with "charts" doesn't automatically mean "I'm using the ***Charts*** module", which is why I wonder ... 1 more question: how about an alternative to using the Views module to create such a similar chart (which uses a free of charge, and open source, charting engine ...)?
Comment: Yes, that's the module I meant. I've tried both the Charts module and the Visualization module, same problem for both. I've not tried it without Views yet, I though this kind of grouping would be something Views could do.
Answer: I believe there is a way to do this by rewriting the field described in this issue: https://www.drupal.org/project/charts/issues/2915097. There is also grouping enabled now if you want to stack your charts. Please download the latest -dev version.

Group date field by day for chart


Passing IEnumerable parameter to WebAPI

Question: I'm getting started with WebAPI and for the most part all is going well. I'm running into a problem with one particular function. This one differs from the other because it's only parameter is of type IEnumerable.
I've set a breakpoint on the first line of the Post() function and I'm hitting the function but the "values" parameter ALWAYS has a count of 0. I verified that the parameter going in on the client side does, in fact, contain an array of integers. If I remove the [FromUri] attribute, the 'values' parameter is NULL instead of a count of 0.
How do I get my array of integers to come through on my WebAPI function?
Here's the WebAPI function:
[System.Web.Mvc.HttpPost]
public void Post([FromUri] IEnumerable<int> values)
{
    if (values == null || !values.Any()) return;

    int sortorder = 1;
    foreach (int photoId in values)
    {
        _horseRepository.UpdateSortOrder(photoId, sortorder);
        sortorder++;
    }
}

Here's the AJAX call (this is using the jQuery UI Sortable feature):
$(".sortable").sortable({
    update: function (event, ui) {
                var newArray = $(".sortable").sortable("toArray");

                $.ajax({
                    url: '/api/photo',
                    type: 'POST',
                    contentType: 'application/json, charset=utf-8',
                    async: true,
                    dataType: 'json',
                    data: JSON.stringify(newArray),
                    complete: function (data) { }
                });
            }
        });

Answer: contentType: 'application/json, charset=utf-8',

should become (the separator between the content type and the charset is a semicolon, not a comma):
contentType: 'application/json; charset=utf-8',

and:
public void Post([FromUri] IEnumerable<int> values)

should become (there are no Uri parameters in a POST request):
public void Post(IEnumerable<int> values)

Now you are good to go assuming that newArray (which you haven't shown here) is an array of integers:
newArray = [ 1, 2, 3 ]

Comment: Figures it was something as stupid as a comma that should be a semi-colon (the code originally was written without [FromUri], that was added as part of my debugging attempts). Thanks!
Comment: @Darin It is possible to extract Uri parameters from a POST request; It is just this instance of usage that it should be from the body, i.e. public void Post([FromBody] IEnumerable values).

Passing IEnumerable parameter to WebAPI


ResponsiveSlides button tabs position

Question: I'm using ResponsiveSlides js to create image sliders. However, is that possible to position the bottom button tabs to be within the image?
my test link:
http://websolutions.my/slider-test/
I tried editing the css but cant able to do so
Answer: Add following code in your css
<code>#slideshow ul.centered-btns_tabs{
     position: relative;
     margin-top: -10px;
     margin-bottom: -10px;
     top: -45px;
     z-index: 999;
}
</code>
Comment: Sorry my bad. Edited code -  `ul .centered-btns_tabs` to be read as `ul.centered-btns_tabs`.

I haven't used  `#slideshow ul` because there are 2 `ul` inside #slideshow.

Above solution worked because of the margin reset in the other ul element. Its better to add the class.
Answer: Add below property to the UL
<code>#slideshow ul {
  font-size: 14px;
  margin-top: -50px;
  position: relative;
  z-index: 99;
  margin-bottom: 40px;
}
</code>

ResponsiveSlides button tabs position


How's this Economics? Type - I Error & Type - II Error: Pregnancy test analogy - is it legit?

Question: I don't have five points. Please move this to your Meta?
Where's the economics or econometrics in Type - I Error & Type - II Error: Pregnancy test analogy - is it legit?? Why was it posted here? Doesn't it belong on math or statistics Stack?
Answer: Econometrics questions are considered on topic here and this is an econometrics question. Econometrics also deals with the issues of type I error & type II error (see for example Verbeek (2008) a guide to modern econometrics pp 31. - and I don't think there is any econometric textbook that would omit discussion of these).
As you can see in our help center, on page that defines what is on and off topic:

Econometrics: Totally on topic, but especially for theoretical questions, you may subsequently try the "All of Statistics" .SE site Cross-Validated.

Here we simply have some overlap with the cross-validated site. Sites sometimes overlap. For example, you can often see people cross-posting programming questions relating to statistics on both Cross-validated and stack overflow because they fit both sites. Cross-validated.SE and Mathematics.SE also overlap (due to statistics being just applied  mathematics).   I would personally agree the question above is better fit for Cross-Validated but it is by no means off-topic by the  rules of the site.


What does Goodfellow mean by "generator conditional variance"?

Question: In Goodfellow's Generative Adversarial Nets, it is mentioned that

Our work backpropagates derivatives through generative processes by using the observation that
  $$\lim_{\sigma \rightarrow 0} \nabla_x \mathbb E_{\epsilon \sim N(0,σ^2 \mathbf I)} f (x + \epsilon) = \nabla_x f (x)$$

What exactly does this mean, and how is it of relevance to the way backpropagation is performed? This is mentioned at the beginning of the paper and is not referred to again, so it's not clear to me why this is significant (or how it affects the learning process).
Right after this, it is said that

We were unaware at the time we developed this work that Kingma and Welling [18] and Rezende
  et al. [23] had developed more general stochastic backpropagation rules, allowing one to backpropagate through Gaussian distributions with finite variance, and to backpropagate to the covariance
  parameter as well as the mean. These backpropagation rules could allow one to learn the conditional variance of the generator, which we treated as a hyperparameter in this work.

So what exactly is different between these two implementations? What does Goodfellow mean by "backpropagating covariance/mean through Gaussian distributions" and "conditional variance of the generator"?
Answer: In both a GAN model and a VAE model, there is a decoder network $g$ (in GAN literature, the decoder is usually called the generator instead).
In both cases, we define a latent variable $z$ to be distributed normally, usually $z \sim \mathcal{N}(0, I)$. A typical GAN or VAE model defines a density on images:
$$p(x) = \int p(z)p(x|z) dz $$ where $p(x|z)$ is gaussian, with mean and variance determined by the generator network: $\mu, \sigma = g(z; \theta)$. In the case of this GAN work, only $\mu$ was determined by $g$, and $\sigma^2$, the "conditional variance of the generator", was fixed as a hyperparameter.
You can sample an image $x$ by first sampling some $z$ from the standard normal prior, then sample $x$ from $\mathcal{N}(\mu,\sigma=g(z;\theta))$. In both the case of VAE and GAN, you'll eventually want to backpropagate some function of $x$, and compute a value such as $\nabla_\theta f(x)$.
Notation overload warning: what is "$x$" in the equation in OP is $\mu$ here, and what is $x$ here is "$x + \epsilon$" in OP.
This is problematic because sampling isn't a function you can differentiate through, so it's not clear how to compute the gradient. One approach taken by Kingma / Welling / Rezende is to "reparameterize" $x$ as $x = \mu + \sigma \epsilon$ where $\epsilon$ is a distributed as a standard normal. Basically this shifts the sampling operation over to $\epsilon$, so that it doesn't "get in the way" of differentiation.  Of course the resulting $x$ is still drawn from the same distribution as before. Note that you can backpropagate through both $\mu$ and $\sigma$ (which are functions of $\theta$) now.
Goodfellow had a different solution to this problem: just substitute $\nabla_\theta f(x)$ with $\nabla_\theta f(\mu)$. Equation 1 states that in the limit, the expectation of the former is the latter.
Comment: Thanks! Would you happen to know of any resources comparing performance of vanilla versus reparametrized GANs?
Comment: I don't think anyone has actually bothered to try it out... as Dougal mentions, most people leave the variance at 0.
Comment: Note that in a GAN, the conditional variance $\sigma^2$ is (almost) always just $0$. You can motivate things that way, via equation 1, but you don't really need to, and later explanations of GANs (including [Goodfellow's 2016 tutorial](https://arxiv.org/pdf/1701.00160.pdf)) don't do that.


mixed linear models comparison

Question: When I get 2 mixed linear Models for a comparison between them
For example,
(B, C, D are factors)
Mdoel 1 <- lmer(A ~ B * C * D + (1|individual), data = Data_1) in script
Outcome in console:
A ~ B * C * D + (1 | individual)
Model 2 <- update(Model 1, . ~ . - B : C : D) in script
Outcome in console :
A ~ B + C + D + (1 | individual) + B:C + B:D + C:D

Also, B,C,D are for fixed effects and (1|individual) is for a random effect

What is the difference between A ~ B + C + D + (1|individual) and A ~ BCD + (1|individual)?

By Model 2 what do mean ". ~ ." and "- B : C : D" both in Script?

2-2. A ~ B + C + D + (1 | individual) + B:C + B:D + C:D
What does mean " B:C + B:D + C:D "?
Answer:
What is the difference between A ~ B + C + D + (1|individual) and A ~ BCD + (1|individual)?

I think you mean B * C * D, not BCD. BCD would represent a variable called BCD, wheras B * C * D means "fit fixed effects for B, C and D, and all the interactions between them" (that is, the three 2-way interactions plus the 3 way interaction). So the difference betwwen A ~ B + C + D  and A ~ B * C * D is that in the former we fit fixed effects for the each of the variables while in the latter we also fit all the interactions.

By Model2 what do mean ". ~ ." and "- B : C : D" both in Script?

You are telling the software to update model 1 by removing  the 3-way interaction B:C:D(because you used the minus symbol -), so that model 2 contains only the 2-way interactions (and the main effects)

What does mean "B:C + B:D + C:D"?

B:C + B:D + C:D tells the software to fit the three 2-way interactions between the variables. Note that
~ B * C * D

is exactly the same as:
~ B + C + D + B:C + B:D + C:D

Comment: Super super thank you I did understand totally :) I have one more question so I did AIC and BIC for model comparison model 1 has more information than model 2 cuz model 2 has no information about 3-way interaction but the outcome of AIC and BIC tells the model 2 is better is it possible?
Comment: You're welcome :) Yes that is possible !

mixed linear models comparison


Erro 1114 (HY000) no MySQL: The table is full

Question: Exportei uma base de dados com uma dimensão de 14GB do MySQL de um computador com SO Windows e pretendo importá-la no MySQL de um computador com o SO Ubuntu 14.04
Utilizei o seguinte comando para exportar a base de dados "dados":
mysqldump -u root -p dados > teste.sql

Depois de exportada tentei importar a mesma no MySQL do computador com SO Ubuntu 14.04 através de:
mysql -u root -p dados < teste.sql.

Após alguns minutos de execução aparece o seguinte erro:

ERROR 1114 (HY000): The table "XXXXXXX" is full

Como ultrapassar este erro?
Answer: Imagino que você esta usando InnoDB então confira o espaço em disco disponível para /usr que é a partição onde comumente esta a base do mysql. Se o espaço não for suficiente para importar, talvez seja melhor o local onde ele salva estas tabelas.( ibdata1, ibdata2 )
Pesquise por:
innodb_data_file_path


How to show that every bounded variation function on $[a,b]$ is differentiable a.e?

Question: I'm struggling with this question for over a week now. I know the proposition is true, but haven't managed to prove it yet. any suggestions anyone?
($f$ is BV on $I$ if $$\sup\left\{\sum|f(b_k)-f(a_k)| :a_{k+1}\gt b_{k}\gt a_{k} ; a_k,b_k\in I\right\}\lt \infty)$$
Comment: Are you allowed to use the fact that monotonic functions are differentiable a.e.?
Answer: Every function of bounded variation is the difference of two monotonically non-decreasing functions. Monotonically non-decreasing functions are differentiable almost everywhere.
Comment: @Snir, see also Michael W. Botsko, An Elementary Proof of Lebesgue's Differentiation Theorem, The American Mathematical Monthly, 110 (2003), 834-838.
Comment: @Snir: These results are quite standard. For example, you can find them both proven in Prof. Tao's book "An Introduction to Measure Theory", which can be found for free online (just ask Google!)
Comment: please prove both these claims
Comment: Snir didn't say his function is real-valued.  If it is complex valued: first show the real and imaginary parts are BV, then use the result here.


Como obter os amigos e seguidores de uma pessoa no Twitter usando a biblioteca tweepy?

Question: A função getting_friends_follwers() abaixo funciona se eu remover o valor 100 de (cursor2.items(100)) . O meu objetivo é pegar estes nomes (seguidores e amigos) e guardar em um arquivo "amigos.txt".
O problema: O nome screen_name possui uma quantidade gigantesca de amigos e seguidores e, assim, a conexão é encerrada pelo Twitter. Pensei em tentar capturar os nomes de 100 em 100 (daí o valor 100 na chamada de cursor2) mas ocorre o seguinte erro:
builtins.TypeError: '<' not supported between instances of 'User' and 'User'

Como corrigir?
f = open("amigos.txt","w")
Meu = []
def getting_friends_follwers():
    # Get list of followers and following for group of users tweepy
    cursor = tweepy.Cursor(api.friends, screen_name="Carlos")
    cursor2 = tweepy.Cursor(api.followers, screen_name="Carlos")
##    for user in cursor.items():
##        print('friend: ' + user.screen_name)

    for user in sorted(cursor2.items(100)):###funciona se eu tirar este valor!!!
         f.write(str(user.screen_name)+ "\n")

         print('follower: ' + user.screen_name)

f.close()
getting_friends_follwers()

Comment: Eu não conheço tão bem o tweepy, mas esse erro deve ser por causa do `sorted`, pois ele não consegue comparar os users retornados pelo cursor para ordená-los
Answer: Tente isso

f = open("amigos.txt","w")
user = '@Carlos'
count=300 # quantidade de seguidores que será retornado pela api (altere para quantidade que quiser, só cuidado com o limite da api)

try:
    followers = tweepy.Cursor(api.followers,screen_name=user).items(count)

    for follower in followers:
         f.write(str(follower.screen_name)+ "\n")

except BaseException as e:
          print('failed on_status,',str(e))
          sleep(3)
f.close()

Esse deve código faz o que você está procurando, porem o usuário @carlos tem o perfil privado, e não é possível coletar seus seguidores.


Não consigo realizar PUT e DELETE em Lista com relacionamento OneToMany Spring Boot

Question: Estou criando uma API via Spring Boot onde preciso fazer o PUT e DELETE de uma classe que será uma lista.
Criei uma classe "A" que vai receber a ligação da classe "B" que terá notação OneToMany, onde consegui com sucesso realizar o post e getAll desta classe B que é uma lista, essas duas classes estão com o crud no controller da clase "A".
Para realizar PUT, tentei de duas maneiras. A primeira é verificando se a classe-a existe, onde é retornado esta mensagem:
"The given id must not be null!; nested exception is java.lang.IllegalArgumentException: The given id must not be null!"
A segunda, não realizei busca pela classe "A", onde só inseri os parâmetros da classe "B" e quando faço isso, ele realiza um post para um outro local que não é o mesmo da URL que criei para o método GET e não consegui encontrar onde foi parar este post criado.
Em anexo, se encontram as duas formas que criei para realizar o put, eu não coloquei o delete, pois para fazer a busca do parâmetro "A" acredito que seja a mesma coisa.
Eu tentei desta forma que esta página explica, porém por eu não int para as variáveis ID e sim long, não consigo realizar o método delete:
https://www.sourcecodeexamples.net/2021/08/spring-boot-one-to-many-crud-example.html
Código com o parâmetro A informado na busca:
<code>@RequestMapping(value = "/{classeAparametro}/classeb/{idClasseB}", method = RequestMethod.PUT)
public Progressao atualizarClasseB(@PathVariable("classeAparametro") String classeAparametro,
        @PathVariable("idClasseB") Long idClasseB, @Valid @RequestBody Classeb classeB) {
    if(!classeARepository.existsById(classeAparametro)) {
        throw new NaoEncontrado("Classe A não encontrado!");
    }
    return classeBRepository.findById(idClasseB).map( idClasseBSet -> {
        classeB.setData_ingresso(classeB.getData_ingresso());
        classeB.setModalidade_acesso(classeB.getModalidade_acesso());
        return classeBRepository.save(classeB);
    }).orElseThrow(() -> new NaoEncontrado("Não existe classeB com este ID!"));
}
</code>
Código sem o parâmetro citado na busca:
<code>@RequestMapping(value = "/{classeAparametro}/classeb/{idClasseB}", method = RequestMethod.PUT)
public Progressao atualizarClasseB(@PathVariable("classeAparametro") String classeAparametro,
        @PathVariable("idClasseB") Long idClasseB, @Valid @RequestBody Classeb classeB) {
    if(!classeBRepository.existsById(idClasseB)) {
        throw new NaoEncontrado("Classe B não encontrado!");
    }
    return classeBRepository.findById(idClasseB).map( idClasseBSet -> {
        classeB.setData_ingresso(classeB.getData_ingresso());
        classeB.setModalidade_acesso(classeB.getModalidade_acesso());
        return classeBRepository.save(classeB);
    }).orElseThrow(() -> new NaoEncontrado("Não existe classeB com este ID!"));
}
</code>
Answer: Não sei se entendi muito bem, mas recomendo não utilizar o controller para criar regras de negocio, minha sugestão: se pretende excluir algo, crie um endpoint para isso e realize o procedimento no service.
Se tem algo na Classe A para excluir, o endpoint da Classe A é responsável por isso, o mesmo não deve excluir dados dos objetos Classe B e Classe C.
Comment: Obrigado pela dica, vou refatorar


Push notification to millions of device + Apns + node.js

Question: My application stack is ios(front-end) and node.js(back-end). I have to send notification to devices.  In my node.js part im using apns module to send notification, its working fine......
Now i have to send Mass notification like at a time consider i have 10,000 devices to notified, the logic what im following is
I'm looping through 10,000 devices and calling apns provider.
1.Why this for loop approach
I have to store each notification details in my mongodb collection, so i followed this approach.
The problem is the notification is received by some devices and that too very late(next day).
I read the link also
https://www.raywenderlich.com/156966/push-notifications-tutorial-getting-started
saying apns will reject.
Is the above approach is correct also any way to make all notification deliverer.
Please share your ideas. Thanks in advance.
Answer: If you need to process each individual notification before/after it is sent I would instead recommend a design change from a loop and have you look at job queue instead.
With this design pattern, instead of your only step being to loop over notifications and send via APN, you push these notification into a queue/messaging system and have workers which pull from the queue and process (send via APN and write to mongo) the notifications. The nice part of this design is that as your application grows you can add on more workers to handle the increased load without rewriting your application/architecture. Once you have it built it may look something like this:

I personally use RabbitMQ for my job queue, but that decision is something you need to research on your own. For example if you don't want to manage the messaging system you could look into something like AWS Simple Queue Service.
Answer: I think looping through 10,000 devices ids and calling APNS provider is not the right way forward. The documentations strictly says here node-apn readme file to reuse apn.Provider rather than recreate it every time to achieve the best possible performance.
If you send notification using arrays of device ids rather than just a device id then you will get a response from the APNS mentioning all the details for each device.


Quick nth-child issue

Question: I've a quick :nth-child issue that I'm struggling to solve. I'm aiming to target every 3rd and 4th item in a grouping of 4 items that form a list.
For example:

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

Item 8

In this example I'd like to target all instances of
 - i've used a lot of nth-child generators to come up with an answer but nothing gets me to what I need.
Any help would be much appreciated!
Comment: @DawoodAwan I think this is just for the example and he doesn't want to use a class
Comment: are they contained within a parent div?
Comment: $('.different') doesn't this work?
Comment: Can you post the previous solutions you've tried so far?
Comment: I was using the classes to highlight the problem, but thanks anyway.
Answer: Use div:nth-child(4n-1), div:nth-child(4n). The logic is simple — you want to select items in groups of four, so 4n would be the common denominator. Since you want to select the penultimate and the last items in the group, 4n-1 and 4n respectively would do the job.
As follow is a simple diagram illustrating my point:
#1
#2
#3  <- 4th item - 1
#4  <- 4th item
#5
#6
#7  <- 4th item -1
#8  <- 4th item

div:nth-child(4n-1), div:nth-child(4n) {
  background-color: #eee;
}

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

Item 8

Comment: @Alex Char: See Thomas's comment on the question.
Comment: This question comes up fairly often, but what doesn't come up fairly often is an explanation of how it works.
Comment: @BoltClock I hope my explanation is sufficient? If not, feel free to supplement it :)
Comment: @AlexChar: I was using classes to highlight the problem.

Quick nth-child issue


Using AutoFixture to create MockFileSystem and its corresponding types

Question: I'm using AutoFixture to test some code that uses System.IO.Abstractions.IFileSystem.
I also have some code that uses IFileInfo and IDirectoryInfo. When I create my AutoFixture Fixture object, I need to accomplish a couple of things:

I need all IFileSystem parameters to be automatically given a frozen (singleton) instance of MockFileSystem.
All IFileInfo instances need to originate from MockFileSystem via MockFileSystem.FileInfo.FromFileName() somehow. If there's a way to take an existing IFileInfo that AutoFixture created and replace it, I can use the FullName property to pass to MockFileSystem so that it's connected to that instance.
Similar to the previous point, I need IDirectoryInfo instances to originate from MockFileSystem.

I personally find the AutoFixture customization system very confusing and lacks proper documentation. There's pieces of information out there but I'm not sure how to accomplish my goals.
Answer: As I struggled through solving my own problem, I eventually came up with this class:
public class MockFileSystemSpecimenBuilder : ICustomization
{
    public void Customize(IFixture fixture)
    {
        var fs = new MockFileSystem();
        fixture.Inject(fs);

        fixture.Customize<IFileInfo>(x => x.FromFactory(() =>
        {
            var name = $"MockFile-{fixture.Create<string>()}";
            return fs.CurrentDirectory().File(name);
        }));

        fixture.Customize<IDirectoryInfo>(x => x.FromFactory(() =>
        {
            var name = $"MockDirectory-{fixture.Create<string>()}";
            return fs.CurrentDirectory().SubDirectory(name);
        }));
    }
}

I add it to my Fixture class after its created in my factory method:
public static Fixture Create()
{
    var fixture = new Fixture
    {
        OmitAutoProperties = true
    };

    fixture
        .Customize(new AutoNSubstituteCustomization {ConfigureMembers = true})
        .Customize(new MockFileSystemSpecimenBuilder());

    return fixture;
}

One thing I really enjoy about this solution is that I do not need to explicitly Freeze() the IFileSystem type anymore. The mock filesystem is intrinsic to this fixture so I always get correct file system behavior. This ends up being less error prone and more convenient.
Answer: I think most of the problems you've described are not really AutoFixture issues, but rather design issues. AutoFixture can help build stuff for you but it doesn't generate behavior, those tasks are usually delegated to the mocking frameworks.
In your case, since you already use the fakes provided by System.IO.Abstractions, all you need to do is instruct AutoFixture to relay the requests for IFileSystem to MockFileSystem.
fixture.Customizations.Add(new TypeRelay(typeof(IFileSystem), typeof(MockFileSystem)));

Now when you want to resolve the same IFileSystem instance you can instead freeze the MockFileSystem instance.
Instructing MockFileSystem to return expected data is a bit trickier but not impossible. Since MockFileSystem accepts file data information during construction only, it means that you'll need to freeze/inject/register the file data dictionary before you create your first MockFileSystem instance.
Your options mainly are to either inject the data fixture.Inject<IDictionary<string, MockFileData>>(myFileData) or register a delegate that can generate your data fixture.Register<IDictionary<string, MockFileData>>(() => GenerateFileData()).
There is the more advanced option to make a custom specimen builder to generate the paths and the file data, but that would require for you to provide the algorithm that would generate this test data.


Where should I implement specific queries in mvc architecture(spring + hibernate)?

Question: I'm implementing an enterprise application using spring mvc and hibernate. The application architecture is composed by the following layers:

View
Controller
Service
Domain
Peristence

The data access is implemented with classic dao pattern and allows CRUD operations + generic queries. Where should I implement specific queries? I guess two solutions:

implement specific queries in the service layer by creating hibernate criteria and passing it to the persistence layer
implement specific queries in DAO classes by create a function for each query and call the functions in the service layer.

What should I do? Are there better solutions?
Comment: Creating complex criteria is typically the sort of persistence-related work that you don't want in your service, but in your DAOs. If the DAOs only contain basic methods that the Hibernate session already has, then they aren't really useful.
Comment: I haven't complex criteria but a lot of simple criteria. This means creating a lot of very simple and fine grain functions in DAO classes so I'm wondering if this is the correct approach.
Comment: It is the right approach. That way, your service can concentrate on what it's for: business logic. And you can test it by mocking the DAOs.
Answer: Service layer is a business layer and there must not be used any persistence provider related interfaces and classes (i.e. Hibernate Criteria API). That's a good practice - you can change your persistence provider without touching any line of code in your service layer.
Don't hesitate to go with your second option (implement specific queries in DAO classes).
Alternatively you can use kind of business dynamic query pattern which is mix of the query and builder pattern (like Criteria API). Here is the simple example:
interface CarQuery {
    CarQuery model(String model);
    CarQuery color(String color);
    List<CarQuery> execute();
}
....
List<Car> cars = carDao.getQuery().model("Jeep").color("green").execute();
....

You can implement HibernateCarQuery that can use Hibernate Criteria API internally. This surely requires some additional efforts but will not pollute your daos with specific methods if you're planning to use them extensively and need ability to build queries dynamically (typical case is multiple filters on UI pages).
Comment: I would go further and say that your service layer should not have to change if you switched ORM providers (i.e. went from Hibernate to something else) -- only the DAO layer should need to change.
Answer: This:

implement specific queries in DAO classes by create a function for each query and call the functions in the service layer.

DAO is shorthand for "Data Access Object" and one of the maim reasons to use the DAO pattern is to comply with the "Single Responsibility Principle". If you have some other classes in your code, other than the DAO classes, which handle Data Access, then you are violating the SRP and it becomes a more or less moot point to use the DAO pattern at all.
More importantly, for maintainability purposes, if other developers are going to maintain your code later on, and they see a bunch of DAO classes lying about, they are not going to expect data access to be happening anywhere else.
Your service layer isn't even supposed to have a dependency on Hibernate. Why would you have that? Your service layer should only depend on your domain layer, which in turn depends on your persistence layer, which is where your Hibernate dependency should be,
Answer: From layered architecture perspective, the sql queries should not be used/declared outside DAO layer. You might have noticed that spring opted to use transaction on service layer since transaction is not the responsibility of DAO and you can also call it as resource leakage/cross cutting concern. Declaring /using sql queries apart from Dao will be considered to be resource leakage/cross cutting concern.
I would like to add something since you touched hibernate and criteria API. When you are using criteria then why should you need separate query? Criteria API is an elegant way to achieve persistence because it is perfect OO approach.
Comment: I mentioned about resource leak.Kindly understand the answer and come up with questions.. Thanks!
Comment: When I talk about query I mean create Critera for query the DB. I don't use separate query. The point is: should I expose hibernate criteria in the service layer? This avoid me from write a lot of simple functions in DAOs but violate the layers separation.
Comment: If you are not bothered about resource leakage/cross cutting concern then go ahead with your approach. but correct way is to implement it in DAO
Answer: It is better to have all the queries in the DAO class methods and run them. If you are working on HQL, have all the queries written in the NamedQuery and access it by name at the run time.
Answer: I use to create a GenericDAO class and an IDao interface and all the other DAOs - let's say a
ClientDAO - extends the Generic.
That being said, I create an interface that extends IDao (which have the basics methods like findAll(), delete(), save())
 E.g.

IClient
public interface IClient extends IDao {
   public void findWithWhere(String where);
}

ClientDaoImpl
public class ClientDaoImpl extends GenericDao<Client> implements IClient {

  public void findWithWhere(String where) {
   //do custom stuff here
  }
}

And then have a ClientDaoImpl object in the Controller (or Service) Layer
That may look a little tiring to do, but, for me, it keeps all the layers well divided and when I have to change a method (maybe receive a new parameter), I simple change the interface to know where I have to implement.
Comment: My solution is more or less the same as your. The point is that I have a lot of different fine grain queries to implement so I have to implement a lot of simple funtions in DAOs (or a complex parametrized one) and I don't know if this is the right way but it seems it is
Comment: You could have QUERIES in your Service Layer, but, in order to keep the pattern, I rather create them in the DAO Layer


Proving Itô's Isometry

Question: Im currently trying to understand stochastic integration.
Reading through some lecture notes I encountered the following proof of Itô's Isometry for simple functions. Here $M_s$ is a $L^2$ integrable martingale starting at 0 and
$$ H_t = \sum_{n = 0}^\infty K_n \mathbb{1}_{(t_{n},t_{n+1}]} $$
is a simple predictable process.
Then
\begin{align}
\mathbb{E}\left[\left( \int_0^\infty H_s dM_s \right)^2 \right] &\overset{1}{=}  \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 (M_{t_{n+1}}-M_{t_n})^2 \right] \\
&\overset{2}{=}\mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[(M_{t_{n+1}}-M_{t_n})^2 | \mathcal{F}_{t_n}\right] \right] \\
&\overset{3}{=} \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[M_{t_{n+1}}^2-M_{t_n}^2 | \mathcal{F}_{t_n}\right] \right] \\
&\overset{4}{=} \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[[M]_{t_{n+1}}-[M]_{t_n} | \mathcal{F}_{t_n}\right] \right] \\
&\overset{5}{=} \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2([M]_{t_{n+1}}-[M]_{t_n}) \right] \overset{6}{=} \mathbb{E}\left[ \int_0^\infty H_s^2 d[M]_s \right]
\end{align}
I have trouble udnerstanding some parts of those equations. So lets go through each of them:

This is just the definition of the stochastic integral for simple functions.

This is the tower property for the conditional expectation.

I dont understand what is happending here.
My calculations based on the comment by Bart:
\begin{align}
\mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[(M_{t_{n+1}}-M_{t_n})^2 | \mathcal{F}_{t_n}\right] \right] &= \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[M_{t_{n+1}}^2-2M_{t_n}M_{t_{n+1}}-M_{t_n}^2 | \mathcal{F}_{t_n}\right] \right]   \\
&= \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \left( \mathbb{E}\left[M_{t_{n+1}}^2-M_{t_n}^2 | \mathcal{F}_{t_n}\right] -2\mathbb{E}\left[M_{t_n}M_{t_{n+1}}| \mathcal{F}_{t_n}\right]  \right) \right]   \\
&= \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \left( \mathbb{E}\left[M_{t_{n+1}}^2-M_{t_n}^2 | \mathcal{F}_{t_n}\right] -2M_{t_n}^2  \right) \right]   \\
&\overset{3}{=} \mathbb{E}\left[ \sum_{n =0}^\infty K_n^2 \mathbb{E}\left[M_{t_{n+1}}^2-M_{t_n}^2 | \mathcal{F}_{t_n}\right] \right] \\
\end{align}
Here I still don't understand how to derive the last line.

I dont understand what is happending here. It is normally argued that $M_t^2-[M]_t$ is a martingale but i dont understand what this has to do with the proof.

Is this the tower property? I lack intuition why this is true.

$[M]_s$ is nondecreasing hence this integral is understood for example in the Riemann-Stieltjes sense.

Can you please help me understand this proof?
Comment: Wrt equation 3: write out the square of the difference between the brackets, then use the conditional expectation wrt Ftn and the martingale property.
Comment: It shouldn't. But using said properties one can show it equals $M_{t_n}^2$, which should then help you get the desired result.
Comment: As user6247850 pointed out there is a slight mistake with one minus sign too much in the expansion of $(M_{t_{n+1}}-M_{t_n})^2$. The rest you already have to obtain equality 3. But you had probably already come to that conclusion.
Comment: Thank you. Why should $\mathbb{E}\left[M_{t_n}M_{t_{n+1}}| \mathcal{F}_{t_n}\right] $ be equal to zero?
Comment: Could you please elaborate this? I'm just too stupid to see how this would help.
Answer: For 3:
There's a slight mistake when you expand out $(M_{t_{n+1}}-M_{t_n})^2$.  It should be $M_{t_{n+1}}^2 - 2 M_{t_n} M_{t_{n+1}} {+} M_{t_n}^2$.  The rest of what you have is good, but now you'll end up with $M_{t_n}^2 - 2 M_{t_n}^2 = -M_{t_n}^2$ in the last line.
For 4:
Let $N_t := M_t^2 - [M]_t$, which is a martingale as you mentioned.  Given $t > s$, we have
\begin{align*}
\mathbb{E}[M_{t}^2 - M_{s}^2|\mathcal F_s] &= \mathbb{E}[M_t^2-[M]_t - (M_s^2 - [M]_s) + [M]_t - [M]_s | \mathcal F_s] \\
&= \mathbb{E}[N_t - N_s+ [M]_t - [M]_s | \mathcal F_s] \\
&= \mathbb{E}[[M]_t - [M]_s | \mathcal F_s].
\end{align*}
Applying this with $t = t_{n+1}$ and $s = t_n$ gives the result.
For 5:
Yes, this is the tower property.  We have
\begin{align*}
\mathbb{E}\left[\sum K_n^2 \mathbb{E}[[M]_{t_{n+1}} - [M]_{t_n} | \mathcal F_{t_n}] \right] &=\mathbb{E}\left[\sum \mathbb{E}[K_n^2([M]_{t_{n+1}} - [M]_{t_n} )| \mathcal F_{t_n}] \right] \\
&= \sum \mathbb{E}[\mathbb{E}[K_n^2([M]_{t_{n+1}} - [M]_{t_n} )| \mathcal F_{t_n}]]\\
&= \sum \mathbb{E}[K_n^2([M]_{t_{n+1}} - [M]_{t_n} )] \\
&= \mathbb{E}\left[ \sum K_n^2([M]_{t_{n+1}} - [M]_{t_n} )\right]
\end{align*}
Comment: @Rooibos Sorry, I didn't see that you hadn't fully resolved line 3.  I added an explanation of that.
Comment: Thanks a lot for the detailed answer. This helped me a lot.

Proving Itô's Isometry


Dualizing the definition of the kernel relation in $\mathbf{Set}$ as a pullback

Question: Given a set function $f : A \rightarrow B$, my book notes that the kernel relation $R_f = \{ (a_1, a_2) | f(a_1) = f(a_2) \}$ along with the corresponding projections is a pullback:
$$\require{AMScd}
\begin{CD}
R_f @>>> A\\
@VVV @VV{f}V \\
A @>>{f}> B
\end{CD}$$
But what about its dual? Let's consider
$$\begin{CD}
A @>{f}>> B\\
@V{f}VV @VVV\\
B @>>> Z
\end{CD}$$
Here, by construction of pushouts in $\mathbf{Set}$, $Z$ is $B \amalg B / \sim$, where $\sim$ is the finest relation such that $\forall a \in A : f(a) \sim f(a)$. But the trivial relation ($\forall b \in B : b \sim b$) will do, so is $Z$ just $B \amalg B$? Did I miss anything?
Answer: That is not correct. Let's set things up with some clearer notation. If we let $i : B \rightarrow B \amalg B$, be the embedding into the first copy of $B$, and similarly let $j$ be the embedding into the second copy, then the relation $\sim$ is the finest satisfying $\forall a \in A : i(f(a)) = j(f(a))$.
The closest analogue of the "trivial relation" is $\forall b \in B: i(b) \sim j(b)$. Even this is not correct, because there is a finer relation: we only have $i(b) \sim j(b)$ for $b \in f(A)$.
Thus the pushout is $B \amalg B$ with the two copies of $f(A) \subseteq B$ identified. Two extreme cases are: $A = \emptyset$, in which case the pushout is just $B \amalg B$; and $f$ is surjective, in which case the pushout is $B$.


Using R's prop.test function to compare subsets of a population

Question: I'm new to CV, so please excuse any faux pas. I've collected sample data from a population and would like to discern whether or not the experimental group outperforms the control group. Let's say I have 10,000 observations within the experimental group and saw 4500 successes. Let's also say that I observed a 47% success rate in my control group (so if my control group is 5% of the total sample size, that would be 235 successes out of 500 trials).
Now, am I right in my usage and interpretation of the following?:
prop.test(4500,10000,.47,alt="less")

data:  4500 out of 10000, null probability 0.47
X-squared = 15.9776, df = 1, p-value = 3.205e-05
alternative hypothesis: true p is less than 0.47
95 percent confidence interval:
 0.0000000 0.4582455
sample estimates:
   p
0.45

Since the p-value is far less than .05, I can safely reject the null hypothesis that that the experimental group will perform as well or better than the control group. Also, I know that 95% of the time, the probability of success for the experimental group will fall between 0% and ~45.8%.
Also, is the prop.test function a good indicator of sufficient sample size? If not, what is?
Thanks!
Comment: Consider these questions: (1) If someone else were to independently sample $500$ members of the control population, would they likely have observed exactly $235$ successes? (Probably not, so how much do you suppose their number of successes would deviate from your number?) (2) Where in your code have you accounted for that sampling variability?
Comment: Really good questions. I suppose I don't really account for sampling variability, right? I drew what I thought was a fairly large random sample, but I'm not certain if I can now draw any statistically significant/accurate conclusions from it. I'd still like to know if I'm using `prop.test` correctly, but if I'm missing some important prerequisites please point me in the right direction. Thank you!
Comment: It seems your real question is less about `prop.test` and more about understanding how to test for a difference in proportions, with a hope to then use `prop.test` or something like it once you figure out what you're trying to do. The hard part isn't the '*how do I do it in prop.test*?' part.
Answer: Your interpretation is incorrect.
"Since the p-value is far less than .05, I can safely reject the null
hypothesis that that the experimental group will perform as well or
better than the control group. "
No. What you did was this: you "tested" if the p from the experimental group is greater or equal 0.47. But you don't know if the p of the control group is actually 0.47. So you could say that the evidence favors the hypothesis that the proportion in the experimental group is less than 0.47, but you could not say that the evidence shows that it is less than the proportion of the control group, which is uncertain.
Actually, just for illustration, assuming the values stated on the question, you wouldn't reject the hypothesis that the proportions are different with prop.test:
success <- c(4500, 235) ## successes in experimental and control
n <- c(10000, 500) ## total observations in experimental and control
prop.test(success, n)

2-sample test for equality of proportions with continuity correction

data:  success out of n
X-squared = 0.6907, df = 1, p-value = 0.4059
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.06587065  0.02587065
sample estimates:
prop 1 prop 2
  0.45   0.47

Notice that once you account for the uncertainty in the other proportion the p-value raises to 0.4.
As for the second sentence:
"Also, I know that 95% of the time, the probability of success for the experimental group will fall between 0% and ~45.8%."
This is a common misconception. What you can say is that the procedure you used constructs confidence intervals that would cover the true value of p 95% of the time. But this is not the same as claiming that 95% of the time you will estimate p on the range of the confidence interval, neither that the true p is inside the confidence interval with 95% probability. This other question may help on this topic.
Comment: Thanks for your explanation. So how do I discern the `p` for the control group so I can use it in my analysis? Or perhaps a better question is: Is the `prop.test` function really what I want here, or is there a simpler method that I should start with?
Comment: We need more information about the experiment to make a better decision, but it seems that `prop.test` would be an adequate first approximation to the problem. You just have to use the information of the successes and sample size of both the control and experiment group, as illustrated on the answer above.


How to use different domains for multilingual sites and keep users logged in?

Question: I have a site that is in French and English. I am able to successfully setup so that users can switch back and forth between the two domains to view either language, however I cannot seem to keep users logged in. When they are logged into the english domain and then switch to the french domain they are required to login again. However if they switch back to the english domain they are still logged in.
As an example we'll use www.english-domain.com as the base (default) domain and www.french-domain.com as the secondary domain
In settings.php I have modified the following, however it doesn't seem to be keeping them logged in. Am I missing something?
/**
 * Drupal automatically generates a unique session cookie name for each site
 * based on its full domain name. If you have multiple domains pointing at the
 * same Drupal site, you can either redirect them all to a single domain (see
 * comment in .htaccess), or uncomment the line below and specify their shared
 * base domain. Doing so assures that users remain logged in as they cross
 * between your various domains.
 */
$cookie_domain = 'english-domain.com';

Note: That I have also flushed both site and browser cache as well as deleted any cookies left on my machine.
Update:
Here are my language settings:
Comment: What do you mean by, "keeping the user logged in"? Are users automatically logged out when they switch between the two domains?
Comment: When they are logged into the english domain and then switch to the french domain they are required to login again. However if they switch back to the english domain they are still logged in.
Comment: Add a dot before site name(`'.english-domain.com'`) and check again
Comment: Unfortunately that didn't work either. I have updated my answer with a bit more info that may or may not be pertinent.
Comment: I also came across this in my research but not sure if it applies to my situation and how to properly add it to settings.php http://drupal.org/node/1348784
Answer: A cookie can only be read from the same domain that set it. This is a fundamental security feature in browsers to prevent other sites from potentially stealing login sessions, and there is nothing simple you can do about it.
With that said, the comment in the settings.php file seems to contradict my statement, and I'm honestly not really sure how to intrepret that. Perhaps it's only meant to be relevant to sub-domains, which are allowed to share cookies, but if so the comment should probably be re-phrased.
There are a number of ways to work around this, where the easiest is to use language prefix sub-domains instead of separate domains.
Baring that, I believe the "best" one is to use a Single sign-on system. There's a comparision of different SSO modules here. This could be a Drupal based solution, which is the way that is most likely to "nicely" integrate with your current Drupal sites, but that's not really necessary.
The Bakery module which is used by *.drupal.org itself is relevant to mention, as well as this related DA answer How drupal.org single sign on works
More info about cross-domain cookies in a Stack overflow question.
Comment: I can understand using SSO if these were different sites, but these are just two domains pointing to the same drupal installation and Drupal is handling the domain switching through the language settings.
Comment: "Just two domains" doesn't help. As I said, it's a _browser_ feature, and the browser can't know, nor does it care, that you run both domains off the same code and they should entrust each other with their cookies. Why that isn't a feature I don't know, but you can probably find out if you read [RFC2109](https://tools.ietf.org/html/rfc2109) :)
Comment: Gotcha. It all makes sense now. I have done some further research and there is no way to use a native Drupal SSO module to achieve what I need. If they were two different installations, I could use SSO server/client because one can be setup as the master and the other as a slave. But I cannot set one site to be both a master and a slave. As in the case of using two domains on one install for multilingual. Going to chalk this up to "can't be done, easily" and moving on. :) Thanks for your help.
Answer: Cross domain cookies are not allowed; this means that english-domain.com cannot set a cookie for french-domain.com.
Cookies can be shared between sub-domains, though: english.example.com can set a cookie that is seen from french.example.com, and vice versa.
To allow users to be logged-in on sub-domains of the same domain, you can use the Bakery Single Sign-On System module.
As far as I know, there aren't Drupal SSO modules for different domains. There are web sites that allows SSO between different domains, such are the ones that are part of Stack Exchange network, where users having an account on Stack Overflow can be automatically logged in on Drupal Answers, or Super User, if they have an account on those sites.
The code to write is more complicated, in this case. What Stack Exchange sites do is the following:
Initial login

Login proceeds as usual
At the end, the user is given a global login token (attached as a cookie)
The next page the user visits takes that token, and (via an IFrame) shovels it to StackAuth
StackAuth verifies the token, and attaches a session key via localStorage

Visiting a different site

JavaScript is used to hit StackAuth via an
StackAuth checks localStorage for a session, and sends it to itself if it is
StackAuth validates the session and passes a login token to the Stack Exchange site via postMessage when the session is valid
The  Stack Exchange site validates the token, looks up the user, and logs the user in

Using cookies, and sub-domains is much simpler. The code used by Stack Exchange requires a browser supporting localStorage, but it works for every domain, not just sub-domains.


Distribution of sum of negative binomial parametrized with varying mean and constant aggregation parameter

Question: From textbook, I know that:
If each $X_i$ is distributed as negative binomial $(r_i,p)$ then $\sum X_i$  is distributed as negative binomial $(\sum r_i ,p)$.
In R: dnbinom(x, size=r, prob=p)
However, I use a parametrization of negative binomial using mean and aggregation parameter [in R: dnbinom(x, size, mu) ]
What can I say from the sum of $X_i$ with such parametrization ?
[edit]
I come back to this question because finally the previous discussion does not solve my problem.
From textboook, I know that NB(size1, prob)+NB(size2, prob)=NB(size1+size2, prob).
But I use the parametrization: NB(size, mu) and then I need to know:
NB(size, mu1)+NB(size, mu2) with size being constant.
Note that mu=size*(1-prob)/prob and then prob=size/(size+mu)
Then prob is not a constant because it varies with mu and I cannot apply the solution with constant prob.
Based on simulations, it seems that :
NB(size, mu1)+NB(size, mu2) = NB(size?, mu1+mu2)
Have you an idea about the solution to estimate the value of size?
Exemple of simulation in R language
size <- 1
mu1 <- 10
V1 <- rnbinom(10000, size=size, mu=mu1)
cat("prob1=", size/(size+mu1))
mu2 <- 20
V2 <- rnbinom(10000, size=size, mu=mu2)
cat("prob2=", size/(size+mu2))

hist(V1)
hist(V2)
hist(V1+V2)

library(MASS)

fitdistr(V1, "negative binomial")
fitdistr(V2, "negative binomial")

fitdistr(V1+V2, "negative binomial")

Marc
Comment: how is $\mu$ related to $r$ and $p$ in this perameterization?
Comment: In this parametrization, p = r/(r+µ) so µ=(r/p)-r=r (1/p - 1)=r (1-p)/p
Comment: @MarcG: I think Glen_b is trying to help you find the solution by yourself: if you know how to switch between both parameterisations then the solution is contained in the question.
Comment: Xi'an is correct -- now you've already done it all yourself apart from the final step of writing the final line (the value for the mu-parameter of the sum). If you know how to switch between parameterizations (as you just did), and you know how to write the distribution of $\sum X_i$ in one form, you can write it in the other form. $\sum r_i$ is a single number, (use $R=\sum r_i$ if it helps); write what the $\mu$ parameter must be in terms of $p$ and $\sum r_i$ (or $p$ and $R$ if it's easier that way; just convert back at the end).
Comment: Please add the `self-study` tag, read its [tag-wiki](http://stats.stackexchange.com/tags/self-study/info) and modify your question to follow the guidelines on asking such questions. (In particular, you'll need to clearly identify what you've done to solve the problem yourself, and indicate the specific help you need at the point you struck difficulty.)
Comment: @MarcG You appear to have created two separate accounts! [Please see this help page](http://stats.stackexchange.com/help/merging-accounts) so you can merge them again. This will mean you can instantly edit your own posts rather than wait for your edits to get approval.
Answer: The solution has been published here:
Furman, E., 2007. On the convolution of the negative binomial random variables. Statistics & Probability Letters 77, 169-172.
The author shows that the sum of negative binomial random variables NB(ri,pi) follows a negative binomial distribution and he shows how to estimate the new parameters.


Docker ENTRYPOINT shell form with parameters

Question: When I have a Docker image with the following line (a Spring Boot microservice):
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]

I can start the container using e.g.:
docker run --rm my_image:1.0.0 --spring.profiles.active=local

and it works, the parameter --spring.profiles.active=local is used. However, when the shell form of ENTRYPOINT is used:
ENTRYPOINT java org.springframework.boot.loader.JarLauncher

this doesn't work any more, the parameters are ignored. I believe the parameters are passed to the /bin/sh -c that is what is used by the shell form.
So how do I pass arguments to the app I want to start when using the shell form?
Comment: Just use the first form (json). Its almost always the better form to use.
Comment: Add `"$@"` if you want your shell to look at extra parameters, possibly `"$0" "$@"` depending on implementation details (particularly, whether Docker passes a stub `$0` value in when it's automatically invoking a shell -- if it doesn't you'll see the first extra argument being ignored when you use only `"$@"` but subsequent ones still working). `ENTRYPOINT java org.springframework.boot.loader.JarLauncher "$@"`, then.
Comment: ...but _really_, TheFool is right. Don't use a shell when you have no good reason for one. (Using this form also means you can't change your base image to be one that doesn't provide `/bin/sh` at all)
Comment: @CharlesDuffy You're correct, `ENTRYPOINT ls "$@"` gets it working, although as you pointed out an extra argument is required
Comment: @charlesduffy will `"$@"` work? The docs state: `The shell form prevents any CMD or run command line arguments from being used`
Comment: @Foo, worth testing, then. There's a good chance that the folks writing the docs (or even the implementation) didn't know that `"$@"` _could_ be used and that it works just fine, but I won't claim to know with certainty either way; depends on whether Docker still passes content through when the shell form is used.
Comment: (if Docker's layering design were done right, though, it would work, because the implementation wouldn't know/care which configuration format was used; properly, `ENTRYPOINT foo` should turn into the exact same thing as `ENTRYPOINT ["sh", "-c", "foo"]`)
Comment: @CharlesDuffy, this has nothing to do with done right or wrong. It actually makes sense the way it is. This is because how calls to exec.Command work. It expects a command and an array of args. If we pass an array in the dockerfile the first item is the command and the rest is the args. But if you give a single string like this, the only way to run it is to wrap it in shell, if you want to avoid ambiguity.  https://pkg.go.dev/os/exec#CommandContext
Comment: you have used it literally in your previous comment quote: *if Docker's layering design were done right, though*
Comment: @TheFool, correct, and I told you above what specific aspect of good software design I was referring to when I said "right". I'm not objecting to Docker's use of a shell; it's reasonable given the design constraints. In the hypothetical world where my suggestion didn't work, I would be objecting to Docker changing how it passes arguments through as a side effect of the user's choice of configuration syntax, because when it _does_ just append arguments to argv -- having identical runtime behavior no matter which config format is used -- the shell syntax suggested works properly.
Comment: @CharlesDuffy, well. And I told you that I think it makes sense. Would love to see your implementation. If we talk about good design, we should rather mention that using a custom file format, the Dockerfile, was a poor decision. They could have used any number of existing config languages and would not need to maintain a parser and all that jazz.
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/250981/discussion-between-charles-duffy-and-the-fool).
Answer: ENTRYPOINT string_here

...has Docker run:
["sh", "-c", "string_here"]

The problem with this is that when you add more arguments, they're added as new elements on the argument vector, as in:
["sh", "-c", "string_here", "arg1", "arg2", "arg3..."]

which means they're ignored, because string_here, when invoked as a script, doesn't look at what further arguments it's given.
Thus, you can use:
ENTRYPOINT string_here "$0" "$@"

where "$@" in shell expands to "$1" "$2" "$3" ..., and $0 is the first argument following -c (which is typically the name of the script or executable, and used in error messages written by the shell itself).


Undefined reference linking static libSOIL.a on g++ linux

Question: I'm trying to build my openGL project, all worked well until I tried to use the SOIL library (http://www.lonesock.net/soil.html) for loading textures from images. This library comes with a SOIL.h and a libSOIL.a files.
That's what I get when doing make:
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/main.o src/main.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/file_ply_stl.o src/file_ply_stl.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/Proyecto.o src/Proyecto.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/Clock.o src/Clock.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/utils.o src/utils.cpp

...

g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/primitives/cilindro.o src/primitives/cilindro.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g   -c -o src/primitives/cono.o src/primitives/cono.cpp
g++ -I./src -Wl,-rpath,./lib -O3 -g -o app -L./lib src/main.o src/file_ply_stl.o src/Proyecto.o src/Clock.o src/utils.o src/Window.o src/Light.o src/Material.o src/PlyModel.o src/Placeable.o src/Moveable.o src/Model.o src/Texture.o src/Geometry.o src/TriGeometry.o src/TriIndGeometry.o src/Entity.o src/Camera.o src/Bounded.o src/Animatable.o src/Scene.o src/Graphics.o src/Vert.o src/Path.o src/Ant.o src/Tank.o src/Chess.o src/primitives/tetra.o src/primitives/cubo.o src/primitives/cilindro.o src/primitives/cono.o -lsfml-window -lsfml-system -lX11 -lXext -lXmu -lXi -lXt -lSM -lICE -lGLU -lGL -lglut -lSOIL
src/Texture.o: In function `Texture::ini(char const*)':
/home/roger/t/ig/prac/proyecto/src/Texture.cpp:9: undefined reference to `SOIL_load_OGL_texture'
collect2: ld devolvió el estado de salida 1
make: *** [app] Error 1

And that's my Makefile:
TARGET = app

MODULES = src/main.o src/file_ply_stl.o src/Proyecto.o src/Clock.o src/utils.o
ENGINE = src/Window.o src/Light.o src/Material.o src/PlyModel.o src/Placeable.o src/Moveable.o src/Model.o src/Texture.o src/Geometry.o src/TriGeometry.o src/TriIndGeometry.o src/Entity.o src/Camera.o src/Bounded.o src/Animatable.o src/Scene.o src/Graphics.o src/Vert.o
GAME = src/Path.o src/Ant.o src/Tank.o
PRIMITIVES = src/Chess.o src/primitives/tetra.o src/primitives/cubo.o src/primitives/cilindro.o src/primitives/cono.o

CXX = g++
LIBDIR = ./lib
INCDIR = ./src

LIBS = -lsfml-window -lsfml-system -lX11 -lXext -lXmu -lXi -lXt -lSM -lICE -lGLU -lGL -lglut -lSOIL

LDFLAGS = -L$(LIBDIR)
CXXFLAGS = -I$(INCDIR) -Wl,-rpath,$(LIBDIR) -O3 -g

$(TARGET): $(MODULES) $(ENGINE) $(GAME) $(PRIMITIVES)
    $(CXX) $(CXXFLAGS) -o $(TARGET) $(LDFLAGS) $(MODULES) $(ENGINE) $(GAME) $(PRIMITIVES) $(LIBS)

clean:
    rm -f $(MODULES) $(ENGINE) $(GAME) $(PRIMITIVES) $(TARGET)

And that's my file hierachy:
.:
lib Makefile src

./lib:
libsfml-audio.so       libsfml-graphics.so.2.0  libsfml-system.so.2    libSOIL.a
libsfml-audio.so.2     libsfml-network.so       libsfml-system.so.2.0  libSOIL.a.1
libsfml-audio.so.2.0   libsfml-network.so.2     libsfml-window.so      libSOIL.a.1.0
libsfml-graphics.so    libsfml-network.so.2.0   libsfml-window.so.2
libsfml-graphics.so.2  libsfml-system.so        libsfml-window.so.2.0

./src:
Animatable.cpp  Clock.cpp         Light.cpp     Moveable.h     random.h         TriGeometry.h
Animatable.h    Clock.h           Light.h       Path.cpp       Scene.cpp        TriIndGeometry.cpp
Ant.cpp         Entity.cpp        main.cpp      Path.h         Scene.h          TriIndGeometry.h
Ant.h           Entity.h          main.h        Placeable.cpp  SFML             utils.cpp
Bounded.cpp     file_ply_stl.cpp  Material.cpp  Placeable.h    SOIL             utils.h
Bounded.h       file_ply_stl.h    Material.h    PlyModel.cpp   Tank.cpp         Vert.cpp
Camera.cpp      Geometry.cpp      matrix.h      PlyModel.h     Tank.h           vertex.h
Camera.h        Geometry.h        Model.cpp     primitives     Texture.cpp      Vert.h
Chess.cpp       Graphics.cpp      Model.h       Proyecto.cpp   Texture.h        Window.cpp
Chess.h         Graphics.h        Moveable.cpp  Proyecto.h     TriGeometry.cpp  Window.h

./src/primitives:
...

./src/SFML:
...

./src/SFML/Audio:
...

./src/SFML/Graphics:
...

./src/SFML/Network:
...

./src/SFML/System:
...

./src/SFML/Window:
...

./src/SOIL:
SOIL.h

I don't know what's going on, and why the compiler doesn't find the library file.
I've got #include "SOIL/SOIL.h" in Texture.h and the call to SOIL_load_OGL_texture in Texture.cpp:9, that's the name on the function as defined in SOIL.h. ¿What I am doing bad?
EDIT
I was using the library provided, I compiled it myself and it worked.
Comment: libSOIL.a is a symbolic link to libSOIL.a.1 //

libSOIL.a.1 is a symbolic link to libSOIL.a.1.0 //

libSOIL.a.1.0 is the static library //

I tried leaving only libSOIL.a as the name of the library (no links), and the same happens...
Answer: I must write down this answer when I compile the SOIL library even the question is asked 4 year ago, because it is so hard to write the CMakeLists.txt myself before I find the CMakeLists.txt.
the CMakeLists.txt
cmake_minimum_required (VERSION 2.6)
project (SOIL)

set (SOIL_BUILD_TESTS false CACHE BOOL "Build tests")

if(ANDROID)
   # Android requires GL ES 2.0 package automatically
  set(LIBRARIES GLESv2)
else()
  find_package (OpenGL REQUIRED)
  include_directories (${OPENGL_INCLUDE_DIR})
  set(LIBRARIES ${OPENGL_LIBRARIES})
endif()

file (GLOB SOIL_SOURCES src/*.c)
file (GLOB SOIL_HEADERS src/*.h)

add_library (SOIL ${SOIL_SOURCES} ${SOIL_HEADERS})
target_link_libraries (SOIL ${LIBRARIES})

install (TARGETS SOIL
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib)
install (FILES ${SOIL_HEADERS} DESTINATION include/SOIL)

if (SOIL_BUILD_TESTS)
    add_executable (test_SOIL src/test_SOIL.cpp)
    target_link_libraries (test_SOIL SOIL)
    install (TARGETS SOIL test_SOIL RUNTIME DESTINATION bin)
endif ()


I have some lyrics with a melody and a chord progression. How do I go about adding music to it?

Question: I wrote some lyrics and added a melody and chord progression. I would like to add more music to it. From what I know, I would have to add a bass track and add a beginning riff, a solo, drums etc..
I only play the guitar and know how to use TuxGuitar(Linux equivalent for GuiatarPro) and I am thinking of using that to add tracks for the instruments that I don't play.
How should I start? In which order would professional musicians go about doing this? Should I start with the bass or drums? Any other tips that will prove helpful to me when adding music?
Comment: Most start with the drums. I know one of my friends actually start with the bass but I guess it depends on you.
Answer: I always start with the drums.  The drum track will serve as a metronome (or click track) to keep the entire arrangement at a consistent tempo and form a framework for all the other instruments that follow.
Since you have a melody in mind, search for a pre-arranged drum pattern that best fits the groove that you want for your song.  Adjust the tempo accordingly.
Next I would play the chord progression to the drum track while singing the melody in my head (or out loud if you can record your guitar independently with the tools at your disposal).
Then I would sing the vocals over the drums and the chord progression.  Singing the vocals establishes the melody of the song.
From there you can add any other instrumental parts you desire in any particular order.
I might do a bass line next and then perhaps some lead fills or solo's if you allow space in your arrangement for a guitar or keyboard or other instrumental solo.
For the intro (beginning riff) decide how long you will make it and play a chord progression during the intro or just leave space by humming an idea for the intro over the drum track for the first couple of measures (however long you want your intro to be). The drum track will let you know where the measures start and end.
Once you finish all the other instrumental parts and record the vocals, you can go back and tweak the drums, adding fills and changing the pattern for the chorus and maybe add an ending.   You can even change the drum track completely as long as you don't change the tempo.


What are advantages of finger pads vs stick pads

Question: I'm looking for a MIDI controller to input drum notes, primarily to recreate the sound of a rock drum set.
Presently I am using a (semi weighted) keyboard. Despite having quite some practice by now, I can never achieve both good dynamics and good timing at the same time. If I play with right dynamics, the timing is off. I get decent timing only if I hit the keys hard, and then I loose dynamics. I end up wasting lot's of time on editing which I would prefer to spend on practicing and recording.
I was searching for MIDI controllers with pads that can be played with fingers (some MIDI keyboards include this type of pads), or pads that can be played with drum sticks.

(photos of products are just to illustrate the type of the pads, not specific manufacturers/models)
What are the advantages or disadvantages of each of these types of MIDI controllers? Are there specific parameters to look for? Is there yet another solution to consider?
Comment: Not really a full answer, but the ones you play with sticks aren't very helpful for anyone who doesn't know how to hit things with drumsticks. That might sound easy (using drumsticks) but beyond the absolute basics, it's not. If you've never played drums with sticks before, the stick pads might be worse than a keyboard for you.
Comment: Using finger pads will be easier than stick pads for anyone who has played a lot of piano and never drums. Stick pads will be easier for people who have played a lot of drums but never piano or any other keyboard instrument. In either case, being able to adjust the response curve and sensitivity to your own style will make a big difference in the results you get.
Comment: Something you might consider before buying gear is seeing if your keyboard has different velocity settings and/or curves. Using a different one might enable you to get the combined dynamics and feel that you're looking for.
Comment: @ToddWilcox thanks for the comment. Yes, I understand that a new instrument will require some practice. Would you consider hitting pads with fingers easier than hitting them with sticks?
Answer: Two major advantages of pads played with sticks are that

You can use your stick technique that you spent hundreds or thousands of hours practicing.  This includes double strokes, triple strokes, rolls...  It's much easier to play with the same timing as you would on a real drum.  That is, if you're a drummer.

You can add the controller to your drum kit.  It's common to see modern drummers with a Roland SPD-SX off to their left, so that they have access to electronic drum sounds or any sounds, really.  Using a finger pad here would be awkward.

Image Source
Of course, if you're not a drummer, the above doesn't really apply.
Answer: I guess any answer to this is going to contain a fair amount of opinion, but here goes anyway.
Over the years I've used most types of input controller [but never a midi kit]. I also actually play drums, so I'm used to sticks; & hand percussion, so I'm used to 'slapping skins' too. I think this last technique probably contributes the most to how I prefer to input drums to a sample set.
The only times I've really used stick input was way back in the 80s when we used a suitcase Simmons for tom fills. These were semi-randomly applied over a drum machine at what felt like the appropriate moments. No-one was thinking about three-handed drumming at the time, so no-one cared these were obvious overdubs.
I wouldn't ever consider one to lay down actual drum patterns, they just don't lend themselves to that style of playing.
Then came drum machines with velocity-sensitive [finger] pads. These were great for getting down a selection of two-bar patterns with some hint towards feel, but pad layout/button size wasn't great. I wouldn't want to try laying down a full part in a single pass that way.
Later, we got to the stage where a DAW could hold huge swathes of samples, so we were no longer forced into using hardware devices with the sounds in. The obvious controller for that was a regular keyboard.
Personally, I never looked back. Of all the input methods this gave me most control. Sure, I'm quite likely to occasionally slap the key next to the one I'm aiming for, but these are a reasonably easy tidy-up afterwards & don't greatly affect the actual groove I'm putting down.
My favourite keyboard ever for this was the Yamaha KX-88, fully weighted piano keys, but in the past 20 years I've just used something with a 'synth' keyboard & just got used to it.
As mentioned in comments - see if you can change the velocity curve of your keyboard so you're not trying to under-hit to get the sounds right. You want as close as you can get to something of the right 'feel' for your playing style.
One trick I've used for years is - never try to put the kick down in the same pass as the rest of the kit. It's unnatural, you'd never play the kick with a hand on a real kit at the same time as the rest of the top of the kit, so don't try to on a keyboard. Imagine the kick, or pre-record a 2-bar cycle if it won't sit in your head without. I just imagine it on the first pass [I tap my foot where it would be] then go for a second pass to lay it down. I always record a drum part top to bottom, in as few takes as possible. I never record bits & chop them together, it kills the groove for me. That way I can pretty much get a drum part down in two passes, then go into clean-up to get rid of the squished notes.
I will do my instrument tracking against just a rough drum part, chopped from bits just to get the timing right, but then once the majority of the 'band' is down I'll replace it properly, top to bottom.
There's another aspect to this - one you get used to over time. No matter how good your hi-hat sample set, without a hat pedal you have to use tricky note assignment to be able to get between closed, half open & open without it sounding unnatural. The samples sets I use do help quite a lot on this, as the hats slightly open with velocity, but it still requires a fair bit of dexterity swapping around 2 or 3 keys to keep this fluid. Hats mapped to black keys makes this so much easier than on white keys.
One last thing, hinted at in the first paragraph. Don't try to play it like a piano. Play it like bongoes, but with one leading finger rather than the whole hand. It just feels better.
Suitcase Simmons… plexi-glass pads you could hit as hard as you like


Need help with the definition of econometric terms

Question: Can somebody please explain in a simpler way, what is meant by "mean correctly specified" and "time averages of the covariates"?
These terms are used in the context of a panel data model.
Thanks
Answer: This is not easy to answer without a reference to a source, but my take would be

The regression equation is correctly specified in the sense that all relevant predictors/regressors/covariates are included so as to be able to give a causal interpretation to the results. "Mean" may refer to the first moment of the dependent variable, so allowing for the possibility that there may be unmodelled effects in the second moments (the variance), e.g., heteroskedasticity.
With panel data, you have several observations for one unit over (typically) time. E.g., you observe several employees over several years.
Comment: This is the comment I received:
"In the panel data case, the correlated random effects approach requires you to compute the time averages of the time-varying covariates and add them. This is supposed to mimic fixed effects."

I have the data for 27 patients over the period of 5 weeks. And let's say one of the covariates is patient's weight which varies across all 5 weeks..
Can you please explain, what I am being suggested to do in that comment?
Comment: Let's say we have just two patients, you and me, observed over three weeks. Your weight is 70, 71, 72 in the three weeks, and mine is 76,77,78. Then your time average is 71, mine is 77.
Comment: Thanks for the comment. Okay, that I understood. How do I add this to the data? I mean, my data is in columns, patient1, week 1, week 2, week 3....week5, and all the corresponding data in columns, then patient 2, from week 1 to 5, and all the corresponding data in columns... how do I add time-averages?

Is it like, the same average of a particular variable repeated again and again for 5 weeks,  (written in columns)

p.s. sorry, i am not very technical with economics
Comment: that seems more like a programming question that will depend on the tool used (and in any case is off topic here)


Bounds of sum by using integral

Question: Is it correct to say?
$$\int _1^{n+1}\:\frac{1}{x^3}dx\le \sum _{k=1}^n\:\frac{1}{k^3}\le 1+\:\int _1^n\:\frac{1}{x^3}dx$$
Therefore:
$$\sum _{k=1}^n\:\frac{1}{k^3}\le 1+\:\int _1^n\:\frac{1}{x^3}dx=\frac{3}{2}-\frac{1}{2n^2}\le \frac{3}{2}$$
And the same thing for the other side.
Answer: Yes, that's right. Since the system won't allow such a short answer, I'll add that indeed
$$\sum_{k=1}^\infty\frac1{k^3}=\zeta(3)\approx1.202\lt\frac32\;.$$

Bounds of sum by using integral


Error while applying function to each row in Python

Question: I am trying to apply a function to each line in a file and the end result is applied to only first line in the file. How can I apply below function to each row so that the print statement gives all the rows with applied function
Sample data
cvdt35_timestamp_s  gps_lat_dd  gps_lon_dd  coord
03:26.8 42.37659    -83.59011667    [42.3765899995963,-83.5901166667541]
03:27.8 42.37659667 -83.59011667    [42.3765966663758,-83.5901166667541]
03:28.8 42.37660667 -83.59012   [42.3766066670418,-83.5901200001438]
03:33.9 42.376665   -83.59017167    [42.3766650001208,-83.5901716664434]
03:35.9 42.37667167 -83.59020667    [42.3766716669003,-83.5902066667875]
03:36.9 42.37667167 -83.59021667    [42.3766716669003,-83.59021666646]
42:33.6 42.37667167 -83.59021667    [42.3766716669003,-83.59021666646]
42:41.8 42.37667167 -83.59021333    [42.3766716669003,-83.590213333567]
42:53.9 42.37682    -83.58996   [42.3768200000127,-83.5899599999189]
42:59.9 42.37697    -83.58992167    [42.3769700000683,-83.5899216666818]

import overpy
import pandas as pd
import numpy as np

df = pd.read_csv("/home/runner/sample.csv")
df.sort_values(by=['cvdt35_timestamp_s'],inplace=True)

api= overpy.Overpass()
box = 0.0005

with open("sample.csv") as f:
  for row in df.index:
    query = 'way('+str(df.gps_lat_dd.iloc[row]-box)+','+str(df.gps_lon_dd.iloc[row]-box)+','+str(df.gps_lat_dd.iloc[row]+box)+','+str(df.gps_lon_dd.iloc[row]+box)+') ["highway"]; (._;>;); out body;'
  print (query)

Result I am getting is as below, instead of list of values
way(42.37647,-83.59042167,42.37747,-83.58942167) ["highway"]; (._;>;); out body;
Comment: your print(query) is outside of the loop for which it is defined.
Answer: Either move your print statement into your loop (see below):
with open("sample.csv") as f:
  for row in df.index:
    query = 'way('+str(df.gps_lat_dd.iloc[row]-box)+','+str(df.gps_lon_dd.iloc[row]-box)+','+str(df.gps_lat_dd.iloc[row]+box)+','+str(df.gps_lon_dd.iloc[row]+box)+') ["highway"]; (._;>;); out body;'
    print (query)

or you can append to a list and then print the list:
queries = []
with open("sample.csv") as f:
  for row in df.index:
    query = 'way('+str(df.gps_lat_dd.iloc[row]-box)+','+str(df.gps_lon_dd.iloc[row]-box)+','+str(df.gps_lat_dd.iloc[row]+box)+','+str(df.gps_lon_dd.iloc[row]+box)+') ["highway"]; (._;>;); out body;'
    queries.append(query)
print(queries)

Comment: @anonymous13 ok great! Do you mind accepting the answer? Thanks!!
Answer: For each iteration in the for loop you are making the query for that row and the variable query is updated by that value. Here, the query you are prining is the last query. You need to first make a list for the query and append each query to the list.
This should solve your problem:
query = list()
for row in df.index:
        querytemp = query = 'way('+str(df.gps_lat_dd.iloc[row]-box)+','+str(df.gps_lon_dd.iloc[row]-box)+','+str(df.gps_lat_dd.iloc[row]+box)+','+str(df.gps_lon_dd.iloc[row]+box)+') ["highway"]; (._;>;); out body;'
        query.append(querytemp)
print (query)


Using a select inside a while read is NOT giving me the result I want

Question: I have a file "in" that contains:
a
b
c

I run the following command:
cat in | while read el; do select yn in "yes" "no"; do echo $yn; done; done;

And get the following output:
1) yes
2) no
#?
#?
#?

What I want is to be able to type 1 or 2 at each iteration.  What do I need to do differently to make this work?
Answer: You're piping input into a while loop.  Within the loop, stdin comes from the preceding command, not from the console.  Your select statement is reading from the same pipe that the read statement is reading from.
Also, you don't need cat.
You need to preserve stdin for use in the select statement.  Try this:
while read el; do select yn in "yes" "no"; do echo $yn; break;
done <&4; done 4<&0 < in

First, note that we're redirecting input using '<', rather than using cat.
This preserves your origin stdin (fd 0) in fd 4, and then inside the loop redirects input to select from fd 4.
Comment: Very cool, this definitely works.

Is there a way to do it with a pipe?  The reason being is that my original input comes from a series of commands instead of a flat file.  I can definitely write the output to a file and redirect from the new file, but I'm curious how to do it with a pipe.
Comment: Yeah, you can do the same thing with a pipe.  In either case (using "<" or "|") you're just redirecting stdin.  Try this:  (cat input | while read el; do select yn in "yes" "no"; do echo $yn; break;
done <&4; done) 4<&0
Answer: select will read an input and execute it repeatedly until EOF; what you almost certainly want is the superficially similar:
cat in | while read el; do select yn in "yes" "no"; do echo $yn; break; done; done

Comment: It still doesn't prompt me for the select and only presents the prompt twice.  I'm still don't get the behavior I want.
Comment: Works For Me (TM)
Comment: It doesn't work for me. I get all output at once and it doesn't wait for input. Similar to what the OP's version does, just sort of in a different order.