Calculating the distribution of maximal value of $n$ draws from a normal distribution

Question: According to normal probability distribution theory which says that for $n$ independent,
identically distributed, standard, normal, random variables $\xi_j$ the expected absolute maximum is
$E(\max|\xi_j|)=\sqrt{2 \ln n}$
Regarding this, why do we need to multiple the above-mentioned estimate by $\sigma$ (Standard Deviation) in order to derive the expected absolute maximum for a normal, random variable with zero mean?
Comment: I don't think that is an equality. I think that is an asymptotic equivalence. It definitely doesn't hold for $n = 1$.
Comment: @K-1, I assume that you asking about the maximum of absolute values of a similar **sequence** of random variables $\eta_j$ the difference being that $\xi_j\sim N(0,1)$ and $\eta_j\sim N(0,1)$? If so please correct the question. Now it asks about the absolute maximum of a **single** normal random variable. The answer to that question is $\infty$.
Comment: It seems that something like $E(\max_j |\xi_j|) = \sqrt{2 \ln n} + C_n$ is correct with $C_n \sim -.3...$ as $n \to \infty$.
Comment: @Hans, Hmm. Can you explain? Leting $(X_n)$ be a sequence of iid standard normals and $b_n = \sqrt{2\log n}$, it is well-known that $b_n(M_n - b_n) \to G$ in distribution where $M_n = \max_{1\leq k \leq n}X_k$ and $G$ is a standard Gumbel random variable. Arguments from extreme-value theory indicate that a dominating random variable exists, so that $\mathbb{E} b_n(M_n-b_n) \to \mathbb{E} G = \gamma \approx 0.57721$. This would seem to contradict your claim. But, perhaps I'm missing something. Maybe the modulus affects things, though this seems doubtful in light of simple symmetry arguments.
Comment: @cardinal - Just a numerical observation with $n \le 2000$. In that range it appears that $E(M_n) \sim \sqrt{2 \log n} - .27...$, but of course that gap could be closing.
Comment: @Hans, I think it is closing. The convergence in distribution is slow. Also, doing a proper simulation in this case is pretty hard since we're looking at the maximum modulus. So, algorithmic nonidealities in the random number generation can come into play. For example, though nobody uses the inversion method for normal random variates, if you did, due to asymmetries in IEEE floating point, the smallest possible normal random variate you can generate is about -37.52, while the largest is only 8.21.
Comment: @cardinal - I actually computed the expected values of the maxima as numerical integrals, using Mathematica. But of course there could be other systematic errors in the integration routines.
Answer: If $\zeta_j = \sigma \xi_j $ for some $\sigma >0$ and some $\mu$ then
$$E[\max|\zeta_j|] =  E[\max|\sigma \xi_j |] = E[\sigma \max| \xi_j|]= \sigma E[ \max| \xi_j|]$$
and this tells us how to move from a standard normal with mean $0$ and standard deviation $1$ to a normal distribution with mean $0$ and standard deviation $\sigma$.
Comment: The modulus got dropped in your answer. Including it destroys the linearity, but not the scale equivariance.
Comment: @cardinal: Thanks - it was only half visible in the question, so I read it as a conditional sign.
Answer: Intuitively: values of the standard normal distribution (including the absolute maximum) 'tend to be' 1 SD = 1 away from 0.
In non-standard zero-mean normal, the data 'tend to be' 1 SD = sigma away from 0.
You could say that as long as you're doing linear stuff, all distances to zero will blow up by a factor sigma.
Comment: so it means that we need to correct the distance estimated from  standard normal distribution hypotheses for non-standard data?


strange tcp network connections 18.104.22.168:ircd

Question:
Possible Duplicate:
My server's been hacked EMERGENCY

I am seeing strange tcp connection 22.214.171.124:ircd  ans it is running perl service
and that process is creating huge load on server
IPV4    TCP     3u  MYIP:58449  ->  184.108.40.206:ircd     ESTABLISHED
can i know is this malware or any other service running and eating my server resources
My os details are centos5.5
Answer: If this isn't something you are expecting then it is most likely malware. Your only real solution is to shut the system down. Get an image for later analysis. Nuke the system from orbit then restore from a known good backup - it's the only way to be sure.
Answer: That's bad - it looks like your server is a part of botnet.
If you don't know where proces runs, check some info about it ('pid' is process id):
ls -l /proc/'pid'/fd
cat /proc/'pid'/cmdline
Find bad script and kill it. I guess, it's located in /tmp, /var/tmp or defined temporary directory for your webserver. It's a good chance, that script was uploaded via bad coded webpages; find it and repair.
Comment: Realistically, the only repair is a clean install and restore from a clean backup.
Comment: I'd tcpdump it first out of interest.
Comment: You cannot repair it with any confidence - it's not your system any more. If your backups are no good then rebuild from your sources...
Comment: Sorry, this is not good advice. Once your system has been rooted, its not yours anymore. In my experience, fiddling around like this is much less cost effective and secure than nuking and starting again.
Comment: If there is a hole in your backup, your server still has a problem.. Better is to know where the hole is and repair it.
Comment: In situations like this the "hole" is that you *don't have control of that system any more*. The only way to be certain that you've fixed that particular hole is to rebuild the system. While there's a lot of value in finding and fixing the exploit used to break into the system, you can't be sure about what **else** has been done to the system once the intruders gained control (indeed, they may well have patched the exploit themselves to safeguard "their" new botnet member from other hackers, as well as installing their rootkit).

strange tcp network connections 149.9.1.16:ircd


Apache Redirect when the page is SSL

Question: I have a redirect from http://example.com to http://www.example.com.
now i want this redirect to work when the user is in https protocol. meaning to redirect from https://example.com to https://www.example.com.
is it possible?
if not, how do i redirect https://example.com/page to http://example.com or http://www.example.com
i hope i was clear
Answer: Redirecting from https://example.com to https://www.example.com works in the same way as redirecting from http://example.com to http://www.example.com, except that you have to put the rewrite rules in your SSL-enabled VirtualHost in addition to your plain HTTP VirtualHost section.
Where you may get a problem is that, for this to work, you need the initial connection to https://example.com to work and be accepted by the browser. For this, you need your certificate to be valid for both www.example.com and example.com. This is typically done by using multiple Subject Alternative Names in your server certificate (you should be able to see this by looking at the details of your certificate). Some CAs issue certificates both with and without the www. prefix.

Apache Redirect when the page is SSL


Configuring 'deep' dependencies with Unity IoC

Question: Let's say I have the following classes and interfaces in my MVC 2 project:
Repositories:
IRepository1, IRepository2, IRepository3
public class ConcreteRepository1 : IRepository1
{
     public ConcreteRepository1()
     {
         ...
     }
     ...
}

public class ConcreteRepository2 : IRepository2
{
     public ConcreteRepository2()
     {
         ...
     }
     ...
}

public class ConcreteRepository3 : IRepository3
{
     public ConcreteRepository3()
     {
         ...
     }
     ...
}

Service classes:
public class Service1
{
    private IRepository1 repo1;
    private IRepository2 repo2;

    public Service1(IRepository1 repo1, IRepository2 repo2)
    {
        this.repo1 = repo1;
        this.repo2 = repo2;
    }
    ...
}

Controllers:
public class Controller1 : Controller
{
    private Service1 srv1;
    private Service2 srv2;

    public Controller1(Service1 srv1, Service2 srv2)
    {
        this.srv1 = srv1;
        this.srv2 = srv2;
    }
    ...
}

I have custom ControllerFactory and I know how to tie concrete repositories to interfaces:
IUnityContainer container = new UnityContainer();
container.RegisterType<IRepository1, ConcreteRepository1>(new TransientLifetimeManager(), new InjectionConstructor());
container.RegisterType<IRepository2, ConcreteRepository2>(new TransientLifetimeManager(), new InjectionConstructor());
container.RegisterType<IRepository3, ConcreteRepository3>(new TransientLifetimeManager(), new InjectionConstructor());

The question is how should I register services' instances and controllers' types inside my custom ControllerFactory to make unity container resolve the whole hierarchy Controller->Service->Repository and avoid calling Resolve inside controllers or services?
Thanks.
Comment: Why do you have more than one Repository? Do you access different datasources? Just curious...
Comment: @Yves M.: I have one repository per aggregate root in my domain layer. I have one data source, but hypothetically this approach supports each aggregate root to be on it's own data source.
Answer: If you have already registered IRepository1-3, so you can get Service1 instance simply calling
container.Resolve<Service1>()

Calling container.Resolve<Controller1>() will automatically resolve dependencies and will create instance of type Controller1.
Sample:
public interface IRepository1 { }
public interface IRepository2 { }
public interface IRepository3 { }

public class ConcreteRepository1 : IRepository1 { }
public class ConcreteRepository2 : IRepository2 { }
public class ConcreteRepository3 : IRepository3 { }

public class Service1
{
    private IRepository1 repo1;
    private IRepository2 repo2;

    public Service1(IRepository1 repo1, IRepository2 repo2)
    {
        this.repo1 = repo1;
        this.repo2 = repo2;
    }
}

public class Service2
{
    private IRepository1 repo1;
    private IRepository2 repo2;

    public Service2(IRepository1 repo1, IRepository3 repo3)
    {
        this.repo1 = repo1;
        this.repo2 = repo2;
    }
}

public class Controller1
{
    private Service1 srv1;
    private Service2 srv2;

    public Controller1(Service1 srv1, Service2 srv2)
    {
        this.srv1 = srv1;
        this.srv2 = srv2;
    }
}

Resolving:
var container = new UnityContainer();
container.RegisterType<IRepository1, ConcreteRepository1>();
container.RegisterType<IRepository2, ConcreteRepository2>();
container.RegisterType<IRepository3, ConcreteRepository3>();

var controller = container.Resolve<Controller1>();

EDIT:
public interface IRepository { }
public class Repository : IRepository { }

public class Service
{
    //[InjectionConstructor]
    public Service()
    {
        Console.WriteLine("Parameterless constructor called");
    }

    public Service(IRepository repository)
    {
        Console.WriteLine("Contructor with IRepository called");
    }
}

private static void Main()
{
    var container = new UnityContainer();
    container.RegisterType<IRepository, Repository>();
    var service = container.Resolve<Service>();
    container.RegisterType<Service>(new InjectionConstructor());
    var service2 = container.Resolve<Service>();
}

Output:
Contructor with IRepository called
Parameterless constructor called

So, when we haven't registered Service (in other words, it has default resolving behavior), Unity constructs Service using greediest constructor by default. When you specify new InjectionConstructor(), this tells Unity to use parameterless constructor. The same behavior can be received if you mark constructor (in this case public Service()) with InjectionConstructorAttribute.
Comment: I see, thanks. And I was trying to register all controllers' types with the code below and when I removed it everything worked: `var controllerTypes = from t in Assembly.GetExecutingAssembly().GetTypes()
          where typeof(IController).IsAssignableFrom(t)
          select t;

   foreach (Type t in controllerTypes)
   {
    container.RegisterType(t, new TransientLifetimeManager(), new InjectionConstructor());
   }`
Comment: So it looks like in this case all services/controllers types get kind of autoregistered with Unity, right? Is there a way I can disable this "autoregistration" and manually register each type/instance one by one or all together?
Comment: @Intruder, Sure, you can, without setting any "disable autoresolving" option. What exception did you catch?
Comment: When I manually register controllers with code provided in my first comment to your answer I get " does not have a constructor that takes the parameters ().  Source : Microsoft.Practices.Unity" exception
Comment: @Intruder, That is because you pass `new InjectionConstructor()` without parameters, therefore Unity tries to resolve `Controller` class using parameterless constructor, and as I see, your `Controller` constructor has 2 parameters. What is main purpose of register types by yourself?
Comment: The purpose is kind of illusion of better control over situation :) Also isn't there some overhead with autoregistering? And another reason is of course to learn what actually happens under the hood. So how should I use that InjectionConstructor in a proper way?
Comment: @Intruder, autoresolving is great feature of containers. As I see, you are using ASP.NET MVC. It has considered infrastructure to use IoC. Also, I've updated my answer with simple sample.
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/4035/discussion-between-intruder-and-kirill-polishchuk)
Answer: surely just the following -
container.RegisterType<Service1>(new TransientLifetimeManager(), new InjectionConstructor());

and the same for Service2. The IoC container will construct the objects for you


Set selected drop down value from database in razor view MVC

Question: I am explicitly creating a dropdownlist in razor mvc view. I would like the default option to be one of the items, the value of that item should come from database
View Code

  @Html.LabelFor(model => model[i].Customer.HomeType, htmlAttributes: new { @class = "control-label col-md-2" })

      @Html.DropDownListFor(model => model[i].Customer.HomeType, new List<SelectListItem>
 {
     new SelectListItem{ Text="House", Value = "House",@(Model[i].CustomerServices.HomeType == "House" ? Selected = true : false)  },
     new SelectListItem{ Text="Apartment", Value = "Apartment" },
     new SelectListItem{ Text="Farm", Value = "Farm" } },
      "Select your home type", htmlAttributes: new { @class = "form-control" })
 }

Controller Code
 cs.HomeType = serviceDetails.HomeType;
 sp.CustomerServices = cs;
 return View("EditCustomerServices", ProviderList);

Model has the data I need but how can I set it to correct Dropdown list value. I tried using short hand if but it gives me an error
Invalid member initializer

Can I add if condition for Selected Boolean?
Comment: why do you use indexer for model -> model[i]
Comment: Because I am running this in loop
Answer: @{
    var selected = Model.CustomerServices.HomeType == "House";
}

  @Html.LabelFor(model => model.Customer.HomeType, htmlAttributes: new { @class = "control-label col-md-2" })

      @Html.DropDownListFor(model => model.Customer.HomeType, new List<SelectListItem>
 {
     new SelectListItem{ Text="House", Value = "House", Selected = selected },
     new SelectListItem{ Text="Apartment", Value = "Apartment" },
     new SelectListItem{ Text="Farm", Value = "Farm" } },
      "Select your home type", htmlAttributes: new { @class = "form-control" })
 }

Comment: What if apartment is selected, there will be too many if conditions. This is not a good way of doing it
Comment: your view model must contain the info about selected item. You should not decide which item is selected in the view, this must be done in the controller.
Comment: how does your model look like?


Is Amazon S3 versioning really a reasonable backup options?

Question: I am in the process of moving my web apps project storage to S3 and I am wondering if S3 versioning is really a good way to handle backups of the data or is there some better way? If the files were all to be deleted or corrupted in some way is it possible and easy to restore an entire bucket using the versioning? If so how? If not, what would be a better backup option for S3?
Answer: Versioning is a great feature, and it should absolutely be used if possible. Having versioning enabled (and using appropriately-provisioned access keys) can save you from all manner of issues.
But.
Versioning won't protect you from;

Loss of control of your AWS account,
S3 downtime or widespread corruption,
Other similar act-of-god type issues.

You need to have backups of your data outside of S3, even if that's an external hard drive that you run <code>$ aws s3 sync</code> against a couple of times a day. Having a backup-of-last-resort is very simple to do, and is very inexpensive.
Answer: No, a backup strategy for off service is required regardless of the service in question.  Aside from the potential for service side corruption, account issues can cause unrecoverable issues.


Infinite sum: $\cos(x)-\cos(3x)+\cos(5x)-\cos(7x) +\cdots$

Question: Question: What is the sum of this infinite series?: $\cos(x)-\cos(3x)+\cos(5x)-\cos(7x) +\cdots$
I know that $\sum \cos(nx)$ is something like $\sum \delta(x-k)$ and $\sum \sin(nx)$ is something like $\cot(k \pi x)$. I tried plotting this in wolfram-alpha and it looks like $\sec(x)$ or perhaps $\sec^2(x)$, but not sure. Any help?
Comment: Wolfram Alpha https://www.wolframalpha.com/input/?i=sum+of+cos(4k-3)+-+cos(4k-1)+for+k+%3D+1+to+k+%3D+infinity shows that the sum does not converge by the root test.
Comment: You can get to the sum by grouping each pair of + terms and - terms together, and then considering the pattern of the averages of the constant multipliers (the average of $\cos(x)$ and $-\cos(3x)$ would be $\cos(2x)$, and find a linear equation for that.
Comment: @TobyMak Perhaps you should check your link, as I don't think WA says it diverges by the root test.  Here's what I see: https://i.stack.imgur.com/pB04R.png
Comment: @Simply Beautiful Art Changed the post
Answer: Obviously it doesn't actually converge. We can compute the finite sums,
$$ S_m(x) = \sum_{k=0}^m (-1)^k \cos{(2k+1)x} = \frac{1}{2}\sec{x}\big( 1+ (-1)^m \cos{2(m+1)x} \big), $$
by using, e.g., the formulae given in this question. So the function oscillates between $\sec{x}$ and $0$, and the oscillations become more rapid as $m$ increases.
We can, however, sum the series to a finite, well-behaved expression using Fejér sums:
$$ \frac{1}{n+1} \sum_{m=0}^n S_n(x) = \frac{1}{2}\sec{x} + \frac{\cos{x}+(-1)^n\cos{(2n+3)x}}{4 (n+1)}\sec^2{x} \to \frac{1}{2}\sec{x} $$
as $n \to \infty$.
Comment: [Fejér's sums](https://en.wikipedia.org/wiki/Fej%C3%A9r%27s_theorem)?  I am more familiar with [Cesàro sums](https://en.wikipedia.org/wiki/Ces%C3%A0ro_summation).
Comment: In the context of trigonometric series, Fejér's name is often attached to the summation process, as well as the kernel and the theorem.
Comment: Thanks. This does match my suspicion that it is sec x. The WA reference I made in my post is this: [link] (https://www.wolframalpha.com/input/?i=cosx+-+cos3x+%2B+cos5x+-+cos7x+%2B+cos9x+-+cos11x+%2B+cos13x+-+cos15x+%2B+cos17x+-+cos19x+%2B+cos21x+-+cos23x+%2B+cos25x+-+cos27x+%2B+cos29x+-+cos31x+%2B+cos33x+-+cos35x)
Comment: Hm, I see. Seems to be naturally the case.
Comment: @Srini One needs to be careful about what "is" means here.
Comment: @Srini Your link only concerns the partial sum of the infinite series.  If this is what you meant, you should update your post to be more clear on this.
Comment: No, no, it is clearly the inifinite sum I am after (very explicitly mentioned in the title). I just posted that link to say why my hunch was towards sec(x). I know that it is partial sum that I linked to. But not because I wanted partial sum, but because I didn't know how to input inifite sum functions
Comment: @Chappers You mean that the sum (as computed infinitely) tends towards sec(x)/2 right? Your reference to "is" is to clarify that it is a limit?
Comment: @Srini See the first line.  It clearly says "Obviously it doesn't actually converge."  More info may be found in the first link, or in my answer, which likewise shows this series does not converge in the ordinary sense.
Comment: @Srini I mean that the ordinary partial sums do not converge to a limit, but the Fejér sums do converge to a limit (except at the singularities of $\sec{x}$, of course).
Answer: For $x=\frac{(2n+1)\pi}2,n\in\mathbb Z$, it is easy to see that
$$\sum_{n=0}^\infty(-1)^n\cos((2n+1)x)=0$$
Since every term is zero.  However, if $x\ne\frac{(2n+1)\pi}2,n\in\mathbb Z$, then it suffices to prove that
$$\lim_{n\to\infty}\cos((2n+1)x)\ne0$$
Or,
$$\limsup_{n\to\infty}\cos((2n+1)x)=1$$
Hence, it diverges.

If one wishes to regularize this series to a finite values, then one may apply an Abel sum:
$$\begin{align}\operatorname{Abel}(S_x)&=\lim_{t\to-1^+}\sum_{n=0}^\infty t^n\cos((2n+1)x)\\&=\Re\left[\lim_{t\to-1^+}\sum_{n=0}^\infty t^ne^{(2n+1)ix}\right]\\&=\Re\left[\lim_{t\to-1^+}\frac{e^{ix}}{1-te^{2ix}}\right]\\&=\Re\left[\frac{e^{ix}}{1+e^{2ix}}\right]\\&=\frac12\sec(x)\end{align}$$
Answer: just a hint
The general term of your series is
$$u_n=(-1)^n\cos( (2n+1)x)$$
$$\lim_{n\to+\infty} u_n=0\iff x=\frac {\pi}{2}+k\pi $$
Answer: For integer $n$, if $\cos x=1,\cos(2n+1)x=1$
and  if $\cos x=-1,\cos(2n+1)x=-1$
In both cases, the series oscillates finitely  between $-1,1$
If $\cos x=0,\cos(2n+1)x=0$ so, each term of the series $=0$
Else as $(-1)^n\cos(2n+1)x=\cos\{n\pi+(2n+1)x\}$
Using $\sum \cos$ when angles are in arithmetic progression,
$$\sum_{n=0}^{m-1}\cos\{n\pi+(2n+1)x\}=\dfrac{\cos\left\{x+(m-1)\left(\dfrac\pi2+x\right)\right\}\sin m\left(\dfrac\pi2+x\right)}{\sin\left(\dfrac\pi2+x\right)}=\cdots=\dfrac{\sin^2m\left(\dfrac\pi2+x\right)}{\cos x}$$


how to assign different default images to different roles

Question: I have only two profiles, public and registration but I have different roles and would like to add an appropriate default image for each role.
Must I create a separate profile for each role to implement this?
Comment: What do you mean by two profiles?  In Drupal, all users have a user profile, but this is generally the same for all users, regardless of their roles.
Comment: I mean in structure - profile types I have set up two types. One for public and one for registration.
Comment: @CharlieMeany are you using the Profile2 module, or the Profile module which comes default with Drupal 7 (intended to help keep old D6 profiles alive), or something else?
Comment: profile2 is what I am using.
Answer: If you want to display an image for a particular user role , say "Role1" , then in page.tpl
<code> <?php global $user;
  if($user->roles[2] == 'Role1')
 {
 print $logo;
 }?>
</code>
Just print the $user to get in what values your role is in the role array.
If you print $user, your role may be in this,
<code> [roles](
 [1] = role1
 [2] = role2
 [3] = role3
       )
</code>
So choose your role and check the condition and accordingly print the $logo however before that get the image in $logo using the image source tag.


regular expression matching in java

Question: Here is my example file :
lineone one
RUN lineone two
lineone three
RUN lineone four

I want to select all lines not starting with run, here is how I did it :
^([^RUN])

Is it possible to match all lines not starting with RUN and then append them to the back of the previous line ? like this
lineone one RUN lineone two
lineone three RUN lineone four

Comment: Your question doesn't describe what your example shows. In your example you are appending lines that *do* start with RUN to the end of the previous line.
Answer: If your example is correct you just need to replace "\nRUN" with " RUN".
System.out.println(yourString.replaceAll("\nRUN", " RUN"));

Result:

lineone one RUN lineone two
lineone three RUN lineone four

ideone
Answer: use str.startsWith("RUN");
Answer: First of all
^([^RUN])

does not work correctly as it will match any line that does not start with either R, U or N.
You should use lookahead:
^(?!RUN)

This should do what you want:
Pattern p = Pattern.compile("\n(RUN)", Pattern.DOTALL);
Matcher matcher = p.matcher("lineone one\nRUN lineone two\nlineone three\nRUN lineone four");
String replaceAll = matcher.replaceAll(" $1");
System.out.println(replaceAll);

regular expression matching in java


Error: Error in file(con, "r") : cannot open the connection

Question: I am trying to run dssp function using bio3d package in R. But I am getting error.
<code>library("bio3d")
pdb <- read.pdb("1CRN")
x <- dssp(pdb, exepath="C:/dssp/dssp.exe")

Error in file(con, "r") : cannot open the connection
</code>
Any suggestions please.
Comment: Do examples in `dssp` work for you? Try including exepath in the example. If not, you could always include path to dssp.exe into your PATH and avoid typing exepath all togehter.
Comment: I've answered your question where you asked it originally: http://biostar.stackexchange.com/questions/19053/secondary-structure-analysis-with-dssp-in-r/19118#19118
Comment: @ThomasH - here is the working link: http://www.biostars.org/p/19002/
Comment: @neilfws That link is broken for me, any chance you could duplicate the answer here as well?
Answer: To solve it you need to fix the dssp function paramenters:
fix(dssp)
rewrite the following line:
system(paste(exepath, "dssp -c ", infile, " ", outfile, sep = ""), ignore.stderr = TRUE)
"-i" istead of "-c" and "-o" intead of " "
chears


Site Map Menu are not translatable

Question: I have created sitemap page using sitemap module to include some menu for displaying on sitemap page. Now I am unable to translate menu titles, while all the links are translated.
Also I added the translation for menu title from Edit-Menu-> Translate.
Answer: Be aware of the Main Menu not translated issue, which states:

When using the default 'Main menu' menu, and outputting that on the sitemap, the title, or menu name, isn't translated ...

I just double checked in a multi language site also where I have the i18n Menu Translation sub-module enabled (one of the Internationalization sub-modules).
I tried editing the Sitemap menu item to be translated, by navigating to admin/structure/menu/manage/navigation. From there I tried to use the "translate" link (for the Sitemap menu item), which one would expect that it can be used to translate that menu item.
However, after doing so, I only get something like "You are not authorized to access this page."
This issue appears to be fixed in the most recent dev version.

Site Map Menu are not translatable


Intuition behind expectation of likelihood ratio function being 1

Question: Suppose we have two probability density functions $f(x)$ and $g(x)$. Let $\mathbb{P}_g$ be the probability measure induced by $g$. Consider the likelihood ratio function $\Lambda(x) = \frac{f(x)}{g(x)}$. Now consider its expected value $E_g[\Lambda]$ w.r.t. $\mathbb{P}_g$. Assuming that $\Lambda(x)$ is integrable, we have
$$ E_g[\Lambda] = \int_{\mathbb R} \frac{f(x)}{g(x)} \ g(x) \ dx = \int_{\mathbb R} f(x) \ dx = 1. $$
While the math is trivial, I don't understand why this is true intuitively. Intuitively, $\Lambda(x)$ measures how more "likely" it is to "draw" $x$ from $f$ vs $g$. Now, if the probability space is given by $\mathbb{P}_g$, then it means that $x$ is actually "drawn" from $g$. Hence, intuitively I would think that "on average" $\Lambda(x)$ is lower than 1, and so $E_g[\Lambda] < 1$.
Where is the flaw in the heuristic reasoning?
Answer: Since we are talking about heuristic reasoning I feel comfortable with sharing my thoughts in a non-formal way (so I'll use loads of " ... ").
The examples might seem child-trivial but this is the way I like to think.
You say that

"..the function $\Lambda(x)$ measures how more "likely" it is to "draw" $x$ from $f$ vs $g$."

The flaw is that you probably misinterpret how you use the function $\Lambda$ to measure whether something is more or less "likely". In particular, you definitely cannot think of it in terms of $\Lambda$ being smaller (or bigger).
Think of it in very easy terms: for example let $f$, $g$ be the "density" of two Bernoulli variables with parameters $p_f$ and $p_g$. Then you'd have
$$
\Lambda (0) = \frac{1-p_f}{1-p_g} , \, \, \, \Lambda (1) = \frac{p_f}{p_g}
$$
Here, as you can see, if $p_f > p_g$ then $\Lambda(0) < 1$ and $\Lambda(1) > 1$, and the two things "level out" proportionally to $p_g$ if the underlying distribution is $g$. One way to see this, observe that the ratio between observations (zeros and ones) under $g$ is proportional the ratio between $\Lambda(1)$ and $\Lambda(0)$. You can imagine/visualize it as follows: imagine you're choosing many times between two things, a bigger and a smaller one, exactly (on average) as many times you need to level them out. (the ratio between number of picks of the two is inversely proportional to the ratio of their sizes).
On the other side, if the underlying distribution actually is $f$, then it's less probable to "observe" zeros rather than ones, so in this case the expected value $E_f[\Lambda]$ will be bigger than one.
This is just a small thought in a very easy (and very discrete) setting, but I think that everything could be translated into continuum without too much effort.
Answer: When $g(x) > f(x)$ you have $0 \le \frac{f(x)}{g(x)} \lt 1$
When $0 < g(x) < f(x)$ you have $1 \lt \frac{f(x)}{g(x)} \lt \infty$
In a handwaving sense, your are correct that the first is more common than the second when measured using $g$, but the second can be very much larger when it does occur; indeed in some cases it can be unbounded above, while the first is restricted to a small interval
I think it is not intuitively obvious which effect dominates. As you say, it turns out that these two effects balance each other so long as $g(x)$ is never $0$ on the support of $f(x)$, and this should not be intuitively implausible


Add CSS class to  element using webform and template file

Question: I have an existing webform in Drupal 7 which I am trying to theme.
So far I managed to add CSS classes, placeholders, prefix, suffix,.. to form elements (fields) but not to the actual <form>-tag?
I tried this:
$form['#attributes']['class'][] = 'form-horizontal';`

But that didn't change anything to the output.
edit: I have found that putting var_dump($form['#attributes']); displays the array below the form-tag. Does that mean that you cannot alter the root element inside a template file (webform-form-8.tpl.php) and that I should implement the hook in template.php?
Answer: I believe your edit is correct. You should add the class in a preprocessor function within template.php.
A similar question and answer can be found here.
Answer: function THEME_form_alter(&$form, &$form_state, $form_id) {
    $form['#attributes']['class'][] = 'my_class';
}

in a template.php


ARIMA grid search for best $p$, $d$, $q$ orders based on BIC metric

Question: i am new to R, and i was hoping to get some answers to some questions i have

is there a function to loop through a range of ARIMA model to get the best order/seasonal based on BIC/AIC ?
do i have always to difference the data if its not not stationary or there might be another reason to that and i dont have to take the differencing ?
i used Auto ARIMA to find the best ARIMA model based on its BIC, and i specifically specified BIC perimeter in the Auto ARIMA function but when i tried to use some candidate ARIMA order i was able to get a lower BIC than what the Auto ARIMA got, how is that possible ?
what could be a better practice, to use the differencing time series data in the arima function , or to use the original one and specify that there is an order of d there?
Answer:
Yes. Use forecast::auto.arima(data, ic="bic")
No. auto.arima() will determine the number of differences needed for you.
auto.arima() will avoid models which could be numerically unstable. For example, the fitted model could have near unit-roots.
Over-riding the automatic selected order of differencing is sometimes useful. But make sure you are aware of the consequences. See https://otexts.com/fpp2/arima-r.html#understanding-constants-in-r
Comment: thank you Dr.Rob , but i might not have made it clear, but in my first questions i was asking about a function that can use a range of orders like the one in JMP. As you look at the ACF, PACF and and what could be the suggested candidate orders and then use a range of them to have JMP loop through that range and spit out the one with the minimum BIC/AIC according to your choice
Comment: auto.arima allows you some control over what models are included in the search. See the help file. The ARIMA function from the fable package provides even more control.


Drupal commerce price markup field

Question: I need to calculate product sell price as gross price * markup (i.e. 1.4) Gross price is stored in field_data_commerce_price table (standard commerce price field). Markup value should be associated with every single product, so I can manage it with views using filters and VBO. I have many product types, so maually created markup field is not a good solution.
Is there any out-of-the-box solution?
and if not:
What is the best way to add new product field using API? (should be added by default for all product types)
Answer: If you need a per product price modificator, your best solution is to have a field with this information, you can probably use a normal numeric field for this purpose.
You've got many examples on how to create a field using the API, for example, drupal commerce itself adds a price field every time a product type is generated by using hook_modules_enabled hook.
/**
 * Implements hook_modules_enabled().
 */
function commerce_product_modules_enabled($modules) {
  commerce_product_configure_product_fields($modules);
}

Then in this function
function commerce_product_configure_product_fields($modules = NULL) {
(...)
      // Loop through and configure the product types defined by the module.
      foreach ($product_types as $type => $product_type) {
        commerce_product_configure_product_type($type);
      }

Next
/**
 * Ensures a base price field is present on a product type bundle.
 */
function commerce_product_configure_product_type($type) {
  commerce_price_create_instance('commerce_price', 'commerce_product', $type, t('Price'), 0, 'calculated_sell_price');
}

And finally
function commerce_price_create_instance($field_name, $entity_type, $bundle, $label, $weight = 0, $calculation = FALSE, $display = array()) {
  // Look for or add the specified price field to the requested entity bundle.
  commerce_activate_field($field_name);
  field_cache_clear();

  $field = field_info_field($field_name);
  $instance = field_info_instance($entity_type, $field_name, $bundle);

  if (empty($field)) {
    $field = array(
      'field_name' => $field_name,
      'type' => 'commerce_price',
      'cardinality' => 1,
      'entity_types' => array($entity_type),
      'translatable' => FALSE,
      'locked' => TRUE,
    );
    $field = field_create_field($field);
  }

  if (empty($instance)) {
    $instance = array(
      'field_name' => $field_name,
      'entity_type' => $entity_type,
      'bundle' => $bundle,

      'label' => $label,
      'required' => TRUE,
      'settings' => array(),

      // Because this widget is locked, we need it to use the full price widget
      // since the currency option can't be adjusted at the moment.
      'widget' => array(
        'type' => 'commerce_price_full',
        'weight' => $weight,
        'settings' => array(
          'currency_code' => 'default',
        ),
      ),

      'display' => array(),
    );

    $entity_info = entity_get_info($entity_type);

    // Spoof the default view mode and node teaser so its display type is set.
    $entity_info['view modes'] += array(
      'default' => array(),
      'node_teaser' => array(),
    );

    foreach ($entity_info['view modes'] as $view_mode => $data) {
      $instance['display'][$view_mode] = $display + array(
        'label' => 'hidden',
        'type' => 'commerce_price_formatted_amount',
        'settings' => array(
          'calculation' => $calculation,
        ),
        'weight' => $weight,
      );
    }

    field_create_instance($instance);
  }
}

So you in summary can use hook_modules_enabled to react and loop the product types and create your field and instance. Another examples of this procedure can be found in commerce recurring, commerce discount, commerce coupon and more.
Comment: Yeah, it depends on the number of fields/product types you've got. If that's not manageable, you can create your own action for VBO and select the field for each type (saving it in a variable, like commerce does for the customer profile address field), that'd give you a single checkbox.
Comment: Thanks, pcambra, I've implemented suggested hook_modules_enabled and "markup" field appeared in all commerce_product bundles, but when I trying to modify entity values with VBO it treats every field instance (one for each bundle) as separate field, so I can't modify markup value for all bundles in one simple action without "crazy checkboxing": (https://dl.dropboxusercontent.com/u/42690200/VBO-checkboxes.PNG)

Drupal commerce price markup field


XPages - unsorted Dojo Data Grid opens incorrect document

Question: I'm using a Dojo Data grid together with a REST service to display view data. When I double click on a row, an XPage is opened. My problem is that, if one of the columns in the grid is not sorted, the wrong XPage is opened. What could be the problem here?
<xe:djxDataGrid id="P_Alle_DDG" store="restService2"
styleClass="DojoViewTable" title="Pendenzen - Alle" autoHeight="20"
rowsPerPage="25" selectable="true" selectionMode="multiple"
singleClickEdit="true" rowSelector="2" style="font-size:12pt"
escapeHTMLInData="true">
<xe:this.onRowDblClick><![CDATA[var idx = arguments[0].rowIndex;
var unid = restService2._items[idx].attributes["@unid"];
var url = 'Reparatur.xsp?documentId='+unid+'&action=openDocument';
window.document.location.href = url;]]></xe:this.onRowDblClick>

UPDATE: With the following JavaScript code the problem has been solved:
var grid = arguments[0].grid;
var index = arguments[0].rowIndex;
var item = grid.getItem(index);
var unid = item.attributes["@unid"];
var url = 'Reparatur.xsp?documentId='+unid+'&action=openDocument';
window.document.location.href = url;

Comment: I would think that the underlining view would need to be sorted for it to work at all.  The sorting doesn't have to be one of the columns that you show in the grid.
Comment: I have sorted all columns in the underlying view. The columns in the underlying view correspond with those displayed in the Dojo Data Grid.
Answer: Tony, try this method of opening the document.  The code if very similar to yours, but the key difference is that I made a view column that contains the unid, I called it "docid".  This works for me.
var grid = arguments[0].grid;
var index = arguments[0].rowIndex;
var item = grid.getItem(index);
var unid = item["docid"];
var url = "New_PO.xsp?doc=" + unid;
window.document.location.href = url;

Comment: Thanks, copying your code directly did not work but after adapting it, all now works well. I have add my working Javascript to my question above. I am using xe:viewItemFileService as the service for my REST service so perhaps that is a factor.


ORA-00904: ORDER BY with UNION ALL

Question: I'm creating a flat file with header. I'm getting an ORA-00904 error, which I think is because of the headers I created that does not have the field 'employee_name' (am I correct with that assumption?).
If yes, how can I can I sort my query without the headers?
The error I get:

    ORA-00904: "employee_id":invalid identifier

My code:

    select 'FILENAME' as filerec from dual
    UNION ALL
    select 'FILEDATE: ' || to_char(sysdate,'mm/dd/yyyy) as filerec from dual
    UNION ALL
    select employee_id || emloyee_name from employee_database as filerec
    order by employee_id;

This is the output I want produce:

    FILENAME
    FILEDATE: 02/27/2015
    200125Ruth Chan
    200126Dan Gonzales
    200135Lisa Mayoral

Comment: The ORDER BY goes for the whole UNION ALL!
Comment: employee_id  is valid column name?? instead of name you can use column number(1,2 ,3..)
Answer: In SQL, you cannot name the columns in ORDER BY with a composite statement, you have to use column position.
Or,
Project the columns explicitly.
In your case, since you have concatenated the columns, you cannot project them explicitly, also, column position would not be of any sense. You could therefore play a small trick. Add a pseudo column, with required values to the rows you want to be sorted first, and then use  NULL value in the pseudo column for which you want the sorting after the first column. So that, the NULLs are always placed in the end of the sort.
For example,
SQL> SELECT filerec FROM (
  2      SELECT 'FILENAME' AS filerec, 1 col FROM dual
  3      UNION ALL
  4      SELECT 'FILEDATE: ' || to_char(SYSDATE,'mm/dd/yyyy') as filerec, 2 col FROM dual
  5      UNION ALL
  6      SELECT empno || ename AS filerec, NULL col FROM emp
  7      ORDER BY 2,1
  8  );

FILEREC
--------------------------------------------------
FILENAME
FILEDATE: 02/27/2015
7369SMITH
7499ALLEN
7521WARD
7566JONES
7654MARTIN
7698BLAKE
7782CLARK
7788SCOTT
7839KING
7844TURNER
7876ADAMS
7900JAMES
7902FORD
7934MILLER

16 rows selected.

SQL>

Comment: A follow question. How can I write a trailer record with the total count of the details:
Answer: Try order by column position number instead of column name
select 'FILENAME' as filerec from dual
UNION ALL
select 'FILEDATE: ' || to_char(sysdate,'mm/dd/yyyy) as filerec from dual
UNION ALL
select employee_id || emloyee_name from employee_database as filerec
order by 1;

or use alias
select 'FILENAME' as filerec from dual
UNION ALL
select 'FILEDATE: ' || to_char(sysdate,'mm/dd/yyyy) as filerec from dual
UNION ALL
select employee_id || emloyee_name as filerec from employee_database
order by filerec;

Comment: It's possible that OP wants the result in select-order first, and only the last select in employee_id order.
Comment: @jarlh, I think it's more then possible; OP said the first 2 unions are headers.
Comment: UNION merge the complete result and order the complete result. No detailed order is possible.
Answer: Putting the last part of the UNION in brackets will probably do the trick:
select 'FILENAME' as filerec from dual
UNION ALL
select 'FILEDATE: ' || to_char(sysdate,'mm/dd/yyyy) as filerec from dual
UNION ALL
(select employee_id || emloyee_name from employee_database as filerec
    order by employee_id);

Edit: Since there is no inherent order, an even better way would be this:
select 'FILENAME' as filerec from dual
, 1 sortorder
UNION ALL
select 'FILEDATE: ' || to_char(sysdate,'mm/dd/yyyy) as filerec from dual
, 2 sortorder
UNION ALL
select employee_id || emloyee_name from employee_database as filerec
, 3 sortorder
ORDER BY sortorder, filerec;

You may need to put this in a subquery, cause you probably don't want to select the attribute sortorder.

ORA-00904: ORDER BY with UNION ALL


How to Check Linearity Assumption in Logistic Regression with a Large Dataset?

Question: I am working with a very large dataset that essentially covers the entire population of interest. I want to assess the linearity assumption between an independent variable and the log(odds) of the dependent variable in logistic regression.

There are different ways to check this assumption, with a typical method being to create a statistical term representing the interaction between each continuous independent variable and its natural logarithm. If any of these terms is statistically significant, the assumption is violated. Solutions include dummy coding the independent variable, or statistically transforming it into a different scale."

Given the dataset's size, how can I effectively check for linearity without overly relying on statistical significance, because with large datasets things tend to be really quickly significant? Or is it even necessary to check this assumption?
Comment: You can graph the scatter plot between each predictor variable and the logit values, and add a smoothed line (such as loess) and then make a judgement as to whether it is straight or not.
Comment: Thanks.  I prefer to look at statistical sources for statistical info.  Of course, other journals are sometimes right, but ... I wouldn't look to American Statistician for advice on emergency medicine, so, I don't do the opposite.
Comment: its's from: Academic Emergency Medicine Volume 18, Issue 10
Comment: Haha, It was more for explaining the method I have seen it more in different sources. But my teacher kind of hinted that checking this assumption might not be necessary at all. But I have a hard time remembering why he said that. Do you have any information about that, or do you think it is necessary?
Comment: Sorry, but I don't.  Is your name a play on the great chess player Peter Leko?
Comment: Yes it is! nice spot :)
Answer: The procedure given in the quote is part of the Box-Tidwell procedure, see

Is there formal test of non-linearity in linear regression?

Comparing coefficients in logistic regression with curvilinear effects

and search this site. A better reference is An R Companion to Applied Regression   which has a detailed treatment.
But there are probably better ways: Use visualization of residuals, see for instance Diagnostic probability plots in logistic regression  and search this site.
But, since you have large data, why not model upfront with modern, flexible models using splines? As an example, in the logistic regression models I see, there is often a predictor age. If this variable covers an appreciable part of a normal human lifelength, there is no chance its effect will be linear. I do not test that, I spline it from the start.
This is consistent with the good advice at When to check model assumptions


MVC PHP - Libraries - Estrutura de Diretórios

Question: estou com uma dúvida em relação da estrutura de pasta da aplicação.
Seguinte, a dúvida surge quando há a necessidade de escrever uma classe que faz integração com um API essa classe seria alocada em que parte? Pois creio que essa classe não teria comportamento de uma componente (não tem qualquer tipo de logica de negócio) e sim talvez de uma lib (Library) que seria uma interface para consumir os recursos da API.
Imaginem a seguinte estrutura:
/app/
 - controllers/
 - components/
 - models/
 - vendors/

Mas a questão é, se API é um pacote escrito por mim, ela não deveria ir para vendors (já que vendors são só para pacotes de terceiros), estou certo? Ou talvez seria ideal ter uma novo diretório librarys/ para pacotes internos da aplicação?
Answer: Em casos como esse a organização é feita da seguinte forma: /vendors/Company/Library/Company/PackageName. Se seu código é genérico e poderá ser reutilizado em outros projetos sem muita necessidade de adaptação, ou seja, como você mesmo disse, se não tem nenhuma regra de negócio, o lugar dele é na vendors.
Comment: Obrigado por responder @hamboldt, entendo, sendo assim posso empacotar todas as classes que não tem logica com da aplicação mas que são utilizas pelo mesmo, em um pacote próprio dentro de `vendors`?
Comment: Exatamente. Quando vamos fazer um software temos que pensar como iremos fazer para que este tenha a maior possibilidade de ter partes reaproveitáveis. Então se conseguirmos separar coisas reaproveitáveis dos códigos específicos de regra de negócio significa que iremos ter cada vez menos trabalho futuramente, podendo reutilizar sem muito esforço nossos códigos passados.

É importante ter em mente também que devemos organizar de modo que podemos presumir onde cada coisa está por exemplo:

* /library/Outlook/Email/Author.php
* /library/Outlook/SMTP/Request.php
* /library/Outlook/SMTP/Response...
Answer: A minha resposta vai no sentido da minha experiência em MVC e especificamente em PHP conforme colocado na pergunta e no que diz respeito à sua estrutura.
Sem entrar em conceitos básicos, a experiência levou-me ao longo dos anos a adoptar uma estrutura diferente daquela que apresenta no entanto sei bem que esta depende muito de caso para caso. Para um projecto que implemente também uma api em conjunto:
/projecto/
   |--api/
   |    |--resources/
   |    |index.php
   |    |.htaccess
   |
   |--application/
   |    |--config/
   |    |--controllers/
   |    |--models/
   |    |--libraries/
   |    |--views/
   |    |--(etc... dentro da lógica)
   |    |.htaccess (deny from all)
   |
   |--public/
   |--vendor/
   |
   |index.php
   |.htaccess

Como pode observar o models fica dentro do application e não fora o seu components penso que será equivalente ao libraries e quanto ao vendors a norma é chamar-lhe vendor no singular caso trabalhe com o Composer... dá mais jeito.
O .htaccess é colocado em cada directório que esteja na raiz do projecto e aí entra a minha resposta directa à sua pergunta sobre o API. Tratando-se de um serviço adicional ao projecto normalmente incluo o directório API na raíz e com o referido .htaccess, desta forma defende os recursos lá presentes por via de uma url amigável.
Recordo que um MVC separa o layout do código PHP e no que respeita ao API deve de separar a implementação de recursos da lógica dos models que por questões de manutenção do código deve de utilizar as implementações em application/models, neste caso :)
Comment: Obrigado por responder @chambelix, mas neste casso eu não estaria gerando redundância já que minha intenção na verdade não é disponibilizar uma `API` e sim a 'classe' de `API` na verdade estará buscando informações de uma API de terceiros. Estou correto, que não seria uma boa?


Views Exposed filter can't input username to filter. Only works with User ID

Question: I created a custom View, where it shows all of a specific content type. In this content type I have a field that is a user reference called "Assigned to". In this view, I want it exposed so one can search by a username, but this exposed filter only takes in the User ID, but not the user name. What am I missing to make this work?
I added a relationship of User:Assigned_To and that didn't seem to work or I'm not configuring it properly.
Answer: Looks like you're filtering by the actual entity reference field field_assigned, which will naturally just take the user ID because the field itself only knows only about the ID of the referenced entity (specifically, it stores the user ID in the target_id)
In order for you to filter against usernames, you:

Add a filter to the username field.

When choosing the field the filter, narrow the category down to User. That way you can find and select the username field.

When configuring the filter, set the relationship to the one that connects your content type to the user via the field_assigned field.

This is needed because you can have another entity reference field that also points to User (e.g. something like a "reviewed by"), and Views needs to know which field you meant to use.

Added bonus: Enable SQL debugging in Views settings. That way you can see the resulting SQL query done by Drupal. This will tell you if you're filtering against the right columns or missing some joins.


Coming from naive set theory, are the following understanding right in ZF system?

Question:
In ZF system, a subset $S'$ of set $S$ is: If $S'$ is a set and for any object $b$ $b\in S'\implies x\in S$.
That is, before being a subset, $S'$ must be a set by itself.
The set of integer $\mathbb{N}$ is set in both naive set theory and in ZF system.
A collection $C$ of an infinite number of arbitrarily selected integer is a subset in
naive set theory. But $C$ is not a subset in ZF system.
The power set of $\mathbb{N}$ in naive set theory is quite different from that in ZF system.
The power set of $\mathbb{N}$ in naive set theory has many more elements than that in ZF system.
Although the power set of $\mathbb{N}$ in ZF system has less element than that of
naive set theory,
The Cantor's power set theorem works always: The power set of $\mathbb{N}$ in ZF system
has a bigger cardinality than $\mathbb{N}$.

Still unsolved question:

In naive set theory, set of real number $\mathbb{R}$ has 1-1 mapping with the power set of $\mathbb{N}$.
Is there a such mapping in ZF system? That is, is $\mathbb{R}$ a set in ZF system? If yes, how is the mapping defined?
Comment: Not sure what you mean by $2$. Can you give me an example of a set of integers in naive set theory that is not a set in set theory? 3. likewise. What do you mean by "many more?" How do you compare the sizes of infinite sets from different theories?
Comment: There is a fundamental misconstuing of what these systems are. Naive set theory is not about something "real." It exists as it does as an informal way of talking about "sets," but sets are not real, as far as we know. Formal systems of reasoning, like ZF, describe a theory that could cover various "models." Even these models are not "real." ZF exists because naive set theory is either contradictory or too weak to be usable formally.
Comment: So, when you are reasoning in ZF, you are finding common arguments that work in all models of set theory that satisfy the rules of ZF. When you are reasoning in naive set theory, it is entirely unclear what you are reasoning about - it is an informal system. (Unless you are using a formalized version of naive set theory...)
Comment: by 2, I mean a collection of an infinite number of arbitrarily selected integer is a subset in naive set theory, but it's not in ZF system. That is an infinite subset should be built by property or other means. An arbitrary infinite subset in naive set theory is not a subset in ZF system.
Comment: There is nothing in ZF that says that there aren't other sets than the ones you can explicitly define. You can't prove it doesn't exist (and, oddly, you can't really prove it does exist in naive set theory.) Again, describe a set that is not in ZF in naive set theory. "An arbitrary collection" is not a definition of a set.
Comment: Yes, this is exactly what I mean. It's not a set by definition. It can be a set if it happen to be the same as a set.
Comment: It is a set. Nothing in ZF says that if $S$ is a set, then it can be written is a certain form. It only says that if you can write out a certain form, it describes a set. That doesn't mean that form describes all sets.
Comment: And you still haven't defined a naive set of integers that is not a set in ZF...
Comment: But when I learn a bit more, it seems that ZF system is first order logic. Is that means the property can accept some thing like "for any", "there exists" ? If yes, supposing C is an arbitrary infinite subset in naive set theory, it's not a subset by definition according to the above discussion. If we define a property P(x)="exist b in C so that x=b", then this property will define C as a set in ZF system. Is this understanding of property correct ?
Comment: What is an "arbitrary infinite subset"? You haven't defined what that means. Is $6$ in your subset? Is $99$? You haven't defined a set. Is it the set of all even integers? The set of all primes? There are certainly sets that you can't describe in terms of the form $\{x\in\mathbb N\mid \phi(x)\}$ for some expression $\phi$, but that doesn't mean that the set doesn't exist just because you can't describe it. (You haven't described your "naive set," yet you insist it exists.)
Comment: In ZF system, a collection of integers is not a set before it's proved to be a set, right ? Now, if we randomly select one integer from every 10 to form a collection, how to prove it's a set ?
Comment: No, that is actually not true in ZF. Everything in ZF is a set, and nothing is not a set. There is no notion of "collection" in ZF. There is no way to even talk about whether a naive set is in ZF, because ZF describes *lots of versions of set theory.* It describes versions of set theory where the continuum hypothesis is true, for example, and versions where it is not.
Comment: A correction to what I said for an arbitrary collection of integers C, 'P(x)="exist b in C so that x=b"' is a property in ZF system. P(x) is not a property in ZF system because C is not an element of first order logic so P(x) is not a valid expression in first order logic.
Answer: Yes. $\sf ZF$ proves that the real numbers and $\mathcal P(\Bbb N)$ have the same cardinality -- or in other words that there is a bijection between the two.
But your question makes some unclear statements. "The power set in $\sf ZF$ has less elements than in naive set theory". Why? What? How? It is true that if $M$ is a countable model of $\sf ZF$ then it only knows about countably many subsets of $\Bbb N$ (or rather, the object that $M$ thinks is $\Bbb N$). But inside $M$ there are only countably many real numbers too (or rather objects that $M$ thinks are real numbers), and moreover, there is a bijection inside $M$ between those two objects.
If one considers working internally inside a model of $\sf ZF$ then the "naive arguments" translate immediately to be true.
Comment: Even in naive set theory, this is not a well-defined set.
Comment: If we select 1 integer randomly from every 10 integer, this forms a subset in naive set theory. Does the same collection forms a subset in ZF system ?
Comment: The question is what do you mean by "random integer". If the choice is a mathematical process, then when formalized internally to a model of $\sf ZF$ the result is a set of integers. Because this "process" is nothing more than a function, and the function is an element of the universe (or the model of $\sf ZF$) so it can be subjected to the same axioms that govern the rest of the sets. In particular axioms like replacement and bounded comprehension.
Comment: {3,11,24,31,45,59,...} is a subset in naive set theory. How to prove it's a set in ZF system ?
Comment: What does the $...$ mean?
Comment: A serial of integer, that forms a subset in naive set theory.
Comment: If you just want a set which includes $3,11,24,31,45,59$ then you can easily find a set like that.
Comment: This is an infinite serial. In naive set theory, there is no difference between collection and set. Any collection is a set.
Comment: I don't think the is naive set theory, I think this is extremely naive set theory. @AsafKaragila
Comment: @pierredeboussy: You seem to be misunderstanding how mathematics works, how set theory works, how naive set theory works. Perhaps it is worth to take a break, open up a couple of books and try to learn more about set theory and mathematics.
Comment: @pierredeboussy: To add on my previous comment. You haven't told me what the ellipsis mean. If you want "a serial of integers" then taking all the integers larger than $60$ as well the few that you mentioned gives you a well-defined set. You haven't clarified anything in your comment reply. You just seemed to have repeated previous arguments.
Comment: He failed in comments to the question to tell me what set he was talking about, too. He seems to think that, in ZF, the only subsets of the integers are of the form: $\{x\in\mathbb N\mid \phi(x)\}$ for one of the neccessarily countable formulae $\phi$. It's not an unnatural assumption, but I can't seem to dissuade him that the axioms don't say that.
Comment: @Thomas: Well, that's almost true. Because you can allow parameters. But then the subset can be its own parameter, which means that $\varphi(x,p)$ is just a simple formula $x\in p$.
Comment: Technically, those are the only sets you can prove exist, not that exist. @AsafKaragila
Comment: @Thomas: No, those are all the sets that exist. Again, we allow parameters, and these parameters are allowed to be the sets themselves.
Comment: Okay, I can't see how that gets you all sets.
Comment: @Thomas: Given any set which exists, $p$, consider the set $\{x\in\Bbb N\mid x\in p\}$. Of course this proof of existence is somewhat circular, but it's still valid.
Comment: Well, that's silly. Obviously an set that exists exists, but that is not a naive "definition," that is just a circular specification...
Comment: @Thomas: True. But we can still prove that "If $p$ is a subset of $\Bbb N$, then $p=\{x\in\Bbb N\mid x\in p\}$". Not a great proof, but it's there...
Comment: Yes, it just isn't helpful for answering his confusion, which is based on the difference between what can be *defined* as a set and the (possibility that) the model contains a lot more. I realize the idea that there is little difference between the naive notion of a definition and an application of specification. I even said above several times that in ZF, and subset of $\mathbb N$ is a set. That is not clearing things up for him...
Answer: 2 is the problem. In naive theory you can pick $\{x| \phi(x)\}$ - a collection of elements that satisfy some condition $\phi$ and call it a set. In ZF you cannot. However you can take $\{x \in X | \phi(x)\}$ - a collection of elements of some set $X$, that satisfy condition $\phi$. There is an axiom for this.


$M\to S^{-1}M$ is an isomorphism iff for all $W$ with $S^{-1}W=0$ all maps $W\to M$ are zero

Question: Let $R$ be a commutative ring, let $S\subset R$ be multiplicative and let $M$ be an $R$-module. Is the following true?

The localization map $M\to S^{-1}M$ is an isomorphism if and only if for all $R$-modules $W$ such that $S^{-1}W=0$, all maps $W\to M$ are zero.

The direction $\Rightarrow$ is easy: it follows from functoriality of localization. But I can't prove or disprove $\Leftarrow$. Note that the condition $S^{-1}W=0$ implies that all elements of $W$ are annihilated by some element of $S$.
Answer: I think $\mathbb{Z} \rightarrow \mathbb{Q}$ (so $R = M = \mathbb{Z}$ and $S = \mathbb{Z} \setminus \{0\}$) provides a counterexample.  Given an abelian group $W$, then $S^{-1} W = 0$ iff $W$ is torsion, i.e., every element has finite order.  Since $0$ is only element of $\mathbb{Z}$ of finite order, given a homomorphism $W \to \mathbb{Z}$, then every element must map to $0$.  Thus the hypothesis on the righthand side is satisfied, but the localization map $\mathbb{Z} \to \mathbb{Q}$ is not an isomorphism.
However, one can show the localization map is injective given the hypothesis on the righthand side.  Letting $\varphi: M \to S^{-1} M$ be the localization map, take $W = \ker(\varphi)$.  Given $m \in \ker(\varphi)$, then $\frac{ms}{s} = \frac{0}{s}$, so there exists $t \in S$ such that
$$
0 = t(ms^2 - 0\cdot s) = s^2 t m \, .
$$
Thus $S^{-1}W = 0$.  By the hypothesis, then the inclusion map $\ker(\varphi) \hookrightarrow M$ must be the zero map, so $\ker(\varphi) = 0$.
Comment: I don't get the part where you prove that $S^{-1}W=0$. (For instance, what is that $s$?) If $m\in\ker\varphi$ then $\frac m1=\frac 01$, that's all. (Do you assume somehow that $R$ has no unit? But then how define the canonical map $M\to S^{-1}M$?)
Comment: Thanks for your reply. I think that there should be a correct result close to what I formulated that keeps the condition that $M\to S^{-1}M$ is an isomorphism, but I can't figure out what it is. Could you help me?
Comment: @user26857 The OP doesn't seem to assume $1 \in S$, in which case one can define $M \to S^{-1} M$ by choosing any $s \in S$ and sending $m \mapsto \frac{ms}{s}$.  Maybe  assuming $1 \in S$ (a very mild assumption) would have made the proof a bit cleaner.
Comment: @user46225 I'll think some more about it and see if I come up with anything.
Comment: Thanks! By the way, I should have said so but I am assuming that $1\in S$, yes.


How can I solve quadratic equations using modular arithmetic

Question: How can I solve quadratic equations using modular arithmetic? E.g.
$$2x^2 + 8x + 2 = 0 \pmod{23}$$
N.b. I have changed the figures from those in my homework question because I don't want a solution I want to understand the process. Consequently the example I gave might not have solutions. For the example I am working from divide the LHS by 2.
Comment: I like that you changed the question so as not to accidentally solve your homework.  But note that 2 is a unit mod 23, so in fact you can legally divide both sides by 2, so this problem is essentially identical to the original one.
Comment: @AaronMazel-Gee Who would have thought that coming up with examples could be difficult :/
Answer: We have $2x^2+8x+2\equiv 0\pmod{23}$ if and only if $x^2+4x+1\equiv 0\pmod{23}$.
Now complete the square. We have $x^2+4x+1=(x+2)^2-3$. So we want to solve the congruence $(x+2)^2\equiv 3\pmod{23}$.
Let $y=x+2$. We want to solve the congruence $y^2\equiv 3\pmod{23}$.
There is general theory that, for large $p$, helps us determine whether a congruence $y^2\equiv a \pmod{p}$ has a solution, and even to compute a solution. But at this stage you are probably expected to solve such things by inspection.
Note that $y\equiv 7\pmod{23}$ works, and therefore $y\equiv -7\equiv 16\pmod{23}$ also works. We have found two solutions, and by general theory if $p\gt 2$ there are either $2$ solutions or none, so we have found all the solutions.
From $y\equiv 7\pmod{3}$ we conclude that $x+2\equiv 7\pmod{23}$, and therefore $x\equiv 5\pmod{23}$.
From $y\equiv 16\pmod{23}$ we conclude that $x\equiv 14\pmod{23}$.
Remarks: If our congruence had been (for example) $x^2+7x-8\equiv 0\pmod{23}$, there would be a bit of unpleasantness in completing the square, since $7$ is odd. But we could replace $7$ by, say, $30$, and complete the square to get $(x+15)^2-225-8$. So our congruence would become $(x+15)^2\equiv 233\pmod {23}$, or equivalently $(x+15)^2\equiv 3\pmod {23}$.
In general, if we have $ax^2+bx+c\equiv 0\pmod{p}$, it is useful to multiply through by the inverse of $a$ modulo $p$ to make the lead coefficient equal to $1$. There are a number of other helpful "tricks."
Comment: For existence, quadratic reciprocity. For construction, details depend on the type of prime, but there are fair to good algorithms.
Comment: What is the general theory that for large $p$, helps us determine whether a congruency $y^2\equiv a \pmod{p}$ has a solution and computes it?
Comment: What would one do differently if the given equation had odd terms? For instance, you can't divide by two the equation $2x^{2}-14x+19\equiv0\pmod p$ .
Comment: @Ido I think we need to move away from the idea of "dividing" by 2, and instead multiply by the inverse of 2. This will be congruent to (p + 1) / 2 for any odd p. So if p was 23 then its a matter of multiplying by 12 to effectively "divide" by 2.
Comment: I am late, but if the base is even, for instance 12, then there is no way to convert that odd coefficient to even, or is it?
Answer: Since $(2,23) = 1$, you pull $2$ out and 'cancel' it. Now complete the square.
At this point you will need to take the square root of $3$.
In the mod $23$ world, by Fermat's little theorem, you know that $3^{22} \equiv 1 \bmod 23$, So $3^{12}$ is most likely $3$. Try it:
$$ 3^{12} \equiv (3^3)^4 \equiv 4^4 \equiv (-7)^2 \equiv 49 \equiv 3 \bmod 23$$
Thus $\pm3^6 \bmod 23$ must be the square roots of $3$. That is $16$ and $7$ are square roots of $3$.
Now your equation looks like $x+2 \equiv \pm7 \bmod 23$. So the solutions are $14$ and $5$.
Answer: The proof of the quadratic formula proceeds by completing the square and then taking a square root.  Completing the square works as long as we can divide by 2.  So as long as we can divide by 2 and take square roots, the quadratic formula gives the roots of the equation.
If the modulus is odd (as in your case), we can always divide by 2.
Whether taking a (modular) square root is possible will depend on the precise equation.


An impossible distribution

Question: Some days ago another user posted a question which was something like this:
$$ A \sim U(0,4)$$
$$B \sim U(0,6)$$
$$A - B \sim U(-4,4)$$
The question was originally to find the distribution of A conditional on B. Immediately two famous users of the community spotted a problem and commented that it was impossible, then providing a hint (if I remember correctly) something like "Write $A = (A-B)+B$". Which today I still do not understand. I found proofs (actually one of these famous users posted some years ago the proof) which claimed that the difference of two iid variables will never yield a uniform distribution, but what about the conditional case? How do we know which distribution is possible and which one is not? If we have $B \sim U(0,4)$, is it now possible?
As part of my journey trying to understand why it was not possible, I tried to replicate the problem, but with discrete scenarios (My technical background is unfortunately not adept to the continuous case...).
In my mind, I am trying to find a joint probability mass where each difference have the same probability of occurrence, and the marginal distributions as well. In plain words, each diagonal sum the same, each row sum the same, each column sum the same.
First with a 3x3 matrix. We can get something like (it should be normalized to 15, otherwise it is not a distribution, but for ease of readibility let's keep it like that):
\begin{bmatrix}
0 & 2 & 3 \\
2 & 2 & 1 \\
3 & 1 & 1 \\
\end{bmatrix}
For a 4x4 matrix:
\begin{bmatrix}
1 & 0 & 2 & 4 \\
0 & 2 & 3 & 2 \\
2 & 3 & 1 & 1 \\
4 & 2 & 1 & 0 \\
\end{bmatrix}
For a 5x5 matrix:
\begin{bmatrix}
0 & 0 & 1 & 3 & 5 \\
0 & 0 & 4 & 3 & 2 \\
1 & 4 & 2 & 1 & 1 \\
3 & 3 & 1 & 2 & 0 \\
5 & 2 & 1 & 0 & 1 \\
\end{bmatrix}
The 4x6 matrix had indeed no solution (Is this related to the comment on $A = (A-B)+B$?)
So I got some questions from here. It seems like there is more than one solution for the 5x5 matrix, and for the other matrices. It makes me think that in the continuous case there may be as well more than one solution ie. for the case where both, A and B are uniform (0,4) and their difference is uniform (-4,4). But how can I find one of those solutions that fit these conditions? How do I identify which conditions are just impossible and which ones can be solved?
Any thoughts on this topic are highly appreciated.
Comment: References to the original questions will be helpful
Comment: The solution is immediate from [general solution sum of two uniform random variables aY+bX=Z](https://stats.stackexchange.com/questions/228421/general-solution-sum-of-two-uniform-random-variables-aybx-z)
Comment: @whuber If I understand what you're saying: there actually are possibilities other than the case when they're affinely related. Let's stick to the case where we want both $A$ and $B$ to have $U[0,4]$ distribution. If you take $A\sim U[0,4]$, and set $B=2-A$ for $A<2$ and $B=6-A$ for $A>2$, then also $B\sim U[0,4]$, and $A-B\sim U[-2,2]$. Via a similar splitting into $k$ cases, I think you can get $A-B\sim U[-4/k, 4/k]$ for any positive integer $k$. Can you get $A-B\sim[-c,c]$ for other $c$?
Comment: @Kjetil How?  The issue here is that $A$ and $B$ are not assumed independent. // The more challenging situation is where $A$ and $B$ have equal expectations.  In that case, consider the situation where $X$ has a uniform distribution on $[-1,1]$ and set $A=2X+2,$ $B = 2-2X.$  Clearly all three of $A,$ $B,$ and $A-B$ have uniform distributions.  I believe this is essentially the only such example: that is, when all three are uniform, $A$ and $B$ must be affinely related.
Comment: @Max - perhaps https://stats.stackexchange.com/questions/640018/finding-probability-involving-dependent-random-variables
Comment: @James Thank you, that is correct.  I recall posting essentially that set of examples some ten years ago but haven't been able to find it.  My search did turn up related threads at https://stats.stackexchange.com/questions/125360, https://stats.stackexchange.com/questions/257779, and https://stats.stackexchange.com/questions/514443.
Answer: If the postulated distributions held, then $E(A) = 2, E(B) = 3$ while $E(A - B) = 0$, can you see the problem here? (hint: linearity of expectation.)
Comment: @OscarFlores "*I was confused and thought dependent variables could have different expectation.*" -- **NO**.  The beautiful thing of the expectation is that the linearity alway holds regardless r.v.s are dependent or independent.
Comment: OscarFlores @whuber's comment to the question gives the solution (up to a zero probability event) to your $A,B\sim U(0,4)$ and $A−B\sim U(−4,4)$ follow-up: for $A \sim U(0,4)$ take $B =4-A \sim U(0,4)$ and so $A-B=2A-4\sim U(-4,4)$.
Comment: At a quick glance, this seems a quick and sound rebuttal to the confusion. BTW, welcome after a long time.
Comment: Amazing, thanks for the note. I was confused and thought dependent variables could have different expectation. I found also a link with more details on it. https://stats.stackexchange.com/questions/130067/how-does-one-find-the-mean-of-a-sum-of-dependent-variables
Comment: Do you have any other thoughts/ hints for how to find a possible distribution when the Expectations match? (For example the case where $A,B \sim U(0,4)$ and $A - B \sim U(-4,4)$). Or in the other case, how to find that there is no such distribution?

An impossible distribution


Usando recursão para contar o numero de vezes que um numero aparece numa lista

Question: Eu criei um algoritmo recursivo para contar o numero de vezes que um certo numero n aparece numa lista ls. Parecia funcionar se a lista ls é bastante pequena (entre 10 e 100), mas se é grande, tipo 1000, parece não funcionar mais: entra numa recursão infinita...
Alguém consegue perceber porquê?
def _count(n, ls, index):
    if index < len(ls):
        if ls[index] == n:
            return 1 + _count(n, ls, index + 1)
        else:
            return _count(n, ls, index + 1)
    return 0

def count(n, ls):
    """Counts how many times n appears in the list or tuple ls."""
    return _count(n, ls, 0)

if __name__ == "__main__":
    from random import randint

    for i in range(10):
        ls = [randint(0, 10) for _ in range(1000)]  # modifica 1000 para 100 para funcionar
        print("List:", ls)

        r = randint(0, 10)
        print("Counting number:", r)

        c = count(r, ls)
        print("Counted:", c)

        if c != ls.count(r):
            raise Exception("Something is wrong with the implementation of count...!")

Answer: Na maioria das linguagens de programação, cada chamada de função precisa de alguns bytes da pilha de execução e em geral existe um limite máximo para o tamanho da pilha. Num computador de Desktop hoje em dia esse limite normalmente é da ordem de centenas de milhares de chamadas mas em Python o limite é bem baixo de propósito:
> import sys
> sys.getrecursionlimit()
1000

Você pode aumentar um pouco o limite com sys.setrecursionlimit mas no fim das contas o ideal é usar um algoritmo com loop, que consuma uma quantidade constante de memória, ao invés do algoritmo recursivo, que consome uma quantidade de pilha diretamente proporcional ao tamanho da entrada. Em linguagens como Python é melhor usar algoritmos recursivos apenas quando o consumo máximo da pilha crescer mais devagar. Por exemplo, muitos em muitos algoritmos recursivos o tamanho máximo da pilha é proporcional ao logaritmo do tamanho da entrada.

Dito isso tudo, existem algumas linguagens de programação que garantem que chamadas recursivas em posição de cauda não gastam espaço adicional na pilha.
Uma linguagem que garante a otimização da recursão de cauda é Lua. Em Lua podemos escrever um algoritmo de count que nunca estoura a pilha.
function _count(x, xs, i, nfound)
    if i <= #xs then
        if xs[i] == x then
            return _count(x, xs, i+1, nfound+1)
        else
            return _count(x, xs, i+1, nfound)
        end
    else
        return nfound
    end
end

function count(x, xs)
    return _count(x, xs, 1, 0) -- Em Lua os índices começam em 1
end

Tive que alterar o seu algorítmo um pouco porque a chamada recursiva em 1 + _count(...) não está em posição de cauda. Mas o importante que eu queria mostrar é que em algumas linguagens é possível sim escrever algoritmos recursivos que não estouram a pilha.
Comment: O limite de recursão é para todas as funções do seu programa e isso inclui o main, a chamada para `count`, a chamada para `len` em `index  < len(xs)`, etc. Se você der `sys.setrecursionlimit` prum número pequenininho dá pra ver isso claramente.


Firebird on FreeBSD

Question: Does anybody know on how to install Firebird 2.0 on FreeBSD 7.2? I have updated the ports and the kernels, but when I try to do a "make install", I get the error:
Please do not build firebird as 'root' because this may cause conflicts with SysV semaphores of running services.
Any ideas?
Comment: Are you compiling as root?
Answer: Thanks for the reply, guys. I finally figured it out. Here's how:
make -DPACKAGE_BUILDING
make install
that finally got it compiled and installed.
Comment: that's definitely a bug in the port.  consider submitting a PR.
Answer: It might be worth your while to download the official source from firebird's site instead of using the FreeBSD port. Have a look here at someone who posted about compiling firebird on freebsd. There may be a macro that needs to be included to get around your problem. As for the make install, I would do this as a normal user first and see what happens. If it fails, then root around looking at the documents to see if there is a macro for compiling and enable it, then recompile the source, and try again.
Hope this helps,
Best regards,
Tom.
Comment: Thanks for the reply, Tom. I finally figured it out. Here's how:

make -DPACKAGE_BUILDING
make install

that finally got it compiled and installed.
Answer: I see some patch on Firebird Tracker for Firebird 2.5
Perhaps, it can be useful for you.

Firebird on FreeBSD


Connecting HumongouS.io to MongoDB on Openshift

Question: I am having issues connecting HumongouS service to my MongoDB on Openshift free plan.
First I get the "server refuse to connect " error which I understand I need to whitelist the ip address given by HumongouS. But it does not look like openshift allows this functionality.
Second, there is the SSL option but not sure how do I set up MongoDB  to use SSL in Openshift.
Please help.
Thanks.
Answer: Go to Security Groups - Change Inbound settings to the I.P.address with which you want to connect
By this, it enables the EC2 instance - Mongo server to allow connection with the given IP address.
Whitelisting means allowing Custom TCP connection to Port 27017 with the IP Adddress (Inbound connection)


Clustering with Self Organizing Maps including time, date and month as attributes

Question: I am about to start up a project on pattern recognition in a highdimensional dataset holding information on transactional salesdata for a company. In that manner I have decided to use the method of SOM to find clusters in the data.
However, I have wondered how it is possible to include the possibility of finding some kind of seasonal pattern? For instance, weekdays are, as far as I know, often modelled as binary. In this way it wont be possible to determine whether there if some certain customer behaviour in weekends, months, etc.
Do there exist a more sophisticated way of including dates, months and time?
Kind Regards
Answer: Assuming that the transactional data is timestamped to a resolution of (say) one second, you will need to bin it into to intervals of hours / days / weeks / months etc in order to form time-series vectors for each product / sales location / salesperson etc.
Seasonal behaviour is most often detected using auto-correlation and the use of a Correlogram. The use of an SOM will help you find clusters of similar sales patterns. Be careful how you normalise your data, though: whether you are more interested in the shape of the sales over time or the size of the sales over time will determine how you should normalise.
Answer: This was exactly what I was searching for
https://ianlondon.github.io/blog/encoding-cyclical-features-24hour-time/
Comment: This is being automatically flagged as low quality, probably because it is so short.
At present it is more of a comment than an answer by our standards. Can you expand
on it? You can also turn it into a comment.


Jetty WebSocket performance

Question: On the Jetty page https://webtide.com/why-choose-jetty/ in the Throughput section it says:

We have designed Jetty for scalable performance under realistic loads of many simultaneous connections and we can achieve excellent results with many tens of thousands of HTTP connections and hundreds of thousands of simultaneous WebSocket connections.

So, what it is so different in the implementation of these 2 protocols, how Jetty can handle only tens of thousands of HTTP connections and in magnitude more WebSocket connections? Are WebSocket connections less expensive? How it can be? If both HTTP and WebSocket implementations utilize Java NIO (just guessing), why there is a difference in the performance then?
Answer: You can only really compare a webSocket connection to an HTTP connection if you measure something they have in common such as sending messages per second.  If you are doing that, then a webSocket has a huge advantage in both bandwidth and in network traffic because it is an already established connection whereas an http connection must be re-established from scratch for every message and establishing that connection for every message has a cost.
Both a webSocket connection and an http connection have a bunch of overhead in order to set up the connection.  There is, at a minimum, the TCP overhead of establishing a TCP connection which looks a bit like this:

Credit to this article
In addition a typical HTTP request will also include some overhead such as cookies and other headers.
The webSocket connection however, does this only once and then keeps the connection open so future messages can just be sent over that connection directly.  The HTTP connection is temporal, it is connected, data sent, response received, then the connection is dropped.  If you need to send another request, another HTTP connection must be established from scratch.
While the stateless nature of HTTP connections is great for conditions where a client only occasionally makes a request and often makes requests to different servers, it is NOT ideal if a given client is making lots of requests to the same host.  In that case, it's much more efficient to establish one connection, keep that connection open and then just send messages over that existing connection when needed.
Given the general context of the article you mentioned and the other aspects it discusses, it appears that this difference is what that article is referring to when mentioning the difference in scalability.
Here's another related answer: Ajax vs Socket.io
Comment: Yes, but in the article I've mentioned they talk about *simultaneous* HTTP and WebSocket connections, so if it is not about data bandwidth, as you mention, so why they could establish much more simultaneous WebSocket connections comparing to HTTP, as the underneath both HTTP and WebSocket connections rely on TCP and Jetty still has to handle and process each request from the client be it HTTP or WebSocket.

Jetty WebSocket performance


Bottom margin Android LinearLayout item in XML ignored, but works in Java code

Question: I am developing an app with an activity with member reactions on a hike event. The reactions are the yellow "balloons" which are made using a LinearLayout. Each item is constructed from a XML file (listitem_deelnemerreactie.xml) which defines the layout for a reaction item. The top level of this layout file is a LinearLayout my itself.
I want some spacing between the separate elements, as well as some right margin. The most straightforward way to so this should be: setting a bottom and right margin on the top-level LinearLAyout element of the listitem_deelnemerreactie.xml  layout file.
But setting the bottom margin on the LinearLayout has no effect on the vertical spacing, though the right margin does have an effect.
The only way to be able to set a vertical margin appears to be: setting is in the Java code, after attaching the inflated view to the container.
See the two images for the effect and the code.
Though setting the margins in the code is a working workaround, I still think it is strange this cannot be achieved in the XML. Why is the bottom margin attribute ignored while the right margin is not?
Any ideas?
Comment: have you ever solved this? I am having the same issue.
Answer: Have you tried to set an android:padding="10dp" for example on your elements to spaced them ?
Comment: Padding works on the internal of an item and does not set outer spacing. The padding is set in the drawable which I use as a background to add some space between the text and the border. It is not meant to set spacing between the separate elements.


Machine learning forecasting / asking for input parameter

Question: I've been working on a weekly web traffic forecast based on time series. The primary business driver is marketing investment (spending)
I have historical spending data but not future spending data. In this case, how can future web traffic be predicted?
I'd like to have an input request switch where I could change the cost input to see future traffic. (If you spend this much, you will receive these many visits; if you do more, you will get more, and vice versa.)
Answer: You can either use your explanatory variable(s) or not.
If you do not want to use your predictors, you can use simple time series models, like exponential smoothing, ARIMA (both not recommended if you suspect intra-yearly seasonality - better to use an STL method for such "long" seasonalities), or many others.
If you do want to use your predictor, you can use simple methods like regression with ARIMA errors, or alternatively run a regression on the predictors, then fit an exponential smoothing model to residuals. (See above on potential seasonality.) Or do a fancy ML method, like Deep Learning, transformers, or Boosting... but quite honestly, I would go for low hanging fruit like a regression first.
However, in the latter case you will need to feed some future predictor values into the model to get a forecast. You can have the user specify future marketing spend, or use any "reasonable" number, like the historical mean spend, or the last spend, or indeed the result from forecasting the spend itself.
This thread contains some references on forecasting.
Comment: To see the result for a predictor value of 1450, just feed that value in exactly as you fed in 800 before. To find thresholds, best to feed in multiple "reasonable" values and fit, e.g., a spline. (Of course, if you just have a simple linear regression, there won't be a threshold.)
Comment: Thank you so much for sharing the references. I would like to use predictors but kind of real-time (that if needed I can change the spend values and the forecast changes on the spot)
Comment: No problem. Use historical spend and demand to fit a model. Persist the model. Then evaluate the model using the supplied spend (or other predictor) values. Model evaluation typically is many orders of magnitude faster than model training, regardless of what kind of model you use.
Comment: Is it called (what-if analysis?), so my model is ready and I have a MAPE of less then 10. But I need to change the predictor value for future. how to extend from baseline model to real time forecast? any source to study that? Thak you in advance
Comment: I'm sorry, but I don't quite understand what your problem is. Is it how to feed the future predictor values into a specific model? That would be a coding question, which is off-topic here, but with an MWE you could ask at StackOverflow in the appropriate tag.
Comment: I will eleborate a little, so model is prepared, futue predictor values are also applied to the model and we have the result. Now my stakeholders would like to the tweek the future values so for example for week 5 the future predictor for cost = 800 and te visit output is 40000, now my stakeholder would like to change the cost to let's say 1450 to see what could be the visit forecast. They actually want to know the threshold point after which even if they spend more there wont be anyincrement in visit number.


Distance between polynomials in euclidean space

Question: In Euclidean space $R[x]_{\leq n}$consisting of polynomials with scalar product i need to find distance between polynomial $f=2$ and subspace of polynomials with zero constant term.
How can i approach to this problem?
Comment: Did you mean a polynomial of degree 2? Or the constant 2?
Comment: If $f = 2$ denotes the constant polynomial $2$, then it belongs to the subspace mentioned. So, I think the distance from the subspace is $0$.
Comment: Remark: the 'subspace of polynomials with non zero constant term' is not a subspace
Comment: @twnly, constant
Comment: @CharlesMadeline, yes, sorry, i edited. With zero constant term
Comment: Given a subspace $U$ and element $v$ of the vector space $W$ not in $U$, the distance of $v$ from $U$ is the length of $v-t$, with $t\in U$ chosen so each $s\in U$ satisfies $(s-t)\cdot (t-u)=0$.
Answer: If your inner product is defined by $\langle p,q \rangle=\int_0^{1} p(x)q(x)dx$ then the answer is $2$. To see this let $f_n(x)=2n(\frac 1 n -x)$ for $x <\frac 1 n$ and $0$ otherwise. it is easy to see that the distance between $f_n$ and $2$ tends to $2$. Now Weirstrass approximation shows that required distance is also $2$.  [Some details:  the distance is obviously $\leq 2$ so we only have to produce some sequence in the subspace whose distance to $2$ tends to $2$. By Weirstrass approximation any continuous function vanishing at $0$  can be approximated uniformly by polynomials vanishing at $0$ and I am using this for $f_n$].


Salvando Imagem base64 com ADODB.Stream

Question: Após converter uma image para base64 utilizando html5 e canvas, envio a string para o ASP Clássico e lá devo salvar esta imagem em um determinado diretório.
Estou utilizando o seguinte código:
    base64String = Trim(Request.Form("base64"))

    Set tmpDoc = Server.CreateObject("MSXML2.DomDocument")
    Set nodeB64 = tmpDoc.CreateElement("b64")
    nodeB64.DataType = "bin.base64"
    nodeB64.Text = Mid(base64String, InStr(base64String, ",") + 1)

    set bStream = server.CreateObject("ADODB.stream")

    bStream.type = adTypeBinary

    call bStream.Open()

    call bStream.Write(nodeB64.NodeTypedValue)

    call bStream.SaveToFile("C:\fotos\imagem.jpg", adSaveCreateOverWrite)

    call bStream.close()
    set bStream = nothing

O cliente está desta forma:
<form method="post">
    <input type="text" id="base64" name="base64" value="" />
</form>
<canvas id="c"></canvas>

O objetivo é redimensionar a imagem, gerar o código base64 e enviar para o servidor. No servidor eu converto o base64 em imagem e salvo ela em um diretório específico.
Está tudo certo se não fosse pelo fato de estar gravando uma imagem preta.
Percebi que a imagem original tem 1.406 bytes e após salvar fica com 691 bytes.
Alguém já passou por isso?
Grato.
Comment: Já que os arquivos são pequenos, tem como postar um hex dump do arquivo original e do arquivo corrompido?
Comment: Seu files[0].type ta retornando o que no javascript quando tu executa?
Answer: Já tentou trocar
Server.CreateObject("MSXML2.DomDocument")
Set nodeB64 = tmpDoc.CreateElement("b64")

por
Server.CreateObject("Msxml2.DOMDocument.3.0")
Set nodeB64 = tmpDoc.CreateElement("base64")

E em último caso
bStream.charSet = "us-ascii"
bStream.position = 0

Tenta:
' type binário
bStream.type = 1

call bStream.Open()
call bStream.Write(nodeB64.NodeTypedValue)

' muda type pra text
bStream.position = 0
bStream.type = 2
bStream.charSet = "us-ascii"

call bStream.SaveToFile("C:\fotos\imagem.jpg", adSaveCreateOverWrite)

Comment: Sem sucesso. A primeira proposta gera o mesmo resultado e a última gera um erro.
Comment: @Onaiggac coloquei outra solução, fique atento a mudança dos type durante o processo
Answer: Onaiggac eu tentei usar o seu código, porém este servidor que eu uso não é tecnologia microsoft e faz muito tempo que não trabalho com servidor asp-classic, mas pelos poucos testes que fiz o problema parece ser no front-end e não no back-end.
De qualquer forma eu modifiquei o código (para funcionar no meu servidor local), troquei isto bStream.type = adTypeBinary por bStream.type = 1 e  bStream.SaveToFile("C:\fotos\imagem.jpg", adSaveCreateOverWrite) por bStream.SaveToFile("C:\ASP\www\imagem.jpg", 2), não acho que faça diferença, mas não custa testar
O código completo ficou assim (funcionou perfeitamente):
save.asp
<%
base64String = Trim(Request.Form("base64"))

Set tmpDoc = Server.CreateObject("MSXML2.DomDocument")
Set nodeB64 = tmpDoc.CreateElement("b64")
nodeB64.DataType = "bin.base64"
nodeB64.Text = Mid(base64String, InStr(base64String, ",") + 1)

set bStream = server.CreateObject("ADODB.stream")

bStream.type = 1

call bStream.Open()

call bStream.Write(nodeB64.NodeTypedValue)

call bStream.SaveToFile("C:\ASP\www\imagem.jpg", 2)

call bStream.close()
set bStream = nothing
%>

Funcionou perfeitamente, usei um HTML de exemplo:
form.asp

    <meta http-equiv="Content-Type" content="text/html; charset=ido-8859-1">
    <title>Test</title>

<form method="POST" action="save.asp">
    <textarea name="base64">/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCADAAQADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWiVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADT0lEQVR42s2Xy08TYRTFDRHCw8LwstCWjlNeooBt59WZYWpfGKMsJBGIC3RjIrvCyr1L/xZlR0hcsqIrSEha6IdUMSL/Bd6TwIT0m5kNGWXxC7f33HPPbYeScOfi4uK/crsPkJ/IpppStwFqfubmPt9FhmbsZ+eyDYCan7m5z3eRbdknpWLpEKDmZ27uc4rUTMqkj2tbSSp/6GcdvaydPS6VSlWAWk2rIdLOlZRy5gZpv9186MGH3chAFneAJmv7ZsZszFlzx4Zu/NAUrTX/NF8pFApVQPXu82fPw6Q16F0xN6Dpii40+SrU64eG3chAFncADZxks9nDXC5XtUyLZbTMy3K5vEJLdsFGeWPZMqz3tIRhxg1oNLOyXl5fvvKhtjLWuysfMpDFHZDRM8e2bVeBaZpHqqzWivni2729vUekJ+YL8290VcdxR5hxAz765Or5bH5t8+um4uVDFv8IVI3RgiowDKNGJian5EY6mf4JUOuazqBhxg1oNFNX0oqvD1ncAXJaZrquV6/QNK2mKuqRIisMoEYPWjOcT/X3IYv/FiRTTFGU6r8AWdwBszOzLJlM1qieCBJkIIs7YPrxNCPqqIMEGQR/wNTDKQYCCvbMcYTJ8UkGAgr2zHGE0dExBvzMiXjCToijp8T55etu1MRZE+fQ+B18jiNIDyQGvMKlEWlVikuHkiidEA30bMMWUMN3HfSKuWKI38PnOEJ8JM6Am0mMit1iTPw1Njq2E4/Ha9fmWogJUKvVZsbHx7+Joniw9HrpAzR+F5/jCLFojAE3U87OhUqF0setra3ZaDR66DY3Ehn5hP7iq8U1es29e68cRxgeGmbAw9hC9M4X5wWaqTfPxYZiUmQociLGxR167fbsPXMcITwYZsDPnFEzPW5z9HjK4fvh7/RH5rOH1zPHEQb6BxjwM6tpVXCbi0ViX9AzdGPVw+uZ4wh9Qh8DfmY5JQu9Qi83N9g/eIC+bdmmh9czxxF6unsY8DI2/9Y39ccue+3+fj7HEUJdIQZQBwef4whdnV0MBBTsmeMIHe0dDAQVzOfwj6DS1tpWD3WGToV7wlkQYDcykMUdsPBiYYUuq7Teba3TEAsC7EYGsrgDiC4iQUwETAJZt+a/47+rJ1iPKnkHqgAAAABJRU5ErkJggg==</textarea>
    <input type="submit">
</form>

Se este código funcionar para ti então realmente o problema é no front-end e você deverá editar a sua pergunta e adicionar o html e javascript (acredito que seja algo com canvas que você esteja manipulando)
Atualização
Efetuei o teste, o problema ocorre quando você envia data:image/jpeg;base64, junto, você deve usar String.replace para remover isto e deixar apenas o base64, o seu código deveria ficar algo como:
$("#base64").val(c.toDataURL(files[0].type).replace(/^data[:]image\/[a-z]+;base64,/, ""));

Um exemplo funcional seria:

    <meta http-equiv="Content-Type" content="text/html; charset=ido-8859-1">
    <title>Test</title>

<form method="POST" action="save.asp">
    <textarea name="base64" id="base64"></textarea>
    <input type="submit">
</form>

<canvas id="c"></canvas>

Atualização 2
Apesar de não estar no escopo da pergunta, o autor detectou o problema, era com o FileReader. O FileReader necessita do evento onload antes de iniciar o envio para o servidor, um exemplo de como deve ficar o código:
JavaScript:
function getImage() {
    var file   = document.getElementById("file").files[0];
    var reader = new FileReader();

    reader.onload = function(e) {
        var img = new Image();
        img.onload = function() {
            resizeImage(img);
        };
        img.src = reader.result;
    }

    reader.readAsDataURL(file);
}

function resizeImage(img) {
    var base64 = document.getElementById("base64");
    var c = document.getElementById("c");
    var ctx = c.getContext("2d");
    ctx.drawImage(img, 0, 0);

    c.width = 50;
    c.height = 50;

    var ctx = c.getContext("2d");
    ctx.drawImage(img, 0, 0, 50, 50);

    base64.value = c.toDataURL("image/jpeg").replace(/^data[:]image\/[a-z]+;base64,/, "");
}

window.onload = function() {
    document.getElementById("file").onchange = getImage;
};

HTML:

<input type="file" name="file" id="file" accept="image/*">
<form method="POST" action="save.asp">
    <textarea name="base64" id="base64"></textarea>
    <input type="submit">
</form>
<canvas id="c"></canvas>

Comment: Adicionei o código front-end que estou utilizando. Alterei meu código back-end como o proposto mas o resultado foi o mesmo.
Comment: @Onaiggac atualizei a resposta, veja se lhe ajuda.
Comment: A linha Mid(base64String, InStr(base64String, ",") + 1) já tem o propósito de remover aquele início.
Comment: Consegui resolver. A rotina js e asp estavam todas certas, porém eu executava a rotina js num momento errado. Eu tenho que executar dentro do método onload do objeto FileReader. Vou considerar sua resposta como certa por ter colocado um exemplo completo e funcional.
Comment: @Onaiggac Otimo trabalho, parabéns por ter encontrado o problema sozinho e obrigado pela gratificação. Note que eu editei a resposta.

Salvando Imagem base64 com ADODB.Stream