Multivariate logistic regression R

Question: I have 6 binary response variables, and I wish to explore what variables can predict these binary outcomes.
glm(cbind(Y1,Y2)~X1+X2, data = dt) doesn't do the work. It gives me an error: eval(family$initialize) : y values must be 0 <= y <= 1, although all variables have been coded into 1 & 0.
Also, both glm(Y1~X1+X2, data = dt) & glm(Y2~X1+X2, data = dt) work. This means data are in the right class
This answer suggested to use glmer(), but glmer() gives me an error saying that random effects were not specified, and I don't need random effects
This answer didn't provide a way to do multivariate logistic regression
Both Bayesian & frequentist methods are welcomed
Comment: I think this is an R usage/coding problem; outcome must be something like `cbind(Y1,Y2)`, not with a plus sign. See the help page for `cbind()`. Also see the help page for `family()`, method 3 for binomial regression : outcome can be "a two-column integer matrix: the first column gives the number of successes and the second the number of failures." Coding per se is off-topic here. If there's a statistical issue beyond coding, please post a new question.
Comment: Welcome to Cross Validated! Does it have to be logistic regression, or would multivariate probit regression work? If not, why must you work with logistic regression?
Comment: @Dave thanks for the reply, Dave! Based on my limited knowledge, I think these two can achieve the same goal, so yes it would work!
Comment: Then there are R packages available for multivariate probit. I don't think I've used [this](https://cran.r-project.org/web/packages/mvProbit/mvProbit.pdf) one, but it looks like it does the trick.
Comment: @Dave thanks for the help Dave. I'm currently trying to make the package work. I was wondering what I should do after finding a significant predictor for the responses. Do I run 6 univariate probit regressions on the significant predictor only?
Comment: It's also not completely clear what you mean by "6 binary response variables." Are these from 6 different types of binary responses, or do you have a single outcome that can be in exactly 1 of 6 categories? If the latter is the case, you should be using multinomial or ordinal regression. See [this page](https://stats.oarc.ucla.edu/r/dae/multinomial-logistic-regression/) or [this page](https://stats.oarc.ucla.edu/r/dae/ordinal-logistic-regression/) for those methods.
Answer: For a true multivariate logistic regression with multiple binary outcomes, any of which can have a "success" for any one observation, you can use generalized estimating equations to estimate and take into account the correlations among observations. You effectively get a set of univariate models, but the correlations are taken into account for inference. The R geepack package provides tools for that.
That is typically done for multiple binary observations over time for the same set of individuals, but the basic principle applies for any set of potentially correlated outcomes.
This paper illustrates use of geepack with clustered binary outcomes.
The suggestion you saw to use glmer() might have envisioned using the outcome type as a predictor in a logistic model (interacted with the predictors of interest) so that you fit the log-odds of a "success" in part as a function of the outcome type. In that case you would want to include the individuals or cases within which you think there are outcome correlations as random effects. There is no direct handling of multivariate binary outcomes in glmer(), so far as I know.
If instead you have 6 potential mutually exclusive outcomes, only one of which can be a "success" at any one observation, you need to use multinomial or perhaps ordinal regression.

Multivariate logistic regression R


Method for constructing large data set from smaller data set?

Question: Is there a method for creating a large data set from a smaller one?
I have a data set of anthropometric variables (e.g. stature, leg length, arm length and so on)
So I have 7 variables and 1774 samples. From this data I would like to create a much larger data set of the same 7 variables but with 100,000 samples.
I know there is an "almost" linear relationship between stature and the other body-measurements but I want random variation in my data.
I am using Python, and I have looked at PCA, multivariate analysis and stuff like that. Here's an illustration of my problem:

Edit:
Here is additional information.
The data contains 7 variables that describes the dimensions of 1774 people. I want to create dimensions of random 100,000 people who are different from each other but still remains realistic.
I tried this method, but cannot figure out how or whether it is possible to generate realistic data using this.
I would assume that the variables are normal distributed.
I know that variable "2-7 correlates" well with "variable 1" which is stature.
Comment: I would turn this round. If there were a good reasoned way to do this, it would need to be in every introductory course and text, as having a small sample and wanting a bigger one remains one of the most common needs. If and only if you have grounds for thinking that your data are close to a brand-name distribution does it make sense to simulate from that distribution. But then it is not your dataset legitimately made bigger, but just a bigger dataset constructed by analogy.
Comment: Yes I would like to generate fake data, but it has to include the correlation between body-segment-lengths, to insure realistic people.
Comment: @KasperRasmussen if so, check my edit - I provided an additional link to thread describing how to do it with examples and code in R. I don't go into further detail so not to duplicate it.
Comment: What is the purpose of creating the larger dataset? Will you use it for inference or for something else?
Comment: The data describes the dimensions of 1774 persons. I want to generate dimensions for 100,000 different sized persons.
Comment: @KasperRasmussen it is not clear for me how would you like to use KDE for generating your data? I also start to wonder if your question is about generating "fake" (simulated) data that is *similar* to your sample?
Answer: If you oversample your sample you would have the same data. Say your sample is: $1,2,3$ and you sample $2N$ values out of it and get: $1,1,2,2,3,3$ - now if you compute some statistic on it (say a mean), then you'll get the same result for both since they contain the same observations appearing with the same probability. Thing that could change is p-value because of sample size, but this would be p-cheating.
If you want to approximate the unknown distribution of your data, then one thing that could be done is to use bootstrap, i.e. sample with replacement $N$ out of $N$ cases $R$ times and use this data to approximate the distribution of your sample. If there is linear relation and you are interested in learning about the distribution of the "variation" around it, then you could use bootstrap in different fashion: fit the linear model and then sample the residuals, so to approximate the distribution of residuals.
You could also conduct a simulation that generates a sample that resembles your data. For example, to generate correlated variables given some covariance matrix you can sample from Multivariate Normal distribution - for learning more on this check this thread.
If the sample size is a problem, then consider using a statistical method that is robust to small sample size - for example Bayesian simulation-based statistical methods often work quite well with small samples.
Comment: If they sub-perturb, make small perturbations of each point so that it is substantially within its distribution, that can help.  They could randomly pick two points and average them, or weighted average them, and use that.  There are about 10 unique methods to estimate central tendency and any of them could be used, not just the classic mean.  You could randomly sample multiple points and get their geometric mean.  One of the risks there is if you sample between two modes, then you put something "out in the boonies" that shouldn't be there.
Comment: Hey @Tim. I have tried to create a co-variance-matrix from my data so I could use the Cholesky decomposition.
I tried to use numpy.cov(data), but some of the eigenvalues were negative. Is there another method, LU decomposition?


Fixing a cycling image background behind main div and floating menu?

Question: I am curious how I would go about keeping a cycling image background still behind the main content while one scrolls the page. Here is a link to the project itself. Feel free to use any of the code you want on your project, including the fixed header menu. I've been working on this template for a while, now.
http://www.famebyname.us/neep.html
Thanks in advance.
Answer: You can do this by using background-attachment: fixed.
Check out this fiddle or the following snippet for a visual example.
Snippet:

body {
  background-image: url(https://support.files.wordpress.com/2009/07/pigeony.jpg?w=688);
  background-repeat: no-repeat;
  background-attachment: fixed;
}

div {
  width: 100%;
  height: 250px;
  display: block;
  opacity: 0.6;
}

#a1 {
  background-color: black;
}

#a2 {
  background-color: green;
}

#a3 {
  background-color: red;
}

#a4 {
  background-color: blue;
}

#a5 {
  background-color: yellow;
}

#a6 {
  background-color: violet;
}

#a7 {
  background-color: purple;
}


Out of Band Access - what's your favorite Terminal / Console Server

Question: I am looking for a 32 or 48 port Console Server for access to routers, switches console ports in our Colo - any recommendations?
The devices it would connect to are mainstream (Cisco, HP, NetApp, etc).
Be nice if it supported RADIUS, encryption, etc.
I like this one from Black Box
http://www.blackbox.com/Store/Detail.aspx/Value-Line-48-Port-Console-Server/LES1148A
But ValueLine makes you buy a dongle for each connection type ($15-20) which can get pricey.
http://www.blackbox.com/Store/Detail.aspx/Cicso-Compatible-Console-Port-Adapter-for-Value-Line-Console-Servers/LES1116A%C4%82CISCO
But this may be required for any Console Server.
Thanks,
Rob
Comment: You can find rollover cable adapters for much cheaper than that.  Shop around, or consider making your own rollover cables (at custom lengths, too!
Answer: We've looked at Raritan devices for this.
We use Raritan KVM switches and they've always worked well for us.  We don't have enough serial devices in one location that we need a serial switch, so we've only ever demo'd a Dominion SX as a test, but it was pretty simple to get it up and running.  Raritan's KSX II does both KVM and serial switch in one unit.
Answer: Jeremy Stretch has been using an Opengear CM4116 console server for his Packetlife Community Lab for some time now.
The lab faq gives some informations about the console server (forbidden commands, send a break), and this link explain how the console server is linked to the lab server, which handles AAA (radius server).
Opengear also gives the source code of their products, if you want to customize your console server.
For my personal needs, I went the cheap way and built my own with a computer, three pci cards w/ 4 serial ports on each, and conserver. This works well and is easily expandable.


Missing piece for combinatorial proof

Question: I am working on a combinatorial proof given that $n-k$ is divisible by $g$ (that is $mod(\frac{n-k}{g})=0$). I am missing a piece for this step:
$\frac{k-1}{n-1}{n+\frac{n-k}{g}-2 \choose n-2}+\frac{k+g}{n}{n+\frac{n-k}{g}-2 \choose n-1}=\frac{k}{n}{n+\frac{n-k}{g}-1 \choose n-1}$
Comment: The correct way to state what you mean is $(n-k)\mod g=0$ or some prefer $n-k \equiv 0 (\mod g)$, although I'm not a big fan of that notation.
Answer: For convenience let $\ell=\frac{n-k}g$; the desired equality is then
$$\frac{k-1}{n-1}\binom{n+\ell-2}{n-2}+\frac{k+g}n\binom{n+\ell-2}{n-1}=\frac{k}n\binom{n+\ell-1}{n-1}\;.\tag{1}$$
We can use the identity $$\binom{n+\ell-1}{n-1}=\binom{n+\ell-2}{n-2}+\binom{n+\ell-2}{n-1}$$ to expand the righthand side of $(1)$ and rearrange it to get
$$\frac{k+g}n\binom{n+\ell-2}{n-1}-\frac{k}n\binom{n+\ell-2}{n-1}=\frac{k}n\binom{n+\ell-2}{n-2}-\frac{k-1}{n-1}\binom{n+\ell-2}{n-2}\;,$$
which immediately simplifies to
$$\frac{g}n\binom{n+\ell-2}{n-1}=\frac{n-k}{n(n-1)}\binom{n+\ell-2}{n-2}\;.$$
Now $n-k=g\ell$, so we can multiply both sides by $\frac{n}g$ to get
$$\binom{n+\ell-2}{n-1}=\frac{\ell}{n-1}\binom{n+\ell-2}{n-2}\;.$$
And
$$\frac{\ell}{n-1}\binom{n+\ell-2}{n-2}=\frac{(n+\ell-2)!}{(\ell-1)!(n-1)!}=\binom{n+\ell-2}{n-1}\;,\tag{2}$$
so all that you have to do to prove $(1)$ is start with $(2)$ and reverse the steps that I used to get from $(1)$ to $(2)$; they are all reversible.

Missing piece for combinatorial proof


Prove for continuous $f$ and $g$,$f(x)<g(x)$ there exists $k$ such that $f(x)+k<g(x)$

Question: Suppose that $f$ and $g$ are continuous on $[a,b]$ and for each $x$, it holds that $f(x)<g(x)$. Prove that there exists $\alpha>0$ such that for each $x$, it holds that $f(x) + \alpha <g(x)$ (Notice that $\alpha$ may not depend on $x$).
We have done a similar problem with $f(x)>0$ and proving $\alpha>0$ exists such that $f(x)>\alpha$, but I'm not exactly sure how to use supremum and infimum to solve the above version.
Comment: Hint: The continuous function $g-f$ attains a minimum value on $[a,b]$.
Comment: Consider the function $h(x)=g(x)-f(x)>0$.
Comment: Hint: $f(x)+\alpha\alpha$.
Answer: The comments answer the question, while the answer does not  (it attempts to prove a statement that the OP already knows how to prove). So, I'm going to record here that the statement about $f<g$ follows by applying the statement about $f>0$ to the function $g-f$.
Answer: It only needs to use finite covering theorem for compact interval $[a, b]$.
This question ,namely :suppose $f(x)>0 ,\forall x\in [a, b]$ ,we need to proof $\exists \alpha>0 , f(x)>\alpha , \forall x\in [a, b].$\
Finite covering theorem tells us that $\exists finite\  x_i and\  \delta>0,such that\  x\in (x_i,\delta)for\ some\  x_i,x\in[a,b].$\
by continuity of $f(x)$,$$f(x)>\frac{f(x_i)}{2}>0,x\in (x_i,\delta)$$
when we select the minimum value of $\frac{f(x_i)}{2},here\  index \  (i) finite.$
thus ,we have solved  this question.
Comment: If fact, strictly speaking,we should use continuity of $f(x)$ firstly. then use finite covering theorem for compact interval $[a, b]$.maybe having finite $\delta_i$.but that doesn't matter.


Using Drupal Queue

Question: I think I am close to getting a queue to work with the Drupal 7 Queue API.  I have some code that returns a value to me that I want to retrieve on cron.
function ...{
  $guid_to_enqueue = $guid;
  //Add GUID into the queue so we can work with it on cron
  $queue = DrupalQueue::get('custom_application_guid_queue');
  $queue->createQueue();
  $queue->createItem($guid_to_enqueue);
}

I then have
function custom_application_cron_queue_info() {
  $queues[ 'custom_application_guid_queue' ] = array(
    'worker callback' => '_custom_application_guid_item',
    'time' => 30,
  );
  watchdog('custom_application_ques_on_cron',  '
' . print_r($queues, TRUE) . '
');
  return $queues;
}

I think the problem is when I try and use the callback.  I am not sure how to iterate over my data that is in the queue.  I know it is in the queue using Drush and if I use Watchdog() with $guid_to_enqueue I can see the right value in the variable.
function _custom_application_guid_item($queues) {
 foreach($queues as $quid_item) {
   watchdog('custom_application_guid_item_foreach', $quid_item);
 }
}

Any help with figuring out what I am doing wrong here is greatly appreciated.  I have also tried getting the number of items in the queue but nothing is getting that far in my code.
I will add that I will rarely have more than two items in the queue if that changes anything.
Thanks,
Josh
Comment: It's first in first out I believe
Answer: Your queue worker callback takes a single queue item to process at a time, and you do not invoke it manually because you have setup cron to do it for you.
Drupal cron will try to process as many queue items within the time that you provided in hook_cron_queue_info(), and this should be first in, first out by the time the queue item was added and then by the primary key.
So in your cron queue worker callback you should expect the data that you enqueued.
function _custom_application_guid_item($guid_enqueued) {
  // Do something with $guid_enqueued
}

Comment: When cron fires and calls the callback, it will pass one item to it from the queue, in this example it is `$guid_enqueued` or call it whatever you want. That is what you will operate on. Is it a string, an array, an object that is stored in the queue item? If it is not an array, the foreach in your post will do nothing, hence, watchdog doesn't get hit. After that, it is then removed from the queue (I think), because it was "processed" out.
Comment: Check the docs on DrupalQueue as well. Basically the callback passes whatever item was stored in the given queue for processing, it will do as many as it can in the given time limit. Here is an article that might shed more light: https://internetdevels.com/blog/drupal-7-queue-api
Comment: Sorry,  I am still new to PHP where is $guid_enqueued coming from?  I thought this would be $queues because I return $queues in custom_application_cron_queue_info().  Also,  I am confused because when I use watchdog() or DSM() there is nothing in these vars.

Using Drupal Queue


Slurs with arrows for glissando-alikes in Lilypond, how to?

Question: I'm using Lilypond 2.20 on Ubuntu to engrave scores for the instrument pipa.
Slurs with arrow is a very common notation in many east Asian music pieces. It sounds similar to glissando in some degree, but not completely the same. The official Lilypond does not offer this out of box. I found some usable code from a German-speaking Lilypond forum. The code is pasted below.

The slur-with-arrow looks already handsome. I'd like the arrow head to be a bit more catching (bigger, and with a solid yet curved arrow head), and the slur line could be thicker too. My question is, how to modify the code below to achieve this? This might be a hardcore Lisp question.....My hat off to the original author of this code.
Here is a folk song from Taiwan containing many usages of this Asian glissando played on a pipa. Pay attention to the musican's left hand, the horizontal movement on a fret demonstrates how to play this kind of glissando. I haven't seen this on classical guitars yet. Very beautiful tunes:

\version "2.20.0"

slurArrow =
\once \override Slur.stencil =
  #(lambda (grob)
    (let* ((slur-dir (ly:grob-property grob 'direction))
           (right-bound (ly:spanner-bound grob RIGHT))
           (right-bound-stem (ly:grob-object right-bound 'stem))
           (right-bound-stem-dir
             (if (ly:grob? right-bound-stem)
                 (ly:grob-property right-bound-stem 'direction)
                 #f))
           (c-ps (ly:grob-property grob 'control-points))
           (frst (car c-ps))
           (thrd (caddr c-ps))
           ;;; corr-values are my choice
           (corr (cond ((not right-bound-stem-dir)
                        '(0 . 0))
                       ((= slur-dir (* -1 right-bound-stem-dir))
                        (cons -0.4  (* 0.2 slur-dir)))
                       (else '(-0.4 . 0.2))))
           (frth (offset-add (cadddr c-ps) corr)))

      (ly:grob-set-property! grob 'control-points
        (append (list-head c-ps 3) (list frth)))

    (let* ((orig (ly:grob-original grob))
           (siblings (ly:spanner-broken-into orig)))
       (if (or (null? siblings)
               (equal? grob (car (last-pair siblings))))
           (let* ((default-stil (ly:slur::print grob))
                  (default-stil-lngth
                    (interval-length (ly:stencil-extent default-stil X)))
                  (delta-x-cps (- (car frth) (car frst)))
                  (diff (- default-stil-lngth delta-x-cps))
                  (delta-iv
                    (cons (- (car frth) (car thrd)) (- (cdr frth) (cdr thrd))))
                  (radians->degree (lambda (radians) (/ (* radians 180) PI)))
                  (ang (radians->degree (atan (cdr delta-iv) (car delta-iv))))
                  (arrowhead (ly:font-get-glyph (ly:grob-default-font grob)
                                            "arrowheads.open.01"))
                  (rotated-arrowhead (ly:stencil-rotate arrowhead ang 0 0))
                  (arrowhead-lngth
                    (interval-length (ly:stencil-extent rotated-arrowhead X))))

             (ly:stencil-add
               default-stil
               (ly:stencil-translate
                 rotated-arrowhead
                 ;; Ugh, 3.8 found by trial and error
                 (cons (+ diff (/ arrowhead-lngth 3.8) (car frth))
                       (+ (cdr frth) 0)))))

            (ly:slur::print grob)))))

\relative c'' {
    \slurArrow
    c'4( c \bar "" \break c c)
    \slurArrow
    \slurDown
    c4( c c c)
    \slurArrow
    \stemUp
    \slurUp
    c4( c d a)
    %% maybe add:
    % \shape #'((0 . 0)(0 . 0)(0 . -0.5)(0 . -0.5)) Slur
    \slurArrow
    \slurDown
    c4( a) \slurArrow f( d)
}

Comment: Well, that's the problem :—). They mostly don't because the people who use them don't read or write scores. I wasn't objecting to your notation, I just wanted to react to your statement that you have never seen something like that on the guitar.
Comment: By the way, this kind of "glissando" looks just like a normal string bend, which is not really used in the classical music but it is abundant in the more modern genres of guitar playing.
Comment: @Ramillies  So how do guitarists notate string bend in a score?
Answer: change arrowheads.open.01 to arrowheads.close.01
for thickness insert these two lines after (append (list-head c-ps 3) (list frth)))
(ly:grob-set-property! grob 'line-thickness 0.8)
(ly:grob-set-property! grob 'thickness 1.2)
experiment with the values (0.8 and 1.2) of these new lines - they are set to defaults for now


Getting IP inside activity causes crash

Question: im creating an android app, to link and transfer socket packets to the server through wifi. In order for this, the IP address of the connected server has to be defined. I used the below function to get the IP address. When i hardcode the SERVER_IP = "0"; the app runs normally. Please help!
    private final String SERVER_IP = getIpAddr();

    public String getIpAddr() {
    WifiManager wifiManager = (WifiManager) getApplicationContext().getSystemService(WIFI_SERVICE);
    WifiInfo wifiInfo = wifiManager.getConnectionInfo();
    int ip = wifiInfo.getIpAddress();

    String ipString = String.format(
            "%d.%d.%d.%d",
            (ip & 0xff),
            (ip >> 8 & 0xff),
            (ip >> 16 & 0xff),
            (ip >> 24 & 0xff));

    return ipString;
}

After Setting up the above, I ran the below codes on the OnCreate function.
class ClientThread implements Runnable {

    @Override
    public void run() {
        try {
            InetAddress serverAddr = InetAddress.getByName(SERVER_IP);
            socket = new Socket(serverAddr, SERVERPORT);
        } catch (UnknownHostException e1) {
            e1.printStackTrace();
        } catch (IOException e1) {
            e1.printStackTrace();
        }
    }
}

As soon as that happens, my android application will stop working.
This is the error that appears:
Java.lang.NullPointerException: Attempt to invoke virtual method 'android.content.Context android.content.Context.getApplicationContext()' on a null object reference
Comment: Try using ActivityName.this.getApplicationContext() to get the context
Comment: Thank you. Plus 1.
Answer: Use this function to get IP (v4 or v6) inside an activity:
// Get IP address from first non-localhost interface
// @param ipv4  true returns ipv4
//              false returns ipv6
// @return address or empty string
public static String getLocalIpAddress(boolean useIPv4) {
    try {
        List<NetworkInterface> interfaces = Collections.list(NetworkInterface.getNetworkInterfaces());
        for (NetworkInterface intf : interfaces) {
            List<InetAddress> addrs = Collections.list(intf.getInetAddresses());
            for (InetAddress addr : addrs) {
                if (!addr.isLoopbackAddress()) {
                    String sAddr = addr.getHostAddress();
                    //boolean isIPv4 = InetAddressUtils.isIPv4Address(sAddr);
                    boolean isIPv4 = sAddr.indexOf(':')<0;
                    if (useIPv4) {
                        if (isIPv4)
                            return sAddr;
                    } else {
                        if (!isIPv4) {
                            int delim = sAddr.indexOf('%'); // drop ip6 zone suffix
                            return delim<0 ? sAddr.toUpperCase() : sAddr.substring(0, delim).toUpperCase();
                        }
                    }
                }
            }
        }
    } catch (Exception ex) { } // for now eat exceptions
    return "";
}

When you need it, just call it this way: getLocalIpAddress(true). You will get the IP as a string for you to use.
Comment: Awesome! I'm not getting the error anymore! Thanks
Comment: Thank you Marcus, keep it up.
Comment: socket = new Socket(serverAddr, SERVERPORT);
Just wondering, if I am connecting to my server correctly with this function? My serverAddr will be the ip address of my wifi i used to connect between my android phone and my hardware.
Comment: I think you are right. The "real" IP of your phone is revealed only when behind a regular data conn, like 3G, 4G and so on.
Comment: Thanks for the reply! I see, so it looks like my TCP client set up is correct. However, my current server connection failure might be due to my hardware not setting up the socket server correctly. I will look into this! Thank you very much! :) I am pretty new to all of these
Answer: Here are a few things that you have to check

Does the Application has the permission to access the WiFi ?
If you are using the getApplicationContext() in a different class , pass a context from the Parent Activity.
Test in on a mobile device not on the emulator.

Getting IP inside activity causes crash


Anova and ChiSquare with 4 groups

Question: I have asked some participants (29) to complete several tasks, and I have the results for the people that successfully completed it, and those who didn't.
I have a breakdown by age (4 groups) and I am trying to prove ($H_0$) that age is independent for the successful completion of the task.
I am trying to understand whether I have to use chi square or ANOVA. I used anova and I was able to reject the $H_0$, but if I use Chi Square, I fail to reject the Nullhypothesis.
Can someone guide me on this, please?
Comment: Could you please clarify if your dependent variable is a count (number of completed tasks) or a binary feature (completed/not completed)?
Comment: Hi, thanks a lot for your reply!
It was a task that they had to do and they would either get a success or unsuccess. Therefore, it is a binary feature.
Thanks again
Answer: ANOVA is better suited for real-valued dependent variables, so in your case chi-square test is a better alternative. If you are doing this in R, you could use prop.test, it implements the same chi-squared when there are more than two groups (age groups, in your case).
Comment: Thanks a lot for your help, I appreciate it.
I had also two other analysis, male-female success-unsuccess, and experience-not, success-unsuccess. Since those are tables 2 x 2 and with the kind of binary feature I had, I used chi square. Do you think that is OK, or you recommend something else?
Comment: It's fine! There are other alternatives like tests for proportion, but chi-squared is perfectly valid as well.
Comment: Thanks a lot for your help!
Last question if I may ask. Thanks again!
I am trying now to compare the results I got as metrics.
The 29 participants would do specific tasks on 3 different websites. I have the average time that it took to complete each task on each website and the target. Can I do ANOVA to check if there is a significant difference between target vs website 1, 2, and 3?
Or there is a better tool to compare the TARGET vs the real time it took?
Thanks again
Comment: Most likely ANOVA would do, but you have to be careful choosing between-subjects or within-subjects ANOVA. Do you have the same 29 participants doing the task on each of the 3 websites, or are they distributed between different websites? In the first case it is within-subjects ANOVA (or, as a nonparametric alternative, Friedman rank sum test); in the second ANOVA should be between-subjects (or you could use Kruskal-Wallis test).
Comment: Thanks so much! I thought it would be ANOVA and now that I have your confirmation I will get to it. It is indeed 29 people doing all the 6 tasks; i didn't divide them.
So I will use the within subject ANOVA. In one column I will put the target time and on the other one all the actual times. Does it sound right?
Thank you!!!!!

Anova and ChiSquare with 4 groups


Virtual firewall to protect hypervisor

Question: I am running an Ubuntu Server 12.10 as a single host connected to a NATed router connected using PPPoE to a optical fiber modem. This server is meant to be accessed from the Internet, but also to be used from the LAN as a SVN, MySQL and what not...
The issue is that the router is not customizable enough to serve, so I was thinking about creating a virtual pfSense firewall using KVM inside of the server itself, removing the need of the router. Is this possible? Can the host ignore and block all traffic coming to itself, but not for the firewall?
I am aware this is not the most desirable environment, I accept suggestions based on budget!
Answer: Yes, you can run a firewall as a virtual machine within this environment.
Before doing so, I'd ideally like to see a couple of NICs dedicated to the firewall VM (SR-IOV is nice, try it!) and additional security such as sVirt running at the hypervisor level to protect VMs from each other.
See also Hardware firewall vs VMware firewall appliance for further discussion.
Comment: The host has 3 NICs, but why would I need them (the host is the only one behind the firewall)? I just need to protect the hypervisor. My problem is trying to find an example of a similar setup using iptables to deny all traffic except what comes out of the virtual firewall

Virtual firewall to protect hypervisor


Second Differential

Question: Let $(x,y,z)$ a coordinate system, $M=\mathbb{R}^3$ and we also denote by $x$ the first coordinate function : $x:M \rightarrow \mathbb{R},\; q=(a,b,c) \mapsto a$.
We have $dx:TM \rightarrow \mathbb{R},\; (q,v=(v_1,v_2,v_3)) \mapsto dx[q](v)=v_1$.
Let $f=dx[q]$.
We have $df:TM \rightarrow \mathbb{R},\; (p,u=(u_1,u_2,u_3)) \mapsto df[p](u)=u_1$.
How to get $d(dx) = df = 0$ ?
Comment: $dx$ has zero differential as a $1$-form. But you differentiate $f=dx[q]$ as a $0$-form, i.e. as a function. In this case your calculation is correct and you don't get $d(dx[q])=0.$
Comment: $d$ is the derivative operator.  So use the properties of the exterior derivative: $df = d(dx) = d(1 dx) = d(1)\wedge dx = 0$
Comment: is there a way by just manipulating the definitions that I recalled and not using the exterior calculus ?
Comment: What you don't do is define the $d$ operator.  So your statement for $df$ is wrong but it'd be hard to see that without some definition of $d$.  To find the correct formula for $df$ you can use the properties of $d$ or just go back to the [limit-integral definition](http://math.stackexchange.com/questions/593920/integral-definition-of-exterior-derivative).
Comment: oki I see if I take the definition in local coordinates:
Given a fucntion $f : \mathbb{R}^3 \rightarrow R, q\mapsto f(q)$, $df$ is defined as:
$df[q](h) = \sum_i h_i \partial_i f(q) $
but then I need to define $d^2 f[q]$ with second partial derivatives and then the null function appears.
Answer: Let's use the definition of the exterior derivative to see why $d(dx)=0$:
Let $x:\Bbb R^3\to \Bbb R$ be given by $x(q) = x(q_1,q_2,q_3) = q_1$.  Then by definition:
$$dx[q](v) = \lim_{t\to 0} \frac {1}{t} \int_{\{q,q+tv\}} x = \lim_{t\to 0}\frac{x(q+tv)-x(q)}{t} = v_1$$
And
$$d(dx)[q](u,v) = \lim_{t_1,t_2\to 0} \frac{1}{t_1t_2} \int_{C} dx$$ where $C$ is the oriented parallelogram with vertices at $q$, $q+t_1u$, $q+t_1u+t_2v$, and $q+t_2v$.  But $\int_C dx=0$ for any such $C$ so $$d(dx) = 0$$
However as Valentin noted in the comments, $d(dx)$ and $d(dx[q])$ are two different things.  $dx$ is a $1$-form but $dx[q]$ is a $0$-form on $T_qM$.  I think that's where the disconnect is here.

Second Differential


How to calculate $\frac{1}{n}\sum_{i=1}^{n}\frac{i}{i+1}$

Question: I have been working with some probabilities and I am not sure if I'm correct but one of my probabilities is reduced to this formula, however, I don't recall a method to solve it:
$$\frac{1}{n}\sum_{i=1}^{n}\frac{i}{i+1}$$
I can't seem to decompose this... What's the approach I should take?
Comment: hint: $\frac{i}{i+1}=\frac{i+1}{i+1}-\frac{1}{i+1}=1-\frac{1}{i+1}$
Comment: Can I use the formula for the sum of r**x with -1 as x and i+1 as r? I am unsure
Comment: You cannot use that, because that only works when $r$ is a constant. Here the $i$ is different for each term.
Comment: What approach should I take then? I am not taking any courses related to this so my assumption is that the formula I found could be found differently... however, I would still like to verify this first.
Answer: Since
$$\frac{i}{i+1}=1-\frac{1}{i+1}$$
we have
$$\frac1n\sum_{i=1}^n\frac{i}{i+1}=1-\frac1n\Bigl(\frac12+\cdots+\frac1{n+1}\Bigr)\ .$$
The sum in brackets on the right hand side is unlikely to have any simple exact evaluation.  However it is given approximately, if $n$ is large, by
$$\ln n-1+\gamma\ ,$$
where $\gamma$ is the Euler-Mascheroni constant.  So
$$\frac1n\sum_{i=1}^n\frac{i}{i+1}\approx1-\frac{\ln n-1+\gamma}{n}\ .$$


I am looking for sampling software for the keyring 49?

Question: I am wondering if anyone knows about sampling software for the keyring 49? I basically have loops that I made in Logic. I would like to use them with my keyring 49. I would like to turn my keyring 49 into a digital sampler. I would also like the loops to beat match or sync on the fly. All of my loops are the same BPM by the way. It is hard for me to beat mach them by ear on Logic ESX24 Sampler. The loops are .wav files. I would also like each key to be mapped separately. Is there any software out there for that?
Basically why I want this is, because I want to do live mixes with my Keyring 49.
Answer: Ableton Live is a program commonly used for live shows, as the name suggests, because you can set up several samples and launch them in various ways, like using the MIDI output from your keyboard.
It automatically syncs the tempo of the new samples you launch.
Answer: Here are some popular software used for loop and sample based music creation. All these programs have convenient tools to map a MIDI keyboard to custom samples.

Ableton Live
Fruity Loops
Reason
NI Kontakt
ACID


Is it possible that $f(g(x))$ has four roots as $1$, $2$, $3$, $4$ if $f(x)$ and $g(x)$ are two quadratic expressions?

Question: This problem is from a previous year mathematics olympiad.

Is it possible that $f(g(x))$ has four roots as $1$, $2$, $3$, $4$ if $f(x)$ and $g(x)$ are two quadratic expressions?

So I tried considering real and non-real cases but couldn't find anything. It would be great if any one can help me.
It would be a great help for other Olympiad aspirants.
Comment: Can anyone plz address the above problem. I need it for my Olympiad preparation.
Answer: Yes it is possible.
Consider the quadratics
$$g(x)=x^2-5x+6$$$$f(x)=x^2-2x$$
In general, given desired roots $r_1,r_2,r_3,r_4$, the existence of $f,g$ depends on whether there exists distinct $i,j,k,l$ such that $r_i+r_j=r_k+r_l$.
Comment: That was what I could figure out. But we need a general proof for it. Can u plz help me with that.
Comment: This is a proof...
Comment: I think the OP means an heuristics to find this example or others maybe...
Comment: @donthousand there are infinite number of values for which the condition may hold can you please look up for that.
Comment: @JAOFELIX Not sure what you are asking. I think this is the point where you need to do the rest of the work yourself.
Comment: Ok then no offence @ don thousand I will keep you posted with the solution. Bye
Answer: Here is heuristic method.
As the goal is to obtain
$$h(x)=\underbrace{(x-1)(x-4)}_{f_1(x)}\underbrace{(x-2)(x-3)}_{f_2(x)}\tag{1}$$
If we remark that
$$\begin{cases}g_1(x)=(x-1)(x-4)=x^2-5x+4\\g_2(x)=(x-2)(x-3)=x^2-5x+6\end{cases}\tag{2}$$
we can write :
$$h(x)=g_1(x)(g_1(x)+2) \ \ \iff \ \ h(x)=f(g_1(x)) \ \ \text{with} \ \  \begin{cases}f(x):=x(x+2)\\g_1(x)=x^2-5x+4\end{cases} \tag{3}$$
which is a second solution.
Please note that this has been made possible because by a "smart" gathering of factors $(x-k)$ in (1) allowing a same first degree term $5x$ in (2) which has been cancelled at the next step.

Edit 1 : there is fact an infinite number of solutions (obtained by a Computer Algebra System : OK, non available for Olympiads) ; among them,

$$\begin{cases}f(x):=x^2+(10-2a)x+(a^2-10a+24)\\g(x)=x^2-5x+a\end{cases}$$
  depending on the real parameter $a$. For example,

taking $a=4$, we retrieve solution (3) above,
taking $a=6$ gives back the solution of @Don Thousand.

Edit 2: A simple - and different - approach is to give the issue a symmetrical aspect.
How ? By replacing $x=1,2,3,4$ by $y=2x-5=-3,-1,1,3,$
in order to have an even polynomial with these roots, i.e.,
$$(y^2-1)(y^2-9)=g(y)(g(y)-8)=f(g(y)) \ \ \text{with} \ \ f(z):=z(z-8).$$
It remains to come back to variable $x$ by replacing $y$ by $2x-5$.
Comment: This is quite a rigorous proof. But can u please try generalise the equations getting certain relation between both the polynomials.
Comment: Thanks a lot @Jean Marie.


Prove that $T$ is one-to-one iff $T$ carries linearly independent subsets of $V$ onto lineraly independent subsets of $W$.

Question: Let $V$ and $W$ be vector spaces and $T:V\rightarrow W$ be linear.
(a) Prove that $T$ is one-to-one iff $T$ carries linearly independent subsets of $V$ onto linearly independent subsets of $W$.
(b) Suppose that $T$ is one-to-one and that $S$ is a subset of $V$. Prove that $S$ is linear independent iff $T(S)$ is linear independent.
(c) Suppose $\mathcal{B} = \{v_{1},v_{2},\ldots,v_{n}\}$ is a basis for $V$ and $T$ is one-to-one and onto. Prove that $T(\mathcal{B}) = \{T(v_{1}),T(v_{2}),\ldots,T(v_{n})\}$ is a basis for $W$.
MY ATTEMPT (EDIT)
(a) We shall prove that implication ($\Rightarrow$) first.
Suppose that $S = \{s_{1},s_{2},\ldots,s_{m}\}\subset V$ is linear independent and $T$ is injective. Then we have that
\begin{align*}
& a_{1}T(s_{1}) + a_{2}T(s_{2}) + \ldots + a_{m}T(s_{m}) = 0 \Longrightarrow\\\\
& T(a_{1}s_{1} + a_{2}s_{2} + \ldots + a_{m}s_{m}) = 0 \Longrightarrow\\\\
& a_{1}s_{1} + a_{2}s_{2} + \ldots + a_{m}s_{m} = 0 \Longrightarrow\\\\
& a_{1} = a_{2} = \ldots = a_{m} = 0
\end{align*}
Thus $\{T(s_{1}),T(s_{2}),\ldots,T(s_{m})\}$ is linear independent, just as it has been claimed.
Let us prove that $(\Leftarrow)$ holds too.
We must prove that $T(v) = 0$ implies that $v = 0$. Indeed, according to the given assumption, if $\mathcal{B}_{V} = \{v_{1},v_{2},\ldots,v_{n}\}$ is a basis for $V$, then $T(\mathcal{B}_{V}) = \{T(v_{1}),T(v_{2}),\ldots,T(v_{n})\}$ is linear independent. Thence we get that
\begin{align*}
& T(v) = T(a_{1}v_{1} + a_{2}v_{2} + \ldots + a_{n}v_{n}) = 0 \Longrightarrow\\\\
& a_{1}T(v_{1}) + a_{2}T(v_{2}) + \ldots + a_{n}T(v_{n}) = 0 \Longrightarrow\\\\
& a_{1} = a_{2} = \ldots = a_{n} = 0 \Longrightarrow v = 0
\end{align*}
and we are done.
(b) Let us prove $(\Rightarrow)$ first.
If $S$ is LI, then $T(S)$ is LI, according to (a).
Let us now prove the implication $(\Leftarrow)$.
If $S = \{s_{1},s_{2},\ldots,s_{n}\}$, then $T(S) = \{T(s_{1}),T(s_{2}),\ldots,T(s_{n})\}$. If $T(S)$ is LI, then one has that
\begin{align*}
& a_{1}s_{1} + a_{2}s_{2} + \ldots + a_{n}s_{n} = 0 \Longrightarrow\\\\
& T(a_{1}s_{1} + a_{2}s_{2} + \ldots + a_{n}s{n}) = 0 \Longrightarrow\\\\
& a_{1}T(s_{1}) + a_{2}T(s_{2}) + \ldots + a_{n}T(s_{n}) = 0 \Longrightarrow\\\\
& a_{1} = a_{2} = \ldots = a_{n} = 0
\end{align*}
Thus $S$ is linear independent, just as desired.
(c) If $T$ is one-to-one, then $T(\mathcal{B}) = \{T(v_{1}),T(v_{2}),\ldots,T(v_{n})\}$ is linear independent. This is because $\mathcal{B}$ is LI as well.
Since $T$ is onto, we have that $T(V) = \text{span}(T(\mathcal{B})) = W$. Consequently, $T(\mathcal{B})$ is a basis for $W$, and we are done.
Could someone double-check my arguments?
Comment: I don't think so.
Comment: My apologies, I didn't read one to one and onto. I just read one to one.
Comment: For the second part of (a): if $T(v)=0$, that means that $\{T(v)\}$ is *not* linearly independent.
Comment: For the backwards direction, you want to show that kerT consists of the zero vector. To do this, you must show that if $v\in kerT$ then $v=0$
Comment: @ArturoMagidin To prove that $T$ is injective is the same as to prove that it is non-singular, that is to say, $T(v) = 0$ implies that $v = 0$.
Comment: @BrickByBrick: I am aware of that; but you are trying to use the fact that $T$ sends linearly independent sets to linearly independent sets in order to show that. The point is that if $T(v)=0$, then the set $\{T(v)\}$ is linearly **dependent**; what does that tell you about the set $\{v\}$?
Comment: Now I see. Thanks.
Answer: The first half of your argument for (a) is fine. The second half, however, is not: your $v_1,\ldots,v_n$ come out of thin air and have no apparent connection with $v$, the vector assumed to be in $\ker T$. I suggest that you assume instead that $T$ is not one-to-one and find a linearly independent set whose image under $T$ is not linearly independent.
One direction of (b) is an application of (a), but the other is not: you still have to show that if $T(S)$ is linearly independent, then so is $S$.
Your argument for (c) is fine.
Comment: First of all, thanks for the comments. I have fixed my arguments and provided a full answer to (b) as suggested. Could you please double-check if I am reasoning rightly this time?
Answer: Your revised attempt to show that if $T$ sends linearly independent sets to linearly independent sets is correct as far as it goes. It suffers form an unwarranted assumption that your space is finite dimensional (though you are probably working just with finite dimensional spaces). This can easily be fixed by taking an arbitrary basis and then just taking finitely many basis vectors needed to express $v$.
But there is a much simpler way: it is enough to show that if $v\neq 0$ then $T(v)\neq 0$. If $v\neq 0$, then $\{v\}$ is linearly independent, hence $\{T(v)\}$ is linearly independent; hence $T(v)\neq 0$.
Alternatively, using contrapositives, if $T(v)=0$ then $\{T(v)\}$ is linearly dependent, which means that $\{v\}$ is linearly dependent, which means that $v=0$.
Comment: Thanks for the contribution. It was really helpful.


Running continually tasks alongside the django app

Question: I'm building a django app which lists the hot(according to a specific algorithm) twitter trending topics.
I'd like to run some processes indefinitely to make twitter API calls and update the database(postgre) with the new information. This way the hot trending topic list gets updated asynchronously.
At first it seemed to me that celery+rabbitmq were the solution to my problem, but from what I understand they are used within django to launch scheduled or user triggered tasks, not indefinitely running tasks.
The solution that comes to my mind is write a .py file to continually put trending topics in a queue and write independent .py files continually running, making get queue requests and saving the data in the db used by django with raw SQL or SQLAlchemy. I think that this could work, but I'm pretty sure there is a much better way to do it.
Comment: Related question: http://stackoverflow.com/questions/3200001/using-crontab-with-django
Answer: If you just need to keep some processes running continually, supervisor is a nice solution.
You can combine it with any queuing technology you like to push things into your queues.


Finding sets which are not $\mu^*$-measurable

Question: Recall that the Dirac measure is defined as follows:
$\mu(A)=\Big\{\begin{array}{ll}
         1 & \mbox{if  $x_0 \in A$}\\
        0 & \mbox{if  $x_0 \notin A$}\end{array}$,
where $x_0$ can be any arbitrary point.
Now, let's define an outer measure which is similar to the Dirac measure:
$\mu^*(A)=\left\{ \begin{array}{ll}
         1 & \mbox{if  $x_0 \in A$}\\
0.5 & \mbox{if  $x_0 \in \bar{A}$ but $x_0 \notin A$}\\
        0 & \mbox{if  $x_0 \notin \bar{A}$}\end{array} \right .$,
where $\bar{A}$ is the closure of $A$. You can check that $\mu^*$ is not a measure but it satisfies the conditions of being an outer measure.
According to this outer measure, I want to find not $\mu^*$-measurable sets in $R^2$ when we consider $x_0$ to be the origin.
Recall that a set $A$ is $\mu^*$-measurable in $R^2$ when for every set $E$ in $R^2$ the following equality is held:
$\mu^*(E)$=$\mu^*(E\cap A)+\mu^*(E\cap A^c)$.
In this problem, I have found that the sets whose boundary passes the origin are not $\mu^*$-measurable. This is because, by considering $E=R^2$, we have: $\mu^*(E)=1$ but $\mu^*(E\cap A)+\mu^*(E\cap A^c)=1.5$.
Could you find other not $\mu^*$-measurable sets according to the defined outer measure? Or, could you prove that there is no other not $\mu^*$-measurable set?
Comment: I took it in good faith that $\mu*$ is an outer measure. However it is not. For instance in $\mathbb R$ let $A_n=(\frac1n,1)$. Then $A:=\bigcup_{n=1}^{\infty}A_n=(0,1)$. Then clearly $\mu^*(A)=\frac12$ and $\mu^*(A_n)=0$ for every $n$. This shows that $\mu^*$ is *not* countably subadditive hence is no outer measure. I got suspicuous when I realized that the "measurable sets" did not form a $\sigma$-algebra.
Comment: This is really helpful for my intuition, Thanks a lot.
Answer: Since outer measures are subadditive it is enough to find conditions on a set $A$ that are sufficient and necessary for the existence of a set $E$ that satisfies:$$\mu^*(E)<\mu^*(A\cap E)+\mu^*(A^c\cap E)$$Let us assume that $A$ contains the origin (for convenience denoted as $0$). This gives no essential loss of generality because a set $A$ is non-measurable iff its complement is non-measurable and one of them contains the origin.
By monotonicity we know that both terms on RHS are not larger than $\mu^*(E)$ and this tells us directly that the inequality can only be satisfied if these terms are both positive.
By substituting for some satisfying $E$ we discern the following possibilities:
$$1<1+\frac12\text{ or }\frac12<\frac12+\frac12$$
In the first case with $0\in E$ and in the second $0\in\overline E-E$.
Taking for both cases $E$ as large as possible we substitute $E=\mathbb R^2$ and $E=\Bbb R^2-\{0\}$ respectively to find the conditions:$$\mu^*(A^c)=\frac12$$for the first case and:$$\mu^*(A-\{0\})=\mu^*(A^c)=\frac12$$for the second.
For this in the first case it is necessary and sufficient that $0\in\overline{A^c}=(A^o)^c$ and combining this with preassumption $0\in A$ we conclude that a set $A$ with $0\in A-A^o$ is not measurable.
For the second case it is also necessary that $\mu^*(A^c)=\frac12$ or equivalently that $0\in A-A^o$ so this case will not provide "new" cases of non-measurable sets.
Dropping the preassumption that $0\in A$ we conclude that a set is non-measurable if $0\in A-A^o$ or if $0\in A^c-(A^c)^o$.
This can be summarized as: $$A\text{ is non-measurable iff }0\in\partial A$$

Edit:
Everything is fine above except that $\mu^*$ is not an outer measure. See my comment on the question.


Divisibility of Differents in Tower of Fields

Question: In Lemmermeyer's class field theory notes, on page 114 there is the following claim. Let $F = \mathbb Q(\sqrt{-5})$, then let $K = F(\sqrt{-1})$. Let $F_1 = \mathbb Q(\sqrt{-1})$ and $F_2 = \mathbb Q(\sqrt{5})$ be the other two quadratic subfields of $K/\mathbb Q$. Then we have that
$$
\operatorname{diff}(K/F) \mid \operatorname{diff}(F_1/\mathbb Q)
\quad\text{and}\quad
\operatorname{diff}(K/F) \mid \operatorname{diff}(F_2/\mathbb Q).
$$
Why is this true? Certainly I see $\operatorname{diff}(K/F)$ divides $\operatorname{diff}(K/\mathbb Q) = \operatorname{diff}(K/F_i)\operatorname{diff}(F_i/\mathbb Q)$ for $i=1,2$ but I don't see how to get the claim.
Answer: The point is that $K/F = F(\sqrt{-1})$ is a "relativization" of $F_1/\mathbb{Q} = \mathbb{Q}(\sqrt{-1})$, so in particular, (i) any element generating the latter also generates the former; (ii) the trace form on the latter extends to the trace on the former. (For $F_2/\mathbb{Q}$ it's the same.)
Now to show the different of the former divides the different of the latter, we can either use (i) together with the "$(f'(\alpha))$ definition" of different, or (ii) together with the "inverse of inverse different" definition (see Wikipedia, especially the Neukirch reference given (p. 198, Chapter III, Theorem 2.5) for the equivalence of definitions).
(For the special case that "relativization" preserves unramified-ness, see Neukirch p. 153, Ch. II, Sec. 7,
Proposition 7.2.)


Autoresizing issue of UICollectionViewCell contentView's frame in Storyboard prototype cell (Xcode 6, iOS 8 SDK) happens when running on iOS 7 only

Question: I'm using Xcode 6 Beta 3, iOS 8 SDK. Build Target iOS 7.0 using Swift. Please refer to my problem step by step with screenshots below.
I have a UICollectionView in Storyboard. 1 Prototype UICollectionViewCell which contains 1 label in the centre (no autoresizing rule). Purple background was to mark a contentView that is generated in runtime by the Cell I guess. That view will be resized properly base on my UICollectionViewLayoutDelegate eventually, but not on iOS 7. Notice that I'm using Xcode 6 and the problem only happens on iOS 7.
When I build the app on iOS 8. Everything is okay.
Note: Purple is the contentView, Blue is my UIButton with rounded corner.

However, on iOS 7, all the subViews inside the Cell suddenly shrink to the frame of (0,0,50,50) and never conforms to my Autoresizing rule anymore.

I assume this is a bug in iOS 8 SDK or Swift or maybe Xcode?

Update 1: This problem still exists in the official Xcode 6.0.1 ! The best work around is like what KoCMoHaBTa suggested below by setting the frame in cellForItem of the cell (You have to subclass your cell though). It turned out that this is a incompatibility between iOS 8 SDK and iOS 7 (check ecotax's answer below quoted from Apple).
Update 2:
Paste this code at the beginning of your cellForItem and things should be okay:
/** Xcode 6 on iOS 7 hot fix **/
cell.contentView.frame = cell.bounds;
cell.contentView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
/** End of Xcode 6 on iOS 7 hot fix **/

Comment: I downloaded xcode 6.1 but still see the same problem in the simulator.
Comment: I am struggling with this right now in my project.  iOS 7 and iOS 8 built using Xcode 5 look fine.  iOS 8 built using Xcode 6 beta 6 looks fine.  iOS 7 built using Xcode 6 beta 6 is having the problem you are describing.  Using Reveal I can see that my UICollectionViewCell has been properly sized.  But the cell's contentView was not resized, even though it's parent, the UICollectionViewCell has Autoresize Subviews turned on.  The size of the contentView is set to what the storyboard has.  I am not using autolayout in this project.  My project is completely objective-c.
Comment: I just wanted to add that this issue still exists as of Xcode 6 / iOS 8 GM seed. @DanielPlamann's answer to force the `contentView` to resize with the cell works fine to workaround the problem. I guess that in iOS 8 Apple changed something about the way cell content views are handled when created in Interface Builder (which is still a bit of a black box anyway). But the fact that it changes behaviour when targeting iOS 7 is surely a bug.
Comment: I found out that this problem still exists in Xcode 6 Beta 5. Anyone experienced this too?
Comment: Same here with Xcode 6 GM, auto layout and a nib-based cell. I fix it by pinning the `contentView` edges to the cell edges.
Comment: Yeah, this is not good. I don't want to put this workaround into a production app at all. Have anyone of you filed this as bug report yet? I'm afraid they don't take this seriously enough.
Comment: @sergiou87 how did you manage to pin the `contentView` to the edges via the nib when `UICollectionViewCell`s don't show the `contentView` in the nib?
Comment: @Acey unfortunately that has to be done in code :-\ We're using a custom subclass of `UICollectionViewCell` as base class for our custom cells.
Comment: Well, in fact it's not unfortunate, because doing it this way we avoid having to do that for every cell :D
Comment: @Acey I just updated the best solution, either you're using nib or not. Putting the block of code in cellForItem is the safest way.
Comment: Happened both on Xcode 6.0 (from AppStore) AND Xcode 6.1 Beta for Yosemite. 'Update 2' works like charm. I just spent half a day on that...
Comment: Apple appears to have fixed this problem in version 8.1 of the iOS SDK in Xcode 6.1 GM (Build 6A1042b). I removed the hack and my collection view cells were sized correctly. Can anyone else confirm this?
Comment: Did you test on iOS 7?
Comment: I never had this issue before iOS 8.1 update, then it started happening (in code compiled with 6.0.1 and also when I recompiled in 6.1). I had to add the workaround to get rid of it.
Comment: None of the solutions still work for me!! Any help?
Comment: http://stackoverflow.com/questions/27265576/static-uitableviewcell-subviews-are-resized-incorrectly-was-off-the-screen-ove I got another problem with static cell
Comment: Guys, the sad news is that, Apple still leaves this bug until the latest iOS SDK.
Answer: contentView is broken. It can be also fixed in awakeFromNib
ObjC:
- (void)awakeFromNib {

    [super awakeFromNib];

    self.contentView.frame = self.bounds;
    self.contentView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
}

Swift3:
override func awakeFromNib() {
    super.awakeFromNib()

    self.contentView.frame = self.bounds
    self.contentView.autoresizingMask = [.flexibleWidth, .flexibleHeight]
}

Comment: I like this answer but you need to call [super awakeFromNib];  Also, I'v thinking about adding a check for iOS 7.1 and less, because I'm not sure how adding these resizing masks affects default behavior on iOS8.
Comment: Did the trick without self.contentView.frame = self.bounds; Think I need it? Thanks anyway!
Comment: Hello Michal, agree with you. But for 100% to be sure, how will it work in next iOSes better to add it I think.
Comment: If you're not using nibs or storyboards, this also works if you put it in applyLayoutAttributes:
Comment: This doesn't work for me. I am not using Autolayout!! Any assistance?
Comment: How do you create cell? From nib?
Comment: We were seeing an issue with ios 8.x that the collection view cell was not displaying all of it's subviews sometimes.  This fixed our issues, but it is slightly different then the main question.
Comment: I added same code in swift. But still app crashing and showing bad execute error at `layout2.estimatedItemSize = CGSize(width: 300, height: 50)` . I added a code in Cell subclass     `self.contentView.frame = self.bounds;
        self.contentView.autoresizingMask = UIViewAutoresizing.FlexibleWidth | UIViewAutoresizing.FlexibleHeight`. amy one know how to rectify in swift
Comment: Adding auto resize subviews in the Nib is the real trick. You don't anything else.
Comment: What if storyboard table has static cells?
Comment: How can `contentView` be broken, still?
Answer: I encountered the same problem and asked Apple DTS for help. Their reply was:

In iOS 7, cells' content views sized themselves via autoresizing
  masks. In iOS 8, this was changed, cells stopped using the
  autoresizing masks and started sizing the content view in
  layoutSubviews. If a nib is encoded in iOS 8 and then decode it on iOS
  7, you'll have a content view without an autoresizing mask and no
  other means by which to size itself. So if you ever change the frame
  of the cell, the content view won't follow.
Apps being deploying back to iOS 7 will have to work around this by
  sizing the content view itself, adding autoresizing masks, or adding
  constraints.

I guess this means that it's not a bug in XCode 6, but an incompatibility between the iOS 8 SDK and iOS 7 SDK, which will hit you if you upgrade to Xcode 6, because it will automatically start using the iOS 8 SDK.
As I commented before, the workaround Daniel Plamann described works for me. The ones described by Igor Palaguta and KoCMoHaBTa look simpler though, and appear to make sense giving Apple DTS' answer, so I'll try those later.
Comment: This is interesting, but I still hope they fix it. This behavior was only introduced in Xcode 6 GM which added iPhone 6 support.  It was working fine in the earlier betas.  I even brought a project back to an earlier beta after I noticed it and it worked as expected.  I hope everyone files bugs with Apple on this issue.
Comment: @arton I filed a bug report the same day I asked DTS for help. It was closed as a Duplicate of 18312246. Not sure how much this helps.
Comment: I've used the work around by sizing the ContentView and works for me.
- (CGSize)collectionViewContentSize {
    return CGSizeMake(self.collectionView.bounds.size.width, self.collectionView.bounds.size.height);
}
Answer: I encountered the same issue and hope that Apple will fix this with the next Xcode version. Meanwhile I use a workaround. In my UICollectionViewCell subclass I've just overridden layoutSubviews and resize the contentView manually in case the size differs from collectionViewCell size.
- (void)layoutSubviews
{
  [super layoutSubviews];

  BOOL contentViewIsAutoresized = CGSizeEqualToSize(self.frame.size, self.contentView.frame.size);

  if( !contentViewIsAutoresized) {
    CGRect contentViewFrame = self.contentView.frame;
    contentViewFrame.size = self.frame.size;
    self.contentView.frame = contentViewFrame;
  }
}

Comment: This is the best solution. Adding this in cellForItem/cellForRow will cause weird artifacts if you're rotating the device and the cell size changes.
Comment: Hm, maybe it's better to place `[cell layoutIfNeeded];` in   cellForItem or cellForRow?
Comment: Yeah I used a similar work around but I did it in cellForItem / cellForRow which also works.
Comment: Hi! I got a problem with this code. For the first time, the boolean contentViewIsAutoresized will be true if it is loaded from storyboard or prototype cell. Only when you do a reloadData, the second it will be correct. So you don't really need to check for the size. Instead just do: self.contentView.frame = self.bounds;
Comment: Confirmed: we should do it in cellForItem or cellForRow because layoutSubviews is only called after the cell is returned. Anything you do before that like drawing stuff will be wrongly calculated.
Answer: Another solution is to set the contentView's size and autoresizing masks in -collectionView:cellForItemAtIndexPath: like the following:
- (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath {

     static NSString *cellID = @"CellID";

     UICollectionViewCell *cell = [collectionView dequeueReusableCellWithReuseIdentifier:cellID forIndexPath:indexPath];

     // Set contentView's frame and autoresizingMask
     cell.contentView.frame = cell.bounds;
     cell.contentView.autoresizingMask = UIViewAutoresizingFlexibleLeftMargin | UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleRightMargin |UIViewAutoresizingFlexibleTopMargin |UIViewAutoresizingFlexibleHeight | UIViewAutoresizingFlexibleBottomMargin;

     // Your custom code goes here

     return cell;
}

This works with Auto Layout too, since auto resizing masks are translated to constraints.
Comment: This is a superior solution as it works with any cell type without subclassing and doesn't require changes in multiple places when you are using more than one type of cell in your collection view.
Answer: In Xcode 6.0.1 contentView for  UICollectionViewCell is broken for iOS7 devices.
It can be also fixed by adding proper constraints to UICollectionViewCell and its contentView in awakeFromNib or init methods.
        UIView *cellContentView = self.contentView;
        cellContentView.translatesAutoresizingMaskIntoConstraints = NO;

        [self addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"H:|[cellContentView]|"
                                                                     options:0
                                                                     metrics:0
                                                                       views:NSDictionaryOfVariableBindings(cellContentView)]];
        [self addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"V:|[cellContentView]|"
                                                                     options:0
                                                                     metrics:0
                                                                       views:NSDictionaryOfVariableBindings(cellContentView)]];

Comment: Only this works now, you have to do it just for IOS 8, and I have to call it after each dequeue ! But this also is not ideal solution because multiple selection do not work as it should visually...
Comment: Best is to use autolayout really, with this bug you save ironically time by use autolayout event in simple case..
Comment: The mask does not work for me; however setting the constraints works!
Answer: This will not work correctly without any of the other mentioned workarounds because of a bug in Xcode 6 GM with how Xcode compiles xib files into the nib format. While I cannot say for 100% certainty it is Xcode related and not having to do with runtime, I'm very confident - here's how I can show it:

Build+Run the application in Xcode 5.1.
Go to the simulator application's directory and copy the compiled .nib file for the xib you are having issues with.
Build+Run the application in Xcode 6 GM.
Stop the application.
Replace the .nib file in the newly built application's simulator folder with the .nib file created using Xcode 5.1
Relaunch the app from the simulator, NOT from Xcode.
Your cell loaded from that .nib should work as expected.

I hope everyone who reads this question will file a Radar with Apple. This is a HUGE issue and needs addressing before the final Xcode release.
Edit: In light of ecotax's post, I just wanted to update this to say it is now confirmed behavior differences between building in iOS 8 vs iOS 7, but not a bug. My hack fixed the issue because building on iOS 7 added the autoresizing mask to the content view needed to make this work, which Apple no longer adds.
Comment: I'm not sure if they legally can release a different xCode than the GM version
Comment: Haha will they go to jail? =P But in all seriousness, sure they can. They had multiple GM versions for Mavericks if you don't remember. It's not too common but it can happen.
Answer: The answers in this post work, what I never understood is why it works.
First, there are two "rules":

For views created programmatically (Ex. [UIView new]), the property translatesAutoresizingMaskIntoConstraints is set to YES
Views created in interface builder, with AutoLayout enabled, will have the property translatesAutoresizingMaskIntoConstraints set to NO

The second rule does not seem to apply to top-level views for which you do not define constraints for. (Ex. the content view)
When looking at a Storyboard cell, notice that the cell does not have its contentView exposed. We are not "controlling" the contentView, Apple is.
Deep dive into storyboard source code and see how contentView cell is defined:
<view key="contentView" opaque="NO" clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="center">
Now the cell's subviews (notice the translatesAutoresizingMaskIntoConstraints="NO"):
<view contentMode="scaleToFill" translatesAutoresizingMaskIntoConstraints="NO" id="NaT-qJ-npL" userLabel="myCustomLabel">
The contentView does not have it's translatesAutoresizingMaskIntoConstraints set to NO. Plus it lacks layout definition, maybe because of what @ecotax said.
If we look into the contentView, it does have an autoresizing mask, but no definition for it:
<autoresizingMask key="autoresizingMask"/>
So there are two conclusions:

contentView translatesAutoresizingMaskIntoConstraints is set to YES.
contentView lacks definition of a layout.

This leads us to two solutions which have been talked about.
You can set the autoresizing masks manually in awakeFromNib:
self.contentView.frame = cell.bounds;
self.contentView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;

Or you can set the contentView translatesAutoresizingMaskIntoConstraints to NO in awakeFromNib and define constraints in - (void)updateConstraints.
Answer: This is the Swift version of @Igor's answer which is accepted and thanks for your nice answer mate.
First Goto your UICollectionViewCell Subclass and paste the following code as it is inside the class.
override func awakeFromNib() {
    super.awakeFromNib()
    self.contentView.frame = self.bounds
    self.contentView.autoresizingMask = [.FlexibleHeight, .FlexibleWidth]
}

By the way I am using Xcode 7.3.1 and Swift 2.3. Solution is tested on iOS 9.3 which is working flawlessly.
Thanks, Hope this helped.
Comment: sorry for my bad English, i meant "works like a charm" :)
Comment: @Aznix No issue..:)
Answer: In swift, place the following code in the collection view cell subclass:
override var bounds: CGRect {
  didSet {
    // Fix autolayout constraints broken in Xcode 6 GM + iOS 7.1
    self.contentView.frame = bounds
  }
}

Comment: This was the answer I was looking for. I used to override setBounds in Objective-C to fix this problem (wasn't sure how to write it in Swift). Thank you :)
Answer: I have found that there are also issues with contentView sizing in iOS 8. It tends to get laid out very late in the cycle, which can cause temporary constraint conflicts. To solve this, I added the following method in a category of UICollectionViewCell:
- (void)fixupContentView
{
#if __IPHONE_OS_VERSION_MAX_ALLOWED < 80100
#if __IPHONE_OS_VERSION_MAX_ALLOWED >= 80000
    if (NSFoundationVersionNumber <= NSFoundationVersionNumber_iOS_7_1) {
        self.contentView.frame = self.bounds;
        self.contentView.autoresizingMask = UIViewAutoresizingFlexibleLeftMargin | UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleRightMargin |UIViewAutoresizingFlexibleTopMargin |UIViewAutoresizingFlexibleHeight | UIViewAutoresizingFlexibleBottomMargin;
    } else {
        [self layoutIfNeeded];
    }
#endif
#endif
}

This method should be called after dequeuing the cell.
Comment: It would be ideal if this method were called automatically. I don't love it, but this could be done by swizzling `dequeueReusableCellWithReuseIdentifier:forIndexPath:`.
Comment: Apple appears to have fixed this problem in version 8.1 of the iOS SDK in Xcode 6.1 GM (Build 6A1042b). Therefore I have updated the code above to not run when using the 8.1 SDK. Once your team has all moved to Xcode 6.1 you can remove this hack entirely.
Answer: I fixed doing this:
override func layoutSubviews() {
   contentView.superview?.frame = bounds
   super.layoutSubviews()
}

see: here
Answer: Just make sure you check the check box "Autoresize subviews" in the nib for that collection view cell. It will work fine on both iOS 8 and iOS 7.
Comment: No it doesn't. This is an embedded prototype cell.


vaadin grid height with pagination controls on mobile devices

Question: I'm using a vaadin-grid that is stacked upon a pagination toolbar (vaadin-grid-pagination) and the problem I'm having is that on mobile devices the pagination doesn't sit at the bottom of the viewport as the grid comes over it.

Ideally (disregard the pagination overflowing) it would look like this:

Now, through the Dev Tools inspector I was able to remove a position: relative property from the scroller and that made things behave as I want them to.

The problem is that I am unable to do this programatically... I tried using a custom css file with these contents:
:host() #scroller {
      width: 100%;
      height: 100%;
      position: absolute;
    }

and use it in the view that is building the grid:
@CssImport(value = "./styles/grid-styles.css", themeFor = "vaadin-grid")

but it doesn't seem to make any difference.
Not sure what the correct approach would be, if any.
Comment: You can inject custom styles using JS and override whatever is there, even if it's generated programatically to start with?
Answer: You should not modify the internal styles of the components. There's no guarantee that they will still behave/function correctly now or in the future. The #scroller selector is part of the private implementation details of the <vaadin-grid> component.
The only CSS selectors that you should style are :host and [part], which can be combined with native pseudo-classes (:hover, etc.), pseudo-elements (::before, ::after, etc.) and documented state attributes ([disabled], [active], etc.). Learn more about the supported selectors.
My suspicion is that the height: 100% on the <vaadin-crud> component is causing it to be too high. You should try using flex: 1 on it instead.
Answer: #1 Try setting the position value using !important ?
:host() #scroller {
      width: 100%;
      height: 100%;
      position: absolute!important; /* add !important */
    }

#2 Otherwise like I suggested you can inject it using JS (here jQuery).
$('#scroller').css('position','absolute');

Making sure to insert that JS snippet as far down the load chain as possible.
Comment: #1 solution doesn't work and the #2 is not an option, unfortunately. Thanks for the reply.
Comment: Maybe try a more specific selector ? i.e. `.my-list-view vaadin-crud vaddin-grid #scroller`
Comment: Unfortunately, that doesn't work either.
Comment: can we know please where are the CSS styles causing issue generated or stored? The screenshot is cut off on the right, but Inspector should state a referall css file or some other info on the styles on the right.
Comment: The styles are generated by vaadin, a framework for building web-apps, so I'm not able to manually modify them.
Comment: I understand that, but still I would be curious to see what Inspector has to say about these styles, if you want me to continue helping you in the troubleshooting.
Comment: Targeting private CSS selectors is not recommended. The `#scroller` selector is part of the private implementation details of the vaadin-grid component.
Comment: @Jouni thanks for that, I was not aware of these best practices. To be really honest, and without trying to be a jerk, I am not a big fan of vaadin-grid or whatever pre-built stuff are related to it and I'm a proponent of not using such things, which is why, I guess, I have naively suggested good ol' targeting. I will keep that in mind however when trying to help pre-built-using-folks.


Pegar a hora do banco de dados MySQL

Question: -Boa noite, pessoal!
Eu tenho uma aplicação que pega dados do banco de dados e apresenta na tela. É uma grade horária de televisão, ou seja, mostra o que está passando ao vivo e o que vai passar depois.
O tipo de dados que representa a hora da programação é DateTime
Gostaria de pegar somente a hora e o minuto, ignorando a data.
Exemplo: no banco está assim (2019-08-31 20:50) Quero pegar apenas o 20:50 e mostrar na tela. Estou fazendo com php.
Como posso fazer isso?
Meu código atual:
<?php
//SELECT
$gh_resultado = $wpdb->get_results("SELECT * FROM Programacao where horario >= '$gh_data' order by horario");

//Mostrando na tela
echo $gh_resultado[0]->horario;
?>

Comment: Utilize a função TIME(seu_campo_datetime) do MySQL.
Answer: Como já explicado só utilizar a função TIME() do mysql
Exemplo.:
SELECT *, TIME(horario) AS horario FROM Programacao where horario >= '$gh_data' order by horario

Lembrando que o nome da propriedade a ser retornada mudara para TIME(propriedade), então é importante que a renomeie novamente para seu antigo nome como
TIME(horario) AS horario

Comment: Deu certo! Muito obrigado pela ajuda!!!!

Pegar a hora do banco de dados MySQL


Autodesk Forge: how to load the model on a node server?

Question: I'm trying to run raytracing analysis on a model from Forge. The processing is pretty heavy and it must be run on the server-side. I'm attempting to load the model into threejs (with all the geometry, materials, etc) on a node server using he svf, but I'm having trouble getting this to work. Has anybody been able to do this or have any suggestions?
Answer: I haven't tried yet, but should be best with something like headless Chrome, where a JavaScript code running on the page can extract the information you need.


Conditionally passing a named keyword argument to a function

Question: I'm not sure if I'm using the correct wording in my question but the situation I have is that I'm calling a function and passing a value to one of its named keyword arguments only if I have a value for it. A simplified example of how I'm currently doing it:
var_1 = data.get("var_1", None)

if var_1:
    some_function(some_arg=var_1)
else:
    some_function()

The function some_function() does not accept setting some_arg to None so I can only call the function with the named argument if I have a valid value for it.
The reason why the above is less desirable for me is that the actual function calls I'm making have a bunch (5+) of named arguments that I set. This results in repeating mostly similar function calls that differ by only one parameter.
Is there a better way/more Pythonic way to do this? I wish I could call the function and set all possible named parameters that I might wish to pass but, at runtime, only pass in the values that I actually have something for.
In case it matters, the function calls are to the AWS Python SDK (Boto3) so I can't alter the behavior of the function.
Comment: Does `some_arg` have a known default, or is it a strictly optional keyword argument (extracted from `**kwargs`, for example)?
Answer: What you can do is:
var_1 = data.get("var_1", None)
fun_kwargs = {'some_arg': var_1} if var_1 else {}
some_function(**fun_kwargs)

If you have multiple parameters you could do:
fun_kwargs = {}
if var1: fun_kwargs['some_arg1'] = var_1
if var2: fun_kwargs['some_arg2'] = var_2
if var3: fun_kwargs['some_arg3'] = var_3
some_function(**fun_kwargs)

Comment: Is this method extensible to multiple arguments? e.g. both a `var_1` and `var_2` for two different kwargs?
Comment: @swigganicks Yes, just fill a `dict` with whatever is available and pass it to the function with `**`. I have added an example with a possible way to do it.


Splitting listviews items help needed

Question: I have some code that loops through a listview and puts the values into 'output1' & 'output2'. However, what is happening is that the results are strung together like thus: 0307NG77775660NG7778. How do I split so that I can include in access db.
There are 2 columns in the listview and they contain values in the format of:
1st row 0307 - NG7777, 2nd row 5660 - 7778 etc. This is a desktop winforms code. Thanks
        Dim BoxList As New List(Of String)
        Dim BoxItem As String
        Dim CustRefList As New List(Of String)
        Dim CustRef As String

        For Each item As ListViewItem In Me.lvSelectedItems.Items
            Dim box As String = item.Text
            BoxItem = item.SubItems.Item(1).Text
            BoxList.Add(box)
            output2 += BoxItem

            Dim cust As String = item.Text
            CustRef = item.SubItems.Item(1).Text
            CustRefList.Add(cust)
            output1 += CustRef
        Next

        output = String.Join(" "c, BoxList.ToArray(), CustRefList.ToArray())

        Dim cmd As OleDbCommand = New OleDbCommand("Insert into Temp (temp, [class]) Values ('" & output1 & "', '" & output2 & "')", oledbCnn)
        dr = cmd.ExecuteReader

Comment: What output do you want? Please show an example
Comment: it *sounds like* instead of `output1 += username` what you want is to add or update a database with those values
Answer: Make username a List(Of String), then just do:
usernameList.Add(username) 'instead of output1 += username

When done with the loop, do this:
output1 = String.Join(" "c, usernameList.ToArray())

You can pick a join character of your choice (if you don't like spaces " ").
EDIT: Example:
Dim usernameList As New List(Of String)
Dim session As String

For Each item As ListViewItem In Me.lvSelectedItems.Items
  Dim username As String = item.Text
  session = item.SubItems.Item(1).Text
  usernameList.Add(username)
  output2 += session
Next

output1 = String.Join(" "c, usernameList.ToArray())

Comment: Many thanks for your help. If I have any problems I shall repost.
Comment: Neolisk I am quite new to vb.net and not advanced enough yet to code what you suggested. Would you have an example you could post please. Thanks
Comment: @user1532468: See my example. I converted `userName`, you can do `session` for learning purposes.
Comment: Many thanks. Just one question, would session be in the same for each as username or a new one. Thanks
Comment: @user1532468: Same ForEach is fine.
Comment: Neolisk it errors with 'Object reference not set to an instance of an object.' which is occuring here: usernameList.Add(username)
Comment: @user1532468: I am sorry, there was a typo - it should have been `New List(Of String)`. I fixed it just now.
Comment: Neolisk. Please look at my amended code. It isn't doing what I thought it would do. It is grouping all the values for each list in a single line. for example, NG565 77889. I need these to be separate for inclusion in db. The user also has the ability to select multiple files, so I don't think this is correct. What is the correct way? Thanks
Comment: @user1532468: Don't Join then. Just use a List you have.
Comment: Neolisk. Are you meaning output =output1 + output2. Thanks
Comment: Neolisk, I am not grasping this. I am getting same item in each output1 and output2. Do I need to change the ite, text somewhere. Thanks
Comment: I have to go out now, but I shall check for any updates in the morning. Many thanks for your patience.
Comment: @user1532468: ExecuteReader is meant for reading. To update, you need [SqlCommand.ExecuteNonQuery](http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlcommand.executenonquery(v=vs.110).aspx). Check [this out](http://stackoverflow.com/questions/6360153/vb-net-sql-server-insert-executenonquery-connection-property-has-not-been-ini). Basically, loop through your lists, and issue INSERT statement for each. `For i = 0 To list.Count` etc. - assuming both counts are the same, you should be able to do use `list1(i)` and `list2(i)`. Notice use of parameters on the link I mentioned.
Comment: Neolisk, do I loop through the lists in the for each loop and execute sqk there or is that incorrect. thanks
Comment: @user1532468: That would be two loops for you. I'm suggesting one loop, using index variable, to access both of your lists in one loop. The rest is okay.

Splitting listviews items help needed


Allowing outbound traffic with APF/iptables for OpenVZ container

Question: I have apf installed on a OpenVZ container (proxmox 2.1).  The config is pretty much vanilla and things are working. My external services like ssh and http are working.  My problem is that all outbound traffic on http/https is blocked. How do I allow all outbound traffic for http/https.
If I change EGF to 1 like this, all inbound and outbound traffic gets blocked
EGF="1"
EG_TCP_CPORTS="21,25,80,443,43,53"
EG_UDP_CPORTS="20,21,53"
EG_ICMP_TYPES="all"

I opened a single outbound rule with the following
# /usr/local/sbin/apf -a downloads.wordpress.org

How do I allow all outbound traffic on http/https without blocking all traffic?
Why would I allow all inbound ssh/http traffic and block all outbound traffic?
Comment: Did you ever find a reasonable solution for this issue?
Answer: I had the same issue - using an OpenVZ container, outbound HTTP and HTTPS were blocked when the inbound filter was on; everything was blocked when the egress filter was on.
The solution is to set the IPTABLES configuration of your container from your hardware. On my server, the default was (in /etc/vz/vz.conf on the hardware):
IPTABLES="ipt_REJECT ipt_tos ipt_limit ipt_multiport iptable_filter iptable_mangle ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_length"

I overwrote this in my container's configuration (/etc/vz/conf/CONTAINERNUMBER.conf on the hardware)
IPTABLES="iptable_filter iptable_mangle ipt_limit ipt_multiport ipt_tos ipt_TOS ipt_REJECT ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_LOG ipt_length ip_conntrack ip_conntrack_ftp ip_conntrack_irc ipt_conntrack ipt_state  ipt_helper  iptable_nat ip_nat_ftp ip_nat_irc ipt_REDIRECT"

Not sure exactly which one did it, but after setting this and restarting the container, APF worked exactly as expected.
Comment: Thanks, I will give that a try.  Never expected to get a response at this point.  Thanks!
Comment: @David I love you, David. You saved my xmas.
Comment: No big deal, I work out :) Merry Christmas
Answer: That's to prevent unwanted outbound traffic to bypass your firewall and consume unnecessary bandwidth, etc.


how to use one collection as a parameter to function call in another collection of the same size?

Question: I have two collections known to be of the same size:
List<Drivers> drivers;
List<Car> cars;

I need to put driver from the i place to i car:
for (int i = 0; i < drivers.Count; i++)
    cars[i].AssignDriver(drivers[i]);

May I do that without loop?
Comment: You need to match driver with car? Or you want to replace entry from cars with some value from drivers? My guess is you want one-to-one relationship between driver and car.
Comment: @PawanMishra i don't understand your question, I just need to call `AssignDriver` method as I wrote in description
Answer: Well, on the lowest level - no. You'll need to iterate anyway.
But there are some nice libraries that make handling such situations easier. For instance, there are Interactive Extensions for .NET. Although, they are not in production state, they make it possible to do it this way:
cars.Zip(drivers, (car, driver) => new { car, driver })
  .ForEach(x => x.car.AssignDriver(x.driver));

Comment: should that be `ForEach(x => x.car.AssignDriver(x.driver));`
Comment: Is it List.ForEach() or custom extension method?
Comment: Neither. It is from the Interactive Extensions package that I referenced. It works with any IEnumerable (the signature is `public static void ForEach(this IEnumerable enumerable, Action action))
Answer: LINQ only for querying object, so you've to live with loop. An other way is to use List.ForEach method but I would suggest not using it since it confusing sometimes, anyway below is example in educational purposes:
drivers.Select((d,i) => new { Index = i, Driver = d })
       .ToList()
       .ForEach(entry => car[entry.Index].AssignDriver(entry.Driver));

Useful links:

"foreach" vs "ForEach"
Comment: It's worthy of noting that this code is less effective: it creates a new list and actually executes a double iteration (once to create the list and then to execute ForEach). Although if performance is not critical, this is fine.
Comment: Absolutely right! ToList() creates a new list instance and copy memory for each item
Answer: LINQ is your friend. Take a look at this:
http://msdn.microsoft.com/en-us/library/bb548891.aspx
Note however, that I would consider your existing approach a good one too.
Comment: You should avoid modifying data (so-called side-effects) inside stateless LINQ operators like `Select`. Their semantics implies this requirement. Operators like `ForEach` in IX are designed specifically to denote that side effects are being introduced.


Complex number reduction problem

Question: Given the complex number $ z = x + yi; x, y \in \!R$. Also, let $ w = \frac{z-1-i}{z+1+i}$. Assuming $\mid w \mid  = 1$, we can affirm that
a) $x + y = 1$
b) $x + y = -1$
c) $x - y  = 1$
d) $x + y = 0$
e) $xy = 1$
SOLUTION
I tried to solve this problem by replaying $z$ in $w$ and expand from, but it seems a dead-end, I ended up with a very long expression with $x$ and $y$.
Answer: $|w|=1$ means $|z-1-i|=|z+1+i|$ so $z$ is equidistant from $1+i$ and $-1-i$ What does that locus look like?
Comment: yes! thank you! the line x + y = 0 is equidistant from those two numbers
Answer: hint
$$|w|=1 \implies$$
$$ |z-1-i|^2=|z+1+i|^2$$
$$\implies $$
$$|(x-1)+i(y-1)|^2=$$
$$|(x+1)+i(y+1)|^2$$

Complex number reduction problem


Dúvida sobre requisito funcional ou não funcional

Question: Recentemente comecei a ler um pouco sobre requisitos funcionais e não funcionais, sobretudo as diversas classificações que um requisito não funcional pode possuir. Entretanto, ainda possuo dificuldades em certas ocasiões de diferenciar ambos.
Para ser mais específico, estou levantando requisitos para um jogo educacional, no qual os jogadores podem escolher um modo de jogo. Basicamente, o modo de jogo permitiria que o usuário jogasse individualmente ou em grupo.
Considerando este requisito em específico, ele seria funcional ou não funcional? E existe alguma técnica ou dica que diferencie facilmente requisitos funcionais de não funcionais?
Answer: Requisitos funcionais são as tarefas de desenvolvimento que SERÃO ou PRECISAM ser DESENVOLVIDAS

RF001 – Incluir produto
RF002 – Alterar produto
RF003 – Consultar produto
RF004 – Excluir produto

Requisitos não funcionais entenda como premissas, exigências de desempenho e limitações, exemplo:

RNF001 - O arquivo XML não pode exceder 500kb.
RNF002 - O envio de arquivos não pode exceder 60 segundos.

Embora deve haver uma REGRA DE NEGÓCIO que impeça alcançar essas fronteiras e limites.
Comment: Considerando portanto que os requisitos não funcionais são limitações, o exemplo que dei de modo de jogo se enquadraria como um requisito funcional?
Answer: Absolutamente correto!
É uma funcionalidade que precisa ser desenvolvida a qual o usuário é OBRIGADO A SELECIONAR O MODO.
Vejamos o caso abaixo:
REQUISITO NÃO FUNCIONAL:
RNF001 - O arquivo XML não pode exceder 500kb.
Esse é um requisito imposto pelos webservers de nota fiscal eletrônica.
E se o arquivo for maior que 500k?
A sefaz vai responder à sua requisição normalmente, embora não aceite o arquivo e te retorne um erro. Mas você não precisou codificar nada. No máximo, vai retornar uma mensagem de erro ao usuário informando que o arquivo foi rejeitado por causa do seu tamanho.
OPCIONALMENTE, você pode especificar um REQUISITO FUNCIONAL associado a este RNF, que seria algo como:
RFxxx - A aplicação deve ser capaz de informar ao usuário que o arquivo XML está próximo/excedeu do/o limite de envio. (alguém precisa codificar isso!)


Number of Orbits of the action

Question: $G=GL_2(\mathbb{R})$ acts  on $\mathbb{R}^2$ via matrix multiplication, I need to find the number of orbits of this action.
$\{Ax: A\in G\}$
I did some calculation and I think It has one orbit , I mean orbit of any non zero vector is whole $\mathbb{R}^2$, but I am not able to prove it.
Thank you for discussion
Comment: Given two non-zero vectors, construct an invertible matrix that sends one to the other. You also need to consider the orbit of the zero vector.
Comment: Then please consider answering the question yourself. This will have the double benefit of removing it from the unanswered list and giving you a chance to have people check if your solution is correct.
Answer: Orbit of $(1,0)^T$ is all of $\mathbb{R}^2$
$A(1,0)^T=$ first collumn of $A$, suppose $x\in$ orbit of $(1,0)^T$ Then $x$ is the first collumn of some invertible matrix $A$ which is not $0$ or else $A$ would not be invertible. so orbit of $(1,0)^T$ is contained in $\mathbb{R}^2$
Now let $\{y,z\}$ be a basis of $\mathbb{R}^2$ then  construct a matrix $A$ making collumn $y$ and $z$ must be invertible!
and $A(1,0)^T=y$ thus $y$ is in the orbit of $(1,0)^T$
Another orbiit is orbit of $(0,0)^T$
Comment: But this orbit is ${\mathbb R}^2 \setminus \{ (0,0) \}$. So there are two orbits in total.

Number of Orbits of the action


What does it mean if test was rejected less than in 5% cases?

Question: I was reading a paper and authors performed multiple tests for Poisson distribution on single vectors. Large dataset of $n$ quite long vectors was used. The test was rejected only $0.02 \cdot n$ times under 0.05 level of significance.
I can imagine non-uniform distribution of p-values and I understand what means 10% of rejections under 5% significance (test's assumptions are not satisfied), but this case is different. Does it mean that suggested test statistic is asymptotically less than theoretical and test is not correct?
(I found test that was used)
Answer: If the null hypothesis is correct, the distribution of the p-value should be uniform, by definition. I have come across two cases in my career where the results looked "too consistent" with the null...out of 50+ tests, we have most p-values $\geq 0.9$!
In cases such as this, I considered two possible explanations:

Samples not truly independent
Incorrect null distribution (if parametric)

Since p-values test for "extremeness" they are not very good at individually testing for an abundance of "closeness"....more than expected by chance.
However, first you need to see how often you'd get $0.02*n$ rejections if the base rejection rate were $0.05$. This is a binomial test.
It could also be bad samples (inconsistent samples) where they do not follow any consistent distribution...this will have unpredictable effects.
In the particular example, the sample sizes were actually quite moderate ($5 \leq n \leq 20)$ an the number of Monte Carlo runs were decent at $10,000$. However, the tests under consideration all involved large-sample approximations of one form or another, so my bets are on my second bullet here...they are really deriving approximate p-vaules.
Comment: @GermanDemidov when I say incorrect null....I mean incorrect null distribution of the *sampling statistic*....in the case of fitting poisson, your null may be asymptotically correct, but if the actual distribution of the null is not near asymptotic, then you are comparing your sample statistic to the incorrect reference distribution.
Comment: Thank you! But incorrect null can only make p values worse...not better, cant they? I can not check result with binomial (actually story starts with http://nar.oxfordjournals.org/content/40/9/e69.full.pdf+html), but number of regions should be really big.


R, remove rows based on the values from multiple column

Question: Suppose i have a dataframe with 100 rows and 100 columns.
For each row, if any 2 columns have the same value, then this row should be removed.
For example, if column 1 and 2 are equal, then this row should be removed.
Another example, if column 10 and column 47 are equal, then this row should be removed as well.
Example:
test <- data.frame(x1 = c('a', 'a', 'c', 'd'),
               x2 = c('a', 'x', 'f', 'h'),
               x3 = c('s', 'a', 'f', 'g'),
               x4 = c('a', 'x', 'u', 'a'))

test

  x1 x2 x3 x4
1  a  a  s  a
2  a  x  a  x
3  c  f  f  u
4  d  h  g  a

Only the 4th row should be kept.
How to do this in a quick and concise way? Not using for loops....
Comment: I mean, I'm not convinced you can do this without for loops
Comment: When asking for help, you should include a simple [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input and desired output that can be used to test and verify possible solutions.
Comment: https://www.rdocumentation.org/packages/base/versions/3.5.1/topics/unique
Answer: Tested on this 20x20 dataframe
library(tidyverse)

N <- 20
df <- matrix(as.integer(runif(N^2, 1, 500)), nrow = N, ncol = N) %>%
  as.tibble()

df

# # A tibble: 20 x 20
# V1    V2    V3    V4    V5    V6    V7    V8    V9   V10   V11   V12   V13   V14   V15   V16   V17   V18   V19   V20
# <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int>
#   1   350   278   256   484   486   249    35   308   248    66   493   130   149     2   374    51   370   423   165   388
# 2   368   448   441    62   304   373    38   375   406   463   412    95   174   365   170   113   459   369    62    21
# 3   250   459   416   128   372    67   281   450    48   122   308    56   121   497   498   220    34     4   126   411
# 4   171   306   390    13   395   160   256   258    76   131   471   487   190   492    21   237   380   129     5    30
# 5   402   421     6   401    50   292   470   319   283   178   234    46   176   178   288   499     7   221   123   268
# 6   415   342   132   379   150    35   323   225   246   496   460   478   205   255   460    62    78   207    82   118
# 7   207    52   420   216     9   366   390   382   304    63   427   425   350   112   488   400   328   239   148    40
# 8   392   455   156   386   478     3   359   184   420   138    29   434    31   279    87   233   455    21   181   437
# 9   349   460   498   278   104    93   253   287   124   351    60   333   321   116    19   156   372   168    95   169
# 10   386    73   362   127   313    93   427    81   188   366   418   115   353   412   483   147   295    53    82   188
# 11   272   480   168   306   359    75   436   228   187   279   410   388    62   227   415   374   366   313   187    49
# 12   177   382   233   146   338    76   390   232   336   448   175    79   202   230   317   296   410    90   102   465
# 13   108   433    59   151     8   138   464   458   183   316   481   153   403   193    71   136    27   454    62   439
# 14   421    72   106   442   338   440   476   357    74   108    94   407   453   262   355   356    27   217   243   455
# 15   325   449   151   473   241    11   154    52    77   489   137   279   420   120   165   289    70   128   384    53
# 16   126   189    43   354   233   168    48   285   175   348   404   254   168   126    95    65   493   493   187   228
# 17    26   143   112   107   350   198   353   439   192   158   151    23   326     4   304   162    84   412   499   170
# 18    88   156   222   227   452   233   397   203   478    73   483   241   151    38   176    77   244   396     9   393
# 19   361   486   423   310   153   235   274   204   399   493   422   374   399    10   215   468   322    38   395   390
# 20   417   124    21   220   123   399   354   182   233    24   397   263   182   211   360   419   202   240   363   187

Removing rows with any duplicates
df %>%
  group_by(id = row_number()) %>%
  gather(col, value, -id) %>%
  filter(!any(duplicated(value))) %>%
  spread(col, value)

# # A tibble: 11 x 21
# # Groups:   id [11]
# id    V1   V10   V11   V12   V13   V14   V15   V16   V17   V18   V19    V2   V20    V3    V4    V5    V6    V7    V8    V9
# <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int>
#   1     1   350    66   493   130   149     2   374    51   370   423   165   278   388   256   484   486   249    35   308   248
# 2     3   250   122   308    56   121   497   498   220    34     4   126   459   411   416   128   372    67   281   450    48
# 3     4   171   131   471   487   190   492    21   237   380   129     5   306    30   390    13   395   160   256   258    76
# 4     7   207    63   427   425   350   112   488   400   328   239   148    52    40   420   216     9   366   390   382   304
# 5     9   349   351    60   333   321   116    19   156   372   168    95   460   169   498   278   104    93   253   287   124
# 6    12   177   448   175    79   202   230   317   296   410    90   102   382   465   233   146   338    76   390   232   336
# 7    13   108   316   481   153   403   193    71   136    27   454    62   433   439    59   151     8   138   464   458   183
# 8    14   421   108    94   407   453   262   355   356    27   217   243    72   455   106   442   338   440   476   357    74
# 9    15   325   489   137   279   420   120   165   289    70   128   384   449    53   151   473   241    11   154    52    77
# 10    17    26   158   151    23   326     4   304   162    84   412   499   143   170   112   107   350   198   353   439   192
# 11    18    88    73   483   241   151    38   176    77   244   396     9   156   393   222   227   452   233   397   203   478

Answer: Use apply to look for duplicates in each row. (Note that this internally converts your data to a matrix for the comparison. If you are doing a lot of row-wise operations I would recommend either keeping it as a matrix or converting it to a long format as in Jack Brookes's answer.)
# sample data
set.seed(47)
dd = data.frame(matrix(sample(1:5000, size = 100^2, replace = TRUE), nrow = 100))

# remove rows with duplicate entries
result = dd[apply(dd, MARGIN =  1, FUN = function(x) !any(duplicated(x))), ]

Comment: Thanks Gregor, your solution also works. I learned a lot from you and Jack. Thanks both. Your solution is more concise i think.
Answer: You can try a series of filters from dplyr. I cooked up some sample data here. If your variables are named then you can use something like the first example. Otherwise the second should work
library(tidyverse)
#> Warning: package 'dplyr' was built under R version 3.5.1
data <- data_frame(
  A = c(1,2,3,4,5,6),
  B= c(1,3,5,7,9,11),
  C = c(2,2,6,8,10,12)
)

data %>%
  filter(A != B) %>%  # This removed the first row
  filter(A != C) # This removed the second row
#> # A tibble: 4 x 3
#>       A     B     C
#>   <dbl> <dbl> <dbl>
#> 1     3     5     6
#> 2     4     7     8
#> 3     5     9    10
#> 4     6    11    12

data %>%
  filter(.[1] != .[2]) %>%
  filter(.[1] != .[3])
#> # A tibble: 4 x 3
#>       A     B     C
#>   <dbl> <dbl> <dbl>
#> 1     3     5     6
#> 2     4     7     8
#> 3     5     9    10
#> 4     6    11    12

Comment: I don't think this works well since OP says they have 100 columns. `choose(100, 2)` gives 4950, which is a lot of pairwise comparisons to type.
Comment: Agreed, this would only work if OP knows which columns shouldn't have duplicated values. Yours generalizes to all columns