iptables matching pattern followed by 4 random values followed by another pattern

Question: I'm trying to create a iptables rule that matches the following pattern in this UDP packet:
0x0000:  0000 030a 0000 0000 0000 0000 0000 0800  ................
0x0010:  4500 0027 5d30 0000 6c11 232a 5164 585d  E..']0..l.#*QdX]
0x0020:  c0a8 6402 fe25 1e61 0013 b382 5341 4d50  ..d..%.a....SAMP
0x0030:  c063 ba71 e2ea 63                        .c.q..c

The pattern is |53414d50c063ba71| followed by a random hex, in this case |e2ea|, followed by |63|.
The rule has to assure that there are 4 digits after the |53414d50c063ba71|, and that after those 4 random digits, there is a |63|.
Right now I have this, but I don't know how to modify it accordingly:
iptables -I INPUT -p udp --dport 7777 -m string --algo kmp \
    --hex-string '|53414d50c063ba71????63|' -j DROP

???? -> How??
Please help me.
Comment: I have seen it's possible to do this with http://www.stearns.org/doc/iptables-u32.current.html, but I don't know how to use it. Another option is to check the first part, just until before the 4 random digits (I know how to do it), and then match the LAST 2 digits with another rule. How to do that?
Comment: @A.B It's always the last position, in my UDP packet example, it's "63", it's always that last position, do you know how to do that with the example packet I've provided?
Comment: I made an answer but you never gave any feedback. So, is that what you wanted?
Answer: As long as the offset is fixed, it's possible to use the u32 match to do this kind of test. If the payload is at the end of the packet, that means the packet must be of constant size too for u32 to be of any use.
Why? That's because u32 can only add unsigned numbers, so it's not possible to match "last 11 bytes" expressed as "at position (packet size - 11)" with u32 because there's a substraction (or an addition of a negative number).
So for this specific example with a payload size of 0x37 (55), as asked in comment, here's the u32 match expression to use (packet being UDP will already have been validated by the udp match before):
iptables -I INPUT -p udp --dport 7777 -m u32 --u32 "0x0>>0x16&0x3c@0x34=0x53414d50&&0x0>>0x16&0x3c@0x38=0xc063ba71&&0x0>>0x16&0x3c@0x3b&0xff=0x63" -j DROP

Iterative explanations (of course, understanding u32's manpage and examples is needed):
0x0 >> 0x16 & 0x3c : adjusted Internet Header Length (IHL) <=> start of UDP header
0x0 >> 0x16 & 0x3c @ 8 : 4 bytes at start of UDP data
0x0 >> 0x16 & 0x3c @ 0x34 : 4 bytes at position (0x34-8) = 0x2c of UDP data
0x0 >> 0x16 & 0x3c @ 0x34 = 0x53414d50 : 4 bytes at position 0x2c == 0x53414d50
&& : logical and
0x0 >> 0x16 & 0x3c @ 0x38 = 0xc063ba71 : 4 bytes at position 0x30 == 0xc063ba71
0x0 >> 0x16 & 0x3c @ 0x3b : 4 bytes at position (0x3b-8) = 0x33 of UDP data
0x0 >> 0x16 & 0x3c @ 0x3b && 0x000000ff : content of last byte of the 4 bytes starting at position 0x33, meaning single byte at (here last) position 0x36
0x0 >> 0x16 & 0x3c @ 0x3b && 0x000000ff = 0x63 : byte at position 0x36 == 0x63


$GL(4,R)$ vs $T(3,1)\times O(3,1)$?

Question: I am told from math class that a general coordinate transformation on a vector is $GL(4,R)$, and in my physics class I am told it is $T(3,1)\times O(3,1)$. Can $GL(4,R)$ do everything $T(3,1)\times O(3,1)$  does, and more? Or is it $T(3,1)\times O(3,1)$ that is 'bigger'?
Answer: The general coordinate transformation group of a $4$-dimensional vector space is $\text{GL}(4)$.
However, in special relativity, the structure of space-time is that of a lorentzian affine space, in which case the group of general coordinate transformations is $T(4)\rtimes O(3,1)$. More explicitely, there is no origin of space, so transformations are not required to fix the origin like in the vector space case. This is the origin of the factor $T(4)$. But the difference between two points lies in a vector space equipped with a lorentzian bilinear form, so we require coordinate transformation to preserve this form. This is why we have $O(3,1)$, which is strictly smaller than $\text{GL}(4)$.
Neither group is included in the other. Instead, we have $\text{GL}(4)\cap (T(4)\rtimes O(3,1)) = O(3,1)$ which is strictly smaller than both.
To recap, the symmetry groups are :

Vector space
Affine space

No bilinear form
$\text{GL}(n)$
$T(n)\rtimes \text{GL(n)}$

Euclidean form
$O(n)$
$T(n) \rtimes O(n)$

Lorentzian form
$O(n-1,1)$
$T(n) \rtimes O(n-1,1)$

$GL(4,R)$ vs $T(3,1)\times O(3,1)$?


Probability of outcomes from overlapping samples of a random variable

Question: I have a feeling there is a straight forward answer to my question, but I'm not sure what the appropriate term to search for is. I'm trying to get a sense of what the probability is of finding some outcome as a subset of a random sample. The challenge as I see it is that the possible subsets of the total sample are not independent.
As a simple example, imagine I flipped a coin 100 times and recorded the results. I'm trying to ask a question like "what is the probability that somewhere in these results there is a run of 5 heads in row (or some other specific sequence)". The probability of flipping a coin 5 times and getting heads each time is pretty easy to calculate, and you have numerous opportunities within 100 total flips, so I can imagine using the binomial distribution, but the question would be exactly how many 'independent' opportunities you have within all the flips.
Naively, you might say that there are 95 chances - you can get all heads over the first five flips, or failing that, over the flips numbered 2 through 6, or 3 through 7, etc - but that's clearly not right, because these subsets of flips overlap each other. If you know that the 5th flip was tails, then you know that all 5 runs of flips that contain it have been 'spoiled'.
On the other end, you might imagine segmenting the 100 flips into 20 completely independent (i.e non-overlapping) runs of flips, but this would seem to undercount your probability - the overlapping runs clearly add some opportunity of finding the sequence of heads.
The actual problem I'm dealing with that caused me to have this question is only very slightly more complicated than the dummy example with the coin flips, because I'm looking in two dimensions (looking for a particular grid of values in an array of random variables). Somehow it feels like there ought to be a way to account for the overlapping samples and come to an intuition for how often you expect to find some specific sequence in the larger sample.
Comment: The one-dimensional problem has already been tackled on this website in several ways, for instance  [Probability of a similar sub-sequence of length X in two sequences of length Y and Z](https://stats.stackexchange.com/questions/565676) and [A fair die is rolled 1,000 times. What is the probability of rolling the same number 5 times in a row?](https://stats.stackexchange.com/questions/492000). In order to tackle your 2d problem one can use similar approaches, but it would help to explain it in more detail.
Comment: You do not have independence.  And you have $96$ rather than $95$ possible strings of $5$, so the expected number of appearances including overlaps of any given sequence of five is $96 \times 2^{-5}=3$, while the expected number excluding overlaps depends on the particular given sequence: for $5$ heads in a sequence on $n$ flips it seems to be about $\frac{1}{31}(\frac n2 - \frac{49}{31})$ so about $1.56$ when $n=100$. This does not help with the probability of ever seeing $5$ heads, which is empirically about $1-0.945595\times 0.982974^n$ for large $n$ about $0.81$ when $n=100$
Comment: @SextusEmpiricus Thanks for those links, very helpful. Reading up on the Markov Chains now. My 2d problem is basically the same sort of thing, except it's a table of what is essentially noise, and the question is, what is probability that some sub-grid of it has some pattern. So, in my, say, 100 by 100 table of random 1s and 0s, what's the probability I can find, say, a 5 by 5 grid of just 1s.
Comment: @MaxMiller Markov Chains are actually the thing where I have doubts whether these can help you (since they are more one dimensional).
Answer: I'm not completely satisfied with this answer, but I think it's better than no answer. I'll attempt to answer the question posed in the comments: in a 100-by-100 matrix of independent Bernoulli trials ($M$), what is the probability of finding a 5-by-5 submatrix, $X$, composed entirely of successes? I use a 0.5 probability of the Bernoulli trials returning 1, but the methodology should work for any probability. The code below is in R.
First, I used a couple Rcpp functions. The first is a fast Rcpp function saved in "C:/temp/findSquare.cpp". It counts how many matrices contain a square submatrix of size n composed entirely of 1s. Each column of the input matrix x holds the values of an nr-by-nc matrix.
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
int findSquare(IntegerMatrix& x, const int& nr, const int& nc, const int& n) {
  const int iter = x.nrow();
  int out = 0;
  for (int k = 0; k < iter; k++) {
    int jj = nc;
    int cnt = 0;
    int found = 0;
    for (int j = 0; j < nc; j++) {
      cnt = (cnt + 1)*x(k, j);
      x(k, j) = (cnt >= n) + 1;
    }
    for (int i = 1; i < nr; i++) {
      int cnt = 0;
      for (int j = 0; j < nc; j++) {
        cnt = cnt*x(k, jj) + x(k, jj);
        x(k, j) = x(k, j)*(cnt >= n) + 1;
        found += x(k, j) > n;
        jj++;
      }
    }
    out += found > 0;
  }
  return(out);
}

The second is for matrix multiplication since matrix multiplication in base R is so slow on my setup. It is saved in "C:/temp/eigenMult.cpp"
// [[Rcpp::depends(RcppArmadillo, RcppEigen)]]
#include <RcppArmadillo.h>
#include <RcppEigen.h>

// [[Rcpp::export]]
SEXP eigenMapMatSquare(const Eigen::Map<Eigen::MatrixXd> X){
  Eigen::MatrixXd X2 = X * X;
  return Rcpp::wrap(X2);
}

// [[Rcpp::export]]
SEXP eigenMapMatMult(const Eigen::Map<Eigen::MatrixXd> A, Eigen::Map<Eigen::MatrixXd> B){
  Eigen::MatrixXd C = A * B;
  return Rcpp::wrap(C);
}

Finally, an R function for raising a matrix to a power using "eigenMult.cpp":
matpow <- function(x, pow) {
  if (pow == 0L) {
    1L
  } else if(pow %% 2L) {
    nextpow <- (pow - 1L)/2L
    if (nextpow) {
      eigenMapMatMult(x, Recall(eigenMapMatSquare(x), nextpow))
    } else x
  } else {
    Recall(eigenMapMatSquare(x), pow/2L)
  }
}

We can get a credible interval via Monte Carlo simulation:
Rcpp::sourceCpp("C:/temp/findSquare.cpp")
# Monte Carlo - take 100 sets of 10k samples
system.time(counts <- sum(replicate(100, findSquare(matrix(sample(0:1, 1e8, TRUE), 1e4), 100L, 100L, 5L))))
#>   user  system elapsed
#> 641.66   27.36  669.52
# 90% credible interval
(pSimBounds <- qbeta(c(0.05, 0.95), sum(counts) + 0.5, 1e6 - sum(counts) + 0.5))
#> [1] 0.0002212588 0.0002728769

These bounds are fairly tight, but they took more than 11 minutes to compute. A simple Markov chain will give quick and easy bounds. Instead of a 100-by-100 matrix, look for $X$ in a 100-by-5 matrix. The probability of getting 1 for all five values in a row is $2^{-5}$. The transition matrix is then:
m <- cbind(matrix(c(rep(1 - 1/32, 5), 0), 6), diag(1/32, 6)[,-6])
m[6, 6] <- 1

Get the probability of finding $X$:
Rcpp::sourceCpp("C:/temp/eigenMult.cpp")
(p1 <- matpow(m, 100)[1, 6])
#> [1] 2.772544e-06

The full matrix has 20 independent sets of 5 columns. A lower bound could be the probability of finding $X$ only in those 20 independent sets. For an upper bound, there are 96 sets of 5 columns in the full matrix, but they aren't independent: if the $X$ is not in columns 1 through 5, it is less likely to be in columns 2 through 6, etc. Treating the 96 sets as independent will over-estimate the probability of finding $X$ in $M$:
(pLower <- 1 - (1 - p1)^20)
#> [1] 5.544942e-05
(pUpper <- 1 - (1 - p1)^96)
#> [1] 0.0002661292

The lower bound is not very impressive, but the upper bound is tighter than what was calculated from 1M samples.
For the final estimate, let's examine what happens if we start with 5 columns and repeatedly add one more column. For six columns, the Markov chain needs to track what is happening in columns 1:5 as well as 2:6. Each set can be in state 0 through 4 (the number of consecutive rows of 5 1s), resulting in 25 states plus 1 absorbing state. As an example, the sequence 0,1,1,1,1,1 in a row will transition columns 1:5 to state 0 and increment the state of columns 2:6 by 1 (unless the system is in the absorbing state).
If we start with 5 columns and add 1 column at a time, we can calculate the probability of finding $X$ in the last 5 columns conditioned on $X$ not existing in the matrix without the last column. As more columns are added, this probability quickly converges, which allows us to estimate the probability of finding $X$ in $M$. The size of the transition matrix grows quickly as more columns are added, so I stop at 11.
# more involved estimate from multiple Markov chains
library(data.table)
library(Rfast) # for eachrow and rowMaxs functions

fTrans <- function(dt, mTrans, transk, n, nn) {
  # iteratively build out state changes
  nmTo <- paste0("to", 1:nn)
  nmFrom <- colnames(dt)
  nmAll <- c(nmFrom, nmTo)
  dtStates <- setnames(as.data.table(rep(list(integer(0)), nn*2L + 1L)), c(nmAll, "k"))
  while (nrow(dt)) {
    ldt <- vector("list", nrow(dt) + 1L)
    ldt[1:nrow(dt)] <- lapply(
      1:nrow(dt),
      function(i) {
        m2 <- eachrow(mTrans, as.numeric(dt[i])) + mTrans
        m2[rowMaxs(m2, TRUE) == n, ] <- n
        cbind(
          setnames(dt[rep(i, nrow(mTrans))], nmFrom),
          setnames(as.data.table(m2), nmTo)
        )[, k := transk][, .(k = sum(k)), nmAll]
      }
    )
    ldt[[nrow(dt) + 1L]] <- dtStates
    dtStates <- rbindlist(ldt)
    dt <- rbindlist(
      list(
        unique(dtStates[, 1:nn])[, k := 2L],
        unique(dtStates[, (nn + 1L):(2L*nn)])[, k := 1L]
      ), FALSE
    )[
      , .(k = sum(k)), nmFrom
    ][
      k == 1L & from1 < n
    ][
      , k := NULL
    ]
  }

  dtStates
}

pGridInArray <- function(n, dims, p = 0.5) {
  # main function
  if (dims[1] < dims[2]) dims <- dims[2:1]
  n1 <- n - 1L
  nn <- dims[2] - n1 # number of column sets
  nn1 <- nn - 1L
  nn2 <- 2^nn1
  # get all possible row values that will not lead to a transition back to the
  # initial state
  m <- as.matrix(expand.grid(rep(list(0:1), nn1)))
  m <- unique(
    do.call(
      rbind,
      lapply(
        1:nn,
        function(i) {
          mm <- matrix(1L, nn2, dims[2])
          if (i > 1L) mm[, 1:(i - 1L)] <- m[, 1:(i - 1L)]
          if (i + n <= dims[2]) mm[, (i + n):dims[2]] <- m[, i:nn1]
          mm
        }
      )
    )
  )
  # for each row in m, which column sets would it increment?
  # aggregate unique counts to feed the creation of a transition matrix
  dtM <- as.data.table(
    m %*% (diag(0L, dims[2], nn) + rep(c(rep(1L, n), integer(nn)), nn)[1:(dims[2]*nn)]) == n
  )[
    , .(k = .N), eval(paste0("V", 1:nn))
  ]
  mTrans <- as.matrix(dtM[, 1:nn])
  class(mTrans) <- "numeric"
  # build a data.table listing all the state changes that don't end in the
  # initial state, and the number of ways the state change can occur
  dtTrans <- setorder(
    fTrans(
      setnames(
        as.data.table(as.list(integer(nn))),
        paste0("from", 1:nn)
      ),
      mTrans, dtM$k, n, nn
    )
  )
  dtTrans[
    , `:=`(
      state1 = rleidv(dtTrans[, 1:nn]),
      state2 = setorder(
        setorder(
          dtTrans[, (nn + 1L):(2L*nn)][, idx := .I]
        )[
          , state2 := rleidv(.SD) + 1L, .SDcols = 1:nn
        ], idx
      )$state2
    )
  ]
  # build the transition matrix, calculate dims[1] transitions, and get the
  # probability of ending in the absorbing state (when the desired submatrix has
  # occurred)
  nStates <- max(dtTrans$state2)
  m <- matrix(0, nStates, nStates)
  m[as.matrix(dtTrans[, state1:state2])] <- dtTrans$k
  m[nStates, nStates] <- 2^dims[2] # absorbing state
  m[, 1] <- 2^dims[2] - sum(dtM$k)
  m[nStates, 1] <- 0
  matpow(m*p^dims[2], dims[1])[1, nStates]
}

# test the pGridInArray function
pGridInArray(2L, c(5L, 3L))
#> [1] 0.3262634
findSquare(as.matrix(expand.grid(rep(list(0:1), 15))), 5L, 3L, 2L)/2^15
#> [1] 0.3262634

pGridInArray(3L, c(5L, 4L))
#> [1] 0.009971619
findSquare(as.matrix(expand.grid(rep(list(0:1), 20))), 5L, 4L, 3L)/2^20
#> [1] 0.009971619

pGridInArray(5L, c(100L, 5L))
#> [1] 2.772544e-06
p1
#> [1] 2.772544e-06

# 5-by-5 filled submatrix in 100 rows x 5-11 columns
p5 <- vapply(5:11, function(i) pGridInArray(5L, c(100L, i)), 0)
dt <- data.table(
  columns = 5:11,
  prob = p5,
# probability of finding the submatrix in the last 5 columns
# given that it is not anywhere else
  marg = c(NA, diff(p5)/(1 - first(p5, -1)))
)[
# project the probability of finding the submatrix in 100 columns
# using the last "marginal" probability
  , proj := 1 - (1 - prob)*(1 - marg)^(100 - columns)
]
dt
#>   columns         prob         marg         proj
#> 1       5 2.772544e-06           NA           NA
#> 2       6 5.454110e-06 2.681573e-06 0.0002574892
#> 3       7 8.135459e-06 2.681364e-06 0.0002574695
#> 4       8 1.081678e-05 2.681347e-06 0.0002574679
#> 5       9 1.349810e-05 2.681345e-06 0.0002574678
#> 6      10 1.617941e-05 2.681346e-06 0.0002574678
#> 7      11 1.886071e-05 2.681346e-06 0.0002574678

# show that the estimate is converging
diff(last(dt$proj, -1))
#> [1] -1.969634e-08 -1.576093e-09 -1.256927e-10  2.543288e-11 -5.694340e-13

# final estimate
last(dt$proj)
#> [1] 0.0002574678

Unfortunately, I don't have a method to place error bounds on the final estimate, nor can I prove that the estimate is strictly converging with each iteration.


Create custom tooltip data C3 pie chart

Question: Im trying to create a detailed tooltip on C3 pie chart (source), here my starting code :
var chart = c3.generate({
     data: {
         columns: [
             ['Error', 72,15,11],
             ['Success', 58,42,5],
        ],
        type : 'pie'
     },
     tooltip: {
         format: {
             title: function (d) { return 'Data ' + d; },
             name : function(name) { return  name; }
         }
     }
});

Here what i want to display :
I Tried with an array in data with no-success
 columns: [
        ['Error': {name : 'Stage 1', value : 72},{name : 'Stage 2', value : 15},{name : 'Stage 3', value : 11}],
        ['Success': {name : 'Stage 1', value : 58},{name : 'Stage 2', value : 42},{name : 'Stage 3', value : 5}],
    ],

Here my JSFiddle
Answer: It seems like grouping the tooltip isn't implemented for pie charts.
The only way I found that works is to make a copy of the data array and return a custom   tooltip:
var columns =  [
             ['Error', 72,15,11],
             ['Success', 58,42,5], ];

var chart = c3.generate({
     data: {
         columns: [
             ['Error', 72,15,11],
             ['Success', 58,42,5],
        ],
        type : 'pie'
     },
     tooltip: {
       contents: function() {
        return "
" + columns[0][0] + "" + columns[0][1] + "
"
                     + "
" +
               "
" + columns[1][0] + "" + columns[1][1] + "
"
       }
     }
});

Now you get the idea. Customize your tooltip however you like.

Create custom tooltip data C3 pie chart


how can we show an antiperiodic function?

Question: How can we graphically show an anti-periodic function? I can't imagine. Maybe I have got no imagination...!!
for example we can show the sin and cos or other periodic functions on the graphs. is it the possible for an anti-periodic one?
Answer: $f(x+p)=-f(x)$ then its anti-periodic
(to see intuitively graphically its like an odd function which is also periodic )
all anti-periodic functions with period $p$ are periodic with period $2p$
not all periodic functions have corresponding anti-periodic function
but all anti-periodic functions are periodic
it is enough to show that your function is periodic and odd with when shifted by half of the period.
example http://www.wolframalpha.com/input/?i=sin%28x%2Bpi%29
$sinx$ is periodic with $2\pi$ period so you check if $sin(x+\pi)=-sinx$ if it is then its anti-periodic with period $\pi$
Comment: i think there is a miss definition of periodicity here. because we say: it is anti-periodic in period ... . or we can say: really we don't have an anti-periodic function and we define some periodic functions in some intervals as anti-periodic.!!?
Comment: @P.A.M unfortunately its how its defined the language used in maths sometimes is confusing here anti-period implies its the same value except its opposite sign of the original value http://mathworld.wolfram.com/AntiperiodicFunction.html
Comment: perhaps it should be called periodic reflection on x-axis but the name anti-periodic is being used for some reason
Comment: although it sounds right but i can't imagine it boy. :) i think we don't have anti-periodicity of a real function.
Comment: technically for any real valued function in interval we can assume it is anti-periodic with period as the size of interval
Comment: here, is sin(x+pi) anti-periodic or is sin(x) anti-periodic in an interval ?
Comment: $sin(x+\pi)$ is anti periodic with period $\pi$ and also periodic with period $2\pi$
Comment: then, it should be so that: sin(x+pi) is anti-periodic compared to the sin(x). not compared to itself!!


laravel 9 create sidebar in admin panel

Question: I would like to make a dashboard for the administrator. I have seen several tutorials but they are all either you have to integrate a package or a template. I am trying to make an admin dashboard from scratch by myself.
Admin home.blade.php
@extends('layouts.app')

@section('content')

Super Admin Dashboard

                    @if (session('status'))

                            {{ session('status') }}

                    @endif
                        This is Admin Dashboard. You must be super privileged to be here !

                        @auth
                            @if(Auth::user()->hasRole('ROLE_SUPERADMIN'))

                                    Dashboard

                                    Categories

                                    SubCategories

                            @endif
                        @endauth

@endsection

I'm trying to make a sidebar in bootstrap 5 and laravel 9 but I can't. When I remove the @extends at the top , the page shows up in white. But I would like to remove this navbar in the dashboard and make a vertical navbar by myself
Categories
SubCategories
I would like the navbar on the left to make a dropdown design and put the categories and subcategories in this dropdown, what should I do if I can't delete the extends with the layout? can someone give me the answer to this problem?
Answer: You can pass parameters to @extend
@extends('layouts.app', ['navbar' => false])

@section('content')
{{-- Your content --}}
@endsection

And in your layouts.app

@if(!isset($navbar) || $navbar)

{{-- Your navbar code --}}

@endif

laravel 9 create sidebar in admin panel


Complete fingering-technique reference book for keyboard player for chords and scales?

Question: I'm now studying harmony and since my instrument is not keyboard, I don't know how to execute complex chords such as 13 or other chords that involve more than 5 notes.
Do you play them with just one hand while playing the melody?
I think I'm looking for a basic fingering bible of all chords (left-hand), scales (R & L), and arpeggios, preferably free (PDF etc.) of course.
Answer: AT most in the left hand, you'll play four tones simultaneously, but more often you're playing just two or three.
Generally when you are harmonizing the melody, you will voice chords in both hands with the melody note on top. A standard way to do this in jazz is put the root and third or root and 7th in the left hand, and fill out two more of the chord tones in the right plus melody note. This gives five tones across both hands, it's grounded because you're including the bass note on the bottom.
When you voice chords, important tones are root, 3rd, 7th, 9th and any altered tones. The fifth is not so important because it doesn't add to the harmony so it's sometimes excluded. So if you pick and choose the important ones to include. You can't include them all. So you have to leave out some.
There is no fingering bible of chords because different styles of music do different things. Gospel players use different chord voicing than pop players, which is different than etc. So it depends on what style of music you're doing.
Comment: Great helpful answer. As I learn in a self-paced manner, and never really went to a piano or keyboard teacher, this post was really helpful. Any further info or redirection to articles or other resources on the subject will be much appreciated!
Comment: You're welcome, Shimmy. Glad I could help.
Answer: For scales and finger exercises, the Hanon studies are good.  You can find them here.  Part II has all the scales and arpeggios.
I wouldn't read the introduction and try to do as recommended (go through all the exercises from end to end over the course of a couple of years).  Those Victorians were gluttons for punishment, after all.  Pick and choose.  That said, scales and arpeggios are important and it's a good idea to learn all of them.


Show that $ \hat{\theta}_2 = Y_{(n)} - \frac{n}{n+1}$ is unbiased estimators of $θ$.

Question: Let $Y_1$, $Y_2$, . . . , $Y_n$ denote a random sample from the uniform distribution on the interval $(θ, θ + 1)$. Let $$ \hat{\theta}_2 = Y_{(n)} - \frac{n}{n+1}$$
Show that $\hat{\theta}_2$ is unbiased estimators of $θ$.
I am told that the density of $\hat{\theta}_2 = Y_{(n)}: g_n(y) = n(y-\theta)^{n-1}, \theta \le y \le \theta +1$. My question is how to find these?
Then, the second part of the solution is $E(\hat{\theta}_2) = E(Y_{(n)}) - \frac{n}{n+1} = \theta$...
However, how does this follow from the previous answer?
Also, I try to compare the efficiency of $θ^1$ relative to $θ^2$... I am wondering how to find the $\text Var [\hat{\theta}_2]$ which I am told that $\text Var [\hat{\theta}_2]= V(Y_{(n)})=\frac{n}{(n+2)(n+1)^2}$ I am wondering how they calculate $\text Var [\hat{\theta}_2]$?
$$E(Y_{(n)}^2) = ny^2(y-\theta)^{n-1} = n\left[\left.y^2\frac{(y-\theta)^n}{n} \right|_\theta^{\theta+1} - \frac{2}{n} \int_\theta^{\theta+1} y(y-\theta)^n  \,dy\right]$$
$$= \left.(\theta+1)^2 - 2\left(y\frac{(y-\theta)^{n+1}}{n+1} \right|_\theta^{\theta+1} - \int_\theta^{\theta+1} \frac{(y-\theta)^{n+1}}{n+1} dy\right) = (\theta+1)^2 -2 \left(\frac{\theta+1}{n+1} - \left.\frac{(y-\theta)^{n+2}}{(n+1)(n+2)}\right|_\theta^{\theta+1}\right)$$
$$= (\theta+1)^2- 2\frac{\theta +1}{n+1} - \frac{1}{(n+1)(n+2)}$$
I check the Latex for the middle two lines... I could not figure out what went wrong please focus on my answer ...If I use $E(Y_{(n)}^2)$ - $E(Y_{(n)})^2$, based on wolframalpha which gives me the following result ...please go to the link https://www.wolframalpha.com/input/?i=%28x%2B1%29%5E2-2%28%28x%2B1%29%2F%28n%2B1%29%29+%2B+1%2F%28%28n%2B1%29%28n%2B2%29%29+-+%28%28x%2B1%29+-+1%2F%28n%2B1%29%29%5E2 ...which is different from the correct answer $\text Var [\hat{\theta}_2]= V(Y_{(n)})=\frac{n}{(n+2)(n+1)^2}$....Could anyone please check why
Answer: We have $Y_i\sim\mathcal{U}(\theta,\theta+1)$ and CDF of $Y_i$ based on Wikipedia
$$
G_{Y_i}(y)=\Pr[Y_i\le y]=\frac{y-\theta}{\theta+1-\theta}=y-\theta.
$$
Here, $Y_{(n)}$ is $n$-th order statistics. Therefore, $Y_{(n)}=\max[Y_1,\cdots, Y_n]$. Note that $Y_{(n)}\le y$ equivalence to $Y_i\le y$ for $i=1,2,\cdots,n$. Hence, for $\theta< y<\theta+1$, the fact that $Y_1,Y_2,\cdots, Y_n$ are i.i.d. implies
$$
G_{Y_{(n)}}(y)=\Pr[Y_{(n)}\le y]=\Pr[Y_1\le y,Y_2\le y,\cdots, Y_n\le y]=(\Pr[Y_i\le y])^n=\left(y-\theta\right)^{n}.
$$
The PDF of $Y_{(n)}$ is
$$
g_{Y_{(n)}}(y)=\frac{d}{dy}G_{Y_{(n)}}(y)=\frac{d}{dy}(y-\theta)^n=n(y-\theta)^{n-1}.
$$
The expected value of $Y_{(n)}$ is
$$
\begin{align}
\text{E}\left[Y_{(n)}\right]&=\int_{y=\theta}^{\theta+1}yg_{Y_{(n)}}(y)\ dy\\
&=\int_{y=\theta}^{\theta+1}yn(y-\theta)^{n-1}\ dy\\
&=n\int_{y=\theta}^{\theta+1}y(y-\theta)^{n-1}\ dy.
\end{align}
$$
Integral above can be solved by using IBP or integration by reduction formula (you may refer here).
$$
\begin{align}
\text{E}\left[Y_{(n)}\right]&=n\left[\frac{y(y-\theta)^n}{n+1}-\frac{\theta(y-\theta)^n}{n(n+1)}\right]_{y=\theta}^{\theta+1}\\
&=\frac{n(\theta+1)}{n+1}+\frac{\theta}{n+1}\\
&=\theta+\frac{n}{n+1}.
\end{align}
$$

ADDENDUM :
Using IBP, let $u=y\Rightarrow du=dy$ and $dv=(y-\theta)^{n-1}\ dy\Rightarrow v=\dfrac{(y-\theta)^n}{n}$. Hence
$$
\begin{align}
n\int_{y=\theta}^{\theta+1}y(y-\theta)^{n-1}\ dy&=n\left[\left.\dfrac{y(y-\theta)^n}{n}\right|_{y=\theta}^{\theta+1}-\int_{y=\theta}^{\theta+1}\dfrac{(y-\theta)^n}{n} dy\right]\\
&=(\theta+1)-\int_{y=\theta}^{\theta+1}(y-\theta)^{n}\ dy\\
&=(\theta+1)-\left.\dfrac{(y-\theta)^{n+1}}{n+1}\right|_{y=\theta}^{\theta+1}\\
&=(\theta+1)-\dfrac{1}{n+1}
\end{align}
$$
Variance can be evaluated as follows
$$
\text{Var}\left[\hat{\theta}_{2}\right]=\text{Var}\left[Y_{(n)}-\frac{n}{n+1}\right]=\text{Var}\left[Y_{(n)}\right]=\text{E}\left[Y_{(n)}^2\right]-\left(\text{E}\left[Y_{(n)}\right]\right)^2.
$$
Since the term $\dfrac{n}{n+1}$ is a constant, then you can ignore it. Variance is invariant with respect to changes in a location parameter. That is, if a constant is added to all values of the variable, the variance is unchanged. You may refer here to learn properties of variance.

Thus
$$
\begin{align}
\text{E}\left[\hat{\theta}_{2}\right]&=\text{E}\left[Y_{(n)}-\frac{n}{n+1}\right]\\
&=\text{E}\left[Y_{(n)}\right]-\frac{n}{n+1}\\
&=\theta+\frac{n}{n+1}-\frac{n}{n+1}\\
&=\theta.
\end{align}
$$
Since $\text{E}\left[\hat{\theta}_{2}\right]=\theta$, we can conclude that $\hat{\theta}_{2}$ is unbiased estimator of $\theta$.
$$\\$$

$$\Large\color{blue}{\text{# }\mathbb{Q.E.D.}\text{ #}}$$
Comment: Could you show me how to use IBP to solve $\text{E}\left[Y_{(n)}\right]$?
Comment: @afsdfdfsaf Let $u=y$ and $dv=(y-\theta)^{n-1}$, then you will get $du=dy$ and $v=\dfrac{(y-\theta)^{n}}{n}$.
Comment: @afsdfdfsaf Done!
Comment: Thanks a lot. I have one more related question: I try to compare the efficiency of $\hat{\theta_1}$ relative to $\hat{\theta_2}$... I am wondering how to find the $V(\hat{\theta_2})$ which I am told that $V(\hat{\theta_2}) = V(Y_{(n)}) = \frac{n}{(n+2)(n+1)^2}$ I am wondering how they calculate $V(\hat{\theta_2})$?
Comment: @afsdfdfsaf Variance can be evaluated as follows
$$
\text{Var}\left[\hat{\theta}_{2}\right]=\text{Var}\left[Y_{(n)}-\frac{n}{n+1}\right]=\text{Var}\left[Y_{(n)}\right].
$$
Since the term $\dfrac{n}{n+1}$ is a constant, then you can ignore it. You may refer [here](http://en.wikipedia.org/wiki/Variance#Properties) to learn properties of variance.
Comment: could you revise it a little bit since it is unreadable now?
Comment: $$
\text{Var}\left[\hat{\theta}_{2}\right]=\text{Var}\left[Y_{(n)}-\frac{n}{n+1}\right]=\text{Var}\left[Y_{(n)}\right]=\text{E}\left[Y_{(n)}^2\right]-\left(\text{E}\left[Y_{(n)}\right]\right)^2
$$
Comment: @afsdfdfsaf I don't know what is wrong with my LaTeX code. you can copy my last LaTeX code to your **ask question editor**. It's readable there.
Comment: could you repaste that under your answer as I am going to change my question at the same time?
Comment: I cannot copy your last latex since some of the text are missing
Comment: I've copied my comment to my answer since an hour ago. Now you can check it. It's already clear enough and it's should be easy to obtain the variance. You can do it by yourself.
Comment: Based your comment, I tried to calculate the $E(Y_{(n)}^2)$...However, I cannot find the correct answer please take a look of my revised questions
Comment: @afsdfdfsaf I just don't get it, what exactly are you asking? Is it $\text{Var}[\hat{\theta}_{2}]$ or $\text{Var}[\theta^{2}]$?
Comment: should be $\text Var [\hat{\theta}_2]$
Comment: @afsdfdfsaf How come I help you if you didn't accept yet this answer? Why you don't ask another question but this time only asking $\text Var [\hat{\theta}_2]$? Maybe other members will help you. I don't wanna edit this answer again. It's already full. If no one help you, I'll try to answer it tomorrow. At this time I couldn't help you. I'm not in home.
Answer: Note that $Y_{(n)}=\max\{Y_1,Y_2,...,Y_n\}$. Given i.i.d. observations the cdf of $Y_{(n)}$ is given by
$$
G_{Y_{(n)}}(y)=\mathsf{P}(\max\{Y_1,Y_2,...,Y_n\}\le y)=[G_{Y}(y)]^n=(y-\theta)^n,
$$
and the density of $Y_{(n)}$ is
$$
g_{Y_{(n)}}(y)=\frac{d}{dy}[G_{Y}(y)]^n=n(y-\theta)^{n-1}.
$$
Thus,
\begin{align}
\mathsf{E}[\hat{\theta}_2]& = \int_\theta^{\theta+1}yn(y-\theta)^{n-1}dy-\frac{n}{n+1} \\
&=\frac{\theta(n+1)+n}{n+1}-\frac{n}{n+1}=\theta.
\end{align}


Android ListView

Question: Como posso fazer com que uma ListView monte os itens da seguinte forma:
Quando tiver apenas um item ele ocupe a tela inteira, se tiver dois divida em 50% o tamanho da tela entre os itens e assim por diante...
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
   xmlns:tools="http://schemas.android.com/tools"
   android:id="@+id/RelativeLayout1"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   android:orientation="vertical"
   tools:context=".ListViewActivity" >

<ListView
    android:id="@+id/listaromaneiosview"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:divider="@color/Black"
    android:dividerHeight="5dp"
    tools:listitem="@layout/cre_004_listaromaneios_adpter" >

</ListView>

<TextView
    android:id="@+id/textoListView"
    android:text=""
    android:layout_width="wrap_content"
    android:layout_height="match_parent"
    android:layout_weight="3"
    android:textSize="@dimen/titulo"/>

<ImageButton
    android:id="@+id/selecionaRomaneio"
    android:layout_width="30dp"
    android:layout_height="match_parent"
    android:layout_gravity="left"
    android:layout_weight="1"
    android:background="@color/White"
    android:contentDescription="@string/app_name"
    android:maxWidth="100dp"
    android:onClick="actionListner"
    android:src="@android:drawable/ic_menu_delete" />

Comment: Tente colocar a altura deles como fill_parent.
Comment: já tentei não deu certo, ele não ficou na tela cheia..
Comment: Qual o layout que vc ta usando?
Comment: RelativeLayout e LinearLayout para o item da lista
Comment: A ListView precisa ficar como fill_parent e o elemento pai do Adapter que o TextView e ImageButton estão precisa estar como fill_parent também. Tenta assim acredito que deve funcionar!
Comment: Acho que esse comportamento não é possível, talvez terá que fazer "na mão" usando um `LinearLayout` e usando o `weight` e `weightSum` dos filhos.
Answer: ListView, do ponto de vista dos filhos, tem altura infinita, não faz sentido um match_parent. O que você precisa é de um LinearLayout (não se esqueça do orientation="vertical") e vai adicionando os filhos com layout_height="0dp" e layout_weight="1" (o valor não importa, desde que seja o mesmo para todos os itens). Por questões de performance, coloque também o weightSum no LinearLayout como a soma dos pesos (weight) dos filhos. É possível fazer isso em tempo de execução também com setWeightSum().
Comment: obrigado vou testar !!

Android ListView


Vertical line test

Question:
A vertical line crossing the x-axis at a point $a$ will meet the set in exactly one point $(a, b)$ if $f(a)$ is defined, and $f(a) = b$.

If the vertical line meets the set of points in two points then $f(a)$ is undefined?
Comment: what is 'the set' here?
Answer: The highlighted proposition is one way of describing the vertical line test, which determines whether $f$ is a function.
If there is one and only point of intersection between $x = a$ and $f(x)$, then $f$ is a function.
If there are two or more points of intersection between $x = a$ and $f(x)$, then $f$ maps a given $x = a$ to two (or more) distinct values, $f(a) = b, f(a) = c, \; b \neq c$, and hence, fails to be a function, by definition of a function. $f$ may, however, a relation.
Answer: (Short answer)
No.  Rather, we conclude $f$ is a relation, not a function.
Response to comment:
A real function of one variable is really saying three things:

It's a real mapping.  This means that the object in question is a mapping from $\mathbb{R}^n \to \mathbb{R}^n$.  In "normal talk," this means that you can put any real number in, and you get real numbers out.  The object in question doesn't accept complex numbers.
It's a function.  This means that every input has exactly one output.  That is, you can't put in a number and be able to get out multiple numbers.
It's of one variable.  This means that there is only one value used for each input.  An example of a multivariable function is $f(x, y, z) = xyz$.  This is a function of 3 variables.

Does this help?
Comment: can you please explain me what is _a real function of one variable_?
Comment: @SamamaFahim See edit
Comment: I'm posing another question which will tell you what's confusing me.
Answer: One definition of a function is  Formal Dirichlet-Bourbaki definition. This defines a function $f: X\to Y$ to be any subset $S$ of the set $X\times Y$ satisfying that if $(x,y)\in S$ and $(x,y')\in S$, then $y = y'$.
In more vague terms: a function is a rule $f$ that assigns to each element of the domain $X$ exactly one element $y$ in the range. We write $y = f(x)$. So that means that if $x_1 = x_2$, then you will necessarily have $f(x_1) = f(x_2)$.
Now if we are talking about functions from (subsets of) the real numbers to the real numbers, then we have a graph. If in independent variable ($x$) is our horizontal axis and the dependent variable ($y$) our vertical axis, then we can talk about the vertical line test. The graph is then $\{(x,f(x))\mid x\in X\}$. If a vertical line intersects the graph in two points, then we have one value of $x$ corresponding to two distinct values of $y$. That is we have $f(x) = y_1$ and $f(x) = y_2$ say.

Vertical line test


How to dynamically update ASP.NET Core View

Question: I am new to ASP.NET Core Razor(or MVC) page web development. I have an ASP.NET core web site deployed in a docker container and I have few other docker containers which are just .NET Core modules. Between the modules and the ASP.NET Core web page they would be talking over an RabbitMQ message bus or an Azure IOT Edge hub. Now if I get a message from one of the .NET Core module,I want to update the ASP.NET Core Index page view. I googled and I couldnt find a clear answer. Is polling using Jquery or Javascript and updating the view using Ajax the only option?
Comment: [SignalR](https://www.asp.net/signalr)
Comment: Thanks. That looks a interesting option. Havent used it before. So how do I populate a SignalR class when I get a message on the hub? I see the code to get the instance of the hub class in javascript, but can I do it when I get data from Queue asynchronously?
Answer: What you need is full duplex communication between your client (webpage) and server (asp.net core MVC).
With a classic MVC Razor webapp, communication is half-duplex; it is one way from your client to your server, and unfortunately, never the other way around.
Popular solutions that would enable what you want to achieve are:

https://github.com/websockets/ws
https://www.asp.net/signalr
Comment: Thanks. I used the DI and got the instance of the hub.
Comment: SignalR seems a best fit looking at the chat sample. Only question I have is how I get instance of the hub class to broadcast message from non hub class. lets say from the program.cs file where i register for RabbitMQ broker messages
Comment: This is another question in itself. You may post another SO question then !


How do I parse every string while comparing two excel spreadsheets?

Question: Good evening,
I'm attempting to compare two excel spreadsheets by using the IF and MATCH functions as follows:
<code>=IF(ISERROR(MATCH(fromADP!$C2,fromSMS!$A$2:$A$4792,0)),"No match found",fromADP!$C2)
</code>
I have two worksheets (fromADP and fromSMS).  I'm trying to compare the two worksheets to find out which records in the fromADP worksheet appear in the fromSMS worksheet.  The MATCH function allows me only three options for the match_type arguement.  I'm using 0, although I'm not sure I understand exactly how the other two options work.  I tried them though without desirable results.
When I use match_type 0 I only get one match - but this is an exact match (as I would expect).  My problem is, some of the records do in fact exist in both worksheets but there are minor differences (for example, "Tony's" vs. "Tonny's" or "Jimmy's Trucking, LLC" vs. "Jimmy's Trucking").
So I'm wondering, is there another way to do this or could there be - perhaps - a vbscript that would parse each string in my lookup_value?  This way, I can find those records where there might be slight differences.
I'm afraid I may simply have to pull out my ruler and pencil and start combing through the spreadsheets, line-by-line.  Any help would be appreciated.
Comment: There is neither VBA nor VBScript code in your question. Questions about Excel formulas are better asked on [SuperUser](http://superuser.com/).
Comment: Thanks, I wasn't aware of that.  I will move the question there.  Thanks for the direction.
Answer: Hi all,
Thanks to the solution offer here, I was able to use the Fuzzy Lookup Addin for Excel to accomplish this task.  Thus, my question is answered and my issue resolved.


Duplicate schema/structured data markup?

Question: Will this structure be problematic?

The reason there could be "duplicate" markup like this for a single category page is that the page may use multiple product templates.
In the current implementation, the markup is dynamically constructed in the product template. For example, if there are two product templates for a single Category Page, the markup will be reconstructed twice, but containing different WebPageElement.
Will this yield bad results? I checked in Google's testing tool and it didn't give me any errors or warnings.
Comment: About your use of Schema.org: 1) I wouldn't use `WebPageElement` at all here. You could directly provide the `Offer` nodes as value of `mainEntity`. 2) Unless a single offer really consists of two products (i.e., you can get them only together), each `Product` should get its own `Offer`. This question isn't the right place to discuss this, though, so if you are interested in it, you could ask on [codereview.se].
Answer: Multiple nodes, same entity
If you have multiple nodes that represent the same entity on a page, the best practice is to give these nodes the same URI as identifier.
With JSON-LD, you can provide identifiers with @id.
So

both of your WebPage items could get "@id": "" (for the current URL; preferably specify your canonical URL here),
both of your BreadcrumbList items could get "@id": "#breadcrumbs",
both of your ListItem-1 items could get "@id": "#breadcrumbs-1", and
both of your ListItem-2 items could get "@id": "#breadcrumbs-2".

That way, Google's SDTT will display each of these items only once, because it now knows that they are about the same entity.
Referencing nodes instead of duplicating them
@id also allows you to reference nodes instead of embedding them (and thereby duplicating their data). See an example.
In your case this would have the advantage that you don't have to duplicate the WebPage/BreadcrumbList/ListItem nodes to begin with. You would specify these nodes once, and each product template would then outpout only the Offer/Product nodes. These nodes could include (reverse) references to the WebPage/etc. (might be easier for you to implement), or the WebPage/etc. could reference these nodes.
Comment: Thanks! I just have one question then, so will having multiple nodes for the same entity affect the page's performance in SERP?
Comment: @herondale: SEO is off-topic on Stack Overflow (it's on-topic on [webmasters.se], e.g., see [Schema.org and SEO](https://webmasters.stackexchange.com/q/108333/17633)). If you mean rich results with "performance in SERP", there should be no reason to assume this. Either a search engine recognizes that these are same entities, or not; either a search engine does something with multiple or with one entity etc.


Can Windows Server intercept SNMP and write to the event log?

Question: We have a software package that can speak SNMP when things go awry.  Currently, we're using Tivoli to monitor SNMP and send out alerts as needed.  We've been informed that our Tivoli installation is going away, and that all operations notifications will need to be SCOM monitoring Event Log entries.
Unfortunately, modifying the software package to write to the event log is a pretty healthy undertaking, and the end result would be the service calling a command-line utility to write to the event log.  Not a pretty solution at all.
So, my question is this. . . Is there anything inherent within Windows Server 2003 that can monitor SNMP traps and write the appropriate data to the event log?  Alternatively, can SCOM be configured to listen for SNMP traps.
Answer: For the first question, nothing that I'm aware of in Windows will run on ServerA, and listen for all SNMP traps that it tries to send and inserts them into the event viewer.
Doing some quick reading however, SCOM will listen for SNMP traps. I don't know if it's easy or the best way, but it's the tool you've got.
Comment: Thanks.  I will check with our operations people and see if they have the understanding of SCOM to implement this.


Using regression splines for values outside of the calibration range

Question: in my question on a load forecast model using temperature data as covariates I was advised to use regression splines. This really seems to be a/the solution.
Now I face the following problem: if I calibrate my model on winter data (for technical reasons calibration can not be done on a daily basis, rather every second month) and slowly spring arises I will have temperature data outside of the calibration set.
Are there good techniques to make the regression spline fit robust for values outside of the calibration range? Any experiences or references? Thanks!
Comment: The general answer is a resounding, emphatic *no* for reasons Stephan Kolassa has outlined (but perhaps not forcefully enough). Unless you have *theoretical* reasons to suppose that the spline ought to extrapolate properly--and that seems unlikely, given that splines are general-purpose and therefore arbitrary--then you should expect the spline to fail, perhaps dramatically, outside the calibration region.
Comment: Thanks for your comment ... I understand the reasoning very well. I might be faced with a situation where I just have to do something .. . and make the smallest error ... but I have to do something.
Comment: What I do is very applied. The forecasts is one of many and there is hope that errors in one forecast do not weigh too much in the aggregate number. Thanks again for your straight comment. Although the situation will not be saved totally applying some flat or linear damped extrapolation could be a very dirty solution. Just, if I have to do something ... and can't do nothing.
Comment: I appreciate that at times you have to do *something*. What that tells us is you should be asking a different question.  Rather than seeking for ways to make a spline "robust," you really want to know something more general: what extrapolation procedures can minimize your loss?  Under some pretty general conditions, there are procedures that are better than splines--but they are all still horrible (unless you're lucky). To do well, you will want to incorporate as much information about the nature of the extrapolation as you can, perhaps by using a theoretical model and additional "soft" data.
Answer: The first thing to do is to make sure you are using something like natural splines. These are smooth piecewise cubic curves inside your training set - but they are only linear outside the training set, so they can be used in (careful) extrapolation. In R, you can use the ns() function in the splines package for this.
And the second thing to do is to be very careful and think about what you are doing. Quite probably you will need to feed additional assumptions into your model as to how the result will look like in summer. For instance, if I regressed sales of canned soup, apples and ice cream on temperature in winter and used the result to predict sales in summer, I would not anticipate a very good forecast. And there is really no way to get a very good result just from the winter data alone.
Comment: I have contacted the authors - maybe they find time to give a short answer.
Comment: I would like to use the package [SemiPar](http://cran.r-project.org/web/packages/SemiPar/SemiPar.pdf) I will check for natural splines there - but do you already know whether this package can do this? Thanks!
Comment: Hm, I don't know. Have you looked into the reference provided in the help page for `spm()`? http://www.stat.tamu.edu/~carroll/semiregbook/ Unfortunately, the help page does not clarify whether the penalized splines it uses are restricted to be linear outside the training set (which I certainly could imagine to be the case).
Comment: I checked out the [manual](http://www.uow.edu.au/~mwand/SPmanu.pdf) and unfortunately I didn't find anything there.


How to implement Sliding tabs in ionic v1?

Question:
I am new in hybrid app development and I want to implement sliding Tabs in ionic v1. I searched for this topic on google but all gives code solution in ionic 2. And I am working on Ionic v1. Is it possible to implement sliding tab like material design in ionic v1?
Answer:
I have solve this issue,It can difficult to create slide tab in ionic 1
  but I have long research after i will found one code of slide tab follow this link to full code of sliding tab
Comment: Above link not available @vishal can you please provide some other link
Comment: Hi Sharad Kale, Sorry for link not longer exist. Ionic 1 have not support for scrolling tab But now you can use ionic 3 for better usage https://stackoverflow.com/questions/45157896/ionic-horizontal-scroll-tab-for-categories


How to post image to URL properly?

Question: I'm trying to post an image to the StockTwits API and am receiving a 422 error as a response saying that the image format can't be recognized. The image is a png file, so it should be a valid file format. I've tried using a jpeg with the same response.
Am I encoding the image incorrectly, or missing some configuration in the HttpClient?
public void Share(string text, string imageFilePath)
{
    using(HttpClient client = new HttpClient())
        using (MultipartFormDataContent formData = new MultipartFormDataContent())
        {
            string      url             = stocktwitsCreateMessageUrl + "&" +
                                            "body="         + Core.Globals.UrlEncode(twit)  + "&" +
                                            "sentiment="    + StockTwitsSentiment.ToString().ToLower();

            byte[]      imageBytes      = File.ReadAllBytes(imageFilePath);
            HttpContent imageContent    = new ByteArrayContent(imageBytes);
            string      fileName        = Path.GetFileName(imageFilePath);
            formData.Add(imageContent, "chart", fileName);
            HttpResponseMessage stockTwitsResponse  = await client.PostAsync(url, formData);
            Stream              responseContent     = await stockTwitsResponse.Content.ReadAsStreamAsync();
            string              result              = new StreamReader(responseContent).ReadToEnd();
            Print(result); // helper method
            if (!stockTwitsResponse.IsSuccessStatusCode)
            {
                LogErrorResponse(result, stockTwitsResponse);
                return;
            }
            else
                LogAndPrint(typeof(Custom.Resource), "ShareStockTwitsSentSuccessfully", new[] { Name }, Cbi.LogLevel.Information);
        }
}

Response:

{
"response": { "status": 422 },
"errors": [ { "message": "We couldn't recognize the image format. Format must be one of: image/jpg image/jpeg image/pjpeg image/png image/x-png image/gif" } ]
}
Answer: Don't forget to set your Content-Type header to "image/png"!
See: https://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.contenttype(v=vs.110).aspx
Comment: Adding the content-type to the imageContent object's headers solved the issue. Thanks!

How to post image to URL properly?


Is a disc that lands on top of a basket considered holed?

Question: This is rare case, but in the event that a disc lands on top of a basket (above the chains and basket in the round area that is often on top of the goal) is it considered holed or does it require an additional throw to actually hole out?
Answer: According to the PDGA competition rules:
QA7: Disc Resting on Top (DROT)
Q: I putted and my disc stayed on top of the basket. Now what?
A: You have not holed out. Mark your lie below the disc and continue.

Applicable rules: 803.13 Holing Out; 803.08 Disc Above Or Below the Playing Surface


Why need to estimate the distribution parameter through MLE when they can calculated them from observation?

Question: MLE is used to estimate parameters of distribution that can best fit our observed data. However, why do we go through these trouble since the parameters can be directly calculated from observed data? For example, to estimate the mean of a Normal distribution, we simply calculate the mean of the sample.
Comment: Remember that parameters are for populations.  Unless you have the entire population, you can't calculate the parameter from what you have. What you can calculate is a statistic, not a parameter. (Parameters - population; statistics - samples).
Comment: What parameters are you talking about?
Comment: The question is broad but clear enough to be answered. It shouldn't have been closed.
Comment: @Pere: I think it was rightly closed - I couldn't  understand what was being asked myself - but as the OP confirms the interpretation given in your answer, I've made a clarifying edit & re-opened it.
Answer: Very often, there are several possible (and reasonable) ways to estimate parameters from data. MLE gives us a sensible way to find one of them with some good properties.
I suppose the question comes from the fact that often the ML estimator is just the estimator anybody would have obviously used without all the trouble of finding the MLE. For example, even without knowing what likelihood is, anybody would rightly guess that the most reasonable way to estimate the mean of a normal distribution is to take the mean of the sample - in fact, our usual trouble is to make students tell apart sample mean from population mean.
However, not all cases are so obvious, because there isn't a self evident parallelism between sample statistics and distribution parameters, and sometimes there are several competing reasonable estimators or there are no simple estimator at all.
For example, mean and variance of Poisson distribution equal the lambda parameter. However in a Poisson sample, mean and variance are likely to be different - although similar. Should we estimate lambda as sample mean or as sample variance? Finding the ML estimator can be a good choice.
Comment: Thanks. The Poisson case is really helpful understanding. However for sampler situation like Gaussian distribution where no constrains like mean = variance in Poisson, can I assume the distribution parameter (mean & variance) estimated using MLE equal to parameter directly calculated from samples (considering that samples data is all we have)?
Comment: We can "assume" that mean and variance of the sample are the MLE for mean and variance of distribution because we can prove it. The only reason we don't need to go through all that trouble is that somebody else did it long ago and we just need to check in a book or in Google. However, it wasn't self-evident before proving it, even in Gaussian distribution. For example, in a Gausian distribution,without checking, I couldn't tell whether sample quartiles are MLE estimators of distribution quartiles, but I'm sure that sample maximum is not the MLE of distribution maximum - which doesn't exist.
Comment: @Pere: MLEs for any property of a Gaussian distribution must be a function of the sufficient statistic $(\sum x_i, \sum x_i^2)$, where $x_i, \ldots, x_n$ are independent Gaussian variates. That rules out sample quartiles' being MLEs of population quartiles. Or recall that MLEs are invariant to reparametrization, so you do know the MLEs of the distribution quartiles: they're $\Phi^{-1}(q, \hat\mu, \hat\sigma)$ for $q=\frac{1}{4}, \frac{1}{2}, \frac{3}{4}$, not the sample quartiles.
Answer: The keyword is "estimation". You can not calculate the exact parameter of a distribution from finite observations, only an estimate of it. Under certain regularity conditions, MLE has good properties (efficiency, asymptotic normality, consistency). Therefore, if you don't know what is the "best" estimation of a parameter, you can use MLE to find it.
Intuitively, MLE shows you what to calculate from your observations to estimate the parameter of interest.


How to select a view inside a custom module form

Question: I would like to have "select" with some views so I can chosse one of them and the submit the selection to display the result. something like this:

And this is close to what I want :
  $form['title'] = array(
'#type' => 'select',
'#title' => t('Display amount'),
//'#required' => TRUE,
'#options' => drupal_map_assoc(array('1','2', '3', '4', '5', '6', '7', '8')),
'#default_value' => 5,
'#ajax' => array(
    'callback' => 'form_ajax_submit',
    'wrapper' => 'contenido-div',
    'method' => 'replace',
    'effect' => 'fade',
   ),
    );

 $form['content'] = array(
'#type' => 'select',
'#title' => t('Content'),
'#options' => array(node_type_get_names()),
'#ajax' => array(
    'callback' => 'form_ajax_submit',
    'wrapper' => 'contenido-div',
    'method' => 'replace',
    'effect' => 'fade',
),
);

$form['views'] = array(
'#type' => 'select',
'#title' => t('views'),
'#options' => array(),
'#ajax' => array(
    'callback' => 'form_ajax_submit',
    'wrapper' => 'contenido-div',
    'method' => 'replace',
    'effect' => 'fade',
 ),
);

Thanks
Comment: Do you need to display nodes in view with settings (display amount and content type)?
Comment: Yes Eugene, that is what I want !
Answer: As Views is a CTools Export UI based module, the correct way to load all views is by using the CTools Export function: ctool_export_crud_load_all()
Example
ctools_include('export');
$views = ctools_export_crud_load_all('views_view');
$options = array();
foreach ($views as $view_id => $view) {
  $options[$view_id] = $view->human_name;
}


jQuery checkboxes error

Question: I am making a form validation and I set the property of other checkboxes disable if the owner checkbox is not checked but it is not working.
Here's my html cod:
<!DOCTYPE html>

    <title>jQuery</title>

Owner:<input type="checkbox" id="owner">

bicycle:<input type="checkbox" id="bicycle">

bike:<input type="checkbox" id="bike">

car:<input type="checkbox" id="car">

and in it min.js is jQuery installation file and ,
here is script.js :
$(document).ready(function(){

        var $checkbox = $("#check").find("input[type=checkbox]");
        $checkbox.prop('disabled', true);

        $("#owner").click(function(){
            var $ownercheck = $(this);

        if ($ownercheck.prop('checked') === true) {
                $checkbox.prop('disabled', false);
        }
        else
        {
            $checkbox.prop('disabled', true);
        }

    });

});

Comment: Put min.js before script.js.
Answer: You have to put min.js above the script.js,
and your html code will look like this :
<!DOCTYPE html>

    <title>jQuery</title>

Owner:<input type="checkbox" id="owner">

bicycle:<input type="checkbox" id="bicycle">

bike:<input type="checkbox" id="bike">

car:<input type="checkbox" id="car">

Answer: You must always load the library first. For issue to be fixed, script.js must go after min.js. Your code is working perfectly fine: http://jsfiddle.net/zb3m3/1/

jQuery checkboxes error


Python array changes values without reason

Question: In the following code, there is no reason I can see why variable "cleary" would change value, but it does. I have restarted pyCharm multiple times but it keeps happening.
import numpy as np

nPeriods = 48
nGens = 1
cleary = np.zeros((nPeriods,nGens,2))
clearz = np.zeros((nPeriods,nGens))

for ii in range(nPeriods):
    for jj in range(nGens):
        temp = cleary
        temp[ii,jj,:] = 1

What am I doing wrong?
Comment: This is expected behavior. Check out the documentation
Comment: Why do you expect that restarting your editor will change the core behavior of the Python language?
Comment: Duplicate: [List changes unexpectedly after assignment. Why is this and how can I prevent it?](https://stackoverflow.com/questions/2612802/list-changes-unexpectedly-after-assignment-why-is-this-and-how-can-i-prevent-it)
Answer: This is the expected behavior. You passed the reference of the ndarray object the name cleary is pointing to to temp when you did:
temp = cleary

You can avoid modifying cleary by assigning a copy of the array to temp:
temp = cleary.copy()

Read How do I pass a variable by reference? to learn more about the underpinnings of name assignments in Python.


Swift Convert A String-As-An-Array, To An Array Of Strings

Question: I have a string:
"["word1","word2"]"
And I want a simple way to convert it to an actual [String].
All the other questions I could dig up on there were about converting int strings to arrays.
I tried doing
Array(arrayLiteral: "["word1","word2"]")
But I get
["[\"word1\",\"word2\"]"]
Manually cleaning up the edges and removing the slashes seems like I'm doing something very wrong.
I'm curious if there's a simple way to convert a an array of strings as a string into an array of strings.
i.e. Convert "["word1","word2"]" to ["word1","word2"]
Solution (Thanks to @Eric D)
    let data = stringArrayString.dataUsingEncoding(NSUTF8StringEncoding)

    var stringsArray:[String]!

    do
    {
        stringsArray = try NSJSONSerialization.JSONObjectWithData(data!, options: []) as? [String]
    } catch
    {
        print()
    }

    print("Array is \(stringsArray)")

Comment: Where is the string comming from ?
Comment: Looks like JSON try using SwiftyJSON to parse it
Comment: There's stuff built in to Foundation, but not built into the Swift standard library as far as I know.
Comment: My app uses Cordova, so this is a command coming from a Javascript interface.
Comment: @LeoDabus I could use SwiftyJSON but I am curious if theres a quick way built into swift where I can move directly to a string array from a string formatted as an array
Answer: Encode your "string array" to data, then decode this data as JSON to a Swift Array.
Like this, for example:
let source = "[\"word1\",\"word2\"]"

guard let data = source.dataUsingEncoding(NSUTF8StringEncoding),
    arrayOfStrings = try NSJSONSerialization.JSONObjectWithData(data, options: []) as? [String] else {
        fatalError()
}

print(arrayOfStrings)  // ["word1", "word2"]
print(arrayOfStrings[1])  // "word2"

Comment: It's called a "guard statement" and you can read about it here: https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/ControlFlow.html#//apple_ref/doc/uid/TP40014097-CH9-ID525 in the "Early exit" section of "Control Flow". It's an alternative to `if let` that lets you use the unwrapped value in the current scope. It's very convenient. :)
Comment: Interesting I've not seen this syntax before! Could you name what `guard let data = source.dataUsingEncoding(NSUTF8StringEncoding),
    arrayOfStrings = try NSJSONSerialization.JSONObjectWithData(data, options: [])` is called? I'd like to read up on this
Comment: Very cool! Might want to update your answer to include the catch ;).
Answer: Agree with comments above, I'd probably use a JSON parser. Failing that (or if you can't for some reason), I do not know of any built-in way; you'd have to do it manually. I'd do something like:
extension String {
    func stringByTrimmingFirstAndLast() -> String {
        let startIndex = self.startIndex.successor()
        let endIndex = self.endIndex.predecessor()
        return self.substringWithRange( startIndex..<endIndex )
    }
}

let str = "[\"word1\",\"word2\"]"
let array = str
    .stringByTrimmingFirstAndLast()
    .componentsSeparatedByString(",")
    .map { string in
        return string.stringByTrimmingFirstAndLast()
    }


Laravel can't see the model of the module in its class

Question: I set up modular system in Laravel 4.1. But I have a problem with a model of a module. Here is my system scheme.
 app/modules
 app/modules/product/
 app/modules/controllers/ProductController.php
 app/modules/controllers/CatalogController.php
 app/modules/models/CatalogProductImage.php
 app/modules/models/ProductCatalog.php
 app/modules/models/CatalogProductImage.php

The problem is that, My ProductCatalog model can't see my CatalogProductImage model and hence it gives me the following error:
 Symfony \ Component \ Debug \ Exception \ FatalErrorException
 Class 'CatalogProductImage' not found

I don't know the reason. Here comes the code:
ProductCatalog.php
 <?php namespace App\Modules\Product\Models;

 use Illuminate\Auth\UserInterface;
 use Illuminate\Auth\Reminders\RemindableInterface;
 use Eloquent;
 use App\Modules\Product\Models\CatalogProductImage as CatalogProductImage;

class ProductCatalog extends Eloquent {
     protected $table           = 'catalog';
     protected $primaryKey  = 'catalogId';
     protected $connection  = 'mysql2';
     public  $timestamps        =  FALSE; //update time stamp varsa true yap, updated_at ve created_at alanlarını otomatik gunceller

     public function category() {
         return $this->hasOne('CatalogCategory', 'catalogCategoryId', 'catalogCategoryId');
     }

     public function images() {
        return $this->hasMany('CatalogProductImage', 'catalogId');
    }
 }

CatalogProductImage.php
 <?php namespace App\Modules\Product\Models;

 use Illuminate\Auth\UserInterface;
 use Illuminate\Auth\Reminders\RemindableInterface;
 use Eloquent;

 class CatalogProductImage extends Eloquent {

     protected $table            = 'catalog_image';
         protected $primaryKey  = 'catalogImageId';
     protected $connection   = 'mysql2';
         public $timestamps      =  FALSE; //update time stamp varsa true yap, updated_at ve created_at alanlarını otomatik gunceller

     public function catalog() {
         return $this->hasMany('ProductCatalog', 'catalogId');
     }

 }

Edit: I changed the CatalogProductImage From Modules... to App\Modules... but still doesn't work.
Comment: try to give full path App\Modules\Product\Models\CatalogProductImage
Comment: I tried it either but it didn't work :(
Comment: Did you issue a `composer dump-autoload` '
Comment: Yes I do it in all changes.
Answer: Did you add the directory in composer.json
"autoload": {
    "classmap": [
        "app/modules",
        "app/commands",
        "app/controllers",
        "app/models",
        "app/database/migrations",
        "app/database/seeds",
        "app/tests/TestCase.php"
    ]
},

Then composer dump-autoload
You can also try to set it in, app/start/global.php
ClassLoader::addDirectories(array(

Comment: Yes I dit it. This is not the answer.
Answer: Ok I found the problem. I was supposed to define full path in hasMany function. Like:
    return $this->hasMany('App/Modules/Product/Models/CatalogProductImage', 'catalogId');

Not like this:
    return $this->hasMany('CatalogProductImage', 'catalogId');


Função return após EOFError [Python]

Question: Boa tarde,
Preciso resolver o seguinte problema utilizando a estrutura While:

Dada uma sequência de valores reais, calcular sua média.

O comando dado pelo professor foi:
while True:
    try:
        valor = float(input())
        # processar o valor
    except EOFError:
        break

E eu desenvolvi o seguinte código:
while True:
    try:
        n = input()
        seq_break = n.split(',')
        armazena = 0
        cont = 0

        for val in seq_break:
            armazena += float(n)
            cont += 1
        a = (armazena/cont)
        print(a)

    except EOFError:
        break

Entretanto, estou tendo problema com a saída. Ao invés de me retornar a média dos valores (expressa na variável a) ele está me retornando ou todos os valores de entrada ou o último valor. Não sei onde eu estou errando.
Comment: Você está executando esse programa por alguma plataforma que o professor disponibilizou?
Answer: Você está iterando sobre os números armazenados no array seq_break com o valor val e não com o valor de n. O correto seria:
while True:
try:
    n = input()
    seq_break = n.split(",")
    armazena = 0
    cont = 0

    for val in seq_break:
        armazena += float(val)
        cont += 1
    a = (armazena/cont)
    print(a)

except EOFError:
    break

Função return após EOFError [Python]


What's the difference between methods of sourcing frontend socket.io?

Question: What's the difference between these two ways of including socket.io?

Answer: In the first you have a local copy of socket.io, in the other you are downloading from the cdn (Content Delivery Network). Either is fine, depending on your needs. If you are deploying an application, then the first may be necessary since the site where you are deploying may not allow you access to the CDN. However for public facing websites, the CDN is a good way to go.
Comment: No. An address like "/socket.io/socket.io.js" is a path relative to the website root, so it will be translated to something like "http://www.mysocketsite.com/socket/socket.io.js". So it's a copy that *you* provide. Both ways will be locked to a specific version. CDNs are optimised for fast delivery, so that may well be faster than hosting the script yourself.
Comment: If it answered the question, please mark as answer/upvote. Cheers!
Comment: Is the local copy of socket.io the copy in the client's browser? So it's whatever version they have? Whereas the other locks to a specific version?
Comment: Oh! Interesting. Thank you!


Controllability properties of discrete vs. continuous systems

Question: I'm not sure what differentiates discrete from continuous systems in trying to prove certain properties of controllability. I have a chain of proofs from class that prove each other for a continuous system:
We'll call our system $ \Sigma = (A,B,C)$
1) $ \Sigma $ is controllable if $(A,B)$ is controllable
2) The Gramian control matrix $W_c(t) $ is positive definite
3) There does not exist $ x=Ty $ with det $\space  T\neq0 $ such that
$T^{-1}AT=\begin{bmatrix} A_{11} \ A_{12} \\ 0 \ A_{22} \end{bmatrix}, \space T^{-1}B=\begin{bmatrix} B_1 \\ 0 \end{bmatrix}$
4) The Kalman rank condition is satisfied.
5) The PBH test is satisfied for rank$[A-\lambda I \space B]=n$ for all $\lambda \in \sigma(A)$ and for all $\lambda\in \mathbb{C}$
How is this applied to the discrete case?
Answer: I believe, not sure, this is all more or less the same discrete case. First all the system matrices need to be discretized [2].

Controllability, exactly the same [1].
I believe this still holds. The matrix equation stays the same $A W_c + W_c A^T = -BB^T$, only the integral gramian equation changes see [3].
I believe also still the same. Since we are now only working with the discretized system matrices.
Same, since we are now only working with the discretized system matrices.
Same, since we are now only working with the discretized system matrices.
Answer: Actually, it is more easy to prove these for discrete time case. Consider this:
Let $x_{k+1} = F x_k + G u_k$. Then we can write the following:
$x_1 = F x_0 + G u_0$
$x_2 = F x_1 + G u_1 = F^2 x_0 + FG u_0 + G u_1$
...
$x_n = F x_{n-1} + G u_{n-1} = F^n x_0 + F^{n-1}G u_0 + F^{n-2}G u_1 + \dots + G u_{n-1}$
$x_n - F^n x_0 = [F^{n-1}G ~~~ F^{n-2}G ~~~ \dots ~~~ G] [u_0 ~~~ u_1 ~~~ \dots ~~~ u_{n-1}]^T$
Remember the definition of controllability (strictly speaking the following one is reachability). We must find a finite $u_k$ such that the state of the system can be transferred to an arbitrary state $x_f$ from any initial state $x_0$. Now, let $x_f = x_n - F^n x_0$. Then we can find such $u_k$ if and only if the Kalman rank condition holds (follows from the last equation above).
Other conditions are equivalent to Kalman rank condition regardless of the system being continuous or discrete (except Gramian condition, here discrete Gramian should be used). They just follow from linear algebra.


How do I calculate the variance-covariance matrix for a set of 2-D data points with errors: (x, y, dy)

Question: I am doing a weighted linear least squares fit to N measured values of the form $(x, y, dy)$, where $x$ is the independent variable, $y$ is the independent variable, and $dy$ is the error estimates for the independent variable. Following the linked wikipedia page, I am able to get the two fit parameters but not yet able to get the parameter errors.  The article lists that the variance-covariance matrix for the parameters, $M^\beta$, can be calculated using
$$M^\beta = (X^T W X)^{-1} W M W^T X(X^T W^T X)^{-1}\,,$$
where $X$ is an $n \times 2$ matrix of the $x$-values and ones, $W$ is an $n \times n$ diagonal matrix with $1/dy_i^2$ values on the diagonal, and $M$ is the variance-covariance matrix for the observations.  How would I calculate $M$ from my $(x, y, dy)$ data?
This related question and this post use $$M_{jk} = \frac{1}{N-1} \sum_{i=1}^{N}(x_{ij} - \bar{x_j})(x_{ik} - \bar{x_k})$$
but should $dy$ somehow affect this?  This is straightforward to calculate but I am not sure which index the $x$ or $y$ values should take (with this only being a 2$\times$2 symmetric matrix it only affects the diagonal).
The article mentions a major simplification when $W = M^{-1}$ but I am not sure if this is the case for my data.  Is there a simple way to know this is the true without calculating $M$?
Comment: I believe the answer to this question requires a clearer explanation of what exactly the "$dy$" values represent.  (I suspect their squares might be proportional to the diagonal elements of $M$, but there's not enough information here to be sure.)  Could you tell us about the $dy$?
Comment: @whuber, The $dy$ values are the error estimates for the dependent variables, $y$.  I just updated the question to reflect this.
Answer: While this does not answer the question asked (and will therefore not be accepted), I provide this response in hopes it will help others and promote worthwhile discussion.
In his book Experimentation: An Introduction to Measurement Theory and Experiment Design, David Baird provides a simple explanation for doing a linear least squares fit using the diffences between the measured and fit values to estimate the error of the fit parameters.  The best fit for the parameters $m$ and $b$ in $$y=mx+b$$ will is determined using his eqn. (6.3):
$$m = \frac{n\sum x_i y_i - \sum x_i \sum y_i}{n\sum(x^2_i) - (\sum x_i)^2} $$
$$b = \frac{\sum(x_i^2)\sum y_i - \sum x_i \sum x_iy_i}{n\sum(x^2_i) - (\sum x_i)^2} $$
After obtaining $m$ and $b$, a standard deviation for the fit parameters can be obtained by calculating the differences of each $y_i$ value from the fit, $\delta y_i = y_i - (m x_i +b)$.  From these differences, one calculates the standard deviation of the data from the fit line using:
$$\sigma_y = \sqrt{\frac{\sum(\delta y_i)^2}{n-2}}$$
and then the standard deviation of the  parameters using
$$\sigma_m = \sigma_y \sqrt{\frac{n}{n\sum{x_i^2}-\left(\sum{x_i}\right)^2}}$$
and
$$\sigma_b = \sigma_y \sqrt{\frac{\sum{x_i^2}}{n\sum{x_i^2}-\left(\sum{x_i}\right)^2}}$$
To include the measurement error, $dy_i$, in the fit one would divide the initial system of equations by $dy_i$ giving $$\frac{y_i}{dy_i} = m\frac{x_i}{y_i} + b \frac{1}{dy_i}$$ then repeat Baird's derivation to get the weighted fit parameters
$$m = \frac{\sum \frac{1}{dy_i}\sum \frac{x_i y_i}{dy_i^2} - \sum \frac{x_i}{dy_i^2} \sum \frac{y_i}{dy_i}}{\sum \frac{1}{dy_i}\sum \frac{x^2_i}{dy_i^2} - \sum \frac{x_i}{dy_i}\sum \frac{x_i}{dy_i^2}} $$
$$b = \frac{\sum \frac{x_i^2}{dy_i^2}\sum \frac{y_i}{dy_i} - \sum \frac{x_i}{dy_i} \sum \frac{x_iy_i}{dy_i^2}}{\sum \frac{1}{dy_i}\sum \frac{x^2_i}{dy_i^2} - \sum \frac{x_i}{dy_i}\sum \frac{x_i}{dy_i^2}} $$
Notice that the $b\frac{1}{dy_i}$ term makes it so you cannot simply divide $x_i$ and $y_i$ by $dy_i$ (as pointed out in the comments below).
Unfortunately, this does not propagate the measurement error into an error in the fit parameters though.
Comment: The trick of "divide observations and predictors by error" (i.e. multiply by weight) works very generally as far as I know. For example the [Wikipedia link](https://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)#Weighted_linear_least_squares) by the OP shows the problem being solved is $\min_\beta\|W^{1/2}(y-X\beta)\|^2$, which shown the $\beta$ will be unchanged by multiplying the $W^{1/2}$ through. However note that in standard regression, not just $x$ but also "$1$" are predictors, i.e. $X=[x,1]$. I am not sure if your formulas account for the $1/dy_i$ "predictor".
Comment: @whuber I see your point but the error on the type of data i am considering is actually quite large.  I am actually trying to fully propagate this large error, and the variance from a straight line, doing both of these in an  accurate manner so I can make a case that the fit values are not as certain as people typically treat them.
Comment: Although this appears to be a correct description of one way to carry out weighted regression (modulo the caveat by @GeoMatt), it doesn't seem likely to solve your problem. The issue is that the $dy$ are only part of the error variance: there is another component reflecting other random deviations of the data from the idealized linear function you are fitting. Weighting by $1/dy$ in this fashion can grossly inflate the influence of data with small $dy$. To see why, consider an extreme case where $y$ is measured perfectly precisely, so $dy=0$. Your solution will force the line through $(x,y)$!
Comment: You could perhaps introduce a "minimum uncertainty", $\sigma_0$, and then use  $\sigma_i^2=dy_i^2+\sigma_0^2$ to compute the weights (a simple [shrinkage estimate](https://en.wikipedia.org/wiki/Estimation_of_covariance_matrices#Shrinkage_estimation)).
Answer: Looking at the formula for $M^\beta$, you can deduce the dimensions of $M$ should be $N \times N$, where $N$ is the number of data points.  Therefore, the variance-covariance matrix is not comparing all the $x$'s to all the $y$'s, which would give a $2 \times 2$ matrix, but instead comparing the $N$ data points.
The wiki article explaining the covariance matrix states that, "Indeed, the entries on the diagonal of the covariance matrix $\Sigma$  are the variances of each element of the vector $\mathbf {X}$."  If these are $dy_i^2$, and we can assume the errors between points are not related, then $M = W^{-1}$ and the simplification applies.


CAPTCHA session reuse attack detected error message when form is submitted

Question: I am experiencing problems with the captcha module 7.x-1.0-beta1 version. I have enabled captcha in webforms for anonymous users. After submitting the form it repeatedly gives me the error message,

CAPTCHA session reuse attack detected

.. This happens for only one of the webform in which captcha is placed. I am using Drupal 7.2 version.
Any solution for this issue will be really helpful.
Comment: Have you tried disabling 'Cache page for anonymous users'?
Answer: I had the same problem because I have an ajax file upload on the form. The solution is to patch the captcha module with the patch found here http://drupal.org/node/918856#comment-6353912 .
You just need to download the patch and put it in the same folder and run in command line:
patch < captcha-patch-10022012.patch

if using git, you can just use :
git apply --directory=your/module/derectory captcha-patch-10022012.patch

Comment: Thanks!  Worked for me, had to patch it using PHPstorm, since I'm on Windows (not by choice).  Netbeans will patch it, too... for any other Windows users out there!
Answer: there is a Patch available for Drupal 7, please check this out:
https://www.drupal.org/node/1395184


Apply Girsanov theorem to a Ornstein-Uhlenbeck process

Question: From the original paper by Schwartz (free access here, page 926):

Assume that the commodity spot price follows the stochastic process$^5$
$$\tag1
    dS = \kappa(\mu-\log S)Sdt+\sigma SdW,
$$
where $dW$ is an increment to a standard Brownian motion. Applying Ito lemma we can see that the log price $X = \log S$ is characterized by an Ornstein-Uhlenbeck process
$$\tag{2,3}
    dX = \kappa(\gamma-X)dt+\sigma dW, \quad \text{ with } \gamma = \mu-\frac{\sigma^2}{2\kappa},
$$
where $\kappa > 0$ measures the degree of mean reversion to the long run mean log price $\gamma$.

Now comes the part which I don't fully understand.

In this model, the commodity is not an asset in the usual sense$^6$ and the spot price, or equivalently the log of the spot price, plays the role of an underlying state variable upon which contingent claims can be written. Under standard assumptions, the dynamics of the Ornstein-Uhlenbeck process under the equivalent martingale measure can be written as$^7$
$$\tag4
    dX = \kappa(\gamma^*-X)dt+\sigma dW^*,
$$
where $\gamma^* = \gamma-\lambda, \lambda$ is the market price of risk (assumed constant)$^8$ and $dW^*$ is the increment to the Brownian motion under the equivalent martingale measure.

I'm trying to understand the steps to derive equation (4); in the article cited in $^7$ (see link below) it is written

With the stated assumptions on the economy (unfortunately, the page with the assumptions is missing from the preview), the appropriate transformation from the actual probability measure to the equivalent martingale measure is
$$\tag{12.14} dZ(t) = dZ^*(t) - \lambda t. $$
where $dZ^*(t)$ has zero mean and variance $dt$ under the *-probability measure.

So it seems that they applied Girsanov theorem. In the following I will use this version of Girsanov theorem from Stochastic differential equations: an introduction with applications by Øksendal

Following the notation of the theorem, in our case $\beta = \kappa(\gamma-X)$ and $\theta = \sigma$. Moreover let $u=\lambda$ (a constant), then
$$ M_t = \exp\bigg(-\int_0^t \lambda \,dB_s + \frac12\int_0^t \lambda^2 \,ds\bigg) $$
so the Novikov sufficient condition holds (recall that $\lambda$ is a constant so it goes out of the integral)
$$ E\bigg[\exp\bigg(\frac12\int_0^T \lambda^2 \,ds\bigg)\bigg] = E\bigg[\exp\bigg(\frac12\lambda^2T\bigg)\bigg] = \exp\bigg(\frac12\lambda^2T\bigg) < \infty \text{ since }\lambda,T < \infty,$$
hence $M_t$ is a martingale and we can apply the theorem, which tells us that $\widehat B = \lambda t+B$, so we have recovered the equation (12.14) above. Now, we have to find $\alpha$ such that $\theta u=\beta-\alpha$, that is
$$
\sigma\lambda = \kappa(\gamma-X) - \alpha \ \implies\ \alpha = \kappa(\gamma-X)-\sigma\lambda = \kappa\Big(\gamma-\frac{\sigma\lambda}{\kappa}-X\Big).
$$
So in terms of $\widehat B$ the process $X$ has the representation
$$\tag5
dX = \kappa(\hat\gamma-X) dt + \sigma d\widehat B, \quad\text{with }\hat\gamma = \gamma-\frac{\sigma\lambda}{\kappa},
$$
resembling equation (4), however notice that $\hat\gamma \ne \gamma^* = \gamma-\lambda$. Does this mean that the choice $u=\lambda$ is wrong? To obtain $\hat\gamma \equiv \gamma^*$ we have to choose $u = \kappa\lambda/\sigma$, however does the Novikov condition still hold with this choice for $u$?

$^5$ $^6$ Ross, S. A,, 1995, Hedging long run commitments: Exercises in incomplete market pricing
$^7$ Bjerksund, P., and S. Ekern, 1995, Contingent claims evaluation of mean-reverting cash flows in shipping, in L. Trigeorgis, Ed.: Real Options in Capital Investment: Models, Strategies, and Applications, Preager (partial preview here)
Answer: I think it might be possible that you got caught up in a sort of 'notation trap' and the vague notion of market price of risk (in this context) did the rest. The most plausible explanation is straightforward:
$$dX_t=\kappa(\gamma-X_t)dt+\sigma dW_t$$
$$dX_t=\kappa(\gamma-X_t)dt+\sigma (dW^*_t-\nu dt)$$
$$dX_t=\kappa(\gamma-\underbrace{\frac{\sigma\nu}{\kappa}}_{:=\lambda}-X_t)dt+\sigma dW^*_t$$
$$dX_t=\kappa(\gamma^*-X_t)dt+\sigma dW^*_t$$
Comment: Yes it's involved in the change from $W_t$ to $W_t^*$. Now I'm trying to find the Bjerksund and Ekern's paper to be sure about the model itself.
Comment: Thank you, let me know if you find it, I tried hard but all I found was the partial preview on google books.
Comment: @soundwave that would be the usual idea, but I want to find the cited paper first
Comment: Ok, I asked it on quant https://quant.stackexchange.com/questions/65916/why-is-the-market-price-of-risk-in-the-one-factor-schwartz-model-different-from
Comment: Thank you for answering, could you explain more about what's happening from $dW_t$ to $dW_t^* - \nu dt$? Moreover, is the Girsanov theorem involved in your proof?
Comment: Meanwhile, may I ask why in the paper Schwartz changes measure to the risk-neutral one? Is it because then the financial model is free of arbitrage?
Comment: Another thing that I don't understand is that in my course and in the book by Shreve the market price of risk is defined as $\frac{\mu-r}{\sigma}$ while in the Scwartz model it is of the form $\sigma\nu/\kappa$, maybe because the definition depends on the context?
Comment: @soundwave since it doesn't have a short answer, you should post that one as a question on here or on the economics/quantitative finance boards


Como declarar variáveis no for em c?

Question: Ele da erro, quando tento declarar uma variável em um laço como for, while.
Antes eu conseguia, tem a ver com a versão da linguagem?
Comment: Mas dá erro onde se não tem código? rs..
Comment: Não sei o que é pior, se é a pergunta sem exemplo, ou as respostas que explicam pra fazer exatamente o contrário da pergunta. @user41625 seria legal ao menos [edit] e por um exemplo de como funcionava antes, pra que o pessoal responda o que precisa ser feito pra continuar funcionando. Ou você mesmo testar e responder, já que pelo comentário do C99, matou a charada. Lembre-se que uma pergunta bem formulada atrai respostas mais objetivas, e ajuda a mais pessoas com a mesma dúvida.
Comment: @Bacco o pior é ter 3 votos positivos em nada. principalmente porque é raro um boa pergunta de C ter 3 votos.
Comment: O pior é este comentário: `Não tem código, só que ele da erro.` @Bacco
Comment: O ideal seria que você colocasse o seu código na pergunta para vermos o que está errado.
Comment: Não tem código, só que ele da erro.
Comment: @user41625 mesmo assim, seria legal vc por o código que usou e não deu certo na pergunta, e postar uma resposta mostrando como resolveu, isso poderia ajudar outras pessoas. Note que quando comentamos sobre problemas na pergunta, não é pra te criticar, e sim para apontar como é que você pode melhorar sua postagem.
Comment: creio que porque está usando a versão c99, ja tinha configurado o codeblocks uma vez e ele funcionava.
Comment: Foi mal, consegui resolver.
Comment: Use essa macro que com certeza vai funcionar `#define FOR(x,y) int x; for(x=0;x<y;x++)` o jeito de usar fica: `FOR(i,5){ puts("string"); }`. Obs: como a macro já faz a declaração da variável, não da para usar a mesma variável dentro do mesmo método, então não se acostume a usar essa macro.
Answer: Na linguagem C89, é obrigatória a declaração das variáveis no início de um bloco de código. Sempre após um abre "{". Seja dentro de um if ou no início do método.
Pode ser que antes você conseguia porque o compilador que você usava não utilizava as mesmas regras que o seu atual compilador (tem compiladores que não seguem só as regras do C89 e aceitam extensões como as GNU. Com isso, é possível declarar variáveis fora de ordem).
Agora ele não deve ser mais já que parou de funcionar. Ou você passou a passar pra ele a flag -pedantic, que instrui o compilador a seguir o padrão ao pé da letra.
Para evitar esse problema, remova o flag, mude o compilador ou declare int i no começo do bloco e use no for ou volte o código pra que o for seja a primeira operação.
int i = 0;

...

for( ; i < SIZE; i++ )
{
}

Answer: Declara antes do FOR, e no for chama a variável declarada
Answer: embora a pergunta não seja muito detalhada você pode fazer assim:
for(int i = 0; i < 10; i++)
  /* seu código aqui*/

ou dessa maneira se a variável é dentro do laço:
int i;
for(i = 0; i < 10; i++){
   int j = 2;
   printf("%d\n", i*j);
}

Como declarar variáveis no for em c?


Check whether a CSS class has a particular attribute or not

Question: It might be repeated question but still need of time is that I want a solution.
I am new to CSS and jQuery, so here is what I am looking for.
I have a css like this
.my-selector {
    background-size: cover;
    something-here1 : 2;
}

Now I want to check whether this my-selector has backgroud-size as one of the elements or not, if it has then I want to add new element in same selector.
This I want to do in JavaScript or using jQuery
Answer: btw: jQuery IS javascript.
if (!$(this).hasClass("my-selector")) {
   //rlemons post [link below] deals with this
}

look here for rlemon's anwer for the this part


sample, replace=F in R

Question: I was wondering how can I determine joint and marginal distribution in R. I believe I should use the function sample and for my exercise also replace=F and also for with iterations.
I was in hospital and I missed some classes in Probability, so I would be really grateful if someone could help me!
Comment: What do you mean by "determine joint and marginal distribution"? If you mean determine the theoretical distribution of some random varible, R won't be able to help you. You could obtain an empirical estimate of the distribution using R
Comment: Well English is not my native language so I jsut translated it and thought it was right, but I guess it was not correct. So I think it's the suggestion you made that I am supposed to do.
Answer: sample(x, size, replace=F) gives a rerandomization of x (i.e. a shuffling of x). If you combine a loop to rerandomize by variables in x, then you preserve the marginal distributions of each variable, but make them independent of one another.
Should you have said replace=T that would give a bootstrap sample of x, for size number of bootstrap samples of x.
Comment: Thanks. Well we have some number of balls of different colours and we should choose 2 of them. X is for the number of yellow balls and Y for green ones. We should determine the probability of choosing yellow and green balls (there's a table: X0-Y0, X1-Y0, X2-Y0, X0-Y1...). In the assignment there's said that we should use "sample", "replace=F", "for" and the number of iterations (e.g. 1000).
Comment: For bootstrap, wouldn't you be sampling WITH replacement? Without replacement you would get the original sample back since the size of each bootstrap sample is equal to the samplle size of x.
Comment: Good heavens, I totally got that backward! Thanks... caffeine. Edited now.
Comment: Yes Christy, you are choosing two balls without replacement, so replace =F.

sample, replace=F in R


Very Slow Virtual Disk I/O, LSI_SCSI Warning on P2V W2K3 64-Bit VMWare Guest

Question: I did a P2V conversion of a production server (W2K3R2 64-bit) a while back using the VMWare conversion tool.  I then installed the free VMWare Server 2.0 product on a Windows server, and am able to boot and interact with the VM guest.  My purpose for this project is to do testing of some patches to our ERP software.
Working with it a bit more, I'm seeing terrible disk I/O performance on the guest OS to the point of it being unusable.  While the virtual disks are on a big fat 1TB SATA RAID1 mirror, I'm getting times of like two hours to copy 250MB files to the virtual disk while the same copy in the host OS environment only takes a minute or two.  I don't expect blazing performance out of this test environment compared to the production system, so I'm okay with some degradation, but this is ridiculous.
I've tried the following:

Updated VMWare Server to 2.01.
Updated Virtual Hardware on VM to V7.
Updated LSI_SCSI driver to latest 64-bit version from LSI web site.
Reducing virtual CPUs to 1.

I still get this error when I Boot the VM:
<code>Event Type: Warning
Event Source:   Lsi_scsi
Event Category: None
Event ID:   26
Date:       8/6/2009
Time:       11:01:46 AM
User:       N/A
Computer:   VULCAN-TEST
Description:
The driver has detected that device \Device\RaidPort0 has old or out-of-date firmware. Reduced performance may result.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 0f 00 18 00 01 00 6a 00   ......j.
0008: 00 00 00 00 1a 00 04 80   .......
0010: 00 00 00 00 00 00 00 00   ........
0018: 00 00 00 00 00 00 00 00   ........
0020: 00 00 00 00 00 00 00 00   ........
0028: 00 00 00 00 00 00 00 00   ........
0030: 00 00 00 00 1a 00 04 80   .......
0038: 00 00 00 00 00 00 00 00   ........
</code>
My assumption is that the terrible I/O performance is related to this complaint from the LSI_SCSI driver.
The host system is running Windows Server 2008 x64 on an Intel E5405 quad-core dual-socket box with one CPU.
I see lots of discussion of I/O issues with VMWare and x64 Windows Server guests but I've tried everything I've seen suggested and so far no joy.
Comment: There's no chance you've got an on access AV scanner in the host slowing access to the virtual disk?
Have you checked the disk queue is short in both the host and the guest?
It might be worth checking the BIOS is bang up to date. a 64 bit guest means VT, and I think some early BIOS/microcode releases had gotchas with VT performance.
Answer: You may want to install ESXi on the host server instead of fighting VMware Server. I have installed VMware Server 2.0 for 4 different clients on Windows 2003 and Ubuntu 8.10 host OSes. The hardware was simliar to to yours in that we were using 7200RPM SATA drives on a PERC RAID controller. When we tried running SBS 2003 and Windows 2003 guests on it the IO performance was atrocious (in the sub 10MB/sec category). When we installed ESXi and migrated the guest OSes on it IO performance jumped to 90MB/sec and the results were much better.
I don't know how flexible you can be with the OS on that server but you may end up saving much more time by just moving to ESXi.
Answer: Have you installed the VMWare tools to the guest? What kind of disks make up the RAID 5 it lives on?
EDIT:
That shouldn't be too horrible although i would really have liked to see SAS drives with 10k spindle speed. But the 7200 drives should be good. I'm more familiar with ESX but the next things i would look at are:

Permon Trace on the host, looking at disk queueing
Change to the BusLogic SCSI Adapter (if Virtual Server has that option) and see if you see the same issues.
Check the host's event log for the same error - update the firmware on the controller if you see it there
As a last resort and if you can get away with it, throw ESXi on a box and copy the vm over ... see if you have the same issues.
Comment: I redid the P2V conversion.  Tried an IDE disk instead of SCSI.  Same badness.  This makes me sad.
Comment: try a fresh, small, install of the same OS into a new VM on the same host/disks - how does that behave?
Comment: VMWare Tools are up to date on the guest.  Actually I was incorrect - the VM disk files live on a RAID mirror (2x1TB SATA drives).  File copies on that volume outside of the VM are pretty fast.
Comment: Disk queues stay < 1 while the 250MB copy from the host to the guest takes an estimated 2 hours.

Virtual Server doesn't give an obvious option to change the SCSI virtual hardware.

The LSI_SCSI error only appears in the guest's event log since the host has an Areca ARC-1680 SATA/SAS storage controller.

Option #4 would be a lot of work as I don't have any 64-bit hardware I could just throw at this on a whim.
Comment: The disk may be a red herring. Can you try attaching a second VHD to the machine and do a copy from one to the other see if you still have bad performance?
Comment: Brian - what's the memory situation on that box? How much is installed in the host system? How much have you made available to the guest?


Show that if the numbers $\sqrt{a^2+b+c+1}$, $\sqrt{b^2+c+a+1}$ and $\sqrt{c^2+a+b+1 }$ are rational, then $a = b = c$

Question: Let $a$, $b$ and $c$ be nonzero natural numbers.  Show that if the numbers $\sqrt{a^2+b+c+1}$, $\sqrt{b^2+c+a+1}$ and $\sqrt{c^2+a+b+1  }$ are rational, then $a = b = c$.
My ideas
For those numebrs to be rational the numbers in the radical must be perfect squares.
I wrote them as some petfect squares.

$a^2+b+c+1=x^2$

$b^2+c+a+1=y^2$

$c^2+b+a+1=z^2$

Atfer summing them up
$a^2+b^2+c^2+2a+2b+2c+3=(a+1)^2+(b+1)^2+(c+1)^2$
Supposing that $a=b=c$ we arrive at the conclusion that we must show that ${a+1}^2=x^2$ and so on.
The form we wrote the numbers as is really close to what we must show. Hope one of can help me! Thank you so much!!
Answer: Hint: Assume, WLOG, $a\geq b\geq c$ and consider the implications of $a^2+b+c+1$ being a perfect square.
Comment: This answer does not appear to offer any additional insight other than what the OP has in their question.
Comment: @Doug Huh?  I don't sum over the three equations instead concentrating on only one.  Anyway, the other answers now cover the same approach.
Answer: It's clear that the numbers in the radical must be perfect squares,you could choose the largest number of $a,b,c$, WLOG, assume $a \ge b \ge c$.
Notice that $$a^2 \lt a^2+b+c+1 \le (a+1)^2$$
you only need to find when the equality holds.
Comment: I have a question. Why can we assume the inequality $a \ge b \ge c$?
Comment: I have a question. Why can we assume the inequality a≥b≥c
?