How does one handle a continuous variable regressed on a low count binary?

Question: Suppose I have a continuous variable $Y$ that takes values in $\mathbb{R}$. Assume that $X \in \{0,1\}$ with only around 25$\%$ of the $X$ variables being $1$. Running a least squares regression here results in an $R^2$ value of near $0$. What kind of analysis can we do to see if there is a relation between $Y$ and $X$ being equal to $1$? Does Poisson regression work?
Comment: You're basically telling us $Y$ falls into two groups and their means are essentially the same.  Changing the model is unlikely to change that result.  It will come down to *what kind* of relationship you are looking for.
Comment: There is no test of influence, only tests of association. You need to supply more information about how $Y$ varies with $X$ before anyone could offer an opinion concerning the applicability of linear regression.
Comment: I am looking for a test to see if $X$ can influence $Y$. Would linear regression be invalid here due to assumptions?
Answer: Poisson regression will not work.  Poisson regression is useful when the dependent variable is a count.  You have a continuous dependent variable.
Your regression seems appropriate and it tells you what linear regression tells you: The means of Y are not very different when X = 0 or X = 1.  You could also do a t-test, it is equivalent.
If you are looking for some other relationship (e.g. the variances of Y are different when X = 0 or X = 1) then there are tests for that, but if you are just trying to find some sort of difference (any sort will do!) then I suggest graphing the data.
If N is small (say, less than 100 or so) then you can do a strip plot (essentially a scatter plot) perhaps with some jittering.  If N is larger, a parallel box plot may be better.
However, when looking for "any sort of difference" I'd be very careful about any statistical tests as you don't have a well formed null hypothesis.
Comment: Re "Poisson regression will not work:"  William Gould of Stata has a remarkably different opinion.  Whether one agrees or not, it's worth being aware of: https://blog.stata.com/2011/08/22/use-poisson-rather-than-regress-tell-a-friend/.
Comment: That certainly is interesting.  Personally, I would take logs only for substantive reasons, not statistical ones. I wonder how robust (linear) regression does and whether Stata has that.  (I know nothing of Stata's capabilities).


What guitar chord is this?

Question: Tony Joe White played Rainy night in Georgia ( Germany '92 ). I'm wondering what chord is this? The chord is at 1min:30sec on video:
Comment: Looks like it could be a maj7 but frankly it's too hard to hear and too hard to see.  I cannot tell if the thumb is being used to fret a note in the bass or just his hand posture.  Try looking at a chord chart and comparing.
Answer: It's pretty clearly a Gmaj7 chord with fingering (from low E to high e):
X X 5 4 3 2
It's the IV chord of the key, coming from a Dmaj7 chord.
Comment: It looks very much like his pinky is on the A string. Probably a trick of foreshortening. The Gmaj7 jibes better with "fake book" chord lists based on the timing in the verse.
Comment: Yes it's Gmaj7. I looked at the chords here: https://tabs.ultimate-guitar.com/tab/tony_joe_white/rainy_night_in_georgia_chords_1502755, and they put G major but it didn't sound like the one Tony played. He was muting sixt & fifth strings with his thumb and that was clearly Gmaj7. Thanks guys
Answer: Difficult to make out exactly with the microphone in the way, but the position of the pinky especially makes it look like a loosely-barred "open-C" pattern starting at the second fret, which would make it a D.

But it looks like he is muting with the index finger, and possibly leaving the G string open, more like:

The internet says that the song is in D. There are multiple versions where some chord lists claim  plain D maj, others using different variations of D.
Comment: I don't see the index being flattened.  Though it's hard to tell, which is why I didn't answer.

What guitar chord is this?


Is the resulting function of plotting two variables imply that all other factors are being held constant?

Question: Say, for instance, I plot two relationships (separately):
(a) Share of income spent on food vs. household income; and
(b) No. of houses in a unit vs. population of a city
Then, for both (a) and (b), I separately run a run a regression of the y-axis variable on the x-axis variable and plot these functions on each respective graph.
Can one say that the resulting regression functions in (a) and (b) holds all other factors constant; even though the function I am plotting isn't controlling for any variable?
For example, in the case of (a) can one say that the graph shows the relationship between share of income spent on food and household income assuming and/or holding constant other factors that could impact the relationship between these two variables, such as demographic compositions of households, etc.?
Answer: No, plotting the two variables against each other does not constitute controlling for all other variables.  Doing this with two different sets of variables also has no effect on that fact.
What you are doing is ignoring all other variables.  To better understand the distinction, it may help to read my answer to: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?  That post is more focused on regression models, whereas your case is about plotting, but the point is ultimately the same.
If you want a plot of a relationship between two variables that holds all else constant, you will need a properly specified model.  Ideally, you would have experimental-ish data where you could guarantee that all other variables are uncorrelated, but if you could knew what all the other relevant variables in the universe were, you could also control for them.  Either way, at that point you could plot the relationship specified in the model by stipulating the mean for all other variables (this would just affect the height of the line).  The other possibility would be to use 'least squares means', better called 'predicted marginal means' to assess a theorized population relationship when all other variables are balanced.  It may help to read How to visualize a fitted multiple regression model? and/or What are LS means useful for?


Please explain this proof of AM-GM inequality

Question: I don't undeestand this proof
How does rhis proof work? What does it mean when it says "Since every time we perform this
operation we create a number equal to A"? Is it taking averages?
Answer: By "create a number equal to $A$, the author means that each time you apply the process, the new sequence that you get ($a_1',a_2',\dots$) has one more term equal to $A$ than the previous sequence ($a_1,a_2,\dots$). Since there are only $n$ terms in the sequence, the process can be applied at most $n$ times before all the terms are equal to $A$.
Comment: Thanks!! But won't there be n/2 terms equal to A? Since we take pairs (a1, a2), (a3,a4).......in each iteration. Could you please tell me how it'll be proven?
Comment: @EHN Nothing prevents us from taking $(a_2,a_3)$ is one of these is $A$.  As long as not all the terms afre equal to $A$, at least one will be $A$, and so the process can again be applied.
Comment: Lets say there are even number of terms, in the last pair we have one term A & another A+h-k, so in the end won't we have (n-1) A terms?
Comment: @EHN Since the average remains $A$ at all stages, once you have $n-1$ terms equal to $A$, the one remaining term will necessarily also be equal to  $A$.
Comment: Yes i get it now, thanks!!


How to prove $\cot ^2x+\sec ^2x=\tan ^2x+\csc ^2x$?

Question: How can I prove the following equation?
\begin{eqnarray}
\cot ^2x+\sec ^2x &=& \tan ^2x+\csc ^2x\\
{{1}\over{\tan^2x}}+{{1}\over{\cos^2x}} &=& {{\sin^2x}\over{\cos^2x}}+{{1}\over{\sin^2x}}\\
{{\sin^2x+\cos^4x}\over{\sin^2x\cos^2x}} &=& {{\sin^4x+\cos^2x}\over{\sin^2x\ \cos^2x}}\\
\end{eqnarray}
Then, what can I do...?
Thank you for your attention.
Answer: HINT:
We know, $$\csc^2x-\cot^2x=1=\sec^2x-\tan^2x$$
Answer: Well if nothing else comes to mind try by hand$$\cot^2 x+\sec^2x=\frac{\cos^2x}{\sin^2x}+\frac 1{\cos^2x}=\frac{\cos^4x+\sin^2x}{\cos^2x\sin^2x}$$
and $$\tan^2x+\csc^2x=\frac {\sin^2x}{\cos^2x}+\frac 1{\sin^2x}=\frac{\sin^4x+\cos^2x}{\cos^2x\sin^2x}$$
And these are equal if $$\cos^4x+\sin^2x=\sin^4x+\cos^2x$$
Now there are various ways to see it. Of course it is easier knowing the standard identities and using them, but they all pretty much boil down to $\sin^2x+\cos^2x=1$, which is in turn another way of writing Pythagoras, and which will definitely help here.
Note also that I've ignored any issue of infinite values or discontinuities. I have been careful not to divide by zero, which is quite easy to do by accident when working with trigonometric sums.
Answer: Hint: $1+\cot ^2x = \dfrac{\sin^2x+\cos^2x}{\sin^2x} = \csc ^2x$, $1+\tan ^2x = \dfrac{\cos^2x + \sin^2x}{\cos^2x} = \sec ^2x$.
Answer: Do you know, $\cot^2(x) + 1 = \csc^2(x)$ and $\tan^2(x) + 1 = \sec^2(x)$
So, $\cot ^2x+\sec ^2x=\csc^2(x)-1+\tan^2(x) + 1=\tan^2(x)+\csc^2(x)$
Thus, we end up with : $\cot ^2x+\sec ^2x=\tan^2(x)+\csc^2(x)$...


Using Scala and StringTemplate, how do I loop through a Map

Question: I have my environment setup nicely using Scala, StringTemplate within the Google AppEngine. I am having trouble looping through a Map and getting it to display in the template. When I assign a simple List of just Strings to the template it works using:
In Scala Servlet:
  var photos = List[String]()

  //... get photo url and title ...

  photos = photo_url :: photos
  template.setAttribute("photos", photos: _*)

In Template:
  $photos: { photo|

  }$

The above works. However, any attempt of creating a Map using url and title and assigning to the template gives me an error. Here is my attempt, which does not work:
In Scala Servlet:
  var photos = List[Map[String,String]]()

  //... get photo url and title ...

  photos = Map("url" -> url, "title" -> title) :: photos
  template.setAttribute("photos", photos: _*)

In Template:
  $photos: { photo|

  }$

This gives me the following error
Class scala.collection.immutable.Map$Map2 has no such attribute: title in template context

Thoughts / Ideas ?
Comment: Deleted my suggestion--I'm just remarking here that my simple idea of creating `case class TitleURL(url: String, title: String)` and using that instead of the map does not work. According to Marcus, it generates an error like "Class FlickrTestServlet$TitleURL$2 has no such attribute: title in template context [templates/flickr_template if(photos)_subtemplate anonymous]".
Answer: Following up on Rex's suggestion, I was able to make it work using a case class with a @BeanProperty annotation for the fields:

case class MyPhoto(@BeanProperty val url: String, @BeanProperty val title: String)

def generateMyPhotos() : String = {
  val tp = new StringTemplate("$photos: { photo|

}$")
  val photos = List(MyPhoto("http://myphoto.com", "my photo"))
  tp.setAttribute("photos", photos: _*)
  tp.toString
}

This worked for me (using the scalasti library for StringTemplate, as you probably also already did).
Comment: Awesome, that worked! Thank you!!  Yep, I am using scalasti wrapper for StringTemplate. The only thing I had to do was add an import for Bean Property  import scala.reflect._
Answer: There's a simple alternative, without using additional packages and annotations. Register a scala object adaptor that knows how to retrieve property values from scala objects and scala collections.
This is now included in the StringTemplate FAQ at:
http://www.antlr.org/wiki/display/ST4/Altering+property+lookup+for+Scala


Matching controls in staggered difference-in-differences

Question: I am planning a staggered difference-in-differences analysis examining the impact of a change in health policy at the county level. The policy was implemented at different times in different counties, and in many counties not at all. I have data for all counties in the country, with only a small fraction having ever implemented the policy.
My question concerns choosing the appropriate control counties to include in my analysis. If I were performing a simple diff-in-diff analysis with a treatment enacted in one county only, I would choose the most closely "matched" county for the control and include only those two. However, given that I have a large number of staggered treatment counties which will all be included in the model and which will contribute variable amounts of data, is there still the same importance to find a 1-to-1 matching control for each? Does it make more sense to use the properties I would match on as covariates in the model and then include more of my available control data, without worrying about matching?
Answer: The selection of control groups in a staggered difference-in-differences (DiD) setup is a little more complicated than in a typical DiD technique, particularly when interventions are implemented at different times across units. Building a conventional one-to-one matched control for every treated county may not be practical or the best effective use of your data, given your many treatment counties and different implementation times. Here are a few considerations for selecting control counties in your staggered DiD analysis:
Covariates vs. Matching: It might be more realistic to incorporate all possible control counties and use covariates in the regression model to account for county-to-county variances rather than attempting to match every treatment county to a control county. By using this method, you may take use of the entire set of data and lower the possibility of bias resulting from faulty matching.
Generalised DiD Framework: You can use all available data, without rigorous matching, from treated and untreated counties, in a generalised DiD framework. Time fixed effects would be included in this model to account for common shocks that affect all counties at each time point, and county fixed effects would be included to account for time-invariant variations between counties.

Covariates vs. Matching: It might be more realistic to incorporate all possible control counties and use covariates in the regression model to account for county-to-county variances rather than attempting to match every treatment county to a control county. By using this method, you may take use of the entire set of data and lower the possibility of bias resulting from faulty matching.

Generalised DiD Framework: You can use all available data, without rigorous matching, from treated and untreated counties, in a generalised DiD framework. Time fixed effects would be included in this model to account for common shocks that affect all counties at each time point, and county fixed effects would be included to account for time-invariant variations between counties.

Synthetic Control technique: The synthetic control technique builds a weighted combination of control units that most closely resembles the properties of each treated unit before the treatment, and you can use it to do a robustness check. This approach can be modified for phased implementation, however it can be especially helpful in cases where there are few treated units.

The fundamental presumption of DiD analysis is that the treated and control groups would have followed parallel courses in the absence of treatment. Validating this assumption for your staggered design is crucial. One way to do this could be to look at pre-treatment patterns in the outcome variable.

Using All Control Data: If you account for the variations between treated and control counties using covariates or fixed effects, adding more control counties without tight matching can improve the accuracy of your results. When you are unsure about the comparability of any particular control unit, this method may be helpful.

Variable Treatment Timing: Because the timing of the treatments varies, you must make sure that the control counties are positioned in relation to each treatment group at the right moment. Group-time fixed effects, which enable the treatment impact to change over time and between groups, can be used to achieve this.

Approaches to Econometrics for Staggered DiD: The intricacies of staggered DiD designs are tackled by the latest advancements in econometrics. The Callaway and Sant'Anna (2020) approach and Sun and Abraham's (2021) estimator are two techniques that offer approaches for estimating treatment effects that change over time and between groups.

References:
Callaway, B. and Sant'Anna, P., "Difference-in-differences with multiple time periods". Journal of Econometrics, Forthcoming, December 2020.
https://www.sciencedirect.com/science/article/abs/pii/S0304407620303948
Sun, L., & Abraham, S. (2021). Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Journal of Econometrics, 225(2), 175-199.
https://www.sciencedirect.com/science/article/pii/S030440762030378X


Monte carlo simulation to forecast growth of a loan portfolio

Question: I have to forecast the future gold loan portfolio growth of a financial firm. I have past 36 month growth data.
I am planning to use Monte Carlo simulation to forecast, but growth is a deterministic process not a random process. How can I apply Monte Carlo for this situation?
Answer: If your model is deterministic and no randomness or uncertainty, there's no point with simulation because all your paths will be identical.
Comment: Thanks a lot friend. I had better study detail about montecarlo.


Read sheet music trouble

Question: I'm new. I got some trouble when read this song, I don't know Bm(b5) meaning. Also some word "Bb" "Dm",... I know it suggestion note make from chord in treble & bass clef. But sometime it lack a note from chord or note in one line can't make a chord as suggestion. Help me, please.
Comment: "Loss of Me" is a popular mistranslation of this piece's title, but the generally accepted translation of the title nowadays is "Rose of May". (The katakana for the last word of the title matches "May" much better than "me".) It's from Final Fantasy IX. (This video game also has some other doubtful translations for its music titles, such as "Sword of Doubt"/"The Wavering Blade" and "Something to Protect"/"Protecting My Devotion".)
Comment: All YOU, as a keyboard player, have to do is play the printed notes.  The chord symbols are a description of the harmonies they contain.  They might be of use to e.g. a guitarist accompanying the piece.  (Be aware that sometimes chord symbols provide a functional harmonic analysis, sometimes they're just utilitarian, indicating what notes to play.  Sometimes they're not a COMPLETE description of what's going on.  Don't waste time analyzing something like the Bm(b5) too closely.  Just note that B, D and F are required.
Comment: @LaurencePayne - I, as a keyboard player, will find those chord symbols (if accurate..!) very useful to change voicing or extemporise later on in songs, instead of playing the same old all through. Probably easier than trying to translate what the harmonies are (supposed to be) from the dots.
Comment: If you extemporise, why keep to the original chords? (Also - why was this question closed?  We demonstrably have more to offer.  And it ISN'T the same question as the one cited.  That was about a piece that WASN'T fully notated.
Answer:
I don't know Bm(b5) meaning

That would be a B minor with  a flat fifth, that is add the seventh (A) and lower the fifth (from F# to F). So the notes become B, D, F, A.

I know it suggestion note make from chord in treble & bass clef. But sometime it lack a note from chord or note in one line can't make a chord as suggestion

Those are not for the person playing the parts on the staff, those are for someone accompanying the player (for example a guitar). Or if you want to improvise, so you know what chords you're supposed to improvise on.
Comment: Strictly speaking, Bmb5 is rarely played as such, and you're right, there's usually a '7' included - Bm7b5. However, that 7 is the melody note here. It also gets called Dm6, and would be better as that in the key of that song, maybe?
Comment: @Tim given the other Dm chords around it, yeah, that would probably make sense. But I guess they used Bm(b5) because of the chord voicing?

Read sheet music trouble


Upload an image and save it as base64

Question: First in nodejs, I have succeeded to upload an image (using multer) through postman. Now I am trying to save it not as a png/jpg file, but as a base64 encrypted text file on disk. Any suggestions ?

const express=require('express');
const mongoose= require('mongoose');
const bodyparser=require('body-parser');
const cookieParser=require('cookie-parser');
const db=require('./config/config').get(process.env.NODE_ENV);
const multer = require("multer");
const path = require('path');
var fs = require('fs');

const imageStorage = multer.diskStorage({
    // Destination to store image
    destination: 'images',
      filename: (req, file, cb) => {
          cb(null, file.fieldname + '_' + Date.now()
             + path.extname(file.originalname))
            // file.fieldname is name of the field (image)
            // path.extname get the uploaded file extension
    }
});

const imageUpload = multer({
    storage: imageStorage,
    limits: {
      fileSize: 1000000 // 1000000 Bytes = 1 MB
    },
    fileFilter(req, file, cb) {
      if (!file.originalname.match(/\.(png|jpg)$/)) {
         // upload only png and jpg format
         return cb(new Error('Please upload a Image'))
       }
     cb(undefined, true)
  }
})

app.post('/api/uploadPicture', imageUpload.single('picture'), (req, res) => {
    const contents = fs.readFileSync(req.file.path, {encoding: 'base64'});
    let buff = Buffer.from(contents).toString('base64');

    console.log(buff);
    res.send(req.file);
}, (error, req, res, next) => {
    res.status(400).send({ error: error.message })
})

Thanks ahead !
Answer: I've some ready snippets which worked well for me before. But I've commented there the part where you save a file with exact image extension. Instead I've saved the file with "txt" extension as you want (in userController).
routes/main.js
router.post('/image/base64', multerUpload.single('image'), userController.uploadBase64Image);

utils/multer.js
const multer = require('multer');
const path = require('path');

module.exports = multer({
    storage: multer.diskStorage({
        destination: function (req, file, cb) {
            const filePath = path.join(__dirname, './../uploads');
            cb(null, filePath);
        },
        filename: function (req, file, cb) {
            const extension = file.mimetype.split('/')[1];
            const fileName = (new Date().getTime() / 1000 | 0) + '.' + extension;
            cb(null, fileName);
        }
    }),
    limits: {
        fileSize: 1024 * 1024 * 2 // MB
    },
    fileFilter: (req, file, cb) => {
        let valid = (file.mimetype === 'image/jpeg' || file.mimetype === 'image/jpg' || file.mimetype === 'image/png');
        cb(null, valid);
    },
});

controllers/user.js
const fs = require('fs');
const path = require('path');
const port = process.env.APP_PORT; // 3000
const appUrl = process.env.APP_URL; // http://127.0.0.1

const uploadBase64Image = async (req, res, next) => {
    try {
        const encoded = req.body.image;
        const base64ToArray = encoded.split(";base64,");
        // const prefix = base64ToArray[0];
        // const extension = prefix.replace(/^data:image\//, '');
        const extension = 'txt';

        // if (extension === 'jpeg' || extension === 'jpg' || extension === 'png')
        // {
            const imageData = base64ToArray[1];
            const fileName = (new Date().getTime() / 1000|0) + '.' + extension;
            const imagePath = path.join(__dirname, './../uploads/') + fileName;
            fs.writeFileSync(imagePath, imageData,  { encoding: 'base64' });

            return res.status(201).json({
                error: false,
                message: "Base64 Image was successfully uploaded.",
                url: `${appUrl}:${port}/images/${fileName}`
            });
        // }
        // else {
        //     return res.status(403).json({
        //         error: true,
        //         message: "Base64 data not valid!",
        //     });
        // }
    }
    catch (e) {
        return res.status(403).json({
            error: true,
            message: e.message,
        });
    }
}

Comment: @Hunter the provided code was from the backend of a mobile app project, so the client side could be anything, as i've tested it with postman as for an API endpoint
Comment: "Cannot read properties of undefined (reading 'split') "
Comment: I've just tested, it's [work for me](https://dropmefiles.com/RcLib). please test it like in image, also you can get base64 from [here](https://www.base64-image.de/). Past the base64 text in form-data as a "image" field
Comment: May I ask How look like code in client side ? @boolfalse

Upload an image and save it as base64


Computation of a series.

Question: NOTATIONS.
Let $n\in\mathbb{N}$. We define the sets $\mathfrak{M}_{0}:=\emptyset$ and
\begin{align}
\mathfrak{M}_{n}&:=\left\{m=\left(m_{1},m_{2},\ldots,m_{n}\right)\in\mathbb{N}^{n}\mid1m_{1}+2m_{2}+\ldots+nm_{n}=n\right\}&\forall n\geq1
\end{align}
and we use the notations:
\begin{align}
m!&:=m_{1}!m_{2}!\ldots m_{n}!,&|m|&:=m_{1}+m_{2}+\ldots+m_{n}.
\end{align}
QUESTION.
I want to evaluate or just bound with respect to $n$ the series
\begin{align}
S_{n}&:=\sum_{m\in\mathfrak{M}_{n}}\frac{\left(n+\left|m\right|\right)!}{m!}\ \prod_{k=1}^{n}\left(k+1\right)^{-m_{k}}.
\end{align}
My hope is that $S_{n}\leq n!n^{\alpha}$ with $\alpha$ independant of $n$.
BACKGROUND.
In order to build an analytic extension from a given real-analytic function, I had to use the Faà di Bruno's formula for a composition (see for example https://en.wikipedia.org/wiki/Faà_di_Bruno%27s_formula). After some elementary computations, my problem boils down to show the convergence of
\begin{align}
\sum_{n=0}^{+\infty}\frac{x^{n+1}}{(n+1)!}\sum_{m\in\mathfrak{M}_{n}}\frac{\left(n+\left|m\right|\right)!}{m!}\ \prod_{k=1}^{n}\left(k+1\right)^{-m_{k}}
\end{align}
where $x\in\mathbb{C}$ is such that the complex modulus $|x|$ can be taken as small as desired (in particular, we can choose $|x|<\mathrm{e}^{-1}$ to kill any $n^{\alpha}$ term from the bound on $S_{n}$).
SOME WORK.
It is clear that we have to to understand the sets $\mathfrak{M}_{n}$ in order to go on (whence the tag "combinatorics"). So I tried to see what were these sets:

for $n=2$ :
\begin{array}{cc}
2&0\\
0&1
\end{array}
for $n=3$ :
\begin{array}{ccc}
3&0&0\\
1&1&0\\
0&0&1
\end{array}
for $n=4$ :
\begin{array}{cccc}
4&0&0&0\\
2&1&0&0\\
1&0&1&0\\
0&2&0&0\\
0&0&0&1\\
\end{array}
for $n=5$ :
\begin{array}{ccccc}
5&0&0&0&0\\
3&1&0&0&0\\
2&0&1&0&0\\
1&0&0&1&0\\
1&2&0&0&0\\
0&0&0&0&1\\
0&1&1&0&0\\
\end{array}

Above, each line corresponds to an multiindex $m$, and the $k$-th column is the coefficient $m_{k}$. We see for example that the cardinal of $\mathfrak{M}_{n}$ becomes strictly greater than $n$ if $n\geq5$. Also, because I wanted to reorder the set of summation in $S_{n}$ into a the set of all multiindices $m$ such that $|m|=j$ for $1\leq j\leq n$, I tried to count given $j$ the number of $m$ such that $|m|=j$; when $n=10$, I counted $8$ multiindices $m$ with length $|m|=4$, so that this number can be greater than $n/2$. Another remark is that the number of multiindices $m$ such that $|m|=j$ becomes larger if $j$ is "about" $n/2$ - don't ask me what "about" means here, I just tried some example and saw this phenomenon.
Comment: There are typos in your configuration, should be
$$n = 2 \to
\begin{matrix}*2 & 0*\\0 & 1\end{matrix},
\quad
n = 4 \to
\begin{matrix}
4 & 0 & 0 & 0\\
2 & 1 & 0 & 0\\
1 & 0 & 1 & 0\\
*0 & 2 & 0 & 0*\\
*0 & 0 & 0 & 1*
\end{matrix}$$ BTW, the first few numbers of your sequences are $1,1,5,41,469,6889$ and it matches the one on [OEIS A032188](https://oeis.org/A032188).
Comment: @achillehui Thank you for the typos, I have corrected it. As for the sequence you are talking about, it seems to give some informations about a generating function, but I am not familiar with this tool. Is it possible to deduce a bound for my series from your link (I do not see how)?
Comment: I don't know what you can get from the link but I'm working on a closed form expression of your series. If I didn't make any mistake, the radius of convergence of $\sum_{n=0}^\infty S_n \frac{x^{n+1}}{(n+1)!}$ is $1 - \log(2)$, this implies $\frac{S_n}{(n+1)!} \sim o( r^n )$ for any $r > \frac{1}{1 - \log(2)}$.
Comment: @achillehui If you could get a closed form, it would be great! My goal is to prove that the series in $x$ converges for some $x\in\mathbb{C}$.
Comment: @Nicolas If both MO accounts [appearing on this question](http://mathoverflow.net/questions/257079/computation-of-a-series) belong to you, you can try to [merge them](http://mathoverflow.net/help/merging-accounts). Related post on this site's meta: [Announcement: New User Merge Policy/Tool](http://meta.math.stackexchange.com/q/8403).
Comment: BTW you should probably mention somewhere also the MO post [Computation of a series](http://mathoverflow.net/q/257079). So that other users interesting in this topic see also the link to the other post. Again, I will add [a link to meta](http://meta.math.stackexchange.com/questions/5085/moderator-supported-official-guidelines-for-legitimate-crossposting/5088#5088).
Answer: First, we will transform $S_n$ to a form easier to manipulate.
Let $C \subset \mathbb{C}$ be a circle of radius $r \ll 1$ centered at $0$.
For any $n \in \mathbb{N}$, $m \in \mathbb{N}^n$, we can single out those $m \in
\mathfrak{M}_n$ with help of contour integrals of the form:
$$\delta_n(m) \stackrel{def}{=} \frac{1}{2\pi i} \oint_{C} s^{\sum_{k=1}^n k m_k} \frac{ds}{s^{n+1}} = \begin{cases}1, & m \in \mathfrak{M}_n\\ 0, & \text{ otherwise }\end{cases}$$
Together with following integral representation of factorial:
$$n! = \Gamma(n+1) = \int_0^\infty t^n e^{-t}dt$$
We have
$$\begin{align}
S_n &= \sum_{m \in \mathbb{N}^n} \delta_n(m) \int_0^\infty \prod_{k=1}^n \frac{1}{m_k!}
\left(\frac{t}{k+1}\right)^{m_k} t^n e^{-t} dt\\
&= \frac{1}{2\pi i}\sum_{m \in \mathbb{N}^n} \oint_C \left[\int_0^\infty \prod_{k=1}^n
\frac{1}{m_k!}
\left(\frac{ts^k}{k+1}\right)^{m_k} \left(\frac{t}{s}\right)^ne^{-t} dt \right] \frac{ds}{s}\\
&= \frac{1}{2\pi i}\oint_C \left[\int_0^\infty \prod_{k=1}^n
\left(\sum_{m_k=0}^\infty \frac{1}{m_k!}
\left(\frac{ts^k}{k+1}\right)^{m_k}\right) \left(\frac{t}{s}\right)^ne^{-t} dt \right] \frac{ds}{s}\\
&= \frac{1}{2\pi i}\oint_C \left[\int_0^\infty \prod_{k=1}^n
\exp\left(\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^ne^{-t} dt \right] \frac{ds}{s}\\
&= \frac{1}{2\pi i}\oint_C \left[\int_0^\infty
\exp\left(\sum_{k=1}^n\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^ne^{-t} dt \right] \frac{ds}{s}\\
&\stackrel{\color{blue}{[1]}}{=} \frac{1}{2\pi i}\oint_C \left[\int_0^\infty
\exp\left(\sum_{k=1}^\infty\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^ne^{-t} dt \right] \frac{ds}{s}\\
&= \frac{1}{2\pi i}\oint_C \left[\int_0^\infty
\exp\left[-t\left(\frac{\log(1 - s)}{s} + 2\right)\right]\left(\frac{t}{s}\right)^n dt \right] \frac{ds}{s}\tag{*1}\\
\end{align}
$$
Next, let

$S(x) \stackrel{def}{=} \sum_{n=0}^\infty S_n \frac{x^n}{n!}$ be the EGF (exponential generating function) for $S_n$.
$\Delta(x) = \sum_{n=0}^\infty S_n \frac{x^{n+1}}{(n+1)!}$ be the series
we want to study its convergence.

They are related by the relation $\Delta(x) = \int_0^x S(t) dt$.
For any $x$ with $|x| \ll r$, $(*1)$ implies
$$\begin{align}
S(x)
&= \frac{1}{2\pi i}\oint_C \left[\int_0^\infty
\exp\left[-t\left(\frac{\log(1 - s)}{s} + 2 - \frac{x}{s}\right)\right] dt \right] \frac{ds}{s}\\
&= \frac{1}{2\pi i}
\oint_C \frac{ds}{\log(1-s) + 2s - x}
\end{align}
$$
Change variable to $y = -\log(1-s) \iff s = 1 - e^{-y}$. When $r$ is small, the image of $C$ in $y$-space is close to circle $C$. We can deform the contour back to $C$ without changing the integral. This leads to
$$S(x) = \frac{1}{2\pi i}\oint_C \frac{dy}{P(x,y)}
\quad\text{ where }\quad
P(x,y) = (2-x-y)e^y - 2
$$
Under the condition $|x| < |y| = r \ll 1$, we have
$$P(x,y) \approx (2 - x - y) (1 + y + O(r^2)) - 2 \approx y - x + O(r^2)$$
This means for fixed $x$ and as a function in $y$, $P(x,y)$ has only
one root inside $C$. Furthermore, the root in $y$ is close to $x$.
Let $\eta$ be that root, we have
$$\begin{align}
P(x,\eta) = 0 &\iff (2-x-\eta)e^\eta - 2 = 0
\iff (\eta + x - 2)e^{\eta + x - 2} = -2e^{x-2}\\
& \implies 2 - x - \eta = -W(-2e^{x-2})
\end{align}
$$
where $W(z)$ is a branch of the Lambert-W function. In terms of $\eta$, we have
$$\begin{align}
S(x) &= \text{Res}_{y=\eta}\left(\frac{1}{P(x,y)}\right)
= \left.\frac{1}{\frac{\partial}{\partial y}P(x,y)}\right|_{y=\eta}
= \frac{1}{(1 - x - \eta)e^\eta}\\
&= \frac{2-x-\eta}{2(1-x-\eta)} = \frac{W(-2e^{x-2})}{2(1+W(-2e^{x-2}))}
\end{align}
$$
Since $S(0) = 1$, we need to choose a branch of Lambert W function with $W(-2e^{-2}) = -2$. The correct branch is the "lower branch" described
in above wiki link. It is usually denoted as $W_{-1}(\cdot)$. In terms of
it, we find
$$S(x) = \frac{W_{-1}(-2e^{x-2})}{2(1+W_{-1}(-2e^{x-2}))}$$
Notice the branches of Lambert W function satisfies ODE
$$z\frac{d}{dz}W(z) = \frac{W(z)}{1+W(z)}\tag{*2}$$
We can integrate $(*2)$ and deduce a closed form expression for $\Delta(x)$:
$$\Delta(x) = \frac12 \int_0^x \left[ z\frac{dW_{-1}(z)}{dz} \right]_{z=-2e^{t-2}} dt = 1 + \frac12 W_{-1}(-2e^{x-2})\tag{*3}$$
$W_{-1}(z)$ has two branch cuts, one terminated at $z = -\frac1e$, the other at $z = 0$. The closest singularity of $\Delta(x)$ to origin is located at $x = 1 - \log(2)$. As a result, $r_0$, the radius of convergence of the power series expansion of $\Delta(x) = \sum\limits_{n=0}^\infty S_n\frac{x^{n+1}}{(n+1)!}$, $r_0$ equals to $1 - \log(2)$. A corollary of this is$\color{blue}{{}^{[2]}}$
$$\frac{S_n}{(n+1)!} \sim o(\rho^n)\quad\text{ for any }\; \rho > \frac{1}{1-\log(2)}
\approx 3.258891353270929$$
As a double check, we evaluate the power series expansion of $\Delta(x)$
using following command Series[1+1/2*LambertW[-1,-2*Exp[x-2]],{x,0,8}]
on WA (wolfram alpha). WA returns
$$\begin{align}
\Delta(x) = & x+\frac{{x}^{2}}{2}+\frac{5\,{x}^{3}}{6}+\frac{41\,{x}^{4}}{24}+\frac{469\,{x}^{5}}{120}+\frac{6889\,{x}^{6}}{720}\\
& +\frac{24721\,{x}^{7}}{1008}+\frac{2620169\,{x}^{8}}{40320}+\frac{64074901\,{x}^{9}}{362880} + \cdots
\end{align}$$
Translate back to $S_n$, this is equivalent to
$$( S_0,S_1,\ldots ) = (1, 1, 5, 41, 469, 6889, 123605, 2620169, 64074901,\ldots )$$
For $n \le 5$, I have checked by hand this is indeed the correct value.
An OEIS search return the sequence OEIS A032188.
Up to $n = 18$, I've verified the $S_n$ extract from expansion of $(*3)$ matches the numbers on OEIS. Look at references there and see whether there is anything useful for your purposes.
Notes

$\color{blue}{[1]}$ - As a function of $s$,
$$\exp\left(\sum_{k=1}^n\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^n\frac{e^{-t}}{s} = \frac{1}{s^{n+1}}A(s)\quad\text{ and }\quad
\exp\left(\sum_{k=n+1}^\infty\frac{ts^k}{k+1}\right) = 1 + s^{n+1}B(s)
$$
where $A(s), B(s)$ are analytic over the disc bounded by $C$. This implies
$$\exp\left(\sum_{k=1}^\infty\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^n\frac{e^{-t}}{s} =
\exp\left(\sum_{k=1}^n\frac{ts^k}{k+1}\right) \left(\frac{t}{s}\right)^n\frac{e^{-t}}{s} + A(s)B(s)$$
Changing the upper bound in the sum within the exponent from $n$ to $\infty$
modifies the integrand by a function analytic over the disc bounded by $C$.
The value of the contour integral over $C$ remains the same.
$\color{blue}{[2]}$ - A more detailed analysis suggests for large $n$, $S_n$ has following approximation:
$$S_n \approx \frac{(2n)!}{\sqrt{8r_0}n!(4r_0)^n}\left( 1  - \frac{r_0}{6(2n-1)} + \cdots \right)\quad\text{ where }\quad r_0 = 1 - \log(2)$$
For $n$ as small as $4$, this formula gives a relative error below $10^{-4}$
(checked against numbers from OEIS). The leading behavior of coefficients of $\Delta(x)$ should be:
$$\frac{S_n}{(n+1)!} \sim O\left(\frac{r_0^{-n}}{\sqrt{8\pi r_0} n^{3/2}}\right)$$
Comment: Absolutely amazing! This is clear and well done!! I just have one question: in the long equation $(*1)$, why can we modify the series of $ts^k/(k+1)$ in the exponential in a sum on $0\leq n\leq+\infty$?
Comment: @Nicolas When you expand
$\exp\left(\sum\limits_{k=n+1}^\infty \frac{ts^k}{k+1}\right)$
 as a power series in $s$, you get something of the form $1 + \sum\limits_{k=n+1}^\infty \alpha_k s^k$. Aside from the constant term, all remaining terms contain a factor $s^{n+1}$. This is enough to cancel the singular factor $\frac{1}{s^{n+1}}$ in the rest of the integrand. Replacing the sum from $0\to n$ to $0\to\infty$ changes the integrand by a function analytic inside $C$. The value of contour integral over $C$ remains the same.
Comment: Of course! I forgot the $s^{-(n+1)}$ term. Ok, now it is perfect to me, I do thank you for your efforts and for the time you've spent to write this answer.
Comment: @achillehui: Clear, concise and instructive! Very nice! (+1)
Comment: @achillehui Almost one year later, I have another question about the long equation (*1): in the third line, you enter the sum of multi indices and exchange it with the product in the integral. I understand the exchange with the product (this is the Cauchy product formula) but I do not see why you can enter the sum in the integral. Dominated convergence theorem seems to be the argument to use, but one needs to show that the series of the integral of the modulus converges, which is quite similar to show the convergence of $S_n$! Have you got any argument to propose? Thanks in advance.
Comment: @Nicolas  Let $|s| = r$, if you repeat the algebra of $(*1)$ start at $3^{rd}$ line by replace everything with their absolute values, you can bound the sum of these absolute values  by an expression of the form $$
\frac{1}{2\pi}\oint_C \left[\int_0^\infty
\exp\left[-t\left(\frac{\log(1 - r)}{r} + 2\right)\right]\left(\frac{t}{r}\right)^n dt \right] \frac{|ds|}{r}$$ For small $r$ (more precisely, $r < 0.7968$ is enough), this integral converges absolutely, so one can exchange the summation of integral in the $3^{rd}$ line without any problem.
Comment: @achillehui Hm and this time you pass through the integral by Beppo-Levi theorem because all is positive. I was so dubious because this is part of an important theorem in my thesis (I will mention your contribution in the acknowledgments by the way). I thank you again for your help!
Answer: Here is a solution of a related problem, followed by a recommendation for the original problem. It would be much simpler if your sum did not have the $n$ in $(n+|m|)!\,$. In that case, we could look at the related sum
$$t_n=\sum_{m\in {\mathfrak{M} }_n}\frac{|m|!}{m!}\prod_{k=1}^n(k+1)^{-m_k}.$$
The sum for the $t$'s comes from a product of exponential generating functions. Because of the factor of $(k+1)^{-m_k}$ in $t_n$ and the term $k\,m_k$ in ${\mathfrak{M} }_n$, we must look at the series
$$1+\frac{\left(\frac{x^k}{k+1}\right)^1}{1!} +\frac{\left(\frac{x^k}{k+1}\right)^2}{2!} +\frac{\left(\frac{x^k}{k+1}\right)^3}{3!} +\cdots=\exp\left(\frac{x^k}{k+1}\right).$$
From multiplying these exponential generating functions, we get
$$t_n=\left[\frac{x^n}{n!}\right]\prod_{k\ge1}\exp\left(\frac{x^k}{k+1}\right).$$
This product turns out to have a nice closed form:
\begin{eqnarray*}
% \nonumber to remove numbering (before each equation)
  \prod_{k\ge1}\exp\left(\frac{x^k}{k+1}\right) &=& \exp\left(\sum_{k\ge1}\frac{x^k}{k+1}\right) \\
    &=& \exp\left(\frac1x\biggl(\log\bigl(\frac1{1-x}\bigr)-x\bigr)\right) \\
    &=& (1-x)^{-x}/\mathrm{e} .
\end{eqnarray*}
The smallest singularity of $(1-x)^{-x}$ is at 1, so a crude approximation would be
$$[x^n](1-x)^{-x}\approx1^n=1$$
and
$$t_n=\left[\frac{x^n}{n!}\right](1-x)^{-x}/\mathrm{e}\approx n!/\mathrm{e}.$$
Certainly, a finer analysis of the singularity of $(1-x)^{-x}$ would give a better approximation and perhaps produce the power $\alpha$ you're seeking.
Now, back to the original problem. It's always the case that $|m|\le n$, so a rough bound on $s_n$ would be $$s_n\le(2n)!t_n\le(2n)!\, .$$ This bound is worse than the hope you expressed, but perhaps good enough for your eventual purposes or perhaps a start for finer analysis.
Comment: Thank you for your interest and your efforts! Could you please explain the link with the exponential generating functions (I am not familiar with generating functions)? Also, I do not understand why we can rewrite $t_n$ with the terms $x^n/n!$ and the product of exponentials (there was no $x$ in your original $t_n$)? Thanks in advance.
Comment: @Nicolas There are a zillion places to learn about generating functions. A couple of my favorites are [Concrete Mathematics](https://notendur.hi.is/pgg/(ebook-pdf)%20-%20Mathematics%20-%20Concrete%20Mathematics.pdf) (chapter 7) and [Generatingfunctionology](https://www.math.upenn.edu/~wilf/DownldGF.html) (the first couple chapters). Generating functions form a really powerful tool, and they're well worth the time to sink your teeth into. After digesting a chapter or two of those texts, this answer might be more meaningful.
Comment: Ok thank you for the reference. Unfortunately, I do not have much time to read it for now... Anyway, good start point here!

Computation of a series.


A simpler way to solve this determinant equation-$2$

Question:
Question Statement:-
  Solve the following determinant equation
  $$\begin{vmatrix}
x & 2 & 3 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}=0$$

My Solution:-
$$\begin{vmatrix}
x & 2 & 3 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}=
\begin{vmatrix}
x+13 & x+14 & x+15 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}\tag{$R_1\rightarrow R_1+R_2+R_3$}$$
$$(x+13)\begin{vmatrix}
1 & 1 & 1 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}+
\begin{vmatrix}
0 & 1 & 2 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}\tag{1}$$
Now lets solve the two matrices obtained in the last step above seperately.
$$(x+13)\begin{vmatrix}
1 & 1 & 1 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}$$
$$=(x+13)\begin{vmatrix}
0 & 0 & 1 \\
2 & x & 4 \\
-(x+1) & -x & x+8 \\
\end{vmatrix}\hspace{3cm}(C_1\rightarrow C_1-C_3, C_2\rightarrow C_2-C_3)$$
$$=x(x+13)\begin{vmatrix}
0 & 0 & 1 \\
2 & 1 & 4 \\
-(x+1) & -1 & x+8 \\
\end{vmatrix}=x(x+13)(x-1)$$
And the second matrix can be simplified as
$$\begin{vmatrix}
0 & 1 & 0 \\
6 & x+4 & -4-2x \\
7 & 8 & x-8 \\
\end{vmatrix}\tag{$C_3\rightarrow C_3-C_2$}$$
$$=-1\times(6x-48+28+14x)=-20(x-1)$$
So, we get
$$\begin{vmatrix}
x & 2 & 3 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix}=0\implies x(x+13)(x-1)-20(x-1)=0\\
\implies (x-1)(x^2+13x-20)=0\implies x=1,\dfrac{-13\pm\sqrt{249}}{2}$$
As can be seen, my solution is way long and it is no better than opening the determinant. So if anyone can provide me with a better, simpler and a more intuitive solution I will be very thankful.
Comment: Typo: $x^2+13x-20$, not $x^2+13-20$
Comment: @paf-Thanks corrected it.
Comment: Your solution is perfectly correct and I don't have a simpler idea.
Comment: @paf - There has to be a smaller solution :'(, becuase I dont think that I am still an amateur in maths so there has to be which is at high school level
Comment: @paf-I saw my comment it sounds kind of rude, what i wanted to say was "There has to be a smaller solution :'( becuase, I think that I am still an amateur in maths so there has to be which is at high school level "
Answer: Your computation is correct.
Here is a different procedure where we use the rule of Sarrus: the determinant is
$$x(x+4)(x+8)+56+144-21(x+4)-32x-12(x+8)\\
=x^3+12x^2-33x+20=(x-1)\cdot(x^2+13x-20).$$
Here you have to guess that $1$ is a root of the third degree polynomial and then divide it by $(x-1)$ in order to obtain the other factor $x^2+13x-20$.
Answer: Your solution is fine but here is a more simple way to solve this problem:
$$\begin{vmatrix}
x & 2 & 3 \\
6 & x+4 & 4 \\
7 & 8 & x+8 \\
\end{vmatrix} $$
= $$-\begin{vmatrix}-\begin{pmatrix}
0 & 2 & 3 \\
6 & 4 & 4 \\
7 & 8 & 8 \\
\end{pmatrix} - x I)\end{vmatrix} = 0$$ where $I$ is the identity matrix.
It lookes similar to the equation for eigenvalue.
So $x$ is eigenvalue for matrix:$$A :=\begin{pmatrix}
0 & -2 & -3 \\
-6 & -4 & -4 \\
-7 & -8 & -8 \\
\end{pmatrix}$$
Because $trace(A) = -12$ and $det(A) = -20$ so the characteristic polynomial of form: $-x^3 -12 x^2 +B x -20$. Where $B$ is an real number.
How to find $B$?
Let $x = 2$ into $det(-A-xI)$,then $det(-A-xI) = -10 = -76+2B \implies B=33$.
So it becomes solve this polynomial:$-x^3 -12 x^2 +33 x -20=0$
EDIT: Another way to find B is to use $x = 1$ as subsitution, as Francesco suggested in comment. Since the row become dependent then the determinant of matrix becomes zero so $-1-12-20+B =0 \implies B = 33$
Comment: you could have found B more simply though, -1 -12 -20 = -33 so B=33, and it's no accident ;) since for x=1 the matrix assumes a pretty nice form and the rows become linearly dependent
Comment: Really smart way, +1 for you
Comment: @FrancescoAlem. Thanks francesco.


I18N, translating in template variables

Question: I've got a theme setup, and this theme has a header region (region--header.tpl.php). There are some small ui elements in this template that require translation, but are not tied to blocks, views or nodes. My understanding is that when using something like <?php print t('Select a topic'); ?> it should become available to the translation backend. But it does not for me? I have the I18N module and Multilingual Variables module installed.
What am I missing?
Answer: Ah I found that one needs to view the site in another language than English in order to have the strings to be registered.

I18N, translating in template variables


How to create a drop down field to categorize content when creating new pages?

Question: I want to have a drop down field available so that when a content author (user) creates a new page, they can select which section of the website this new content should get placed.
For example, my top navigation may be About Us > Services > Products. A user creates a new page that should go on in the Products section. When they go to add the new page, there should be a drop down field that shows About Us, Services, Products. The user would select Products and when they save the page, that page will automatically get placed in the Products section of the website.
Can anyone tell me how to achieve this?
Comment: a proper answer is dictated by how you are creating those sections, and what you want people to see when they go to those pages
Answer: The drop-down can be created as a Taxonomy.
The section pages can all be one View with a Contextual Filter for each taxonomy term.
Answer: Consider using the Taxonomy Menu module for this. Here is an excerpt from its project page:

Transforms your taxonomy vocabularies into menus with ease!
Features

Custom Menu per vocabulary
Uses the default taxonomy term path or custom paths
Integrate with Views
Integrate with Pathauto: use URL/path aliases
Does not interfere with other menu modules

By creating an hierarchical taxonomy, it should be straight forward for your content editors to use this feature. That would take care of the "dropdown" field you're looking for already.
Next you'd have to think of how to implement what you consider "section" of the website, how would you technically implement that? Could it be something like "can be accessed by a user who has access to role ABC?". I'm thinking of a few possible techniques for that, based on the Rules module, possible with integrations from a few other modules also, such as:

Flag.
Content Access.

So please edit your question to better explain what you mean by "section". After you do, I can update my answer accordingly.


An example of Noether normalization

Question:
Let $A=k[x_1,x_2]/(x_2^2-x_1^3+x_1)$. As an example of Noether normalization, determine elements $y_1,\ldots,y_m\in A$, algebraically independent over $k$, such that $A$ is a finite $k[y_1,\ldots,y_m]$-algebra.

This is a problem in the Klaus Hulek's Elementary Algebraic Geometry. I think the book's proof of Noether normalization is not actually constructive...
Could anyone show me how to determine the $y_1,\ldots,y_m$ ?
Comment: Well, it doesn't say "application", it says "example.  Have you tried some things?  Have you looked at simpler examples?  Here's the question that should always be asked first: What is the simplest finitely generated $k$-algebra for which you cannot calculate a noether normalization?
Comment: Oh, and one more question, perhaps an important one: How large do you think $m$ is, in this example?
Comment: I didn't encounter any example of this kind before...so I have got no clue to how I may find such y1~ym. Certainly, m must be lower than 3. Could you give any hint?
Comment: If you know this basic theorem relating the dimension to transcendence degree, you can determine $m$ immediately, assuming that you know the Krull dimension of $A$.
Comment: Honestly, almost anything you try will lead to progress here.  Like I said if the problem is too hard, try to do it for something with one variable.
Comment: The proof of Noether normalization which can be found in almost every book on commutative algebra is constructive. It can be used to find specific elements as required.
Answer: The Krull dimension of $A$ is $1$, so almost every choice of $y\in A-k$ is good.

An example of Noether normalization


How to extract weights of hidden layers with Keras API

Question: I have a simple sequential neural network which I would like to use to train a classifier. It is made of one input layer, 3 hidden layers and one output layer, as follows:
<code>    #sequential type of model
model = Sequential()
#stacking layers with .add
model.add(Dense(len(ytrain), activation='relu', input_dim=100))
model.add(Dropout(0.5))
model.add(Dense(len(ytrain), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(len(ytrain), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(len(ytrain), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(5, activation='softmax'))
</code>
How can I extract the weights associated with each hidden layer. The ultimate goal is to then use the activation function to compute the probability of each label to be the correct one.
Hope you understand. Any kind of help is appreciated.
Answer: <code>weights = [layer.get_weights() for layer in model.layers]
</code>


Combination of cross-correlation and t-test

Question: I have a coarse and noisy image data taken from two different but geometrically similar samples (pixel size are all the same). I want to prove that there is a statistically significant differences between the images from the two samples.
What I am thinking is calculating cross-correlation, and conducting t-test.
Cross-correlation between the images of the same sample should be higher than the correlation between the images of the different samples. There are about 30 images from each sample, so I can get 30C2 x 2 values from the correlation of the same sample and 30 x 30 values from the correlation of different sample. I wonder t-test is applicable to statistically prove the difference of those two groups of values.
Any comments & advice are greatly appreciated!
Comment: That depends. For example, if I were to take a picture of myself, and compare it to a picture of myself that is rotated 180 degrees, the two pictures would be statistically different pixel by pixel, but, in a larger sense, not different at all. I think one has to allow for what image differences are permissible and which are not, e.g., translations, rotations, flips, plastic deformations etc.
Comment: Thanks for your comment. You can assume that there is no rotation, flips, translations etc.  The images has been taken in the exactly the same way. It is like one is "new sample" and the other is "degraded sample", and there is no other difference. But because of the noise in the image, even the images from the same sample look a little different. Do you think cross-correlation and t-test still works?
Comment: No, testing of that type is irrelevant to the hypothesis of unchanged information content. All information is context dependent. For example, suppose that an image is Gaussian smoothed, then comparison to the original should then be done AFTER deblurring of the smoothed image.
Comment: Sorry, I have some confusion.. My understanding so far is that the combination of cross correlation & t-test can work, but the samples may have some problems.. Is that correct? Do you mean there should not be a problem as long as I compare raw signals/images?
Comment: I also thought that even if the information is context-dependent, isn't it possible to analyse statistical difference under that specific context (as long as every image is under the same context)? I am sorry if it is a silly question. I am not good at statistics.
Answer: In a known noise environment, e.g., Gaussian, Poisson, the statistical significance for an imaging feature for noise reduction and image reconstruction can be set by the user, e.g., see the Pixon method of image reconstruction. In other words, an image is noise and information, and let us call the latter the latent image. Next, if we wish to compare two images that only differ by noise content, and have the same latent image, then we know a priori that there is no significance of difference between those images, but there is a noise difference. Now, any test that we would perform that quantifies both noise in an image and the identical latent images, e.g. t-testing, would test for a detectable difference in the noise, not the images.
More generally, suppose we have two images and we wish to establish whether the latent images are the same, and discard the less than helpful noise. For example, to compare a lakeside on a clear day, with the same lakeside barely visible in a fog, we would first apply image reconstruction, and then any rotations, translations, deformations, or contrast adjustments of those images to best superimpose them. Then, if the difference between the two adjusted images is only noise, and is without visible structure, then there is no significant structure to that difference. It helps to look at this rather than just talk about it, as follows.

Follow up answer: A Pixon would be a paraboloid of revolution. That is the element that would be examined for tolerance of fit for the Pixon method. However, many Pixons would be included in a 2D map, and the agreement of fit is local, not global. It may be to compare images that you might settle for a less than ideal global procedure. For example, after noise reduction, rank all the pixel magnitudes in each image from least to greatest, and compare the distributions. That could be done using, for example, Spearman's rank correlation. That may avoid some of the image comparison problems like different image intensity, rotation and translation. One could even smooth both images (identically, and after noise reduction) to filter out additional noise for preventing false positive differences.
Comment: Thank you for the very informative answer. But I am afraid that it does not not answer my question. I got images from two different samples: 30 images from sample A and 30 images from sample B. Two samples generate **two different latent images**, but both are contaminated by noise. And I want to use t-test to prove there is a significant difference between the two samples A & B without a noise reduction measures. I am not trying to examine the difference between the images from the same samples (of course it just sees a noise).
Comment: T-test is conducted to distinguish two groups. One group has values obtained by cross-correlation of images between the sample A & B. The other group has values obtained by cross-correlation of images within the sample A (or B).
Comment: If two images are noise reduced so that the noise reduced latent images are available, then they can be compared. Using a *t*-test to do that would not be my first choice, [Mann-Whitney U test](https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test) would be better.
Comment: Thanks, I will see if I can reduce the noise from my data. Mann-Whitney test seems more suitable than t-test. But can you apply that test to multi-dimensional data (2D image) ? The reason I was thinking about (normalised) cross-correlation is it can express the similarity of images with single value so that hypothesis testing can be easily followed. If you could give me information of how to test the differences of multidimensional data with Mann-Whitney test, that would be very helpful!
Comment: Thanks for the follow-up answer. I will try using Spearman's rank correlation, and I will also try Mann-Whitney U test with cross-correlation. I have still difficulty to understand Pixon method. I may ask you a question in the future.


Points, Vectors, and Planes Oh My! (Finding parallel vectors and intersecting vectors)

Question: I'm quite stumped on the following problem:
Consider the planes 4x + 1y + 1z = 1 and 4x + 1z = 0.
(A) Find the unique point P on the y-axis which is on both planes. (_, _, __)
(B) Find a unit vector u with positive first coordinate that is parallel to both planes __I + __J + _K
(C) Use the vectors found in parts (A) and (B) to find a vector equation for the line of intersection of the two planes r(t) = __I + __J + _K
Work thus far:
I've figured out (A) is (0,1,0)
Now, I know that the dot product of <4,1,1> and u will = 0, as well as the dot product between <4,0,1> and u...
I'm quite stumped. Can anyone help me out by giving me hints? I'd prefer that the entire solution isn't given yet, so I can work through it. I'll respond as quickly as I can.
Thank you,
Landon
Answer: (B) Write $\mathbf{u}=(a,b,c)$. You want $(4,1,1)\cdot (a,b,c) = 0$, so that means $4a+b+c=0$. You also want $(4,0,1)\cdot(a,b,c)=0$< so that means $4a+c=0$. This tells you that you must have $b=0$, and $c=-4a$. So the general form of vectors that are parallel to both planes is $(a,0,-4a)$. Since you want it to be a unit vector, you want $a^2 + 0^2 + (-4a)^2 = 1$, or $17a^2=1$. This gives you the absolute value of $a$ by solving for $a$, from which you can find one with positive first coordinate.
(C) Now that you have a point in both, and a direction parallel to both, that point and direction will give you a line that is common to both planes, giving (C).
Comment: Thank you! I've been having a terrible time with all this vector-space stuff. (B) was real easy after you explained it. I struggled a little longer on (C), because I'm quite the fool. I figured it out finally though :D


Mean and variance of ranks

Question: Consider rank data 1 to n with two groups, n=n1+n2, how would one test the null that the two groups have equal rank distributions using MOMENTS? (Wilcoxon is not the answer)
Is MLE possible to do the above?
If the ranks are for time periods, how does that affect the mean and variance.
Comment: Surely you can identify the mean of the numbers $1,2,3,...,n$.
Comment: Could you elaborate on "Wilcoxon is not the answer"? The Wilcoxon test (using the usual adjustment for ties when mean ranks in tied groups are assigned to each group) is a perfectly valid test based on the first two moments of the ranks. When the asymptotic distribution is used to compute p-values, it is *identical* to a pooled-variance z-test.
Comment: Cross-posted at [Math SE](http://math.stackexchange.com/questions/1068565/mean-and-variance-of-rank-data-with-and-without-ties).
Comment: There's no need to consider uniform distributions. The ranks take known values; under the null (plus the assumptions) you're randomly sampling (without replacement) from those known values.
Comment: How exactly will you rank ties? There are several conventions and the answer depends on which you choose.
Comment: To me the question is trying to look at ordinal and fractional ranking systems. Does this answer that?
Comment: This seems like routine bookwork, such as might be set as work or revision for some subject. Please add the `self-study` tag, and read its [tag wiki](http://stats.stackexchange.com/tags/self-study/info), modifying your question as suggested there.
Comment: Of course, I can handle that. Ties would give the same mean if I'm thinking about that correctly. But the notation for variance considering ties is giving me issues.
Comment: I've edited to decrease the content of things that I already have to make my confusion more clear. Showing this by Wilcoxon is "not allowed" which is why I'm confused on this. So, considering 1 to n, is it now more of a uniform distribution and I can continue with the mean and variance from that distribution? All I know of MLE is that I "think" that the uniform distribution would be suitable. I am certainly clueless on how considering the ranks for time periods would change this.
Answer: The question could be clearer -- in particular, the way it's phrased, there are any number of things we could test for when assessing equality of rank distributions using moments - are we comparing the first moment, the first two moments, the first three, ... the first $n$ moments?
Let's just look at the first moment to start:
The ranks of $n$ items without ties are the numbers $1,2,\ldots,n$
Under the null, $n_1$ ranks are selected randomly from those $n$ to be the ranks of the first group and $n_2$ ranks are selected randomly to form the ranks of the second group.
So under the null the average rank in the first group will be $\frac{n+1}{2}$ and so will
the average rank in the second group.
If we take the difference in the average ranks in the sample we get a statistic that would be a reasonable test statistic for equality of rank distributions.
But that's an equivalent test to the using sum of ranks in say the first sample (they are linear transformations of each other with constant coefficients at given sample sizes). Which is to say, if you're comparing first moments, you are doing a Wilcoxon test.
Note that the variance of ranks is also not so hard. If you have a random selection of $n_1$ values (without replacement) from the set $1,2,...,n$, then the average squared rank in the first sample is fairly easy to work out, and similarly the variance of the mean rank in the first sample, and so on.
You could of course construct some statistic that looked at second moments, or first and second moments or some function of the first three moments and so on.

If the ranks are for time periods, how does that affect the mean and variance.

It depends on what kind of dependence (if any) there is; it might make no difference; it might affect the variance depending on the particular kind of dependence.

Mean and variance of ranks


Fail to read information from a web site using Google Apps Script

Question: I have tried to use "UrlFetchApp.Fetch" in google apps script to retrieve data from https://metoc.ndbc.noaa.gov/web/guest/jtwc".
However, the information highlighted in  red as shown here "web capture" cannot be captured in the file I downloaded. Please help, thanks.
var FILE_NAME = 'data.txt'
var Google_DRive_ID = 'your google drive folder id'
var RESOURCE_URL = 'https://metoc.ndbc.noaa.gov/web/guest/jtwc'
var folder = DriveApp.getFolderById(Google_DRive_ID);
var exportUrl = RESOURCE_URL
var data = UrlFetchApp.fetch(exportUrl)
folder.createFile(FILE_NAME, data)

Answer: When I saw the source of https://metoc.ndbc.noaa.gov/web/guest/jtwc, it was found that the source you want is included in iframe with the id="_48_INSTANCE_0SiamlX2KcM6_iframe". The source URL is src="/ProductFeeds-portlet/img/jtwc/html/coop.jsp?". When your script is modified using the URL, it is as follows.
Modified script :
var root = "https://metoc.ndbc.noaa.gov";
var src = "/ProductFeeds-portlet/img/jtwc/html/coop.jsp?"; // You can also use "/ProductFeeds-portlet/img/jtwc/html/coop.jsp"

var FILE_NAME = 'data.txt'
var Google_DRive_ID = 'your google drive folder id'
var RESOURCE_URL = root + src;
var folder = DriveApp.getFolderById(Google_DRive_ID);
var exportUrl = RESOURCE_URL
var data = UrlFetchApp.fetch(exportUrl)
folder.createFile(FILE_NAME, data)

If I misunderstand your question, I'm sorry.
Comment: Hi Tanaike, First of all, thanks for your enlightenment. I learnt from you the use of 'root' + 'src'. Actually, the 'cool.jsp' doesn't get the information I need. But I just found another 'src' i.e. src = "/RSSFeeds-portlet/img/jtwc/jtwc.rss?" + (new Date()).getTime()". I used it. It finally works now. Thanks.
Comment: @warsur ``jtwc.rss`` is included in ``coop.jsp``. I couldn't notice that what you want is it. I'm sorry for taking your time. But I'm glad your problem was solved.


Infinite series question involving integrals

Question:
Can someone help me solve this tricky infinite series problem?
I tried to find the indefinite integral of the nth term but my solution didn't make sense at all.

I suspect there must be an alternative better approach.
Any help would be appreciated.
Comment: Use the given indication and show where you stuck
Comment: Agreed.  Use the substitution you were given.  Do not evaluate the integral.  (P.S. In any case, your evaluation is incorrect.)
Comment: How? show how that works.
Comment: You're right, I've corrected my attempt.
Answer: First let's try the substitution in evaluating $u_n$: we let $T = t - \pi$, so $du = dt$. The bounds $t = n\pi$ and $t = (n+1)\pi$ get sent to $T = (n-1)\pi$ and $T = n\pi$, respectively. Also note that we get $\sin{t} = \sin(T + \pi) = -\sin{T}$. Then we can write $u_n = \int_{n\pi}^{(n+1)\pi}\frac{\sin{t}}{t}dt = \int_{(n-1)\pi}^{n\pi}\frac{-\sin{T}}{T + \pi} dT$. Thus $u_{n+1} = \int_{n\pi}^{(n+1)\pi}\frac{-\sin{T}}{T + \pi} dT$.
Now note that $|\int f(t)dt| \leq \int|f(t)|dt$, so we can bound $u_{n+1}$: $|u_{n+1}| = |\int_{n\pi}^{(n+1)\pi}\frac{-\sin{T}}{T + \pi} dT| \leq \int_{n\pi}^{(n+1)\pi}|\frac{-\sin{T}}{T + \pi}| dT \leq \int_{n\pi}^{(n+1)\pi}\frac{\sin{T}}{T + \pi} dT \lt \int_{n\pi}^{(n+1)\pi}\frac{\sin{T}}{T} dT = u_n \leq |u_n|$
Comment: Thanks nilradical1


How do I import taxonomy terms

Question: This is almost 6000 rows of taxonomy terms ranging over 8 columns that i'd like to import from csv. I would like to use it for facet search.
The CSV is located here (google_product_category).
The format is like:
<code># Google_Product_Taxonomy_Version: 2015-02-19
1 - Animals & Pet Supplies
3237 - Animals & Pet Supplies > Live Animals
2 - Animals & Pet Supplies > Pet Supplies
3 - Animals & Pet Supplies > Pet Supplies > Bird Supplies
7385 - Animals & Pet Supplies > Pet Supplies > Bird Supplies > Bird Cage Accessories
</code>
Answer: As popular Drupal 7 modules like Taxonomy CSV import/export and Taxonomy Manager are not ported to Drupal 8, I recommend a very simple but functional module called term_csv_tree_import.
After installing, go to <code>/csv-tree-import</code> and paste your taxonomies into the form. It assigns parent values from right to left, empty intermediate elements are allowed. You have to convert your CSV into the following format:
<code>"element","subelement","subsubelement"
</code>
Comment: Noob here, how do you access /csv-tree-import I have installed and activated your module but i cant find where to use the option from admin
Comment: It's your Drupal install directory + the path, for example: http://www.example.com/csv-tree-import
Comment: Ah was expecting /term_csv_tree_import instead of ..../csv-tree-import. Thanks
Answer: At the time of this writing, the options on drupal.org are:

Taxonomy Import
Term CSV Export Import
Migrate Source CSV

I'd evaluate based on the method you're most comfortable with, the issues in the queues and the number of sites using each module.
Answer: You can also refer to the following module:
https://www.drupal.org/project/hti
However, CSV file needs to be formatted a bit differently. You would find a sample screenshot there.
Comment: What  does this module do?

How do I import taxonomy terms


Understanding the two-stage choice paradigm

Question: In Manski - The structure of random utility models the following example is proposed:
Consider the alternative set space a = $(\alpha,\beta,\gamma)$ with the attribute representation: $X = \begin{bmatrix} & \alpha & \beta & \gamma & \cr x_1 & 1 & 1 & 2 \cr x_2 & 0.5 & -0.5 & -1 \end{bmatrix}$.
The individuals/decision-makers $\sigma$ and $\tau$ are characterized by the attributes: $S = \begin{bmatrix} \sigma & \tau \cr 1 & 0 \end{bmatrix}$.
Finally the function $w(x,s)$ which defines the utility of the utility maximizing agents $\sigma$ and $\tau$ is given by $w(x,s) = x_1 + x_2\cdot s$.
The aim is to compute the probabilities: $P_t(a\in^c C)$ which is the probability that a decision-maker $t\in T$ will choose the alternative $a$ from a possible choice set $C$. By the definition of the alternative space a one can think of $C$ as any non-empty subset of any particular order of the alternatives $\alpha, \beta$ and $\gamma$. Hence there will be quite a few possible choice sets $C$ from which a decision-maker can choose. Manski denotes the choice set space, the set of all possible non-empty subsets $C$, with $\Gamma$ and $T$ represents the population of all possible decision-makers.
To calculate $P_t(a\in^c C)$ additional information is given. The joint distribution $M_{\Gamma T}((C,t))$ of possible choice sets $C$ and the decision-makers $t$ is given by:
$M_{\Gamma T}((\alpha,\beta,\gamma),\sigma) = \frac{2}{36}$ for each of the six ordered choice sets whose elements are $\alpha,\beta$ and $\gamma$
$M_{\Gamma T}((\alpha,\beta),\sigma) = \frac{1}{12}$ for each of the two ordered choice sets whose elements are $\alpha$ and $\beta$
$M_{\Gamma T}((\alpha,\beta,\gamma),\tau) = \frac{1}{36}$ for each of the six ordered choice sets whose elements are $\alpha,\beta$ and $\gamma$
$M_{\Gamma T}((\alpha,\beta),\tau) = \frac{2}{12}$ for each of the two ordered choice sets whose elements are $\alpha$ and $\beta$
To calculate the probabilities $P_t(a\in^c C)$ for the case that the information of $x_2$ is not observed we first need to calculate the vector $W_{Ct}$ which  contains the utilities for the alternatives contained in $C$ for a decision-maker $t$. In this case this will yield:
$\bar W_{(\alpha,\beta,\gamma),\sigma} = \begin{pmatrix} 1.5\\ 0.5\\ 1\end{pmatrix}$ and $\bar W_{(\alpha,\beta,\gamma),\tau} = \begin{pmatrix} 1\\ 1\\ 2\end{pmatrix}$
For the calculation of the probability $P_t(a\in^c C)$ Manski gives the following two formulas:
$(1) \ P(\bar W_{Ct} | r_{Co},s_{to}) = \frac{ \sum_{ (\tilde C, \tilde t):r_{\tilde Co} = r_{Co}, s_{\tilde to} = s_{to}, W_{\tilde C\tilde t} = \bar W_{Ct} } M_{\Gamma T}((\tilde C,\tilde t))}{\sum_{ (\tilde C, \tilde t):r_{\tilde Co} = r_{Co}, s_{\tilde to} = s_{to}} M_{\Gamma T}((\tilde C,\tilde t))}$
$(2) \ P_t(a\in^c C) = \sum_{\bar W_{Ct}:\bar w_{at}\ge \bar w_{\tilde at}, \tilde a\in C}P(\bar W_{Ct} | r_{Co},s_{to})$
So clearly in order to calculate $P_t(a\in^c C)$ we need to calculate (1) first. This is done by summation over $M_{\Gamma T}((\tilde C,\tilde t))$ for all possible choice problems $(\tilde C,\tilde t)$ for particular values of $r_{\tilde Co},s_{\tilde to}$ and $W_{\tilde C\tilde t}$ namely $r_{Co},s_{to}$ and $\bar W_{Ct}$. To do that we need to consider the definition of $r_{Co}$ and $s_{to}$. Manski proposes that $X$ and $S$ are split up into observable and non-observable parts like $X = [X_o,X_u]$ and $S = [S_o,S_u]$ where the "o" donates the observable, and "u" the non observable part. Now we can write $X_o$ as $X_0 = (x_{ao}, \forall \ a\in $ a). The definition of $r_{Co}$ is now given by $r_{Co} = (x_{ao}, \forall \ a\in C)$. So the only difference between $X_o$ and $r_{co}$ is the set on which $x_{ao}$ is defined on, since $r_{Co}$ considers all possible choice-sets $C$, $X_o$ only defines $x_{ao}$ on the alternative space a.
Now my dilemma: Since I need to calculate (1) in order to calculate (2), I need to sum up the joint probabilities $M_{\Gamma T}((\tilde C, \tilde t))$ over possible values of $r_{Co}, s_{to}$ and $\bar W_{Ct}$. Since the value of $s_{to}$ is given by the integers $1$ or $0$ and $\bar W_{Ct}$ is just one possible utility-vector depending on which choice-set $C$ we are currently looking on, I only need to figure out what $r_{Co}$ actually is. The definition is quite clear but for this example I'm not quite sure. Since $x_2$ is not observed I thought of $r_{Co}$ as a vector of the observed outcomes of $x_1$, i.e., $r_{Co} = (1.5,0.5,1)$ for $C = (\alpha,\beta,\gamma)$. But if I consider $r_{Co}$ in this way the calculation will not work since (1) will always be 1 and the sum in (2) can get larger then 1 which is of course not possible.
I'm stuck with this problem a couple of days now and any help will be much appreciated!
Answer: This would be my answer for my question if someone would ask me. With no further knowledge, what the solotion would look like, though I already now the solution because it's part of the previous mentioned paper (page 237), I would argue like:
(it's going to be a quite long answer, but in my opinion necessary to explain all the details)
The solution, for the sets of three alternatives, i.e., for all ordered sets composed of $\alpha,\beta$ and $\gamma$ is given by Manski with:

$P_\sigma(\alpha\in^c (\alpha,\beta,\gamma))=\frac{1}{2}$
$P_\sigma(\beta\in^c (\alpha,\beta,\gamma))=\frac{1}{2}$
$P_\sigma(\gamma\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\alpha\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\beta\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\gamma\in^c (\alpha,\beta,\gamma))=1$

It's obvious that the sum $\sum_{a\in C}P_t(a\in^c C)$ needs to be $1$ for any $C$ and any $t\in T$.
If we calculate the utility-vectors $W_{Ct}$ for any $C$ and $t\in T$, i.e., for any ordered choice set $C$ composed of three alternatives $\alpha,\beta$ and $\gamma$ and any order choice set $C$ composed of the two alternatives $\alpha$ and $ \beta$ we will get:
$\bar W_{(\alpha,\beta,\gamma),\sigma} = \begin{pmatrix} 1.5\\ 0.5\\ 1\end{pmatrix}$ for any combination of $(\alpha,\beta,\gamma)$ and $t=\sigma$
$\bar W_{(\alpha,\beta),\sigma} = \begin{pmatrix} 1.5\\ 0.5\end{pmatrix}$ for any combination of $(\alpha,\beta)$ and $t=\sigma$
$\bar W_{(\alpha,\beta,\gamma),\tau} = \begin{pmatrix} 1\\ 1\\ 2\end{pmatrix}$ for any combination of $(\alpha,\beta,\gamma)$ and $t=\tau$
$\bar W_{(\alpha,\beta),\tau} = \begin{pmatrix} 1\\ 1\end{pmatrix}$ for any combination of $(\alpha,\beta)$ and $t=\tau$
If we have a look at $t=\sigma$, with no further knowledge but the information that a decision-maker is a utility maximizing agent, we would think that in any case, i.e., for any choice set $C$ of three or two alternatives, the agent would always choose the alternative $\alpha$.
Similarly the agent $t=\tau$ would choose $\gamma$ in any case in which he picks a alternative from set composed of three alternatives. For the case in which he picks alternatives from a set composed of two alternatives he would be indifferent between $\alpha$ and $\beta$.
If we have a look at formula (1) these ideas would translate into:
$P(\bar W_{(\alpha,\beta,\gamma),\sigma} | r_{(\alpha,\beta,\gamma),o}, s_{\sigma,o})=\frac{\frac{2}{36}}{6\frac{2}{36}} = \frac{1}{6}$ for any of the six possible orderings of the vector $\bar W_{(\alpha,\gamma,\beta),\sigma}$, i.e., for each of the six ordered choice sets $C$ whose elements are $\alpha,\beta$ and $\gamma$.
$P(\bar W_{(\alpha,\beta),\sigma} | r_{(\alpha,\beta),o}, s_{\sigma,o})=\frac{\frac{1}{12}}{2\frac{1}{12}} = \frac{1}{2}$ for any of the two possible orderings of the vector $\bar W_{(\alpha,\gamma),\sigma}$, i.e., for each of the two ordered choice sets $C$ whose elements are $\alpha$ and $\beta$.
Similarly we  get
$P(\bar W_{(\alpha,\beta,\gamma),\tau} | r_{(\alpha,\beta,\gamma),o}, s_{\tau,o})=\frac{2\frac{2}{36}}{6\frac{2}{36}} = \frac{1}{3}$
$P(\bar W_{(\alpha,\beta),\tau} | r_{(\alpha,\beta),o}, s_{\tau,o})=\frac{2\frac{1}{12}}{2\frac{1}{12}} = 1$
Comments: The reason for the $2$ being a part in the denominator of $P(\bar W_{(\alpha,\beta,\gamma),\tau} | r_{(\alpha,\beta,\gamma),o}, s_{\tau,o})$ and $P(\bar W_{(\alpha,\beta),\tau} | r_{(\alpha,\beta),o}, s_{\tau,o})$ is, that we only observe three distinct vectors of $W_{C,\tau}$, though we've six different orderings of the choice set $(\alpha,\beta,\gamma)$. For instance, for the two cases $C=(\alpha,\beta,\gamma)$ and $C'=(\beta,\alpha,\gamma)$, we will observe the same vector $W_{C,\tau}$. Since, by formula (1), we need to count how often we observe $W_{\tilde C, \tilde t} = \bar W_{C,t}$, this is what the $2$ in the denominator essentially does.
If we calculate (2), for a certain size of $C$ (like $|C|=3$), we need to consider all the values of $P(\bar W_{C,t} | r_{C,o}, s_{t,o})$ for which a value $\bar w_{a,t}$ is at least as big as all the other values $\bar w_{\tilde a, t}, \tilde a\in C$. If we do that, we will get the following values:
(all probabilities will be the same for any ordering of $(\alpha,\beta,\gamma)$. The "$\in^c (\alpha,\beta,\gamma)$" only adresses the set we are currently looking on)
$P_\sigma(\alpha\in^c (\alpha,\beta,\gamma))=6\frac{1}{6} = 1$
(there are six distinct cases of $W_{C,\sigma}$ in which the value $\bar w_{\alpha,\sigma}$ is greater then all the other values )
$P_\sigma(\beta\in^c (\alpha,\beta,\gamma))=0$
$P_\sigma(\gamma\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\alpha\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\beta\in^c (\alpha,\beta,\gamma))=0$
$P_\tau(\gamma\in^c (\alpha,\beta,\gamma))=3\frac{1}{3} = 1$
(there are only three distinct cases of $W_{C,\tau}$, hence the $3$ )
For $|C|=2$ we will get:
$P_\sigma(\alpha\in^c (\alpha,\beta))=2\frac{1}{2} = 1$
$P_\sigma(\beta\in^c (\alpha,\beta))=0$
Now the forumla (2) seems to fail for $t=\tau$, since the imposed condition not only accounts for the cases in which $\bar w_{a\tau}$ is greater then $w_{\tilde a,\tau}$ but also if the equality between those two quantities is true. In this particular case, $|C|=2,t=\tau$ all the values of $w_{\tilde a, \tau}, \tilde a\in C$ are equal. So by the definition of (2) we would count all the cases of $P(\bar W_{(\alpha,\beta),\tau} | r_{(\alpha,\beta),o}, s_{\tau,o})$ for each alternative. Here we only observe one possible value of $\bar W_{C,\tau}$, i.e., $W_{(\alpha,\beta),\tau}=W_{(\beta,\alpha),\tau}=\begin{pmatrix} 1\\ 1\end{pmatrix}$. So by (2) we would get:
$P_\sigma(\alpha\in^c (\alpha,\beta))=1$
$P_\sigma(\beta\in^c (\alpha,\beta))=1$
,since the condition "$\bar w_{a,\tau} \ge w_{\tilde a, \tau}, \tilde a\in C$" is true for both cases, i.e., for the summation of $a=\alpha$ and $a=\beta$. But this cannot be true because $\sum_{a\in (\alpha,\beta)}P_\tau(a\in^c (\alpha,\beta))=2\ne 1$. I would suggest the following work arround:
Instead of (2) I would suggest we could use
$(3) P_t(a\in^c C) = \sum_{\bar w_{a,t} > w_{\tilde a, t}, \tilde a\in C}P(\bar W_{C,t}|r_{Co},s_{to}) + \sum_{\bar w_{a,t} = w_{\tilde a, t}, \tilde a\in C}\frac{P(\bar W_{C,t}|r_{Co},s_{to})}{2}$
This equation would, in my optinion, better adress the issue of a agent being indifferent between all possible alternatives. The former mentioned issue, leaving all the other results unchanged, would be, by using (3) instead of (2):
$P_\sigma(\alpha\in^c (\alpha,\beta))=\frac{1}{2}$
$P_\sigma(\beta\in^c (\alpha,\beta))=\frac{1}{2}$


Out of memory when doing a big query?

Question: I received this error when trying to do big query.
java.lang.OutOfMemoryError: Java heap space

I've searched and found that applying setAutoCommit(false) and setFetchSize methods to my prepared statement might help handling big query.
However, when I used it, I received this error.
java.sql.SQLException: Illegal value for setFetchDirection().

What is the proper and easy way to handle large query?
What is the proper way to use setFetchSize?
Comment: It would be easier to advise with sample code.  The most common answer to this kind of thing though is to break the query into more managable chunks.  Do you really need all this data at once?
Comment: How many rows do you get while running the same query on `SQL` Console, also try increasing the `heap_size` for java
Comment: possible duplicate of [Streaming large result sets with MySQL](http://stackoverflow.com/questions/2447324/streaming-large-result-sets-with-mysql)
Comment: @Simonatmso.net - I'm afraid that I cannot share that. But what I can tell is query string is not optimized and somewhat horrible. Too bad, it's not easy to modify it (it's already bad before I come to work on it ;P).
Comment: @Akash I've also tried "LIMIT 0,1" and it's still unable to query. I'll check heap_size.

Sorry I forgot to mention that I use java, JDBC, and mysql.
Comment: @YaK - How to use setFetchSize with prepared statement?
Comment: @AtthaponJunpun-eak that makes it pretty hard to offer anything but generic advice then, such as that already offered by others.  Specific advice to your problem generally requires an example of how you are trying to perform something, as there may be something in how you are performing something that someone else sees that is the cause for your problems.  Up to you though really.
Answer: Assuming you are using the MySQL Connector/J driver provided by MySQL, I believe the solution is found in this manual page (notice parameter 1 of Connection::createStatement()):

If you are working with ResultSets that have a large number of rows or
large values, and cannot allocate heap space in your JVM for the
memory required, you can tell the driver to stream the results back
one row at a time.
To enable this functionality, create a Statement instance in the following manner:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
              java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);

Comment: Is it possible to use setFetchSize with prepared statement?
Comment: Since `public interface PreparedStatement extends Statement`, I suppose it is. Likewise, I would consider using the method `Connection::prepareStatement(String sql, int resultSetType, int resultSetConcurrency)` to create such a `PreparedStatement`.
Answer: maybe do a lazy search, pull for example just ID's or something then when you want to use / display your data perfrom a query on just that one ID?
or maybe instead run it in a thread so it just goes off and does it in the background

Out of memory when doing a big query?


MVC Use Json to get Data and append to table

Question: I'm trying to call a web API and use json to bind the data to a table. When I run the debugger in Chrome it tells me it is getting a 404 error. When i call the API through a URL it is working fine. I think the problem is the script I am running in the view.
Here is my code
Controller
    public class WorkspaceController : Controller
{
    // GET: Portal/Workspace
    public ActionResult Index()
    {
        return View();
    }

    [HttpGet]
    public JsonResult GetData()
    {
        var contactId = 3311;

        List<WorkItemActiveGridModel> collection = new List<WorkItemActiveGridModel>();

        HttpClient httpClient = new HttpClient { BaseAddress = new Uri(string.Format("{0}://{1}", Request.Url.Scheme, WebConfigurationManager.AppSettings[ConfigurationKeys.DataServicesUri])) };
        httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        HttpResponseMessage httpResponseMessage = httpClient.GetAsync("Api/WorkItem/Active/" + contactId).Result;
        if (httpResponseMessage.IsSuccessStatusCode)
        {
            collection = JsonConvert.DeserializeObject<List<WorkItemActiveGridModel>>(httpResponseMessage.Content.ReadAsStringAsync().GetAwaiter().GetResult());
        }

        return Json(collection, JsonRequestBehavior.AllowGet);

    }
}

}
View
@model dynamic
@{
ViewBag.Title = "Customer Portal";
Layout = "~/Areas/Portal/Views/Shared/_LayoutPage.cshtml";
}

Open Items

            WorkItemID
            Title
            State
            PercentComplete
            OwnerName
            ContactName
            ReleaseDtm


archive_page() from archive module in drupal 6 shows all nodes not date specific

Question: I installed Archive module and used following code in a custom template:
print archive_page('all', 2012, 10, 12);

but it shows every single node I posted, How can I make it load nodes according to date I specified in the code?
Answer: I just tested archive page in Drupal 6.26 and using Archive 6.x - 1.4 . It is working as expected.  Maybe you are  date and month in the archive_page. Please find the detailed explanation below.
The parameters of archive_page() are the following ones:

$year
Number of year.
$month
Number of month.
$day
Number of day.

So print archive_page('all', 2012, 10, 12); will return the nodes created on October 12, 2012. I just followed the below process to verify the functionality.

I installed archive module
I enabled the module
I enabled content types in "admin/settings/archive"
I created two pages using Page one, and Page two as titles

Page one's authored date is October 13, 2012
Page two's authored date is October 15, 2012

I enabled the PHP filter module
I created a block with the below code, and enabled it on the right side bar
module_load_include('inc', 'archive', 'archive.pages');
print archive_page('all', 2012, 10,  13);

It does display the following output where only "Page one" is the one authored on October 13, 2012.
Comment: +100 Thanks, I did as you said but it didn't work for me. anyway thanks for your help.
Answer: Here is How I fix my problem:
Created a view that prints out months (link), enabled pagination and added some fields to print and now I get months list with post titles in pages.


HTML Table to CSV File in powershell

Question: Hi can anyone explain this:
$Table = $HTML -split '\<\/table\>' -replace '\<[^\?t]+[^\<]*','' -replace ' \w+="\w+%?"','' -replace '\<t\w+\>',',' -replace ',,\s?|,\r\n','' -replace ',\s',',' -replace ':,',': ' -replace "%","`r`n" -replace "\r\n,","`r`n"
# $Table | Out-File -FilePath "File.csv"

Comment: The answer from [@broffutt](https://stackoverflow.com/users/17801412/broffutt) gives some explanation about what the concerned syntax does but in general, I recommend against [attempting to parse HTML with regular expressions](https://blog.codinghorror.com/parsing-html-the-cthulhu-way/).
Instead use a dedicated HTML parser as the [**HtmlDocument** class](https://learn.microsoft.com/en-us/dotnet/api/system.windows.forms.htmldocument?view=windowsdesktop-6.0), see also: [Extracting HTML table as CSV](https://stackoverflow.com/q/67161977/1701026)
Answer: That code goes through and uses regex to replace certain characters with a comma to create comma separated values, or CSV.
It then uses Out-File to push the CSV data to a CSV file. Out-File literally just sends the data to a file.
If you are curious what the regex means, you can go to this site and paste each regex expression. The right hand side of the site will explain in detail what it is doing.

HTML Table to CSV File in powershell


cURL errors: undefined references

Question: I've found this thread while looking for a way to download a URL to my pc:
Download file using libcurl in C/C++
I downloaded curl for windows and merged all 3 folders to their existing counterparts in the MinGW directory: include, lib, and bin.
Now, whenever I compile the code, I got the following errors that I know they are due to not linking the proper libraries, but in the link I provided their is no sufficient info on what to link:
obj\Debug\main.o
In function main':
D:\CPP Scrap\curl1\main.cpp:11: undefined reference to __imp__curl_easy_init'
D:\CPP Scrap\curl1\main.cpp:13: undefined reference to __imp__curl_easy_setopt'
D:\CPP Scrap\curl1\main.cpp:14: undefined reference to __imp__curl_easy_perform'
D:\CPP Scrap\curl1\main.cpp:17: undefined reference to __imp__curl_easy_cleanup'
=== Build finished: 4 errors, 0 warnings ===

It shows that the header curl/types.h is being included but I cannot find it in the [latest] version of curl I downloaded, so I'm guessing I have to link with something, question is, what is it?
Code:
#define CURL_STATICLIB
#include <stdio.h>
#include <curl/curl.h>
#include <curl/types.h>
#include <curl/easy.h>
#include <string>

size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream) {
    size_t written;
    written = fwrite(ptr, size, nmemb, stream);
    return written;
}

int main(void) {
    CURL *curl;
    FILE *fp;
    CURLcode res;
    char *url = "http://localhost/aaa.txt";
    char outfilename[FILENAME_MAX] = "C:\\bbb.txt";
    curl = curl_easy_init();
    if (curl) {
        fp = fopen(outfilename,"wb");
        curl_easy_setopt(curl, CURLOPT_URL, url);
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
        res = curl_easy_perform(curl);
        curl_easy_cleanup(curl);
        fclose(fp);
    }
    return 0;
}

Comment: Did you mean to show us the error messages? Looks like you forgot to add them.
Comment: I'll guess you're getting undefined symbol errors, because you're missing `curllib.lib`. Does your download have these files? Have you set up your compilation / IDE to link with that file?
Comment: @birryree

I downloaded the curl files in a zip from their official website. I do not have **curllib.lib**, but I do have **libcurl.a**, and linking with it doesn't eliminate the errors.

Note, the curl files are also missing **curl/types.h**
Comment: @ErnestFriedman-Hill

These are the errors:
`
obj\Debug\main.o||In function `main':|
D:\CPP Scrap\curl1\main.cpp|11|undefined reference to `__imp__curl_easy_init'|
D:\CPP Scrap\curl1\main.cpp|13|undefined reference to `__imp__curl_easy_setopt'|
D:\CPP Scrap\curl1\main.cpp|14|undefined reference to `__imp__curl_easy_perform'|
D:\CPP Scrap\curl1\main.cpp|17|undefined reference to `__imp__curl_easy_cleanup'|
||=== Build finished: 4 errors, 0 warnings ===|
`
Answer: I don't know the details for Windows (I never used that system, but I am using GNU/Linux since 1993 with happiness), but you probably need to link the libcurl library (maybe using files like libcurl.lib and/or libcurl.dll or something similar).
But did you consider to use GNU/Linux as your development platform? If you want to use C++ & Curl, it makes a lot of sense (because Linux has a good free C++ compiler, GCC, and because libcurl is probably developed mostly for and on Linux). And the ability to glance into all the source code of your system enables you to learn a lot of things.
Comment: Thanks for asnwering. I am actually using the GNU minimcal compiler for Windows operating systems (MinGW ofcourse). So I don't think there must be much of a difference. I am linking to **libcurl.a**, but not to any dll. It still doesn't work, gives me an undefined error to the **curl_easy** functions

cURL errors: undefined references