Bootstrap, Monte Carlo

Question: I have been set the following question as part of homework:

Design and implement a simulation study to examine the performance of the bootstrap for obtaining 95% confidence intervals on the mean of a univariate sample of data. Your implementation can be in R or SAS.
Aspects of performance you may want to look at are confidence interval coverage (i.e., what proportion of times the confidence interval contains the true mean) and Monte Carlo variation (i.e., how much do the upper and lower confidence limits vary between simulations)'

Does anyone know how to go about the Monte Carlo variation aspect of this? I can't seem to even work out an algorithm or anything. Is it to do with Monte Carlo integration?
Thanks!
Answer: This confusion between bootstrap procedures and Monte Carlo procedures keeps recurring, so perhaps this is as good a place as any to address it.  (The examples of R code may help with the homework, too.)
Consider this implementation of the bootstrap in R:
boot <- function(x, t) { # Exact bootstrap of procedure t on data x
    n <- length(x)       # Must lie between 2 and 7 inclusive.
    if (n > 7) {
        stop("Sample size exceeds 7; use an approximate method instead.")
    }
    p <- c(n, 1:(n-1))
    a <- rep(x, n^(n-1))
    dim(a) <- rep(n, n)
    y <- as.vector(a)
    while (n > 1) {
        n <- n-1
        a <- aperm(a, p)
        y <- cbind(as.vector(a), y)
    }
    apply(y, 1, t)
}

A quick look will confirm that this is a deterministic calculation: no random values are generated or used.  (I will leave the details of its inner workings for interested readers to figure out for themselves.)
The arguments to boot are a batch of numeric data in the array x and a reference t to a function (that can be applied to arrays exactly like x) to return a single numeric value; in other words, t is a statistic.  It generates all possible samples with replacement from x and applies t to each of them, thereby producing one number for each such sample: that's the bootstrap in a nutshell.  The return value is an array representing the exact bootstrap distribution of t for the sample x.
As a tiny example, let's bootstrap the mean for a sample x = c(1,3):
> boot(c(1,3), mean)
> [1] 1 2 2 3

There are indeed four possible samples of size $2$ with replacement from $(1,3)$; namely, $(1,1)$, $(1,3)$, $(3,1)$, and $(3,3)$.  boot generates them all (in the order just listed) and applies t to each of them.  In this case t computes the mean and those turn out to be $1$, $2$, $2$, and $3$, respectively, as shown in the output.
Where you go from here depends on how you want to use the bootstrap.  The full information about the bootstrap is contained in this output array, so it's usually a good idea to display it.  Here is an example where the standard deviation is bootstrapped from the sample $(1,3,3,4,7)$:
hist(boot(c(1,3,3,4,7), sd))

Now we are prepared to talk about Monte Carlo simulation.  Suppose, say, we were going to bootstrap a 95% upper confidence limit on the SD from a sample of $5$ by using the upper 95th percentile of its bootstrap distribution.  What properties would this procedure have?  One way to find out is to suppose the sample were obtained randomly from, say, a uniform distribution.  (The application will often indicate what a reasonable distributional assumption may be; here, I arbitrarily chose one that is simple for computation but not easy to deal with analytically.)  We can simulate what happens by taking such a sample and computing the UCL:
> set.seed(17)
> quantile(boot(runif(5, min=0, max=10), sd), .95)[1]
     95%
3.835870

The result for this particular random sample is 3.83587.  This is definite: were you to call boot again with the same set of data, the answer would be exactly the same.  But how might the answer change with different random samples?  Find out by repeating this process a few times and drawing a histogram of the results:
> boot.sd <- replicate(100, quantile(boot(runif(5, min=0, max=10), sd), .95)[1])
> hist(boot.sd)

Were we to do another set of simulations, the random draws would come out different, creating a (slightly) different histogram--but not very different from this one.  We can use it with some confidence to understand how the bootstrap UCL of the SD is working.  For reference, notice that the standard deviation of a uniform distribution (spanning the range from $0$ to $10$ as specified here) equals $10/\sqrt{12} \approx 2.887$.  As one would hope for any UCL worth its salt, the majority (three-quarters, or 0.75) of the values in the histogram exceed this:
> length(boot.sd[boot.sd >= 10/sqrt(12)]) / length(boot.sd)
[1] 0.75

But that's nowhere near the nominal 95% we specified (and were hoping for)!  This is one value of simulation: it compares our hopes to what is really going on.  (Why the discrepancy?  I believe it is because bootstrapping an SD does not work well with really small samples.)
Review

Bootstrap statistics are conceptually just the same as any other statistic like a mean or standard deviation; they just tend to take a long time to compute.  (See the warning message in the boot code!)
Monte-Carlo simulation can be useful for studying how a bootstrap statistic varies due to randomness in obtaining samples.  The variation observed in such simulation is due to variation in the samples, not variation in the bootstrap.
(Not illustrated here) Because bootstrap statistics can take a lot of computation (apparently, up to $n^n$ calculations for samples of size $n$), it is convenient to approximate the bootstrap distribution.  This is usually done by creating a "black box" program to obtain one value at random from the true bootstrap distribution and calling that program repeatedly.  The collective output approximates the exact distribution.  The approximation can vary due to randomness in the black box--but that variation is an artifact of the approximation procedure.  It is not (conceptually) inherent in the bootstrap procedure itself.
Comment: +1 This is very nice. I don't think I could have done it better! I would just add that I have some papers on the performance of the bootstrap in error rate estimation and for confidence intervals when estimating variances. These results can be found in the second edition of my bootstrap methods book and my book on bootstrapping with applications to R.
Comment: Amazon links to the books are here: http://www.amazon.com/Bootstrap-Methods-Practitioners-Researchers-Probability/dp/0471756210/ref=sr_1_2?s=books&ie=UTF8&qid=1343146566&sr=1-2&keywords=bootstrap+methods   http://www.amazon.com/Introduction-Bootstrap-Methods-Applications/dp/0470467045/ref=sr_1_3?s=books&ie=UTF8&qid=1343146628&sr=1-3&keywords=bootstrap+methods
Comment: I would just say that the idea is to simulate data from known distributions apply the bootstrap procedure of interest (e.g. confidence intervals for a parameter or estimation of a classifier's error rate) and you can evaluate how well it does because you know the truth.
Answer: The bootstrap is a Monte Carlo technique, in that it involves some type of random sampling. If you run the bootstrap twice on the same set of data, you'll get different answers. The more samples you use in your bootstrap, the less variation you'll get.
The coverage refers to the variation of the results on different data sets from the same hypothetical sampling distribution.
Comment: To be precise (pedantic?), we should clarify that the bootstrap is *not* per se a Monte Carlo technique. However, it is usually *approximated* by using Monte Carlo simulation since it is not often tractable to calculate the particular quantities of interest. I find the wording of the problem very prone to misinterpretation as well! One must be careful to distinguish *sampling error* from *Monte Carlo approximation* error. The latter is only sensibly estimated by repeating the MC bootstrap approximation many times on the *same* sample, as you point out. :)
Comment: thanks, I'll assume it just means to come up with code to see if the confidence limits vary between the simulations then!
Answer: I'm not sure what is meant exactly by "Monte Carlo variation" per se, either.  Certainly it should be possible to look at how much variation there is amongst iterations in things like the value of the upper (or lower) limit, for example (hint).  Perhaps they only want you to do this for the Monte Carlo, and not the bootstrap?  That isn't a requirement I would have for an exercise, though.  It may be best to just ask what is meant by that phrase.
Comment: Fair enough, what I'm wondering now is, are you good to go at this point, or do you need extra hints about how to approach what we've guessed the question is asking for?
Comment: So, on each iteration, are you storing the upper & lower limits (hint)?
Comment: Thanks for your help! Perhaps I will just look at the variation between upper and lower limits for now and ask about Monte Carlo later. Now just to work out how I go about that..
Comment: There are several possibilities here. One is upper-lower (ie, the difference). My interpretation of the phrasing in the question is variation in upper, & variation in lower (ie, 2 different variations), but I suppose I can't be sure.  On a different note, is part of your question *how* to conduct MC simulation in R?
Comment: I came to the conclusion that it was looking for 2 different variations aswell, so think I will try that!
Comment: No that is the full question, and the only time it mentions Monte Carlo. We've been told that we can choose what bootstrap methods to evaluate, what distributions to sample from, how many simulations to perform etc, so I assume the whole question is open to interpretation...
Comment: I have been trying a few things but none that are working well in R, hints would be appreciated if you have any! :)
Comment: OK, so do you need hints about how to conduct a MC or bootstrap simulation in R, or about how to assess the results (the variation)?
Comment: I have a bootstrap simulation (both parametric and nonparametric) but not sure how find out how much the upper and lower
condence limits vary betwen simulations
Comment: Yes, I suppose I could just find a way to order the upper limits and find the difference between highest and lowest for a large number of simulations? (and do the same for lower limits)
Comment: That sounds a bit like looking at the IQR (inter-quartile range) of the sampling distribution of the upper (/ lower) limit. There may be other ways to assess the amount of variability in a sample, as well (hint). Which method would be ideal would depend on various factors, among others, what has been emphasized in your class.
Answer: My understanding of this homework assignment is that it is asking you to do a Monte Carlo simulation for a certain statistical technique. That is, you simulate a bunch of random data sets, apply this technique to these data set, and store the numbers to summarize later in a convenient way (means, simulated probabilities, etc.)
Now, the technique in question is the bootstrap, which involves Monte Carlo simulation within it (unless, as whuber demonstrated, you are asked to perform the exact bootstrap, which is highly unlikely). So as outcomes of your simulations, you may be storing the sample mean, the sample standard deviation, and the limits of confidence interval for the mean obtained by the bootstrap.
I think to make this study interesting, it is worth taking a small size sample ($n=10$) from a skewed distribution (exponential). That way, the standard CLT-based confidence interval will likely have inadequate performance: say, for 90% confidence interval that would need to miss 5% at either end, you will see something like 10% and 2%. The bootstrap CI may show better performance (say, 6% and 4%).
Comment: One thing I have learned from working with the exact bootstrap is that the standard normal-theory CI will often work much *better* with small samples even when they are skewed.  The bootstrap comes into its own with large samples: the theory is an asymptotic one, not an exact one.
Comment: @whuber, that's an interesting observation. I usually stress the asymptotic nature of the bootstrap (see Peter Hall's The Bootstrap and Edgeworth Expansions book to understand why), but failed to do it this time.

Bootstrap, Monte Carlo


Microsoft Security Essentials for servers?

Question: Microsoft Security Essentials seems to be a fine, lightweight AV solution, and it's getting a lot of positive reviews. Microsoft is explicitly positioning it as a solution for home PCs, but are there any reasons not to run it on a server? We use McAfee Total Protection Service on our desktops, but it's not entirely compatible with the server, and in any event the software firewall and link protection features aren't necessary on the server anyway. We have a single SBS 2008 server running things, so I want to make sure it has decent protection. Is MSE the answer?
Answer: if you care about licensing, then - no.

Use. You may install and use any
  number of copies of the software on
  your devices in your household for use
  by people who reside there or for use
  in your home-based small business.

http://www.microsoft.com/security_essentials/eula.aspx
m
Comment: That's changed now: Small Business. If you operate a small business, then you may install and use the software on up to ten (10) devices in your business.
Comment: and you find a way of bypassing the O/S detection as well....
Answer: There is Microsoft OneCare for Servers, it's a paid option.


Semi-martingale vs. martingale. What is the difference?

Question: Can someone please explain (preferably in layman's terms) what is the difference between semi-martingales and martingales?
I have found the following sentence on Wikipedia:

In probability theory, a real valued process X is called a semimartingale if it can be decomposed as the sum of a local martingale and an adapted finite-variation process.

I assume semi-martingale is a more general term than martingale. Is that correct?
Comment: First you need to understand the definitions of local martingales and adapted finite-variation processes. This requires a bit of measure theory which is not a very layman-friendly topic. It is true that semimartingales are a generalization of martingales, and in fact are the class of (stochastic) integrators for which the Itô integral is defined.
Answer: Martingale and semi-martingale have very precise mathematical definitions, so it's definitely not something easy to understand. I'll try to give some intuition without too much mathematical details.
Martingale is a stochastic process that it's expectation is equal to the current value. This means:

No clear trend (like a random process)
Your previous knowledge can't help you to predict the future

For example, if you have the Google stock, and it's $100. You might assume the daily stock movement is a martingale. The probability of it reaching 110 is the same as the probability of it reaching 90. On average, the most likely stock price after a week is still 100. If you need to make a bet on the most likely stock price after one week of trading, you should go for 100. Note in our example, we don't need to know the past stock price, only today's price is relevant. Since your best prediction is actually today's price, you aren't actually predicting anything.
Mathematically:
$$\mathbf E(X_{n+1}|X_1,\ldots,X_n)= X_n.$$
Semi-martingale is similar to martingale but it's not always a martingale. For example, if you can somehow use the past stock data to predict accurately Google stock price for the first week (and only the first week), it won't be a martingale process. Starting from the second week, the process becomes a martingale again.
Mathematically:

You can think semi-martingale like a merge of martingale (more precisely local martingale) and a non-martingale process.
Comment: A [càdlàg](https://en.wikipedia.org/wiki/C%C3%A0dl%C3%A0g) is basically a function whose left and right limits exist, and right-continuous.


Country code in taxonomy shown as flag

Question: I am using ISO 3166-1 alpha-2 codes for countries as a taxonomy in several pages of a given type.
When listing those pages in a view, I would like to display the corresponding country flag, rather than the two-letter code, e.g.

for
it
Is that possible, perhaps using CSS+JavaScript?
Many thanks!
Answer: For example code for a Drupal-based approach, check out the Flags module.
Also take a look at the Language Icons module, although this particular module works on languages, not taxonomy.
If you just want a CSS + JS solution, you should be asking on Stack Overflow, not here.
Comment: Flags module seems like a great option for this case. I'd just like to add that one of the common ways to implement it is to add the Flag field to your Taxonomy, so each term would have both the letter code and the image. Then in Views you pull the flag info through a Relationship, or through content View modes (you can create a mode where the letter code is always replaced with the image or something similar).

Country code in taxonomy shown as flag


Views conditional filter

Question: Is it possible for me to apply a filter in a view on a field only if that field is not empty? In other words if the field is empty the filter is not applied (and the row is shown), and if the field is not empty the filter is then applied and determines whether to show the row or not based on the field value.
My particular use case is a job expiry date. If no expiry date is entered I want the job to show, but if an expiry date is entered, then this determines if the job is displayed or not based on the current date.
Answer: It's possible to set up 2 filters:

Filter operating on date (as you described)
Filter "job expiry" field is empty

Then use the drop-button right of Filter crieria, go to And/Or, Rearrange and change the operator for group to OR
Let me know, if you need further assistance.

Views conditional filter


Como modificar uma janela no Windows Forms?

Question: Como alterar a aparência do Window borde, Title bar e button, e status bar no Windows Forms?
Comment: Jhonatas, é impossível alguém te responder uma pergunta se você não disser qual é a pergunta.
Comment: Do Visual Studio ou dos programas que você cria **no** Visual Studio?
Comment: Programas criados no VS
Comment: WinForms ou WPF?
Comment: Window-form's !
Comment: O que você quer alterar, especificamente?
Comment: No momento foi mais uma duvida, mas acho que a cor ou o formato.
Comment: O formato então. Como mudo o formato meio arrendondado para retilíneo e uniforme?
Comment: Da borda do título?
Answer: Nas propriedades do seu form, você pode escolher o BorderStyle entre algumas opções disponíveis fornecidas pelo próprio VS, e isto ira alterar os itens que citou. Todos estes estilos serão baseados no seu sistema operacional(Ex: se você tem windows 8 suas bordas serão quadradinhas, se esta no Windows XP suas bordas vão ser arredondadas, ou qualquer Visual Style que você tenha instalado)

Você também terá outras opções como:
ControlBox (true/false): Exibe ou não os controles Minimizar,Maximizar, restaurar tamanho e fechar.
Icon: Define o ícone do seu form que sera (ou não, dependendo do seu FormBorderStyle) exibido na TitleBar, barra de ferramentas e no Alt+tab
e algumas outras(é só dar uma olhadinha nas propriedades que você vai encontrar todas!)
Se não gostar de nenhum, também pode setar o FormBorderStyle para None e criar seu próprio visual(Aka Gambiarra) e dar ao form o formato dos seus sonhos!


GTK with Java on Windows

Question: I searched for GTK java bindings and found java-gnome but java-gnome not support Windows os why ? is it possible to developing gtk apps with java-gnome on windows ?
Comment: https://github.com/JavaGtk/JavaGtk ?
Answer: According to this; there is no version available for Microsoft Windows unlike most other GTK+ bindings. So I'm afraid you cannot fulfill your purpose but you can do the same using other languages

GTK with Java on Windows


Is there a relation between the polynomial that generates a splitting field and its minimal polynomial?

Question: It's not too hard to show that the splitting field of $f(x) = x^3 - 2 \in \mathbb{Q}[x]$ over $\mathbb{Q}$ is $\mathbb{Q}(2^{1/3}, \omega)$ where $\omega$ is a nontrivial third root of unity (a little working shows that it can also be written as $\mathbb{Q}(2^{1/3}, \sqrt{-3})$). However, as one expects of a degree 6 extension, the minimal polynomial is also of degree 6: $x^6 + 9x^4 - 4x^3 + 27x^2 + 36x + 31$. Is there any relation between this higher degree minimal polynomial and the third degree polynomial we started with, considering they both generate the same splitting field? This is meant to be a general question, the above is just an example.
EDIT: A clarification and generalisation of the question. Apologies for the earlier lack of clarity, I see that I made too many assumptions and such.
Given a field $K$ and a polynomial $f(x)\in K[x]$, supposing we can find a field extension $K(\alpha_1, ..., \alpha_n) \cong K[x]/(f)$, is there any relation between $f$ and the minimal polynomial of some linear combination of the $\alpha_i$ (that involves all $\alpha_i$.
The reason I ask this is that I would expect there to be something relating them as both the original polynomial and the resulting minimal polynomial will result in the aforementioned field extension.
Comment: You wrote : "the minimal polynomials is also..." . **What** "*the* minimal polynomial" are you talking about??
Comment: I'm sorry, I'm afraid I'm not sure what you're asking. Minimal polynomials are unique, are they not?
Comment: Would you like to explain us how you got the 6th degree polynomial?
Comment: @rwmak "Minimal polynomial" in fields extensions is *always* wrt some element. You give there a sixtic and call it "minimal polynomial": of **what** element and how is this related to the whole question?
Comment: Ah, apologies. I skipped a step there. It's the minimal polynomial of the element $\sqrt{-3} + 2^{1/3}$. I worked it out by setting $\beta = \sqrt{-3} + 2^{1/3}$ and taking exponents so only rational numbers were left as coefficients. So the question then is this: given a field $K$ and a function $f(x)\in K[x]$, supposing we can find a field extension $K(\alpha_1, ..., \alpha_n) \cong K[x]/(f)$, is there any relation between $f$ and the minimal polynomial of some linear combination of the $\alpha_i$ (that involves all $\alpha_i$.). I'll edit my main question to reflect this clarification.
Answer: If $K$ is a field of characteristic zero, if $\alpha_1,\dots,\alpha_n$ are algebraic over $K$, if there is no proper subset $S$ of $\{\,\alpha_1,\dots,\alpha_n\,\}$ such that $K(S)=K(\alpha_1,\dots,\alpha_n)$, then there is some linear combination $\alpha=\sum c_i\alpha_i$ such that $K(\alpha_1,\dots,\alpha_n)=K(\alpha)=K[x]/(f)$, where $f$ is the minimal polynomial for $\alpha$.
In the other direction, if you start with $f$ irreducible over $K$, then there will be lots of ways to find $\alpha_1,\dots,\alpha_n$ such that $K(\alpha_1,\dots,\alpha_n)=K[x]/(f)$, and there's no reason to think there's a linear combination $\alpha=\sum c_i\alpha_i$ such that the minimal polynomial of $\alpha$ is related to $f$ (although there's a certain amount of wiggle room here, since it's not entirely clear what you mean by "any relation").
Consider this example. Let $K$ be the rationals, let $f$ be the minimal polynomial for $\sqrt2+\sqrt6$, then $K(\sqrt2,\sqrt3)=K[x]/(f)$, but there is no rational linear combination of $\sqrt2$ and $\sqrt3$ with minimal polynomial $f$.
Comment: Sorry, 'any relation' is indeed vague, but I was really looking for an answer like yours that shows there is nothing 'intuitive' or 'obvious' linking them as such. Thank you for the concise answer.


tkinter python - place() a widget to the top-right corner

Question: I am trying to place() my widget on the top-right corner of the window
I can't use pack() or grid() here.. (due to some reasons)
Answer: Use relx and rely (not necessary for this case, could also use y) and set the anchor to 'ne' (north east - top right):
from tkinter import Tk, Label

root = Tk()

lbl = Label(root, text='Hello World')
lbl.place(relx=1, rely=0, anchor='ne')

root.mainloop()

It basically positions the top right corner of the widget to a relative (relative to its master) x of 1 which basically is the right side of the window and y to 0 which is top. Basically the top-right corner of the widget is placed in the top-right corner of the window


How do I pass Anti Forgery Token when doing CRUD items from jTable to MVC5?

Question: When doing CRUD calls from jTable to my MVC5 controller how do I pass along the AntiForgeryToken? In order to get the action to complete I have to comment out the ValidateAntiForgeryToken portion which seems rather unsafe. If not then I get a "An error occured while communicating with the server." message.
jTable code:
    $(document).ready(function () {

    //Prepare jtable plugin
    $('#CandidateTable').jtable({
        title: 'Candidates',
        actions: {
            listAction: '@Url.Action("List")',
            deleteAction: '@Url.Action("Delete")',
            updateAction: '@Url.Action("Edit")',
            createAction: '@Url.Action("Create")'
        },
        fields: {
            ID: {
                key: true,
                create: false,
                edit: false,
                list: false
            },
            FirstName: {
                title: '@Html.DisplayNameFor(model => model.FirstName)',
                width: '15%'
            },
            MiddleName: {
                title: '@Html.DisplayNameFor(model => model.MiddleName)',
                width: '15%'
            },
            LastName: {
                title: '@Html.DisplayNameFor(model => model.LastName)',
                width: '15%'
            },
            AnonymousID: {
                title: '@Html.DisplayNameFor(model => model.AnonymousID)',
                width: '15%'
            },
            Email: {
                title: '@Html.DisplayNameFor(model => model.Email)',
                width: '15%'
            },
            GUID: {
                title: '@Html.DisplayNameFor(model => model.GUID)',
                width: '15%',
                create: false,
                edit: false
            }
        }
    });

    //Load person list from server
    $('#CandidateTable').jtable('load');
});

ASP.NET MVC 5 Create Action:
    [HttpPost]
    [ValidateAntiForgeryToken]
    public JsonResult Create([Bind(Include = "FirstName,MiddleName,LastName,AnonymousID,Email")] Candidate candidate)
    {
        try
        {
            if (ModelState.IsValid)
            {
                candidate.GUID = System.Guid.NewGuid();
                candidate.IsActive = true;
                candidate.DateAdded = DateTime.Now.ToUniversalTime();
                candidate.DateModified = null;
                db.Candidates.Add(candidate);
                db.SaveChanges();

                return Json(new { Result = "OK", Record = candidate });
            }
            else
            {
                throw new Exception("Form is not valid! Please correct it and try again.");
            }
        }
        catch (Exception ex)
        {
            ModelState.AddModelError("", "Unable to save changes. Try again, and if the problem persists see your system administrator.");
            return Json(new { Result = "ERROR", Message = ex.Message });
        }
    }

UPDATE:
I figured it out. I needed to add an arbitrary field (AFT) and then add a custom input which @HTML.AntiForgeryToken outputs. Now i just need to figure out how to do this on Deletes since Delete only passes the primary key (id)
  $('#CandidateTable').jtable({
        paging: true,
        pageSize: 15,
        sorting: true,
        defaultSorting: 'LastName ASC',
        title: 'Candidates',
        actions: {
            listAction: '@Url.Action("List")',
            deleteAction: '@Url.Action("Delete")',
            updateAction: '@Url.Action("Edit")',
            createAction: '@Url.Action("Create")'
        },
        fields: {
            ID: {
                key: true,
                create: false,
                edit: false,
                list: false
            },
            FirstName: {
                title: '@Html.DisplayNameFor(model => model.FirstName)',
                width: '15%'
            },
            MiddleName: {
                title: '@Html.DisplayNameFor(model => model.MiddleName)',
                width: '15%'
            },
            LastName: {
                title: '@Html.DisplayNameFor(model => model.LastName)',
                width: '15%'
            },
            AnonymousID: {
                title: '@Html.DisplayNameFor(model => model.AnonymousID)',
                width: '15%'
            },
            Email: {
                title: '@Html.DisplayNameFor(model => model.Email)',
                width: '15%'
            },
            __AFT__: {
                create: true,
                edit: true,
                list: false,
                input: function (data) {
                    return '@Html.AntiForgeryToken()';
                }
            }

        }

Comment: The solution I use is typically adding a custom header to the request with the value of the anti forgery token, and then check it on the server. I'm unsure how to do this with jTable, however. If you want some guidance with a straight up AJAX example, let me know.
Comment: Oh very cool approach. My way did feel a bit "hacky." Thank you!
Comment: @kcabrams : please add update as an answer, worked for me
Comment: but yes, it did not work on delete
Answer: I have found how to add this AntiForgeryToken to delete action of jTable. You have added __AFT__ to the create and update actions correctly, no need to add headers and fetching in controller. For adding it to delete action you can write custom delete as below. Updated your code
var tokenId = '@Html.AntiForgeryToken()';

$('#CandidateTable').jtable({
    paging: true,
    pageSize: 15,
    sorting: true,
    defaultSorting: 'LastName ASC',
    title: 'Candidates',
    actions: {
        listAction: '@Url.Action("List")',
        deleteAction: function (postData) {
            postData.__RequestVerificationToken = $(tokenId).val();
            return $.Deferred(function ($dfd) {
                $.ajax({
                    url: '@Url.Action("Delete")',
                    type: 'POST',
                    dataType: 'json',
                    data: postData,
                    success: function (data) {
                        $dfd.resolve(data);
                    },
                    error: function () {
                        $dfd.reject();
                    }
                });
            });
        },
        updateAction: '@Url.Action("Edit")',
        createAction: '@Url.Action("Create")'
    },
    fields: {
        ID: {
            key: true,
            create: false,
            edit: false,
            list: false
        },
        FirstName: {
            title: '@Html.DisplayNameFor(model => model.FirstName)',
            width: '15%'
        },
        MiddleName: {
            title: '@Html.DisplayNameFor(model => model.MiddleName)',
            width: '15%'
        },
        LastName: {
            title: '@Html.DisplayNameFor(model => model.LastName)',
            width: '15%'
        },
        AnonymousID: {
            title: '@Html.DisplayNameFor(model => model.AnonymousID)',
            width: '15%'
        },
        Email: {
            title: '@Html.DisplayNameFor(model => model.Email)',
            width: '15%'
        },
        __AFT__: {
            create: true,
            edit: true,
            list: false,
            input: function (data) {
                return tokenId;
            }
        }

    }

I have tested and tried this myself. Hope this helps.
Answer: I had this code lying around, maybe it will help you. I'm guessing jtable uses $.ajax for it's requests so I guess it should work.
// Setup CSRF safety for AJAX:
$.ajaxPrefilter(function(options, originalOptions, jqXHR) {
    if (options.type.toUpperCase() === "POST") {
        // We need to add the verificationToken to all POSTs
        var token = $("input[name^=__RequestVerificationToken]").eq(0);
        var headers = {};
        headers["__RequestVerificationToken"] = token.val();
        if (!token.length) return;

        var tokenName = token.attr("name");

        // If the data is JSON, then we need to put the token in the QueryString:
        if (options.contentType.indexOf('application/json') === 0) {
           // Add the token to the URL, because we can't add it to the JSON data:
           options.headers = headers;
        }
   }
});

Answer: First thing to do is add a function to your View that will generate your anti-forgery token:
@functions{
    public string TokenHeaderValue()
    {
        string cookieToken, formToken;
        AntiForgery.GetTokens(null, out cookieToken, out formToken);
        return cookieToken + ":" + formToken;
    }
}

The @functions block lets you add functions to your Views. They're useful in keeping all your functions in one spot which helps with organization.
Then, in your Javascript you would make an AJAX call and add the token as a RequestVerificationToken header:
$.ajax("api/values", {
    type: "post",
    contentType: "application/json",
    data: {  }, // JSON data goes here
    dataType: "json",
    headers: {
        'RequestVerificationToken': '@TokenHeaderValue()' //Can be named whatever
    }
});

Now create the following utility method in your solution:
void ValidateRequestHeader(HttpRequestMessage request)
{
    string cookieToken = "";
    string formToken = "";

    IEnumerable<string> tokenHeaders;
    if (request.Headers.TryGetValues("RequestVerificationToken", out tokenHeaders))
    {
        string[] tokens = tokenHeaders.First().Split(':');
        if (tokens.Length == 2)
        {
            cookieToken = tokens[0].Trim();
            formToken = tokens[1].Trim();
        }
    }
    AntiForgery.Validate(cookieToken, formToken);
}

And call it like:
try{
   Utils.ValidateRequestHeader(request);
}
catch(HttpAntiForgeryException exc){
   //VALIDATION FAILED! RELEASE THE HOUNDS
}

ref: http://www.asp.net/web-api/overview/security/preventing-cross-site-request-forgery-(csrf)-attacks


remove uiwebview href grey selection

Question: i have searched the net, but the given solutions doesn't fit my needs..
So i have a webview, and my webview loads data like this:
NSString *path = [[NSBundle mainBundle] bundlePath];
NSURL *baseURL = [NSURL fileURLWithPath:path];
[_webView loadHTMLString:htmString baseURL:baseURL];

That html string has anchor tags with href e.g -  and i trigger some events on shouldStartLoadWithRequest.
Now the problem is i don't know how to remove the grey selection when this attribute is pressed. I saw people are doing this:

and some similar things, but i want to do it other way. By other way i'm talking that i don't want to change my htmlString, because it may have a lot of them, and injecting style in every seems not the perfect solution..
So i tried doing like this:
[_webView stringByEvaluatingJavaScriptFromString:@"document.documentElement.style.webkitTapHighlightColor = \"rgba(0,0,0,0);\""];

after i load html string, but it doesn't work.
So anybody has some suggestions, or knows what am i doing wrong?
EDIT:
Or maybe a simple injection of javascrypt for the behavior i want?
Thanks in advance!!!
Answer: This code should do the trick:
- (void)webViewDidFinishLoad:(UIWebView *)webView

    NSString *js = @"var styleNode = document.createElement('style');\n"
                    "styleNode.type = 'text/css';\n"
                    "var styleText = document.createTextNode('a {-webkit-tap-highlight-color:rgba(0,0,0,0)}');\n"
                    "styleNode.appendChild(styleText);\n"
                    "document.getElementsByTagName('head')[0].appendChild(styleNode);\n";

    [_webView stringByEvaluatingJavaScriptFromString:js];
}

Comment: Super!!! Exactly what i was looking for!! Thank you very much! I've edited your answer a bit and even disabled the touch callout :)) Wouldn't have done it without your help!
Comment: This is excellent advice. Thanks! I basically ripped the js in a separate js file that I've been using for my webview but this way is great too especially if you want the js snippet controlled in isolation.

remove uiwebview href grey selection


Is the sound of a semi-hollowbody guitar significantly different from that of an equivalent solid body?

Question: I'm wondering if the sound of a semi-hollowbody electric guitar (i.e. one that looks hollow but in fact has a solid central block) really differs that much from an equivalent solid-body guitar with the same pickups and hardware. I'll illustrate with a couple of examples.
At first glance, the Epiphone Sheraton II looks like a blues/jazz style of guitar, with its archtop shape and F-holes:

And the Epiphone Les Paul is a classic blues/rock/metal kind of instrument with its legendary solid-body sustain:

But on inspection I can't see what's really acoustically that different between the two guitars. The Sheraton's body isn't completely hollow: it has a solid block of mahogany in the centre (beneath the strings and bridge). The Les Paul has a completely solid mahogany body. Both have the same LockTone Stopbar/Tune-o-matic bridge, and both feature alnico Classic Humbucker pickups - all of which will have a significant influence on the guitar's sound.
The only obvious difference is the larger body on the Sheraton and the hollow wings with the F-holes. So my question is: do the hollow wings and F-holes on a semi-hollowbody guitar really make a noticeable difference to the sound, or are they mostly/only aesthetic?
Footnotes:
I understand there are many different Epiphone Les Paul models - some with Gibson rather than Epiphone pickups, for example - but if we keep the spec as similar as possible to the Sheraton (e.g. the Les Paul Standard model linked to above), would you really be able to distinguish the sound of the two guitars?
Please let's keep it objective! I'm not asking for opinions on which is better, or why you should really buy a Gibson/get a Mac/whatever. Just whether the two sound noticeably different.
Comment: A few reasons. 1) It's not too easy for me to get to a music shop. 2) Because then only I learn the answer, whereas this way many people do. 3) Because one person's judgment in one venue on one amp is likely to be subjective, whereas here we can (hopefully) crowdsource a more objective consensus. 4) Because StackExchange is all about asking interesting questions - what's the problem exactly? :)
Comment: Check out this closely-related question: http://music.stackexchange.com/q/2501/36
Comment: Go to a music store and play one of each and see if you can hear any difference. Why discuss in the abstract something that you haven't held in your hands and played and heard?
Comment: From my experience there is a slight difference in sound and tone but the guitars still sound very similar depending on amp and other equipment.
Answer: Any guitar tone is going to be affected by the materials that the guitar is constructed from and this will contribute to things like "sustain" and resonance.  I can't give you a direct answer between the two guitars you are showing but I do know that one of the features of a Les Paul is a solid mahogany body and a fixed (not bolt on) neck. Due to the heavy weight of the mahogany recent models have appeared that have been hollowed out to reduce weight. Purists stick with the solid models. Some basses and guitars have the neck going all the way through the guitar so that the string vibration will span the same piece of wood.
I am guessing here, but would venture to say that part of the tone of a hollow body is due to the internal acoustics and the way bridge is attached.  Many jazz musicians play hollow bodies and arch tops for the smoother tone it provides.
That said, a significant portion of the sound is from the guitar electronics (pickups and tone circuit) and much R&D has gone into the correct matching of these parts in various guitar models - Les Paul - Humbucking Pickups, 500K pots with specific capacitors (film, oil in paper, foil, etc...).  You can actually make an electric guitar from a 2x4 some tuners and pickup.
You could probably get more info from www.mylespaul.com in the Luthier forum.
Comment: Since electric-acoustic guitars are really susceptible to nasty howling feedback, I suspect that hollowbodies have different feedback characteristics to solidbodies when played with loud amps and lots of gain. But I don't know enough to write an answer of my own. Care to add something on this topic?
Answer: I don't have experience of the Sheraton, but in comparing a Les Paul with some semi-hollow body / semiacoustics I have played I would say the key features for me are:

The semiacoustics give a bigger 'ring' - and by that I mean a combination of long sustain and a sensitivity to feedback, so you get a much bigger feel to chords in a loud environment. That said, it can be too much - I tend to use no distortion, just a little overdrive when using one as it gets too muddy otherwise, where a Les Paul can cope quite happily with shedloads of distortion and stay clear.
For a nice round jazzy tone I would always go with the semiacoustic - for similar reasons to the previous one; a Les Paul can sound a bit thin with a clean amp but a semiacoustic through a clean Marshall can give a warm round tone with some lovely mids.
Answer: There is a difference in sound. This comes down to vibrational characteristics. The semi-hollow body has a slight acoustic factor which creates more vibration for a slightly longer duration. This will give one a "cleaner " sound quality as opposed to a solid body. A semi hollow will become "muddy" under heavy distortion as opposed to a solid body. Another factor is "natural harmonics" and "pinch harmonics". Semi acoustics give longer, brighter natural harmonics whereas solid bodies are better for pinch harmonics. Which is better comes down to ones playing style and taste. Truck vs car. Hopes this helps.
Answer: I have owned an Epiphone Dot which I traded for a Grestch Electromatic and I own a Les Paul. If you play them acoustically, you will hear the difference between the hollowbody and solid body. Hollow has that wider sound which is very neat.
Answer: Acoustically they will sound different.  I have a thinline Telecaster semihollow which is hollow on only one side.  If I strum a chord and then cover and uncover the f hole I can hear the sound of the ringing chord change.  Does this affect the sound of the guitar when it is plugged into an amp?  Hard to say, but it probably does have some effect.
Answer: I have just started to make custom electric guitars after my ex-wife kept buying me crappy guitars from HSN to make up for pawning my Gibson! I did a lot of research on wood and on the different shapes of the guitar. Here is what I know:
The most important factor in producing quality sound with an electric guitar is its weight and density, along with the luthier, or guitar maker. How it is made will determine if you can even tune your guitar!
For electric guitars, the heavier the wood, the brighter and cleaner the sound it will produce. For a bass guitar, it has to be made from a heavy dense wood, otherwise it will not play right (it may also difficult to keep it tuned).
Lighter woods make for a muddy sound. If you can find wood that is in the middle of light and very dense, that would be ideal. The in-between woods produce a fat, rich tone with good sustain.
One thing I noticed is that larger body guitars will have some of its wood carved out, which is, I suspect, the semi-hollow bodies accomplish. It also makes those guitars lighter in weight, but not too light.
The problem with the Randy Jackson and the Esteban's models from HSN, and I suspect the Keith Urban guitars (all made in China) the quality of wood used can be different and even suspect from one guitar to the next. I mean, who is going to remove some of the finish to see what wood was used?
My wife bought me an Esteban guitar after she made me pawn my Gibson and not only did it sound horrible, but I think the high "E" string was 20 gauge! Even after I filed the bridge down so the string at the 12th fret wasn't 1/4 inch from the fret board, it was hopeless.
This issue is really about physics. The strings vibrate to a resonate of the wood. This is why a 13 gauge (.013) string-set sounds good on one guitar, and on another with the same shape will need a 10 gauge string-set (either .010 or .011).
So what woods would I want to use to build a guitar you ask?
For the body, Maple provides good sustain and brighter tones. If I want a guitar to give warmer tones, Alder and Rosewood are good choices. Both are heavier and dense wood. Gibson and Hamer use Korina wood, which gives a well balanced warm tone, great clarity, definition and sustain.
Korina is a lighter hardwood that would be my first choice if I can get a block of it.
For the neck and fingerboard, I would want a wood that is hard and dense and strong. the only three I would use is either Maple, Bubinga or Ebony.
Comment: Oh, FYI: Fender used Alder througou the 60's in building all Strats. So that is why your guitar sounds twice as good and Alder is one of those inbetween woods I was talking about.

Hope that helps.
Comment: Not sure I'd agree on 'heavier wood, better sound' as a sweeping, global statement. Best sounding guitar I ever owned was a '64 Strat, weighed half what a modern one does, sounded twice as good.
Comment: That's because Fender used only Alder in building the 64 Strat, except for white blonde finished guitars and starburst patterned Strats. Fender used Northern Ash wood instead because the grain is better.

I left out Ash in my original post because the Southern Ash tree is different than its northern cousin. The Southern Ash is too soft and not as dense.

I didn't want to confuse anyone, because they are two separate trees.
Answer: Theoretically there should be a difference because acoustic volume (surface contact of the wave) of a guitar is inversely proportional to the sustain (dissipation of energy); also the frequency damping of different guitar must be different. Keeping both unplugged, if the semi-body produce a stronger acoustic that the solid body, it will in theory, gives proportionally weaker sustain.
You may tell the difference better between hollow and solid-body. Or, if you try Gibson instead of cheap Epiphone.
Comment: How predictable. Clearly you didn't read the last paragraph of the question. :(
Comment: huh? the hollow wings lowers the mass, give more surface area for the acoustic to be released through the F-holes. If you cover the F-hole or stuff up the hollow of course you will find a noticeable difference. the reverse is also true.


How can I alter the contents of a form before they are submitted?

Question: I'd like to do something like the following:
Let's say that I intended to allow users to input their Social Security Number, in the format 123-45-6789, which is a field in a "Person" content type. When saving to the DB though, I'd like to store this SSN as an integer. So, before the form gets saved, I want to remove the dashes. How can I do this? I can't find an appropriate hook that will allow me to alter the data BEFORE it is submitted. Adding a submit handler would not work, I believe, since this additional handler would trigger after the data is already saved.
EDIT: I'd now like to change this value before the individual element's #element_validate function is used. I tried changing the callback of this element's #element_validate function, but then the actual value isn't being changed before being submitted to the database, and I get an insertion error.
EDIT2: I'm attempting to use value_callback, but still having the same issue - either I'm using this incorrectly, or this is not firing before the element's validator. Either way, the result is that rather than attempting to insert "123456789" into the database, "123-45-6789" is attempted, and since this is an integer field, insertion fails.
Answer: You may be interested in form_set_value().  You would use this within the form validation function.  There are examples of its use posted below the API documentation in the comments if you need further clarification.
For help creating a validation function, see the Form API Quickstart Guide (D6) or How to use the Form API (D7).
Comment: This was a great start! I'm now running into one additional problem - the data that I need to change is in a format that would be invalid if left unchanged. So, to use the SSN example again - if I want to store "123-45-6789" as an integer, I must remove all of the dashes. BUT, before my function which uses form_set_value is called, the element's validator triggers, and sees that "123-45-6789" is not an integer. How can I get around this?
Comment: You can use a value callback, see my answer
Answer: Technically, to alter it before it is submitted, you'd need to do that in the client with JavaScript. However, what you are looking for I think is altering it before it is saved.
For nodes, you can implement hook_node_presave($node) where you can change any properties of the node object right before it is saved.
Form fields also support a #value_callback, which allows you to alter the actually submitted value and return something else/changed.
Comment: Any chance you could give me an example of it's usage? I'm attempting to use it, and I'm still having the issue that the element's validation is either not taking the value from this value callback, or it's firing before the value callback. I'll post the updated code in my question.


PHP eval and heredoc don't play nicely

Question:
Possible Duplicate:
heredoc with eval code execution

So I have the following in function.php:
eval("\$content = <<<TEMPLATE\n
               asdf
\nTEMPLATE;");

And I keep getting an error saying:
Parse error: syntax error, unexpected $end, expecting T_VARIABLE or T_END_HEREDOC or T_DOLLAR_OPEN_CURLY_BRACES or T_CURLY_OPEN in /var/www/function.php(10) : eval()'d code on line 5

I cannot figure out what the problem is. There is obviously an ending for the heredoc syntax, does heredoc just not like to play nice with eval?
Comment: Why would you do such a thing?
Comment: What are you trying to accomplish? This looks like all kinds of bad. See: http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem
Comment: I agree with @NullUserException. Please explain what you are trying to do so we can properly help you / prevent you from doing "stupid" things. `eval()` is considered `evil`.
Comment: @legion Don't do that. Just because someone else is doing it doesn't mean they're doing it right. In general, I would avoid using templating engines in PHP, period - because PHP itself *is* a templating engine. There *is* a better solution, it's called [MVC](http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller). Look at how an MVC framework handles views (templates): http://codeigniter.com/user_guide/general/views.html. You get (somewhat) clean HTML, without losing performance or flexibility.
Comment: Even with TEMPLATE; on its own line it doesn't work.
Comment: @NullUserException I thought that too, but wouldn't the \n be interpreted as a newline *before* the string is evalled at all?
Comment: @GolezTrol Hmmm, it makes sense. I didn't catch the quotes.
Comment: I wouldn't do this either, but it makes you wonder why it isn't possible. I think it would be more challenging to answer the question than to save this user from doing stupid things. We have made our points. :0
Comment: Very interesting theoretical question. But I see no actual uses in practice. What are you trying to do? I'm 99% sure there's a better solution.
Comment: HTML templates where you can use php variables, I know MyBB uses it as do many other packages that use templating.
Comment: I'm not too worried about it I'm just doing it for a class so that my partner who doesn't know PHP can just create the HTML/CSS files elsewhere and I don't have to worry about integrating the PHP in those files. I would normally just use php directly in the template itself but its easier for him this way
Answer: HEREDOC syntax is ended by the delimiter defined at the start, followed by a semicolon, followed by a newline. You do not have the newline, therefore it is not being recognised as the end of the HEREDOC. Add an extra \n after TEMPLATE; and it should work fine.
Comment: Thank you that was it and I feel stupid now, I kept thinking it was before hand.

PHP eval and heredoc don't play nicely


if-else statement in Ef along with MVC

Question: I have three fields that I want to base on them filter my result.
I used an if-else statement, but it is very awkward, and base on my fields, I had to write some if blocks statements.
Is there an easier way to do this?
public ActionResult Search(String title, int? category_id, string productCode)
{
    IEnumerable<products> result;
    using (DBModels db = new DBModels())
    {
        if (title != null && category_id == null && productCode == null)
        {
            result = db.products.Where(c => c.title.Contains(title))
                                .Include(x => x.media)
                                .Include(p => p.categories)
                                .ToList();

            return View(result);
        }
        else if (category_id != null && title == null && productCode == null)
        {
            result = db.products.Where(c => c.category_id == category_id)
                                .Include(x => x.media)
                                .Include(p => p.categories)
                                .ToList();

            return View(result);
        }
        else if (productCode != null && title == null && category_id == null)
        {
            result = db.products.Where(c => c.id.ToString().Contains(productCode))
                                .Include(x => x.media)
                                .Include(p => p.categories)
                                .ToList();

            return View(result);
        }
        else if (productCode != null && title != null && category_id != null)
        {
            result = db.products.Where(c => c.title.Contains(title)
                                         && c.category_id == category_id
                                         && c.id.ToString().Contains(productCode))
                                .Include(x => x.media)
                                .Include(p => p.categories)
                                .ToList();
        }
    }

    result = db.products.Where(c => c.title.Contains(title)
                                 || c.category_id == category_id
                                 || c.id.ToString().Contains(productCode))
                        .Include(x => x.media)
                        .Include(p => p.categories)
                        .ToList();

    return View(result);
}

Answer: You can use IQueryable
public ActionResult Search(string title, int? category_id, string productCode)
{
    IEnumerable<products> result;
    using (DBModels db = new DBModels())
    {
        var query = db.products
                      .Include(x => x.media)
                      .Include(p => p.categories)
                      .AsQueryable();

        if (!string.IsNullOrWhiteSpace(title))
        {
            query = query.Where(c => c.title.Contains(title));
        }

        if ( category_id != null)
        {
            query = query.Where(c => c.category_id == category_id);
        }

        if (!string.IsNullOrWhiteSpace(productCode))
        {
            query = query.Where(c => c.id.ToString().Contains(productCode));
        }

        result = query.ToList();
        return View(result);
    }
}

if-else statement in Ef along with MVC


How to determine the code for settings.php config override based on .yml?

Question: I am trying to put my API keys in a separate file instead of storing them in config management.
For example, I installed the Commerce Stripe module.
When I export the config, it looks like this:
commerce_payment.comerce_payment_gateway.stripe_test.yml
uuid: 430a9544-c852-4a14-b7b0-3a833a02dd06
langcode: en
status: true
dependencies:
  module:
    - commerce_stripe
id: stripe_test
label: 'Stripe Test'
weight: null
plugin: stripe
configuration:
  publishable_key: thisishidden
  secret_key: thisishidden
  display_label: Stripe
  mode: test
  payment_method_types:
    - credit_card
conditions: {  }
conditionOperator: AND

Now, I want to store publishable_key and secret_key in a file, but how do I determine the "path" to call them?
$config['whatgoeshere?']['configuration']['secret_key'] = 'abcdefg';

Answer: The first key is the name of the config object/file, so... commerce_payment.comerce_payment_gateway.stripe_test.
The rest you already figured out, it's an array that's recursively merged into the other one.
Comment: Actually its more like ```$config['system.performance']['js']['preprocess'] = FALSE;``` ... just to be clear about the structure. So the root(fist key) is the config name(yml file name) and the rest is structure as set in the config(yml) file itself.
Comment: true, I was answering specifically to the "whatgoeshere?" part in the question, there already is example with just that part missing
Comment: This is not available in any documentation anywhere that I was able to find! (With regards to how to reference the nested yaml items)
Comment: @NikLP there are several examples in settings.php, as a start.
Comment: You're correct (naturally). I had not been able to see this clearly enough to formulate a correlation between the nested yamls and the array syntax. We might do well to add a small entry in the docs on config overrides outlining this specific syntax.


Output text with hyperlinks

Question: If you have text coming from a database such as:
 "New Apple TV Offers 8 GB of Internal Storage, 256 MB RAM http://t.co/fQ7rquF"

Is there a helper method that takes that text and wraps the web address as a anchor tag?
Answer: Off the top of my head I would run a regular expression to match a link pattern and throw the anchor tags around it.
Found this on google:
using System.Text.RegularExpressions;

public string ConvertURLsToHyperlinks(string sInput)
{
    return Regex.Replace(sInput, @"(\bhttp://[^ ]+\b)", @"$0");
}

It looks for "http://" followed by anything that's not a space, separated by word boundaries.
Comment: This will work in the example stated in the question, but many URLs in the real world end in a slash (`/`) or other punctuation characters and your regex wouldn't match that.
Comment: This didnt work actually. I copied the RegEx from the link Kevin gave and that worked - Regex.Replace(sInput, @"(\bhttp://[^ ]+\b)", @"$0");
Comment: @Timwi - how can we modify the Regex to fix it?
Comment: @Timwi - it seems fine with a slash on the end
Comment: @Jon: No, it doesn't. It doesn't linkify the slash. (And other punctuation characters.)
Comment: Well spotted. How can we fix this?
Comment: adding the literal `"` to the `[^ ]` group should take care of that.
Answer: i believe there is no such helper method but u can create one
public static class helper{
public static string AnchorHelper(this htmlHelper helper, string text)
{
   //here u can use kevin's function to generate anchors
   return ConvertURLsToHyperlinks(text);
}
}

you have to add corresponding namespace in ur view and then u can just use it like other html helpers
<%=Html.AnchorHelper(TextwithUrls)%>

Comment: The class needs to be marked `static` for extension methods to be permitted in it.

Output text with hyperlinks


Connection with my db using pgAdmin4

Question: I am study with this tutorial: https://www.youtube.com/watch?v=Ho3DKKopC2E&list=PLDmvslp_VR0xlwr5lAx2PDsZLu7oIOhpX&index=20
When I connect to db, I want to connect the db like the turorial:

But I failed, because I don't know how to do with it:

Does anyone knows how to do with it?
Answer: Add new server steps:

Open pgAdmin4

Right click in "Servers" in Browser panel

Enter some meaningful name for Server eg: "PostgreSQL9.6"

Click on "Connection" tab

Enter "Host" (default is "localhost" if database is installed on your own local system otherwise IP address of system where PostgreSQL database is installed)

Enter "Port" (default port for PostgreSQL server is "5432")

Enter "Maintenance Database" name (default is "postgres")

Enter "User Name" (default user is "postgres")

Enter "Password" (Password which you gave when you installed PostgreSQL database for "postgres" user)

Click on CheckBox "Save password?" to save your password in pgAdmin4 for    future use.

Now as you can see, "Save" button gets enable in dialog, just click on it.
You will see that your server is now listed under "Servers" and you should be connected to your database.
Answer: You are getting error because You did not specified Name for server, Click on "General" tab provide name for your server like "PostgreSQL9.6"
Refer the pgAdmin4 documents:
Connecting to server:
https://www.pgadmin.org/docs4/dev/connecting.html
Creating new server:
https://www.pgadmin.org/docs4/dev/server_dialog.html
While creating new Server you should have all the required parameters like,
<code>Host (default is "localhost" if database is installed on local system), Port(default is "5432"), Maintenance Database Name (default is "postgres"), User Name (default is "postgres"), Password
</code>
Without above required information you can't connect to PostgreSQL database server.
Comment: But I get this issue: http://img1.ph.126.net/fhmXJbIaKSaaSFRW8bP5Gw==/6632074918351502842.png, could you give me a details steps?
Answer: localhost did no work for me, but my server ip (192.168.xx.x) address worked. The x needs to be replace with you ip address.
Comment: If localhost doesn't work, you're probably missing it in your hosts file.  But the inability to resolve 'localhost' can potentially mean you've messed your name resolution up.  Good luck!
Answer: If you created a database from the docker, use the 'docker inspect ID_IMAGE' to see the IP address. This IP address will be put in pgadmin in the connection tab in the first input "host name/address".
If you mirrored the default postgress port on docker-compose.yml for 5432 like
ports: 5432:5432
the IP address will be input "host name/address" and will only receive this value '0.0.0.0'
Comment: It worked for me using "docker inspect CONTAINER_ID", then the IPAddress was under NetworkSettings.IPAddress

Connection with my db using pgAdmin4


How do I pick a good value of num_epochs?

Question: I'm running a program that uses an RNN to build a language model of some sample text: https://github.com/crazydonkey200/tensorflow-char-rnn When training it, I currently use the default value of num_epochs (50). Passing --verbose 1 to the program, I notice that the perplexity seems to stop decreasing at around 25 or 30 epochs, and since perplexity is just exp(train_loss) this means that the training loss is stabilizing as well.
Is it a good idea to limit the number of epochs to the point where the training loss stops decreasing significantly? I assume that means the model isn't getting any better from extra training, right?
Answer: There are a few schools of thought on this topic.

Continue training until training loss stops decreasing, then stop. This can be as simple as squinting at a graph and making a judgement call, or as sophisticated as statistically measuring whether loss is decreasing or not.
Continue training until loss stops decreasing (as in 1), then reduce the learning rate (say, by cutting it in half). The logic here is that some models do better with larger learning rates early on, but smaller learning rates later.
Use some function, like a very slow exponential decay, or a monotonic decreasing sigmoidal function, to slightly decrease the learning rate at each iteration.
Super-convergence evangelists have their own, ornate methods.


Inserting random characters to MYSQL Database

Question: I want to add 100 entry to users table numbers field, random characters length is 10, all entry should be unique as well. How can i achieve this using MYSQL query code ?
Or do i need to use PHP ?
Help me with code snippets please. Thanks.
Comment: you want to update exist rows ?  or insert new rows ?
Answer: in mysql u can do like :
insert into table ( SUBSTRING(MD5(RAND()) FROM 1 FOR 10) , field2 , field3) , ( SUBSTRING(MD5(RAND()) FROM 1 FOR 10) , field2 , field3) , .........

..............
in php see this 2 links :
Short unique id in php
What is the best way to generate a random key within PHP?
Comment: hello, i want to add 100 entry once. How can i achieve ? I have to use insert instead of update right ?
Comment: hi haim, how to insert 100 rows by just one query ?
Comment: i show the example in my answer with comma u can separated 100 insert
Answer: That may create duplicates and it's too long. This is 10 char long:
UPDATE users SET numbers = SUBSTRING(MD5(RAND()) FROM 1 FOR 10)

However, you could still get duplicate values.
So you could put a unique restraint on your column. Then try the update. If there are duplicates, you will get an error. Then just try again. You only have 100 entries, so it's probably fine.
Is this for passwords?
If so, I'd recommend encrypting the value. Of course you then have the problem of not knowing what the value is. So you could create a temporary table, insert the random values in there. Then encrypt the values as they are inserted from the temp table into the real table. You can then use the temp table for reference (giving the users their passwords, etc). Hope that helps.
Comment: I do realize this answer is 6 years old, but still storing the plain text password even if in a separate "temp" table/column is just defeating the purpose of hashing and is a security hole. Passwords should be **irretrievable** whatsoever. And about "*giving the users their passwords*" -- Passwords should never be "given", and rather they should be entered by the end user.
Answer: UPDATE TABLE users SET number_field = MD5(RAND());

Comment: If rand returns the same number twice, wouldn't that create a violation of the uniqueness constraint?
Comment: will only work in mysql, in other DBMS you would need to truncate the md5
Comment: hi, i want to add 100 entry by once ? How can i ?
Comment: Hi @sum1stolemyname you are right. my mistake. A function counting from 1 to 100 will be better. But anyway just wanted to point a direction for solution.
Comment: @spotlightsnap: are you talking about 'entry' (which I read as add to field for one user or 'row', as in : add 100 users with a 10-char number_field for each???
Comment: hi lexu, it's like this, i have table name called users, at users table, there's field name called numbers, i want to generate 100 row and insert into that users table numbers field. Currently the above code generate only 1 row at a time. thanks
Answer: You can't generate an unique random number. Over time, the randomness will generate a number already stored. You need to make a "quasi-random" number, meaning that it's a number based on another data, but it just looks random. You can use the primary key on the table as the base number to generate the "fake-random" number
INSERT INTO myTable(primaryKey,quasiRandom)
    SELECT IFNULL(MAX(primaryKey),0)+1, CAST(CONCAT(IFNULL(MAX(primaryKey),0)+1,CHAR(FLOOR(RAND()*26)+65),FLOOR(100+RAND()*(500-100)))
 AS CHAR(10)) AS quasiRandom FROM myTable


Confused About Rudin's Opening Statements on his Chapter on Distributions

Question: I have a few doubts regarding Rudin's opening paragraphs in Chapter 6 of Functional Analysis. The passage (with small edits) reads:

A complex function $f$ is said to be locally integrable if $f$ is measurable
and $\int_K |f| < \infty$ for every compact $K \subseteq R$. We let $\mathcal{D} = \mathcal{D}(\mathbb{R})$ be the vector space of all $\phi\in C^\infty(\mathbb{R})$ whose support is compact. Then

$\int f\phi$ exists for every locally integrable $f$ and for every $\phi\in\mathcal{D}$.
$\mathcal{D}$ is sufficiently large to assure that $f$ is determined (a.e.) by the integrals $\int \phi f$. (To see this, note that the uniform closure of $\mathcal{D}$ contains every continuous function with compact support.)
If $f$ happens to be continuously differentiable, then
$$\int f'\phi = -\int f\phi' \ \ \ \ (\phi\in\mathcal{D}).$$
If $f\in C^\infty(\mathbb{R})$, then
$$\int f^{(k)}\phi = (-1)^k\int f\phi^{(k)} \ \ \ \ (\phi\in\mathcal{D}).$$
The compactness of the support of $\phi$ was used in these integrations by parts.

Regarding 2: while it is true that the uniform closure of $\mathcal{D}$ contains all continuous functions with compact support, how does that imply $f$ is determined by the $\int \phi f$?
Regarding 3: it seems to me that the actual equation should be
$$\int f'\phi = f\phi -\int f\phi' \ \ \ \ (\phi\in\mathcal{D}).$$
Why does Rudin omit the $f\phi$ term?
Regarding 4: again, it seems to me the actual equation should be
$$\int f^{(k)}\phi = \sum_{i=0}^{k-1}(-1)^if^{(i)}\phi^{(k-i-1)} + (-1)^k\int f\phi^{(k)}.$$
Why does Rudin omit the second term?
Lastly: how is compactness of the support of $\phi$ used?
Comment: The integrals here are definite integrals (from $-\infty$ to $+\infty$). That should explain 3) and 4).
Comment: You are asking too many questions simultaneously. When you ask a question you should wait for an answer, study the answer and then ask the next one.
Comment: @geetha290krm Oh of course. That does solve the issue!
Answer: The second remark ($\mathcal{D}$ being sufficiently large to determine $f$ a.e. through its various integrals) alludes to the following fact: if $f$ and $g$ are locally integrable with respect to a $\sigma-$finite Borel measure $\mu$ of $\mathbb{R}$ and
$$\int_\mathbb{R} f\phi \, d\mu = \int_\mathbb{R} g\phi \, d\mu \qquad \text{ for all } \phi \in \mathcal{D},$$
then $f = g$ a.e.[$\mu$].
To prove the above statement, it's enough to show that if
$$\int_\mathbb{R} f\phi \, d\mu = 0 \qquad \text{ for all } \phi \in \mathcal{D}$$
then $f = 0$ a.e.[$\mu$]. Suppose there is an open interval $A \subset \mathbb{R}$ such that $f$ is positive a.e.[$\mu$] on $A.$ Take $K \subset A$ a (nondegenerate) proper compact subinterval (so that $\mu(A \smallsetminus K) > 0$) and pick a nonnegative $\phi \in \mathcal{D}$ such that $\operatorname{supp} \phi \subset A$ and $\phi = 1$ on $K$ (you can always construct these smooth bump functions using smooth cutoff functions). Then
$$0 = \int_\mathbb{R} f\phi \, d\mu = \int_A f\phi \, d\mu = \int_K f\phi \, d\mu + \int_{A \smallsetminus K} f\phi \, d\mu \, > \, \int_K f \, d\mu.$$
This contradicts the positivity a.e.[$\mu$] of $f.$ A similar argument works for $f$ negative a.e.[$\mu$], and thus we conclude that $f$ must be zero a.e.[$\mu$] on any open interval, and also on $\mathbb{R}.$
The third and fourth remarks are explained by noting that the boundary terms in the integration-by-parts formula always vanish since every $\phi \in \mathcal{D}$ has compact support.
Lastly, you want the functions $\phi \in \mathcal{D}$ to have compact support so the integrals $\int_\mathbb{R} f\phi$ are all finite for every locally integrable $f$ (and because of the nice "topological/real analysis" properties that you get when you work with these function spaces).
Comment: I'm a little confused on the following implication. "$f$ is not positive a.e. or negative a.e. on any open interval" $\implies$ "$f$ is zero a.e. on every open interval." Could you expand on this step?
Comment: What if for every open interval, $f$ is positive over some set of nonzero measure, and also zero or negative over some set of nonzero measure? Is this impossible?
Comment: Actually, now I think the implication is false. I asked [this](https://math.stackexchange.com/questions/4763163/can-mathbb-r-be-partitioned-into-two-sets-with-positive-measure-on-every-inte) question, which was motivated by my comments above, and people have pointed to the existence of functions which contradict this implication.


sql query related

Question: Hi I have an "Student" table with below records
INSERT into Student(studId,name)
Values(1,A)
values(2,B)
values(3,C)
values(4,D)

I have a "Department" table with below records
INSERT into dept(deptId,Deptname,Emp Id)
Values(D1,Phy,1)
values(D2,Maths,2)
values(D3,Geo,3)

How can i find the student who does not belong to any department i.e. in this case the result should be "D".
I know the left outer join would return all the records from the student table but i am only interested to get 1 record i.e.: of student "D".
Comment: This is not a forum to ask people to do your homework.
Comment: Look into something like "where dept.deptid is null"
Comment: Any chance you could improve your subject? Any question about SQL is "sql query related"
Answer: SELECT  *
FROM    Student
WHERE   studId NOT IN (SELECT EmpId FROM dept)

Find all Student.studId entries that do not already exist in the dept.EmptId column as an entry. (Assuming I read your table correctly)

Ideally though, you should probably break out the "Student" and "Department" and creating a joining table (maybe called) "Student_Department" that links the keys of each table.
+--------------+       +--------------------+        +--------------+
| Student      |       | Student_Department |        | Dept         |
|--------------|       |--------------------|        |--------------|
| studId       | <-----| studId             |    .-> | deptId       |
| name         |       | deptId             | --'    | name         |
| ...          |       +--------------------+        | ...          |
+--------------+                                     +--------------+

This allows you to only have to define a student and department once, but can assign one student to multiple departments, one department to multiple students, or any combination therein.
Comment: Hey. How did you make that textualized (if that's word) diagram? [Related](http://stackoverflow.com/q/16479425/1309352).
Comment: @BleedingFingers: Index and middle fingers (hunt'n'peck) ;-)
Answer: Pseudo-code:
select *
from students
where id not in(select a.id from students a
                inner join department b where b.id in('D1','D2','D3'))

Comment: Aaron; Thanks for the response but the inner query what you have mentioned will be ok if the department ids are limited but what if the department ids are too many, in that case will the below query work?"select * from students where id not in(select a.id from students a inner join department b where b.id in(select distinct deptID from dept))"
Comment: You need to know what you don't want to return.  Therefore, if you know the dept you don't want to return just change the `IN` in the subquery to `NOT IN` the dept id you don't want.
Answer: While I agree with some commenters that this sounds like a homework problem, I'll answer with a question...

I know the left outer join would return all the records from the student table but i am only interested to get 1 record i.e.: of student "D".

OK, let's say you run the following query:
SELECT * FROM Student
LEFT OUTER JOIN dept ON Student.studId = dept.EmpId

You'd get the results:
studId    name    deptId    deptName   EmpId
1         A       D1        Phy        1
2         B       D2        Maths      2
3         C       D3        Geo        3
4         D       NULL      NULL       NULL

Can you add a WHERE clause to this query that will filter out only the data you want? :)
Comment: Oh well.  Hopefully reading through these answers and comments will still do anonymous some good.
Comment: For all those who think this is a homework question, let me clarify its not so. every trivial query has its own importance. here i am trying to understand different cases of join, hence thought of getting confirmed with some experienced guys in this forum. Also I dont have a sql query analyzer handy dut to which i am posting these queries..also pls see my above comments in which I have mentioned the sql query from my end too.
Answer: SELECT s.name
FROM Student s
LEFT JOIN Dept d ON d.empId = s.studId
WHERE d.empId IS NULL

Comment: I realize that this is a old post. But could you please elaborate.

sql query related


Regex to extract minute and seconds

Question: How can I extract minutes and seconds from a NSString? Examples:
3'15" or 3' 15" (3 minutes 15 seconds)
28" (28 seconds)
2' (2 minutes)
50 (default is in seconds, 50 seconds)

and stores into two NSIntegers?
Comment: `(\d+')|(\d+[^'])` The second var will always be the seconds and the first will be the minutes
Answer: Not entirely convinced that RegEx matching is the best way to read numbers out of a string, but here is a snippet that works:
NSString *str = @"12'34\"";

NSString *minutePattern = @"[[:digit:]]+'";
NSString *secondPattern = @"[[:digit:]]+\"?";

NSRange minuteRange = [str rangeOfString: minutePattern options: NSRegularExpressionSearch];

NSRange  secondStartRange;
NSString *minuteString = @"";
if ( minuteRange.location != NSNotFound)
{
    minuteString = [str substringWithRange: minuteRange];
    NSUInteger secondStartPt = NSMaxRange(minuteRange);
    secondStartRange = NSMakeRange(secondStartPt, str.length - secondStartPt);
}
else {
    secondStartRange = NSMakeRange(0, str.length);
}

NSRange secondRange = [str rangeOfString: secondPattern
                                 options: NSRegularExpressionSearch
                                   range: secondStartRange];

NSString *secondString = @"";
if ( secondRange.location != NSNotFound)
{
    secondString = [str substringWithRange: secondRange];
}

NSInteger minutes = [minuteString integerValue];
NSInteger seconds = [secondString integerValue];

NSLog(@"Minutes: %d, seconds: %d", minutes, seconds);

Note that it will parse a string like 5'21 as 5 minutes 21 seconds, not sure if that is what you want, but that can be fixed with an extra test.
Answer: NSArray *data = @[@"3'15\"", @"3' 15\"", @"28\"", @"2'", @"50"];
NSString *pattern = @"(\\d++')|(\\d++(\"{0,1})$)";

NSRegularExpression *regExp = [NSRegularExpression regularExpressionWithPattern:pattern
                                                                        options:NSRegularExpressionCaseInsensitive
                                                                          error:nil];
for (NSString *string in data) {
    NSInteger mi = 0;
    NSInteger ss = 0;

    NSArray *match = [regExp matchesInString:string options:0 range:NSMakeRange(0,string.length)];
    for (NSTextCheckingResult *result in match) {

        NSString *value = [string substringWithRange:result.range];
        if ([value hasSuffix:@"'"]) {
            mi = [value integerValue];
        } else {
            ss = [value integerValue];
        }
    }
    NSLog(@"%@ - %d %d",string,mi,ss);
}

Comment: It is always preverable to not only post code but some kind of explanation of it as well

Regex to extract minute and seconds


Where do I find Date class in Zend Framework 2.0?

Question: I have started my ZF2 application from Skeleton-Application.
I want instance of Date class in one of my controller.
I tried to search Date.php class under...

library\Zend\

but could not found.
Does ZF2 has not Date.php at all or I need to palce it from somewhere else?
Answer: Zend_Date has been removed in favor of PHP 5.3 DateTime API http://es.php.net/manual/en/book.datetime.php
Comment: Thank you Maks3W for valuable information. As of now i will proceed with writing my own Plugin to wrap Date utilities.
Comment: Hanmant Please, when you finish send a Pull Request to the ZF2 GitHub repository to propose include your code with the framework
Comment: Thank you Maks3w. But how can i then use Date utilities in controllers? I mean which plugin or class should i intantiate for Date utilities?
Comment: I don't understand, Which Date utilities? almost all features of Zend_Date are provided by PHP 5.3
Comment: Maks3w, let say for example, in ZF 1.x it has wrapper class 'Zend_Date' (derived from Zend_Date_DateObject).
 This class has friedly methods to operate on Dates like...
 Zend_Date::getUnixTimestamp
 Zend_Date::mktime
 Zend_Date::isYearLeapYear
 Zend_Date::dayOfWeek
 etc...
 My question is 'Does ZF2 has a wrapper class as 'Zend_Date' in ZF1.x? If not then do i need to use plain PHP Date statements throughout the code or need to write my own wrapper?'
Comment: No, the decision of ZF2 board was remove Zend_Date and don't supply any wrapper or other kind of friendly use. Personally I prefer to have a wrapper because, IMHO, PHP 5.3 DateTime API is very ugly and cryptographic
Comment: Hi again Maks3w, I just completed with my DateTime plugin for ZF2 [named as DateManager]. Actually I was needed it because my application is mostly oriented around date, time and timezone utilities. Written plugin is not a state-of-art one but it is good :)
Answer: Don't forget to call DateTime() in Zend Framework 2 like base class.
Like:
<code>$date = new \DateTime($str);
</code>
where <code>$str</code> is a string from which you create date object.


What is the highest version of Bobo-Browse that is compatible with Lucene 3.0.3?

Question: ...or maybe this question could be shortened to - are Bobo-Browse and Lucene using semantic versioning?
I am contributing NuGet build script to a .NET port of the Java Bobo-Browse faceted search engine, which has been upgraded to be compatible with Lucene.Net 3.0.3 (the latest version currently available). However, I don't know what version of Bobo-Browse this is technically a port of. The source files don't resemble the Java versions very much.
I went through the Java source files and noticed that the version numbers don't line up between Bobo-Browse and Lucene. That is, Bobo-Browse 3.0.0 depends on Lucene 3.5.0. I tried going back further, but I can't seem to find any version numbers in the source. I then tried doing a Google search and found 2 documents here and here that both agree that version 2.5.0 of Bobo-Browse is compatible with Lucene 3.0.3.
My question is: Does that mean version 2.5.1 of Bobo-Browse is also compatible with Lucene 3.0.3? Would Bobo-Browse 3.0.0 or 3.0.1 or another higher version be compatible with Lucene 3.0.3?
Some Context
Actually, the port was originally done on the Lucene.Net 2.9.4 version by one developer (but given the mysterious version number 126.96.36.199 in the source files, and never officially "released") and then upgraded to Lucene.Net 3.0.3 by another developer, and there is no information about what version number this is supposed to be (and believe me, I have asked). So, this question is really to help get to the bottom of 2 different issues:

What version number should we give this release Bobo-Browse.Net?
What version of Bobo-Browse should we be porting over the unit tests from in order to better confirm that it works like that version?

The thing is - the port seems to work fine. But it doesn't seem right to either guess what version number this is supposed to be or stick 3.0.3 on it just because that is the version of Lucene.Net it is compatible with. It really should line up with the Bobo-Browse version in some logical way, but I need more information to be sure of what we are doing.
Answer: After quite a bit of research I concluded that:

Lucene and Bobo-Browse are using semantic versioning.
The major version of Bobo-Browse normally aligns with the major version of Lucene.
The highest version of Bobo-Browse that is compatible with Lucene 3.0.3 is Bobo-Browse 3.2.0.


NotePad++ Changing Few Number Entries

Question: Here is a simple lists where I like to change the numbers: the entries are as below and it got over 300 entries like it
tom112
smith113
harry114
linda115
cindy106
samantha147

It need to be changed to
tom212
smith213
harry214

...and so on.
Please assist using notepad++ regular expression.
Thanks.
Comment: What would you like to replace 'tom917' with? Or will it always be 'name1xx'
Comment: Is this even the right tag for Notepad++?
Answer: Assuming it's just a matter of replacing a name followed by a number starting with 1 with the same number but starting with 2 instead:

Ctrl + H for search & replace.
Check Regular expression under Search Mode.
Next to Find what type or copy in ([a-zA-Z]+)1([0-9]+).
Next to Replace with type or copy in \12\2.
Click Replace All and that should do it.

Add any other characters that might appear in the name before the number inside the first set of brackets with a-zA-Z.

NotePad++ Changing Few Number Entries


Beethoven and parallel minor keys

Question: I read somewhere recently that Beethoven considered the minor and major keys to be "one and the same"
The quote i reference was not referring to the relative minor (ie. Am in key of C), but was referring to the parallel minor (Cm in key of C).
Could anyone elucidate further on what, if anything, the original author meant by this?
Thanks
Edit:
Here is the original source, it's at 10:37: https://youtube.com/clip/Ugkx5etB0VIZa5NHXxqxgp9N4ul74H7fF3Bm
Comment: "I read somewhere recently…" If you can provide a quote why not provide the source as well?
Comment: Not only Beethoven. I think a lot of musos consider the same. I did when writing stuff 50 yrs ago, not even understanding the 'theory' behind it.
Comment: I doubt this claim when Wikipedia maintains an article on [Beethoven and C minor](https://en.wikipedia.org/wiki/Beethoven_and_C_minor) and does not do the same for Beethoven and C major.
Comment: Could you please add more about the context, or identify or help us find the source? I could imagine someone meaning *something* by this phrase—maybe about modal mixture—but it would just be an unfounded guess without more explanation.
Comment: Remembering a quote without remembering who it is attributed to is a common occurence for me! - Here it is at 10:37 -  https://youtube.com/clip/Ugkx5etB0VIZa5NHXxqxgp9N4ul74H7fF3Bm
Comment: Thanks! I'll let someone else explain better, but I heard this referred to as "modal ambiguity." I heard about it more when looking at Brahms, but I guess Beethoven was leading towards that: The idea of "I'm in C." "Major or minor?" "Does it matter? Kinda both. Just call it C." It's kind of like taking the idea of [modal mixture](https://viva.pressbooks.pub/openmusictheory/chapter/modal-mixture/) and following it so far that it's hard to say which is the "real" mode anymore.
Comment: If you analyze enough Beethoven, you will start to see frequent modal mixture, but I'm not sure if that's the same as treating parallel keys the same. It's also not unique to Beethoven. Also, when a Beethoven piece in a major key has a PAC it's to the major I and likewise for minor. So he might mix modes in the middle, but he still retains the sense of the home mode.
Comment: @Dekkadeci Well, they don't need two articles, since Beethoven considered them one and the same. ;-)
Comment: @Aaron - If that were really true, I'd expect the Waldstein sonata, which is in C major, to actually get mentioned in "Beethoven and C minor" articles. Haven't found a single mention in any such article so far (and I checked the Wikipedia one just now again).
Comment: @Dekkadeci Just to make sure it's clear, my comment was entirely tongue-in-cheek. As is this one, about the Waldstein sonata, which is commonly misidentified as C major, but which is actually in C minor — the two, of course, being one and the same. (And, just to be even more clear for everyone else, I think it's actually a very interesting question and hope someone can provide a well-documented answer.)
Comment: @Dekkadeci Funny we completed our formal analysis of the first movement of the Waldstein in class yesterday and the modal mixture came up. Particularly because the first (and only) cadence in the primary theme is a half cadence preceded by a C *minor* chord. My point being, modal mixture in the Waldstein is definitely well-known and discussed, whether or not it has made it into the articles you have checked. Perhaps the article writers feel that no more needs to be said about the Waldstein because it's already famous for that half cadence and other C minor areas.
Comment: @ToddWilcox - Huh, I remember reading a different (probably unconfirmed) source saying that the parallel major/minor keys were considered to be more equivalent and providing less drama/contrast than the relative major/minor keys in earlier Classical music. At the very least, Mozart *loves* using the parallel minor within his second theme groups of sonata-allegros and also often in transitions to prepare the exposition's second theme group's dominant key.
Comment: I suppose the video mentions using the functions in both modes with the same tonic interchangeably. For instance, a C major phrase ends with Fm-G7-Cm cadance.
Answer: I think the key to understanding the idea is to separate the concepts of tonal and modal in regard to scale degrees and functional harmony versus modal color.
In a nutshell, the tonal scale degrees are ^1, ^4, and ^5, and the modal scale degrees are ^3 and ^6. ^1, ^4, and ^5 provide the roots for the all important tonic, subdominant, and dominant chords, while ^3 and ^6 provide the thirds for the tonic and subdominant chords. ^3 and ^6 can be either major or minor and so define the mode of a key.
To some extent you could say scale degrees ^2 and ^7 have mixed roles, but in regard to functional harmony the raised ^7 degree, the leading tone, is critical to dominant chords and it's "function" is to define the tonic in both major and minor keys.
Another way to say this is: the scale degrees ^1, ^4, ^5, and leading tone ^7 do not vary between the major and minor forms of a key. The functional aspect of harmony doesn't change between major and minor forms of a key: ^5 goes to ^1, leading tone ^7 goes to ^1, and ^4 goes to ^3. That's the essence of functional tonic/dominant harmony. The variable major or minor aspect of ^3 and ^6 only provides "color", determines the mode, for a key.
A common convention in harmonic analysis is to use upper case Roman numerals for major chord quality and lower case for minor quality, like I for a major triad and i for a minor triad. But there is also a convention that does not distinguish major/minor quality and only give upper case Roman numerals to indicate chord roots. By that second convention IV V I only means "subdominant, dominant, tonic triads with no regard to modal quality.
So, we can define harmonic function without knowing the details of mode. You can think of that as either modal ambivalence or as accepting a much more "colorful" harmonic style. I suppose if you push that idea far enough you could say there really aren't separate major and minor tonalities but simply a single tonality. There is no name for it, except perhaps "tonal harmony."
Comment: I like this answer. Shouldn't (4th para.) ^5 stay as ^5 though? +1.
Comment: Yes, for upper voices, I meant in the bass.
Answer: This is a really basic answer, but it is one example I can think of where I have seen this. The source you provide doesn't seem to give a lot of detail about what he means as "identical", but I would say that strictly speaking only identical things are identical. One way you can treat the parallel major and minor keys similar, though, is its relationship to the dominant chord. This is something I have noticed in some of Bach's and Ponce's music.
One thing you could do to give a song a strong sense of thematicism, while getting a novel sound is repeat an idea you already did in the major key in the parallel minor later in the song. It is very easy to modulate to the parallel minor because it has the same dominant chord.
The simplest example I can think of that I have seen in Ponce's music is this: "my musical idea is that the bass line is going to walk up the scale. I will harmonize then with the tonic and dominant chords where possible, and then I will repeat this same idea later using a similar melody in the parallel minor. "
Can't find a perfect example in the same song, but here is one. In the Allemande of Ponce's suite in Am he uses the pattern I show below in A minor. Then in the next song, the Sarabande from that suite, he uses that pattern in A major. I tried to offset and stretch the scores so you can see the patterns line up.
My Simplified Example

Example from Ponce Suite in La Minuer
Comment: "One way you can treat the parallel major and minor keys similar, though, is it's relationship to the dominant chord".  I think you are correct here. In tonic-dominant harmony, a C harmonic minor scale will work perfectly well on the G7. That is currently my line of thinking?

Beethoven and parallel minor keys


Proving $\neg (p\land \diamond q)$

Question: Assume the Necessitation Rule and the Distribution Axiom (https://en.wikipedia.org/wiki/Modal_logic#Axiomatic_systems) of modal logic, and also assume the axiom $p\land \diamond q\to \diamond(q\land\diamond p)$. Suppose that $\neg(\diamond p\land q) $ is provable (one can apply all standard inference rules and all tautologies of propositional logic). I'm trying to show that $\neg (p\land \diamond q)$ is also provable.
Assume by contradiction $p\land \diamond q$. By modus ponens, $\diamond(q\land\diamond p)$ or $\neg \square \neg (q\land \diamond p)$. We also have $\neg (q \land \diamond p)$. But I don't see how to obtain a contradiction.
Answer: Since $\lnot(q\land\lozenge p)$ is provable, then it is a theorem and the necessitation rule can be applied.
Do so.

Proving $\neg (p\land \diamond q)$


R: changing String values to NA, getting coercion error

Question: I've got a dataset called data that looks like this:
  Year  Population
1 2005  3000
2 2006  4000
3 2007  5000
4 2008  6000
5 2009  NP
6 2010  NP
7 2011  NP
8 2012  6000
9 2013  3000

I want to get the average of the count column, but since NP is not a numeric value I want to convert it to NA. I've used the following code:
data %>% mutate(Count = as.numeric(Count))

However I get an error message saying that NAs have been introduced by coercion. How can I convert all of the values to numeric and avoid getting this error?
Comment: It is not an error message, just a warning
Answer: It is not an error, just a warning message.
1) We can wrap with suppressWarnings
data$Count <- suppressWarnings(as.numeric(data$Count)))

e.g.
v1 <- c(1, 2, 3, 'a', 'b')
suppressWarnings(as.numeric(v1))
#[1]  1  2  3 NA NA

The most direct method of converting is to apply the as.numeric without any other manipulations.  The warning here is a friendly message.
2)  If we want to convert to numeric without getting the warnings, another option would be to replace the NP to `NA
library(dplyr)
data %>%
      mutate(Count = as.numeric(na_if(Count, 'NP')))

3) Or another option is to replace the elements to NA based on non-numeric character
library(stringr)
data %>%
    mutate(Count = as.numeric(replace(Count,
           str_detect(Count, '\\D'), NA)))

We can get the summarised output within summarise or mutate
data %>%
      mutate(Count = sum(as.numeric(replace(Count, str_detect(Count, '\\D'), NA)), na.rm = TRUE))

Answer: There are many ways in which you can do this.
Couple of ways include -

Using grepl we can turn all the values in Population column that has a non-digit character to NA and convert it to numeric.

df$Population[grepl('\\D', df$Population)] <- NA
df$Population <- as.numeric(df$Population)
df

#  Year Population
#1 2005       3000
#2 2006       4000
#3 2007       5000
#4 2008       6000
#5 2009         NA
#6 2010         NA
#7 2011         NA
#8 2012       6000
#9 2013       3000

With parse_number :

df$Population <- readr::parse_number(df$Population)

After doing this you can perform all the mathematical operations as usual with na.rm = TRUE.
mean(df$Population, na.rm = TRUE)
sum(df$Population, na.rm = TRUE)