Which correlation analysis method should be prioritized when different methods yield completely different rankings?

Question: I've been working with an industry-related dataset where I need to analyze the correlation between a specific output (y) and several inputs (x1, x2, x3, etc.). During my search, I came across various correlation analysis methods such as Pearson, Kendall, Spearman, and Mutual Information. However, I'm facing a challenge: different methods yield completely different correlation rankings. In some cases, the variations are quite significant. For instance, one input might be the most correlated with y according to the Mutual Information method, but it ranks as one of the least correlated in other methods.
This inconsistency poses a dilemma for my analysis, which is crucial for an industry-related dataset. They need a clear understanding of how inputs correlate with y. I'm considering combining results from all methods but I'm worried that some might not be well-suited to my dataset, leading to inaccurate conclusions.
Could you advise on the most suitable correlation method for a general analysis? Can Mutual Information be considered the best method as it quantifies the "amount of information" obtained about one random variable by observing the other random variable? Are there any methods I should avoid because they are designed just for specific situations?
Also, I would appreciate any tips on how to effectively conclude and integrate the findings from different methods.
Comment: Your use of the word "correlate" will tend to be interpreted in terms of some kind of formal correlation coefficient here on CV, but is that what you really mean?  "How inputs correlate with y" sounds like you are looking for some form of *multiple regression.* Compared to looking at individual "correlations" (however those might be defined), multiple regression goes much more deeply into the mutual relationships among the x's and y by accounting for the relationships among the x's, too.
Comment: Minimal comment to allow future readers to make full sense of this thread. OP posted the question, got answers as below, and also made some comments in reply to answers, giving more context to their question, and asking for more advice. Then for whatever reasons OP deleted their comments, and indeed their question, and deleted the account. The deletion of the question was reversed, but the OP's comments are no longer visible. Several comments by various people seeking to engage with the OP positively have also been deleted.
Answer: I have to say that this question is, to me, posed backwards, for several quite different reasons. Here are some.

Only exceptionally can different correlations be expected to agree, as they are based on quite different criteria. Indeed mutual information criterion isn't even a correlation in the traditional but useful sense of being bounded by $-1$ and $1$ with interpretable limits. This isn't resolved by discussions in principle about which is best, which is no more fruitful than discussing which pen or bag or car is best.

When the evidence from correlations is puzzling or ambiguous, the only real way to resolve that is to look more closely at the data, preferably with some graphics, and indeed this is a lot of work for any non-trivial dataset. This could show up all sorts of quite different set-ups, including

(a) there is no appreciable relationship, so all correlations are more or less futile or unhelpful
(b) there is a relationship but not one described even roughly by a straight line or monotonic curve
(c) something is going on but you may need something like a transformation (e.g. if outliers are dominating Pearson correlation) to think about it appropriately.

There is more recent work purporting to give more versatile measures of correlation, such as energy correlation or the work of Chatterjee. The optimists think that these are exciting and clever ideas. The pessimists think that naive users will confuse answers to "Is there a strong correlation?" with "Is there a relationship and what is it?" which in my experience is always closer to the real scientific or practical problem.

As a very simple example: What is the correlation between time of day and temperature? It's about zero, presumably, because rising limbs as the air heats up are balanced by falling limbs with the opposite. But naturally there is a relationship there best described in other terms.
Comment: You're asking several good extra questions in your comments, but that makes a single focused reply even harder. The question is getting closer to what do I need to know to build a model using machine learning on a large dataset.
Answer: I agree with the answers by Frank and Nick. Adding to them:

Could you advise on the most suitable correlation method for a general
analysis?

No, and I'd be leery of anyone who did so. Frank's answer goes about as far as I think you could go in this direction, but it is also, essentially a "no".

Can Mutual Information be considered the best method as it quantifies
the "amount of information" obtained about one random variable by
observing the other random variable?

I don't think mutual information is a correlation.  As to when you should use which, and what they are for, see this thread  But, often, you want both.

Are there any methods I should avoid because they are designed just
for specific situations?

The methods you mention are all of fairly general application. But the method you should avoid is thinking that there will be one method that works in all cases. One of my favorite statistical quotations is

There are no routine statistical questions, only questionable
statistical routines.  --- Sir David Cox.

I also wonder why you are ranking the correlation methods. This obscures things. The gap between the highest and lowest measure may be small or large. If it is quite small, then the rankings are pretty much arbitrary, if they are large then the very fact that there is a big difference indicates that you need to look more carefully.
Also, why correlation at all? You call y an "output" and the various x's "inputs".  Correlation doesn't distinguish. If you want to look at outputs and inputs, then some sort of regression is the usual way.
Finally, it would help if you told us what industry, what variables, how they are measured, what your sample size is, and, most crucially, what you want to find out, not in statistically terms, but in substantive ones.
Comment: #1 Good to do EDA.  I would use graphs. 150 graphs is not so many.  For the large number of samples, you can make the plotting symbol a very small dot, or transparent, in a scatter plot.  For choice of correlation, see Frank's answer.

#2  There's a whole literature on model building. I wouldn't use correlation in this way as part of that. You probably have some substantive knowledge that can guide things.
Answer: When the variables are almost continuous and relationships are not expected to be very non-monotonic my default choice is Spearman's $\rho$.  If you expect non-monotonic relationships that do not change directions more than once, then a 2 degree of freedom generalization of $\rho$ is handy.  See https://hbiostat.org/rmsc/multivar
If examining more than a few variables, it may be important to quantify the uncertainty in rankings of coefficients.  Https://hbiostat.org/bbr/hdata gives some simulation and bootstrap approaches.
Note that the minimum sample size needed to estimate a single correlation coefficient well is 400.


Using Excel "IF" Function to Assign Months to Cells

Question: I have a spreadsheet containing multiple cells that are formatted as dates.  What I would like to do is write an "IF" statement that I can copy and paste that will check the first value in the cell and then come up with a month.  For example, I have a cell that has a value of 9/30/2014 and I am trying to do the following:
IF(LEFT(A2,1) = 9, "September", "NULL")

This is throwing me a value of "NULL" for some reason.  Any guidance on how to get this to work would be appreciated.
Answer: Dates are not handled as strings. They are the number of days since 1900-Jan-0. You need to convert the date to its month number.
=IF(MONTH(A2)=9,"September","Null")

Answer: If your source cell A2 is formatted as text, enclose the value 9 within double quotes.
Like this =IF(LEFT(A2,1) = "9", "September", "NULL")
Comment: It is formatted as a date, though.  How do I deal with that?
Comment: =IF(MONTH(A2) = 9, "September", "NULL")
Answer: The simplest way to return the month from a date is:
=TEXT(A1,"mmmm")

Example:


Spring integration - less boilerplate?

Question: I have started using Spring-integration for listening on JMS-queues, and it's working satisfactory, but I have somewhat of an issue with the amount of xml-configuration required in order to set up a listener. Most of this is boilerplate, what changes is really just the jms-instance and the name of the queue to listen to, and the class and method to invoke upon receiving a message. Here is an example:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:int="http://www.springframework.org/schema/integration"
       xmlns:jms="http://www.springframework.org/schema/integration/jms"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
                           http://www.springframework.org/schema/integration/jms http://www.springframework.org/schema/integration/jms/spring-integration-jms.xsd">
  <bean id="jms.queue.myQueue" class="org.apache.activemq.command.ActiveMQQueue">
    <constructor-arg value="${myApplication.jms.queue.myQueue}" />
  </bean>
  <bean id="jms.container.myJMSListender"
        class="org.springframework.jms.listener.DefaultMessageListenerContainer"
        destroy-method="destroy">
    <property name="connectionFactory"
              ref="jmsConnectionFactory" />
    <property name="destination"
              ref="jms.queue.myQueue" />
    <property name="sessionTransacted"
              value="true" />
  </bean>

  <int:publish-subscribe-channel id="channel.myQueue" />

  <jms:message-driven-channel-adapter id="channelAdapter.myQueue"
                                      container="jms.container.myQueue"
                                      channel="channel.myQueue"
                                      acknowledge="transacted"/>

  <int:service-activator id="serviceActivator.myQueue"
                         input-channel="channgel.myQueue"
                         ref="myQueueJMSListener"
                         method="handleMessage" />
</beans>

As you can see, a lot of config for something fairly simple, and with quite a lot of JMS-listeners this becomes tedious, both to read and write.
Is there a way to configure listening to a queue with Spring-integration that requires less boilerplate? I've looked into creating my own XML-tag, but I'm kinda hoping that there is a simpler solution.
Comment: You don't have to create the `MessageListenerContainer` instead you can pass the connection factory and destination to the `message-driven-channel-adapter` saves you the listener definition. As described http://docs.spring.io/spring-integration/reference/html/jms.html#jms-message-driven-channel-adapter here.
Answer: You only need an external container if you need to configure properties that are not exposed on the namespace.
<jms:message-driven-channel-adapter id="channelAdapter.myQueue"
                                  destination="jms.queue.myQueue"
                                  connection-factory="jmsConnectionFactory"
                                  channel="channel.myQueue"
                                  acknowledge="transacted"/>

Comment: Cool, that helps at least. Using desination-name instead of destination also removes a bit of config, so getting there :)
Answer: Found that Spring integration has annotation support.
Requires some setup, but this can be used for multiple listeners:
  <amq:connectionFactory id="jmsConnectionFactory" brokerURL="${jms.host}">
    <amq:redeliveryPolicy>
      <amq:redeliveryPolicy maximumRedeliveries="0" />
    </amq:redeliveryPolicy>
  </amq:connectionFactory>

  <bean id="cachingJmsConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
    <property name="targetConnectionFactory" ref="jmsConnectionFactory" />
  </bean>

  <jms:annotation-driven />

  <bean id="jmsListenerContainerFactory" class="org.springframework.jms.config.DefaultJmsListenerContainerFactory">
    <property name="connectionFactory" ref="cachingJmsConnectionFactory" />
  </bean>

Then, listening to a certain queue is as simple as:
@JmsListener(destination = "${myQueueName}")
public void listen(@Payload final String payload) {

Spring integration - less boilerplate?


Elementary Measure Theory Problem

Question: I'm trying to solve the following exercise but I always end up missing some vital step along the way, so any help would be much appreciated!
Let $\mathbb{Q}$ be the set of all real rational numbers, and let $I_Q = \{[a, b)_Q : a, b ∈ \mathbb{Q}\}$ where
$[a, b)_Q = \{q ∈ \mathbb{Q} : a ≤ q < b\}$.
(a) Prove that $σ(I_Q) = P(\mathbb{Q})$, where $P(\mathbb{Q})$ is the collection of all subsets of $\mathbb{Q}$ and $σ(I_Q)$ is the sigma algebra generated by $I_Q$.
(b) Let $µ$ be counting measure on $P(\mathbb{Q})$, and let $ν = 2µ$. Show that $ν(A) = µ(A)$
for all $A ∈ I_Q$, but $ν \neq µ$ on $σ(I_Q) = P(\mathbb{Q})$. Why doesn't this contradict the uniqueness of measures Theorem?
Comment: What is $\sigma$?
Comment: Presumably the $\sigma$-algebra generated by the given collection.
Comment: Precisely! Should have mentioned it. Sorry!
Answer: For part (a), since $\mathbb{Q}$ is countable, is is enough to show that each singleton is in $\sigma(I_\mathbb{Q})$.
For part (b), look at the hypothesis that the original measure has to be $\sigma$-finite for the extension to be unique.
Comment: Every set in $I_\mathbb{Q}$ is infinite.
Comment: Could you please elaborate on (b) a little more?
Comment: @V. Krumov: clearly, if the uniqueness theorem does not hold, then one of its hypotheses must not be satisfied. In this case, the hypothesis to worry about is that the original (pre-extended) measure has to be $\sigma$ finite.
Comment: I really cannot see how can ν(A)=µ(A) if v=2µ? Clearly it follows for the empty set and the infinite but for all $A∈I_Q$?

Elementary Measure Theory Problem


Is there anything like "implied density" in this experiment?

Question: A basket of balls is dropped into a maze. When a ball is dropped into the maze at the top it moves downward, pulled by gravity, through a series of nails. The ball then falls down to a new level where it encounters another nail. Finally, at the bottom of the maze, the ball falls into one of the troughs.

If enough balls are dropped into the maze, we get a distribution of balls which is a mixture of two normal distributions with parameters: $x_{1}\sim\mathcal{N}(\mu_{1},\sigma_{1})$, $x_{2}\sim\mathcal{N}(\mu_{2},\sigma_{2})$, and $w_{1}=1-w_{2}$ (weight).

As you can see, there are many troughs. Moreover, there are $\tau$ levels of nails. If $X$ is the number of a trough, we know the probability of our balls falling in a trough lower than $X$ after $\tau$ levels of nails. Let's call the density function $f(\tau)=f^{*}$. Now we can repeat this experiment by adding $t$ levels of nails after the first $\tau$ levels of nails, getting a total of $T=\tau + t$ maze levels. Because nails are randomly distributed, we get a distribution of balls which is a mixture of two normal distributions with different parameters than before. We get the probability density function $f(T)=g^{*}$:

So now we know that, after $\tau$ maze levels, our p.d.f. is $f^{*}$; then we know that, after $T=\tau + t$ maze levels, our p.d.f. is $g^{*}$. My question: what's the p.d.f $f(T - \tau)=f(t)$? In other words, how to obtain the p.d.f. implied by this experiment related to balls falling from level $t$ to level $T$ in this case and under what hypothesis does your answer hold?
Comment: "nails are randomly distributed" — They are? I thought they were arranged in a fixed pattern. What procedure is used to randomly decide where to place the nails?
Comment: The description could use some history details:  Galton's quincunx (circa 1898)(https://en.wikipedia.org/wiki/Bean_machine).  Also, a reference to the claim that the result is a mixture of two normals would be interesting.
Comment: It seems that nails are arranged so that there will be $n = 15$ random bounces, each left or right with probability about 1/2. Then with the numbers of the bins at the bottom as you have labeled them $(0, 1, \dots, 15).$ we can say that the ball comes to rest at bin $X,$ where $X$ is the number or bounces to the right. So (assuming a little more precision than an actual physical apparatus would have) the implied dist'n is $X \sim \mathsf{Binom}(n=15,\,p=1/2).$ In a classroom demo, one might comment that the dist'n is nearly normal.
Comment: @Kodiologist: I'm sorry because the picture may be misleading. Although the picture shows a fixed pattern, it's just an example. Actually playing with the normal mixture parameters can return a normal distribution. So the procedure to place the nails is the one which returns your favourite normal mixture.
Comment: @BruceET: let the levels/steps/random bounces go towards $\infty$. I think this should make the central limit theorem come into play.
Comment: @JimB: the claim that the result is a mixture of two normals is just a further complication and an attempt to introduce some generalization, because for properly chosen parameters we have the normal distribution. You can use the normal p.d.f. if you want.
Comment: I get it that the binomial distribution will be well-approximated by a single normal distribution for the standard/symmetrical/50-50 chance of going to the left or right at each level.  But without a specific alternative nail design, I see that you've presented no justification that a bimodal distribution will arise (let a alone a mixture of two normals).  It would be an interesting problem to solve with specified alternative nail designs.  But the distribution will always be discrete and starting out with continuous distributions seems to be jumping the gun a bit.
Comment: @JimB: ok, so let's simplify the problem by getting rid of nails and mixtures (I can amend the original question if you think this is needed: the risk here is seeing the wood for the trees). $f^{*}$ is the p.d.f. of a stochastic process that goes from $t=0$ to $t=\tau$; $g^{*}$ is the p.d.f. of the same process from $t=0$ to $t=T>\tau$. Question: is it possible to get the p.d.f. of the process from $t=\tau$ to $t=T$ just by knowing $f^{*}$ and $g^{*}$?
Answer: This is an extended comment to simply show my confusion as to the details of the problem.
I think that you need to start with the discrete nature of the problem (at least that's what your figure and initial description implies to me).  I appears that there are discrete time steps and just two potential outcomes at each nail.
One way to generalize Galton's device is to assume that at each nail (or vertex) there is a probability of the ball going to the left or right.  This would be a Bernoulli random variable.  For 5 levels (4 levels of transition and 1 final level) the probabilities might be described as follows:

where $p_{i,j}$ is the conditional probability of the ball going to the left of the nail given that the ball is in position $j$ of row $i$.
If $p_{i,j}=p$, then when there are $n$ rows of nails, the probability of landing in the $k$-th trough is given by
$$\binom{n-1}{k-1}(1-p)^{k-1}  p^{n-k}$$
When all of the $p_{i,j}$ probabilities are different, then the distribution is a bit more complex and can be multi-modal.  But it remains a discrete probability distribution.  Here is an example with 5 levels:

One might be interested in the distribution of $X_\tau$ (where a ball will land after $\tau$ levels or the joint distribution of $X_\tau$ and $X_{\tau+t}$ but if so, that is not clear (to me) from your description.
You've mentioned continuous probability density functions which might be handy for a large number of levels and varying nail probabilities but that would seem to come into play after the set-up is completely described.
OK.  You raise a good question.  I'm just not seeing the necessary details to move forward.


Image with text as a resource file

Question: My client gave me images with text on them in png to use as buttons in the app.
Is it bad to use them that way or should I ask for images without text and I use the android:text property to write text on it?
I found the image with text in a resource file to be good that text is already position with the right font but I feel it is not good approach
Thanks
Answer: I don't think that is a bad approach, many apps use already generated images this way, so I can't see why this can be a bad way of doing it.
Answer: It is clearly not a good approach. Yes, you should use android:text to write text. See Android runs on different sized devices, so your button will have a different size in different devices and the text may not be readable in some of them.
But when you use the text attribute, your background image may have a bad quality, but the text will still be readable.
Comment: But when using images on different device, it will be in dp so really it will show up as the same size regardless of the screen size. no?

Image with text as a resource file


How to resolve designated initialization error for UITableViewController?

Question: I'm new to swift and I'm having problems declaring my initializer in my PlacesTableViewController class. It prompts me "Super.init isn't called before returning from initializer" and when I added the Super.init, it prompts me:

"Must call a designated initializer of the superclass
  'UITableViewController'"

this is my PlacesTableViewController.swift file:
class PlacesTableViewController: UITableViewController {

    var pa55DB : COpaquePointer = nil
    var selectStatement : COpaquePointer = nil;

    var chapterdata : Array<Entry> = []
    var ety : Entry
    var sqlString : String = "";

    var chpData : ChapterData

    init(pa55DB : COpaquePointer,selectStatement : COpaquePointer, chapterdata : Array<Entry>, ety : Entry, sqlString : String) {
        super.init()
        self.pa55DB = pa55DB
        self.selectStatement = selectStatement
        self.chapterdata = chapterdata
        self.ety = ety
        self.sqlString = sqlString

    }

    required init!(coder aDecoder: NSCoder!) {
        fatalError("init(coder:) has not been implemented")
    }

    override func viewDidLoad() {
        super.viewDidLoad()

        chpData = ChapterData(newEntryArray : chapterdata, entryV : ety, paDB55 : pa55DB, selectStatement : selectStatement, sqlString : sqlString)
        chpData.openDB()
        chapterdata = chpData.prepareStartment()

        println("Testing...")
    }
...
}

Here's my another class file called ChapterData.swift, used to open database and retrieve data
class Entry {

    let image : String!
    let heading : String!
    let title : String!
    let description : String!
    public init(image : String, heading : String, title : String, desc : String) {
        self.heading = heading
        self.image = image
        self.title = title
        self.description = desc
    }
}

class ChapterData {

    var paDB : COpaquePointer = nil;
    var selectStatement : COpaquePointer = nil;
    var sqlString : String = "";

    var newEntryArray : Array<Entry> = [];
    var entryV : Entry;

    init(newEntryArray : Array<Entry>, entryV : Entry, paDB55 : COpaquePointer, selectStatement : COpaquePointer, sqlString : String){
        self.newEntryArray = newEntryArray
        self.entryV = entryV
        self.paDB = paDB55
        self.selectStatement = selectStatement
        self.sqlString = sqlString
    }

    func openDB() {
        var paths = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true)[0] as! String
        var docsDir = paths.stringByAppendingPathComponent("pa55.sqlite")

        if (sqlite3_open(docsDir, &paDB) == SQLITE_OK) {
            println("success open database")
        }
        else {
            println("failed to open database")
        }
    }

    func prepareStartment() -> Array<Entry>   {
        sqlString = "SELECT * FROM Chapter ORDER BY CID ASC"
        var cSql = sqlString.cStringUsingEncoding(NSUTF8StringEncoding)
        sqlite3_prepare_v2(paDB, cSql!, -1, &selectStatement, nil)

        println("startment start")

        while (sqlite3_step(selectStatement) == SQLITE_ROW) {

            let image_buf = sqlite3_column_text(selectStatement, 3)
            var image = String.fromCString(UnsafePointer<CChar>(image_buf))

            let heading_buf = sqlite3_column_text(selectStatement, 1)
            var heading = String.fromCString(UnsafePointer<CChar>(heading_buf))

            let title_buf = sqlite3_column_text(selectStatement, 2)
            var title = String.fromCString(UnsafePointer<CChar>(title_buf))

            let desc_buf = sqlite3_column_text(selectStatement, 4)
            var description = String.fromCString(UnsafePointer<CChar>(desc_buf))

            entryV = Entry(image: image!, heading: heading!, title: title!, desc: description!)
            newEntryArray.append(entryV)
        }
        println("select statement successful executed")
        return newEntryArray
    }

}

Comment: The rules are complicated but obvious once you understand them. Read the Swift language book. It explains it all.
Comment: I've looked over your file, your designated initializer issue IS resolved, however this caused another error: Property 'self.et' not initialized at super.init call. So your ety: Entry needs to be initialized properly. You should probably ask a new question regarding this as it's general practice to ask one question per post.
Comment: So now the problem left is just properly initialize my ety: Entry, can I continue with the codes I've shared with you(to debug)?
And thanks so much for your help.
Answer: The init mess is a known limitation of the ObjC <-> Swift bridging.
One way to work around this is to delegate to super's init(nibName:bundle:) initializer.
class PlacesTableViewController: UITableViewController {

    var chapterdata: [Entry]
    var chpData: ChapterData?
    var ety: Entry
    var pa55DB: COpaquePointer
    var selectStatement: COpaquePointer
    var sqlString: String

    init(pa55DB: COpaquePointer, selectStatement: COpaquePointer, chapterdata: [Entry], ety: Entry, sqlString: String) {
        self.pa55DB = pa55DB
        self.selectStatement = selectStatement
        self.chapterdata = chapterdata
        self.ety = ety
        self.sqlString = sqlString

        super.init(nibName: nil, bundle: nil)
    }

    required init(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    // …
}

Comment: My Build success, however, when i run my project, there is a fatal error saying init(coder:) has not been implemented.
Thread 1: EXC_BAD_INSTRUCTION (code=EXC_1386_INVOP, subcode=0x0)
Comment: Are you loading the view controller from a storyboard or nib/xib file? If so then you'll also have to implement `required init(coder: NSCoder)` properly which is currently not implemented.
Comment: Yes, I'm using the view controller from my storyboard. How do I implement the required init(coder: NSCoder), is there any references I can look at to follow?
Comment: Warning: May cause unwanted side effects. For instance, I pushed a `UITableViewController` using @fluidsonic's hack and the `UITableView` was all messed up. It's `UITableViewSectionHeaders`'s were sized all wrong and  the section header font was different. Be careful using this.
Answer: See if this resolves your issue:
override init(style: UITableViewStyle) {
    super.init(style: style)
}

init(pa55DB : COpaquePointer,selectStatement : COpaquePointer, chapterdata : Array<Entry>, ety : Entry, sqlString : String) {
    super.init(style: UITableView.Style.plain)
    self.pa55DB = pa55DB
    self.selectStatement = selectStatement
    self.chapterdata = chapterdata
    self.ety = ety
    self.sqlString = sqlString

}

UITableView Class Reference
From the Swift Programming Language Guide:

Designated initializers are the primary initializers for a class. A
  designated initializer fully initializes all properties introduced by
  that class and calls an appropriate superclass initializer to continue
  the initialization process up the superclass chain.

[read more]

Update:
The above code actually resolves the issue with your designated initializer. You now need to properly initialize your self.ety property.
Comment: override init(style: UITableViewStyle){

    }
//error: Super.init isn't called before returning from initializer

init(pa55DB : COpaquePointer,selectStatement : COpaquePointer, chapterdata : Array, ety : Entry, sqlString : String) {
        super.init(style: UITableViewStyle.Plain)
        self.pa55DB = pa55DB
        self.selectStatement = selectStatement
        self.chapterdata = chapterdata
        self.ety = ety
        self.sqlString = sqlString

    }
//error at super.init(style: UITableViewStyle.Plain) -> Property 'self.et' not initialized at super.init call
Comment: you'll need to update your super.init in your initializer as well, add super.init(style: UITableViewStyle.Plain) to your init(pa55DB...
Comment: sorry, but isn't super.init(style: UITableViewStyle.Plain) already added in the code you showed above?
Comment: give me a moment, i finding how to upload my project to github. :(
Comment: sir, can I have you email to share the folder with you?
Comment: Done, I've just finished added all the files in
Comment: what's the project name, you're not making it simple for me, you've got nearly 10 projects in that folder
Comment: maintpt2 that the project name. Sry for that. I'll reput the project individually to the share folder.
Answer: Maybe try something like this:
class PlacesTableViewController: UITableViewController {

   var chapterdata: [Entry]
   var chpData: ChapterData?
   var ety: Entry
   var pa55DB: COpaquePointer
   var selectStatement: COpaquePointer
   var sqlString: String

   convenience init(pa55DB: COpaquePointer, selectStatement: COpaquePointer, chapterdata: [Entry], ety: Entry, sqlString: String) {
       self.init()

       self.pa55DB = pa55DB
       self.selectStatement = selectStatement
       self.chapterdata = chapterdata
       self.ety = ety
       self.sqlString = sqlString

   }

   required init(coder aDecoder: NSCoder) {
       super.init(coder: aDecoder)
   }

   // …
}

Answer: UITableViewController isn't anything good for advanced use cases. You can always subclass UIViewController and put an UITableView property in it, just like this:
class PlacesTableViewController: UIViewController {

    let tableView = UITableView(frame: CGRectZero, style: .Plain)

    override func loadView() {
        view = tableView
    }

    // Custom designated initializer
    init(pa55DB : COpaquePointer,selectStatement : COpaquePointer, chapterdata : Array<Entry>, ety : Entry, sqlString : String) {
        // setting up these properties
        super.init(nibName: nil, bundle: nil)  // designated initializer of the superclass 'UIViewController'"
    }
}

And it works like UITableViewController.
Other utilities UITableViewController provides, like refreshControl you can also set one for yourself, like clearsSelectionOnViewWillAppear mechanism can also be implemented in viewWillAppear() or didSelectRowAtIndexPath depending on your needs.
Comment: What, this is one of the worst statements I've seen on Stackoverflow to date. UITableViewController's should be used over raw UITableView's more often than not. Why in the world would you recreate what's already been done.
Comment: Until you find UITableViewController is really a _bad joke_ in Swift, and even _crash_ on iOS 8 iPad. https://forums.developer.apple.com/thread/16621
Comment: That's an initializer issue between swift and objc which in no way supports your poor claim. As someone who's been in the industry for years working on apps with user bases the size of Uber, I will never take you seriously if you call a uitableviewcontroller a joke lol. To me, it sounds like you just flat out don't know how to use it or have poor architecture paradigms. Also, I recommend looking into issue reports before blaming a class implementation lol.


Google Appscript problem with export to PDF when used by another user

Question: I have been making a sheet that helps me with the process of making invoices throught the "Invoice" tab on the sheet I will link below. Yes - I made sure to get rid of any info, do not worry.
The problem I have, is that if I try to export the Invoice tab to a PDF, this works as intended and the file is automatically named correctly following a certain structure in Appscript.
However, when someone elses uses this sheet, for some reason it saves the file on the wrong name (happens to always be the name of one customer above the one you are invoicing under. In my sheet, meaning that if they try to make an invoice for Example2, the file will save and give it a name based on info from Example1 even when the invoice tab is properly showing all data.
I am lost and looking to see if anyone can see what is happening.
File: https://docs.google.com/spreadsheets/d/1TF1xncuX_P47dSvQnrrbZIdOpTG20rQ4UvNpywiGwbs/edit?usp=sharing
Appscript:
<code>function onOpen() {
  var submenu = [{name:"Save PDF", functionName:"generatePdf"}];
  SpreadsheetApp.getActiveSpreadsheet().addMenu('Export', submenu);
}

function generatePdf() {
  var sourceSpreadsheet = SpreadsheetApp.getActive();

  var sheets = sourceSpreadsheet.getSheets();
  var sheetName = sourceSpreadsheet.getActiveSheet().getName();
  var sourceSheet = sourceSpreadsheet.getSheetByName(sheetName);
  var titleName = sourceSpreadsheet.getRange("G1").getValue();
  var pdfName = titleName;

  var parents = DriveApp.getFileById(sourceSpreadsheet.getId()).getParents();
  if (parents.hasNext()) {
    var folder = parents.next();
  }
  else {
    folder = DriveApp.getRootFolder();
  }

  var destSpreadsheet = SpreadsheetApp.open(DriveApp.getFileById(sourceSpreadsheet.getId()).makeCopy("tmp_convert_to_pdf", folder))

  var sheets = destSpreadsheet.getSheets();
  for (i = 0; i < sheets.length; i++) {
    if (sheets[i].getSheetName() != sheetName){
      destSpreadsheet.deleteSheet(sheets[i]);
    }
  }

  var destSheet = destSpreadsheet.getSheets()[0];

  var sourceRange = sourceSheet.getRange(1,1,sourceSheet.getMaxRows(),sourceSheet.getMaxColumns());
  var sourcevalues = sourceRange.getValues();
  var destRange = destSheet.getRange(1, 1, destSheet.getMaxRows(), destSheet.getMaxColumns());
  destRange.setValues(sourcevalues);

  var theBlob = destSpreadsheet.getBlob().getAs('application/pdf').setName(pdfName);
  SpreadsheetApp.flush();
  var newFile = folder.createFile(theBlob);
  SpreadsheetApp.flush();
  DriveApp.getFileById(destSpreadsheet.getId()).setTrashed(true) ;
</code>
Comment: In your showing script, it seems that the sheet name of the active sheet is used as the filename of `theBlob`. If you want to use the constant sheet name, how about directly put the specific sheet name to `sheetName` of `var sourceSheet = sourceSpreadsheet.getSheetByName(sheetName);`? But if I misunderstood your question, I apologize.
Comment: I'm not sure this is the issue here, but I have found it useful to call `SpreadsheetApp.flush()` *before* generating a pdf (i.e. before your line that starts `var theBlob = ...`), to force commit any changes.
Comment: Is it possible your friend is pressing the Invoice button before committing a change to the cell?  Try this: type 1 under FACTUURNUMMER, then press Enter. Then type 2 in that cell, and without pressing enter, immediately press the Invoice button.  The script will run with the example 1 data, because the value 2 was not yet committed.  Make sure they press enter in the cell (or click away) before pressing the Invoice button.
Comment: @AaronDuniganAtlee , the issue has been resolved, thank you for your comments. This was a user-end problem and I've passed them on to chill down on the executions of the files. I did add the extra flush however, for good measure.
Answer: The issue has been resolved, thank you for your comments. This was a user-end problem and I've passed them on to chill down on the executions of the files. I did add the extra flush however, for good measure like mentioned above.


How to add File [] Array content into ArrayList?

Question: I have an array:
File [] temp=null;

And I have an arrayList of File type:
List <File> tempList = new ArrayList <File>();

Now I want to add the content from temp to tempList. So anyone can please tell me How do I this?
Comment: loop through the `temp` array and add each File to the `tempList`
Comment: Just so others know, adding array contents to an `ArrayList` using a for loop will give you a warning in Android Studio that says it prefers you use the `ArrayList`'s `addAll()` method. You could still do it, but for some reason Android Studio doesn't prefer it (maybe it's more efficient to use the method?)
Answer: Try this
tempList.addAll(Arrays.asList(temp));

Comment: 'http://stackoverflow.com/questions/16433915/how-to-copy-file-from-one-location-to-another-location' can you plz see this problem.
Answer: If you are not going to update the content of the array (add/removing element), it can be as simple as
List<File> tempList = Arrays.asList(temp);

Of course, if you want a list that you can further manipulate, you can still do something like
List<File> tempList = new ArrayList<File>(Arrays.asList(temp));

Comment: The second line of code should be `List tempList = new ArrayList<>(Arrays.asList(temp));`, note the diamond in `ArrayList<>`
Comment: Dunno why I missed that lol. Lemme fix that
Answer: use following
List<File>tempList = Arrays.asList(temp);

Answer: You can iterate through the array and add each element to the list.
for (File each : temp)
  tempList.add(each);

Answer: You can use a collections library call for this:
Arrays.asList(temp);


How to get total number of rows on a block of a view

Question: On hook_block_view_alter() I need to get a total number of rows that this block displays.
For example, I have a view with the following block:

On hook_block_view_alter() I do the appropriate checks in order to weed out unnecessary blocks and get the one shown in the figure above.
function mymodule_block_view_alter(array &$build, \Drupal\Core\Block\BlockPluginInterface $block) {
  if (!preg_match('~^views_block:taxonomy_tag_list~', $block->getPluginId())) {
    return;
  }

  $view = $block->getViewExecutable();
  // Gets the number of rows
  $totalRows = $view->get_total_rows;
}

But for some reason, $totalRows contains NULL.
Here is the object of the current view:

Why can't I get the total number of rows? Maybe I'm using the wrong hook?
Comment: just curious, why you don't use HEADER (add) in this views display - you can select there "Result summary" with default content "Displaying **start** - **end** of **total**", plus many other tokens related to the current views, and replace with what you want (bold here should be prepended with @ sign but editor here recognize the sign as reference to a user)
Answer: Use a views_view_ hook, i.e. views_view_unformatted,  views_view_list  etc... I tried this with a PHP count for my views block and it worked great.

My function is:
function MYTHEME_preprocess_views_view_unformatted(&$vars) {
  $num_rows = $vars["rows"];
  $count = count($num_rows);
  echo '
This view has ' . $count . ' rows

';
}

This yielded the number of rows in the view.
Printed, I see:
Comment: I think no need to loop over `$num_rows` just use `count` function like `$count = count($num_rows); `
Comment: @berramou Good idea, answer tested and updated.
Answer: No need for any hook.
Adding {{ view.total_rows }} in any views template, eg views-view.html.twig is enough
Answer: No need for any coding. Edit the view and under "Header" or "Footer" add a "Text area" (Category: Global) and see the available global token replacements.
To get the total number of rows add: [view:total-rows] and save the view.
Comment: You're right, it's very simple with the token. But the limit is that we cannot make conditions. I needed it to handle the singular/plural of the text displayed after the counter.


CapacityProviderStrategies of EC2Service shows the error, The specified capacity provider ****** was not found

Question: From this article (that is solved but this article is related)
I made two AutoScalingGroup, two AsgCapacityProvider and two ecs on ec2 service
Now I want to attach these CapacityProvider to each service.
However, There is  error
The specified capacity provider myapp-dev-admin-cp was not found
const adminCapacityProvider = new ecs.AsgCapacityProvider(this, 'admin-asg-capacity-provider', {
  autoScalingGroup:adminAutoScalingGroup,
  capacityProviderName: `myapp-${targetEnv}-admin-cp`
});

const adminCapacityProvider2 = new ecs.AsgCapacityProvider(this, 'admin-asg-capacity-provider2', {
  autoScalingGroup:adminAutoScalingGroup2,
  capacityProviderName: `myapp-${targetEnv}-admin-cp2`
});

cluster.addAsgCapacityProvider(adminCapacityProvider);
cluster.addAsgCapacityProvider(adminCapacityProvider2);

const ecsAiService = new ecs.Ec2Service(this, 'AiService', {
  cluster,
  taskDefinition:aiTaskDefinition,
  serviceName: `myapp-${targetEnv}-ai-service`,
  enableExecuteCommand:true,
  securityGroups: [adminServiceSg],
  vpcSubnets:{subnetType: ec2.SubnetType.PUBLIC },
  capacityProviderStrategies:[
    capacityProvider:`myapp-${targetEnv}-admin-cp`// error occurs here
  ]
})

I googled around and found for fargate, FARGATE or FARGATE_SPOT should be put here, however my one is ECS on EC2.
What should I use for capacityProviderStrategie's props?
Thanks for any help.
Answer: This might be due to the fact that the service is being created before the capacity provider.
To create an implicit dependency (while also removing the need to specify the name of the capacity provider in two places), try this:
...
  capacityProviderStrategies:[
    capacityProvider: adminCapacityProvider.capacityProviderName
  ]
...

This will let CDK know that your service depends on the capacity provider and only create it after it's been provisioned.
Comment: Thank you very much, I understand the orders of items created. It solves my problem!


How to solve conversion error in SQL Server?

Question: Please let me know the issue with following query I am trying to join on msg_id in some cases message id will be hexadecimal so I am converting to decimal and joining but I get this error:

Msg 245, Level 16, State 1, Line 3
  Conversion failed when converting the varchar value 'e52b7ff2-0607-4b84-b326-7268af76f41d' to data type int.

This is the code:
select *
from ODS..RPL1 a
left join SAP_T1..RPL1 b on a.Mobileno = b.Mobileno
                         and a.Trans_ID = b.Trans_ID
                         and (case
                                 when charindex('-', a.msg_id) > 0
                                    then a.MSG_ID
                                 when ISNUMERIC(a.MSG_ID + 'e0') = 1
                                    then a.MSG_ID
                                 when ISNUMERIC(a.MSG_ID + 'e0') = 0
                                      and CHARINDEX('-', a.msg_id) = 0
                                    then CONVERT(int, CONVERT(VARBINARY, a.MSG_ID, 2)) end) =
                               (case
                                   when charindex('-', b.msg_id) > 0
                                      then b.MSG_ID
                                   when ISNUMERIC(b.MSG_ID + 'e0') = 1
                                      then b.MSG_ID
                                   when ISNUMERIC(b.MSG_ID + 'e0') = 0
                                        and CHARINDEX('-', b.msg_id) = 0
                                      then CONVERT(int, CONVERT(VARBINARY, b.MSG_ID, 2)) end)
where
    a.msg_id = 'e52b7ff2-0607-4b84-b326-7268af76f41d'

Comment: could you provide us more information about your table ?
Comment: could you please provide reference value for "a.msg_id"?
Comment: The error suggests the msg_id column of ODS..RPL1 is int but your literal is uniqueidentifier string literal.
Comment: hi all thanks for your quick replay i got solution i really thanks for you
Answer: You have logic like this:
(case when charindex('-', a.msg_id) > 0
      then  a.MSG_ID
      when ISNUMERIC(a.MSG_ID + 'e0') = 1
      then a.MSG_ID
      when ISNUMERIC(a.MSG_ID + 'e0') = 0 and
           CHARINDEX('-', a.msg_id) = 0
      then CONVERT(int, CONVERT(VARBINARY, a.MSG_ID, 2))
 end)

A case expression only returns one type.  In this case, the type will be an integer, because that is the most restrictive (that is how the conversion rules work).  Hence, returning msg_id will fail with the conversion to an integer.
Adding an explicit conversion back to a string will fix the immediate problem:
(case when charindex('-', a.msg_id) > 0
      then  a.MSG_ID
      when ISNUMERIC(a.MSG_ID + 'e0') = 1
      then a.MSG_ID
      when ISNUMERIC(a.MSG_ID + 'e0') = 0 and
           CHARINDEX('-', a.msg_id) = 0
      then CAST(CONVERT(int, CONVERT(VARBINARY, a.MSG_ID, 2)) as VARCHAR(255))
 end)

Whether this does what you want is unclear, because your question does not specify that.  If this doesn't do what you want (but it does fix the conversion), then ask another question with sample data and desired results.
Comment: @RamThakur if the answer is correct then you should accept it


What is a 'nail attack'?

Question: I heard this term the first time while browsing acoustic covers of songs in Youtube. I tried searching but there seems to be different variations, and none of the tutorials I found were quite clear.
What is it? How is it really done?
Example I found of it being used was here:
Comment: Could it just mean using your nails to attach the string rather than a pick?
Comment: @ggcg Which is why I'm trying to find out how exactly it is done, and what this 'special practice' is you're referring to for the nail attack. Especially if there are different variations.
Comment: "Exactly what is done" is hard to explain which is why lessons with a flamenco instructor would be good.
Comment: @ggcg I mean, it looks like at first glance, strumming using the nails. I tried but the sound produced is clearly different and there's something different about it that I can't figure out what...
Comment: For one thing you may just not be good at the technique yet.  Your nails might be slight different in shape, thickness, edge etc.  These things all affect the sound.  Classical guitarists devote a lot of energy to shaping the nails.  Then there is special practice for the attack.
Comment: After watching this video it seems like this is a self taught technique, not really classical flamenco or other traditional style.  He is describing what he's doing but in another language.  I personally do not like the sound of nails on steel string acoustics.
Answer: I think that "nail attack" in that case is used in the same way as "pick attack" is, that is to describe the way the nail "attacks" (touches) the string.
For example the attack can be flat (pick/nail parallel to string) or angled with consequences on tone and volume..

What is a 'nail attack'?


Rails 5 - How to create custom scaffold generator?

Question: The goal is for command...
bin/rails generate custom_scaffold Thing
... to generate the following 6 files:
db/migrate/201812031331_create_things.rb
app/models/thing.rb
app/controllers/things_controller.rb
app/serializers/thing_serializer.rb
test/fixtures/things.yml
test/integration/requests/things_request_test.rb

... using Rails 5.
My current setup does generate app/models/thing.rb but does not populate it with Thing.
Expected:
class Thing < ApplicationRecord

end

Currently:
class <%= class_name %> < ApplicationRecord

end

I have read through these Rails guides but to little avail.
Does anyone have a working example?

My setup:
# lib/generators/custom_scaffold/custom_scaffold_generator.rb

class CustomScaffoldGenerator < Rails::Generators::NamedBase
  source_root File.expand_path('templates', __dir__)

  def create_files
    copy_file 'migration.rb', "db/migrate/#{timestamp}_create_#{plural_name}.rb"
    copy_file 'model.rb', "app/models/#{file_name}.rb"
    copy_file 'controller.rb', "app/controllers/#{plural_name}_controller.rb"
    copy_file 'serializer.rb', "app/serializers/#{file_name}_serializer.rb"
    copy_file 'fixture.yml', "test/fixtures/#{plural_name}.yml"
    copy_file 'request_test.rb', "test/integration/requests/#{plural_name}_request_test.rb"
  end

  private

  def timestamp
    Time.now.utc.strftime('%Y%m%d%H%M%S')
  end
end

# lib/generators/custom_scaffold/templates/model.rb

class <%= class_name %> < ApplicationRecord

end

# lib/generators/custom_scaffold/templates/controller.rb

module V1
  module Public
    class <%= class_name.pluralize %>Controller < ApplicationController

    end
  end
end

# lib/generators/custom_scaffold/templates/migration.rb
# Ignore for now

# lib/generators/custom_scaffold/templates/serializer.rb
# Ignore for now

# lib/generators/custom_scaffold/templates/fixture.yml
# Ignore for now

# lib/generators/custom_scaffold/templates/request_test.rb
# Ignore for now

# Gemfile

source 'https://rubygems.org'

ruby '2.4.1'
gem 'rails', '~> 5.1.6'
gem 'puma', '~> 3.7'
gem 'pg'
gem 'rack-cors', require: 'rack/cors'
gem 'olive_branch'
gem 'fast_jsonapi'
gem 'awesome_print'
gem 'byebug', '~> 10.0', groups: %i[development test]
gem 'yaml_db'

group :development do
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'mina', '~> 1.2', require: false
  gem 'mina-puma', require: false
  gem 'rubocop', require: false
  gem 'annotate', require: false
end

Answer: You need to specify the file as a Thor template. Rails uses Thor templates for generating templates with ERB style code inside them.
Replace:
copy_file 'model.rb', "app/models/#{file_name}.rb"
With:
template 'model.rb.tt', "app/models/#{file_name}.rb"
By adding the .tt extension you're telling the generator to handle the file as a Thor template, which will interpret the Ruby code (ERB style) inside the file and then create a file with that same name minus the .tt extension. Any file you have without the .tt extension the generator will copy wholesale, without executing any of the code inside.
A useful tip: Sometimes you want to leave some ERB code inside a Thor template file without it being executed. By default any ERB style tags inside a .tt file will be process and in it's place a string will be written to the output file. You can avoid the processing of ERB tags but using a double percent sign in the tag.
For example, lets say you have a file named foo.erb.tt, which will create the file foo.erb when your generator runs. Let's also say we have a article_name variable and it's value is Breaking News
If you put <%= article_name %> in the file it will write Breaking News to the foo.erb.
If you put <%%= article_name %> (notice the %%) it will write <%= article_name %>to the foo.erb.
I found the following reference handy when learning this stuff.

Rails Application Templates article at Rails Guides.
Creating and Customizing Rails Generators & Templates article at Rails Guides.
Thor Actions Docs these are the commands used in our template.rb file.

Thor comes with several actions that help with script and generator tasks. You might be familiar with them since some came from Rails Templates. They are: say, ask, yes?, no?, add_file, remove_file, copy_file, template, directory, inside, run, inject_into_file and a couple more.
Comment: I wish I could give you more upvotes. Using `template` and adding `.tt` works! Thanks!
Comment: I also  much appreciate the extra tip and documentation links. Just read through.


Ubuntu Firewall blocking DNS requests to VMWARE guest with outbound traffic rules added and stateful matching done

Question: This is a unique problem related to DNS queries(tcp/udp 53) even with correct UFW/iptables rules and does not relate to /etc/resolv.conf
I can ping google DNS 22.214.171.124 but cannot resolve google.com after turning on the host ufw.
Tried searching everywhere and updated ufw and iptables accordingly to allow DNS queries and allow established connection.
My numbered ufw status related to DNS is
Screenshot UFW rules
Stateful matching
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
Note: vmnet 1/192.168.191.1/24, 192.168.191.130(guest), eno4 Host public interface.
The strange thing is when I turn off the host ufw, everything works perfectly on guests side. However, I am able to ping any IP addresses even after turning on the firewall.
sudo ufw status numbered | grep 53
FAILED TCPDUMP on UDP/TCP 53 only after pinging google.com via 220.127.116.11
Screenshot (UFW on host)
Note: UFW disabled on guest
Comment: This question appears to be off topic. Home and end user computing questions can be asked on [su].
Answer: After a lot of help from serverfault.com :P and an encouragement boost from Michael Hampton, I finally solved the missed ack flags for DNS queries while everything else worked perfectly.
The real problem was:
-systemd-resolved somehow did not allow DNS queries from any other IPs except 127.0.0.1 after turning on UFW
The solution is to mangle every output from the guest VM by setting mark to 2
iptables -t mangle -A OUTPUT ! -s 192.168.191.130-j MARK --set-mark 2
where 192.168.191.130 is the cluster where you would not want to block UDP from.
CHEERS!!


Building libcurl using MSYS

Question: I have a makefile which builds libcurl static library to use in my project (works perfectly on Linux).
I'm trying to build libcurl on Windows 7 using MSYS but I am struggling.
After solving the 'errors' in the configure script the make command was run and I have received this error:
file.c:569: error: too many arguments to function 'Curl_pgrsSetDownloadCounter'

at which point the make file terminates.
Comment: Are there any warnings? Did you make changes to the code or did you just do the standard ./configure && make? Which version(s) of curl and MSYS? I compiled curl-7.33 on MSYS recently and had no trouble. I just checked and this function does indeed take two arguments so something else might be going wrong in the build.
Comment: Well there were a bunch of other errors which I didn't include (only the error preceding the termination). I'm using CURL 7.34 and MSYS 1.0. I use ./configure --disabled-shared.
Comment: Hmm. Like you I can build libcurl fine using MSYS but I get those errors when running make recursively from my makefile.
Answer: Assuming you download MinGW/MSYS from here:
https://sourceforge.net/projects/mingw/files/latest/download?source=files
Install MinGW/MSYS and make sure you edit msys/etc/fstab to add the following line (with tab between two paths):
C:/<path_to_MinGW>      /mingw

Then create using notepad or notepad++ an ANSI text file called build.sh and paste the following code in it:
(you will need to change the two paths on the top of the file to reflect your folders; also in your case NASM is not needed but I suggest you keep it)
#!/bin/sh

# ==================================================================================================================================
#      _____      __                 ______           _                                       __
#     / ___/___  / /___  ______     / ____/___ _   __(_)________  ____  ____ ___  ___  ____  / /_
#     \__ \/ _ \/ __/ / / / __ \   / __/ / __ \ | / / / ___/ __ \/ __ \/ __ `__ \/ _ \/ __ \/ __/
#    ___/ /  __/ /_/ /_/ / /_/ /  / /___/ / / / |/ / / /  / /_/ / / / / / / / / /  __/ / / / /_
#   /____/\___/\__/\__,_/ .___/  /_____/_/ /_/|___/_/_/   \____/_/ /_/_/ /_/ /_/\___/_/ /_/\__/
#                      /_/
# ==================================================================================================================================

# make sure you edit msys/etc/fstab
# add: C:/<path_to_MinGW>       /mingw

LIBRARY_SOURCES="/d/MyLibs"
NASM="/d/mobileFX/Projects/Software/Coconut/IDE/Studio/etc/3rd_party/nasm/nasm.exe"

cd $LIBRARY_SOURCES

# ==================================================================================================================================
#      __  ____  _ __
#     / / / / /_(_) /____
#    / / / / __/ / / ___/
#   / /_/ / /_/ / (__  )
#   \____/\__/_/_/____/
#
# ==================================================================================================================================

function move_head
{
    count=`ls $1 | wc -l`
    if [ "$count" -eq "1" ]
    then
        t=`ls $1/* -1 -d`
        for s in `ls -d -1 $1/**/*`
        do
            mv $s $1
        done
        rm -rf $t
    fi
}

# ==================================================================================================================================
#      __  __          __      __          __  ____       _______       __
#     / / / /___  ____/ /___ _/ /____     /  |/  (_)___  / ____/ |     / /
#    / / / / __ \/ __  / __ `/ __/ _ \   / /|_/ / / __ \/ / __ | | /| / /
#   / /_/ / /_/ / /_/ / /_/ / /_/  __/  / /  / / / / / / /_/ / | |/ |/ /
#   \____/ .___/\__,_/\__,_/\__/\___/  /_/  /_/_/_/ /_/\____/  |__/|__/
#       /_/
# ==================================================================================================================================

function update_mingw
{
    mingw-get install msys-wget-bin
    mingw-get install libtool
    mingw-get install unzip

    # Download package config and copy executable to mingw/bin
    wget -c http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/pkg-config_0.23-3_win32.zip
    7z x ./pkg-config_0.23-3_win32.zip -o./pkg-config -y >& /dev/null
    rm -f ./pkg-config_0.23-3_win32.zip
    cp ./pkg-config/bin/pkg-config.exe $MINGW/bin/
    rm -rf ./pkg-config

    # Download glib
    wget -c http://ftp.gnome.org/pub/gnome/binaries/win32/glib/2.28/glib_2.28.1-1_win32.zip
    7z x ./glib_2.28.1-1_win32.zip -o./glib -y >& /dev/null
    rm -f ./glib_2.28.1-1_win32.zip
    cp ./glib/bin/libglib-2.0-0.dll $MINGW/bin/
    rm -rf ./glib

    # Fixes pkg-config
    curl -L -s -o gettext-runtime_0.18.1.1-2_win32.zip "http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/gettext-runtime_0.18.1.1-2_win32.zip"
    7z x ./gettext-runtime_0.18.1.1-2_win32.zip -o./gettext -y >& /dev/null
    rm -f ./gettext-runtime_0.18.1.1-2_win32.zip
    cp ./gettext/bin/intl.dll $MINGW/bin/
    rm -rf ./gettext

    # NASM
    rm -rf $NASM >& /dev/null
    curl -L -s -o nasm-2.07-win32.zip "https://sourceforge.net/projects/nasm/files/Win32%20binaries/2.07/nasm-2.07-win32.zip/download"
    7z x ./nasm-2.07-win32.zip -o$NASM -y >& /dev/null
    rm -f nasm-2.07-win32.zip
    t=`ls $NASM/* -1 -d`
    for i in `ls -d -1 $NASM/**/*`; do mv $i $NASM; done
    rm -rf $t

}

# ==================================================================================================================================
#       ____                      __                __   __    _ __                    _
#      / __ \____ _      ______  / /___  ____ _____/ /  / /   (_) /_  _________ ______(_)__  _____
#     / / / / __ \ | /| / / __ \/ / __ \/ __ `/ __  /  / /   / / __ \/ ___/ __ `/ ___/ / _ \/ ___/
#    / /_/ / /_/ / |/ |/ / / / / / /_/ / /_/ / /_/ /  / /___/ / /_/ / /  / /_/ / /  / /  __(__  )
#   /_____/\____/|__/|__/_/ /_/_/\____/\__,_/\__,_/  /_____/_/_.___/_/   \__,_/_/  /_/\___/____/
#
# ==================================================================================================================================

function download_libraries_sources
{
    # IMPORTANT: Make sure you download files as <library_name>.tar.gz

    echo "Download common libraries (stable versions 21 Mar 2016), please wait..."

    curl -L -o curl.zip                 "http://curl.haxx.se/download/curl-7.47.1.zip"
    curl -L -o openssl.tar.gz           "https://www.openssl.org/source/openssl-1.0.2g.tar.gz"
}

# ==================================================================================================================================
#       ______     __                  __     ___ __
#      / ____/  __/ /__________ ______/ /_   / (_) /_  _________ ________  __   _________  __  _______________  _____
#     / __/ | |/_/ __/ ___/ __ `/ ___/ __/  / / / __ \/ ___/ __ `/ ___/ / / /  / ___/ __ \/ / / / ___/ ___/ _ \/ ___/
#    / /____>  </ /_/ /  / /_/ / /__/ /_   / / / /_/ / /  / /_/ / /  / /_/ /  (__  ) /_/ / /_/ / /  / /__/  __(__  )
#   /_____/_/|_|\__/_/   \__,_/\___/\__/  /_/_/_.___/_/   \__,_/_/   \__, /  /____/\____/\__,_/_/   \___/\___/____/
#                                                                   /____/
# ==================================================================================================================================

function delete_sources
{
    # Delete all library foldes
    echo "Deleting old sources, please wait..."
    for i in `ls -d */`; do
        t=$(basename "$i")
        echo "+ deleting $t, please wait..."
        rm -rf $t >& /dev/null
    done
}

function extract_library_sources
{
    # Save sources
    mkdir ./.src >& /dev/null

    # Extract all library sources
    echo "Extracting sources, please wait..."

    for i in `ls *.zip`; do
        echo + extracting $i...
        7z x $i -o$(basename "$i" .zip) >& /dev/null
        mv $i ./.src
    done

    for i in `ls *.gz *.xz *.bz2`; do
        n=`( tar -tf "$i" | head -1 | cut -f1 -d"/" ) 2>/dev/null`
        t=$(basename "$i" .tar.gz)
        t=$(basename "$t" .tar.xz)
        t=$(basename "$t" .tar.bz2)
        echo "+ extracting $i to $t ($n) ..."
        tar xf $i >& /dev/null
        mv $n $t
        mv $i ./.src
    done

    # Detect master folders within library root and move them to root
    for i in `ls -d */`; do move_head $i; done
}

# ==================================================================================================================================
#      ______                      _ __        __    _ __                    _
#     / ____/___  ____ ___  ____  (_) /__     / /   (_) /_  _________ ______(_)__  _____
#    / /   / __ \/ __ `__ \/ __ \/ / / _ \   / /   / / __ \/ ___/ __ `/ ___/ / _ \/ ___/
#   / /___/ /_/ / / / / / / /_/ / / /  __/  / /___/ / /_/ / /  / /_/ / /  / /  __(__  )
#   \____/\____/_/ /_/ /_/ .___/_/_/\___/  /_____/_/_.___/_/   \__,_/_/  /_/\___/____/
#                       /_/
# ==================================================================================================================================

function compile_libraries
{
export "INCLUDE_PATH=/usr/local/include"
export "LIBRARY_PATH=/usr/local/lib"
export "BINARY_PATH=/usr/local/bin"

export "CFLAGS=-I/usr/local/include"
export "CPPFLAGS=-I/usr/local/include"
export "LDFLAGS=-L/usr/local/lib"

export "PKG_CONFIG_PATH=/usr/local/lib/pkgconfig"

# ==================================================================================================================================
# openssl - OK
# OpenSSL is a well known standard library supporting SSL, i.e. the encrypted HTTPS web protocol.
# Depends on: nothing
# Required by: libcurl
# ==================================================================================================================================

cd $LIBRARY_SOURCES/openssl
./Configure mingw --prefix=/usr/local shared
make depend
make
make install

# ==================================================================================================================================
# libcurl - OK
# libcurl is a well known library supporting URLs (networking, web protocols)
# Depends on: libz, OpenSSL
# Required by: XMLHttp
# ==================================================================================================================================

cd $LIBRARY_SOURCES/curl
./configure --prefix=/usr/local --enable-shared=no --with-zlib=/usr/local
make
make install-strip
make distclean
./configure --prefix=/usr/local --enable-shared=yes --with-zlib=/usr/local
make
make install-strip

}

update_mingw
download_libraries_sources
delete_sources
extract_library_sources
compile_libraries

Normally your could use this script to build any C/C++ library using MinGW/MSYS on Windows, such as cairo, glew, jpeg, libjpeg-turbo, libpng, libxml2, sqlite, zlib, freetype, fontconfig, etc.

Building libcurl using MSYS


Feeding arguments into Views

Question: I'm not sure how to frame this question, so I'm sorry if the answer already exists.
Basically, I want to know how to modify the way arguments are fed into views. For example, say I'm presenting a income statement and balancesheet, it seems like its formatting the links as www.example.com/balancesheet/ibm and www.example.com/incomestatement/ibm
Is there a way to make it www.example.com/ibm/balancesheet or www.example.com/ibm/incomestatement, so I can create views and have them grouped this way?
My problem is instead of grouping content by views, I want to try to group views by content so if someone searches for IBM they basically get all the views that apply.  I'm open to any suggestions anyone has.
Answer: Each item between the slash after the domain name of the website is called an argument. Take, for example, example.com/one/two/three/four/five -- in this case one is the first argument, two is the second argument, ect.. Drupal has a function that separates these arguments named arg() which takes an argument as a number. arg(1) when called while loading this example URL will always return 'one' while arg(5) will always return 'five'. This way context can be passed into all the modules in a consistent way. I just want to make sure you see arguments for what they are.
Here is why. You are creating a two separate pages for items that should be on the same page. Instead of creating pages you might consider blocks in views and place the many blocks on the same page. This way you can, for example, use taxonomy terms to tag content as IBM, Intel or Apple. Create a balance sheet and tag it IBM. Create another tag it Apple. Create an income statement and tag it IBM. Create another and tag it Apple.
Then create a basic page with a brief description of the IBM corporation and a logo using the path alias module to change the name of the URL to example.com/ibm. Then add the two views as blocks to the page. Each view will be called when the user goes to example.com/ibm and arg(1) will be ibm. Then you can use the arguments (in View 3 contextual filters) to use this argument to define the Taxonomy term to filter the information in the database with. In this case IBM. Now you might have 12 reports for IBM for each category going back 4 years. Views will pull them all and you can use a field of title that links to the node to have a link going to each one. Then you can also use the date filter to make sure that the most recent one is on top. Once the View block is created, Views will just load the other view using once again arg(1) to define the taxonomy term to find all the other reports information in the database.
Now there are more advanced ways to trigger which Views get called with modules like Context or Panels which are super high powered tools that focus on just this simple purpose.  You might be interested in the videos on Panels at Node One or the Mustard Seed Media video on the Context module.
(edit) To create a similar tabbed look like companies on the WSJ you can use either Quick Tabs for an ajax tabbed effect or creating a Views page and assigning it a Menu Tab for the menu item so that several tabs are found on the same page.
Comment: @lostsol, you only need to create one template which has the company name as the argument. I'm working on a similar sounding project in the uk so mail me if you want some more general collaboration.
Comment: Hey Thanks so much Adam.  I'll look into this, not 100% sure if it'll work as my firm has about 15 years worth of company research that we're trying to post online.  Creating individual first pages for each company would cause alot of extra work for me, plus they are adding companies all the time I wouldn't want them to have to get IT's help everytime they finish researching a new firm.  But we want to eventually get community feedback around it so I might have to go this route(since I'm not sure if I can add comments to views).
Comment: Well, I just made up a very arbitrary example. Building Views and using arguments and relationships is an art form so it takes practice. You would of course do something such as make a content type that functions as the base for each company. Then you would use one of the many different solutions for importing information into a Drupal database to create as many of the pages for each company that you need. If you have a lot of data in spreadsheets you might want to look at the Data module with the Table Wizard. This way you can have tabular data that is also made available to views.
Comment: Awesome thank you Jeremy. I'll get cracking on this over the weekend and reach out to you.  I should hopefully be done but I think I've gone through almost every problem possible so maybe I can help and I'm sure you'll see flaws in my approach and have improvements suggestions.  Thanks so much.
Comment: Thanks Adam.  I am using data to access an external database which has all the data in it. Table Wizard was easier but didn't seem to do the joins correctly..I think presending the data is good now just looking for cool ways to present it.  I'm going to work on your suggestions today and let you know the result.  Thanks as always for your suggestions and insight.

Feeding arguments into Views


Como usar select em uma tabela que não está relacionada?

Question:
Preciso construir uma consulta SQL que liste o
valor a pagar por cada km, a quantidade
de km utilizado e o valor total a pagar por
cada locação
É um exercicio de SELECT e inner/left join mas não sei como relacionar LOCACAO com TABELA.
Obrigado!
Comment: Mas a tabela está relacionada. `LOCACAO > VEICULO > TABELA_VEICULO > TABELA`. Basta entender como cada uma se relaciona e fazer teus JOINS.
Answer: Vou tentar te ajudar pelo o que eu entendi da sua pergunta. Em banco de dados temos o conceito de chave primaria e chave estrangeira, chave primaria é um identificador único de um registro de uma tabela. Já a chave estrangeira é uma referencia a chave primaria de outra tabela. Tendo uma noção básica disso você pode relacionar tabelas diferentes através das chaves primarias e chaves estrangeiras. E para ajudar com a parte dos inner joins vou deixar o link do w3schools https://www.w3schools.com/sql/sql_join_inner.asp


not able to set readonly property of textboxes in a panel in windows form to true in vb.net

Question: i want to enable the read only property of all the text boxes in a panel in windows forms to true  during form load but  its not working. i am using the below code.when i am debugging the code it skips that part.not sure why??
The below code does not work,it skips that part as if there are no text box controls in the particular panel.
Private Sub lockgroupcontrols()
    For Each TextBox As TextBox In Pnltransaction.Controls.OfType(Of TextBox)()

        TextBox.ReadOnly = True
    Next
End Sub

Answer: Your code seems correct, so, I can only imagine that, if it skips the For Each, then you don't have any textboxes inside the PnlTransaction panel.
Sometimes this happens when you draw the Panel over preexisting textboxes. You think the controls are inside the panel, but in reality they are under the panel and you see them because the panel background is transparent.
Try to move the panel on a different place, drag&drop the textboxes over the panel and then reposition the panel.
Comment: perfect steve. i was about to answer my own question.when i checked it.the controls where not in the panel.thanks.i had drawn it over the existing controls
Comment: hi guys,if the above code is not working in your vb.net code then make sure to check that whether the container has all the controls inside it.
Answer: You shouldn't use TextBox as name for your vars
   For Each **TextBox** As **TextBox** In

try it with for ex.
   ...For Each **tBox** As **TextBox** In
         tBox.ReadOnly = True....


Best option to cluster variables (not cases) in R

Question: In SPSS, if I use the hierarchical clustering procedure, I have the ability to cluster both variables and cases using a variety of methods and distance measures.  For this task, I would like to use R to cluster my variables.
For context, my data come from a survey and the respondents were able to select multiple items from a block of options. In my datafile the data are coded as 0,1.
I am trying to learn R so any help you can provide will be greatly appreciated.
Comment: How many respondents and items do you have? Can we assume there are different blocks of items, or do you put all items into the same box, so to speak? (This can help to figure out the best clustering solution.)
Comment: I'm not sure I understand how your dataset looks like. But you may want to transpose your data matrix (turning variables into cases and cases into variables) and run clustering. Would you find it useful to see how to do this in R?
Comment: Does this relate to [biclustering](http://en.wikipedia.org/wiki/Biclustering)? That's a method for clustering both variables and cases simultaneously. See also [this previous Q](http://stats.stackexchange.com/questions/7419/getting-started-with-biclustering).
Answer: Hi
Take a look at the varclus function in the Frank Harrel's Hmisc package.
<code>require(Hmisc)
? varclus
</code>


how to display same polygon on 2 maps present in different divs in the same page

Question: I am using Google maps api V3.
I have a polygonpoints array as like in this Link's variable 'triangleCoords'.
I am generating maps dynamically using a button click..
I want to place the same polygon to be included in all maps.
As i am trying, the Polygon only displayed on the latest Map that is generated.
I want the Polygons to be like as in the Image Link
Answer: You can only add a google.maps.Polygon to one map.  If you have multiple maps, you need to make one for each.
<code>function initialize() {
    var myLatLng = new google.maps.LatLng(24.886436490787712, -70.2685546875);
    var mapOptions = {
      zoom: 4,
      center: myLatLng,
      mapTypeId: google.maps.MapTypeId.TERRAIN
    };

    var bermudaTriangle;

    var map = new google.maps.Map(document.getElementById('map-canvas'),
        mapOptions);
    var map2 = new google.maps.Map(document.getElementById('map-canvas2'),
        mapOptions);
    var map3 = new google.maps.Map(document.getElementById('map-canvas3'),
        mapOptions);

    var triangleCoords = [
        new google.maps.LatLng(25.774252, -80.190262),
        new google.maps.LatLng(18.466465, -66.118292),
        new google.maps.LatLng(32.321384, -64.75737),
        new google.maps.LatLng(25.774252, -80.190262)
    ];

    // Construct the polygon
    var bermudaTriangle1 = new google.maps.Polygon({
      paths: triangleCoords,
      strokeColor: '#FF0000',
      strokeOpacity: 0.8,
      strokeWeight: 2,
      fillColor: '#FF0000',
      fillOpacity: 0.35
    });
    var bermudaTriangle2 = new google.maps.Polygon({
      map: map2,
      paths: triangleCoords,
      strokeColor: '#FF0000',
      strokeOpacity: 0.8,
      strokeWeight: 2,
      fillColor: '#FF0000',
      fillOpacity: 0.35
    });
    var bermudaTriangle3 = new google.maps.Polygon({
      map: map3,
      paths: triangleCoords,
      strokeColor: '#FF0000',
      strokeOpacity: 0.8,
      strokeWeight: 2,
      fillColor: '#FF0000',
      fillOpacity: 0.35
    });

    bermudaTriangle1.setMap(map);
  }
</code>
example


What is the value of real positive $a$ for which $\ln x= ax^4$ has exactly one real solution?

Question: $\ln x = ax^4$
I found a solution using graph. Another method exists, eventually using analysis?
Answer: We can rearrange the equation to say that we're looking for an $a$ such that
$$
\frac{\ln(x)}{x^4} = a
$$
has exactly one solution.  In other words: if we look at the graph of $y = f(x) = \ln(x)/x^4$, we are looking for the $y$-values that are attained exactly once.
In order to get the important features the graph of $f(x)$, it suffices to observe that:

$\lim_{x \to 0^+} f(x) = -\infty$
$\lim_{x \to \infty} f(x) = 0$
$f(x)$ is increasing on $(0,e^{1/4})$ and decreasing on $(e^{1/4},\infty)$.

These facts can be verified by the usual "intro calculus" methods.
Since $f$ attains a maximum at $x = e^{1/4}$, we see that $f(e^{1/4}) = \frac 1{4e}$ is the only positive value of $a$ that is attained exactly once.
Comment: Very interesting idea!!! Thank you very much!
Answer: Take $f(x) = ax^4 - \ln x$.  We want that to equal $0$ exactly once.
Informally in terms of graphs that means $f(x)$ "touches" $y=0$ once but does not "cross" it.  So $f(x) = 0$ is a local maximum or minimum.
So we need an $a$ and an $x$ so that $f(x) = 0$ and $f'(x) = 0$.
So $f'(x) = 4ax^3 - \frac 1x = 0$
$4ax^4 - 1 = 0$
$ax^4 = \frac 14$
$x = \pm (\frac 1{4a})^{\frac 14}$  So $a > 0$.
We also have $ax^4 - \ln x = 0$
So $\ln x = ax^4 = \frac 14$
So $x = e^{\frac 14}$ and $x = \pm (\frac 1{4a})^{\frac 14}$  (we can note that this means that $x$ is positive but that's not really that important as it is the $a$ we are interested in.)
So $x^4 = e = \frac 1{4a}$ and $a = \frac 1{4e}$
Answer: Set $ax^4-\ln x$ and its derivative equal to $0$, so the root is a turning point. You get $4ax^3=1/x$ and $\ln x=ax^4=\frac{1}{4}$, so $a=\frac{1}{4}(\exp\frac{1}{4})^{-4}=\frac{1}{4e}$.
Comment: Why ln x = ax^4 = 1/4  ?
Comment: multiply both sides of 4ax^3=1/x by x/4
Answer: As $x \to 0+$, $\ln(x) \to -\infty$ while $a x^4 \to 0$ so  $\ln(x) < a x^4$ for
$x$ sufficiently small.  On the other hand, $ \ln(x) < a x^4$ for $x$ sufficiently large if $a > 0$.  To have a single solution, you'll want the two graphs tangent.  Since $\ln(x) - a x^4$ is strictly concave, this implies there is only one solution.


Schur test and it's relation to representation theory

Question: I was told by analyst who doesn't know about such things that Schur's test relating to boundedness of integral operators is somehow a version of Schur's lemma on irreducible representations, the group being $\mathbb R$. The two statements look very different to me (although there seem to be many things named after Schur and I have little experience in rep theory). Does anyone know the details of this connection?
Answer: It is true that there are many things named after I. Schur, and they are not necessarily otherwise related! :)
I can't quite guess what your colleague had in mind, but it is true that in many fortuitous circumstances relevant operators are given as integral operators, and if the two-variable "kernel" $K(x,y)$ of such an operator is in $L^2$ of the product space $X\times X$, then it is not only "bounded" (really, continuous, which is more explanatory), but is a "Hilbert-Schmidt" operator, which is an easily-testable case of "compact operator". Self-adjoint operators are our best friends, because their spectral theory is as close to finite-dimensional spectral theory of self-adjoint operators as one could reasonable hope: orthogonal basis of eigenvectors.
Thus, compact topological groups (not $\mathbb R$...) turn out to have the nice property that all Hilbert-space repns decompose as direct sums of irreducibles.
Already the real line $G=\mathbb R$ does not behave this well: $L^2(G)$ does not decompose as a direct sum of irreducible repns of $G$. Rather, Fourier inversion shows that it is a "direct integral" of irreducibles, each generated by the (not $L^2$!!!) functions $x\to e^{i\xi x}$.
Comment: Meant to say "self adjoint COMPACT" operators on Hilbert spaces...


Why does $1+\omega=\omega$ with ordinal addition

Question: In my notes, it says that $1+\omega=\omega \neq \omega+1$.
I understand why the two expressions are not equal, but why does this equality holds:
$$1+\omega = \cup\{1+n:n\in\omega\}=\omega$$
Answer: The definition of ordinal addition says that when $\gamma$ is a limit ordinal, $\alpha+\gamma = \sup_{\beta<\gamma} (\alpha+\beta)$.
Since $\omega$ is a limit ordinal, $1 + \omega = \sup_{n<\omega} (1 + n)$.
The set $\{1 + n\mid n<\omega\}$ is the set of all finite ordinals (except $0$), so its supremum is $\omega$.

In case it helps: You can also characterize ordinal addition as follows. If $\alpha$ and $\beta$ are ordinals, then in particular they are linearly ordered sets, and you can look at the new linear order with domain $\alpha \sqcup \beta$ ($\sqcup$ denotes disjoint union) in which every element of $\alpha$ is less than every element of $\beta$. The new linear order is still a well-order, and its order type is exactly the ordinal $\alpha+\beta$.
So you can picture $1 + \omega$ as taking $\omega$ and adding a new least element (putting $1$ to the left of $\omega$). You should easily be able to convince yourself that this has the same order type as $\omega$ (draw a picture!). On the other hand, you can picture $\omega + 1$ as taking $\omega$ and adding a new greatest element (putting $1$ to the right of $\omega$). This is different from $\omega$, since $\omega$ has no greatest element.


Eigenvalues of a matrix close to a tridiagonal Toeplitz matrix

Question:
I am trying to find all the eigenvalues of $P$ defined below:

$$P=\begin{bmatrix}
0.5&0.5&0&0&\cdots 0&0 \\
0.25&0.5&0.25&0&\cdots 0&0\\
0&0.25&0.5&0.25&\cdots 0&0\\
\vdots \\
0&0&0&0&\cdots 0.5&0.5
\end{bmatrix}_{n\times n}$$
So $P$ has $0.5$ along the main diagonal. It has $0.25$ on diagonals above and below the main diagonal except for the first and last row. Hence $P$ is not exactly a Toeplitz matrix.
My attempt: A paper I'm looking at, gives eigenvalues of the following Toeplitz matrix
$$
Q=\begin{bmatrix}
b&a \\
c&b&a \\
&\ddots&\ddots&\ddots \\
&&c&b&a \\
&&&c&b
\end{bmatrix}
$$
as $\lambda_j = b+2\sqrt{ca}\cos\left(\frac{j\pi}{n+1} \right)$
So I'm wondering if I will be able to find eigenvalues of $P$ even though its not a Toeplitz matrix precisely?
Comment: I suspect that [this wikipedia page](https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors_of_the_second_derivative#The_discrete_case) could be helpful
Comment: You have a typo in the eiegenvalues of the Toeplitz matrix, the coefft of the $\cos$ has to be $2\sqrt{ac}$.
Comment: There's clearly an eigenvalue $0$ with multiplicity $1$ and another $1$ of multiplicity $1$.
Comment: @ancientmathematician Thanks, corrected the typo.
Comment: Did u try to look at $Px = \lambda x$ and expand linear equations?
Answer: $\def\a{\alpha}$
Let $u$ be an eigenvector of the matrix $P$ and assume the elements of the eigenvector have the form:
$$
u_k=e^{\alpha k}+ae^{-\alpha k},\tag1
$$
with some parameters $a$ and $\alpha$, which are to be found.
Obviously for all $k=2\dots(n-1)$
$$
(Pu)_k=\frac14\left(e^{\a k}+ae^{-\alpha k}\right)\left(e^\alpha+2+e^{-\alpha}\right)
=\frac14\left(e^\alpha+2+e^{-\alpha}\right)u_k.\tag2$$
Thus it remains only to find such $a$ and $\alpha$ that the equation $(2)$ is satisfied for $k=1$ and $k=n$ as well.
For $k=1$:
$$\begin{align}
&\frac12(e^{\a}+ae^{-\a}+e^{2\a}+ae^{-2\a})=\frac14\left(e^\a+2+e^{-\a}\right)\left(e^\alpha+a e^{-\alpha}\right)\\
&\iff
e^{2\a}+ae^{-2\a}=1+a \iff a=e^{2\a}.\tag{3}
\end{align}$$
For $k=n$:
$$\begin{align}
&\frac12(e^{\a(n-1)}+ae^{-\a(n-1)}+e^{\a n}+ae^{-\a n})
=\frac14\left(e^\a+2+e^{-\a}\right)\left(e^{\a n}+a e^{-\a n}\right)\\
&\iff e^{\a (n-1)}+ae^{-\a(n-1)}=e^{\a (n+1)}+ae^{-\a(n+1)}\\
&\iff e^{\a (n-1)}+ae^{-\a (n-1)}=e^{2\a}e^{\a(n-1)}+ae^{-2\a}e^{-\a(n-1)}\\
&\stackrel{(3)}\iff \left(e^{\a (n-1)}-e^{-\a (n-1)}\right)\left(1-e^{2\a}\right)=0\\
&\iff \a_m=\frac{\pi\,i}{n-1}m,\quad m=0\dots n-1
.\tag{4}
\end{align}$$
The corresponding eigenvalues are:
$$
\lambda_m=\frac14\left(e^{\a_m}+2+e^{-\a_m}\right)=\left(\frac{e^{\frac{\a_m}2}+e^{-\frac{\a_m}2}}2\right)^2=\cos^2\frac{\pi m}{2(n-1)},\quad m=0\dots n-1.\tag5
$$
Since all $n$ eigenvalues are distinct we are done.
Comment: I see, I missed the $i$ in $\alpha_m$ earlier. Thanks.
Comment: @manifolded It is equal, with no doubt.
Comment: Thanks, I'm looking at $(5)$ now, how do you have $\frac{e^{\alpha_m/2}+e^{-\alpha_m/2}}{2}=\cos\left( \frac{\alpha_m}{2} \right)$?
Comment: @manifolded I used: $\frac{e^{ix}+e^{-ix}}2=\cos x$
Comment: I see that $(Pu)_k = \frac{1}{4}(e^{(k-1)\alpha}+ae^{-(k-1)\alpha})+\frac{1}{2}(e^{k\alpha}+ae^{-k\alpha})+\frac{1}{4}(e^{(k+1)\alpha}+ae^{-(k+1)\alpha})$, for $k=2,\cdots,n-1$, by definition of matrix multiplication but that is not equal to $(2)$?
Comment: May I know the intuition behind choosing $u_k=e^{\alpha k}+ae^{-\alpha k}$? Thanks.
Comment: I invented this tool several years ago working on a similar problem. It works whenever $(Au_k)=bu_{k-1}+au_{k}+bu_{k+1}$ for all $k$ except for the boundary. The intuition behind: this looks like a discrete version of the linear differential equation of second order which usually has general solution $A_1e^{\lambda_1x}+A_2e^{\lambda_2x}$.
Answer: (Not a solution)
I numerically found that the answer should be
$$
\lambda_j = \sin^{2}\left(\frac{j\pi}{2(n-1)}\right), \quad 0\leq j \leq n-1
$$
but I only have a vague idea to prove this. If we define $B_n = 2P_n - I_n$, then it is enough to show that the eigenvalues of $B_n$ are
$$
\cos\left(\frac{j\pi}{n-1}\right),\quad 0\leq j\leq n-1.
$$
It seems that the characteristic polynomial of $B_n$ is
$$
\phi_n(x) = \frac{1}{2^{n}}(x^2-1)U_{n-2}(x)
$$
where $U_n(x)$ is $n$-th Chebyshev polynomial of second kind. However, I'm not sure how to prove this, although it seems that using induction might give recurrence formula for $\phi_n$ which resembles that of $U_n$ a lot.
Comment: For the numerical solution, may I know if you found $\lambda_j^{(n)}$ for different $n$ and then conclude $\lambda_j$ is what you have above? Thanks.
Comment: @manifolded Yes, I just plotted the eigenvalues and it reminds me $\sin^2$.
Comment: May I know why it is enough for $B_n$ to have its eigenvalues as $\cos \left(\frac{j\pi}{n-1} \right)$? I didn't understand how $B_n=2P_n-I_n$ implies that?
Comment: @manifolded It follows from $\sin^2(x)=(1-\cos(2x))/2$.
Comment: I see, I thought if $B_n$ has eigenvalues of $\cos\left(\frac{j\pi}{n-1} \right)$, then $\frac{I_n-B_n}{2}$ will have eigenvalues $\sin^2 \left(\frac{j\pi}{2(n-1)} \right)$ as opposed to $\frac{I_n+B_n}{2}$ which is what you have. I maybe wrong?


Practical difference between "sampling" and "re-sampling with replacement"

Question: What is the difference between "sampling" and "re-sampling with replacement" from a practical point of view (I mean how do I code that) ?
When one say: "sampling $\tilde{x}_t^{(i)}$ from $p(x_t | x_{t-1}^{(i)})$", this mean that we generate $\tilde{x}_t^{(i)}$ from the conditional probability distribution $p(x_t | x_{t-1}^{(i)})$, so this is the next random value to generate from a uniform or a gaussian distribution for example, right ?
Now, when one say: "re-sampling with replacement N values $\{x_t^{(i)}; i=1,\dots,N\}$ from the set $\{\tilde{x}_t^{(i)}; i=1,\dots,N\}$ according to the associated weights/probabilities $\{w_i; i=1,\dots,N\}$". Does this mean that we choose each value $\tilde{x}_t^{(i)}$ according to the probability $w_i$, and repeat this until we get N values ? So how do you do that practically in a pseudo code ?
EDIT:
I understand that re-sampling here means reselecting the values from the weighted set, by choose randomly, but according to the weights. So if we had three values, with weights 0.99, 0.005 and 0.005, then the most likely outcome of resampling would be three values of the first type. But how do I code this in a general case ?
Comment: Yes to both questions. For the first one, you need to be able to simulate from the conditional distribution, which is known. For the second one, you might be interested on the command R [command](http://stat.ethz.ch/R-manual/R-patched/library/base/html/sample.html) `sample` in order to implement the re-sampling.
Comment: @Procrastinator, that comment (with some more detail) has the makings of a nice answer, if you have time to give one
Comment: Actually, I believe you need to be careful in your terminology. You can sample from a set of data, both with and without replacement. You can also sample from a (parametric) distribution. I believe "re-sampling" is asociated with "bootstrapping", which might be a good place to look.
Comment: @Wayne Yes, actually I was implementing a bootstrap filter, and what I was looking for in the second point was "re-sampling with replacement"
Comment: @Procrastinator Well, I'm not using R, but I want to code that by my self in C++ or python. So for the first one, say that the conditional distribution is the standard gaussian distribution, are the generated samples $\tilde{x}_t^{(i)}$ some values in [0,1] (i.e. probabilities according to normal distribution), or they are any random values that can be generated according to a normal distribution ?

For the second one, can you give a pseudo code algorithm to do this resampling of N values according to the weighted set of N values that we had ? Since I'm not using R.

Thanks.
Answer: I am going to try to expand my previous comment and correct some notation (let me know if I am mistaken). (Thanks for the suggestion @Macro)
1. In this case you need to know explicitely what $p(x_j\vert x_{j-1})$ means (Note that here $x_{j-1}$ is a known value and this distribution depends on this value somehow). An example of this is
$$x_j\sim \mbox{Normal}(x_{j−1},1).$$
In this case, in order to sample from $x_j$, you just need to sample from the a normal distribution with mean $x_{j−1}$ and variance $1$. For other examples you can check the wikipedia page for Markov chains or check other transition kernels.
2. A pseudocode for obtaining a sample of size $M$ with replacement is as follows

Starting with a known sample  $\{x_i,i=1,...,N\}$, repeat $M$ times
(i). Consider partitioning the interval $(0,1)$ into $N$ subintervals $I_1=(0,w_1)$,
      $I_2=(w_1,w_1+w_2)$,..., $I_N = (\sum_{j=1}^{N-1}w_j,1)$.
(ii). Simulate $u\sim U(0,1)$.
(iii). Identify the interval $I_j$, $j\in\{1,...,N\}$, such that $u\in I_j$.
(iv). Take the sample $x_j$.

As you can see, there is a difference between these two methods.
In the first one you have a model, $p(x_{j}|x_{j−1})$ ,  and you simulate from it. Starting from, say $x_0$, the next value $x_1$ is sampled from $p(x_1|x_0)$; then the second value  $x_2$ is sampled from $p(x_2|x_1)$ and so forth. For this purpose you need to be able to simulate from the conditional distributions $p(\cdot\vert\cdot)$. This is, you obtain a sample $\{x_i,i=1,...,N\}$
In the second sampling method (re-sampling) you already have a sample $\{x_i,i=1,...,N\}$ and you obtain a new sample by picking elements from the original one. This is similar to the 'urn game'. Imagine you have an urn with $N$ elements of different sizes (illustrating the different weights), you pick one of them, record the result and put it back in the urn. This experiment is repeated $M$ times.
I hope this helps.
Comment: Ok, since it's context-dependent, I'll create later a new question about the construction of the conditional distribution in the context that I'm working on. I'll let you know when I do it. Thanks.
Comment: Thanks, this is a great and complete answer. For the transition conditional distribution $p(x_t|x_{t-1})$, is it possible to define this model according to my own problem (for which I want to apply particle filter) instead of using some well known distributions such as $x_t \~ Normal(x_{t-1}, \sigma)$; or in most of cases we generally use some existing distributions (like uniform, normal, etc) ?
Comment: @user995434 The constrution of the conditional distribution is a completely different problem. This has to be done in such a way that the convergence to the objective distribution is ensured. It cannot be any distribution. Please, have a look a this [link](http://en.wikipedia.org/wiki/Particle_filter) for some references.
Comment: Well, so the construction of the conditional distribution according to a given problem, is also one of my concerns, and I have no idea about how to create a convenient conditional distribution. Should we think of this as a function which reflects how the estimated variable x is supposed to evolve over time ?
Comment: @user995434 Possibly but it is context-dependent. I think it would be a better idea that you expand your comment, giving a bit more of details of your context, in a new question. This is, explaining what is the goal of the simulation, what have you done, ...
Answer: Drawing a sample means sampling without replacement from a population.  If you think of this like an urn with distinctly numbered balls in it, it means to take k and each time the urn has one less ball because the number you draw each time is not returned to the urn.  Sampling with replacement means that each time the ball is returned to the urn.  Hence when sampling with replacement you can get repeat numbers for the selected balls. When sampling with replacement each of the N balls has the same 1/N chance of being selected each time.  When sampling without replacement the probability of being drawn changes based on the results of the preceding draws.  Resampling is term used for a set of statistical techniques.  With resampling you have an original sample drawn and then to make inferences about the population based on the sample you do addition sampling from the original sample.  As an example the bootstrap is a resampling method.  The ordinary bootstrap samples N times with replacement from an original sample of size N.  Although resampling is usually done with equal weights it has been generalized to consider unequal weighting schemes. I am not sure what the original intent of your question was but if my interpretation is correct this description is missing from the other answer.  The bootstrap was mentioned in one of the comments.
Comment: I was in fact implementing the bootstrap filter. What is missing about the resampling in the previous answer ?!
Comment: @user995434 Procrastinator explained the concept of resampling but did not connect it to the inference techniques that are called resampling methods namely the bootstrap, subsampling, and permutation methods.  Also with the bootstrap M is usually set equal to N.  The only exceptions would be to use a smaller value for m because the ordinary bootstrap is inconsistent.
Comment: I'm actually using M equal to N (fixed number of particles). What do you mean by "the ordinary bootstrap is inconsistent" ? Should I use another ameliorated bootstrap technique ? I'm actually using [this algorithm](http://www.aiqus.com/upfiles/PFAlgo.png)
Comment: There are parameters that the ordinary bootstrap does not estimate correctly.  One example estimation of the mean when the variance is infinite.  Bickel and Ren showed that a so-called m out of n bootstrap can give the proper results as long as m goes to inifinty at a slower rate than n.  This is covered in some detail in Chapter 9 "When Bootstrapping Fails Along with Some Remedies for Failures" of my book "Bootstrap Methods: A Guide for Practitioners and Researchers" Second Edition.(2007).
Comment: Here is a link to where it is given on amazon: http://www.amazon.com/Bootstrap-Methods-Practitioners-Researchers-Probability/dp/0471756210/ref=sr_1_1?s=books&ie=UTF8&qid=1339520307&sr=1-1&keywords=bootstrap+methods+a+guide+for+practitioners+and+researchers.
Answer: The practical difference between "sampling" and "sampling with replacement" is really in the accounting part of the probability weights, whether or not you can't have (or don't want) your whole data set in memory, and if sample ordering is important. If you wish to pursue it, Ordering is more related to "random-shuffling", which is out of context here. A dirth of information related to "sampling with replacement" is available in the context of "bootstrapping" and "online algorithms" are a standard remedy for memory constraints, which are also out of context here.
If you simply have $N$ observations, each will have an equal weighting (a uniform distribution) in a single random sample. So here "sampling with replacement" is equivalent to a repeated single random sampling. I typically generate a random index between $0$ and $N-1$, and select that index from my $N$ observations... then repeat $M$ times to get the specified sample size.
So we're all on the same page, you generate a random integer to use as an index as
              RandomIndex = (int)((max - min + 1) * RND + min) where $RND$ is your favorite random number generator, returning a real number (decimal) between $0$ and $1$. Most often the standard function for your environment is sufficient, and here we use $0$ for the min and $N-1$ for the max.
However, you stated that you have frequency weights available for this problem, defined as  $\{w_i; i=1,\dots,N\}$ in your notation; not too different really. We just need to tell our RandomIndex function to use a non-uniform distribution defined by your $\{w_i\}$. This puts a wrapper function around the $RND$ function. I typically use something cheap like
               Wtd.RND = function(weights[]) {
                         R = RND; index = 1; sum = weights[0];
                         while (sum ≤ R) {
                             sum += weights[index]; index++;}
                         return index;
                         }
Plug the wrapper function back in, and simplify some algebra, to get your new weighted index:
               WeightedIndex = (int)(N * Wtd.RND)
Now just select the weighted index from the $N$ observations... then repeat $M$ times to get the specified sample size.
Comment: Well, this a better algorithm than what I've implemented from the post of @Procrastonator, since it seems cheaper. By the way, "bootstrapping" and "online algorithms" are not out of context here, because I'm trying to use particle filters (e.g. bootstrap filter) for an online clustering problem.