Error-checking for permissions and size when reading/writing files

Question: In Android, if I want to copy a file from one place to another:

How do I check that I have the correct Manifest permissions? I know you include the permissions in the manifest, but let's say we don't know that we included it or not -- how do we catch the exception if we lack the permission?
How can I verify that I have read access for the source file?
How can I verify that I have write access for the destination?
How can I verify that there is enough room to perform the write?
How can I verify that the write was performed successfully?
Answer:
how do we catch the exception if we lack the permission?

That varies by permission.

How can I verify that I have read access for the source file?

Call canRead() on the File.

How can I verify that I have write access for the destination?

Call canWrite() on the File.

How can I verify that there is enough room to perform the write?

You can't, insofar as by the time you have made the calculation, the value may have changed due to the behavior of other apps on the device.
You can use methods on StatFs to determine the available space, but I would give yourself a significant buffer.

How can I verify that the write was performed successfully?

Start by calling getFD().sync() on your FileOutputStream, after flush() and before close(), to ensure all bytes are written to disk and are not being cached by the kernel and filesystem implementations. For most cases, that's sufficient.
You are also welcome to read the content back in and verify it against whatever you have in memory, to confirm that the file appears to contain what it should.
Comment: @KaliMa: "Does copying/moving a file require write access?" -- you need to able to write to the destination `File` for a copy. For a rename/move, you need to be able to read and write both the original location and the destination.
Comment: @KaliMa: "In what sense?" -- where and how exceptions are thrown due to missing `READ_CONTACTS` will be substantially different than for `CAMERA`, or `ACCESS_FINE_LOCATION`, etc. "I'm referring to things like (for example) android.permission.WRITE_EXTERNAL_STORAGE or android.permission.READ_EXTERNAL_STORAGE " -- IIRC those will throw `SecurityExceptions` on affected disk I/O.
Comment: @KaliMa: So? You need a `File` object pointing to where you want the file to go. Call `canWrite()` on that `File` object. That is standard Java file I/O, nothing unique to Android.
Comment: @KaliMa: Ah, OK. For not-yet-existent files, I think you need to check write access on the parent directory (in this case, `getFilesDir()`). My apologies for not thinking that through. BTW, you can get rid of `+ ""`.
Comment: @KaliMa: You should be able to call `canWrite()` on a `File` object pointing to the directory (e.g., `getFilesDir().canWrite()`).
Comment: @KaliMa: `getFilesDir()` is a `File`. There is no `File(File)` constructor. In your previous example, you were calling `File(String, String)`, and that is not necessary, as there is a `File(File, String)` constructor.
Comment: I don't understand the canWrite() example. Does copying/moving a file require write access? I was referring more to checking if I have the ability to write to the source medium but this probably ties into (1).
Comment: "That varies by permission." In what sense? I'm referring to things like (for example) android.permission.WRITE_EXTERNAL_STORAGE or android.permission.READ_EXTERNAL_STORAGE
Comment: "you need to able to write to the destination File for a copy" That's what I mean: if I am trying to move a file to a new location, the file does not yet exist in that location
Comment: I ask because if I do File file = new File(this.getFilesDir(), "testfile.txt"); and then output file.canWrite(); it returns false even though it's internal storage
Comment: That is part of my question, though (checking write access of the directory). I was probably unclear: When I asked "How can I verify that I have write access for the destination?" I refer to the directory, not a file already in the directory.
Comment: Seems like I did need the "" for some reason: `File file = new File(this.getFilesDir() + "");` and `file.canWrite();` returns true, but `File file = new File(this.getFilesDir());` doesn't even compile. Apparently the constructor doesn't exist for a single file object input unless a filename is provided?
Comment: I see now at http://developer.android.com/reference/java/io/File.html -- I wonder why they don't have a File(File) constructor. At any rate this discussion has been helpful. Thank you!


ASG replacement with same IP

Question: Summary
When a node fails, or is otherwise removed, how can an ASG be configured to respawn that node with the same private IP as the one it is replacing?
I notice that there is support for allocating an administrator chosen address but I'm not sure how to apply this to ASG's.
Context
We have a service, Consul, that we're trying to make more robust ideally with ASG's since that can automatically respawn a terminated instance (failed or otherwise). The issue is that Consul consists of a set of masters and you can't just replace them with different nodes without explicitly joining the new node to the cluster and cleaning up the now defunct node.
So it seems simpler just to make a failed node get replaced with a fresh version of its former self. That primarily means it having the same private interface IP as before. How can this be accomplished?
More generally - how are people managing clustered services that have some kind of awareness of the participating nodes?
Answer: You could make use of an Elastic IP (EIP).

Start by allocating an EIP that you want to always be connected to the instance in your ASG, this is done using the AWS CLI or through the AWS console (EC2 > Elastic IPs).
When that is done, you should add a script to your AMI that associate the new instance with your allocated EIP. Use the aws ec2 associate-address command available in the AWS CLI tools. This script should be run on boot or whenever your application is ready.

By doing it this way, the new instance will be assigned the same public IP as the old instance had.
Comment: Thanks for the info but we need to IP's to be in a private network; this is not a public facing service. And it would be better if the IP could be an actual interface on the instance that the service can bind to.
Answer: That's not now ASGs are supposed to be used.
Instead, put a load balancer (either ELB or one of your own creation) in front of your ASG nodes and then add/remove back-end nodes from the load balancer as needed.
If you need awareness of nodes participating in your ASG, then you can use the API to enumerate ASG instances.
Comment: We could use some external tool to watch the ASG and reconfigure the cluster as needed however it seemed simpler to just get the same IP assigned (if possible).

The issue with an LB is that the back end nodes speak to each other and track which are present (or not); they are not stateless. If we add/remove nodes enough then the cluster quorum will be lost since we need to explicitly rejoin new nodes and clean up the failed ones.

ASG replacement with same IP


Uniform Convergence of $f_n(x)= \sin(x+\frac 1n)$

Question: Prove that $\sin(x+(1/n)) \rightarrow \sin x$ uniformly on $R$ using Mean Value Theorem.
I'm not sure how to go about this. Because the problem asks to use MVT, I tried taking the derivative of the function in order to find the supremum and use that to show that the supremum was less than some epsilon to prove uniform convergence, but I am having a hard time.
Comment: What did you try? $\cos (x)$ is bounded by $1$ and it is surely sent to $0$ by the $\frac{1}{n}$ term
Comment: Can you think about the picture first before attempting the proof?  What does the graph of $\sin(x + \frac{1}{n})$ look like in comparison to $\sin{x}$?
Answer: For all $x\in \mathbb R$,
$$\sin\left(x +\frac 1n \right) - \sin x = \sin' (y)\left( x +\frac 1n  -x\right)$$
for some $y\in (x, x+\frac 1n)$ by the MVT. Note that the right hand side is always smaller than $\frac{1}{n}$.
Comment: beat me by 10 seconds, sigh +1
Answer: Since $|\sin'(x)| \le 1$ for all $x$, we have
$| \sin(x+ {1\over n})-\sin x| \le {1 \over n}$ for all $x$.
Answer: For all $n \geq 1$, by mean-value theorem we have
$$
\Big| \sin \Big( x + \frac{1}{n} \Big) - \sin x \Big| \leq \Big( x + \frac{1}{n} - x \Big) \sup_{x < t < x + \frac{1}{n}}| \cos t|
\leq
\frac{1}{n}
$$
for all $x \in \mathbb{R}$;
if $\varepsilon > 0$, then $n \geq \Big\lceil \frac{1}{\varepsilon} \Big\rceil + 1$ only if $\frac{1}{n} < \varepsilon$;
we have thus proved this: for every $\varepsilon > 0$, we have $n \geq \Big\lceil \frac{1}{\varepsilon} \Big\rceil + 1$ only if
$$
\Big| \sin \Big( x + \frac{1}{n} \Big) - \sin x \Big| < \varepsilon
$$
for all $x \in \mathbb{R}$


Detect specific tag match in Catch2 at runtime

Question: I have integration tests in my Catch2 project that depend on some expensive global state being set up. I'd like to only initialize that global state when the test runner will actually be testing the systems that depend on it.
What I have appears to work, but it's a little horrifying... it depends on rather a lot of implementation details in the Catch config.
Here's my main:
#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
...

int main(int argc, const char* argv[])
{
    // Construct a fake TestCaseInfo to match against the [integration] tag
    const char * expensive_tag = "integration";
    Catch::SourceLineInfo fake_source_line("?", 0)
    Catch::TestCaseInfo fake_test_case("?", "?", "?", {expensive_tag}, fake_source_line);

    Catch::Session session;
    session.applyCommandLine(argc, argv);
    auto test_spec = session.config().testSpec();
    const bool want_integration_tests = test_spec.matches(fake_test_spec);

    if(want_integration_tests)
    {
        do_expensive_setup();
    }

    return session.run();
}

And then my test file is just:
#include "catch.hpp"
...

TEST_CASE("expensive-to-initialize system", "[.integration]")
{
    REQUIRE(expensive_setup_is_done());

    SECTION("has property 1") { ... }
    SECTION("has property 2") { ... }
    ...
}

Note that because there are multiple sections (and, in my actual project, multiple test cases) that depend on the global setup, I can't just move the initialization into the top of the TEST_CASE.
Is there a better way?
Answer: Just do the initialization on demand, using something like std::call_once:
TEST_CASE("expensive-to-initialize system", "[.integration]")
{
    static std::once_flag init_flag;
    std::call_once(init_flag, do_expensive_setup);

    // ...
}

This will ensure that do_expensive_setup is called one time, but only if needed. If there are multiple places that need this setup, just wrap this in a function.
Note that if do_expensive_setup throws, it will may be called a second time. But once the function successfully exits, that's it.
Comment: @NoSenseEtAl Yep, that works just fine too (modulo the missing parens).
Comment: Oh wow, did not know about `std::call_once()`. That's perfect!
Comment: would static const bool dummy = do_expensive_setup(), false; also work? Some might find it uglier but I do not mind it since I know about how statics work.


Dependency Injection by @Produce annotation in this example

Question: this is a simple number-guess example to demonstrate dependency injection in Java EE 7. What I don't quite understand is the two injected fields in Game.java, "maxNumber" and "randomNumber". I understand that they can be injected because there are two @produce methods in Generator.java, @Produce int next() and @Produce int getMaxNumber().
However, the method name "next()" doesn't match "randumNumber" injected in Game.java. How does this work?
Also, I tried changing "getMaxNumber()" to something else "getMMMaxNumber()" in Generator.java. The program still works fine. So, I really don't understand how these two fields are injected into Game.java from Generator.java.
Please help understand this. Thanks really in advance.
Game.java
package org.jboss.as.quickstarts.numberguess;
@SuppressWarnings("serial")
@Named
@SessionScoped
public class Game implements Serializable {
/**
 * The number that the user needs to guess
 */
private int number;

/**
 * The users latest guess
 */
private int guess;

/**
 * The smallest number guessed so far (so we can track the valid guess range).
 */
private int smallest;

/**
 * The largest number guessed so far
 */
private int biggest;

/**
 * The number of guesses remaining
 */
private int remainingGuesses;

/**
 * The maximum number we should ask them to guess
 */
@Inject
@MaxNumber
private int maxNumber;

/**
 * The random number to guess
 */
@Inject
@Random
Instance<Integer> randomNumber;

public Game() {
}

public int getNumber() {
    return number;
}

public int getGuess() {
    return guess;
}

public void setGuess(int guess) {
    this.guess = guess;
}

public int getSmallest() {
    return smallest;
}

public int getBiggest() {
    return biggest;
}

public int getRemainingGuesses() {
    return remainingGuesses;
}

@PostConstruct
public void reset() {
    this.smallest = 0;
    this.guess = 0;
    this.remainingGuesses = 10;
    this.biggest = maxNumber;
    this.number = randomNumber.get();
}
}

Generator.java
@SuppressWarnings("serial")
@ApplicationScoped
public class Generator implements Serializable {

private java.util.Random random = new java.util.Random(System.currentTimeMillis());

private int maxNumber = 100;

java.util.Random getRandom() {
    return random;
}

@Produces
@Random
int next() {
    // a number between 1 and 100
    return getRandom().nextInt(maxNumber - 1) + 1;
}

@Produces
@MaxNumber
int getMaxNumber() {
    return maxNumber;
}
}

MaxNumber.java
@Target({ TYPE, METHOD, PARAMETER, FIELD })
@Retention(RUNTIME)
@Documented
@Qualifier
public @interface MaxNumber {

}

Random.java
@Target({ TYPE, METHOD, PARAMETER, FIELD })
@Retention(RUNTIME)
@Documented
@Qualifier
public @interface Random {

}

Answer: Injection is not done by method name, but rather the two values are distinguished by their qualifiers @Random and @MaxNumber.  The respective method names of the @Produce-annotated methods are not relevant, they only need to return a matching type.


why isn't the marginal distribution needed when using a conjugate prior?

Question: What is a good explanation as to why you wouldn't have to integrate to find the posterior when you use a conjugate prior. Most examples (like for instance: http://www.youtube.com/watch?v=0XD6C_MQXXE) multiply the prior times the likelihood and end with 'look how nice the (unscaled) posterior is of the same distribution as the prior and we didn't need to integrate'. But they never calculate the marginal distribution and divide by it. That kind of makes the conjugate look like a diversion from the fact that the integration problem still isn't solved at all..so why isn't the marginal distribution needed?
Comment: When the product of prior by likelihood is equal to a standard distribution up to a constant, there only exists one constant that turns this product into the standard distribution. It need not be computed anew each time.
Answer: You can do it more explicitly if the "by inspection" trick bothers you. Suppose that $X\mid\theta\sim\mathrm{Bin}(n,\theta)$ with prior $\mathrm{Beta}(a,b)$. Then, the posterior density is proportional to
$$
  \theta^{x+a-1} (1-\theta)^{n-x+b-1} \, .
$$
The full posterior density is
$$
  \pi(\theta\mid x) = A\;\theta^{x+a-1} (1-\theta)^{n-x+b-1} I_{[0,1]}(\theta) \, ,
$$
in which the normalization constant $A$ must be determined by the condition that the posterior density must integrate to one. Therefore, we have
$$
  1 = \int_{-\infty}^\infty \pi(\theta\mid x) \, d\theta = A \int_0^1 \theta^{x+a-1} (1-\theta)^{n-x+b-1} \, d\theta
$$
$$
  = A \frac{\Gamma(x+a)\Gamma(n-x+b)}{\Gamma(n+a+b)} \int_0^1 \frac{\Gamma(n+a+b)}{\Gamma(x+a)\Gamma(n-x+b)} \theta^{x+a-1} (1-\theta)^{n-x+b-1} \, d\theta \, .
$$
The integral above is equal to one, because you are integrating the density of a $\mathrm{Beta}(x+a,n-x+b)$ random variable. Hence,
$$
  A = \frac{\Gamma(n+a+b)}{\Gamma(x+a)\Gamma(n-x+b)} \, ,
$$
and the full posterior is
$$
\pi(\theta\mid x) = \frac{\Gamma(n+a+b)}{\Gamma(x+a)\Gamma(n-x+b)}\theta^{x+a-1} (1-\theta)^{n-x+b-1} I_{[0,1]}(\theta) \, ,
$$
meaning that $\theta\mid X=x\sim\mathrm{Beta}(x+a,n-x+b)$.
After doing this a couple of times, you will appreciate the shortcut provided by the "by inspection" way.
Comment: Thanks for the elaborate answer, this backward reasoning approach I didn't see before, unfortunately I don't entirely get it. Isn't the part between the '=' and the integral sign, on the fourth formula line a bit artificial? (isn't moving 1/B(x+a,n-x+n) behind the integral sign already like moving the constant A behind the integral sign?)
Answer: Actually the marginal is needed, it is just that the marginal is known and we don't need to integrate to find it.  Think of all the trig identities in the back of the calculus book.  We could re derive them by integration and the tricks used the first time, but if we find an integral that matches one of the identities then we can just look at what the table says and have the solution without the work.  The same works with the conjugate prior times the likelihood, we could find the marginal by integration, but we know that integrals following a certain pattern have a given marginal, so we just plug that in rather than doing the hard work over again.
Answer: This question seems like a mix-up between two situations

Conjugate prior available: The marginal distribution is known and we have it (of course in some situations, you have some integral in there when you talk about some general conjugate prior for a GLM).
No conjugate prior available: The marginal distribution is usually not available analytically, but this does not matter so much if we do MCMC sampling, which only requires the likelihood and the prior density.

Of course people sometimes introduce the general situation (and have in my that for practical problems you'll end up doing MCMC sampling) by starting with conjugate priors. They will then sometimes not go into all the nice properties/analytical solutions/what you can do with mixtures of conjugate priors etc., because in practice you can rarely use these things - even if in the particular situation they would be applicable.


php, mysql, update variable with particular id

Question: So i have this database with registering. And in the backend i want to be able to change their status of payment.
I want a basic textarea where i can paste the ID of the user, and then an okay button to send the method, and then i click okay, i want it to update the users status with the particular id.
Can anyone help how to make connection to the database, where i only update their status if they have that id?
I now the html, i just need the php part.
I don't have any idea on how to do it, so i haven't tried anything yet.
Comment: Please try to use the search; http://stackoverflow.com/questions/23945476/php-sql-update-array
Comment: If you're wondering why you got dowvoted , then consider reading [this](http://stackoverflow.com/help/mcve). We expect your attempt for the problem you're trying to search, provided input, expected output and errors if any. Consider adding this stuff and we will glad to help.
Comment: possible duplicate of [Best way to write PHP SQL Update Statement](http://stackoverflow.com/questions/5433561/best-way-to-write-php-sql-update-statement)
Answer: For database connection, use this:
database.php
<?php
try{
    $db = new PDO('mysql:host=localhost;dbname=your db name', "username" , "password");
    $db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

}catch(PDOException $e){
    print "error in connection" . $e->getMessage();
}

update-Id.php
for udating/changing the value of user Id stored in database
<?php
    require_once 'database.php';
    $userId = $_POST['userId'];

$stmt = $db->prepare("UPDATE tablename SET userId  = ?");
        $stmt->execute(array($userId));


Java: reading objects from file into different derived classes

Question: I have class Employee, and two derived classes Hourly and Salary. Instantiations of those derived classes are placed in the same Array List called employeeList. When I read these objects from file using ois.readObject() how can I ensure the data ends up in the appropriate derived class? Thank you!
Answer: I assume they are extensions of the Employee class ? If you have an array like this:
List<Employee> emps = ...

then you can do something like:
for (Employee e : emps) { if (e instanceof Hourly) ... }

You still need to check the elements. You can also use filter them like this:
emps.stream().filter(e -> e instanceof Hourly).collect(Collectors.toList())

Comment: thank you for your help!
Answer: Note: I am not familiar with ObjectInputStream, so I am not exactly sure if this dynamic dispatch would work here, but I recommend giving this a try:
You could have your employee interface look like this:
interface Employee {
    // Whatever other methods you have defined

    void populateFieldsFromObject(Object obj);
}

Then each subclass would implement this method differently:
class HourlyEmployee implements Employee {

    // Other methods/fields...

    @override
    void populateFieldsFromObject(Object obj) {
        if (!(obj instanceof HourlyEmployee)) {
            throw new IllegalArugmentException("Object not of correct type");
        }
        HourlyEmployee objEmp = (HourlyEmployee)Object;
        this.hourly = objEmp.hourly;
    }
}

class SalaryEmployee implements Employee {

    // Other methods/fields...

    @override
    void populateFieldsFromObject(Object obj) {
        if (!(obj instanceof SalaryEmployee)) {
            throw new IllegalArugmentException("Object not of correct type");
        }
        SalaryEmployee objEmp = (SalaryEmployee)Object;
        this.salary = objEmp.salary;
    }
}

Then you can just iterate through your List of employees, and call:
employee.populateFieldsFromObject(ois.readObject());, as Dynamic Dispatch should automatically determine which method to call based on the type of the object.
Let me know if this works.
Comment: thank you for your help! That's helpful. I ultimately ended up just using print writer because of time constraints for my class assignment, but I will still need to learn this skill so I appreciate  your response. Once I attempt this technique again I'll follow up and let you know how it goes.


Using the definition of $f$ is $O(g)$ proof:

Question: I'm studying for my discrete math class and I don't understand how to prove big O notation. I understand that $f$ is $O(g)$ of another if $f(x) \le c g(x)$ holds.

How would I go about proving $\sin x$ is $O(1)$ using the definition.
Also how do you disprove? Suppose you're asked to disprove that $\frac {x^3+x} {x+1}$ is not $O(x)$ using the definition?

Can you show me the steps and explain why each step was applied in simple terms?
EDIT:  How would you do a question like this $e^x$ is not $O(x^5)$ Big O or/and $1$ is not $O(1/x)$.
Comment: Anyway, $\sin x \le 2 \cdot 1$, so taking $c=2$ will be enough.
Comment: @Crostul Also would I be able to take c=1? and is that all you would show to get full marks on a test..?
Comment: @Newbie: That's correct, you may take $c=1$ in that question. Note, though, that the second one does not make sense: disprove that $\frac {x^3+x} {x+1}$... what?
Comment: @AlexM. Sorry I meant to say show that x^3+x / x+1 is not O(x). Brain fart lol.
Comment: Assume it is...
Comment: @RickSanchez Yeah I know you assume that it is, but if you have to show that it is not O(x), how do you go about that?
Comment: Assume that it is $O(x)$, then derive a contradiction.
Answer: Notice that, in general, $\sin x \le 1 = 1 \cdot 1$, so it is trivial to understand that $\sin x$ is $O(1)$ where $c=1$.
For the second, assume that there exist $c>0$ such that $\frac {x^3 + x} {x+1} \le cx$; usually these problems are given for $x>0$, so your assumption would be equivalent to $x^3+x \le cx^2 + c$, or $x^3 - cx^2 + x - c \le 0$, but this is clearly false because that 3rd degree polynomial tends to $\infty$ (which is positive) for $x \to \infty$. Hence, the assumption is false.
Comment: equivalent to x3+x≤cx2+cx3+x≤cx2+c, or x3−cx2+x−c≤0x3−cx2+x−c≤0,  I don't understand this. And I need steps to proof it so that I will receive full marks on a test.
Answer: For a), simply take $c=1$. Anyway, any constant larger than $1$ is okay.
For b), suppose by contradiction that there exists $C>0$ such that $$\frac{x^3+x}{x+1} \le C x$$
Multiplying by $1/x$ gets you
$$\frac{x^2+1}{x+1} \le C $$
But this implies that $\frac{x^2+1}{x+1}$ is bounded, contradicting
$$\lim_{x \to \infty} \frac{x^2+1}{x+1} = + \infty$$
Comment: Is this because it is f(x)/g(x)? I somewhat understand. Suppose you wrote exactly that to answer a question on a test, would you receive full marks or would you have to be a bit more thorough with showing your proof?
Comment: I would give a full mark for such an answer.
Comment: I see. Thanks very much for your help. I figured out the rest of the practice questions but there's one that I can't figure out. Using the definition I have to proof that e^x is not O(x^5).  ---- So far I just wrote suppose e^x = O(x^5) Then there exists constant C such that for all large x we have e^x <= c*x^5. However e^x/x^5 approaches infinity so it is a contradiction. Is this correct? And where do I go on from here?
Comment: @Newbie Yes, your proof that $e^x$ is not $O(x^5)$ is correct.


Command not found

Question: I downloaded the AWS CloudWatch command line api and I have also set the env_path variable that is AWS_CLOUDWATCH_HOME=local/usr/CloudWatch.
But when I run mon-cmd, I get command not found error in the console. I am working on ubuntu 10.04 server which is a EC2 instance.
It's been to couple of days I am struck with this problem, in spite of setting the path variables correctly I am facing this problem.
Kindly help me out
Comment: Can you find this command `mod-cmd` in your system somewhere? Use `locate` or `find` to check.
Answer: Find the file with find / -type f -name mon-cmd 2>/dev/null then add the folder where the file is located to $PATH by running: export PATH=$PATH:/path/to_mon-cmd/
Answer: Command not found usually means exactly that - that the operating system can't locate the command you are trying to execute. The operating system only looks in a select few folders for the application. This list of folders is defined by the PATH variable.
You can view your current PATH variable with:
echo $PATH

One possible output of this is:
    /usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin:/root/bin
You will note, in the above output that I have my AWS API tools in /opt/aws/bin. Edit your PATH variable to include the location of the mon-cmd program. You can do so, either:
Temporarily (from bash prompt):
PATH=$PATH:/path/to/api-tools

Or permanently, by modifying (or appending) the PATH in your users' .bash_profile:
PATH=$PATH:/path/to/api-tools
export $PATH

Be careful! Ensure you do not leave out the $PATH - this includes the existing PATH and only adds the new one - otherwise you will overwrite your existing PATH and no programs will work if they are not run with an absolute path.
You can verify that you have done so successfully by running (which should return the correct location)
whereis mon-cmd

(Sample output: mon-cmd: /path/to/api-tools/mon-cmd)
Alternatively, you can run the command directly from wherever it is located:
/path/to/api-tools/mod-cmd`

Command not found


Has anybody except Shahid Afridi hit 12 runs off one ball in cricket?

Question: Shahid Afridi has hit 12 runs off one ball (video link).
Except him, has another batsman ever in cricketing history hit 12 runs off one ball?
Comment: All: please don't get into arguments about semantics. If it's clear what "double six" is *intended* to mean, then that's OK. We don't always have to use the precise rulebook terminology for everything (it's very rarely referred to as a "Boundary 4", and crossing sports nobody calls it "kicks from the penalty mark"). And as a general reminder to everyone, please try and keep within the spirit of the [Code of Conduct](https://sports.stackexchange.com/conduct). Thank you.
Answer: No. Afridi is not the only player to do so, and arguably didn't deserve it, unlike
Travis Birt
January 2012: Big Bash League, for Hobart Hurricanes v Melbourne Stars
Clint McKay took the fifth over bowling to Birt, beginning it with 0, 4, 4, 6.
His first attempt at the fifth ball was a no ball hit for Boundary Six (total: 6, 1 bye). His second attempt was a repeat of the first (total: 12 runs, 2 byes).
On the third delivery he finally kept his foot back, and Birt played it on for the wicket.


How would I make it so that whenever the tie conditional is met, it loops and grabs a new entry from random.choice?

Question: For my final project in unit 2 of my python class, we had to make a relatively simple rock, paper, and scissors game. I really enjoyed the class and wanted to get extra credit for it, so I made it more complicated with functions and whatnot. My issue is that whenever the condition for tying is met, what I want my code to do is loop back to where it asks you to play again, and get a new selection from random.choice([x, x, x, etc.]). How would I implement this?
import random
import time

rps = random.choice(['rock', 'paper', 'scissors', 'lizard', 'spock'])
def win_cond():
    print('i choose...')
    time.sleep(0.9)
    print(rps)
    time.sleep(1)
    print('I WIN!!!')
    time.sleep(2)
    print('nerd!')
    play()

def lose_cond():
    print('i choose....')
    time.sleep(0.9)
    print(rps)
    time.sleep(1)
    print('GAH!!! I LOSE!!!')
    play()

def play():
    play = input('you wanna play!? yes or no / y or n: ')
    if (play == 'yes' or 'y'):
        game()
    else:
        print('okay lol')

def game():
    rpsU = input('welcome to the game ong!! choose rock, paper, scissors, lizard, or spock! ').lower()
    rps = random.choice(['rock', 'paper', 'scissors', 'lizard', 'spock'])
    print(rps)

    if (rpsU == 'rock' and (rps == 'paper' or rps == 'spock')):
        win_cond()
    elif (rpsU == 'rock' and 'rps' == 'rock'):
        print('tie,,')
        play()
    elif (rpsU == 'rock' and (rps == 'paper' or rps == 'spock')):
        lose_cond()

    if (rpsU == 'spock' and (rps == 'lizard' or rps == 'paper')):
        win_cond()
    elif (rpsU == 'spock' and rps == 'spock'):
        print('tie,,')
        play()
    elif (rpsU == 'spock' and (rps == 'scissors' or 'rock')):
        lose_cond()

    if (rpsU == 'scissors' and (rps == 'spock' or 'rock')):
        win_cond()
    elif (rpsU == 'scissors' and rps == 'scissors'):
        print('tie,,')
        play()
    elif (rpsU == 'scissors' and (rps == 'paper' or 'lizard')):
        lose_cond()

    if (rpsU == 'paper' and (rps == 'lizard' or 'scissors')):
        win_cond()
    elif (rpsU == 'paper' and rps == 'paper'):
        print('tie,,')
        play()
    elif(rpsU == 'paper' and (rps == 'spock' or 'rock')):
        lose_cond()

    if (rpsU == 'lizard' and (rps == 'rock' or 'scissors')):
        win_cond()
    elif (rpsU == 'lizard' and rps == 'lizard'):
        print('tie,,')
        play()
    elif (rpsU == 'lizard' and (rps == 'spock' or 'paper')):
        lose_cond()

    if (rpsU != 'rock' and rpsU != 'paper' and rpsU != 'scissors' and rpsU != 'lizard' and rpsU != 'spock'):
        print('what the freakazoids!?')

play()

Comment: You have a function to play the game. If it returns a tie, just call it again.
Comment: I've tried that, but what I need is a new output from rps' random.choice.
Comment: I have tried it, it is already take a new output !
Comment: That's the thing, I can't figure out how to do so, as it gives me the first generated output every time I call the function again.
Comment: @WalidGhalem Does it? From what I've been doing, it hasn't been doing so.
Comment: You have a small set of choices, you should try many times I think.
Comment: You have a different global variable named rps, which might be used in some places outside the game() function. Do you know how to pass arguments to a function?
Comment: @KennyOstrom I assume its by placing the argument within the parenthesis?
Ex. find_time(distance)
Comment: Yeah try to put (rps) as a parameter to all your functions because it takes the global array every time
Comment: Did that - prints nothing.
Comment: What did you get in terminal ?
Comment: I get the usual prompts asking if I want to play, but after that it prints nothing.
Answer: Ok, if i understand correctly, you want the game to take a new random value every time, but the problem here is that you have a global variable rps that is the same as the local variable in game, you should delete the global variable, and leave only the one in game, and use it as a parameter on win_cond(rps) and lose_cond(rps). Now, note that the program is not complete, it doesnt cover all the possible combinations, for example, what happens if i choose rock and the computer chooses scissors?
Heres the code without the minor fixes:
import random
import time

def win_cond(rps):
    print('i choose...')
    time.sleep(0.9)
    print(rps)
    time.sleep(1)
    print('I WIN!!!')
    time.sleep(2)
    print('nerd!')
    play()

def lose_cond(rps):
    print('i choose....')
    time.sleep(0.9)
    print(rps)
    time.sleep(1)
    print('GAH!!! I LOSE!!!')
    play()

def play():
    play = input('you wanna play!? yes or no / y or n: ')
    if (play == 'yes' or 'y'):
        game()
    else:
        print('okay lol')

def game():
    rpsU = input('welcome to the game ong!! choose rock, paper, scissors, lizard, or spock! ').lower()
    rps = random.choice(['rock', 'paper', 'scissors', 'lizard', 'spock'])
    print(rps)

    if (rpsU == 'rock' and (rps == 'paper' or rps == 'spock')):
        win_cond(rps)
    elif (rpsU == 'rock' and 'rps' == 'rock'):
        print('tie,,')
        play()
    elif (rpsU == 'rock' and (rps == 'paper' or rps == 'spock')):
        lose_cond(rps)

    if (rpsU == 'spock' and (rps == 'lizard' or rps == 'paper')):
        win_cond(rps)
    elif (rpsU == 'spock' and rps == 'spock'):
        print('tie,,')
        play()
    elif (rpsU == 'spock' and (rps == 'scissors' or 'rock')):
        lose_cond(rps)

    if (rpsU == 'scissors' and (rps == 'spock' or 'rock')):
        win_cond(rps)
    elif (rpsU == 'scissors' and rps == 'scissors'):
        print('tie,,')
        play()
    elif (rpsU == 'scissors' and (rps == 'paper' or 'lizard')):
        lose_cond(rps)

    if (rpsU == 'paper' and (rps == 'lizard' or 'scissors')):
        win_cond(rps)
    elif (rpsU == 'paper' and rps == 'paper'):
        print('tie,,')
        play()
    elif(rpsU == 'paper' and (rps == 'spock' or 'rock')):
        lose_cond(rps)

    if (rpsU == 'lizard' and (rps == 'rock' or 'scissors')):
        win_cond(rps)
    elif (rpsU == 'lizard' and rps == 'lizard'):
        print('tie,,')
        play()
    elif (rpsU == 'lizard' and (rps == 'spock' or 'paper')):
        lose_cond(rps)

    if (rpsU != 'rock' and rpsU != 'paper' and rpsU != 'scissors' and rpsU != 'lizard' and rpsU != 'spock'):
        print('what the freakazoids!?')

play()

Comment: Yeah, I noticed that, I already fixed that.
Comment: I'm trying, and its doing the same thing, would you mind posting your code?
Comment: I did the same thing and it worked
Comment: @CodingQuestions Okay! It now gets a new output for rps, but stops for whatever reason when tying.
Comment: @CodingQuestions Or not. I'm sorry. Thanks for your help! I'd upvote it if I can.


Countable partition of a probability space

Question: I am trying to prove the following statement. Even though it seems almost obvious that it must be true, I am having trouble with making my arguments precise.
Let $\{D_i: i\in \mathbb{N}\}$ be a countable partition of $\Omega$, i.e. $\cup_ {i\in \mathbb{N}} D_i =  \Omega$ and $D_i \cap D_j =\emptyset$ whenever $i\neq j$. Define $\mathcal{D}$ as the $\sigma$-algebra generated by $\{D_i\}$. Show that $G \in \mathcal{D}$ if and only if $G = \cup_{i\in \mathbb{I}}D_i$ for some $\mathbb{I} \subseteq \mathbb{N}$.
The sufficiency part immediately follows from the definition of a $\sigma$-algebra. To show the necessity as I argue as follows. $G \in \mathcal{D} \Rightarrow G^\mathsf{c} \in \mathcal{D}$. Since $G \cup G^\mathsf{c} = \Omega = \cup_ {i\in \mathbb{N}} D_i$ and $D_i \cap D_j =\emptyset$, both $G$ and $G^\mathsf{c}$ must be the unions of some elements of $\{D_i\}$ since I cannot break down $D_i$s to smaller elements within $\mathcal{D}$. The last sentence is not good. But how do I make myself precise here?
Edit: (Based on Karolis Juodele's answer)
$$D_k^\mathsf{c} = \Omega\setminus D_k = \cup_ {i\in \mathbb{N}} D_i \setminus D_k = \cup_ {i\in \mathbb{N}\setminus \{k\}} D_i$$
I do the same for the countable unions of $\{D_k\}$.
Answer: From wikipedia:

A subset $Σ ⊂ 2^X$ is called a σ-algebra if it satisfies the following three properties:

X is in Σ.
Σ is closed under complementation: If A is in Σ, then so is its complement, X\A.
Σ is closed under countable unions: If A1, A2, A3, ... are in Σ, then so is A = A1 ∪ A2 ∪ A3 ∪ … .

You need to show that the complement of every $D_i$ is a union of the rest  of them and that the complement of every union of $D_i$ is another such union. Then we have that property 2 adds no new sets and that your sigma algebra only contains unions.
$$(\bigcup_{j \in J} D_j)^c = (\bigcup_{i \in I} D_i) \setminus (\bigcup_{j \in J} D_j) = ((\bigcup_{i \in I \setminus J} D_i) \setminus (\bigcup_{j \in J} D_j)) \cup ((\bigcup_{i \in J} D_i) \setminus (\bigcup_{j \in J} D_j)) =\\= (\bigcup_{i \in I \setminus J} D_i) \setminus (\bigcup_{j \in J} D_j) = \bigcup_{i \in I \setminus J} (D_i \setminus (\bigcup_{j \in J} D_j)) = \bigcup_{i \in I \setminus J} (D_i \setminus (\bigcup_{j \in J} D_j \cap D_i)) =\\= \bigcup_{i \in I \setminus J} (D_i \setminus (\bigcup_{j \in J\cap \{i\}} D_j \cap D_i)) = \bigcup_{i \in I \setminus J} (D_i \setminus \emptyset) = \bigcup_{i \in I \setminus J} D_i$$
Where the property disjointness of $D_i$ is used in the = going from line 2 to line 3.
Comment: Thanks for your answer. I can follow your reasoning but my problem was with giving a precise answer. Can you prove the claims in your answer rigorously or check my edit?
Comment: I should rephrase my answer in a way that's more comfortable to state. First construct a set of unions of $D_i$, then show that this set is, in fact, a sigma algebra. You still need to show that complements of unions of $D_i$ are also unions of $D_i$.
Now, the actual mechanics of showing this rigorously seem uncomfortable. Your attempt sorely lacks a mention of empty intersections since the last equality doesn't hold in general.
Comment: If you're aware of it, I think you could use [σ-algebra generated by a function](http://en.wikipedia.org/wiki/Sigma-algebra#.CF.83-algebra_generated_by_a_function). The function $x \in D_i \mapsto i$ is a map from $\Omega$ to $\mathbb I$ and unions of $D_i$ are inverse images of the power set of $\mathbb I$.
Comment: I am aware of the fact that the last equality requires the disjointness property and I used it intuitively there. Thanks for your attempts but I haven't really got the rigorous answer I was looking for.
Comment: I don't see any lack of rigor in the use of inverse image of a function, if you're allowed to use that property. Regardless, I've added more basic and complete derivation to my answer.


Combine two dataframe columns in R

Question: Suppose I have a dataframe such as:
A  B  C
a  a  a
a  b  b
d  e  a
and I want to create a new column in the dataframe that is B and C combined (call it D).
A  B  C  D
a  a  a  aa
a  b  b  bb
d  e  a  ea
Comment: Here's a similar question: http://stackoverflow.com/q/30927899/1191259
Answer: You can use paste.
  df1$D <- do.call(paste0, df1[c('B', 'C')])

Or
  df1 <- transform(df1, D= paste0(B,C))

Combine two dataframe columns in R


Find $\lim_{k \to \infty} \frac{k^k}{(k+1)^{k+1}}$

Question: I have spent the better part of this day trying to solve $$\lim_{k \to \infty} \frac{k^k}{(k+1)^{k+1}}$$ When I write this in Wolfram, it says the limit $= 0$, but I don't understand how it got to that conclusion.
Comment: You need to know that $\lim_{n\to\infty}(1+\frac{1}{n})^n$ is well-known to be $e$.
Comment: Hint:  write your expression as $\left(\frac k{1+k}\right)^k\times \frac 1{1+k}$ and remark that the first term goes to $\frac 1e$.
Comment: This isn't rigorous, but it might help from an intuitive perspective. When taking the limit of $(k + 1)^{k + 1}$, the dominant term is always going to be the first term, so we can approximate it as $k^{k + 1}$. $k^k / k^{k + 1} = 1/k$; taking the limit as $k$ goes to $\infty$, we get $0$.
Comment: @almagest Actually you only need to know that $(1+\frac1n)^n$ is bounded. You don't need to know its limit exists.
Comment: @DavidK Good point! But you are unlikely to know one without the other!
Comment: @almagest Pedagogically, most students who know enough to show that $(1+\frac1n)^n$ is bounded _do_ know its limit, because that limit is very likely among the first really useful limits one learns. But it's a lot easier to prove the bounds than to prove the limit, and if you had to justify the answer on something closer to first principles (rather than "my teacher told me this limit"), the approach of merely proving boundedness is a good one, I think.
Answer: For all $k > 1$,
$$ 0 < \frac{k}{k+1} < 1 $$
and
$$ 0 < \left(\frac{k}{k+1}\right)^k < 1. $$
But
$$ \frac{k^k}{(k+1)^{k+1}} = \frac{1}{k+1} \left(\frac{k}{k+1}\right)^k. $$
Therefore
$$ 0 < \frac{k^k}{(k+1)^{k+1}} < \frac{1}{k+1} $$
for all $k>1$, and since $\lim_{k\to\infty} \frac{1}{k+1} = 0$,
by the squeeze theorem
$$ \lim_{k\to\infty} \frac{k^k}{(k+1)^{k+1}} = 0 $$
Answer: Hint for real numbers $(k+1)^{k+1}\geq k^k$ thus as the k becomes larger the denominator becomes so large that the whole fraction tends to $0$.
Answer: Recall that
$$\lim_{k \to \infty} \frac{k^k}{(k+1)^k} = \lim_{k \to \infty} \left( 1-\frac{1}{k+1} \right)^k = \lim_{k \to \infty} \left( \left( 1-\frac{1}{k+1} \right)^{k+1} \right)^{\frac{k}{k+1}} =\left( \frac{1}{e} \right)^1 = \frac{1}{e}$$
Now, your limit is
$$\lim_{k \to \infty} \frac{k^k}{(k+1)^k} \cdot \frac{1}{k+1} = \lim_{k \to \infty} \frac{k^k}{(k+1)^k} \cdot \lim_{k \to \infty} \frac{1}{k+1} = \frac{1}{e} \cdot 0 = 0$$
Answer: First, prove that :
$$ \lim \limits_{k \to +\infty} \Big( \frac{k}{k+1} \Big)^{k} = \frac{1}{e}. $$
Then, it follows that :
$$ \lim \limits_{k \to +\infty} \frac{k^{k}}{(k+1)^{k+1}} = 0 $$
because :
$$ \frac{k^{k}}{(k+1)^{k+1}} = \frac{1}{k+1} \Big( \frac{k}{k+1} \Big)^{k}. $$
Answer: $$\lim_{k \to \infty} \frac{k^k}{(k+1)^{k+1}}=\lim_{k \to \infty} \frac{k^k}{(k+1)^{k}}\cdot\frac{1}{k+1}=\lim_{k \to \infty} \left(\frac{1}{1+\frac1k}\right)^k\cdot\frac{1}{k+1}$$
$$\lim_{k \to \infty}\left(\frac{1}{1+\frac1k}\right)^k=e^{\lim_{k \to \infty}\left(\frac{1}{1+\frac1k}-1\right)\cdot k}=e^{\lim_{k \to \infty}\left(\frac{-1}{1+\frac1k}\right)}=e^{\lim_{k \to \infty}\left(\frac{-k}{1+k}\right)}=\frac1e$$
Thus,
$$\lim_{k \to \infty} \frac{k^k}{(k+1)^{k+1}}=\lim_{k \to \infty} \left(\frac{1}{1+\frac1k}\right)^k\cdot\lim_{k \to \infty} \frac{1}{k+1}=\frac1e\cdot0=0$$
Answer: It's clear that $k^k$ is less than $(k+1)^k$. So you can make an upper bound of $\frac{1}{k+1}$ which clearly converges to zero. Hope this helps..


Why do std::string operations perform poorly?

Question: I made a test to compare string operations in several languages for choosing a language for the server-side application. The results seemed normal until I finally tried C++, which surprised me a lot. So I wonder if I had missed any optimization and come here for help.
The test are mainly intensive string operations, including concatenate and searching. The test is performed on Ubuntu 11.10 amd64, with GCC's version 4.6.1. The machine is Dell Optiplex 960, with 4G RAM, and Quad-core CPU.
in Python (2.7.2):
def test():
    x = ""
    limit = 102 * 1024
    while len(x) < limit:
        x += "X"
        if x.find("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 0) > 0:
            print("Oh my god, this is impossible!")
    print("x's length is : %d" % len(x))

test()

which gives result:
x's length is : 104448

real    0m8.799s
user    0m8.769s
sys     0m0.008s

in Java (OpenJDK-7):
public class test {
    public static void main(String[] args) {
        int x = 0;
        int limit = 102 * 1024;
        String s="";
        for (; s.length() < limit;) {
            s += "X";
            if (s.indexOf("ABCDEFGHIJKLMNOPQRSTUVWXYZ") > 0)
            System.out.printf("Find!\n");
        }
        System.out.printf("x's length = %d\n", s.length());
    }
}

which gives result:
x's length = 104448

real    0m50.436s
user    0m50.431s
sys     0m0.488s

in Javascript (Nodejs 0.6.3)
function test()
{
    var x = "";
    var limit = 102 * 1024;
    while (x.length < limit) {
        x += "X";
        if (x.indexOf("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 0) > 0)
            console.log("OK");
    }
    console.log("x's length = " + x.length);
}();

which gives result:
x's length = 104448

real    0m3.115s
user    0m3.084s
sys     0m0.048s

in C++ (g++ -Ofast)
It's not surprising that Nodejs performas better than Python or Java. But I expected libstdc++ would give much better performance than Nodejs, whose result really suprised me.
#include <iostream>
#include <string>
using namespace std;
void test()
{
    int x = 0;
    int limit = 102 * 1024;
    string s("");
    for (; s.size() < limit;) {
        s += "X";
        if (s.find("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 0) != string::npos)
            cout << "Find!" << endl;
    }
    cout << "x's length = " << s.size() << endl;
}

int main()
{
    test();
}

which gives result:
x length = 104448

real    0m5.905s
user    0m5.900s
sys     0m0.000s

Brief Summary
OK, now let's see the summary:

javascript on Nodejs(V8): 3.1s
Python on CPython 2.7.2 : 8.8s
C++ with libstdc++: 5.9s
Java on OpenJDK 7: 50.4s

Surprisingly! I tried "-O2, -O3" in C++ but noting helped. C++ seems about only 50% performance of javascript in V8, and even poor than CPython. Could anyone explain to me if I had missed some optimization in GCC or is this just the case? Thank you a lot.
Comment: @AshBurlaczenko: In java strings are immutable and pooled, therefore dead slow. Normally you use stringbuilders to assemble strings. This whole thing here is comparing apples and oranges anyways.
Comment: @AshBurlaczenko possibly because strings in Java are immutable. I suppose `s += "X"` is a performance killer there. That's the reason `StringBuilder` exists.
Comment: Try `s.reserve(limit);` before the loop.
Comment: You are testing a mixture of operations, you should probably try to divide the test into different tests that perform different checks for performance, for example: growing strings, or finding, or ... currently you cannot know where the time is being spent. And BTW, this is probably a quite useless test to decide on a language...
Comment: You're also including the runtime startup and termination of each language in your results.
Comment: Erm, how did C++ perform more poorly than CPython? (You show 5.5s with C++, 8.8 seconds with CPython)
Comment: Not as relevant to the C++ part of the Question, but Matt Joiner is correct in his assessment of the Java solution.  Use a StringBuffer to concatenate instead of a String.  Java Strings are not mutable, and it will create a new String every time you += to it.  From http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/StringBuffer.html, "A string buffer is like a String, but can be modified. "
Comment: @REDSOFTADAIR I confirm that a different compiler can help. On my machine compiling with (visual studio 2010) cl /EHsc -O2 gives 1.753 s (close to your result) while python 2.7.1 takes 6.266 s (also more or less in line with the OP results)
Comment: You should ask someone who knows the language to implement your test case. Simply using StringBuilder in Java cuts the running time from 75 seconds to 4 on my machine.
Comment: @AndrewWalker I don't think the console output is supposed to occur except for a single line at the end...
Comment: Maybe try a different compiler as well. I compiled the sample using Visual Studio 2010. Runtime 1.79 Seconds on a XEON 2.8 GHz.
Comment: @WuShu: Do you intend on loading a character at a time? If so, then this test is representative of your usage, and the best thing you can do it not deciding a language, but looking for a better approach. Dividing the test will provide with more information, as it will let you know for each language what operations are most expensive and will thus let you optimize your processing for that and still the test will be absurd. Just in C++, changing from `s+="X"` to `s.push_back('X')` will probably have a greater impact than the differences with other languages...
Comment: You might also want to try profile guided optimizations -- in C and C++ this can be significant for loops like this (node.js/V8 are certainly compiled with such things on)
Comment: It appears that GNU libstdc++ inline coded a for loop for std::string find. Dumb move. memmem and strstr are much more optimized with hand-written versions in assembly taking advantage of SSE2 and SSE4 extensions.
Comment: Yeah, it's really 50s, and I tried several times. I supposed Java would perform better than Python at least, though.
Comment: Get rid of the console output from your testing, it will be skewing your results
Comment: @David: It would be meaningless if I test each type of operation separately, for there won't be winners on all operations. This test is to emulate a server daemon that listens on a socket or waits on the terminal for character stream, and analysis it in real time. But for simplicity, the test program omits a lot of not-so-relevant operations.
Comment: @David: I tried s.push_back('X') but only improved about 0.1s. As of others' and my option, the bottle neck is mainly the searching procedure. And I also tested string::find is quicker than strstr or memmem if "-O" disabled, but much slower if "-O2" enabled. So string::find maybe hard to optimized.
Comment: @Billy: Well, it's not so fair to directly compare CPython and C++ using 'time' command, for CPython spends more time than C++ to start the engine, that is, loading and compiling into byte-code. So I supposed CPython faster if it C++ didn't exceeds much much more, which of course, not very accurately.
Comment: @AshBurlaczenko: Yeah you are right. I tested with StringBuilder and Java gave the score of 2.8s, the best performance within all the tests, which of course still slower than strstr in pure C (about 1.6s). While strangely Java with StringBuilder still performed about 30% slower than Nodejs on my Thinkpad R400 laptop.
Comment: @Wu: You can make the same argument against Java or Node. (Try writing a minimal hello world python app and subtract the time that takes from the time your benchmark took; that should get rid of startup delay)
Comment: @PlasmaHH Note that Lua strings are also immutable and pooled (interned), and I found that a Lua version of this test runs over twice as fast as the next fastest performer (nodejs/v8).  [about 1.5s for Lua vs. about 3.1s for nodejs]
Answer: It's not that std::string performs poorly (as much as I dislike C++), it's that string handling is so heavily optimized for those other languages.
Your comparisons of string performance are misleading, and presumptuous if they are intended to represent more than just that.
I know for a fact that Python string objects are completely implemented in C, and indeed on Python 2.7, numerous optimizations exist due to the lack of separation between unicode strings and bytes. If you ran this test on Python 3.x you will find it considerably slower.
Javascript has numerous heavily optimized implementations. It's to be expected that string handling is excellent here.
Your Java result may be due to improper string handling, or some other poor case. I expect that a Java expert could step in and fix this test with a few changes.
As for your C++ example, I'd expect performance to slightly exceed the Python version. It does the same operations, with less interpreter overhead. This is reflected in your results. Preceding the test with s.reserve(limit); would remove reallocation overhead.
I'll repeat that you're only testing a single facet of the languages' implementations. The results for this test do not reflect the overall language speed.
I've provided a C version to show how silly such pissing contests can be:
#define _GNU_SOURCE
#include <string.h>
#include <stdio.h>

void test()
{
    int limit = 102 * 1024;
    char s[limit];
    size_t size = 0;
    while (size < limit) {
        s[size++] = 'X';
        if (memmem(s, size, "ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26)) {
            fprintf(stderr, "zomg\n");
            return;
        }
    }
    printf("x's length = %zu\n", size);
}

int main()
{
    test();
    return 0;
}

Timing:
matt@stanley:~/Desktop$ time ./smash
x's length = 104448

real    0m0.681s
user    0m0.680s
sys     0m0.000s

Comment: FWIW the difference between Python 2.7 and 3.2 is just under 10%. It is possible that PEP 393 will remove that difference in Python 3.3. Also it might be worth mentioned that searching strings in Python uses a form of Boyer-Moore so when the string gets longer it should gain an advantage over languages that do a plain search.
Comment: In java, using StringBuilder instead of String speeds it up (on my machine) aproximately 4 times, rest is searching. In java, string is immutable, so what he is doing is atrociously wrong string manipulation in java. Then there is issue of timing VM start instead of timing usefulactions (it is issue for all languages on VM, not just java)
Comment: @WuShu: actually, C and C++ are probably the sharpest knives. It's just that Python and Node.js may be chainsaws. It's heavy and sometimes overkill, but when you tire in C++ you appreciate the "batteries included" approach they took in Python.
Comment: @Wu Shu: If the specific pattern to search for is fixed and predetermined, you can construct an automaton to search for the pattern. This will be much faster than repeated calls to `std::string::find`.
Comment: @Matt: Well, the C program is too extreme... I didn't tried to make a battle or contest between languages, for each language has its optimization in different ways. I just want to find a language that can proceed strings with quite good efficiency. The program just described a case that a program reads from an input(console or socket), maybe decrypts it then, and searches through the string for a specified pattern. My test program simplified the procedure and was just a demo, of course. The result just reminds me C++ is not always the sharpest knife. And thanks anyway :)
Answer: So I went and played a bit with this on ideone.org.
Here a slightly modified version of your original C++ program, but with the appending in the loop eliminated, so it only measures the call to std::string::find(). Note that I had to cut the number of iterations to ~40%, otherwise ideone.org would kill the process.
#include <iostream>
#include <string>

int main()
{
    const std::string::size_type limit = 42 * 1024;
    unsigned int found = 0;

    //std::string s;
    std::string s(limit, 'X');
    for (std::string::size_type i = 0; i < limit; ++i) {
        //s += 'X';
        if (s.find("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 0) != std::string::npos)
            ++found;
    }

    if(found > 0)
        std::cout << "Found " << found << " times!\n";
    std::cout << "x's length = " << s.size() << '\n';

    return 0;
}

My results at ideone.org are time: 3.37s. (Of course, this is highly questionably, but indulge me for a moment and wait for the other result.)
Now we take this code and swap the commented lines, to test appending, rather than finding. Note that, this time, I had increased the number of iterations tenfold in trying to see any time result at all.
#include <iostream>
#include <string>

int main()
{
    const std::string::size_type limit = 1020 * 1024;
    unsigned int found = 0;

    std::string s;
    //std::string s(limit, 'X');
    for (std::string::size_type i = 0; i < limit; ++i) {
        s += 'X';
        //if (s.find("ABCDEFGHIJKLMNOPQRSTUVWXYZ", 0) != std::string::npos)
        //    ++found;
    }

    if(found > 0)
        std::cout << "Found " << found << " times!\n";
    std::cout << "x's length = " << s.size() << '\n';

    return 0;
}

My results at ideone.org, despite the tenfold increase in iterations, are time: 0s.
My conclusion: This benchmark is, in C++, highly dominated by the searching operation, the appending of the character in the loop has no influence on the result at all. Was that really your intention?
Comment: @Matthieu M.: Boyer-Moore does not gain you anything here, because the first character of the search-for string is not found at all in the search-into string. On the contrary, it may be adding some overhead, needlessly processing the search-for string in every iteration of the loop.
Comment: @sbi: and that is when one notes than in C++ `find` is O(N), while in `Python` `indexOf` uses Boyer-Moore (as noted by Duncan in a comment). Once again, "batteries included".
Comment: Are we sure that string::find(const char*) isn't just implemented in terms of string::find(const string&)? If it was, memory allocations could be expensive here.
Comment: @Kylotan: I tested both. No visible difference.
Comment: @MikeNakis: Indeed, I tested it and even doing the loop invariant code motion by hand (to move the pattern analysis out of the loop) the boyer-moore search was still slower. Therefore I suspect they use something even trickier, perhaps closer to `memmem`.
Comment: @MatthieuM. WRT C++ find being linear time, am I missing something? Boyer-Moore is also linear time, isn't it?
Comment: @Tim: linear can be faster than linear :)
Comment: @MatthieuM. of course, but the wording of your comment implied a contrast of some sort. OK, no more language lawyering from me.
Comment: I suspect the reason `s += 'X';` results in `0s` is because the compiler is detecting the fact the loop is just doing `s += 'X';` `limit` times within a loop and instead generating the equivalent of `std::string s(limit, 'X')`, or perhaps even realising that the actual contents of the string aren't relevant and merely replacing the `s.size()` with `limit` - i.e. either the `s += 'X';` or even `s` is entirely optimised away. While I don't doubt the `find` is the bulk of the operation, I expect its inclusion prevents `s += 'X';` and `s` being optimised away.
Answer: The idiomatic C++ solution would be:
#include <iostream>
#include <string>
#include <algorithm>

int main()
{
    const int limit = 102 * 1024;
    std::string s;
    s.reserve(limit);

    const std::string pattern("ABCDEFGHIJKLMNOPQRSTUVWXYZ");

    for (int i = 0; i < limit; ++i) {
        s += 'X';
        if (std::search(s.begin(), s.end(), pattern.begin(), pattern.end()) != s.end())
            std::cout << "Omg Wtf found!";
    }
    std::cout << "X's length = " << s.size();
    return 0;
}

I could speed this up considerably by putting the string on the stack, and using memmem -- but there seems to be no need. Running on my machine, this is over 10x the speed of the python solution already..
[On my laptop]
time ./test
X's length = 104448
real    0m2.055s
user    0m2.049s
sys 0m0.001s
Comment: The reason std::search is faster than std::string::search is the because (by convention?) std::search is implemented in the header which gives the compiler much more room to optimise. std::string::search on the other hand is not. (And because this is calling the function so many times, it makes a big different)
Comment: @Heptic: Um. `std::string` is but a typedef for `std::basic_string`, which is a template, and as such fully implemented in headers.
Comment: Confirmed. g++ 4.4.3. In my test 5s for search, 12.5s for find (both in the same exe; my test times are longer as I pre-created the string with `std::string s(limit,'X');` I.e. search and find had more work to do.) CONCLUSION: stdlib find() on g++ has lots of potential for optimization!
Comment: Wow; added a memmem() version, and it is 0.75s (using the same string, accessed via c_str()). (Actually, it was 0; the whole loop seemed to get optimized away; so I added some minor computation to the loop to stop that.) NEW CONCLUSION: find() and search() are doing something weird, that even -O3 cannot optimize, or memmem is using some special CPU feature. Fascinating!
Comment: @Pharap: All true!
Comment: @sbi Unless `std::basic_string` is specialised, in which case the functions may be implemented in a `.cpp` because they then behave as functions on a 'normal' class rather than a templated class. Also some compilers attempt to 'precompile' headers using various tricks, which may also interfere (though I don't think GCC is one of those as far as I remember)
Answer: That is the most obvious one: please try to do s.reserve(limit); before main loop.
Documentation is here.
I should mention that direct usage of standard classes in C++ in the same way you are used to do it in Java or Python will often give you sub-par performance if you are unaware of what is done behind the desk. There is no magical performance in language itself, it just gives you right tools.
Comment: Well, checked this in practice. Replacing s += "X" with string s(102*1024, 'X'); made enormous speed improvement ( real 0m0.003s in my VBox ). std::string::reserve not helping though, despite what I have said ( should have had same effect in my opinion though ). Need to investigate a bit more.

Edited: lol, only now have paid attention to the way for loop is stated :) ok, rollback everything
Comment: Of course building the string made enormous speed improvement. You then bypass the loop completely... You need to change the loop condition to iterate on a `i = 0` variable if you allocate the string first, and then you'll notice that the search is the real issue.
Comment: On my machine adding `s.reserve(limit)` before the loop makes no perceptible difference to performance.
Comment: I agree with what you are saying *in general*, but have you tested this?  With gcc 4.6 I don't get any speedup when using `string::reserve`.  Can you show how to do the concatenation in a fast way, exploiting the knowledge of how the classes work in the background?
Comment: Is that really an issue here? Each `string::operator++` is only appending a single character, so memory reallocation and copying shouldn't be a big drain.
Comment: Yes I tried s.reserve(102 * 1024), but no help. It's about 5.895s and little improvement. I think the bottle neck is in string search.
Answer: My first thought is that there isn't a problem.
C++ gives second-best performance, nearly ten times faster than Java. Maybe all but Java are running close to the best performance achievable for that functionality, and you should be looking at how to fix the Java issue (hint - StringBuilder).
In the C++ case, there are some things to try to improve performance a bit. In particular...

s += 'X'; rather than s += "X";
Declare string searchpattern ("ABCDEFGHIJKLMNOPQRSTUVWXYZ"); outside the loop, and pass this for the find calls. An std::string instance knows it's own length, whereas a C string requires a linear-time check to determine that, and this may (or may not) be relevant to std::string::find performance.
Try using std::stringstream, for a similar reason to why you should be using StringBuilder for Java, though most likely the repeated conversions back to string will create more problems.

Overall, the result isn't too surprising though. JavaScript, with a good JIT compiler, may be able to optimise a little better than C++ static compilation is allowed to in this case.
With enough work, you should always be able to optimise C++ better than JavaScript, but there will always be cases where that doesn't just naturally happen and where it may take a fair bit of knowledge and effort to achieve that.
Comment: Stringstream will not give any performance improvements here. `std::string` is already mutable and can insert in constant amortized time.
Comment: The performance is bounded by the `find` call, not the allocation. For example, testing the 2nd point, there is just not difference (at all).
Comment: @Matthieu - well, I didn't say any of my ideas would definitely make a difference. However, the second point is **all about** the `find` call. The point is to use a different overload of `find` which takes the search pattern as an `std::string` rather than as a C string, and thus (possibly but not definitely) avoid `strlen` calls within the `find` call. Another thought is that since the search pattern is constant, a compiled-pattern approach may work faster (Boyer-Moore string search, for example), but that's cheating - unless e.g. JavaScript optimizers are much smarter than I'd expect.
Comment: I tested a naive Boyer-Moore (building the table at each step) and it performed worse. The needle is very small (26 characters) compared to the size of the haystack (104448 characters), so the extra complexity balances the speed-up that could be expected. I guess that building the table outside could help... but maybe not as much as expected.
Answer: What you are missing here is the inherent complexity of the find search.
You are executing the search 102 * 1024 (104 448) times. A naive search algorithm will, each time, try to match the pattern starting from the first character, then the second, etc...
Therefore, you have a string that is going from length 1 to N, and at each step you search the pattern against this string, which is a linear operation in C++. That is N * (N+1) / 2 = 5 454 744 576 comparisons. I am not as surprised as you are that this would take some time...
Let us verify the hypothesis by using the overload of find that searches for a single A:
Original: 6.94938e+06 ms
Char    : 2.10709e+06 ms

About 3 times faster, so we are within the same order of magnitude. Therefore the use of a full string is not really interesting.
Conclusion ? Maybe that find could be optimized a bit. But the problem is not worth it.
Note: and to those who tout Boyer Moore, I am afraid that the needle is too small, so it won't help much. May cut an order of magnitude (26 characters), but no more.
Comment: There is no `A` in the hay-stack, so it should just check each character in the string that it is not found and not look at the other characters of the pattern. You seem to be describing `find_any_of` method, which again should find the `'X'` very fast here.
Comment: @UncleBens: not at all, I am talking about `find`, which even for a string pattern should stop on the first character of the pattern if it does not match and move on in the haystack. The fact that looking for a single character `A` (the first character of the pattern) is only 3 times faster confirms my suspicion that it is not the pattern search that is slow, but simply that looking for a pattern in such a long string so many times is plain slow in itself.
Answer: For C++, try to use std::string for "ABCDEFGHIJKLMNOPQRSTUVWXYZ" - in my implementation string::find(const charT* s, size_type pos = 0) const calculates length of string argument.
Answer: I just tested the C++ example myself. If I remove the the call to std::sting::find, the program terminates in no time. Thus the allocations during string concatenation is no problem here.
If I add a variable sdt::string abc = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" and replace the occurence of "ABC...XYZ" in the call of std::string::find, the program needs almost the same time to finish as the original example. This again shows that allocation as well as computing the string's length does not add much to the runtime.
Therefore, it seems that the string search algorithm used by libstdc++ is not as fast for your example as the search algorithms of javascript or python. Maybe you want to try C++ again with your own string search algorithm which fits your purpose better.
Comment: Well, if you remove string::find, this is just string concatenation, and this would be no much difference between languages/runtimes optimized for string: string in C++ is also much more optimized than in C (string as an array of char). string::find is not just a test for searching algorithm, but also a test for traversing string. I'll make another test.
Answer: C/C++ language are not easy and take years make fast programs.
with strncmp(3) version modified from c version:
#define _GNU_SOURCE
#include <string.h>
#include <stdio.h>

void test()
{
    int limit = 102 * 1024;
    char s[limit];
    size_t size = 0;
    while (size < limit) {
        s[size++] = 'X';
        if (!strncmp(s, "ABCDEFGHIJKLMNOPQRSTUVWXYZ", 26)) {
            fprintf(stderr, "zomg\n");
            return;
        }
    }
    printf("x's length = %zu\n", size);
}

int main()
{
    test();
    return 0;
}

Answer: Your test code is checking a pathological scenario of excessive string concatenation. (The string-search part of the test could have probably been omitted, I bet you it contributes almost nothing to the final results.) Excessive string concatenation is a pitfall that most languages warn very strongly against, and provide very well known alternatives for, (i.e. StringBuilder,) so what you are essentially testing here is how badly these languages fail under scenarios of perfectly expected failure.  That's pointless.
An example of a similarly pointless test would be to compare the performance of various languages when throwing and catching an exception in a tight loop.  All languages warn that exception throwing and catching is abysmally slow. They do not specify how slow, they just warn you not to expect anything. Therefore, to go ahead and test precisely that, would be pointless.
So, it would make a lot more sense to repeat your test substituting the mindless string concatenation part (s += "X") with whatever construct is offered by each one of these languages precisely for avoiding string concatenation. (Such as class StringBuilder.)
Comment: `std::string::operator+=` *is* the construct offered by C++ for avoiding the thing that in Java causes String concatenation to be slow. `std::string` is a mutable class, same as `StringBuilder`. TBH I think it's a bit confusing that the question is "why is C++ slow?", but includes Java results that are *waaay* slower, prompting various people to explain why the Java results are slow. Which is irrelevant to the question ;-)
Comment: I have just checked the example code myself, and it turns out that almost all of the runtime is spend during string search.
Comment: @swegi which languages did you check? I expect it may vary between them. With Python 2.7 the code as written takes 13.1s on my system, removing the `find` call it takes 0.019s. So the string concatenation (at least on Python) is the irrelevant part of the test. This is probably only true because the C version of Python uses reference counting and can do the concatenation in-place when it can detect that there is only one reference to the string.
Comment: o_O -- OK, then there is something totally weird going on. Prior to posting my answer I checked the documentation of all the find() and indexOf() methods in all of the above languages to make sure that they all perform straight non-regex, case-sensitive search. So, if search is the problem despite the triviality of the task, I do not know what to say.
Comment: Well, I checked only the C++ example, I think you are right for the really poor performance of the Java example.
Comment: @MikeNakis - a straight search can benefit from knowing the length of the search pattern - limiting the range where it searches and even dismissing the possibility of searching for a long pattern in a short string at all. A C string doesn't know it's own length, so either the `find` operation will use a linear-time `strlen` on the pattern for each call, or else it will work without that advanced knowledge of the length and do more work searching as a result.
Comment: @Steve314 I thought the same, but I tried searching for "A" instead of "ABCDEFGHIJKLMNOPQRSTUVWXYZ" and there is no change in the running time. (Speaking of the C++ sample.)
Comment: @MikeNakis - actually, now I think about it, I'd expect that result. A straight string search for "ABC..." in a string of "XXX..." reduces to a straight search for 'A' (each check starting at a particular position fails at the first character - `'A' != 'X'`). This even raises questions about just how smart optimizers may be - find the right language and compiler and maybe all the searching will be optimized away (proven unnecessary) at compile-time. With Haskell and GHC, for example, you may find that the final executable just outputs a constant "x's length ..." string.
Comment: @Steve314 I beg to differ; if that was possible, then the "Halting Problem" would be solvable. http://en.wikipedia.org/wiki/Halting_problem.
Comment: @MikeNakis - this program isn't all possible programs. Haskell may well do what I said, not by recognising a specific special case, but because there's no run-time specific data to block it from executing pretty much the whole program at compile-time. For the more specific optimisation, is it possible for a particular compiler to spot that the string to search will never contain an "A"? Yes. But it's an unlikely case in itself, except in that it might be a special case of a more general (but still not universal) optimisation.
Comment: @MikeNakis - while I won't say that *will* happen, and especially not in any particular language, if you have studied Haskell to any depth it might change some of your assumptions about what is and what is not plausible.
Comment: OK, I got your point, you pointed out my mistake when you said "this program isn't all possible programs".
Answer: It seems that in nodejs there are better algorithms for substring search. You can implement it by yourself and try it out.
Answer: As mentioned by sbi, the test case is dominated by the search operation.
I was curious how fast the text allocation compares between C++ and Javascript.
System: Raspberry Pi 2, g++ 4.6.3, node v0.12.0, g++ -std=c++0x -O2 perf.cpp
C++ : 770ms
C++ without reserve: 1196ms
Javascript: 2310ms
C++
#include <iostream>
#include <string>
#include <chrono>
using namespace std;
using namespace std::chrono;

void test()
{
    high_resolution_clock::time_point t1 = high_resolution_clock::now();
    int x = 0;
    int limit = 1024 * 1024 * 100;
    string s("");
    s.reserve(1024 * 1024 * 101);
    for(int i=0; s.size()< limit; i++){
        s += "SUPER NICE TEST TEXT";
    }

    high_resolution_clock::time_point t2 = high_resolution_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::milliseconds>( t2 - t1 ).count();
    cout << duration << endl;
}

int main()
{
    test();
}

JavaScript
function test()
{
    var time = process.hrtime();
    var x = "";
    var limit = 1024 * 1024 * 100;
    for(var i=0; x.length < limit; i++){
        x += "SUPER NICE TEST TEXT";
    }

    var diff = process.hrtime(time);
    console.log('benchmark took %d ms', diff[0] * 1e3 + diff[1] / 1e6 );
}

test();


Show a linear map $\varphi: V \rightarrow V$ is a homomorphism if and only if $g \varphi= \varphi$ for all $ g \in G$.

Question: Can anyone help me to show the if and only if. I have tried to show it by assume the first part and show the second part and converse but I can't figure it out.
Lemma:
Let $V$ be a representation of finite group $G$. Then the set of linear maps from $V$ to $V$ forms a vector space with pointwise addition and scalar multiplikation, and if we have for $g \in G$ and a linear map $\varphi: V \rightarrow V$ define the map $g \varphi $ by $(g \varphi )(v)=g(\varphi (g^{-1}v))$, then this defines a representation of $G$ on the set of linear maps $V \rightarrow V$.
Further, a linear map $\varphi: V \rightarrow V$ is a homomorphism if and only if $g \varphi= \varphi$ for all $ g \in G$.
Answer: By definition, if $V$ and $W$ are $G$-modules and $\varphi\colon G\longrightarrow H$ is a map, then $\varphi$ is a $G$-module homomorphsm if$$(\forall g\in G)(\forall v\in V):\varphi(gv)=g\varphi(v).$$But this is equivalent to$$(\forall g\in G)(\forall v\in V):\varphi(v)=g\varphi\left(g^{-1}v\right).$$
Comment: Thank you! But what about the other way? If I assume that $g \varphi = \varphi$ how do I show that the map is a homomorphism?
Comment: Because the assertion $g\varphi=\varphi$ means$$(\forall g\in G)(\forall v\in V):\varphi(v)=g\varphi\left(g^{-1}v\right).$$
Comment: Thank you so much :) I know im asking many questions, but can you help me understand why the map $g \varphi $ is a representation? I have tried  to understand why it is a representation by looking at the definition of a representation, but I don't get it. $G$ is a group. And a representation of $G$ is a vectorpsace $V$ together with a homomorphism $\rho: G \rightarrow GL(V)$. Is it because the map $\varphi$ is a homomorphism that $g\varphi$ is a representation?
Comment: The map$$\begin{array}{ccc}G\times V&\longrightarrow&V\\(g,\varphi)&\mapsto&g\varphi\end{array}$$is a representation if $G$ because$$(\forall g,h\in G)\bigl(\forall\varphi\in GL(V)\bigr):(gh)\varphi=g(h\varphi).$$


How do I rename all files index.php containing a certain folder in the path?

Question: I want to rename all wordpress admin index files into index_disabled in the folder
/var/www/

I need a script like
find /var/www/ -path '*wp-admin/index.php' -exec mv {} $(dirname {})/index_disabled

but this doesen't work like this
the problem is this: How do i use the {} operator in find execution inside exec
my bash skills are not good enough to complete this ;)
Comment: The bash code will be nearly trivial but you have to explain what exactly you want. What do "all files like" and "$(locate wp-admin/index.php)" stand for? "All files in a known directory"? "All *.php files"? "A directory that needs to be found"?
Answer: You can use the find command for this:
find /var/www/ -path '*wp-admin/index.php' -exec mv -v {} {}_disabled \;

This will rename all files to {old_name}_disabled.
Comment: this doesent work: Warnung:  `Unix Dateinamen enthalten gewöhnlich keine Schrägstriche (anders als Pfadbezeichnungen).` in english something with slash in filenames
Comment: for finding a path and file combination you will indeed need to use the -path argument to find instead of the -name argument. I have updated my answer accordingly.
Comment: you forgot the -path argument?
Comment: ok, so i need something like this: `find /var/www/ -path '*wp-admin/index.php' -exec mv {} $(dirname {})/index_disabled` but that doesent work ;) the problem is this: http://serverfault.com/questions/500122/how-do-i-use-the-operator-twice-in-find-exec
Comment: I have it!!! `find /var/www/ -path '*wp-admin/index.php' -exec mv {} {}_disabled \;` thanks!!! ;)


Post stratification weights and the survey package in R

Question: I have survey data that needs to be weighted, and to help me with this task, I have access to the full joint distributions of the variables I want to use. As I understand it, I should use the postStratify() function in the survey package for this task, but I must admit that I am quite lost when it comes to the required syntax.
I have tried to use marginal distributions and the rake() function, this is quite straight-forward:
http://www.r-bloggers.com/survey-computing-your-own-post-stratification-weights-in-r/
But how do I incorporate all my joint distributions into the postStratify() function? The manual only lists a simple example and is a bit vague, to me at least. I have full joint distributions for 5 different variables.
Moreover, if want to use my weighted result with a function that does not take weight as an input, how should I go about and "duplicate" my rows in the final data? When I tried the rake() function I could only trim weights to an interval with a min value of 0.87. I was thinking I could round all weights to integers.
Comment: some syntax examples https://github.com/ajdamico/usgsd/search?utf8=%E2%9C%93&q=postStratify
Comment: Please demonstrate your syntax.
Comment: I just answered a similar question on `postStratify()` syntax [here](http://stats.stackexchange.com/q/146841/5739).
Answer: I made an example for using post-stratification as below. I find that in the 'strata' argument, if specify as a formula with two or more variables, it implies that the post-stratification is done for their joint distribution. (Initially, I thought I need to add an interaction term in the formula, but it turned out to be unnecessary!)
library(survey)

data(api)
summary(apiclus1)

# original design
# suppose 'pw' is sampling weight
dclus1<-svydesign(id=~dnum, weights=~pw, data=apiclus1, fpc=~fpc)

### post-stratify on only one categorical variable ###
# on school type
pop.types <- xtabs(~stype, data=apipop)
dclus1p<-postStratify(dclus1, ~stype, pop.types)

# compare sampling weights with post-stratified weights
summary( attr(dclus1p$postStrata[[1]], "oldweights" ) )
summary( attr(dclus1p$postStrata[[1]], "weights" ) )

### post-stratify on joint distribution of two categorical variables ###
# school type and sch.wide

apiclus1$stype.sch.wide <- interaction( apiclus1$stype, apiclus1$sch.wide )
apipop$stype.sch.wide <- interaction( apipop$stype, apipop$sch.wide )
pop.types <- xtabs(~stype.sch.wide, data=apipop)

dclus1<-svydesign(id=~dnum, weights=~pw, data=apiclus1, fpc=~fpc)

# first method
dclus2p<-postStratify(dclus1, ~stype.sch.wide, pop.types)
summary(weights(dclus2p))

# second method
pop.types <- xtabs(~ stype + sch.wide, data=apipop )
dclus3p<-postStratify(dclus1, ~stype + sch.wide, pop.types)
summary(weights(dclus3p))

Answer: If you are having syntax trouble calculating post stratification weights and trimming I can recommend using www.spinnakerresearch.nl. No syntax needed. I believe you can trim max weights. However rounding weights to integers does not make sense as you will need to weight some cases down. Meaning a weight factor between 0 and 1.
Comment: Hi, welcome to CrossValidated. Please note that you must reveal any affiliation to products you recommend, each time that you do so.  Please see this section of the [help/behavior], in particular the section under "***Avoid overt self-promotion***". You should also make certain when you do so that you clearly address the *specific issues raised in the question* in your answer.


Linear Transformation in Lin Alg

Question: Let $T : R3 → R3$ be a linear transformation which projects vectors onto the plane π with equation $x−y + 2z = 0$. Find a matrix $A$ such that $T = T_A$
I know how to complete the question I just had a quick inquiry. So I know I need to choose two vectors that span the plane and are orthogonal. It is easy to choose vectors that span the plan but is there a quick and easy way to find two vectors that span the plane and are orthogonal without just guessing what possible combination would work? I know that the two vectors in this case are 1 1 0 and 1 -1 -1
Comment: @Arthur how so?
Comment: I know to pick x and y but just figuring out if they are orthogonal might take a bit longer if I was in a time crunch on an exam
Comment: Sorry, I read your question wrong.
Answer: To get two orthogonal vectors in the plane, first find one vector in the plane (say $(1,1,0)$), then take the cross product with the normal vector $(1,-1,2)$:
$$
(1,1,0)\times(1,-1,2)=(1\cdot2-(-1)\cdot0,0\cdot1-1\cdot2, 1\cdot(-1)-1\cdot1)\\
=(2,-2,-2)
$$
which may be scaled back to $(1,-1,-1)$.
Comment: if I was to just leave it without scaling back is that fine?
Comment: Of course. Simplifying is just personal preference.
Comment: perfect thanks again

Linear Transformation in Lin Alg


Visualization of a Standard Two Dimensional Polyhedron

Question: I am reading this book on linear programming, and the authors give an excellent exposition of the topic by the interplay between the underlying algebra and geometry. Their main approach is to motivate the main results by geometry and then carry out the mechanics by algebra. In this avenue, they have given some examples of visualizing a standard two dimensional polyhedron $P$, which is a subset of a six dimensional Euclidean space $\mathbb{R}^6$ as shown in the figure below.
$\qquad\qquad\quad\qquad$
To make things more precise and concrete, let me give some definitions and an example to work with.

Definition. Let $A\in\mathbb{R}^{m,\,n}$ be a matrix with $m$ rows and $n$ columns where $\text{rank} A = m$. The set of points $P=\{\mathbb{x}\in\mathbb{R}^n | A\mathbb{x}=\mathbb{b}, \, \mathbb{x} \ge \mathbb{0} \}$ is said to be a standard polyhedron in $\mathbb{R}^n$. The dimension of $P$ is defined to be the dimension of the smallest affine subspace of $\mathbb{R}^n$ that contains $P$.

It is also straight forward to conclude that

Lemma. Let $S=\{\mathbb{x}\in\mathbb{R}^n | A\mathbb{x}=\mathbb{b}\}$. Then $S$ is the smallest affine subspace that contains $P$. Furthermore, we have $\dim P = \dim S = \dim \text{null} A = n - m$.

Example. Let us consider the following polyhedron $P\subset\mathbb{R}^4$
\begin{align}
P = \Big\{ &\mathbf{x} =
\begin{bmatrix}
x_1, & x_2, & x_3, & x_4
\end{bmatrix}^\top \in \mathbb{R}^4 \Big| \\
&1 x_1 - 2 x_2 + 1 x_3 + 2 x_4 = 2, \\
&2 x_1 + 1 x_2 + 0 x_3 - 1 x_4 = 2, \\
& x_1, \, x_2, \, x_3, \, x_4 \ge 0
\Big\}
\end{align}
It is easy to check that $\dim P = 4 - 2 = 2$. So, it is a two dimensional object and it may be suggesting that there is a way to visualize it in $\mathbb{R}^2$. I am wondering that if we can obtain the algebraic equations that describe $P$ in $\mathbb{R}^2$ and helps us to visualize it. The first thing that comes to my mind is to project $P$ on $\mathbb{R}^2$. How can we do this effectively, so that we can get something like the above figure?
Question 1. Given the algebraic equations of a standard polyhedron (as in the example), I am wondering if there is a way to obtain such a visualization?
Question 2. Given that the projection idea works, it seems natural to ask that is the projection of a polyhedron a polyhedron too?
Comment: Yes, the projection of a polyhedron is a polyhedron. However, projection of one plane onto another not parallel to it distorts shapes, so i don't think it is your best option. What you need to do is establish some coordinate sytem on the plane $S$. They chose the basic feasible solution $x$ as their origin. But to get a full coordinate system, you will need at least a choice of the direction of the positive $x$ axis, and which half-plane the positive $y$-axis will lie in - effectively whether you are looking at the plane from the same side as the $\Bbb R^n$ origin, or the opposite side.
Comment: Sorry, but I can't. The geometry and the linear algebra I understand well, but I know little else about linear programming. I have no idea what the "basic feasable solution" is - only that this apparently is the point $x$ in the diagram. What I've said is just the result of experience with similar operations that arose from entirely different sources.
Comment: Once you have the coordinate system, the lines are the intersections of the hyperplanes $x_i$ with $S$ and the polyhedron itself is the section of $S$ that lies on the positive side of all of them.
Comment: @PaulSinclair: Thanks paul for your comment. Nice intuition. You are welcomed to write an answer. Can you work out the details in the example that I have given or any other example that you find more useful to convey the idea?
Answer: Answer 1. In the example we have $m=2$, $n=4$ and $n-m=2$. The pertinent matrices and vectors can be defined as follows
\begin{align}
A &= \begin{bmatrix}
1 & -2 & 1 & 2 \\
2 & 1 & 0 & -1
\end{bmatrix} \\
\mathbf{x} &=
\begin{bmatrix}
x_1 & x_2 & x_3 & x_4
\end{bmatrix}^\top \\
\mathbf{b} &=
\begin{bmatrix}
2 & 2
\end{bmatrix}^\top.
\end{align}
It easy to check that
$
\mathbf{u} =
\begin{bmatrix}
1 & 0 & 1 & 0
\end{bmatrix}^\top
$
is a basic feasible solution or equivalently a vertex of $P$. This means that $A\mathbf{u} = \mathbf{b}$, and for the list of basic indices
$
\mathbf{\beta} =
\begin{pmatrix}
1 & 3
\end{pmatrix}
$,
the basis matrix
\begin{equation}
B =
\begin{bmatrix}
a_{\beta_1} & a_{\beta_2}
\end{bmatrix}
=
\begin{bmatrix}
a_{1} & a_{3}
\end{bmatrix}
=
\begin{bmatrix}
1 & 1 \\
2 & 0
\end{bmatrix}
\end{equation}
is invertible and for every $i \notin \beta$ we have $u_i = 0$. Define the nonbasic list of indices as
$
\mathbf{\nu} =
\begin{pmatrix}
2 & 4
\end{pmatrix}
$
and let the nonbasic matrix be
\begin{equation}
N =
\begin{bmatrix}
a_{\nu_1} & a_{\nu_2}
\end{bmatrix}
=
\begin{bmatrix}
a_{2} & a_{4}
\end{bmatrix}
=
\begin{bmatrix}
2 & 2 \\
1 & 1
\end{bmatrix}.
\end{equation}
Introducing the following basic and nonbasic variables as
\begin{align}
\mathbf{x}_\beta =
\begin{bmatrix}
x_1 & x_3
\end{bmatrix}^\top, \qquad
\mathbf{x}_\nu =
\begin{bmatrix}
x_2 & x_4
\end{bmatrix}^\top,
\end{align}
we can rewrite the $A\mathbf{x}=\mathbf{b}$ as $B\mathbf{x}_\beta + N \mathbf{x}_\nu=\mathbf{b}$ and solving for $\mathbf{x}_\beta$ results in
\begin{equation}
\mathbf{x}_\beta = B^{-1}\mathbf{b} - B^{-1}N\mathbf{x}_\nu = \mathbf{u}_\beta - B^{-1}N\mathbf{x}_\nu,
\end{equation}
which in component form reads as
\begin{align}
x_1 &= 1 - \frac{1}{2} x_2 + \frac{1}{2} x_4, \\
x_3 &= 1 + \frac{5}{2} x_2 - \frac{5}{2} x_4.
\end{align}
Now, we can rewrite $P$ by eliminating the basic variables from its representation as below
\begin{align}
P = \Big\{ &\mathbf{x} =
\begin{bmatrix}
1 - \frac{1}{2} x_2 + \frac{1}{2} x_4, &
x_2, &
1 + \frac{5}{2} x_2 - \frac{5}{2} x_4, &
x_4
\end{bmatrix}^\top \in \mathbb{R}^4 \Big| \\
& 1 - \frac{1}{2} x_2 + \frac{1}{2} x_4 \ge 0, \\
& 1 + \frac{5}{2} x_2 - \frac{5}{2} x_4 \ge 0, \\
& x_2 \ge 0, \, x_4 \ge 0
\Big\}.
\end{align}
Next, to make a visualization of $P$, it is suggesting to visualize the following
\begin{align}
Q = \Big\{ &\mathbf{x} =
\begin{bmatrix}
x_2, &
x_4
\end{bmatrix}^\top \in \mathbb{R}^2 \Big| \\
& x_2 - x_4 \le 2, \\[0.3em]
-& x_2 + x_4 \le \frac{2}{5}, \\
& x_2 \ge 0, \, x_4 \ge 0
\Big\}.
\end{align}
To make some mathematical justification for this, we can define an bijective affine function $f:Q\to P$ with $Q\subset\mathbb{R}^2$ such that $f(Q) = P \subset \mathbb{R}^4$ and in particular
\begin{align}
f(\mathbf{x}_\nu) =
f\big(
\begin{bmatrix}
x_2, &
x_4
\end{bmatrix}^\top
\big) &=
\begin{bmatrix}
1 - \frac{1}{2} x_2 + \frac{1}{2} x_4, &
x_2, &
1 + \frac{5}{2} x_2 - \frac{5}{2} x_4, &
x_4
\end{bmatrix}^\top \\
&=
\begin{bmatrix}
1 & 0 & 1 & 0
\end{bmatrix}^\top
+
\begin{bmatrix}
- \frac{1}{2} x_2 + \frac{1}{2} x_4, &
x_2, &
\frac{5}{2} x_2 - \frac{5}{2} x_4, &
x_4
\end{bmatrix}^\top \\
&= \begin{bmatrix}
1 \\
0 \\
1 \\
0
\end{bmatrix}
+
\begin{bmatrix}
-\frac{1}{2} & \frac{1}{2} \\
1 & 0 \\
\frac{5}{2} & -\frac{5}{2} \\
0 & 1
\end{bmatrix}
\begin{bmatrix}
x_2 \\
x_4
\end{bmatrix} \\
&= \begin{bmatrix}
1 \\
0 \\
1 \\
0
\end{bmatrix}
+
\begin{bmatrix}
1 & 0 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 1 & 0 & 0 \\
0 & 0 & 0 & 1
\end{bmatrix}
\begin{bmatrix}
\frac{1}{2} & -\frac{1}{2} \\
-\frac{5}{2} & \frac{5}{2} \\
1 & 0 \\
0 & 1
\end{bmatrix}
\begin{bmatrix}
x_2 \\
x_4
\end{bmatrix} \\
&= \mathbf{u} + \Pi
\begin{bmatrix}
-B^{-1} N \\
I_2
\end{bmatrix}
\mathbf{x}_\nu
\end{align}
Due to the existence of such an bijective affine function, we prefer to visualize $Q$ rather than having no picture of $P$ in mind! It is noteworthy to mention that lines and parallelism are persevered under affine transformations. Furthermore, since both $P$ and $Q$ are convex, it can be shown that there is a one-to-one correspondence between their vertices under $f:Q\to P$. In conclusion, $Q$ seems to be a good candidate for visualizing $P$.
Remark. This is indeed similar to what we do in the implicit function theorem.

Answer 2. Yes, the projection of a polyhedron is a polyhedron. One approach to prove this is by Fourier-Motzkin elimination. See Corollary 2.5 of Chapter 2 of this book.


iframe has zero height inside parent with no height and bottom padding

Question: I'm using the classic responsive design trick of applying a percentage based padding-bottom and zero height to an element in order to make it maintain a certain aspect ratio. Inside this element is an iframe with a height of 100%.
This works an intended in chrome, but firefox and IE doesn't show the iframe, as if it would have no height. I have tried applying box-sizing: content-box as a workaround for IE, but it did nothing.
Fiddle: http://jsfiddle.net/jgsnK/
How can I make the iframe behave like in chrome in the other browsers?
Answer: What you'll need to do is position your iframe using position:absolute; with position:relative; on your .wrapper
.wrapper {
    position:relative;
    height: 0;
    width: 100%;
    padding-bottom: 56.25%;
}
.frame {
    position:absolute;
    top:0;
    right:0;
    left:0;
    bottom:0;
    width: 100%;
    height: 100%;
}

Have a look at this DEMO
FURTHER:
If you plan on doing something like this regularly throughout your document I would suggest adding an internal div that does this same function and leave your iframe without the absolute positioning
HTML

CSS
.wrapper {
    position:relative;
    height: 0;
    width: 100%;
    padding-bottom: 56.25%;
}
.abs-inner{
    position:absolue;
    top:0;
    right:0;
    left:0;
    bottom:0;
}
.frame {
    width: 100%;
    height: 100%;
}

Something like this DEMO
Answer: The height: 100%; means match the height of element with the height of its parent i.e. 0px. You can use relative + absolute positioning to achieve the desired result i.e. match the height with the height of the element plus padding:
.wrapper {
    width: 100%;
    padding-bottom: 56.25%;
    position: relative;
}
.frame {
    position: absolute;
    left: 0;
    top: 0;
    width: 100%;
    height: 100%;
}

Demo here
Note: for pixel perfect results you might need to zero-out the marginwidth, marginheight and frameborder attributes on the iframe.


prove: For any finite-dimension vector space $V$ with ordered basis $\beta$, $\phi_\beta$ is an isomorphism.

Question:
For any finite-dimension vector space $V$ with ordered basis $\beta$, $\phi_\beta$ is an isomorphism.

My work: Let $\beta=(v_1,v_2, \dots ,v_n)$ be a n-dimensional ordered basis for $V$. To check $\phi_\beta$ is an isomorphism, we need to prove it is injective and surjective.
Let $x \in V$, then $x=a_1v_1+\dots+a_nv_n$ for $a_i \in F$.
Injective: we need to prove for all set of vectors $x$ in $V$, $\phi_\beta(x)$={$o_n$}
Here, $\phi_\beta(x)=[x]_\beta$=$(a_1,\dots,a_n)^t=(0,\dots,0)^t$, thus $N(T)={0}$.
To prove surjective, since the linear mapping is defined $\phi_\beta:V \rightarrow F^n$, and that $dim(v)=dim(F^n)$, by the theorem ( Let $V$ and $W$ be vector spaces of equal dimensions, then $T$ is onto), we see it is surjective.
Hence, it is an isomorphism.
Can I leave like that?
Comment: The proof for surjectivity is wrong. Just because the dimensions of domain and codomain equal, it need not be surjective. You need to show that the dimensions of domain, codomain and image equal each other.
Comment: You should define $\phi_\beta$ before you start with your proof.
Comment: the injective part is not as clear and precise as it could be. i am writing one right now
Comment: is injective part ok? Also I forgot to add that there is a definition beforehand that defines $\phi_{\beta}$
Answer: Let $V$ be a $n$-dimensional vector space over the field $F$ and $\beta=(v_1,\dots,v_n)$ a basis.
Define the function
$$
\phi_\beta : V \to F^n, \quad x=\sum_{i=1}^n x_i v_i \mapsto (x_1, \dots, x_n)
\quad \text{for} \quad x_i \in F \,.
$$
The function is an isomorphism iff $V, F^n$ are vector spaces, and $\phi_\beta$ is injective, and surjective.

The vector space argument is given by definition.
Injective: check if $\phi_\beta$ only maps $0_V$ onto $0_{F^n}$.
\begin{align}
  0_{F^n} &\overset{!}{=} \phi_\beta(x)
  = (x_1, \dots, x_n)
  \\
  \implies x_i &= 0_F \quad  \forall i
  \\
  \implies x &= 0_V
\end{align}
Thus, $\phi_\beta$ is injective.
Surjective: check that every vector in $F^n$ is mapped onto.
Let $x_{F^n}=(x_1, \dots, x_n) \in F^n$ be given.
Define $x_V=\sum_{i=1}^n x_i v_i \in V$ and we get $\phi_\beta(x_V) = x_{F^n}$.
Thus, $\phi_\beta$ is surjective and finally we showed it is an isomorphism.