3-dimensional light up cube, # of rows/cols/diags in/on a 4 × 4 × 4 cube

Question:
Imagine a 3-dimensional cube (much like a 4 × 4 × 4 Rubik's cube) except the planes of the cube cannot be twisted individually and instead of faces with different colors, it is clear (see thru) but with a single light in each of the $64$ little cubes contained within the $1$ big cube.
The $64$ lights are connected to $64$ switches, all of them initially turned off.  We now turn on a light by choosing a switch uniformly at random, toggling it (so that the light turns on if it was off, and turns off if it was on), and repeat this lighting process for a total of $32$ times.
At the end, we therefore have at most half of the $64$ lights lit — but the actual number is likely to be less, since some lights may be toggled several times.
I'd like to confirm the answers to the following five questions:
$1$. What will be the average number of lights on after all $32$ random numbers are processed sequentially, meaning that the corresponding lights were toggled properly from them, possibly multiple times each?
$2$. At the end of the lighting process, we want to know what is the probability that only $1$ complete row, column, or diagonal will be illuminated.  That is, $4$ adjacent/connected lights that complete one of those.  It may help if you think about and answer question 3 first.
$3$. How many possible complete unique/distinct rows, columns, and diagonals are there in/on the cube?  Remember they can go thru the cube, not just along the faces.  Also, they can be on the inner planes such as planes $2$ and $3$ (of $4$). They must be collinear.  You cannot go around a corner for example and get $4$ in a row.  All $4$ lights must lie in a line segment of length $4$ with length (in this context) meaning # of adjacent lights illuminated in a collinear fashion.
$4$. What is the ideal number of lights on that will give us the highest probability of having exactly $1$ complete row/col/diag?
$5$. How many random lights (minimum) need to be on to guarantee that at least $2$ complete rows/cols/diags are illuminated?  That is, the probability of exactly $1$ complete row,col,diag becomes $0$.

I used computer simulation and got the following results so far:

On average about $20.4$ lights will be on.
I get about $31.89$%.  The result is somewhat surprising to me because of all the possible ways to get $1$ complete row/col/diag, you would think out of $20$ or so lights (on average), you would get multiple of those more often.  My simulation program is even telling me that getting $8$ complete is even possible, but very unlikely.  I don't know what the max is, maybe $20$ or so if all $32$ lights stay lit and they just happen to be perfectly placed for maximum lit rows/cols/diags.
I counted $76$ distinct complete rows, columns, and diagonals (so far) of length $4$.
It appears that the ideal # of lights on is $23$, which gives about a $39.5$% chance of exactly $1$ complete row/col/diag. $22$ was a close 2nd with a $39.3$% chance. $24$ came in 3rd with a $38$% chance.
I think the minimum is $47$.  The algorithm I used is very simple and very fast. Obviously you cannot have any $4$ in a row, col, or planar diagonal, so I simply started with $48$ (of $64$) lights on, making sure to turn off $1$ in each row, col, and planar diagonal (but not checking 3D diagonals).  My simulation program told me I had $3$ complete "lines" at that point.  I then checked to see if I could get only $1$ line lit by testing each of the $48$ on lights by turning them off one at a time then putting them back on before the next test.  That dropped it to $2$ lines illuminated.  I then looped thru combinations of $2$ on lights (there are $48 \choose 2$ of them which is only $1128$).  That dropped it to only $1$ line lit.  F.Y.I. my program told me there were $27$ two light combos that (when turned off) dropped it to only $1$ line lit.  So with that info, we just take the penultimate distinct # of lit lights we tried (which was $47$) and that is our answer.

The correct answer for Question $2$ depends on getting Question $3$ correct, so that is why I am asking for some help, because I want to get this exactly right.
For the diagonals, I counted $2$ per face, so that is $12$ facial diagonals (imagine an X shape on each face), $12$ inner planar diagonals which are Xs also but on planes $2$ and $3$, twisting the cube in all $3$ dimensions.  I also count $4$ inner "3D" diagonals that go thru the "heart" of the cube.
Can someone please help me confirm these numbers?
Comment: These are 3 separate questions.
Comment: Yes but they are all related so I asked them together here.  Anyone can answer any subset of the questions if they think they have a good answer/method.  I am working on them myself but having some difficulty and could use some help,  I made this scenario up myself and was curious if the math is a "bear" to compute the answer to question $2$.
Comment: What is the source of this problem?
Comment: I made this up myself from "scratch" but it was motivated from a 4x4 lottery ticket then I extended that idea into 3D and added the lights.  I am not getting the "hard" answers I am looking for.  I am getting formulas and methods instead.  I was hoping for actual numbers for answers.
Comment: @David Click on Edit to edit your post. In the editing bar (where it allows you to bold, italicize, etc.,) click on the one where it looks sort of like a picture frame/image of a mountain. It will then allow for you to upload a picture file.
Answer: Question 1
This is a simple Markov chain.
Let $o_i$ be the number of lights ON at step $i$, then
$$o_{i+1}=\begin{cases}
o_i-1,&p=\frac{o_i}{64}\\
o_i+1,&p=1-\frac{o_i}{64}\\
\end{cases}$$
And $o_0=0$
You can build the matrix for this (I'm not going to) and solve for 32 steps.
Question 2
For 4 specific lights in row, calculate the chance that each of them is "hit" an odd number of times and note how many of the 32 shots each combination uses up (hint: it can be $4, 6, \dots,32$).
Then for all other rows, calculate the chance that at least 1 of the bulbs is hit an even number of times (including 0) and allowing for the fact that some bulbs are common with the illuminated row.
Question 3
From each face there are 16 rows/columns. There are 3 independent faces. Therefore there are 48 rows/columns.
Diagonals that are 4 long can only start from the corners. There are 2 in each 1x4 block. There are $4+4+4$ of these. 24.
Plus there are 4 diagonals that go from corner of the cube through the "guts" to the other corner.
$48+24+4=76$
Comment: I like the idea of using a Markov Chain for the first question - it's quite elegant. Though, it would probably be easier just to calculate the probability that a given light is on, and multiply by $64$ (which works because of the linearity of expectation), but not nearly as enlightening.
Comment: The advantage of the Markov chain is it can be extended to answer Question 2. Your idea is much easier for Question 1 - wish I had though of it!
Comment: @Dale M, how do you get only $3$ diagonals?
Comment: There are 3 diagonals with 4 lights - if you are willing to consider diagonals with less than 4 lights then there are more
Comment: There are 4 "full" diagonals that cross from one corner of the cube to the opposite one, not 3. If you also consider "semi-diagonals", where one of the three coordinates is constant, there are $3\cdot2\cdot4=24$ of those.
Comment: Right - my mistake
Answer: Question 1:
This one is actually pretty easy.
It helps to start by considering a single light.
The number of times the light is toggled is binomially distributed, with parameters $n = 32$ and $p = 1/64$, and the probability that it ends up being lit at the end of the process is the sum of the probabilities that it is toggled an odd number of times, i.e.:
$$P = \sum_{k=1}^{16} f(2k-1;32,\tfrac1{64}) = \sum_{k=1}^{16} {32 \choose 2k-1} \left( \frac1{64} \right)^{2k-1} \left( 1-\frac1{64} \right)^{33-2k},$$
where $f(k;n,p) = {n \choose k} p^k (1-p)^{n-k}$ is the probability mass function of the binomial distribution.  A laborious but straightforward evaluation of the sum, best carried out by a computer, then yields the result:
$$P = \frac{932357239278482603486754898815444262218545775615}{2923003274661805836407369665432566039311865085952} \approx 0.318972$$
Now, by the linearity of the expected value, the expected number of lit lamps is simply the total number of lamps times the probability of each lamp being lit, i.e. $64 P \approx 20.41423$.
Ps. I did the calculation in Maxima, but any language that has an arbitrary-precision rational type will do — or you can even use floats, if you only want an approximate numerical value.  Here's the code I used:
f(k,n,p) := binomial(n,k) * p^k * (1−p)^(n−k);
P: sum(f(2*k-1 , 32 , 1/64), k, 1, 16);
float(P);
float(64 * P);

Question 3:
The third question, is a simple exercise in 3D visualization and enumeration:

For each of the three axes, there are 16 columns running parallel to the axis for a total of 3 × 16 = 48 columns.
For each of the three axes, there are also 2 × 4 = 8 diagonals running perpendicular to that axis (from one edge of the cube to the opposite), for a total of 3 × 8 = 24 diagonals.
Finally, there are 4 "inner diagonals" running from one corner of the cube to the opposite one.

Thus, the total number of ways you can arrange four sub-cubes in a row in a 4 × 4 × 4 cube is 48 + 24 + 4 = 76.

Question 5:
For this question, we can equivalently ask for the minimum number $n$ of unlit lamps needed to get at least one unlit lamp in every line of four lamps except one.  The minimum number of lit lamps needed to guarantee at least two fully lit lines of four lamps is then $k = 64 - n + 1$.
An upper bound for this is given by the fact that each 4×4×1 plane must contain at least four unlit lamps (as otherwise it would contain at least one fully lit row and one fully lit column).  Thus, it is clear that at least 16 unlit lamps are needed to keep the number of fully lit lines below two.  Thus, $n \ge 16$, and so $k \le 49$.
Conversely, the following configuration (0 = off, 1 = on; each square is one slice of the cube) demonstrates that $5 + 4 + 4 + 5 = 18$ unlit lamps can be arranged to leave only one fully lit line of four lamps (an inner diagonal from top left to bottom right):
1100  0111  1011  0110
1011  1110  1101  0111
1110  1011  0111  1101
0111  1101  1110  1011

Thus, the minimum number of lit lamps needed to guarantee at least two fully lit lines of four is at least 47, and no greater than 49.
In fact, it turns out that the configuration above is optimal (although not uniquely so), and that the minimum number of lit lamps needed to guarantee at least two fully lit lines of four is $k = 47$.  I have confirmed this by first generating all the 1636 possible 4 × 4 planes with 4 or 5 unlit lamps and 0 or 1 fully lit lines, and then searching all possible combinations of such planes for a combination with at most 17 unlit lamps and at most 1 fully lit line.  No such combination was found, proving that $n = 18$ unlit lamps is the minimum needed to achieve less than two fully lit lines.
(The trick that made the search feasible was making sure to prune early any partial configurations containing two fully lit lines or two planes with 5 unlit lamps, since no such combination can appear in an $n=17$ solution.  This pruning allowed me to only check 40,478,800 full cubes, which was easily doable in a few minutes even on my old laptop.)
Ps. The configuration above was found by noting that there are only 10 ways to arrange 12 lit lamps and 4 unlit ones on a 4 × 4 plane without getting any fully lit rows, columns or diagonals.  It further turns out there's a way to stack four such planes into a cube in such a way as to leave only three inner diagonals (and no other lines of four lamps) fully lit:
1101  0111  1011  1110
1011  1110  1101  0111
1110  1011  0111  1101
0111  1101  1110  1011

Turning off two more lamps then leaves only one of the diagonals fully lit.

Question 2:
Your even-numbered questions seem, at least to me, quite a bit harder than the odd-numbered ones.  I don't have an analytical solution for them, but I did run a stochastic simulation of my own to confirm your results.  It does appear that the probability of getting exactly one fully lit line of four lights with 32 toggles is about $0.318951 \pm 3\times10^{-6}$ (95% confidence interval, calculated using a normal approximation from 100 billion trials).  The proximity of this value to the probability of a single lamp being lit seems like a rather remarkable coincidence, but I can't really see any way it could be anything but a coincidence.
(The maximum number of fully lit lines of four lamps I observed in my simulation was 16, but the theoretical maximum obtainable is at least 20, achieved by lighting all 32 lamps in one half of the cube.  I suspect that's the best one can do with 32 toggles, although I haven't proven that.)

Question 4:
As for this question, my stochastic simulation confirms your result that 23 lights is optimal, with a probability of $0.3945 ±0.0001$ of getting exactly one line of four lit lamps.  For randomly toggled lamps, as in your first question, it appears that the optimum is either 36 or 37 toggles, both of which give a probability of $0.3297 ±0.0001$; I'd need to run more trials to tell if there's any difference between them, and if so, which way.
Interestingly, the expected number of lit lamps is only about 21.8 for 36 toggles, or 22.1 for 37 toggles, about one lamp short of the optimal fixed light count of 23, showing that the variance and higher moments of the light count distribution also play a role.
Comment: Also, I see a way to get $21$ "lines" lit with $32$ on lamps.  Fill the entire front face (that is $16$ lights and $10$ "lines"), then fill the rest of the top, ($12$ more lights gives $9$ more lines), then on the left side, fill the 2nd row from the top ($3$ lights and $1$ line), then finish the diagonal on the side face ($1$ light, $1$ line).
Comment: Actually, my first arrangement had 40 lit lamps, not 30; that was a silly mistake.  But I managed to improve it by two, based on the arrangement you posted.
Comment: @David: Uhm... but I have a counterexample right there, showing how to arrange 42 lights so that there's only one line of four.
Comment: @David: I managed to narrow the answer to question 5 down to between 47 and 50; see edited answer above.
Comment: Ilmari, my original "hunch" of $48$ seems like a good guess then.  When I have more time later, I will code your new matrix and help verify it for correctness.  Gotta love computers for tedious counting problems ("on paper").  Also, I think a mathematical proof of this type of question would be interesting.  Can a "stochastic simulation" be used as a mathematical proof?
Comment: In principle, that'd work, but the problem is that just the number of combinations of 47 lit lamps is ${64 \choose 47} = 1{,}379{,}370{,}175{,}283{,}520$. That makes naïve brute force search a bit impractical (although just barely feasible, if you have enough spare computers lying around). I do think, however, that with some clever optimizations it should be possible to reduce the search space enough to make it practical. (Ps. I got the upper bound down to 49, by realizing that 15 unlit lamps must always leave at least three fully lit lines of four.)
Comment: The optimization I was mainly thinking of was precomputing the 4×4×1 planes with 3, 4, 5 or 6 unlit lamps and 0 or 1 fully lit lines. Any cube with only one fully lit line can only contain at most one plane with such a line, and thus all the other planes must contain at least 4 unlit lamps. For 17 unlit lamps, that means we can have at most one plane with 6 unlit lamps, or two with 5. That should leave a small enough number of combinations to easily check by brute force.
Comment: @David: I just ran an exhaustive search, and can confirm that the answer to question 5 is 47.
Comment: Impressive, nice work. I will type into my program and try to confirm. I wonder if it would have been easier to start with $48$ (since that is exactly $3/4$ of $64$), and just have the program turn off $1$ lit light at a time, looking for a drop from $(2+)$ complete lit lines to only $1$. If none were found, then reinitialize to $48$ and drop all combinations of $2$ lit lights (leaving $46$ on). That might work if a "winning" $46$ matrix can be made from the initial $48$ matrix minus some $2$ light combination. $48 \choose 2$ (=$1,128$) cases is a very small number for a computer to simulate.
Comment: Check out this link to a $4$x$4$x$4$ lighted cube controlled using an Arduino.  I wish they would show all $76$ "lines" one a a time.  It is amazing that the # of combinations of lighting $32$ of the $64$ possible lights is almost $2$ quintillion (almost $2$ * $10^{18}$) which is about $10$% of all possible $2^{64}$ combinations of lights. There are many of these videos on YouTube but I liked the music on this one: https://www.youtube.com/watch?v=r1cjHSGuNTU
Comment: Ilmari, thank you for your assistance on these $5$ questions.  Your analysis and explanations were very informative and gave me a deeper understanding of this interesting set of questions.  Of course they could be extended to larger cubes, but then the code needed to check it may become much longer (depending on the language used).  For example what if we instead had an $8$x$8$x$8$ cube?  I find the math, algorithms, analysis, speed of simulation, visualizations... all very interesting for problems like this.  I awarded you the +100 bounty and appreciate all your hard work on this.
Comment: Just F.Y.I., to get the optimal $23$ lights on to give the highest chance of getting exactly $1$ "line" lit requires $40$ random toggles. Also, no matter how many (large number of) random toggles you do, the # of lights on will "settle" to $32$ which is half. I did $100,000$ iterations of $1,000$ random toggles and my simulation program is telling me $32$ lights should be lit on average. When I ran the simulation, the minimum I got (per iteration) was $16$ and the max was $48$.  I wonder what is so special about those numbers.
Comment: @David: Yes, the mean number of lights lit will converge to 64/2 = 32 as the number of flips tends to infinity. You can see this from the fact that, if the number of lights currently lit is less than 32, a random flip is more likely to increase it, whereas if it is greater than 32, a random flip will most likely decrease it. By similar reasoning, one can even calculate the stationary distribution of the number of lights lit after a large number of flips. I don't think there's anything very special about 16 and 48, except that they're about the same distance from 32.


Stop the code after a time period

Question:
Possible Duplicate:
Stop running the code after 15 seconds

I am dealing with the code of a packet sniffer and I just want to make some modifications to it. Now I am trying to edit it so that once I start the program, it will only capture the packets for 15 seconds. Below is the part of code which intercepts the packets and as you can see I am dealing with Try/Catch/Throw and it works like a loop.
        public void Start() {
            if (m_Monitor == null)
            {

                try
                {

                    m_Monitor = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.IP);
                    m_Monitor.Bind(new IPEndPoint(IP, 0));
                    m_Monitor.IOControl(SIO_RCVALL, BitConverter.GetBytes((int)1), null);
                    m_Monitor.BeginReceive(m_Buffer, 0, m_Buffer.Length, SocketFlags.None, new AsyncCallback(this.OnReceive), null);
                }
                catch
                {
                    m_Monitor = null;
                    throw new SocketException();
                }
            }
    }
    ///
    /// Stops listening on the specified interface.
    ///
    public void Stop() {
        if (m_Monitor != null) {
            m_Monitor.Close();
            m_Monitor = null;
        }
    }
    ///
    /// Called when the socket intercepts an IP packet.
    ///
    /// <param name="ar">The asynchronous result.</param>
    ///
    private void OnReceive(IAsyncResult ar) {
            try
            {
                int received = m_Monitor.EndReceive(ar);
                try
                {
                    if (m_Monitor != null)
                    {
                        byte[] packet = new byte[received];
                        Array.Copy(Buffer, 0, packet, 0, received);
                        OnNewPacket(new Packet(packet));
                    }
                }
                catch { } // invalid packet; ignore
                m_Monitor.BeginReceive(Buffer, 0, Buffer.Length, SocketFlags.None, new AsyncCallback(this.OnReceive), null);
            }
            catch
            {
                Stop();
            }
    }

How do you think I can modify this code so that it will be stopped once started after 15 seconds for instance? I have tried to use DateTime but it was not successfull and I couldn't kind of break this so-called loop.
Comment: well this is the extended question... I was asked to open another question...
Comment: @iSi Can you add some code which actually invokes `Start()`
Answer: I think you need to measure time and stop the code after particular time say "15 seconds" ,StopWatch class can help you out.
// Create new stopwatch instance
Stopwatch stopwatch = new Stopwatch();
// start stopwatch
stopwatch.Start();
// Stop the stopwatch
stopwatch.Stop();
// Write result
Console.WriteLine("Time elapsed: {0}",stopwatch.Elapsed);
// you can check for Elapsed property when its greater than 15 seconds
//then stop the code

Elapsed property returns TimeSpan instance you would do something like this.
TimeSpan timeGone = stopwatch.Elapsed;

To fit your scenario you can do something like this
Stopwatch stopwatch = new Stopwatch();
TimeSpan timeGone;
// Use TimeSpan constructor to specify:
// ... Days, hours, minutes, seconds, milliseconds.
// ... The TimeSpan returned has those values.
TimeSpan RequiredTimeLine = new TimeSpan(0, 0, 0, 15, 0);//set to 15 sec

While ( timeGone.Seconds < RequiredTimeLine.Seconds )
{
    stopwatch.Start();
    Start();
    timeGone = stopwatch.Elapsed;
}
Stop();//your method which will stop listening

Some useful links
MSDN StopWatch
Note:
Im' sure that i have given enough pointers that anyone can sort his problem through these basic approach ,kindly also check my comments for another approach.
Comment: @iSi I checked your code again but could'nt find what you posted here ,How about using events if time == 15 seconds raise `onStop` which can then call stop method.
Answer: You could use
System.Threading.Thread.Sleep(15000);

and put the code after it. the code will be executed after the defined delay.
Good Luck
Comment: This is not my intention... this will only put the thread at sleep for 15 seconds and then will run it anyways...
I need to start the program and then let it run for 15 seconds and then have it stopped automatically...
Comment: @iSi check my suggestions ,and let me know
Comment: @dotNetSoldier I undetstand your point... You want me to write a condition to check the elapsed time and if it is exceeding 15 seconds, it will stop, right?
Well, I guess I have basic problems with the program as I am kind of lost between the Try/Catch pieces of the code and have no clue where to put your codes...
Comment: @iSi I'm smiling right now this happens when our thoughts are too much occupied due to the problem , I will try to give you some pointers..
Comment: @dotNetSoldier in the code that I am trying to edit, it just does not work... I edited it in 100 different ways but just does not work. do you think you can download this code from here and see if you can change it the way I want? (The code you get from this link will be a bit different from the one you see here though)
http://www.codeproject.com/Articles/17031/A-Network-Sniffer-in-C
Comment: @iSi I didn't find the code in the question on the codeProject uploaded project.
Comment: @dotNetSoldier that is a different one actually but kind of similar... please check this new one on the MJsnifferForm.cs file...
Answer: Given that your code uses a separate thread to do the actual work, you could set it up with this rather small amount of code:
using System;
using System.Threading;

namespace ConsoleApplication1
{
    class Program
    {
        static YourSnifferClass sniffer = new YourSnifferClass();
        static AutoResetEvent done_event = new AutoResetEvent(false);

        static void Main(string[] args)
        {
            sniffer.Start();

            // Wait for 15 seconds or until the handle is set (if you had a
            // Stopped event on your sniffer, you could set this wait handle
            // there: done_event.Set()
            done_event.WaitOne(TimeSpan.FromSeconds(15));

            // stop the sniffer
            sniffer.Stop();
        }
    }
}

Hope that helps.

Stop the code after a time period


ASP.net checkbox always checked

Question: <asp:CheckBox ID="isSubscribed" runat="server" /> Subscribe to mailing list?

<asp:Button runat="server" CssClass="btn" ID="btnPrefUpdate" OnClick="updatePrefs" Text="Update" />

This fires in the code behind:
protected void updatePrefs(object sender, EventArgs e)
{
    Response.Write(isSubscribed.Checked);
    Response.End();
}

But it's always coming out as true!  Whether it is checked or not!  I know I'm doing it wrong, could someone show me how to access this value properly?
Comment: Do you have anything in your Page_Load event which ever sets it as true? Have you tried putting "isSubscribed.Checked.ToString"?
Comment: Ah yes, in my page load I have isSubscribed.Checked = Master.isSubscribed; how do I get the on click event to run before that one?
Comment: as a side note I think Respose.Write is evil and should be avoided if possible. It can make debugging your code a nightmare :(
Answer: like @Curt said, it looks to me like you have something in your page_load. if you set the value in Page_Load make sure it is inside the following if statement
if(!Page.isPostBack)
{
    isSubscribed.Checked = true;
}

Answer: You are doing it the right way. The boolean property Checked should just say True or False (I even tested it). Is your Page_Load doing something with the checkbox? In other words, is the value of the checkbox somehow (re)set when the post back is occuring (the postback of the button click).
In your Page_Load method you could include:
if (!this.IsPostBack)
{
// Set default or loaded values for controls
}

Comment: Yes, stupidly it sets it to the profile value in page load, how can I stop that code from loading before the button click event?
Comment: @Tom Gullen see my answer, or you could set it in the aspx page
Comment: if (!this.IsPostBack)
// Do your thing.

ASP.net checkbox always checked


Maven Class path error multiple SLF4J bindings

Question: I have been getting this error while trying to do a MAVEN INSTALL. I tried exclusions, but not sure the where to include in pom file. Let me how and what exclusion tags should i include in my pom file. I am also attaching my pom file snippet where to include the exclusions`SLF4J: Class path contains multiple SLF4J bindings.

SLF4J: Found binding in
  [jar:file:/C:/Users/147188/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
  [jar:file:/C:/Users/147188/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.10.0/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
  explanation. SLF4J: Actual binding is of type
  [ch.qos.logback.classic.util.ContextSelectorStaticBinder]

POM file:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <exclusions>
    <exclusion>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
    </exclusion>
</exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>

    <exclusions>
    <exclusion>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
    </exclusion>
        <exclusion>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-logging</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-log4j2</artifactId>
    <exclusions>
    <exclusion>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
    </exclusion>
</exclusions>
</dependency>

Comment: Did you solved that?
Answer: 1 Run
mvn dependency:tree

to see which package import org.slf4j
2 keep one, and exclude other
   <exclusion>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
    </exclusion>

Comment: to be more specific , you can use "mvn dependency:tree -Dincludes=:slf4j*"
Answer: Correct way to exclude default logging, and configure log4j for logging.
Add this dependency in Spring Boot project, if it is not already there
<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter</artifactId>
 <exclusions>
     <exclusion>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-logging</artifactId>
     </exclusion>
 </exclusions>
</dependency>

Refer Spring Logging - How To.
Comment: the only answer that helped me. thanx
Answer: That message indicates that you are bringing in both logback-classic and log4j-slf4j-impl, which both want to be the logging framework for SLF4J to bind to. If you're not sure which dependency is bringing in which other dependencies, I find it very useful to run "mvn dependency:tree" to see the tree of dependencies being used. That should give you enough information to figure out which logging framework binding you need to exclude.
As the documentation that the warning points to you says,

Embedded components such as libraries or frameworks should not declare a dependency on any SLF4J binding but only depend on slf4j-api. When a library declares a compile-time dependency on a SLF4J binding, it imposes that binding on the end-user, thus negating SLF4J's purpose. When you come across an embedded component declaring a compile-time dependency on any SLF4J binding, please take the time to contact the authors of said component/library and kindly ask them to mend their ways.

You pretty much want to exclude all actual logging frameworks from all dependencies, so that the only logging framework being used is the one you've explicitly added. I even often find it useful to set up some maven-enforcer-plugin bannedDependencies rules to ensure that I don't accidentally bring in another logging framework when updating my dependencies. It can also be helpful to use dependencyManagement sections in your POM to ensure that all your dependencies use the same version of slf4j-api.
Answer: I ran into same log4j-slf4j multiple binding issue. There are multiple reasons which caused this issue (Struggled a lot for this issue :) ). Please find below comments.

I was using Spring-Boot version 1.4.7, which internally configured with slf4j-1.7.25 version. Below is the url link, where you can find Spring-Boot version and respective module versions.

https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-dependencies/
It seems when you use "spring-boot-starter" dependency, SB(Spring-Boot) will automatically download all these dependent modules. In my project, i was using slf4j-1.7.21, which created multiple slf4j version issue. In short, SB was not able to identify which slf4j version to use at runtime. So, first I changed my slf4j version to 1.7.25(which is compatible with Spring-Boot version 1.4.7).

Next you need to exclude right module (in my case exclusion of log4j-over-slf4j, logback-classic worked). You need to add below exclusions in all of spring-boot-starter-** module. For Eg : Here I am adding exclusion for spring-boot-starter-data-redis.

Hope this will resolve issue.


Angular.js Directive Template - If Then Conditional For What Template To Use

Question: In angular.js, for the directive I would like to specify the property of the template key according to what the attr.checkType is using with if then. How could I do this? The only place I know to place logic would be in the link function, but the template key/property goes outside the link function.
angular.module('monitorApp')
.directive("countDown", [ 'sseHandler', function (sseHandler) {
    function _bindMessage(scope, message) {
        scope.countFrom = message;
    }
    return {
        restrict: 'E',
        scope: {
            countFrom: "=",
            checkType: "@"
        },
        //WOULD LIKE TO PLACE IF THEN TO SPECIFY WHAT TYPE OF TEMPLATE TO
        //USE ACCORDING TO ATTR.CHECKTYPE.
        template :  if (attr.checkType == "foo") {
                      '
'+
                      'Next update in &nbsp {{countFrom}}'+
                      '&nbsp seconds} else
                       {

 HI

                       },

        link: function(scope, element, attr){
           scope.countFrom = sseHandler.broadcastStamp[attr.checkType] || "initializing";
           scope.$on('ONE_SEC', function() {
               resultType = attr.checkType;
               seconds = sseHandler.broadcastStamp[attr.checkType];

               if (seconds < -3) {
                   seconds = "Snap! Broadcast is late." +
                          " Something is broke";
               }
               else if (seconds <0) {
                   seconds = "running a little late";
               }
               _bindMessage(scope,seconds);
           });
        }
    }
}]);

Answer: You simply need to use a single template, and use ng-if inside the template:
template : '
' +
               '
'+
                  'Next update in &nbsp {{countFrom}}'+
                  '&nbsp seconds}' +
               '
' +
               '
HI
' +
           '
'

Answer: Below you'll find a directive that choose among a list of templates according to the "temp" attribute value
module.directive('templateSwitch', ['$compile', function ($compile) {
var getTemplate = function (Type) {
    var typelist = {
        template1: '
view1
',
        template2: '
view2
'
    };
    return typelist[Type];
}
return {
    restrict: 'E',
    link: function (scope, element, attrs) {
        var templatelinker = function (listType) {
            var html = getTemplate(listType);
            element.html(html).show();
            $compile(element.contents())(scope);
        };
        templatelinker(attrs.temp);
    }
};
}])

Use it as follow
<template-switch temp="template2"></template-switch>


Other ways of customizing Unity web player

Question: I'm using the free version of Unity. I saw that there's a way around custom in-game loading screens with "Application.GetStreamProgressForLevel" instead of "Application.LoadLevelAsync".
I know you can change it in Unity Pro; is there a method of changing the screen in the free version?
Answer: From the docs:

"[M]odifying the loader images is only possible with Unity Pro."

Unfortunately, there's no way around it with the free version.


Kubernetes + Rancher: multiple VLANs

Question: I intend to deploy a k8s + Rancher cluster on my local network, but my environment has several VLANs, with pfsense acting as a firewal and router between such VLANs.
My cluster resides in XCP-NG as a hypervisor and I will inform the VLANs that it should pass on to the cluster nodes.
I intend to have some services in different VLANs, because I have VLAN for development, DMZ, production, management, etc., in that I would like to know if I have to take a different approach during the deployment of K8s + Rancher due to my environment?
To deploy a cluster that works with pods on multiple VLANs, must the cluster nodes have multiple NICs, each on a VLAN that I intend to use?
For example, if my cluster has 6 nodes, 3 master and 3 workers, must they be in the same VLAN or are they in different VLANs and having communication between them is enough?
If I want to deploy a pod on the development VLAN, and my cluster resides on the management VLAN, would that be possible?
Thanks in advance for your help.
Comment: Typically, you wouldn't implement that type of access restriction external to k8s. K8s itself has many options for network policies and ACLs, depending on which network subsystem you choose.
Answer:
If I want to deploy a pod on the development VLAN, and my cluster
  resides on the management VLAN, would that be possible?

This is not possible, kubernetes Clusters have it's own internal network. This network is completely segregated from your local network.
While deploying your kubernetes cluster (doesn't matter if it's rancher or any other on-premises kubernetes cluster) you can define on which CIDR your cluster will sit on.
You may be thinking: So if kubernetes has it's own network, how can I talk to the applications I deployed in my cluster?
You can expose your resources by using a Service or a Ingress. For example: When you create a service with type: LoadBalancer your service will allocate a external or public IP address (endpoint)  that can be accessed from your internal network.
$ kubectl get svc
NAME                                      TYPE           CLUSTER-IP   EXTERNAL-IP      PORT(S)                      AGE
custom-nginx-svc                          LoadBalancer   10.0.10.18   188.8.131.52   80:31549/TCP                 11d
echo-svc                                  LoadBalancer   10.0.10.14   188.8.131.52   80:30668/TCP                 11d
kubernetes                                ClusterIP      10.0.0.1     <none>           443/TCP                      11d
nginx-ing-nginx-ingress-controller        NodePort       10.0.9.184   <none>           80:31745/TCP,443:31748/TCP   25h
nginx-ing-nginx-ingress-default-backend   ClusterIP      10.0.1.169   <none>           80/TCP                       25h

As can be seen in the example above, there are two services with external IP defined.
In your scenario you need these External IPs to be IPs from your local network. This can be achieved using MetalLB.
In MetalLB you can define which IPs from your local network will be used. For example, the following configuration gives MetalLB control over IPs from 192.168.1.240 to 192.168.1.250:
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.1.240-192.168.1.250

This is tying MetalLB to only one range and that's not what you need. So, please take a look at this article where it's explained how you can create IPPools and use them.

Kubernetes + Rancher: multiple VLANs


Rigorous Covering Space Construction

Question:
Construct a simply connected covering space of the space $X \subset \mathbb{R}^3$ that is the union of a sphere and diameter.

Okay, let's pretend for a moment that I've shown, using van Kampen's theorem or some other such method, that X has the fundamental group $\mathbb{Z}$, and I have in mind a covering space that consists of a row of spheres each connected to the next by a line segment, along with a map from that row of spheres to X that takes the line segments to the diameter, and the spheres to the sphere. It's not hard to sow that this choice is simply connected.
My question is not how to do all of that. My question is, at this point, how do I show for certain that it is a covering space for X? The only definition I can find for a covering space, hence the only way to be certain that I have one in hand, is this:

A covering space of X is a space X' with a map $p: X' \rightarrow X$ so that there exists an open cover $U_\alpha$ of X so that for each $\alpha$, $p^{-1}(U_\alpha)$ is a disjoint union of open sets in X', each of which is mapped by p homemorphically onto $U_\alpha$.

First problem: What is the best/easiest way, handed some space X in general and with a covering space X', p in mind, to find an open cover that might fit this bill? I'm sure it isn't that hard for this particular case, but it's easy to imagine (and I think  I even have on another problem in the same set) a more difficult case.
Second problem: A generic open cover might consist of uncountably infinitely many sets. How can I be certain that every single one of the $U_\alpha$'s has a preimage that's a disjoint union of sets, each of which is homeomorphic by p to $U_\alpha$? I've had some trouble with correctly showing two spaces to be homeomorphic by a map in the past, and I don't want to make a mistake of that kind again.
I may be worrying over nothing here, but whenever geometric intuition comes into algebraic topology I'm never completely sure just how much explanation is considered enough for that intuition, or how to translate it into a more rigorous proof of the idea. So far I've made the mistake of not explaining enough, and I'd prefer to err on the other side from now on.
(I know my "X'" should be X-tilde, but don't know how to code for that)
Comment: $\tilde X$ can be done via `\tilde X`. Also try `\widetilde X` $\widetilde X$
Answer: Without information about $X$ you can't really write an "explicit" universal cover. What you can do is write a generic $\widetilde X$ using some general construction (for example the one found in Hatcher, $\widetilde X = \{ (x, \gamma) \mid \gamma : I \to X, \gamma(0) = x \} / \sim$). But that's not a very maniable construction.
If you're able to compute explicitly the fundamental group and find for example a "small" generating set and simple relations between the generators, it's easier to find the universal cover -- as you did in this example. Basically for every generator you cut your space in two in the middle of the loop, and then you glue an infinite number of spaces in a string (like you did). Then if there are relations you identify things. But this technique starts to become very complicated when there are many generators, or when it's not obvious how to "cut out" the space (think about $\mathbb{RP}^2$, what do you cut?)...
The upshot of all that is that describing the universal cover explicitly isn't easy in general. Working with the properties of the universal cover is easier, namely: it's simply connected and it's a covering space of the original space... (Which implies all the nice lifting properties you know). Knowing that it exists is a powerful theorem, and one isn't usually too concerned with how the universal cover looks (well, I guess it depends on applications).
As for checking whether a map is a covering map: it's enough to check the covering condition for a basis of the topology. So in your example, you can check it for very small open sets that are homeomorphic to either a segment, a disk, or a disk with a segment glued in the middle (a picture will make all that easier). For these open sets, it's more or less easy to check the covering condition, if you've defined your covering space correctly.
Answer: For a space $X$ which does admit a universal cover there is a quite explicit construction but you need  to take account of all base points of $X$, as Najib states, and which can be given in the following form.
First one forms the fundamental groupoid $\pi_1(X)$. There are two functions $s,t:\pi_1(X) \to X$, namely the source and target.  As a set, the universal cover $\tilde{X}_x$ of $X$ at a point $x \in X$ is just $s^{-1}(x)$, the Star of $\pi_1(X)$ at $x$, or, equivalently, the Costar, $t^{-1}(x)$.  Under the given local conditions on $X$ the topology on $X$ can be lifted to a topology on $\pi_1(X)$, making it in fact a topological groupoid. This method of using lifted topologies via covering morphisms of groupoids, in particular covering morphisms of $\pi_1(X)$, is dealt with in Chapter 10 of Topology and Groupoids. In particular, 10.5.8 shows how to topologise  $\pi_1(X)$ (actually it deals with $\pi_1(X)/N$ where $N$ is a normal subgroupoid of $\pi_1(X)$).
Now you may be able to see how to construct the universal cover of you example, at a specific point of it.
Alternatively, the homotopy type of the example you give is that of $S^2 \vee S^1$, and the universal cover of that at the wedge point is easily seen to be a copy of $\mathbb R$ with an $S^2$ glued to each integer point. Can you see from this how to get the universal cover of your example at a particular point, e.g. one end of the given diameter?
Update September 24, 2014: Here is a description of the universal cover. Consider the real line $\mathbb R$ with a blob at each integer point. Now expand each blob to a $2$-sphere. That is the universal cover at some point of your example $X$.
To see how this fits with the construction, star at one point $a$ of your $2$-sphere $S$ with $b$ as its opposite point. Now $S$ is simply connected so there is one path class in $\pi_1S(a,x)$ for each $x \in S$. However from $\pi_1X(a,b)$ we get more path classes going back along the diameter joining $b$ to $a$. These give the line joining the first sphere to the next one.
I also think this is a good example to compare with the more usual one of the universal cover of $S^2 \vee S^1$.

Rigorous Covering Space Construction


Need open source simple 3D software renderer

Question: Is there any good open source software renderer for C#?
I need a small and simple library for rendering simple 3D objects, Not DirectX or XNA.
Comment: because they need to be installed and they are not open source.
Comment: OpenGL isn't really open source either, so I don't think (on Windows, at least) you'll find a fully open source hardware renderer. Typically, however, that's not an issue, and plenty of open source software uses one or the other. It depends on your intent/requirements.
Comment: Did you find one? How simple do you need it?
Answer: Try Axiom which is a C# port of Ogre 3D. I have not worked with Axiom myself, but with Ogre 3D and its a robust 3D library with a good community. Alternatively check out the game engine list on GPWiki and see if you find anything that suits your needs.
Comment: Axiom is not a software renderer, it uses DX or OpenGL (SlimDX and other C# wrappers) underneath.
Comment: The question _specifically states_ not XNA (so Axiom's similarity further rules it out) and asks for a **software** renderer. This is not an answer to the current question (why the OP needs a software renderer, I don't know, but for now, that's what it says).
Comment: XNA is not a software renderer per sé either but a programming framework which, among other things, supports 3D rendering - a similarity it shares with Axiom. IMHO the OP's main intend was to find some libraries he can use to render 3D objects without being restricted by some kind of license.
Comment: It is not the similarity that plays a role but what *distinguishes* both, namely Axiom being Open Source. But as, as you state, we do not exactly know what the OP actually needs the libraries for, and as long as that is the case any discussion is just an exchange of opinions.


Fail2ban banning odd IP address that are not in logs (0.0.0.4, 0.0.0.5, etc)

Question: Server: Nginx
Fail2ban version: v0.9.3
It seems like no matter what I try, I cannot get fail2ban to find the correct host from the log entry consistently.
/etc/fail2ban/filter/expanse-bot.conf:
[Definition]
failregex = ^(\d{2}|\d{3}) \| <HOST> \| .*\"Expanse indexes the network.*

The ^(\d{2}|\d{3}) catches port 80 or 443. I originally tried a wildcard at the beginning of the line with ^.*<HOST>, but that did not work.
Log entry:
443 | 18.104.22.168 | - | [14/Mar/2021:11:08:23 -0500] | redacted-domain.com | "GET / HTTP/1.1" | 200 | 144126 | "-" | "Expanse indexes the network perimeters of our customers. If you have any questions
 or concerns, please reach out to: firstname.lastname@example.com" | - | 18.104.22.168:1234

In /var/log/fail2ban.log, it is showing these odd IP addresses which do not exist in the log:
2021-03-14 11:07:02,716 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 0.0.0.3
2021-03-14 11:07:03,656 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 0.0.0.4
2021-03-14 11:07:03,865 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 0.0.0.5
2021-03-14 11:07:04,075 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 0.0.0.6

However, then it bans correctly?:
2021-03-14 11:13:48,075 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 18.104.22.168
2021-03-14 11:13:51,288 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 126.96.36.199
2021-03-14 11:15:19,595 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 188.8.131.52
2021-03-14 11:16:30,884 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 220.127.116.11
2021-03-14 11:18:14,208 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 220.127.116.11
2021-03-14 11:19:39,513 fail2ban.actions        [10818]: NOTICE  [expanse-bot] Ban 188.8.131.52

The configuration in my /etc/fail2ban/jail.local:
[expanse-bot]
enabled = true
filter = expanse-bot
logpath = /var/log/nginx/access.log
port = http,https
maxretry = 1
findtime = 3
bantime = 86400
action = iptables-allports[name=expanse-bot]

Once it passes through those odd IP addresses, it does as intended. I just don't understand what the 0.0.0.* means when it's not present in the log? Is there something I am missing in the main configuration?
Answer: Although I don't understand how it is possible with anchored failregex like your, I'll try to explain how fail2ban works here.
Tag <HOST> can find hosts also by its hostname. To avoid this you can either use tag <ADDR> instead (>= v.0.10 only) or set usedns = no in jail.
Don't you really have some lines starting with 2-3 digits and pipe and some text hereafter? I mean some multi-line log line...
Fail2ban normally stores the matches to database, so you can also gets the matched lines from there:
sqlite3 'file:/var/lib/fail2ban/fail2ban.sqlite3?mode=ro' "select * from bans where jail = 'expanse-bot' and ip like '0.0.0.%'"

By the way (\d{2}|\d{3}) can be replaced by \d{2,3} and catch-all .* at end does not matter at all and can be safely removed (RE is not anchored at end of message). But .* in the middle (before "Expanse indexes the network") can be replaced by something more precise.
If your RE was unanchored initially (and this odd addresses occurring at restart time) - this may be old, previously matched and banned tickets (fail2ban restores active IPs after restart).
Just remove them with manually unbanning, see fail2ban-client --help | grep unban.
Comment: It appears I had forgotten `usedns` was not set to "no." That fixed the problem. Thank you. I should have thought to check the sqlite table, but removed the file and restarted fail2ban before I could see it.


Does Spring Transaction Management Work with Spring WebFlux?

Question: Does Spring's support for RDBMS transaction management also work in Spring WebFlux?
For example, assuming proper configuration, will a method annotated with the @Transactional annotation use the Spring transaction manager and rollback the transaction if an error occurs?
If transaction management does work, must a @Transactional method actually throw and exception, or must the Mono or Flux return type emit an error signal?
I know JDBC is inherently blocking, and thus any JDBC operations must be bridged from blocking to reactive, or vice-versa.
The Spring transaction manager works by using a ThreadLocal (right?) which I'm assuming won't work in a Reactor environment, since Reactor is frugal with threads and a single thread can swap out one unit of work for another while the first is waiting on I/O.  I know Reactor has the Context object which is conceptually similar to ThreadLocal (right?) but I haven't seen any documentation that mentions that transaction makes use of it.  Additionally, all JDBC operations that occur within a transaction must use the same Connection which might be tough to do in a reactive context.
My organization has experience with WebFlux and Cassandra, but Cassandra has a native reactive driver.
Thank you!
Answer: EDIT: This answer is not valid anymore for Spring Framework version 5.2 M2 and higher. See this article. Thank you @Florent Dupont for mentioning this.
AFAIK the Spring standard transaction management does not work with WebFlux.
Using the @Transactional will not work because when an annotated method is called, the transaction mechanism will save the state of the transaction inside the ThreadLocal of the calling thread. As you have said it yourself, this does not work. It blocks AND it shares state.
However, you can use a .runOn(Schedulers.parallel()) in order to send the blocking code to another thread. This way you can have a thread pool with blockable threads which you can configure to have the same size as you DB connection pool.
But even so you still cannot rely on @Transactional because of the way the tread pool reuses the threads. In a standard Servlet architecture you have one thread per HTTP request. When the response is sent back, the thread is stopped, which closes the transaction. In this case though, the Reactor scheduler does not close the threads and reuses them for other events. So even if you CAN block you still have the same problem as before.
You do have the Context option you've mentioned and I think this would work for Mono. I'm not sure if it would work for a Flux (I'm thinking that all events in a Flux share the same context, which is what you don't want).
Another option is to use a Tuple2 with T1 as the business object and T2 the transaction context. I cannot recommend this though since you mix business logic with technical stuff and it overcomplicates things.
My best bet would be to do the transaction/connection management yourself:

Get DB connection
Open TX
Do blocking IO stuff
Close TX
Close/release DB connection

all in one code block on a blocking thread.
This will be safer (no leaks) and easier to understand. Also because you basically do everything yourself, you can chose what kind of error handling works best for your scenario.
Comment: That's what I thought.  Thank you!
Comment: this answer is not valid anymore. @Transaction is now available with Spring Reactive. see https://spring.io/blog/2019/05/16/reactive-transactions-with-spring
Answer: @Transactional is now valid on Spring Reactive.
I won't go into details here. Everything's explained in this post : https://spring.io/blog/2019/05/16/reactive-transactions-with-spring.
Comment: That's great news.  Thank you!  I've worked with R2DBC a little and it's nice to see that `@Transactional` is now supported.  Great work Spring Team!
Comment: This article is of 15th June 2019. Was it working from that time? If yes, was it supporting DB other than MongoDB from that time?
Comment: Basically, if you are able to find R2DBC driver for database you are willing to use, than it should work just fine.


Three-Level Multilevel Meta-analysis: What exactly are the three levels? Multilevel vs. Multilevel SEM Approaches

Question: Completely edited:
If a meta-analysis include many studies that reported multiple effect sizes within each study, multi-level meta-analysis is one way to account for this dependency in effect sizes. However, there seems to be two approaches: one that utilize the traditional multilevel models and another that utilize multilevel structural equation modeling (uses metaSEM package; Cheung, 2014)
In regular multi-level modeling, the three levels are:
Level 1- effect size
Level 2- within-study variation
Level 3- between-study variation
However, it's unclear to me what the three levels are for the multilevel SEM approach. Is it the same? Some papers that used metaSEM and cited Cheung 2014 (e.g., Lebuda et al., 2016) describes the three levels as:
Level 1- study participants
Level 2- effect sizes (within-study variation)
Level 3- study (between-study variation)
My questions are:
1. Did Lebuda et al., 2016 have it right that those are indeed the three levels?
2. If so, how is it possible to have level 1 as study participants given that in meta-analysis you typically don't have data at the participant level?
3. What's the added advantage to using multilevel SEM vs. just multilevel approach?

Citations:
Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., ... & Weersing, V. R. (2017). What five decades of research tells us about the effects of youth psychological therapy: A multilevel meta-analysis and implications for science and practice. American Psychologist, 72(2), 79.
Cheung, M. W. L. (2014). Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychological Methods, 19(2), 211.
metaSEM example
Paper that used metaSEM to do multilevel SEM meta-analysis and describes the three levels:
Lebuda, I., Zabelina, D. L., & Karwowski, M. (2016). Mind full of ideas: A meta-analysis of the mindfulness–creativity link. Personality and Individual Differences, 93, 22-26.
Comment: Can you give us some more details of what is troubling you as from your description I am not sure whether Cheung and Weisz are actually describing the same situation. Perhaps you could give an example? Also providing references for the two papers you cite would be very helpful.
Comment: @mdewey Thank you so much for your response!/
Comment: @mdewey Sorry, I'm new to stackexchange, so I didn't know hitting enter would automatically submit the comment. Thanks for responding! Because of characters limit, I elaborated in the original post instead of down here. Please let me know if I'm still not making sense! Any help will be very much appreciated!
Answer:
Those are indeed the three levels. When Lebuda says'study participants', he/she means that the variance at the first level refers to the sampling variance, and this variance is directly related to the number of participants of a given study (that is why it is called 'study participants' level). The sampling variance is assumed to be known and estimated in advance.

I think this point is already answered with the previous. It's true that in meta-analysis we don't normally have the raw data. However, we do have the sample size, and with that sample size (and with some other data) we can calculate the sampling variance, which is the variance at first level.
Both approaches are practically the same, both are based on the same regression model. The difference is that Cheung(2014) proposes to use SEM techniques, and that makes his approach more flexible in the sense that constrains among parameters can be imposed. Cheung talks more about it in this article:

Cheung, M. W.-L. (2014). Modeling dependent effect sizes with threelevel
meta-analyses: A structural equation modeling approach.
Psychological Methods, 19, 211–229. https://doi.org/10.1037/
a0032968
Summarizing: unless you want to fit a model with constrains among some parameters, both methods work equally good.


currying of 'flexible types' in F#

Question: Small part of a code to highlight the problem:
open System.IO

let do_smth i (stm : #System.IO.Stream) =  // val do_smth : 'a -> #Stream -> unit
    (*....*)
    ()

type SomeOps = SomeOps with
    static member op (i : int) = do_smth i

let test_currying i = do_smth i            // val test_currying : 'a -> (Stream -> unit)
                                           // NB: Stream, not #Stream
let main() =
    use stm = new System.IO.MemoryStream()
    test_currying 42 stm  // OK, upcasted to Stream somehow
    SomeOps.op 42 stm     // compiler error!

Can someone explain, why is compiler's behavior so different in last two lines? And why we lost information (about flexible #Stream) in test_currying function?
Answer: This looks like a bug to me. The compiler handles differently functions constructed using let and functions written as static member.
In particular, I think that it does not insert flexible types for member arguments that are not explicitly declared in the member declaration (that is, arguments that are either the result of partial function application or are created using the fun construct).
The simplest example that demonstrates the behavior does not use flexible type and looks like this:
type A =
  static member op = fun (s:obj) -> ()
module B =
  let op = fun (s:obj) -> ()

A.op "A" // Error
B.op "A" // Ok

According to the specification (Section 14.4.2, posted by Brian in a deleted (?) answer), a flexible type that allows using supertypes should be inserted regardless of whether the invoked function is member or a let-bound value.
Comment: I don't think that what the compiler does at the moment is a correct behavior. I don't see a reason why the two cases should behave differently. That's why I said that it looks like a bug to me.

currying of 'flexible types' in F#


String com quebra de linha class PHPExcel

Question: Caros, estou com um pequeno problema ao gerar um arquivo excel utilizando a classe PHPExcel.
Faço uma consulta no banco e obtenho uma string com uma quantia x de chars, e insiro este valor em um template em excel. Porém quando a string ultrapassa 43 chars eu preciso fazer com que a planilha excel faça um merge de duas linhas ou mais para que o texto não apareça cortado; O processo de mesclar as células de acordo com a quantidade de chars está funcionando, porém quando o texto é inserido na célula, existe umas quebras de linhas e espaços a mais que não deveriam existir, o que deixa o texto bagunçado.
Gostaria de saber como posso inserir o texto nas celulas mescladas do mesmo modo como esta no banco, sem as quebras de linhas e espaços adicionais.
<?php
    //configurações do banco
    ini_set('max_execution_time', 300);
    include_once("../../function/conexao.php");
    include_once '../../Classes/PHPExcel/IOFactory.php';
    $objReader = PHPExcel_IOFactory::createReader('Excel2007');
    $objPHPExcel = $objReader->load("../../Templates/PedidoCompra.xlsx");
    // Create new PHPExcel object
    //$objPHPExcel = new PHPExcel();
    // Set document properties

    if(isset($_POST['PDC'])){
    $Dados = $_POST['PDC'];
    //atualização dos dados do pedido
    $data1 = explode("/",$Dados[10]);
    $Dados[10] = substr($data1[2],0,4)."/$data1[1]/$data1[0]";
    if(empty($Dados[16])){
        $Dados[16]="0.00";
    }
    if(empty($Dados[9])){
        $Dados[9]="0.00";
    }
    if(empty($Dados[16])){
        $Dados[16]="0.00";
    }
    if(empty($Dados[17])){
        $Dados[17]="0.00";
    }
    //if($_POST['btPDC']=="ATUALIZAR"){
    $sql_up = "update financeiro_pedido_compra set
    pgto = '$Dados[12]',
    contato='$Dados[3]',
    data='$Dados[10]',
    desconto='$Dados[16]',
    IPI='$Dados[9]',
    FRETE='$Dados[17]',
    status='GERADO',
    data_modif=now(),
    usuario='$_SESSION[Usuario]' where pedidopk = $Dados[1]";
    mysql_query($sql_up);
    //echo $sql_up;
    if ($error = mysql_errno()) die("");
    //}
    $sql = "select nome,endereco1,num_end1,bairro1,cep1,cidade1,uf1,cgc,insc_est,concat(ddd1,'-',num1) as tel,cond_pgfk from financeiro_cliente where cgc='".$Dados[0]."'";
    $DadosCliente = mysql_fetch_assoc(mysql_query($sql,$conexao));
    // SELECIONAR OS DADOS PARA PLANILHA
    $query = mysql_query($sql,$conexao);
    // INICIAMOS A CRIAÇÃO DA TABELA
    $TotIPI = ($Dados[9]/100)*$Dados[8];
    $TotIPI = number_format($TotIPI, 2, ',', ' ');
    $Total = $Dados[8] + $TotIPI;
    $Dados[11] = strtoupper($Dados[11]);
    $Dados[6] = strtoupper($Dados[6]);
    $objPHPExcel->setActiveSheetIndex(0)
                ->setCellValue("C3", "$DadosCliente[nome]")
                ->setCellValue("C4", "$DadosCliente[endereco1], N°: $DadosCliente[num_end1]")
                ->setCellValue("C5", "$DadosCliente[bairro1]")
                ->setCellValue("C6", "$DadosCliente[cep1]")
                ->setCellValue("I4", "$DadosCliente[cidade1]")
                ->setCellValue("I5", "$DadosCliente[uf1]")
                ->setCellValue("I6", "$DadosCliente[cgc]")
                ->setCellValue("I7", "$DadosCliente[insc_est]")
                ->setCellValue("I8", "$DadosCliente[tel]")
                ->setCellValue("C10", "$Dados[1]") //pcn
                ->setCellValue("F10", "$Dados[2]")//data emissao
                ->setCellValue("I10", "$Dados[12]")
                ->setCellValue("C7", "$Dados[3]");//ordem
    $select=mysql_query("select item,concat(substr(mpfk,3,2),codigo),descricao,qtd,un,vlrun,vlrtot from financeiro_pedido_compra_item where pedidofk = '$Dados[1]' order by item") ;
    $row =14;
    $Total=0;
    $data1 = explode("/",$Dados[10]);
    $Dados[10] = substr($data1[2],0,4)."/$data1[1]/$data1[0]";
    while($Itens = mysql_fetch_array($select)){
    $objPHPExcel->setActiveSheetIndex(0)
                ->setCellValue("A$row", "$Itens[0]")//codigo
                ->setCellValue("C$row", "$Itens[1]")//codigo
                ->setCellValue("E$row", "$Itens[2]]")//descricao
                ->setCellValue("J$row", "$Itens[3]")//qtd
                ->setCellValue("K$row", "$Itens[4]")//un
                ->setCellValue("L$row", "$Itens[5]")//Val Un
                ->setCellValue("N$row", "$Itens[6]")//Total
                ->setCellValue("O$row", "$Dados[9]")//ipi
                ->setCellValue("P$row", "$Itens[3]")//qtd
                ->setCellValue("Q$row", "$Dados[10]");//data entrega
    $objPHPExcel->getActiveSheet()->mergeCells("C$row:D$row");
    $objPHPExcel->getActiveSheet()->mergeCells("L$row:M$row");
    //Verificação de quantidade de chars e mescla de linhas
    $Char = strlen($Itens[2]);
    if($Char>43){
        $IntDiv=(int)($Char/43);
        $row2=$row+$IntDiv;
        $objPHPExcel->getActiveSheet()->mergeCells("E$row:I$row2");
        $row+=$IntDiv+1;
    }else{
        $objPHPExcel->getActiveSheet()->mergeCells("E$row:I$row");
        $row++;
    }
    //$objPHPExcel->getActiveSheet()->getStyle("E$row")->getAlignment()->setWrapText(false);
    $Total+=$Itens[6];
    }
    //}/*
    $TotIPI = $Total * ($Dados[9]/100);
    $TotalGeral = $Total+$TotIPI+$Dados[17];
    $TotalGeral=$TotalGeral-$Dados[16];
    $objPHPExcel->setActiveSheetIndex(0)
                ->setCellValue("L38", "$Total")//codigo
                ->setCellValue("P38", "$TotIPI")
                ->setCellValue("Q38", $TotalGeral)
                ->setCellValue("N38", "$Dados[17]");//codigo
    $objPHPExcel->getActiveSheet()->getRowDimension('3')->setRowHeight(12);
    $objPHPExcel->getActiveSheet()->getRowDimension('4')->setRowHeight(12);
    $objPHPExcel->getActiveSheet()->getRowDimension('5')->setRowHeight(12);
    $objPHPExcel->getActiveSheet()->getRowDimension('6')->setRowHeight(12);
    $objPHPExcel->getActiveSheet()->getRowDimension('7')->setRowHeight(12);
    $objPHPExcel->getActiveSheet()->getRowDimension('8')->setRowHeight(12);
    $arquivo = 'PedidoCompra'.date("Ymd")."_ $Dados[1]";
    // Criamos uma tabela HTML com o formato da planilha
    }
    $objPHPExcel->getActiveSheet()->setTitle('Custo');
    $objPHPExcel->setActiveSheetIndex(0);
    header('Content-Type: application/vnd.openxmlformats-officedocument.SpreadsheetML.Sheet');
            header("Content-Disposition: attachment;filename=\"$arquivo.xlsx\"");
            header('Cache-Control: max-age=0');
            // If you're serving to IE 9, then the following may be needed
            header('Cache-Control: max-age=1');
            // If you're serving to IE over SSL, then the following may be needed
            header ('Expires: Mon, 26 Jul 1997 05:00:00 GMT'); // Date in the past
            header ('Last-Modified: '.gmdate('D, d M Y H:i:s').' GMT'); // always modified
            header ('Cache-Control: cache, must-revalidate'); // HTTP/1.1
            header ('Pragma: public'); // HTTP/1.0
            $objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');
            $objWriter->save('php://output');
    exit;

?>
Comment: Qual é sua variável do texto ?
Comment: estou usando dentro do array $Itens[2]
Comment: Faça isso - `nl2br($Itens[2])`
Answer: PREG_REPLACE
http://php.net/manual/pt_BR/function.preg-replace.php
->setCellValue("E$row", preg_replace('/\s/',' ',"$Itens[2]]"); # Descrição
Comment: Diego, obrigado pela contribuição, porem este procedimento não funcionou do modo que espero, ele apenas inseriu quebra de linha no excel, e na verdade eu gostaria de remover as quebras de linhas contidas no texto da célula.
Comment: Editei meu Poste.
Comment: Perfeito!!! muito obrigado Diego.


How can bootstrapping help in meeting assumptions of linear regression

Question: I was reading through my stat book and it was written that bootstrapping can relax the distribution assumptions for linear regression generalizability. I do not quite understand what assumptions we might have for linear regression distribution (maybe have something about residuals).
I am attaching the screenshot(book text), it would be great if anyone can shed some light on this topic.
Comment: See sec 126.96.36.199, of the same book, in particular, p272. The most commonly used tests and confidence intervals and prediction intervals for regression all assume normality of errors.
Answer: One general use of bootstrapping (not the only one) is to estimate parameters and their p values, confidence intervals and so on when there is no analytic solution.
The usual methods of getting p values and CIs in OLS regression rely on analytic methods. These analytic methods assume that the errors are normally distributed.  But bootstrapping does not assume this.


using $_POST values in form alter to add validation

Question: it's not the first time that i get into trouble using $_post in a node form alter, this time i'd like some assistance.
basically, i need to pass some parameters to a form (and i should not pass the appended to the url, as query parameters). but $_POST values don't work as expected (they work until reaching validation phase).
So.. in a page i pass as hidden value
<form method="post" action="/node/add/">
    <input type="hidden" name="mydesideredParameter" value="itsValue">
</form>

In the form_alter function of this form, at the very beginning, i try to assign this value to $form_state (and it works, I watchdog it later and it's correct)
if (array_key_exists('mydesideredParameter', $_POST)) {
    $form_state['mydesiredConsequence'] = 'dance';
}

Later, inside form_alter, at the very end, i state:
if ($form_state['mydesiredConsequence'] === 'dance'){
    $form['#validate'][] = 'myDancingFunction';
};

No complains happen on form building, but when trying to submit the form, the validation funcion doesn't get called and php complains about index that doesn't exist no more
Notice: Undefined index mydesiredConsequence

Nothing similar happens when i use this step and the values are in the url(via drupal_query_parameters) , but trying to make this work in $_POST seems a bit awkward. (i obviusly can imagine that the second form-rebuilding ignores $_POST, as it was a former request; but that should be the intended purpose of assigning the value to $form_state).
Answer: You should use $form_state['storage']. This will cache your data and make it available throughout the forms lifecycle.
Comment: I'll try it, but i suppose the index ['storage'] has no special behaviours compared to $form_state['callMeAsYouLikeIt']; it's just a standard naming convention (at least in D7)
Comment: I believe you may be right in it just being a convention but there is not enough context to know what else could be going wrong here.


Desktop Notification in asp.net

Question: I'm using below javascript for desktop notification in asp.net. On button click this event is working fine with permission but getting error while implementing from code behind.
and this code added on my webpage...

<code> <script type="text/javascript">
        function notifyMe() {
            if (!("Notification" in window)) { alert("This browser does not support desktop notification"); }
            else if (Notification.permission === "granted") {
                var options = {
                    body: "This is the body of the notification",
                    icon: "stupidcodes.com.png",
                    dir: "ltr"
                };

                var notification = new Notification("Hi there", options);

                notification.onclick = function () {
                    window.open("http://www.stupidcodes.com/");
                };
            }
            else if (Notification.permission !== 'denied') {
                Notification.requestPermission(function (permission) {
                    if (!('permission' in Notification)) {
                        Notification.permission = permission;
                    }
                    if (permission === 'granted') {
                        var options = {
                            body: 'This is the body of the notification',
                            icon: 'stupidcodes.com.png',
                            dir: 'ltr'
                        };
                        var notification = new Notification('Hi there', options);
                        notification.onclick = function () {
                            window.open('http://www.stupidcodes.com/');
                        };
                    }
                });
            }
        }
</code>

Is there any method to add notification like this in asp.net
you can check this code working here
Comment: did you get this working ?
Comment: No still not @SachinTrivedi
Answer: Please check out my blog post. I have created a sample that will fire notification when document is loaded. You can leverage the code as per your requirements.
Comment: Let see will check for sure can you reply my current question @Sachin
Comment: Please specify what is the error that you are getting.
Comment: http://stackoverflow.com/questions/33255591/get-filter-the-date-rand-in-sql-server/33255764#33255764 check this

Desktop Notification in asp.net


How to integrate $\int_0^{2\pi} \frac12 \sin(t) (1- \cos(t)) \sqrt{\frac12 - \frac12 \cos(t)}\,dt$

Question: How to integrate
$$\int_0^{2\pi} \frac12 \sin(t) (1- \cos(t)) \sqrt{\frac12 - \frac12 \cos(t)}\,dt$$
I know the solution is $0$, but I don't know how one gets this.
Comment: P&P: Periodicity and parity.
Answer: \begin{align}
\frac{1}{2}\sin t\left(1-\cos t\right)\sqrt{\frac{1}{2}-\frac{1}{2}\cos t} &= \frac{1}{2}\sin t\left(1-\cos t\right)\sqrt{\frac{1}{2}}\sqrt{1-\cos t} \\
&= \frac{1}{2\sqrt{2}}\sin t\left(1-\cos t\right)^{3/2}\,dt
\end{align}
so let $u = 1-\cos t\to du = \sin t \,dt$ which makes the problem easier to solve.
Comment: So does this go down to 0 when integrated with the limits? I can't get it to work?
Comment: $u(0) = u(2\pi) = 0$ so the integral will be $0$ in any case. But you can compute the integral and find the result you desire. But please follow the comment by Daniel where you do not have to compute the integral directly.
Comment: I get 1/2sqrt2 x integral u^(3/2) with the limits of 1 and 0, but this doesn't give 0, where am I going wrong?
Comment: $\cos(2\pi) = 1 = \cos(0)$. I am not sure how you are getting $0,1$?
Answer: Note that
$$\int_a^{b} f(t)dt=\int_a^{b} f(a+b-t)dt$$
Thus
$$I=\int_0^{2\pi} \frac12 \sin(t) (1- \cos(t)) \sqrt{\frac12 - \frac12 \cos(t)}\,dt=\int_0^{2\pi} \frac12 \sin(2\pi-t) (1- \cos(2\pi-t)) \sqrt{\frac12 - \frac12 \cos(2\pi-t)}\,dt=-\int_0^{2\pi} \frac12 \sin(t) (1- \cos(t)) \sqrt{\frac12 - \frac12 \cos(t)}\,dt=-I$$
Hence
$$2I=0$$
or
$$I=0$$
Answer: You can set $t=2u$, so
$$
\sqrt{\frac{1-\cos t}{2}}=\sqrt{\frac{1-1+2\sin^2u}{2}}=|\sin u|
$$
Since $0\le t\le 2\pi$ is the same as $0\le u\le \pi$, you can ignore the absolute value and your integral becomes
$$
\int_0^\pi\sin2u(1-\cos2u)\sin u\,du
=
\int_0^\pi 4\sin^4u\cos u\,du=
\left[\frac{4}{5}\sin^5u\right]_0^\pi=0
$$
But there's a much slicker way: set $t=u+\pi$, so the integral becomes
$$
-\frac{1}{2}\int_{-\pi}^{\pi}
\sin u(1+\cos u)\sqrt{\frac{1+\cos u}{2}}\,du
$$
Set $f(u)=\sin u(1+\cos u)\sqrt{\dfrac{1+\cos u}{2}}$ and note that $f(-u)=-f(u)$.


Algoritmo para cálculo e distribuição de elementos na tela

Question: Estamos aqui na empresa querendo criar uma vitrine com o efeito de reorganização estilo o que é usado pelo próprio stackexchange https://stackexchange.com/sites
Estive pensando em algoritmos de cálculo de posição, mas não estou chegando algo conclusivo de como reorganizar itens e calcular os espaços vazios.
Alguma lógica para este efeito em específico?
Answer: Dê uma olhada nesse plugin http://isotope.metafizzy.co/, talvez resolva seu problema.
Ele tem diversas opções de layout, mas talvez a que voc6e tenha interesse seja a Masonry. Nela os blocos de tamanho diferente são encaixados, como se fosse um muro de tijolos (daí o nome). A documentação descreve assim:

Masonry é o layout padrão no Isotope. Os elementos são organizados de forma inteligente, dentro de um grid vertical. Para cada elemento, o script calcula a melhor posição dentro do grid.

O algoritmo que determina a posição dos elementos é esse aqui:
_masonryLayout : function( $elems ) {
    var instance = this,
        props = instance.masonry;

    $elems.each(function(){
        var $this  = $(this),
            //how many columns does this brick span
            colSpan = Math.ceil( $this.outerWidth(true) / props.columnWidth );
        colSpan = Math.min( colSpan, props.cols );

        if ( colSpan === 1 ) {
            // if brick spans only one column, just like singleMode
            instance._masonryPlaceBrick( $this, props.colYs );
        } else {
            // brick spans more than one column
            // how many different places could this brick fit horizontally
            var groupCount = props.cols + 1 - colSpan,
                groupY = [],
                groupColY,
                i;

            // for each group potential horizontal position
            for ( i=0; i < groupCount; i++ ) {
                // make an array of colY values for that one group
                groupColY = props.colYs.slice( i, i+colSpan );
                // and get the max value of the array
                groupY[i] = Math.max.apply( Math, groupColY );
            }
          instance._masonryPlaceBrick( $this, groupY );
        }
    });
}

Pelo que eu pude entender o que acontence é:

O grid é iniciado com uma lista das posições iniciais de cada coluna. Ex: [0, 0, 0, 0]
Pra cada elemento, o algoritmo determina em quais colunas ele caberia, baseado na largura. Então, se são 5 colunas, um elemento com a largura de 3 colunas, ele pode entrar nas colunas 1 ou 2.
Em cada uma das colunas possíveis de encaixar o elemento, é verificado qual o lugar mais alto em que é possível inserí-lo.
O elemento é inserido nos espaços mais altos possíveis, e a primeira posição disponível das colunas que ele ocupa é atualizada.

É um processo esquisito de se explicar, mas bem simples de visualizaar, se você ver os exemplos no site do Isotope. Ou, melhor ainda, se montar um pra você e futucar nele.
Comment: Olá Leandro, muito top este plugin. Vai servir de referência também para futuros trabalhos, mas gostei mais do masonry por ser gratuito e bem mais simples
Answer: Recomendo o uso do Masonry pra isso, é bem simples: http://masonry.desandro.com/
Ele não depende do jQuery, mas pode ser usado integrado com o mesmo.
Código fonte:
/*!
 * Masonry v3.1.4
 * Cascading grid layout library
 * http://masonry.desandro.com
 * MIT License
 * by David DeSandro
 */

( function( window ) {

'use strict';

// -------------------------- helpers -------------------------- //

var indexOf = Array.prototype.indexOf ?
  function( items, value ) {
    return items.indexOf( value );
  } :
  function ( items, value ) {
    for ( var i=0, len = items.length; i < len; i++ ) {
      var item = items[i];
      if ( item === value ) {
        return i;
      }
    }
    return -1;
  };

// -------------------------- masonryDefinition -------------------------- //

// used for AMD definition and requires
function masonryDefinition( Outlayer, getSize ) {
  // create an Outlayer layout class
  var Masonry = Outlayer.create('masonry');

  Masonry.prototype._resetLayout = function() {
    this.getSize();
    this._getMeasurement( 'columnWidth', 'outerWidth' );
    this._getMeasurement( 'gutter', 'outerWidth' );
    this.measureColumns();

    // reset column Y
    var i = this.cols;
    this.colYs = [];
    while (i--) {
      this.colYs.push( 0 );
    }

    this.maxY = 0;
  };

  Masonry.prototype.measureColumns = function() {
    this.getContainerWidth();
    // if columnWidth is 0, default to outerWidth of first item
    if ( !this.columnWidth ) {
      var firstItem = this.items[0];
      var firstItemElem = firstItem && firstItem.element;
      // columnWidth fall back to item of first element
      this.columnWidth = firstItemElem && getSize( firstItemElem ).outerWidth ||
        // if first elem has no width, default to size of container
        this.containerWidth;
    }

    this.columnWidth += this.gutter;

    this.cols = Math.floor( ( this.containerWidth + this.gutter ) / this.columnWidth );
    this.cols = Math.max( this.cols, 1 );
  };

  Masonry.prototype.getContainerWidth = function() {
    // container is parent if fit width
    var container = this.options.isFitWidth ? this.element.parentNode : this.element;
    // check that this.size and size are there
    // IE8 triggers resize on body size change, so they might not be
    var size = getSize( container );
    this.containerWidth = size && size.innerWidth;
  };

  Masonry.prototype._getItemLayoutPosition = function( item ) {
    item.getSize();
    // how many columns does this brick span
    var remainder = item.size.outerWidth % this.columnWidth;
    var mathMethod = remainder && remainder < 1 ? 'round' : 'ceil';
    // round if off by 1 pixel, otherwise use ceil
    var colSpan = Math[ mathMethod ]( item.size.outerWidth / this.columnWidth );
    colSpan = Math.min( colSpan, this.cols );

    var colGroup = this._getColGroup( colSpan );
    // get the minimum Y value from the columns
    var minimumY = Math.min.apply( Math, colGroup );
    var shortColIndex = indexOf( colGroup, minimumY );

    // position the brick
    var position = {
      x: this.columnWidth * shortColIndex,
      y: minimumY
    };

    // apply setHeight to necessary columns
    var setHeight = minimumY + item.size.outerHeight;
    var setSpan = this.cols + 1 - colGroup.length;
    for ( var i = 0; i < setSpan; i++ ) {
      this.colYs[ shortColIndex + i ] = setHeight;
    }

    return position;
  };

  /**
   * @param {Number} colSpan - number of columns the element spans
   * @returns {Array} colGroup
   */
  Masonry.prototype._getColGroup = function( colSpan ) {
    if ( colSpan < 2 ) {
      // if brick spans only one column, use all the column Ys
      return this.colYs;
    }

    var colGroup = [];
    // how many different places could this brick fit horizontally
    var groupCount = this.cols + 1 - colSpan;
    // for each group potential horizontal position
    for ( var i = 0; i < groupCount; i++ ) {
      // make an array of colY values for that one group
      var groupColYs = this.colYs.slice( i, i + colSpan );
      // and get the max value of the array
      colGroup[i] = Math.max.apply( Math, groupColYs );
    }
    return colGroup;
  };

  Masonry.prototype._manageStamp = function( stamp ) {
    var stampSize = getSize( stamp );
    var offset = this._getElementOffset( stamp );
    // get the columns that this stamp affects
    var firstX = this.options.isOriginLeft ? offset.left : offset.right;
    var lastX = firstX + stampSize.outerWidth;
    var firstCol = Math.floor( firstX / this.columnWidth );
    firstCol = Math.max( 0, firstCol );
    var lastCol = Math.floor( lastX / this.columnWidth );
    // lastCol should not go over if multiple of columnWidth #425
    lastCol -= lastX % this.columnWidth ? 0 : 1;
    lastCol = Math.min( this.cols - 1, lastCol );
    // set colYs to bottom of the stamp
    var stampMaxY = ( this.options.isOriginTop ? offset.top : offset.bottom ) +
      stampSize.outerHeight;
    for ( var i = firstCol; i <= lastCol; i++ ) {
      this.colYs[i] = Math.max( stampMaxY, this.colYs[i] );
    }
  };

  Masonry.prototype._getContainerSize = function() {
    this.maxY = Math.max.apply( Math, this.colYs );
    var size = {
      height: this.maxY
    };

    if ( this.options.isFitWidth ) {
      size.width = this._getContainerFitWidth();
    }

    return size;
  };

  Masonry.prototype._getContainerFitWidth = function() {
    var unusedCols = 0;
    // count unused columns
    var i = this.cols;
    while ( --i ) {
      if ( this.colYs[i] !== 0 ) {
        break;
      }
      unusedCols++;
    }
    // fit container to columns that have been used
    return ( this.cols - unusedCols ) * this.columnWidth - this.gutter;
  };

  // debounced, layout on resize
  // HEADS UP this overwrites Outlayer.resize
  // Any changes in Outlayer.resize need to be manually added here
  Masonry.prototype.resize = function() {
    // don't trigger if size did not change
    var previousWidth = this.containerWidth;
    this.getContainerWidth();
    if ( previousWidth === this.containerWidth ) {
      return;
    }

    this.layout();
  };

  return Masonry;
}

// -------------------------- transport -------------------------- //

if ( typeof define === 'function' && define.amd ) {
  // AMD
  define( [
      'outlayer/outlayer',
      'get-size/get-size'
    ],
    masonryDefinition );
} else {
  // browser global
  window.Masonry = masonryDefinition(
    window.Outlayer,
    window.getSize
  );
}

})( window );

Comment: no caso, ser mais simples é uma tremenda **vantagem**, já que ele está interessado no *algoritmo* (apenas se ele quiser + recursos, aí sim, que estude o código do Isotope... mas com certeza será melhor dominar o algoritmo simples antes de partir pro complexo...)
Comment: Mesmo que este link seja uma boa sugestão, esta resposta não terá validade se um dia o link deixar de funcionar. Por isso, e porque é importante para a comunidade ter conteúdo aqui mesmo, seria melhor você responder com respostas mais completas. Você pode elaborar mais sua resposta? Um resumo do conteúdo do link já ajudaria bastante! Saiba mais sobre o assunto neste item do nosso FAQ da Comunidade: [Respostas que apenas contêm links são boas?](http://meta.pt.stackoverflow.com/questions/705/respostas-que-apenas-contem-links-sao-boas)
Comment: Aproveitando: acho o masonry melhor para se estudar que o isotope (que é baseado no masonry e veio depois).
Comment: Olá Gabriel, muito obrigado pela sugestão \0/
Salvou meu trampo aqui e meu tempo também. Um ótimo plugin =)
Comment: O código do Masonry é consideravelmente mais simples do que o do Isotope: https://github.com/desandro/masonry/blob/master/masonry.js
Comment: @J.Bruni sim, só quis dar uma segunda opção pra ele, eu acho o Isotope legal, mas as vezes não precisa de todos recursos dele.
Comment: @J.Bruni vamos ver o que o Douglas vai achar, de qualquer jeito eu dei +1 pro Leandro pois a resposta dele foi certeira sem dúvidas.


Proof that $A\in M_{n\times n}\mathbb{R}$ is a permutation matrix if and only if it is the product of elementary matrices of type $P_{ij}$.

Question: We define a permutation matrix $A\in M_{n\times n}\mathbb{R}$ as a matrix that has all its coefficients 0 except from a 1 in each row and each column. Then, proof that $A\in M_{n\times n}\mathbb{R}$ is a permutation matrix if and only if it is the product of elementary matrices of type $P_{ij}$.
Comment: This amounts to say that a permutation is the product of transpositions...
Comment: Maybe it can help you that every invertible matrix is the product of elementary matrices (for example notice that if $A$ is invertible, then it is $A^{-1}$. Then apply Gaussian elimination and you will obtain the identity matrix. That means that $QA^{-1}=I_n$ for a certain matrix $Q=E_1\dots E_k$, hence $A=Q=E_1\dots E_k$. I have used the fact that row reduction is the equivalent of multiplying to the left for elementary matrices). Also a permutation matrix is obtained by permuting the columns of the identity matrix, which is a Gauss operation.
Answer: I would answer this way. First, show that the set $S$ of permutation matrices is a subgroup of $GL_n$ (it is not hard to prove it). Then, for $M \in S$, let $\sigma$ be the unique permutation on $\{1,\ldots,n\}$ such that $M_{i,j} = \delta_{i,\sigma(j)}$ where $\delta$ stands for the Kronecker symbol. You can show $\sigma$ exists by constructing it : $\sigma(j)$ is the rank of the only line for which there is a $1$ at the $j$-th place. Now, show that $M \in S \mapsto \sigma \in\mathfrak{S}_n$ is a group homomorphism. It is surjective (as you can construct the desired matrix for a chosen $\sigma$ by the formula above) and for cardinality reasons, it will be an isomorphism. Thus, the results is the same as saying that $\mathfrak{S}_n$ is generated by transpositions, which is true.


How to calculate an average across columns based on values in adjacent columns in Pandas

Question: I have an example dataset attached. My actual dataset is much larger. The "yr" columns are split between "cd" and "qty" and go from yr 10 to yr 1. Not every row contains a full set of data, some contain zeros.
Example data:

Sales
yr10cd
yr10qty
yr9cd
yr9qty
yr8cd
yr8qty

42
A
45
A
47
A
49

56
T
58
A
52
0
0

78
A
75
0
0
0
0

I want to be able to take an average of the qty columns (yr10qty, yr9qty, yr8qty, etc.) but only if the associated indicator (yr10cd, yr9cd, yr8cd, etc. ) in the adjacent column is an "A" value. if the associated indicator is 0 or any other value like "T", I do not want to include it in my average calculation.
I have tried using a function that utilizes if statements to append values to a list and then average if non zero. Then, using df.apply to apply that function to each row in the df. Unfortunately, I keep getting 0 for all my averages, which is not expected.
Here is the expected output. It is an added column to my df that contains the average for that row.
Expected output:

Sales
yr10cd
yr10qty
yr9cd
yr9qty
yr8cd
yr8qty
average

42
A
45
A
47
A
49
47

56
T
58
A
52
0
0
52

78
A
75
0
0
0
0
75

I have tried searching stack overflow for sometime now and none of the solutions I have come across have worked for my specific scenario.
Answer: Another possible solution:
d = (df.filter(like='cd', axis=1).eq('A') * df.filter(like='qty', axis=1).values)

df['average'] = d.where(d.ne(0)).mean(axis=1)

Output:
   Sales yr10cd  yr10qty yr9cd  yr9qty yr8cd  yr8qty  average
0     42      A       45     A      47     A      49     47.0
1     56      T       58     A      52     0       0     52.0
2     78      A       75     0       0     0       0     75.0

Answer: Take advantage of the alternate pairs cd/qty, mask the non A, then compute the mean :
tmp = df.set_index("Sales") # put aside the S

cds = tmp.iloc[:, ::2]; qty = tmp.iloc[:, 1::2]

df["average"] = (qty.mask(cds.ne("A").set_axis(qty.columns, axis=1))
                     .mean(axis=1).reset_index(drop=True))

Ouptut :
print(df)

   Sales yr10cd  yr10qty yr9cd  yr9qty yr8cd  yr8qty  average
0     42      A       45     A      47     A      49     47.0
1     56      T       58     A      52     0       0     52.0
2     78      A       75     0       0     0       0     75.0

Answer: In some situations where the structure of the columns can be unknown it can be useful to expand out into a MultiIndex to perform operations which allows us to take advantage of pandas indexing to ensure data integrity during our computation.
Assuming a structure of yr[num]cd paired with yr[num]qty we can isolate those columns and create a multi-index such that the [num] values are in their own level.
v = df.filter(regex='^yr\d+(cd|qty)$')
v.columns = (
    v.columns
    .str.replace(r'yr(\d+)(cd|qty)', r'\2_\1', regex=True)
    .str.split('_', expand=True)
)

Here I've isolated the interesting columns into the variable v and restructured the columns so that the cd and qty values are in level 0 and the numbers are in level 1 through the use of replace and split.
V looks like:
  cd qty cd qty cd qty
  10  10  9   9  8   8
0  A  45  A  47  A  49
1  T  58  A  52  0   0
2  A  75  0   0  0   0

Note there are many ways to restructure the columns into a MultiIndex. Here's another example for reference:
v.columns = (
    v.columns
    .str.split(r'(\d+)', regex=True, expand=True)
    .droplevel(0)
    .swaplevel(0, 1)
)

Depending on the column name format different approaches may be better at restructuring.

The primary benefit of a MultiIndex with this level order is that we can very easily get access to all cd columns and qty columns by accessing them v['cd'] and v['qty].
v['qty'] for reference:
   10   9   8
0  45  47  49
1  58  52   0
2  75   0   0

The great thing about this is that regardless of column order, we can reliably align computations between 10, 9 and 8.
This allows us to filter out where is equal to 'A' v['qty'].where(v['cd'].eq('A')):
     10     9     8
0  45.0  47.0  49.0
1   NaN  52.0   NaN
2  75.0   NaN   NaN

Then take the mean across rows v['qty'].where(v['cd'].eq('A')).mean(axis='columns'):
0    47.0
1    52.0
2    75.0
dtype: float64

This has the same index as df so we can very simply assign the values back
df['Average'] = v['qty'].where(v['cd'].eq('A')).mean(axis='columns')

df with the new column:
   Sales yr8cd yr9cd yr10cd  yr10qty  yr9qty  yr8qty  Average
0     42     A     A      A       45      47      49     47.0
1     56     0     A      T       58      52       0     52.0
2     78     0     0      A       75       0       0     75.0

Again something really great about this approach is that our initial data column order does not matter.
Imagine a situation where our cd columns are grouped with numbers in ascending order and our qty columns are grouped with numbers in descending order.

Sales
yr8cd
yr9cd
yr10cd
yr10qty
yr9qty
yr8qty

42
A
A
A
45
47
49

56
0
A
T
58
52
0

78
0
0
A
75
0
0

Or, perhaps more realistically, imagine a scenario where someone accidentally dragged one of the columns out of order. The approach outlined here will still result in the correct average values because the calculation and value filtering is aligning based on the numerical values and not their relative position within the DataFrame.
0    47.0
1    52.0
2    75.0
dtype: float64

So, while this is likely not the fastest solution or the most memory efficient, it is fairly performant while not sacrificing any alignment integrity checks that indexing can provide.

Complete working example with version number
import pandas as pd  # v2.1.2

df = pd.DataFrame({
    'Sales': [42, 56, 78],
    'yr10cd': ['A', 'T', 'A'],
    'yr10qty': [45, 58, 75],
    'yr9cd': ['A', 'A', '0'],
    'yr9qty': [47, 52, 0],
    'yr8cd': ['A', '0', '0'],
    'yr8qty': [49, 0, 0]
})

v = df.filter(regex='^yr\d+(cd|qty)$')
v.columns = (
    v.columns
    .str.replace(r'yr(\d+)(cd|qty)', r'\2_\1', regex=True)
    .str.split('_', expand=True)
)

df['Average'] = v['qty'].where(v['cd'].eq('A')).mean(axis='columns')

print(df)

Comment: Hi @Henry, thank you for your solution. I found it quite helpful and comprehensive. I have a similar issue I am facing, but I do not know if I should post another question. Let's say I want to see if two or more of the values in "v" for each row are less than 10% of the average I just calculated, would I be able to perform operations using "v" to answer that question?
Comment: Yes @Taylor3417 you could use `v` for that as well if you wanted to. Just compare `v['qty']` to `df['Average']` (maybe something like `v['qty'].lt(df['Average'] * .1, axis=0)`) and count the number of True values in each column. You might consider asking another question which would allow you to further explain what exactly you mean and give answerers a better chance to write out a solution in an answer field rather than comments.
Answer: Row-Method with for loop way:
df = pd.DataFrame(data)

def calculate_average(row):
    qty_values = []
    for i in range(10, 1, -1):
        cd_col = f'yr{i}cd'
        qty_col = f'yr{i}qty'
        if cd_col in row and qty_col in row and row[cd_col] == 'A':
            qty_values.append(row[qty_col])
    if qty_values:
        return sum(qty_values) / len(qty_values)
    else:
        return 0

df['average'] = df.apply(calculate_average, axis=1)
print(df)

Output:
   Sales yr10cd  yr10qty yr9cd  yr9qty yr8cd  yr8qty  average
0     42      A       45     A      47     A      49     47.0
1     56      T       58     A      52     0       0     52.0
2     78      A       75     0       0     0       0     75.0

Vectorized way:
df = pd.DataFrame(data)
cd_columns = df.columns[df.columns.str.contains('cd')].to_list()
qty_columns = df.columns[df.columns.str.contains('qty')].to_list()

df[qty_columns] = df[qty_columns].apply(pd.to_numeric, errors='coerce') # replace non-numeric values with NaN
avg_qty = np.where(df[cd_columns] == 'A', df[qty_columns], np.nan) # average
df['average'] = np.nanmean(avg_qty, axis=1)

print(df)

Output:
   Sales yr10cd  yr10qty yr9cd  yr9qty yr8cd  yr8qty  average
0     42      A       45     A      47     A      49     47.0
1     56      T       58     A      52     0       0     52.0
2     78      A       75     0       0     0       0     75.0

Answer: Here's a very scrappy answer that should work:
temp_df= df.copy()
columns = list(df.columns)
for col in columns:
    if col.endswith('cd'):
        adj_col = col.replace('cd','qty')
        temp_df[adj_col] = np.where(temp_df[col] != 'A', None, temp_df[adj_col])

df['average'] = temp_df[[c for c in columns if c.endswith('qty')]].mean(axis=1,skipna = True)

Explanation: Create a temporary df. For each 'cd' column, if the value is not 'A', replace the value of the adjacent 'qty' column with None.
Then when we take the row-wise average of all 'qty' columns, we can skip 'qty' columns with None values.
Finally, when we take the row-wise averages, we simultaneously create a new column in the original df and assign the result of those averages.