Do (systems of linear equations with scalars and unknowns from different algebraic structures) occur widely?

Question: Generally in linear algebra one studies systems of linear equations where both coefficients and unknowns belong to the same field. I would not be the first person to notice that a system like
$$\begin{array}{rrrcl}\mathbf{x}_1&+\mathbf{x}_2&+2\mathbf{x}_3&=&\mathbf{b}_1\\\mathbf{x}_1&+\mathbf{x}_2&+\mathbf{x}_3&=&\mathbf{b}_2\end{array}$$
(scalars in $\mathbb{R}$ and unknowns & 'constants' $\mathbf{x}_i,\mathbf{b}_j \in V$ where $V$ is an arbitrary vector space over $\mathbb{R}$) can be solved by row-reduction, assignment of free parameters and back-substitution to get $$\left[\begin{array}{c}\mathbf{x}_1\\\mathbf{x}_2\\\mathbf{x}_3\end{array}\right]
= \left[\begin{array}{c}2\mathbf{b}_1-\mathbf{b}_1\\\mathbf{0}_V\\\mathbf{b}_1-\mathbf{b}_2\end{array}\right] + \left[\begin{array}{c}-\mathbf{t}\\\mathbf{t}\\\mathbf{0}_V\end{array}\right]
= \left[\begin{array}{c}2\mathbf{b}_1-\mathbf{b}_1\\\mathbf{0}_V\\\mathbf{b}_1-\mathbf{b}_2\end{array}\right] +
\left[\begin{array}{c}-1\\1\\0_{\mathbb{R}}\end{array}\right]
\left[\begin{array}{c}\mathbf{t}\end{array}\right] \text{ for } \mathbf{t} \in V$$
where $\left[\mathbf{t}\right]$ is a $1\times1$ matrix. Parameters do not factor out the same way as when $V=\mathbb{R}$.
Wikipedia mentions linear equations over a ring and over polynomials (Gröbner basis theory), but in both cases coefficients and unknowns are from the same algebraic structure.
My question is: are there broad areas in mathematics where such systems of equations (scalars from a field, unknowns from a general vector space) occur? Or, more generally, are there broad classes of linear equations with scalars and unknowns from different algebraic structures?
Answer: Yes. Here is a sketch of an example.
Suppose that you want to understand the behavior of the solutions a system of equations as the "parameters" of the equations vary.
To be concrete, imagine we have the equation:
$a_0 x_0 + a_1 x_1 = 0$
Here we can think of the $a_i$ as $\mathbb{R}$-valued functions on some space $X$, which you can take as a first approximation to be the line $\mathbb{R}$. We will take our functions $a_i$ to be polynomials in $\mathbb{R}[t]$.
So at each point $p$ we have a fixed equation $a_0(p) x_0 + a_1(p) x_1 = 0$, and these equations vary nicely if we move around on $X$ nicely (nice can be: continuous, smooth, algebraic, etc.).
Above every point on $X$ we can put a copy of $\mathbb{R}^2$, with coordinates $x_i$, and so above each point on $X$ there is a corresponding set of solutions to the equations. These solutions "move" in a algebraic fashion as we move around on $X$.
Maybe at this point you should try to draw a picture of this. Over each point in the line there should be a line of solutions (or a plane possibly if $a_0 = a_1 = 0$), and your polynomial functions $a_i$ control how it moves.
I said above that taking $X$ to be the line $\mathbb{R}$ is only a first approximation. In fact, one gets a more flexible theory if you include other points in this line - this is a mysterious point to explain without going into technical details, so please just accept it for now.
For example, we will associate a point to the entire line $P$, or a point to the pair $(i,-i)$. (These correspond in some fashion to the polynomials $(0)$ and $(t^2 + 1)$, respectively.)
Once this is done, it no longer makes sense to think of the $a_i$ as being real valued functions everywhere - over the point $(i,-i)$ they will take complex values, and over the point $P$ they will take values in the field of rational functions $\mathbb{R}(t)$. But note that since we chose polynomials in $\mathbb{R}[t]$ for the $a_i$, they still know how to be functions on $\mathbb{C}$, or $\mathbb{R}(t)$: $t$ evaluated at $(i,-i)$, which amounts to the map $R[t] \to R[t]/(t^2 + 1) \cong \mathbb{C}$ is $i$, $t$ evaluated at $P$ (the map $R[t] \to R(t)$) is $t$.
And over these points it makes sense for the same reasons to put the vector spaces $\mathbb{C}^2$ or $\mathbb{R}(t)^2$ over the corresponding points (after all, the equations now have complex coefficients, so it makes less sense to look for solutions $\mathbb{R}^2$). In each of these vector spaces, the functions $a_i$ define an equation $a_ix_i =0$, which defines a subspace.
Maybe you can try to draw a (rough) picture now? (I certainly can't, but it helps to try.) One should still get the sense that the set of solutions moves nicely, even though the space in which it lives may change from point to point.
So the moral of this story is that there are situations where you can study equations whose fields of coefficients change in some controlled way. As you saw, the situation is described by something vaguely geometric.
The study of these objects is called scheme theory, which is a topic in algebraic geometry.


Inserting to sqlite dynamically with Python 3

Question: I want to write to multiple tables with sqlite, but I don't want to manually specify the query ahead of time (there are dozens of possible permutations).
So for example:
def insert_sqlite(tablename, data_list)
    global dbc
    dbc.execute("insert into " + tablename + " values (?)", data_list)

tables_and_data = {
                   'numbers_table': [1,2,3,4,5],
                   'text_table': ["pies","cakes"]
                   }

for key in tables_and_data:
    insert_sqlite(key, tables_and_data[key])

I want two things to happen:
a) for the tablename to be set dynamically - I've not found a single example where this is done.
b) The data_list values to be correctly used - note that the length of the list varies (as per the example).
But the above doesn't work - How do I dynamically create a sqlite3.execute statement?
Thanks
Answer: a) Your code above seems to be setting the table name correctly so no problem there
b) You need a ?(placeholder) per column you wish to insert a value for.
when i recreate your code as is and run it i get the error message:
"sqlite3.OperationalError: table numbers_table has 5 columns but 1 values were supplied".
A solution would be to edit your function to dynamically create the correct number of placeholders:
def insert_sqlite(tablename, data_list):
    global dbc
    dbc.execute("insert into " + tablename + " values (" + ('?,' * len(data_list))[:-1] + ")", data_list)

after doing this and then re-executing the code with an added select statements (just to test it out):
dbc.execute("""
select * from numbers_table
""")
print(dbc.fetchall());

dbc.execute("""
select * from text_table
""")
print(dbc.fetchall());

I get the result:
[(1, 2, 3, 4, 5)]
[(u'pies', u'cakes')]

Comment: Decent analysis, and you tested your suggestion too. Good answer!
Comment: Thanks. I had tried that method of tablename placing but it wasn't working. I've since discovered I had a bad table name. Anyway, I just dropped it in and it worked perfectly. Many thanks. And welcome to SO! :-)


Prerequisites for Stanley's Enumerative Combinatorics

Question: I want to learn combinatorics, and have read people's messages around here recommending Enumerative Combinatorics by Stanley; most of these suggestions also state that it's a dense book. If by dense they mean that it is really detailed, I have no problem with it, but it also makes me wonder if I need any previous knowledge before reading it.
Thanks in advance.
Comment: Plenty of "mathematical maturity".
Comment: By the way, I highly recommend Bruce Sagan's new book called Combinatorics: The Art of Counting. A draft is freely available from the author's website [here.](https://users.math.msu.edu/users/bsagan/Books/Aoc/final.pdf) The book is a gentle introduction (at the graduate level) to the field of enumerative combinatorics, and may be an easier read than Stanley's books. You may find it interesting to know that Sagan's PhD advisor was Stanley.
Answer: I think "dense" might be meant in the sense of packing in a lot of mathematics per page. In one sense, this is "detailed" but in another it isn't, as readers may be expected to fill in or work out certain details themselves. Being comfortable with doing this is part of the mathematical maturity that Angina Seng mentions in the comment above.
If you're new to combinatorics, it might be wiser to first read one of the many excellent undergraduate combinatorics books out there or possibly a discrete mathematics book. See, for instance, the extensive list here.
As for prerequisites, starting from scratch would be tough going. A solid background in linear and abstract algebra, along with some complex analysis will help.


How can I change UIButton title color?

Question: I create a button programmatically..........
button = [UIButton buttonWithType:UIButtonTypeRoundedRect];
[button addTarget:self action:@selector(aMethod:)
forControlEvents:UIControlEventTouchDown];
[button setTitle:@"Show View" forState:UIControlStateNormal];
button.frame = CGRectMake(80.0, 210.0, 160.0, 40.0);
[view addSubview:button];

how can I change title color?
Answer: You can use -[UIButton setTitleColor:forState:] to do this.
Example:
Objective-C
[buttonName setTitleColor:[UIColor blackColor] forState:UIControlStateNormal];

Swift 2
buttonName.setTitleColor(UIColor.blackColor(), forState: .Normal)

Swift 3
buttonName.setTitleColor(UIColor.white, for: .normal)

Thanks to richardchildan
Comment: For instance [buttonName setTitleColor:[UIColor blackColor] forState:UIControlStateNormal];
Comment: I can't believe [UIButton setTitleColor:forState:] works but btn.titleColor = [UIColor x] does not..
Comment: Superb answer. Its ridiculous how you have to set the state just to change the color though... :(
Comment: @Lengus I don't see how you can access `btn.titleColor`, there is just `btn setTitleColor` available
Comment: How about with RGB ?
Answer: You created the UIButton is added the ViewController, The following instance method to change UIFont, tintColor and TextColor of the UIButton
Objective-C
 buttonName.titleLabel.font = [UIFont fontWithName:@"LuzSans-Book" size:15];
 buttonName.tintColor = [UIColor purpleColor];
 [buttonName setTitleColor:[UIColor purpleColor] forState:UIControlStateNormal];

Swift
buttonName.titleLabel.font = UIFont(name: "LuzSans-Book", size: 15)
buttonName.tintColor = UIColor.purpleColor()
buttonName.setTitleColor(UIColor.purpleColor(), forState: .Normal)

Swift3
buttonName.titleLabel?.font = UIFont(name: "LuzSans-Book", size: 15)
buttonName.tintColor = UIColor.purple
buttonName.setTitleColor(UIColor.purple, for: .normal)

Comment: Please don't simply post the code. Give some explanation or information or usage about your code. For example, see [this answer](http://stackoverflow.com/a/16893057/756941).
Answer: Solution in Swift 3:
button.setTitleColor(UIColor.red, for: .normal)

This will set the title color of button.
Answer: With Swift 5, UIButton has a setTitleColor(_:for:) method. setTitleColor(_:for:) has the following declaration:

Sets the color of the title to use for the specified state.

func setTitleColor(_ color: UIColor?, for state: UIControlState)

The following Playground sample code show how to create a UIbutton in a UIViewController and change it's title color using setTitleColor(_:for:):
import UIKit
import PlaygroundSupport

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = UIColor.white

        // Create button
        let button = UIButton(type: UIButton.ButtonType.system)

        // Set button's attributes
        button.setTitle("Print 0", for: UIControl.State.normal)
        button.setTitleColor(UIColor.orange, for: UIControl.State.normal)

        // Set button's frame
        button.frame.origin = CGPoint(x: 100, y: 100)
        button.sizeToFit()

        // Add action to button
        button.addTarget(self, action: #selector(printZero(_:)), for: UIControl.Event.touchUpInside)

        // Add button to subView
        view.addSubview(button)
    }

    @objc func printZero(_ sender: UIButton) {
        print("0")
    }

}

let controller = ViewController()
PlaygroundPage.current.liveView = controller

Answer: If you are using Swift, this will do the same:
buttonName.setTitleColor(UIColor.blackColor(), forState: .Normal)
Hope that helps!

How can I change UIButton title color?


symfony2 mais de um security voter para um bundle

Question: estou tendo problemas em criar mais de um voter para cada classe
a ideia é ter um voter para cada classe para poder realizar a segurança de acesso a mesma, o primeiro voter funcionou mas o segundo sempre da acesso negado
notei também que no services.yml só é possivel adicionar um só, seria possivel criar mais de um voter para um bundle?
exemplo:
security.access.post_voter:
        class:      AppBundle\Security\Authorization\Voter\StateVoter
        public:     false
        tags:
            - { name: security.voter }

security.access.post_voter:
        class:      AppBundle\Security\Authorization\Voter\CityVoter
        public:     false
        tags:
            - { name: security.voter }

ele não deixa repetir o security.acess.post_voter
Answer: o problema era porque não precisa ser chamado de post_voter, me enganei no tutorial
security.access.state_voter:
        class:      AppBundle\Security\Authorization\Voter\StateVoter
        public:     false
        tags:
            - { name: security.voter }

security.access.city_voter:
        class:      AppBundle\Security\Authorization\Voter\CityVoter
        public:     false
        tags:
            - { name: security.voter }

tudo certo agora


Is there any relationship between a step and an alphabetical note?

Question: I'm learning music theory for the first time.  I recently learned that a fret is equivalent to a half step.  For whatever reason, when I think of a half step I imagine going from one A => A# per-se.  Looking at the the guitar neck, sometimes there are 2 half steps between notes (A => B), other times there is only 1 (B => C).
Is there any relationship between the alphabet name of a note and the distance between the notes (interval)?  And isn't A# the same as B?  So what does that make 6th fret low E?
Answer: The musical alphabet consists of 7 notes: A-B-C-D-E-F-G. All the notes have 2 half steps between them, except for B->C and E->F - between those, there is only 1 half step.

And isn't A# the same as B? So what does that make 6th fret low E?

No, A# is A# - 1 half step above A.(It can also be called Bb - 1 half step below B). You need to go 2 half-steps to get to B, as explained. The 6th fret on your E string is A# or Bb. The 7th fret - 2 half steps above A - is B. It is the same note as the second fret on your A string - 2 half steps above A, the open string.
The relationship between the number of steps and the name of a note can get a bit complicated, but those are the basics.
Comment: All music thing - that is the way the system works. It is ancient, and the reason it is that way has to do with the way the human ear hears a scale of seven notes.
Comment: Is the B->C and E->F being 1 half step a guitar thing or all music thing?
Answer: Get a picture of a piano keyboard.  Write in the letter names on the white notes.  C is the one just below a group of two black notes.   When you get to G, start over with A.
Yes, if you start on a white note, a sharp always means 'next note up'.  Sometimes this takes you to a black note (C to C#).  Sometimes to another white one (E to F).  So, E# is F.  Sometimes it's clearer to spell it as E#, though just for now I think I'll ask you to take that on trust!
Guitar is not the easiest instrument to learn theory on :-)
Comment: @P.Brian.Mackey - you can get a little casio keyboard for a few dollars. Use it to peck out and visualize your scales and chords. I play guitar and (mostly) bass and I started doing that a few years ago - it is a great help. (Eventually you'll also learn how to play some piano - a very good thing to know.)
Answer: In the Western world, music wise, we split an octave into 12 equal parts - now called 12EDO, or equal temperament.
We also split (usually) an octave into what probably sounds like seven 'equal' parts. We call those major and minor scales.
Obviously, with 12 divisions on one hand, and 7 on the other, there's going to be trouble. In a key/scale of 7 notes, each must have a separate letter name, otherwise writing them on manuscript would be horribly messy.
So, we use letter names A to G, then start again.Unless we are German, and bring H into the equation, making it all more interesting!
Now, using the 7 letter names, only once each, there are going to be musical gaps between each on either a semitone (one step/fret) or a tone (two steps/frets).
Developing over many centuries, we now have A>B=T, B>C=S, C>D=T, D>E=T, E>F=S, F>G=T, G>A=T. When keys other than the ubiquitous C major are used, some of those notes need to be sharpened or flattened in order to sound in tune.
In the case of, say, Eb major, the spacing of the notes need the E to be flat, and also A and B. So it ends up having 3 flat notes in that key.
Another well used layout is the keyboard, where the white keys are called ordinary (natural) notes, A>G, and the black keys get called sharps OR flats, depending on the key of the prevailing piece.
Don't know where the terms 'step and half-step' come from, but they're universally known as tone (T) and semitone (S). They're sometimes portrayed as W (whole) and H (half), which seem to correspond to your terms better.


Java - How can these 2 objects equal each other?

Question: Hi I am revising and understand the difference between equals() and '==' however this code really confuses me:
public class MyTest {
    public static void main(String[] args) {
        Integer w = new Integer(1);
        Integer x = 1;
        double z = x;
        System.out.println(z == w);
    }
}

How can a double object have the same reference as an Integer object in memory when using '==' ? Cannot understand this. Thank you for reading!
Answer: If you compare primitive and wrapper using == operator, then the wrapper values will be un boxed and compared with primitive value.
From JLS 15.21.1

15.21.1. Numerical Equality Operators == and !=
If the operands of an equality operator are both of numeric type, or
  one is of numeric type and the other is convertible (§5.1.8) to
  numeric type, binary numeric promotion is performed on the operands
  (§5.6.2).
Comment: @MironBalcerzak It has nothing to do with Integer caching (by the way w is not cached, it is a whole new integer).
Comment: Thank you, I had a tricky question like this in the preparation for my exam.
Comment: @MironBalcerzak To expand on assylias' answer, `Integer` caching only comes into effect if you use the `Integer#valueOf()` method, if memory serves
Comment: Thank you all very much, I understand perfectly now. Will make sure to remember this for my exam :)
Comment: it's not true what u r writing Abimaran.
Comment: @MironBalcerzak, Then what's correct? Can you link to JLS?
Comment: @Abimaran, check Integer.IntegerCache. I will look for JLS and link in couple of moments - once I find it... "adjustments" like that are not on the 1st page of java doc. ;-/
Answer: A double is not a reference, because it is a primitive type. It is a value instead. Double would have been a reference.
When you write z == w, where z is double-typed and w is Integer-typed, the runtime definitely has to compare something. And, obviously, it has to compare things of the same kind. It will not compare the double value with the Integer reference. Instead, it will unbox the Integer and will compare the int value obtained after unboxing with the double value. Both being 1, the boolean operator will return true.
Answer: Here objects are not compared but primitive types are compared with feature of unboxing for Integer.


Prove there is either a chain or an antichain of infinite cardinal.

Question: Let $K$ be an set of infinite cardinal, $X$.  Let $(K,\le)$ be a partially ordered set. Prove there is either a chain $C$ such that $|C|=X$ or there is an antichain $A$ such that $|A|=X$. I guess I should be using Zorn's lemma, but I don't know how. I would truly appreciate it if you helped me.
Answer: This is consistently false even for regular cardinals, a Suslin tree is a tree is size $\aleph_1$ with no uncountable chain or antichain. The existence of a Suslin tree is independent of the axioms of set theory.
Answer: This is not true in general. For $n\in\omega$ let $\langle C_n,\le_n\rangle$ be a well-order of type $\omega_n$. Let $K$ be the disjoint union of the sets $C_n$, and let $\le$ be the union of the orders $\le_n$; $\le$ is a partial order on $K$. $K$ has cardinality $\omega_\omega$, but every chain in $K$ has cardinality $\omega_n<\omega_\omega$ for some $n\in\omega$, and every antichain in $K$ has cardinality at most $\omega$ (since it can contain at most one member of each $C_n$).
Comment: Does it make any use in ordinal?
Comment: @donnaIceberg: I'm not sure what you mean, I'm afraid. Can you explain your question a little more?


Intersection of sequentially compact sets

Question: Let A, B (non-empty) sequentially compact sets.
Then the intersection of A and B is sequentially compact.
One can prove this hypothesis by selecting a sequence in the intersection and observing a convergent subsequence in A the limit of which is a. Then there exists an subsubsequence in B and as the limits coincide and the hypothesis is proven.
Now, I am wondering, if the following is a valid proof (or it is just a reformulation of the argument in proof 1):
Again, select a sequence in the intersection and observe a convergent subsequence. As the sequence and thus the convergent subsequence with it's limit are in (say) A, the limit must be in B, too, otherwise there are elements in the sequence which are not in B, which is a contradiction to the givens. Shouldn't that proof the hypothesis as well?
Answer: The answer is No if you are working in a general topological space. If the condition is restricted to a metric space then both your sequentially compact subsets become closed by being compact spaces in the metric space. Then your argument should work.
What you are looking for should be a sequentially compact subset $B$ in some first-countable topological space which is not closed, thus the limit of some sequence $(x_n)$ is not contained in $B$. Choose another sequentially compact subset $A$ which is closed contains $(x_n)$, then although $(x_n)$ contains itself as a subsequence, the limit is not in $B$.
See the top answer on this post:
Sequentially compact subset of Hausdorff space is Closed?


Conflito entre Spring MVC e PrimeFaces

Question: Estou com um problema para integrar o Spring MVC com o PrimeFaces. Parece que dá um conflito entre eles.
Tem a configuração no web.xml
<code><servlet>
    <servlet-name>Faces Servlet</servlet-name>
    <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
</servlet>

<!-- Map these files with JSF -->
<servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>/faces/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>*.jsf</url-pattern>
</servlet-mapping>
<servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>*.faces</url-pattern>
</servlet-mapping>
<servlet-mapping>
    <servlet-name>Faces Servlet</servlet-name>
    <url-pattern>*.jsp</url-pattern>
</servlet-mapping>
</code>
o arquivo do faces-config.xml com o Spring

            org.springframework.web.jsf.el.SpringBeanFacesELResolver

E mesmo assim não funcionou.
Já tentei esse tutorial e o tutorial do Spring Web-flow que fala sobre o Spring-Faces e não funcionou.
O código fonte quebrado com o problema encontra-se no Github.
Comment: Bom acho que você deve estar falando sobre o Spring e não o Sring-MVC pois o tutorial que você passou não trata do Spring-MVC, melhor que postar o código nesse caso seria você postar a saida do seu servidor de aplicação ou servlet (jboss,tomcat) e também postar o seu pom.xml para que os colegas possam te ajudar com mais precisão.
Poste também o faces-config.xml e o applicationContext.xml
Answer: Sugiro que você comece seu projeto a partir de um arquétipo do Maven (como um dos dois que habilitam JSF + Spring desse repositório). Se quiser algo ainda mais ágil pode tentar também o suporte a JSF do Spring Roo. Em ambos os casos o PrimeFaces vem pré-configurado para você.
É claro que não existe solução perfeita e você terá que fazer upgrade manual de algumas bibliotecas no primeiro caso (caso queira trabalhar com as versões mais recentes do Spring e do PrimeFaces) e / ou aprender um pouco sobre o Spring Roo no segundo. Porém esse trabalho tedioso de configuração é feito para ti.
Answer: Boa tarde pessoal.
Muito obrigado Anthony por ter tentado me ajudar, mas percebi que não era exatamente isso que queria.
O que estava interessado em fazer está contido aqui:
https://stackoverflow.com/questions/18744910/using-jsf-as-view-technology-of-spring-mvc
Obrigado

Conflito entre Spring MVC e PrimeFaces


How do I get an output with Stargazer?

Question: I tried to create a regression table output using the Stargazer-Package. I did everything like the documentation said, but somehow I cant get it to work.
This is my code:
import pandas as pd

from IPython.display import display

from stargazer.stargazer import Stargazer

import statsmodels.api as sm

#Import

daten = pd.read_excel(r"G:\Meine Ablage\Kinga\Daten\Rice-Data.xlsx", sheet_name="Data")

#Festlegen des Regressands

Y = daten['yield']

### Komplett Regression

est = sm.OLS(endog=daten['yield'], exog=sm.add_constant(daten[daten.columns[4:8]])).fit()

est2 = sm.OLS(endog=daten['yield'], exog=sm.add_constant(daten[daten.columns[4:10]])).fit()

stargazer = Stargazer([est, est2])

stargazer

This is the error I get: Out[4]: <stargazer.stargazer.Stargazer at 0x1689be8e730>
How do I get it to run so that I can see the output (table with comparison)?
Answer: You can try forcing jupyter notebook to render the HTML by
from IPython.core.display import HTML
HTML(stargazer.render_html())

How do I get an output with Stargazer?


Parsing CSV with regex

Question: I am trying to read below from CSV and split accordingly:
"1000";" ";"";;0;0;0;0
"1001";"tit;le1";"desc";0;0;0;0
"1002";"title2";"desc2";0;0;0;0
"1003";"title3";"desc123 desc23 desc2 de
sc34 dfd desc45 desc454;,

dfd desc desc";0;0;0;0
"1004";"tilte4";"desc5";0;0;0;0

I am using [^;"][^"]+ and could get outcome of the strings between " " if not empty in above snippet.
I need the empty contents as well. Can you please help to correct the regex here [^;"][^"]+
Comment: May I suggest editing the title to better represent the actual problem?
Comment: Would it be possible to get a full-working, actual code-snippet?  This will make it easier to see the problem and to answer!  Thanks.
Comment: Is using a ready-made JavaScript CSV-parser an option?
Comment: Wouldn't it be easier for you to split every line with the separator ";" ?
Comment: @felixjiricka No, because ";" can occur in a value, not only between values.
Comment: Yes, cannot use ";" as it occurs between strings as well.
Comment: I am reading csv file from nintex workflow and getting the strings between " " by using regular expression. I am testing my regular expression in url (https://regex101.com/r/mw03oC/2)
Comment: @UshaRamani Sometimes, a regex is not the solution. Please see [How can I parse a CSV string with JavaScript, which contains comma in data?](https://stackoverflow.com/a/41563966/1115360) (you'll need to change the `','` to `';'`).
Comment: @AndrewMorton, why not? I think a regex is both possible and preferable.
Comment: @ByteEater If your regex is RFC 4180-compliant, it would be very useful if you added it as an answer in the Q&A I linked to.
Comment: It isn't. That RFC allows only for commas as separators. But otherwise I'll try to find time soon to look into it.
Answer: You want to get the stuff between ""? And there are no other " than those used as delimiters? Then "(.*?)" should work.
Comment: The same without a reluctant quantifier: `"([^"]*)"` or `"([^"]*).`.
Comment: Neither of the 3 expressions worked for me. I am trying in URL (https://regex101.com/r/mw03oC/2)
Comment: Did you enter them including the "s?
Comment: Oh, and if I understand correctly, there's a multiline string to be matched. In that case you should use the `s` flag with the expression. Thus: `/"(.*?)"/gs`.
Comment: tried this, but it doesnt capture the 0s and empty ex - ;;0;0;0;0 in the first line  and similarly in other lines all 0s.
Comment: Tried adding "(.*?)"|[^;"][^"]+ , except for the null/empty between ;; , everything seems to be now captured. i can split 0;0;0;0 further . but how to capture something between ;;
Comment: Then I didn't understand from you question what you wanted. So the ;s are separators, except when between "s, which always come in pairs and delimit strings – is that the complete specification? If so, see my new answer.
Comment: Thank you ByteEater. I have achieved with below regex  "(.*?)"|;;|[^;"][^"]+ . ; are seperators but i need anything in between ;; as it could be an empty one that need to be captured as a result.
Answer: With the spec from my latest comment the right expression is /(("?).*?\2)(;|$)/gsm.
Comment: I included parentheses for grouping in the previous answer, assuming that you don't want the "s. Now I think you do, but the parentheses are needed for a different reason: your target with this expression is the 1st capturing group, not the whole match.
Comment: For completeness, a version stripping the "s from values:`/ ("?)(.*?)\1(;|$)/gs`, with capturing group 2.
Comment: This worked as well. I am actually trying to split as each row, but because i couldnt split as rows, i am now thinking to split as each string and then create a row item later. I think you might be able to help me with my actual problem , because if i can split as single row , it would save me a lot. I had posted another post for an answer. https://stackoverflow.com/questions/64807334/regular-expression-to-split-strings-in-csv?noredirect=1#comment114583387_64807334. It would be great if you can help me here as well.
Comment: I've just noticed that there are no ;s at line ends, yet you want separation there (unless inside a string). So I edited the answer adding the `m` flag, so that e.g. `0` and `"1001"` aren't lumped together.
Comment: The question you linked is closed. Please edit it and apply for reopening or post yet another one which hopefully won't be closed. But specify your objective clearly, and preferably include the desired list of matches under the sample text.

Parsing CSV with regex


Possible to perform segue without the use of StoryBoard?

Question: I have been creating my Swift project completely programmatically thus far. I have created detailed Navigation Controllers and Table View Controllers without touching the StoryBoard (SB) once. I have come to a point where I would like to pass data from a TableViewCell click onto another ViewController, however, I believe that I need to use a segue and create/identify it inside of the SB. Since this application has become pretty complex over time, it has become quite difficult to mimic all of the view controllers inside of the SB, and creating any changes inside of the SB is not reflecting any changes inside of the Views on the simulator (I have paired the views inside of the SB to their respective class, yet nothing is working. Even the segue is not being recognized when the identifier matches up). Therefore, I have a couple of questions.

Is it possible to perform a segue programmatically? In other words, is it possible to pass data from one view controller to another without touching the story board?
Is there a simple guide or technique one can follow in order to mimic their code inside of the StoryBoard? With a simple application, it shouldn't be too difficult. However, is there any other way in order to portray the application in the StoryBoard if someone has been creating it completely programmatically?
Comment: Check out [this link](https://medium.com/@jacqschweiger/how-to-segue-programmatically-using-delegates-in-swift-e333a9800f5). It might give you a few pointers.
Comment: Thank you @vadian and @ iabuseservers for your help. For anyone else that may be having these issues, I suggest checking out the this video as well! Thanks again. Video: https://www.youtube.com/watch?v=70OhFMzRBZM
Answer: Interface Builder (IB) is just a GUI for programmatic development. Anything you can do in IB you can definitely do programmatically but not everything you can do programmatically you can do in IB, because IB is just for interface building—and just some interface building, not all.
segue is just IB terminology. In iOS, you can only display a view controller three ways: push (via UINavigationController), present (via a presentation object vendor UIViewControllerTransitioningDelegate), or show (a more generic approach to displaying a view controller).
Since you're new to programmatic iOS development (the best kind, IMO), a quick 101:
class ProgrammaticViewController: UIViewController {

    override func loadView() {

        // do not call super.loadView()
        // most IB developers don't even know this method exists
        //   because this is where IB does its work
        // add all of your view objects here (scroll views,
        //   table views, buttons, everything)

        setView()
        addTableView()
        ...

    }

    override func viewDidLoad() {
        super.viewDidLoad()

        // do call super.viewDidLoad(), however
        // do your post-view setup here, like adding observers
        //   or fetching data
        // this method is called after the entire view
        //   has been loaded into memory so consider that

    }

    // other lifecycle events that come after viewDidLoad where you
    //   can perform last-second work include viewDidLayoutSubviews(),
    //   viewWillAppear(), viewDidAppear(), etc.

    deinit {
        // do any cleanup here like deactivating timers, removing
        //   observers, etc.
    }

    // MARK: Methods

    func setView() {

        // if you're creating your view controller programmatically,
        //   you must create the view controller's actual view property
        //   and it must be done before adding any subviews to the
        //   view controller's view

        view = UIView()
        view.frame = UIScreen.main.bounds
        view.backgroundColor = UIColor.white

    }

}

If you are using auto layout (and you probably should be), you don't always need to explicitly set the frame of the view with view.frame = UIScreen.main.bounds. If it's the root view controller of the app, it's not needed—the window owns that view controller and it will set the frame. If you've then put a UINavigationController in the root and you're using auto layout, none of the view controllers you push to ever need their frame set as well. Therefore, the only real time you need to explicitly set the view's frame using something like view.frame = UIScreen.main.bounds is when you present a view controller modally. This is because a modally-presented view controller is not owned by the window or a navigation controller; it exists temporarily inside a transient container view. In this case, you must set its frame.
Passing data forward programmatically is simpler than it is for IB developers. Just instantiate a view controller, inject one of its (non-private) properties with a value, and push, present, or show it. Here's what a UITableView delegate for didSelectRowAt may look like:
func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {

    let p = indexPath.row
    let detailViewController = SomeDetailViewController()

    detailViewController.someProperty = searchResults[p]
    navigationController?.pushViewController(detailViewController, animated: true)

}

And this is obviously doable because you've created that property in that view controller and not given it a private access modifier.
class SomeDetailViewController: UIViewController {

    var dataObject: SomeType? // injectable
    private var notInjectable: SomeType?

}

UPDATE: IB is clearly not the future of iOS development as far as Apple is concerned, with the introduction of SwiftUI, which is another way of programmatically developing iOS. Personally, I hate GUI programming and I'm happy to see it slowly phase out.
Comment: Good Sir... THANK YOU SO MUCH. Not only did you explain my problem, I just got a great overview that will stick with me forever. Everything was clear and very helpful, but one quick question: I see that inside of the "setView()" function, you have declared the "view.frame" to be "UIScreen.main.bounds". Isn't the view.frame already set to be the Screen's main bounds? In other words, I usually don't declare that, and if I want some view to stretch the whole screen, I declare its width and height to be : view.frame.width and view.frame.height (respectively). In other words, is that needed? Thanks
Comment: A view controller's frame must be set somewhere if you're going pure-programmatic. But not every view controller will need it set inside itself with `view.frame = UIScreen.main.bounds`. For example, a child view controller can have its view's frame set by its parent. The root view controller also doesn't need it set since it's owned by `UIWindow` whose frame you set in AppDeleagte with `window = UIWindow(frame: UIScreen.main.bounds)`. But this is when you go pure-programmatic, where you have deleted the storyboard file and reconfigured your app not to launch to it.
Comment: and another question if you are still willing to help: When the data is pushed over to the variable "dataObject", how can I use that variable to be portrayed, lets say, inside of a label? Because whenever I try to access the value of that variable, I get the following error: Instance member 'dataObject' cannot be used on type ''. Any idea?
Comment: Here's a classic setup: https://gist.github.com/iabuseservers/d3f2202c6f21f16f8c50773f88f978a8
Answer:
Is it possible to perform a segue programmatically?

Sure. UIStoryboardSegue has initializers that you can use to create a segue, and it has a perform() method that you can call to make the segue do it's thing. But segues are mostly a tool that makes it easier to transition between scenes in a storyboard, so there's little reason to want to do that if you're not using storyboards in the first place.

In other words, is it possible to pass data from one view controller to another without touching the story board?

Again, yes. Storyboards didn't even exist until iOS 5, and you can bet people were writing apps with transitions between view controllers well before that. The answer you've already gotten about how to do that is fine. More generally, though, view controllers are just objects, and all that one needs to send information to another is a valid reference to the destination controller.

However, is there any other way in order to portray the application in the StoryBoard if someone has been creating it completely programmatically?

If you mean that you want to switch over to using storyboards, then yes, there are some good ways to do that. If you mean that you want to somehow represent what your app is doing programmatically inside a storyboard but continue to do everything programmatically, then no, there's no way to do that (and really no point in doing it).
Transitioning to storyboards doesn't have to be an all or none kind of thing. You can model just one view controller in a storyboard and load it as needed using init(name:bundle:) (to initialize the storyboard itself) and instantiateViewController(withIdentifier:) (to load a particular view controller), and then just use that view controller the same way you do now. All the code that you use to set up that view controller and it's view hierarchy can then be removed, since you're now loading it from the storyboard instead. Once that's working, repeat the process for other view controllers, expanding your storyboard and excising the setup code as necessary.


Probability that the histogram for the sum of two dice has an expected shape

Question: Suppose that two 6-sided dice are thrown $n$ times and that the sum after each throw is plotted on a histogram. Let $s_i$ be the frequency of the sum $i \in \{2, 3, \dots, 12 \}$. As $n \rightarrow \infty$, the probability that
$s_i \le s_{i+1}$ for $i \in \{2, 3, \dots, 6\}$ and
$s_{i} \ge s_{i+1}$ for $i \in \{7, 8, \dots, 11\}$
approaches $1$ (because the further the sum is from $7$, the less likely it is to occur). But after only a finite number of throws, what is the probability that the above expression is true? Can this be expressed as a closed-form function of $n$?
Comment: @JeanMarie I thought about asking for the solution to the general $(s, t)$-problem  when $s$ $t$-sided dice are thrown $n$ times, but the case $(2, 6)$ seemed like a good start (and is more practically relevant).
Comment: ... Knowing that the limit is a triangular distribution. There is a almost surely a closed form expression at least under the form of a multiple integral... with 12 integral signs...
Comment: Why not begin with a simpler problem with two identical dies wearing 3 zeros and 3 ones in order to answer the same question about histograms of the sum of their values in the much simpler set $(0,1,2)$ such that $s_0 \le s_1$ and $s_1 \ge s_2$...
Answer: As long as you can express a "dice pool" evaluation like this as a finite state machine with transitions determined by tuples (outcome, number of throws rolling that outcome) and keep the number of states in check, it's possible to compute the result in polynomial time of reasonable order. Here's a crude explanation of the algorithm; I'm working on a more refined explanation but it's not ready yet.
I implemented this approach as part of my hdroller Python library. You can try this computation in your
browser using this JupyterLite notebook.
State transition function
By default, hdroller presents outcomes to the transition function in ascending order. Therefore, we can define the state transition like this:
import hdroller

class IsTriangleIsh(hdroller.EvalPool):
    def __init__(self, rise_end, fall_start):
        self.rise_end = rise_end
        self.fall_start = fall_start

    def next_state(self, state, outcome, count):
        # State consists of the previous count,
        # or -1 if the rise-fall pattern was violated.
        if state is None:
            return count
        if state == -1:
            return -1
        if outcome <= self.rise_end and count < state:
            return -1
        if outcome > self.fall_start and count > state:
            return -1
        return count

    def final_outcome(self, final_state, _):
        return final_state >= 0

evaluator = IsTriangleIsh(7, 7)
# 2d6
die = 2 @ hdroller.d6

Manually checking the result for 1 and 2 throws:
print(evaluator.eval(die.pool(1)))
print(evaluator.eval(die.pool(2)))

Denominator: 36

Outcome
Weight
Probability

False
30
83.333333%

True
6
16.666667%

With a single throw, the only accepting result is if the roll is exactly 7. There are 6 ways for this to happen.
Denominator: 1296

Outcome
Weight
Probability

False
1140
87.962963%

True
156
12.037037%

With two throws, there are three accepting possibilities:

Both throws roll 7. (6 * 6 = 36 ways)
The first throw rolls 7, the second throw rolls 6 or 8. (6 * 10 = 60 ways)
The reverse of the above. (60 ways)

Total = 156 ways as computed.
Plotting
The exact chance for 500 throws takes about 4 minutes on my computer when running in JupyterLite. Results are memoized so we are not starting from scratch at each number of throws.
max_n = 500
x = [n for n in range(1, max_n+1)]
y = []

import time

start_ns = time.perf_counter_ns()
for n in x:
    y.append(evaluator.eval(die.pool(n)).mean() * 100.0)
end_ns = time.perf_counter_ns()
elapsed_s = round((end_ns - start_ns) * 1e-9)
print(f'Computation time: {elapsed_s} s')

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(16, 9))
ax.plot(x, y)
ax.grid()
ax.set_xlabel('Number of throws')
ax.set_ylabel('Chance of fitting shape (%)')
ax.set_xlim(left=0, right=max_n)
ax.set_ylim(bottom=0)
plt.show()

Unless I've made an error somewhere, the convergence to 100% chance is pretty slow by the standards of throwing physical dice.
By the time you reach 500 throws you're probably better off using statistical methods rather than exact calculation, but I'll leave that to someone more versed in that approach.
Comment: I appreciate your effort, but I was looking for an exact solution using statistics instead of a numerical solution.
Comment: The solution *is* exact combinatorics; there's no Monte Carlo, floating point arithmetic, numerical convergence, or anything like that. (The "convergence" above refers to reaching 100% chance of fitting the shape with enough throws, not to the algorithm.) You could write down the solution in terms of binomial-weighted sums nested ten deep, which would be in some sense mathematically cleaner, but computationally much more expensive (especially if you increase the number and/or size of dice).
Comment: I see the value in your approach and I was wrong to call it a numerical solution, even though it is limited. I have upvoted and accepted the answer.


Write the following as an algebraic expression of x; sinh(lnx)

Question: So I have a questions asking for $sinh(log_ex)$
sinh(x) = $\frac{e^x-e^{-x}}{2}$
So $sinh(log_x)$ = $\frac{e^{lnx}-e^{-lnx}}{2}$
$\frac{e^{lnx}-e^{lnx^{-1}}}{2}$
= $\frac{x - x^{-1}}{2}$
=$\frac{x-1}{2x}$
However the answer is $\frac{x^2-1}{2x}$
Can't seem to find where we went wrong
Comment: $\frac{x - x^{-1}}{2} =\frac{x^2 - 1}{2x}$
Answer: $$ \frac{x-x^{-1}}{2} = \frac{x-x^{-1}}{2} \frac{x}{x} = \frac{x^2-1}{2x}$$
Comment: Doesn't $\frac{x^{-1}}{2}$ = $\frac{1}{2x}$? Therefore the answer would be $\frac{x}{2}-\frac{1}{2x}$?
Comment: @user88720:  In what way is that different from what I wrote?  $\frac{x^2}{2x} = \frac{x}{2}$...
Answer: You were incorrect when saying $\dfrac{x-x^{-1}}{2}=\dfrac{x-1}{2}$.
$$\frac{x-x^{-1}}{2}=\frac{x}{2}-\frac{x^{-1}}{2}$$
$$=\frac{x}{2}-\frac{1}{2x}$$
$$=\frac{x^2}{2x}-\frac{1}{2x}$$
$$=\frac{x^2-1}{2x}$$
Hope I helped.
Comment: I fixed a minor typo. Cheers.


Am I interpreting my mixed effects Cox regression models properly? (two-level nested survival data)

Question: I: Data structure issue (two-level nested hierarchy in survival data)
[This analysis was done in R v4.0.1, using the Coxme package for mixed effects Cox regression models]
I have collected data on ants trying to survive in different thermal conditions, and I hypothesize that the surface area of these ants as well as the chemical compounds they possess determine their survivability. These ants were assayed in groups of 20 (called "Samples") replicated for each treatment, and these samples came from a few different nests. The response variable (survival, as a Surv(Time,Event) object) is measured for each individual, but each individual's body size is averaged at the level of "Sample", and their chemical compound data is averaged at the level of "nest". This hierarchical structure to the data looks like this:
> head(df, n=10)

##    NEST SAMPLE TIME EVENT       SA     X1675   X1700    X1760
## 1   ABJ   ABJ1   16     1 3.374059 0.3677422 2.46877 1.230814
## 2   ABJ   ABJ1   16     1 3.374059 0.3677422 2.46877 1.230814
## 3   ABJ   ABJ1   16     1 3.374059 0.3677422 2.46877 1.230814
## 4   ABJ   ABJ1   20     1 3.374059 0.3677422 2.46877 1.230814
## 5   ABJ   ABJ1   20     1 3.374059 0.3677422 2.46877 1.230814
## 6   ABJ   ABJ2   16     1 3.184234 0.3677422 2.46877 1.230814
## 7   ABJ   ABJ2   16     1 3.184234 0.3677422 2.46877 1.230814
## 8   ABJ   ABJ2   16     1 3.184234 0.3677422 2.46877 1.230814
## 9   ABJ   ABJ2   16     1 3.184234 0.3677422 2.46877 1.230814
## 10  ABJ   ABJ2   20     1 3.184234 0.3677422 2.46877 1.230814

Here we can see one NEST (ABJ) that multiple SAMPLES were taken from (ABJ1,ABJ2,etc.), and while surface area (SA) is unique to each sample (3.37, 3.18, etc.), the chemical data (X1675, X1700, etc.) is unique to each nest. Because of this structure, I don't think I can just do a usual Cox Proportional Hazard model (like coxph()).
II: Comparing models, considering random effects
To deal with this nested hierarchy, I tried the following approach of mixed effects model comparison to see if adding my surface area and chemical data as predictor variables would change the AIC scores/log likelihood(loglik) of models allowing each NEST and SAMPLE to have a random intercept. I may not be explaining this well, but here is what that comparison looked like (method based on this YouTube video, and the models are using coxme()):
> anova(m1,m2,m3)

## Analysis of Deviance Table
##  Cox model: response is  Surv(Time, Event)
##  Model 1: ~1 + (1 | nest/Sample)
##  Model 2: ~1 + sa + (1 | nest/Sample)
##  Model 3: ~1 + sa + (1 | nest/Sample) + X1675 + X1880 + X2600 + X2700 +      X2800 + X2900 + X3100
##    loglik   Chisq Df P(>|Chi|)
## 1 -9946.9
## 2 -9946.0  1.7814  1     0.182
## 3 -9930.6 30.8123  7 6.733e-05 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Model 1 consideres only random effects for each SAMPLE nested within NEST. Model 2 adds on the sample-level metric of surface area, and we see no change in loglik. Model 3 adds on the nest-level chemical data, and we see a significant change to loglik. So, my interpretation of this is that the chemical data, despite being a nest-level metric that is identical across many individuals, still adds meaningful "explanatory power" to the differences we see in survivability. The problem is, I would be satisfied here if this model considered all predictor variables I want to include. There are 7 more chemical compounds I want to include, but coxme() can't handle too many predictor variables while considering random intercepts, and I've read that, as a general rule of thumb, random effects + many predictor variables = less accurate coefficient estimates. Also, while these model comparisons tell us vaguely what predictor variables explain survivability, the hazard ratios that can be generated from Cox Proportional Hazard models (with no random effects) offer the best metrics for judging how each variable influences survivability.
III: The problem (am I following the rules?)
This is where I am least certain about my model formulation + choice. I figured, if a model containing random effects + chemical data is better than a model of just random effects, then the random effects are not critical to consider. Therefore, we can thus examine a model of just the chemical data, and focus on the hazard ratios to estimate how each chemical compound influences ant survivability. So, my questions, stated finally, would be: is it safe to ignore random effects because of my model comparisons so far, or do I need to stick to a model incorporating random effects, thus limiting the amount of predictor variables I want to include?
I can provide more information where needed. Thanks for the help!
Comment: Why do you say "*if a model containing random effects + chemical data is better than a model of just random effects, then the random effects are not critical to consider*" ? It is not safe to remove the random effects unless you account for non-independence withing the groups in some other way, such as fitting fixed main effects and interactions for them. If you don't do this then your inferences will be biased.
Answer:
Because of this structure, I don't think I can just do a usual Cox Proportional Hazard model (like coxph()).

That's not necessarily true, and if it is true it might not be for the reason you think. The fact that, for technical reasons, many individuals have the same values for areas and concentrations doesn't by itself rule out a simple coxph() analysis. The problem is that survival among individuals in one nest or sample might differ systematically from survival in another nest or sample under otherwise identical conditions. You need to take such correlations among associated individuals into account.
If you only have "a few" different nests, you might be able to treat nests as fixed effects. Then you could use the standard coxph() function, with a cluster variable handling the correlations within samples. For anything more elaborate, coxme() is a reasonable choice.

I've read that, as a general rule of thumb, random effects + many predictor variables = less accurate coefficient estimates.

This depends on the details. If there are more than a few groups, modeling with random effects typically poses less of a problem in this respect than does modeling with fixed effects. With random effects you are mainly modeling the variance among the effects rather than the individual effects themselves, reducing the number of parameter values you have to estimate from the data.
The number of events is typically the major limiting factor for reliably fitting a survival model. The usual rule of thumb is to have about 15 events per coefficient that you are estimating in the model to avoid overfitting.
You have surface area and 14 compounds* that you would like to evaluate as fixed effects, along with some variances among random effects. So if you have something like 300 or more events, you probably would be OK with random effects for samples and nests, and fixed effects for all your compounds and surface area. Yes, the variance of coefficient estimates does increase as you try to estimate more coefficients and thus reduce the residual degrees of freedom, but if you have a large enough event/coefficient ratio to avoid overfitting that shouldn't be a big problem.

I figured, if a model containing random effects + chemical data is better than a model of just random effects, then the random effects are not critical to consider. Therefore, we can thus examine a model of just the chemical data ...

As Robert Long's comment implies, that would not be "following the rules." One way or another, you need to take into account the lack of independence among observations that you have with this type of structure. One can discuss whether that is best done with nests treated as fixed versus random effects, or whether random effects in a mixed model versus clusters in a standard Cox model are the best ways to handle samples. But you have to take the correlations among observations into account somehow.

*A couple of cautions about the chemical-concentration analysis. First, it's not clear when the concentrations are measured. For survival analysis those concentrations should be those present at the event times. If, for example, concentrations are measured at the end of the experiment, then it might be that the deaths are leading to the concentration changes rather than vice-versa. Second, if you have fewer nests than the 14 compounds you want to evaluate, then including all the compounds along with the nests as predictors will lead to a linear dependence that might pose problems with random-effect modeling of nests and almost certainly will be an issue if nests are treated as fixed effects. If you've been having problems with coxme() when trying to include all 14 compounds, that (not some inherent problem with the number of fixed-effect predictors in a mixed-effects model) might be why. You might consider a ridge-regression type of penalization on the concentrations to get around that.


Unlocking a LUKS encrypted root partition remotely via SSH

Question: My server is running CentOS 6.4, fully encrypted using LUKS.
The question is. What can I do to be able to insert the passphrase after a server reboot, thru ssh?
The only referente I found, was about installing dropbear and busybox, and editing the file initramfs.

But I cound't find any initramfs on CenOS. busybox is already installed, I was able to install dropbear thru rpm.
So, is there an equivalent to initramfs in CentOS?
Answer: It's called initrd, not initramfs there.
See http://www.centos.org/docs/2/rhl-cg-en-7.2/buildkernel-initrd.html
http://wiki.centos.org/TipsAndTricks/CreateNewInitrd


Android Studio - IBus prior to 1.5.11 may cause input problems. See IDEA-78860 for details

Question: Android Studio 1.5
Build #AI-141.2422023, built on November 12, 2015
I just updated my Android Studio on Ubuntu 15.10 when it gave me the prompt that a new version is available. After the patch download was finished, I chose the option to use my previous Android Studio settings when it asked for the prompt. But after that it is giving me the cannot resolve symbol ... error everywhere in the project and also showing me the pop-up warning

System Health
  IBus prior to 1.5.11 may cause input problems. See IDEA-78860 for details.

I have tried cleaning and rebuilding the project. But nothing is working. What do I do? Please Help.
Thank you.
Answer: When IDEA is used on a system with IBus is installed and running keyboard input sometimes became blocked by IBus daemon. Event Dispatching Thread is active. All other events are delivered in order - i.e. it is possible to select text or call menu actions by a mouse.
Solution:
Either upgrade IBus to version 1.5.11 or add "export IBUS_ENABLE_SYNC_MODE=1" to your ~/.profile, then restart a session.
Alternatively, you can turn off IBus at System Settings | Language Support | Keyboard input method.
IDEA 15 will show a warning if an incompatible version of IBus is in use.
checkout this link for detals https://youtrack.jetbrains.com/issue/IDEA-78860
Comment: Going through the link you provided, I assume that this problem is not even remotely attached to `Android Studio` giving errors everywhere after the update, if  I am not wrong. The problem with IBus is completely different. That said, can you tell me how to upgrade my IBus? I found this link (http://www.ubuntuupdates.org/package/core/wily/main/base/ibus), should I use it?


Erro ao utilizar monolog: PHP Fatal error: Class 'Monolog\Logger' not found

Question: Tentei utilizar monolog na minha aplicação, porém, depois da instalação feita de acordo com a documentação, eu tento importar e utilizar, mas vem uma mensagem de erro informando que não foi possível encontraá-lo.
<?php

use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Monolog\Handler\FirePHPHandler;

// Create the logger
$logger = new Logger('my_logger');

Utilizei apenas o código acima.
Comment: Está utilizando o Composer ou alguma outra ferramenta de autoload?
Comment: @AndersonCarlosWoss Utilizo o Composer
Comment: E o arquivo de autoload foi inserido no projeto em algum momento?
Comment: @AndersonCarlosWoss Sim sim, está dentro da pasta 'Vendor'
Comment: Inserido no sentido de dar `include` ou `require` neste arquivo. Desculpe-me caso souber usar o Composer e essas perguntas parecerem básicas, mas é que o erro parece estar em o PHP não conseguir carregar a classe e a ausência do `include` faria todo sentido.
Comment: Pode ser que o básico me ajude, pq não uso muito php, então não conheço muito. Não, o arquivo autoload não foi carregado, imaginei que só utilizar o `use Monolog\Logger;` ja era o bastante para que funcionasse
Comment: Agora sim funcionou. Coloca uma resposta dizendo pra mim realizar um require no autoload. @AndersonCarlosWoss
Answer: Conforme discutido nos comentários, buscando entender melhor o problema, a falta do carregamento do arquivo autoload do Composer faz com que o PHP não saiba como buscar as classes. Composer faz mágica, mas nem tanto. Após a instalação, deve existir um diretório vendor em sua aplicação, com as dependências de projeto. Antes de usá-las, faça a inserção do arquivo autoload, no início de cada arquivo:
<?php

require __DIR__ . '/vendor/autoload.php';

use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Monolog\Handler\FirePHPHandler;

// Create the logger
$logger = new Logger('my_logger');

O arquivo vendor/autoload.php é o padrão do Composer, que faz toda a mágica acontecer.

Nota: o arquivo autoload deve ser incluído uma vez a cada requisição tratada no PHP. Isto é, se sua aplicação faz requisições a diversos arquivos PHP, o require autoload deve estar presente em todos eles. Se ela for baseada em alguma arquitetura, tal como MVC, em que todas as requisições são tratadas em apenas um arquivo, basta, apenas, colocar nele para estar acessível em todo o projeto.


Find the sum of the angles

Question: The question is below.

The solution says:
Solution

My questions:
I'm wondering why it was concluded that the path goes around the center three times. Also, what does that have to do with the external angles?
Comment: May be relevant:http://math.stackexchange.com/questions/1387660/find-the-sum-a-b-c-d-e-f-g
Answer: Draw a ray from anywhere in the open center region passing through the point $A$ and ending well outside everything in the figure. Then, starting at $A$, begin tracing the figure until you return to $A$ again. Count how many times you hit the ray you drew (don't count the starting point as a hit, but you do count the ending point as a hit). You will find you hit the ray three times -- once for every circuit around the central point you chose.
The external angles are how much you turn as you "round the corner" around the outside. If you go around once, you have turned $360^{\circ}$.
Comment: How do you get that there are $3$ different paths? Can you list them out?
Comment: There aren't 3 different paths. Just one long winding path: A to D to G to I to B to E to H to C to F to A. You will cross the ray the first time when you go from I to B. Keep going until you get back to A again (you are only traveling along the edges from vertex to vertex).
Answer: At Wikimedia there's a good animation of winding number. For external angles there's a picture of a figure where external angles add up to $360^\circ$. Similar thing can be seen when the path goes around more than once, in your example it goes around three times, so external angles add up to $3\cdot360^\circ$.
Answer: Envision a car that starts at $A$ driving towards $F$, then keeps going until it ends up back at $A$ again.
I suspect the idea that the path goes around the center three times is meant to be intuitive, though whether it is or not is up to debate. Regardless, it can be helpful to have a more rigorous way to tell these things. Here's one: consider some line passing through $A$ and going off somewhere between $F$ and $E$. Trace the path from $A$ with your finger, counting every time you cross it from right to left.
You'll get two, and since you start and end at $A$, count that as another crossing. So three.
Take a second to convince yourself that when the car turns around an angle, it rotates a number of degrees equal to the external angle to (which measures $180^{\circ}$ minus) that angle.
For example, (not thinking about the above diagram at all), if the car is on Main St. and needs to take Elm St, which branches off slightly to the right,
<code>   Elm
  /
 /
 |
 |
Main
</code>
then Elm St. makes a very wide angle with Main St., but the car itself doesn't need to turn very much.
But if the car needs to turn onto Crazy St., as below:
<code> | \
 |  \
Main \
 |    \
 |   Crazy
</code>
then the car needs to turn wildly, even though Crazy St. makes a very small angle with Main St.
If the car needs to make a U-turn, then it needs to turn a full $180^{\circ}$, since Main St. forms a $0^{\circ}$ angle with itself.
So for an internal angle $\alpha$ and external angle $\beta$ we have $180^{\circ} - \beta = \alpha$.
The amount the car turns is $3 \cdot 360^{\circ} = 1080^{\circ}$, because that's three full turns. That's also the sum of all the external angles. Summing everything up we get (because there are $9$ points):
$$
\begin{aligned}
9 \cdot 180^{\circ} - \text{sum of $\beta$'s} &= \text{sum of $\alpha$'s}\\
9 \cdot 180^{\circ} - 1080^{\circ} &= \text{sum of $\alpha$'s}\\
\end{aligned}
$$
Bonus: This idea can be used to show that the sum of the internal angles of a triangle is always $180^{\circ}$. Which already puts you ahead of the wisest geometers of the Dark Ages.
Comment: How did the car make $3$ full turns? I thought it just turned on Main, Elm, and Crazy St.? Is each one of those turns $360^{\circ}$?
Comment: @JacobWillis: The Crazy St. etc examples are just examples of how to think about the relationship between internal angle and external angle. They've got nothing to do with the specific problem.

Find the sum of the angles


Imagem chega nula no servidor

Question: Estou enviando dados usando o retrofit, observo toda a requisição até o fim e recebo o código 200 mais no lado do servidor a imagem chega nula, como estou enviando a imagem e textos o texto chega corretamente, sem problemas contudo a imagem não, já alterei o código, pesquisei olhei tutoriais, inicialmente eu estava enviando uma lista de imagens, mudei estou enviando apenas um mais este não é o problema meu servidor é em Java
Todo o processo começa obtendo imagem da camera, os dados da activity são gravados no banco SQLite e posteriormente pela ação do usuário ele chama um serviço que faz o envio do registro para o servidor.
Aqui a interface WebServiceApi.java
@Multipart
@POST("fw_file.rule")
public Call<WsViagemResponse> sendViagem3(
        @Part("metodo") RequestBody metodo,
        @Part("parametros") RequestBody parametro,
        @Part MultipartBody.Part images
);

aqui o código na classe responsável por transmitir os dados
MultipartBody.Part file1 = prepareFilePart("arquivo1.jpg", v.getFoto());
// Retrofit
WebServiceApi apiService = ApiClient
            .getClient(cURL)
            .create(WebServiceApi.class);

    Call<WsViagemResponse> call = apiService.sendViagem3(
            createPartFromString("fw_addViagem"),
            createPartFromString(parametros),
            file1
    );

@NonNull
private MultipartBody.Part prepareFilePart(String fileName, byte[] content) {

    RequestBody requestFile =
                   RequestBody.create(MediaType.parse("image/*"), content);

    return MultipartBody.Part.createFormData("file", fileName, requestFile);
}

@NonNull
private RequestBody createPartFromString(String descriptionString) {
    return RequestBody.create(
            MediaType.parse(MULTIPART_FORM_DATA), descriptionString);
}

para as dependências
compile 'com.squareup.retrofit2:retrofit:2.1.0'
compile 'com.google.code.gson:gson:2.7'
compile 'com.squareup.retrofit2:converter-gson:2.1.0'
compile 'com.github.anshulagarwal06:Simplify-Permissions:v1'
compile 'com.squareup.okhttp3:logging-interceptor:3.3.1'

Não sei o que pode estar errado uma vez que no logcat eu vejo toda a requisição sendo feita
D/OkHttp: fw_addViagem
D/OkHttp: --55fa9778-7456-403f-be6b-48ab3c386c9b
D/OkHttp: Content-Disposition: form-data; name="parametros"
D/OkHttp: Content-Transfer-Encoding: binary
D/OkHttp: Content-Type: multipart/form-data; charset=utf-8
D/OkHttp: Content-Length: 82
D/OkHttp: 1|3|2|06/01/2017 17:38:55|01/02/1900 00:00:00|01/02/1900
D/OkHttp: --55fa9778-7456-403f-be6b-48ab3c386c9b
D/OkHttp: Content-Disposition: form-data; name="file";   filename="arquivo1.jpg"
D/OkHttp: Content-Type: image/*
D/OkHttp: Content-Length: 16053

vem depois deste log a imagem e no final
D/OkHttp: <-- 200 OK http://10.0.10.25:8082/webrun/fw_file.rule (111ms)

Answer: O problema aqui tem a ver com os parâmetros do meu Servidor, ele espera três cujos nomes são: metodo, parametro e arquivo
A interface tem os três porém nela não nomeamos o ultimo
@Part MultipartBody.Part images

A solução esta no método prepareFilePart, onde antes eu tinha "file" devo trocar por "arquivo" casando com o parâmetro esperado
@NonNull
private MultipartBody.Part prepareFilePart(String fileName, byte[] content) {

        RequestBody requestFile =
               RequestBody.create(MediaType.parse("image/*"), content);

        return MultipartBody.Part.createFormData("arquivo", fileName, requestFile);
    }

Imagem chega nula no servidor


How to Set-Up VPN that is Client and Server Independent

Question: I have about 5 clients who are all running some iteration of Microsoft Windows Server, be it Server 2003, Server 2008, Server 2013.
I will commonly connect to their networks/machines in a few different ways depending on if I'm in the same building or not in the building.

When I'm in the building I can simply connect to the WiFi and start a remote session using Microsoft Remote Desktop and enter the IP.
When I'm not in the building one of the clients has a LogMeIn account which I connect to and then eventually make my way to the server.

What I'm looking for is advice or a set of instructions to allow me to connect to each of my clients servers with as little extra software as possible (ie by using existing services already installed in Windows Server or settings/software already available on their in-house routers being simple Linksys, SonicWall or even Cisco).
And then once I've set up this VPN connection I'd like to be able to connect to it, independent of what system I'm on. If I'm on OSX or Windows 7 I'd like do have VPN connections for each of my client locations, and then be able to RDP into their server or any machine on that network.
I've tried setting up VPN's a few of their servers but I wasn't sure I was going through the process correctly. Do I need to forward ports? Do I need to forward ports to their router? Their server? Isn't there a VPN service already available on Windows Server? How do I configure it to allow connections from wherever I might be? What if I'm on the road traveling and I need to remote into their machine so I tether from my cell phone and do a quick fix. How can I have the connection accepted independent of what the connecting IP is?
Answer: There's no magic here. VPN is a complex ecosystem, with many different vendors, many different technologies, and many different configurations. There is no "VPN Master Software" or the like that will let you do what you want.
About the closest configuration I can conceive of is to configure persistent site-to-site tunnels from each client location back to your network, or back to a single server under your control. With that in place along with proper routing rules, you'll be able to, yourself, VPN to that single point, and then route traffic to each client.
This will not be a simple thing to set up, though, and you should keep in mind the following items:

If any of your clients have overlapping subnets, the above scenario will not work. You'll need to re-number their network.
For the above to work, you'll likely need hardware or software that supports IPSec on both ends of each VPN association
You'll need to ensure that clients on one client's network cannot route traffic to other clients' networks
Protect your "bastion" host very well, as that server or network has the keys for each and every one of your clients networks

If I were to offer a piece of advice? Forget your plan. Instead, work to migrate your clients to a consistent router/firewall/vpn platform so that you have a consistent means of connecting to their networks. OpenVPN is a good option here, as there are well-maintained OpenVPN clients for every OS and mobile platform out there.
Answer: Honestly, you're asking a lot here. EEAA points out many of the potential issues and stumbling blocks you'll face in getting something implemented based on your desires.
Personally, my suggestion would be to use something like GoToAssist, TeamViewer or LogMeIn. You already have a client using LogMeIn. Each of these require only that you install the software on your client machines and your workstation/laptop. I can't speak to TeamViewer and LogMeIn, but GoToAssist has both a Windows and a Mac client/application. IMO, You'll find this to be the easiest way to remotely connect to and manage your client systems.
Trying to find a common VPN solution that fits into every client scenario sounds like an exercise in frustration. You'd need to manage firewall/router settings (and quite possibly might need to purchase hardware), configure VPN client software/settings, etc., etc.


Product of Bernoulli Random Variables

Question:
I am stuck on "part (a)" of this question.
My approach:
X, Y and Z are independent Bernoulli random variables.
So, XYZ is also a Bernoulli random variable, since it can take only two values - {0,1}
(1 when all of them are 1 and 0 when any of X, Y or Z is 0)
Therefore P(XYZ = 0) = 1 - {0.5*0.5*0.5} = 7/8
Now, 7/8 = 0.875, which is greater than 3/4 or 0.75.
But I don't think this is the right approach to solving this question, since I wasn't able to arrive at the inequality explicitly.
I would be very grateful, if somebody can help me with this question.
Comment: You don't seem to realize this fact: Pairwise independence is _weaker_ than complete independence.  I.e. the problem statement only gave you enough to conclude that $P(XY=1) = P(X=1)P(Y=1) = 1/4$ and $P(YZ=1) = P(Y=1)P(Z=1) = 1/4$ and $P(ZX=1) = P(Z=1)P(X=1) = 1/4$.  However, the problem stmt does _not_ give you enough to conclude that $P(XYZ=1) = P(X=1)P(Y=1)P(Z=1) = 1/8$.  In fact the $Z$ defined in part (b) is the classic example of pairwise independence but not complete independence.
Comment: Thank you so much for the answer!
Answer: Oh, no, you were informed that the variables are pairwise independent, but that does not assure you that they will be mutually independent.
So you cannot justify saying: $\mathsf P(XYZ{=}1)=\mathsf P(X{=}1)\mathsf P(Y{=}1)\mathsf P(Z{=}1)$.   It may not be true (and indeed, part (b) indicates that it will not be true).
However, you can say $\mathsf P(XYZ{=}0) = \mathsf P(X{=}0\cup Y{=}0\cup Z{=}0)$ and then use the principle of inclusion and exclusion to express this as a series of seven terms.   The information given will allow you to evaluate six of these seven terms, and there is an inequality you do know about the seventh propability measure.
Go to it.
Comment: Oh, alright!

Thank you so much for the help!

Product of Bernoulli Random Variables


How can I find out what is causing interrupts on Windows?

Question: Occasionally I come across servers (Windows 2003 and 2008) with high processor % interrupt time. Is there a way to see what program or device is causing the interrupts?
Answer: After digging through the documentation (based on the other answers here), this is the process I ended up using:

Capture the ETW log of the problem
The easiest way to do this is using the Windows Performance Recorder. I'm not sure when it first appeared, but seems to be built in on recent versions of Windows. Set the profile to CPU usage.

or, using an elevated command prompt, navigate to the folder which contains it and use the command-line tool xperf:
xperf -on base+interrupt+dpc

Note, you will need to close Process Monitor or any other app which uses ETW or you will get the following error: xperf: error: NT Kernel Logger: Cannot create a file when that file already exists. (0xb7).
Stop tracing / save the log
xperf -d interrupt_trace.etl

Open the trace in Windows Performance Analyzer (part of Windows Performance Toolkit); some places mention using xperfview instead.
Expand Computation -> CPU Usage (Sampled) -> DPC and ISR Usage by Module, Stack, right-click and add graph to analysis view

This pointed right to the driver in question. In this case, HDAudBus.sys is using a constant 10.82% of my cpu via interrupts, which is exactly what Process Explorer was showing me.
Comment: Nice one! Very well done.
Comment: One correction - you need to expand Computation-> CPU Usage (Sampled) - "CPU Usage" is ambiguous.
Comment: @Unknown123, if you click through the dialog, it will let you uncheck everything but "Windows Performance Toolkit", which brings it down to 159 MiB for me, which still seems like a lot, but I guess it's worth it if you can find your problem
Comment: Well explained. In my case, it turned out to be audiodg.exe. As soon as I killed it, DPCs went to almost nothing. I found additional details on how to resolve this here: http://windows-exe-errors.com/fix-audiodg-exe-high-cpu-usage-in-windows-7/
Comment: Does it work on Windows 10? Any other tools available? I don't want to install GiB files of the whole pack.
Comment: Instead of using WPR GUI ([included in Windows ADK](https://docs.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder)) or `xperf` ([no longer supported](https://docs.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-toolkit-technical-reference)), one may use the WPR command-line ([shipped with Windows 10](https://www.nextofwindows.com/windows-10-comes-with-windows-performance-recorder-wpr-exe-built-in), _~Winbiquous_). In above steps, 1: `wpr -start cpu`; 2: `wpr -stop output.etl`. A folder named `output.etl.NGENPDB` will also be created for .NET symbols.
Comment: @Unknown123 for Windows 10 you need to install Windows Performance Analyzer from the Windows Assessment and Deployment Kit (Windows ADK). Start here: https://docs.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder
Comment: In my case I had 10% DPC in Task Manager, but most of it was ntoskrnl.exe in the trace. However, expanding the stack of that module revealed a 3rd party service "RfeCo10X64.sys", which was part of Killer Performance Suite. I uninstalled that software (some network prioritization system that was doing the opposite of what it attempted) and my problem was resolved.
Comment: Thank you SO MUCH. I tried all sorts of things but I couldn't fix the high interrupt usage. This points you right at the problem drivers.
Answer: If you can handle low-level system tools;

Windows Performance Analyzer (WPA)
Windows Performance Analyzer (WPA) is
  a set of performance monitoring tools
  used to produce in-depth performance
  profiles of Microsoft Windows
  operating systems and applications.

After you learn how to use xperf; check out;

The DPC/ISR Action
The DPC/ISR action produces a text
  report that summarizes the various
  metrics regarding DPCs and ISRs. The
  usage for this action is:
Copy Code
  -a dpcisr [-dpc -isr -summary -interval [n]  -bucket [n] -range T1 T2 ]
Option
Description
dpc
Show statistics for DPC only
isr
Show statistics for ISR only
summary
Show summary report
interval [dt]
Show usage report for intervals of dt,
  default is 1 second
bucket [dt]
Show histogram for intervals of dt,
  default is 2 seconds
range T1 T2
Show delays between T1 and T2
If no data type is specified, default is to show report for both DPC

and ISR. If no report type is
  specified, default is to print all
  three kinds of report.
Answer: Here's the best article I've found on how to do this, with tutorials, screenshots, and download links to the relevant tools:
http://www.msfn.org/board/topic/140263-how-to-get-the-cause-of-high-cpu-usage-by-dpc-interrupt/
Comment: Welcome to Server Fault! Generally we like answers on the site to be able to stand on their own - Links are great, but if that link ever breaks the answer should have enough information to still be helpful. Please consider editing your answer to include more detail. See the [FAQ](http://www.serverfault.com/faq) for more info.
Answer: Two great tools are LatencyMon and DPC Latency Checker.
Comment: [From here](https://msfn.org/board/topic/140263-how-to-get-the-cause-of-high-cpu-usage-by-dpc-interrupt/): If you use Windows 8, don't use the "DPC Latency Checker tool". Due to internal Kernel changes in Windows 8, the "DPC Latency Checker tool" shows DPC spikes of over 1000µs all other the time. Those VALUES are not correct!


Can ARCH, GARCH, and GARCH derivative models be used outside of finance?

Question: I'm learning about these models and seems like they can only be used in finance, and the reasoning is the assumption that variance is returns squared. Is this correct?
Can ARCH/GARCH be used to model processes outside of finance? What are some examples? What are some models used to model nonconstant variance outside of finance?
Answer: The GARCH-type of models do not assume that variance equals squared return, but squared return is a proxy for variance. This is a purely statistical observation and does not depend on use in finance. The reason for the popularity of GARCH models in finance is that they capture some stylized facts (typical patterns) of financial time series, e.g. volatility clustering and heavy tails (Cont, 2001).
The seminal paper by Engle (1982) used the newly introduced ARCH model for a macroeconomic time series, namely, inflation in the United Kingdom. Campbell and Diebold (2005) used a GARCH model for temperature forecasting (and achieved pretty decent accuracy compared to professional whether forecasts). There are examples in other areas, too, if you look for them. Hence, (G)ARCH models are definitely applicable beyond finance.
References

Campbell, S. D., & Diebold, F. X. (2005). Weather forecasting for weather derivatives. Journal of the American Statistical Association, 100(469), 6-16.
Cont, R. (2001). Empirical properties of asset returns: stylized facts and statistical issues. Quantitative Finance 1(2), 223-236.
Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econometrica: Journal of the Econometric Society 50(4), 987-1007.


Why my action is not dispatched after polling with a takeWhile

Question: I have an angular effect that is triggered by myFirstAction and is supposed to poll till a file was created in the back end, and then dispatch another action:
  myEffect = createEffect(() =>
    this.actions$.pipe(
      ofType(Actions.myFirstAction),
      withLatestFrom(
        this.store.select(Selectors.fileName)
      ),
      switchMap(([payload, fileName]) =>
          this.pollTillFileCreated(fileName)),
        map(() => Actions.mySecondAction()),
        catchError((error) => {
           return of(Actions.errorAction(error));
        })
      )
    );

and my polling method is:
  private pollTillFileCreated(fileName: string): Observable<boolean> {
    return timer(0, 1000).pipe(
      concatMap(() => this.service.fileExist(fileName)),
      takeWhile((doesExist: boolean) => !doesExist),
      takeLast(1)
    );
  }

Though I can see that my HTTP call returns doesExist: true,  mySecondAction is not being dispatched.
Answer: The short answer was I needed to add inclusive: true:
takeWhile((doesExist: boolean) => !doesExist, true),

The explanation for my issue is, that in my simple case my file is generated quickly in the back-end, so the first poll will return true.
so it will be the first and the last poll.
when takeWhile is used without inclusive: true it will not keep going until the predicate will return false and will not include it.
In that case takeLast(1) will have nothing to take, since we are excluding the last (which is also the first).
This way we are not returning anything, so there is nothing to pass to the map.


Verifying ClearCase files have been labeled properly

Question: My company uses ClearTeam Explorer for version control. Sometimes when either I or a co-worker apply a label, we get an error message. Today my comrade got
"An error occurred during a ClearCase operation.
 CRMAP7009: CCRC command "WriteProperties" failed:
CCRC WANServer: Error: No permission to perform operaton "change event".
 CCRC WAN Server:Error: Mustbe one of : user associated with event, object owner, element owner, VOB.."
According to the above message, you would think the operation failed, correct?
However, it appears looking at a random sampling of the files that everything was labeled correctly.
It is alarming to get this message and then find out it appears to have done what it said it didn't??? - There will always be lingering doubt unless you can verify that every single file was labeled properly.
Early on, I ignored this error and continued making labels for various versions. Now, I want to verify because I have lots of GB of files and there is no way I can hand verify everything.
Offhand, it seems like I want to make a view based on the date the label was created (maybe months ago) and then have a script to show any files that don't have a label.
Comparing files between 2 labels might work also provided at least one of the labels has "all the files" labeled?
Any guidance is appreciated.
Answer:
According to the above message, you would think the operation failed, correct?

No, and there is an APAR PM79463 in progress on that one.
(an APAR is an Authorized Program Analysis Report, or APAR, that is: "a formal report to IBM development of a problem caused by a suspected defect in a current release of an IBM program")
(CTE = " ClearTeam Explorer")
Error description

User tried to apply label to an element on CTE, and error messages were shown although label was applied.
So user tried to apply label to an element on HomeBase in native    client after logging in Windows as the same user as user who was used in CTE, and succeeded to apply label without errors.
  If executed user is not the owner nor creator of the object but belongs the same group of VOB owner, the issue was occurred on CTE although the operation was succeeded on HomeBase.
Error messages are shown on pop up window as follows.

<code>  CRMAP7009: CCRC command 'WriteProperties' failed:
  CCRC WAN Server Error: no permission to perform operation 'change event'
  CCRC WAN Server Error: Must be one of : user associated with event:
    object owner, element owner, VOB owner, member of ClearCase group
</code>

So you can try and put labels through the ClearCase HomeBase gui, which seems more robust.
Comment: You're the man, thanks! Having this kind of feedback gives regular users some hope :-)


Classification: keeping false positive in training set

Question: I am working on a classifier, with a large number of possibles classes, and also a no class class.
My training set is made of the output of a hardcoded logic that is currently made for classification. Obviously, this hardcoded method is not perfect, hence the classifier I am working on now.
The output of the hard coded method has true and false output. My first thought was to only keep the true output (when the output was right), but I am wondering if there is a way to also keep the false output, so the model can train on it, knowing that it is not the right output.
If a class is ABC, I can create 2 classes ABC_true, and ABC_false, but how would the model know that we are talking about the same class, one for its true output, one for its false output?
Would encoding each possible class as dummies features for the model be a solution, and setting the dummy to one for each class that I am sure it is not the right outcome?
Thank you
Comment: If the classifier is hardcoded logic, how are you doing any training at all? It's also not clear by what you mean by "keep" the true/false output - the training set is what it is, regardless of how you classify it. Are you talking about some kind of boosting algorithm, where you iteratively reweight training examples so that misclassified samples are given more weight?
Comment: Thanks for your comment. I want to get rid of the hardcoded classifier, so I am developing a ML one. But I am going to use the output of the currently hardcoded classifier as a training set for the ML one.
Answer: Definitely don't do the last thing you mentioned, you don't want to add features that wouldn't be in real data which specifically state what class(es) a particular sample is (or isn't).  Your classifier will likely rely heavily on those and since they (presumably) won't be present when you're actually using the classifier for its intended purpose.
As for the creating two classes thing, the classifier wouldn't know that it's a TP/FP of the same class but I don't really see what the value in labeling like that would be anyway.  If you're gonna go through and check your training data for incorrectly labeled samples I think you should just change them to the correct class.  By making TP/FP labels for your classes you're just pushing the classifier to make the same mistakes that, if I understand correctly, you're replacing the current method for.
Comment: Thank you Shabuki, it makes sense. I am going to just keep the true classification then. I just wished that I could also hint the classifier on common mistakes (mistakes that are made by the hardcoded classifier now).
Comment: No, don't just keep the true classification - keep the ground-truth, i.e. keep the labels that the hard-coded classifier was compared to (as though the hard-coded classifier was never there at all).


Combine tornado gen.coroutine and joblib mem.cache decorators

Question: Imagine having a function, which handles a heavy computational job, that we wish to execute asynchronously in a Tornado application context. Moreover, we would like to lazily evaluate the function, by storing its results to the disk, and not rerunning the function twice for the same arguments.
Without caching the result (memoization) one would do the following:
def complex_computation(arguments):
    ...
    return result

@gen.coroutine
def complex_computation_caller(arguments):
    ...
    result = complex_computation(arguments)
    raise gen.Return(result)

Assume to achieve function memoization, we choose Memory class from joblib. By simply decorating the function with @mem.cache the function can easily be memoized:
@mem.cache
def complex_computation(arguments):
    ...
    return result

where mem can be something like mem = Memory(cachedir=get_cache_dir()).
Now consider combining the two, where we execute the computationally complex function on an executor:
class TaskRunner(object):
    def __init__(self, loop=None, number_of_workers=1):
        self.executor = futures.ThreadPoolExecutor(number_of_workers)
        self.loop = loop or IOLoop.instance()

    @run_on_executor
    def run(self, func, *args, **kwargs):
        return func(*args, **kwargs)

mem = Memory(cachedir=get_cache_dir())
_runner = TaskRunner(1)

@mem.cache
def complex_computation(arguments):
    ...
    return result

@gen.coroutine
def complex_computation_caller(arguments):
    result = yield _runner.run(complex_computation, arguments)
    ...
    raise gen.Return(result)

So the first question is whether the aforementioned approach is technically correct?
Now let's consider the following scenario:
@gen.coroutine
def first_coroutine(arguments):
    ...
    result = yield second_coroutine(arguments)
    raise gen.Return(result)

@gen.coroutine
def second_coroutine(arguments):
    ...
    result = yield third_coroutine(arguments)
    raise gen.Return(result)

The second question is how one can memoize second_coroutine? Is it correct to do something like:
@gen.coroutine
def first_coroutine(arguments):
    ...
    mem = Memory(cachedir=get_cache_dir())
    mem_second_coroutine = mem(second_coroutine)
    result = yield mem_second_coroutine(arguments)
    raise gen.Return(result)

@gen.coroutine
def second_coroutine(arguments):
    ...
    result = yield third_coroutine(arguments)
    raise gen.Return(result)

[UPDATE I] Caching and reusing a function result in Tornado discusses using functools.lru_cache or repoze.lru.lru_cache as a solution for second question.
Answer: The Future objects returned by Tornado coroutines are reusable, so it generally works to use in-memory caches such as functools.lru_cache, as explained in this question. Just be sure to put the caching decorator before @gen.coroutine.
On-disk caching (which seems to be implied by the cachedir argument to Memory) is trickier, since Future objects cannot generally be written to disk. Your TaskRunner example should work, but it's doing something fundamentally different from the others because complex_calculation is not a coroutine. Your last example will not work, because it's trying to put the Future object in the cache.
Instead, if you want to cache things with a decorator, you'll need a decorator that wraps the inner coroutine with a second coroutine. Something like this:
def cached_coroutine(f):
    @gen.coroutine
    def wrapped(*args):
        if args in cache:
            return cache[args]
        result = yield f(*args)
        cache[args] = f
        return result
    return wrapped

Comment: Thanks Ben. Yes. You're right. I already tested `functools.lru_cache` and it works like a charm. Regarding your example, I assume `cache` is something provided by outer context as you don't seem to be defining a anything inside the outer decorator closure. Nonetheless, your example is perfectly clear for me to grasp the idea. Cheers!


Prove $A+B= A \cup B$ if and only if $A \cap B = \emptyset$ using the definition of $A+B$

Question: Let $A$ and $B$ be sets. Define the symmetric difference of $A$ and $B$, written $A+B$, by $A+B=(A \cup B) \backslash (A \cap B)$.
Prove $A+B= A \cup B$ if and only if $A \cap B = \emptyset$.
My attempt:
Part 1b. If $A+B = A \cup B$, then $ A \cap B = \emptyset$
$A+B=(A \cup B) \backslash (A \cap B)$.
There shouldn't be any elements inside $A \cap B$.
$A+B=(A \cup B) \backslash (\emptyset)$
Therefore, $A \cap B = \emptyset$.
Edit: What if I put...
Definition: Let $(A \cup B)$ and $(A \cap B)$ be sets. The complement of $(A \cup B)$ relative to $(A \cap B)$, written $ (A \cup B)\backslash (A \cap B)$ is the set $(A \cup B) \backslash (A \cap B) = [x: x \in (A \cup B) \land x \notin (A \cap B)]$
While the elements are inside $(A \cup B)$, they don't belong in $(A \cap B)$
hmmmm they isn't any element in common...so it's an emptyset... wait a sec... emptyset means that there aren't any elements.
Part 2b. If $A \cap B = \emptyset$, then $A+B = A \cup B$
$A+B=(A \cup B) \backslash (\emptyset)$
Definition: Let $(A \cup B)$ and $\emptyset$ be sets. The complement of $(A \cup B)$ relative to $\emptyset$, written $ (A \cup B)\backslash \emptyset$ is the set $(A \cup B) \backslash \emptyset = [x: x \in (A \cup B) \land x \notin \emptyset]$
There are elements inside $A \cup B$, but not in $A \cap B$.
As a result, $A+B = (A \cup B)$
Did I do this correctly?
Comment: To prove sets $C,D$ are equal, you need to show that each element of $C$ is an element of $D$, and vice-versa.
Comment: huh? well... this is a biconditional statement right? I did split them up into two parts .
Comment: Yes, that's true, but within each part you are claiming that certain sets are equal to others.  For example, to prove that $(A\cup B)\setminus \emptyset=(A\cup B)$ you said some things, but you did not prove that the two sets are equal.
Comment: You need to add more details to part1, you didn't prove anything. As for part2, your idea is correct but you need to formalize/write it better.
Comment: hmm so how do I prove that they're equal? Like... $A+B=(A \cup B) \backslash (\emptyset)$.... use the definition? Definition: Let (A∪B) and ∅ be sets. The complement of (A∪B) relative to ∅, written (A∪B)∖∅ is the set (A∪B)∖∅=[x:x∈(A∪B)∧x∉∅]... there are no elements in $A \cap B$ since there are elements in the $A \cup B$
Comment: @dani_s I'm new at proof writing...and yeah I do need more details for part one. I just jumped to a conclusion. I should've included a definition.
Comment: @usukidoll note that $x \notin \emptyset$ is true by definition of empty set, therefore $x \in A \cup B \land x \notin \emptyset$ is equivalent to $x \in A \cup B$, i.e. $(A \cup B) \setminus \emptyset = A \cup B$
Comment: @dani_s I've edited my post and included some things in part one...but I just don't know how to conclude it... If x doesn't belong to $A \cap B$, then?!!!
Comment: @usukidoll try a proof by contradiction: suppose there exists $x \in A \cap B$, then prove that $x \notin A + B$ but $x \in A \cup B$, which would imply that $A + B$ and $A \cup B$ cannot be equal. (More technically you would be proving the contrapositive.)
Comment: alright so if there were elements inside $A \cap B$, then there can't be any elements at all in $A+B$, yet x belongs to $A \cup B$.. So.... hmmm....argh some statements are easier to prove than others ...
Answer: I will try to show you the result exploiting the definitions.
Define the symmetric difference of $A$ and $B$ (written $A+B$), by

$A+B = (A \cup B) \backslash (A \cap B)$.

We have that

$x \in A+B \quad$ iff $\quad ( x \in A \lor x \in B) \quad$ and $\quad \lnot (x \in A \land x \in B)$

But we have also that

$x \in A \cup B \quad$ iff $\quad ( x \in A \lor x \in B)$

Now we want that $A + B$ must be equal to $A \cup B$.
If we "compare" the two above condition, we can write them as

$P \land Q$ and $P$, and we want that $P \land Q \equiv P$.

By truth-table, we can check that $P \land Q \equiv P$ iff $Q$ is identically true.
So, we want that our condition $\lnot (x \in A \land x \in B)$ must be identically true.
This is simply :

$\forall x \lnot (x \in A \land x \in B)$.

By definition, $x \in A \cap B$ iff $x \in A \land x \in B$.
So, our condition is : $\forall x (x \notin A \cap B)$ and this, in turn, means that $A \cap B$ is $\emptyset$ (the empty set).
Comment: thanks. this is the approach that I wanted to go...not by contradiction because I got lost further than when I was trying to prove the actual statement.
Comment: You are welcome. Please, think at my suggestions : *(i)* use the definitions; *(ii)* progress step-by-step; *(iii)* for me, there is no magic in *proof by contradiction*: use it only when you do not have any other resource.
Answer: A + B := A\B U B\A. We will use this definition to prove the claim and it is equivalent to the definition given in the statement above.
===> If A + B = A U B. Then we show the intersection of A and B to be empty. If this is not true then there is a y in A and B. So y in A U B and therefore y in A + B. So that means y is in A\B or B\A. But neither of these can happen since y is in both sets A and B. Thus we obtain a contradiction. So the intersection of A and B is empty.
<=== If A intersects B is empty. We show A + B = A U B. We have A U B = A\B U B\A U C,
where C is the intersection of A and B. So if C = empty set, then A U B = A\B U B\A = A + B