Sorting by composite cohorts stored in different columns

Question: Suppose I have a data frame, and 3 of its columns contain parameters. Thus all possible cohorts are combinations: suppose the columns have "duck", "cat", "dog", another "white", "grey", "black", the last one "small", "big". Thus, cohorts are small grey duck, big black cat, and so on.
Thus, how can group the data frame by all possible cohorts stored therein? I will need this to calculate certain values for all cohorts. Thank you.
Comment: `DataFrame` has `groupby()`. Is `data_frame.groupby(column_list).(...)`, e. g. `data_frame.groupby(["animal", "colour", "size"]).max("age")`, not doing what you need?
Answer: Somehow you have to split your series into unique elements (whether those are lists or pandas.Series).
For example :
cols1 = set(df['col1'])
cols2 = set(df['col2'])
cols3 = set(df['col3'])
df = pd.DataFrame([[x,y,z] for x in cols1 for y in cols2 for z in cols3], columns=['col1', 'col2', 'col3'])
df.drop_duplicates(keep="first", inplace=True)

You could also do this using a loop (looping over df.columns.tolist()) and storing series values into a dict.
(This help you gather all the possible combinations of cohorts)
EDIT :
To group combinations in you first dataframe, you could do something like this :
df.groupby(['col1', 'col2', 'col3'])['col4'].mean()
See pandas.groupby's doc
EDIT 2 :
Another way to get all your combinations could be (using pandas only) :
from functools import reduce
cols = [df[[col]].drop_duplicates(keep='first') for col in ['col1', 'col2', 'col3']]
for sub_df_col in cols:
  sub_df_col['CARTESIAN_PRODUCT'] = 1 #you now have a new column which the same value everywhere
df2 = reduce(lambda  left,right: pd.merge(left,right,on=['CARTESIAN_PRODUCT'], how='outer'), cols).drop('CARTESIAN_PRODUCT', axis=1)

Credits to @everestial007 for the reduce/lambda solution
Comment: @Versteher When you say "all possible combinations/cohorts", do you mean "the theoretical extent/powerset" or do you mean "all combinations that can be observed in the actual dataframe"?
Comment: thank You. thus, I am supposed to loop myself a new data frame, where unique values from the columns are. But will I be able a.) find out all combinations of cohorts, as I need to know all possible combinations of values from all 3 columns? My cohorts need to be composite; b.) will I be able to calculate anything in my new data frame, I will need to calculate data in the original data frame.
Comment: This answer does not yet contain the grouping that OP asked for.
Comment: @valid : can't figure whats is asked about this grouping operation without any sample though... But you're right
Comment: @Versteher : just edited it, but I'm starting to suspect I did not understood your question at first... If groupby is the answer you were looking for, let me know and I will delete this post.
Comment: groupby cannot be a solution, in my understanding of it. The core word, for which I can't find an answer in cohort analysis resources in the net, is "composite". Suppose, I have a three columns: countries, gender and OS. Thus all possible cohorts are: "US male Windows", "France female Linux" etc. Afterwards I will need to calculate xcertain values for all of the cohorts. thus the question is in three steps:
Comment: 1.) what is combinatorics formula to calculate all cohorts. If there were 1 set, it would be 3!/(set - 3)!. but we have three separate sets (countries, gender, OS), that's why I call cohorts composite. 2.) how can review all (composite) cohorts that actually are given in the dataset? 3.) then how will I calculate values for eacxh cohort?
Comment: exempli gratia, `````list = [['US', 'male', 'Linux'], ['UK', 'female', 'Windows'], ['UAE', 'male', 'Android'], ['KSA', 'male', 'ios'], ['USA', 'female', 'Windows']] df = pd.DataFrame(list, columns = ['country', 'gender', 'OS']) ```` this DF has composite cohorts: each cohort consists og 3 elements
Comment: That's exactly what my first answer was : recreating a new dataframe with all combinations. If you want to compute any calculation afterwards, you may merge the new dataframe to the old one (beeing carefull to use different names for your variables : for example `df_new = df_combinations.merge(df_init, on=['col1', 'col2', 'col3'], how="left")`). Though any "groupby" applied to the final dataframe won't give you any result for combination not there at first...
Comment: in fact, from the point of maths, a solution of finding ALL possible combinations does indeed exists: if we have m values of one cohorts, n values of the other cohort, thus a number of all their possible combinations is m*n. This m*n has to be distributed over a value of entries in the table, thus a number of all possible composite cohorts is (multiplication of values of each cohort) to the power of the number of values of entries.
Comment: But this does not help much in practise, since we usually want to know not all possible combinations, but all ones given in a database. I suppose a pandas method should exist that would run through all columns, find all composite cohorts (combination of cohorts present in the database) and advise, which of them occur in the database
Comment: I disagree : pandas is just perfect the way it is, it's meant to work on matrix, not to shuffle each column independantly. If you want to find every combination, you have to work on each column on your own (using those as a vector, a list or a set, whatever). This is what I did in my first example using sets and list comprehension. I'm pretty sure this is efficient enought, but I'll add another way using only vectors, it may be easier for you to understand
Comment: Just added the solution working with pandas only. It will probably be faster on big datasets, even though I'm not sure of how much...


Define a rectangle with one edge of known length and whose two opposing corners are tangent to two circles of known size and location.

Question: Im trying to wrap a wire on two nails on a flat plane. I need to calculate the angle of a length of wire with respect to the x axis, given the thickness of two nails, and their locations, and different wire thicknesses.
To do that, I believe it requires defining a wire segment as rectangle with two opposing corners tangent to two known circles as in the picture below.
Comment: @KeithReynolds As amd has explained, the answer involves extending one of the $2$ circles so it's radius is $e + f$ instead. Then the line joining those $2$ circles are tangent to both circles. As such, the solution involves solving for this tangent line (from which you can get it's angle with the $x$-axis), which has already been done several times on this site. I should've seen this earlier myself.
Comment: For my part, I wouldn't mark this as a duplicate, although it does reduce to a previously-answered question.
Comment: Please draw a figure for us.
Comment: Please edit your post to state which part(s) of the wire are you trying to figure out an angle, and an angle with respect to what (e.g., the positive $x$-axis). Thanks.
Comment: @KeithReynolds Thanks for providing more details. I believe I've figured out your answer, but I first want to confirm a few things. The $2$ circles are the nails and the rectangle is the wire. The center of the upper circle is $(a,b)$ & the lower circle is $(c,d)$. The radii of both circles is $e$. The width of the wire is $f$. Finally, I'm assuming $a \lt c$ and $b \gt d$, although the calculations don't actually depend on this. Please let me know if any of this is incorrect. Thanks. P.S. I was planning on giving an exact answer, with amd's answer, I believe, being only approximate.
Comment: @JohnOmielan Yet another instance of how to compute common tangents to a pair of circles adds little to this site, IMO. How to solve that part of the problem has been explained many times already.
Comment: @JohnOmielan those assumptions are correct.
Comment: @JohnOmielan It is exactly the same problem as finding the common tangent to a pair of circles, just not to the original ones given.
Comment: @amd I feel a bit silly. Of course, it is with one of the circles being somewhat larger. Sorry about my oversight & thanks for explaining it.
Comment: Thanks to both JohnOmielan amd. I also feel silly, because increasing the radius of one of the circles escaped me as well. Despite solving involves determining the line tangent to two circles has been covered, this question as it is,  may be useful to others who also did not think of defining a new circle with radius e+f.
Answer: The simplest approach that comes to mind is to increase the radius of one of the circles by the wire thickness and then compute their common inside tangents. If you want to keep things symmetric, you could instead grow both circles by half the wire width. If you do the latter, then you're effectively working with the wire's centerline.
Questions about computing common tangents to two circles are pretty common here, so if you don't know how to do that, a simple search will turn up many methods. This question came up near the top of my search just now. If you use the symmetric approach I mentioned above, since the two circles have the same radius, their inside tangents pass through the midpoint of the centers, so you can reduce the problem to that of finding tangents to a single circle through that point.

P.S. Using the symmetric construction, the angle can be computed directly. Setting $D$ to be the distance between circle centers and measuring angles clockwise, we have for the angle $\alpha$ of the line joining the two centers $$\cos\alpha = \frac{c-a}D \\ \sin\alpha = \frac{b-d}D.$$ If we're wrapping clockwise around the top nail, we want to increase this by the angle $\beta$ between this line and the centerline of the wire, which, as noted above passes through the midpoint of the circle centers and is tangent to the circles with radius increased by $f/2$. We therefore have $$\sin\beta = {e+f/2\over D/2} = \frac{2e+f}D \\ \cos\beta = \sqrt{1-\sin^2\beta}.$$ Finding $\alpha+\beta$ then becomes a matter of applying any of the trigonometric sum-of-angles formulas and computing the inverse trig function. I don't have a good feel for whether this is going to be more or less work than computing the tangent line.


Problem with arrow position - range input

Question: I found an input range script on the Internet that meets my expectations.
I began to process it under myself. And everything works ok except the cloud position under the dot. I tried different ways but still the script does not work as it should. I would like the triangle from the cloud to be always under the dot regardless of the position.

$.fn.WBslider = function () {
    return this.each(function () {
        var $_this = $(this),
            $_date = $('input', $_this),
            $_title = $('.setyear', $_this),
            thumbwidth = 85, // set this to the pixel width of the thumb
            yrnow = 200;

        $_date.on('input change keyup', function () {
            var $_this = $(this),
                val = parseInt($_date.val(), 10);

            if (val < 70) {
                val = '< 70';
            }
            if (val === '') { // Stop IE8 displaying NaN
                val = 0;
            }

            $_title.text(val);

            var pos = (val - $_date.attr('min')) / ($_date.attr('max') - $_date.attr('min'));

            // position the title with the thumb
            var thumbCorrect = thumbwidth * (pos - 0.5) * -1,
                titlepos = Math.round((pos * $_date.width()) - thumbwidth / 4 + thumbCorrect);

            $_title.css({
                'left': titlepos
            });

            // show "progress" on the track
            pos = Math.round(pos * 99); // to hide stuff behide the thumb
            var grad = 'linear-gradient(90deg, #fb824f ' + pos + '%,#e2e5ec ' + (pos + 1) + '%)';
            $_date.css({
                'background': grad
            });

        }).on('focus', function () {
            if (isNaN($(this).val())) {
                $(this).val(0);
            }
        }).trigger('change');

        $(window).on('resize', function () {
            $_date.trigger('change');
        });
    });
};

$(function () {

    $('.slider').WBslider();

});
.startyear,
.endyear {
    float: left;
    color: #333;
    text-align: right;
    font-weight: bold;
}

.endyear {
    text-align: left;
}

.setyear {
    position: absolute;
    bottom: -65px;
    left: 50%;
    text-align: center;
    font-weight: bold;
    white-space: nowrap;

    min-width: 85px;
    box-shadow: 1px 3px 5px #d5d7db;
    border-radius: 5px;
    border: 1px solid #e4e7eb;
    background-color: #ffffff;
    padding: 10px 25px;

    color: #494e53;
    font-size: 12px;
    font-weight: 600;

}
.setyear:before {
        position: absolute;
        margin-top: -21px;
        content: URL("https://image.ibb.co/eSib69/range_arrow.png");
        background-color: transparent;
}

.range {
    position: relative;
    float: left;
    padding: 0 0.9375rem;
}

input[type=range] {
    -webkit-appearance: none;
    display: block;
    height: 4px;
    padding: 0;
    background: #FFE014;
    box-sizing: content-box;
    border-right: 0 !important;
}

input[type=range]:focus {
    outline: none;
}

input[type=range]::-webkit-slider-thumb {
    -webkit-appearance: none;
    width: 20px;
    height: 20px;
    border: 7px solid #fff;
    border-radius: 25px;
    background: #fb824f;
    box-shadow: 0 2px 9px rgba(185, 185, 185, 0.75);
}

<form>

    max. 250 kg

      <input type="range" name="date" id="date1" min="70" max="200" step="1" value="Please enter the year of build" required>

    max. 250 kg

</form>
Answer: I tried to find out the position of type range via linear gradient. It might not be accurate but to an extent, it does the work.
$.fn.WBslider = function() {
    return this.each(function() {
        var $_this = $(this),
            $_date = $('input', $_this),
            $_title = $('.setyear', $_this),
            thumbwidth = 85, // set this to the pixel width of the thumb
            yrnow = 200;

        $_date.on('input change keyup', function() {
            var $_this = $(this),
                val = parseInt($_date.val(), 10);

            if (val < 70) {
                val = '< 70';
            }
            if (val === '') { // Stop IE8 displaying NaN
                val = 0;
            }

            $_title.text(val);

            var pos = (val - $_date.attr('min')) / ($_date.attr('max') - $_date.attr('min'));

            // position the title with the thumb
            var thumbCorrect = thumbwidth * (pos - 0.5) * -1,
                titlepos = Math.round((pos * $_date.width()) - thumbwidth / 4 + thumbCorrect);

            if ($("#date1").attr("style") != undefined) {
                titlepos = 23 + parseFloat($("#date1").attr("style").split(" ").pop().split("%")[0]);
            } else {
                titlepos = 70;
            }
            $_title.css({
                'left': titlepos
            });

            // show "progress" on the track
            pos = Math.round(pos * 99); // to hide stuff behide the thumb
            var grad = 'linear-gradient(90deg, #fb824f ' + pos + '%,#e2e5ec ' + (pos + 1) + '%)';
            $_date.css({
                'background': grad
            });

        }).on('focus', function() {
            if (isNaN($(this).val())) {
                $(this).val(0);
            }
        }).trigger('change');

        $(window).on('resize', function() {
            $_date.trigger('change');
        });
    });
};

$(function() {

    $('.slider').WBslider();

});

https://jsfiddle.net/4Lqmw1sz/4/
Comment: Thanks but i need a solution like this http://jsfiddle.net/paciek/xpvt214o/877753/ only in clear jQuery


recursively $x_n$ by the equation $x_{n+1} = x_n^2 + x_n -3$ for $0≤n$,

Question: Consider the succession of real numbers defined recursively $x_n$ by the equation $x_{n+1} = x_n^2 + x_n -3$ for $0≤n$, Determine the initial values  $x_0$for which you get a constant succession $x_n$.
I really stuck with this problem of sequences some help to solve this please!! ...
Answer: If a solution is constant, then $x_n = x_{n+1} = x_{n+2} =  \ldots$ , so we can set $x_n = x_{n+1} = x$ in the recurrence relation obtaining:
$$ x = x^2 + x - 3$$
Hence $x^2 = 3 = x_0^2$, so $x_0 = \sqrt{3}$ or $x_0 = -\sqrt{3}$. Plugging both into the recurrence relation shows that they indeed lead to constant solutions.
Answer: Hint: if $x_0^2=3$, then
$$x_{1} = x_0^2 + x_0 -3=x_0.$$


Mac Management Without Permission and Security

Question: I was going through some literature on managing OS X laptops and asked someone some questions about usage scenarios when using the MacBooks. I asked someone more knowledgeable than I about whether it was possible for my Mac to be taken over if I were visiting another site for a conference or if I went on a wifi network at a local coffee house with policies from an OS X Server with workgroup manager (either legit for the site or someone running a version of OS X Server on hardware they have hidden somewhere on the network), which apparently could be set up to do things like limit my access to Finder or impose other neat whiz-bang management features.
He said that it is indeed possible for it to happen as it would be assigned via the DHCP server and the OS X server would assume my Mac is a guest and could hand out restrictions and apparently my Mac will happily accept them without notifying me or giving me an option, unlike Windows which I believe would need to be joined to a domain before it becomes "managed" by Active Directory.
So my question is as network admins and sysadmins with users traveling with MacBooks, is there a way to reasonably protect your users from having their machines hijacked without resorting to just turning off networking all the time? Or isn't this much of a security hazard? What threat does this pose to the road warriors in your businesses?
Answer: Your friend is potentially correct:
If your Mac is set to bind to get its directory search path automatically then it can bind to a DHCP supplied LDAP server.  The danger here is that root is root, the MacBook doesn't realise that it's not the root account from its "home" LDAP directory and happily treats it as if it were.
If you set the directory search path to custom then it only binds to LDAP servers you have specified and totally avoids the problem.
Also as you mention, if you have no need of LDAP - for instance if you authenticate against a Windows AD domain or just have a local account you can uncheck the LDAP v3 box.
See chapter 2 of the OpenDirectory Admin guide for more info.
Comment: So I can prevent it from happening by using the directory utility to get rid of binding to any LDAP directory that's not the local MacBook?
Comment: Which is...duh...what you just said. I just have to figure out how to point it just to the local system's LDAP directory...I'll have to search the link you supplied...
Comment: I set the directory utility to search local in the search policy and now only have "BSD Flat File and NIS" and "Local" set in Services, unchecking LDAP3 and Active Directory. Hopefully that'll mitigate the threat of having control unknowingly stolen.
Comment: Yes, just having local - which you can't get rid of anyway - will protect you.  Of course it only works if you don't need to bind to a LDAP server.
Comment: Hopefully that mitigates the management-without-permission thing. I'd hate to have to troubleshoot my own laptop and finding out it's because of "stuck" management settings after being on another network during a conference or hotel stay. And hwo kn


Sum of Cauchy distributed random variables

Question: Problem: Let $X_1, X_2, \ldots $ be independent $C(0,1)$ and set $S_n = \sum_{k=1}^n X_k$. Show that $\frac{1}{n}\sum_{k=1}^n \frac{S_k}{k}\sim C(0,1)$.

Using the characteristic function it is easy to get that $\frac{S_k}{k}$ is $C(0,1)$. But $Y_k=\frac{S_k}{k}$ are not independent for different $k$ so that cannot be applied directly in this case.
Comment: Hint: Try re-arranging the sum $$\begin{align} &X_1\\+&\frac{1}{2}(X_1+X_2)\\+&\frac{1}{3}(X_1+X_2+X_3)\\+&\cdots\end{align}$$ into a weighted sum of independent $X_i$'s.
Comment: OP: What happened to this question?
Comment: Sorry that I didn't follow up. I followed Dilip's advice and it worked out well.
Answer: Let $T_n=\frac1n\sum\limits_{k=1}^n\frac1kS_k$, then $T_n=\sum\limits_{k=1}^nt_{k,n}X_k$ for some coefficients $(t_{k,n})_{1\leqslant k\leqslant n}$ summing to $1$, whose values are irrelevant. Now, $u$ times a centered Cauchy random variable with parameter $v$ is a centered Cauchy random variable with parameter $uv$, and the sum of independent centered Cauchy random variables with parameters $u$ and $v$ is centered Cauchy with parameter $u+v$.
Hence, if the $(X_k)_k$ are independent and each $X_k$ is centered Cauchy with parameter $x_k$, then $T_n$ is centered Cauchy with parameter $t_n=\sum\limits_{k=1}^nt_{k,n}x_{k}$. In the present case, $x_k=1$ for every $k\geqslant1$ hence $t_n=1$ for every $n\geqslant1$.


htaccess for Angular app in a subfolder

Question: I have an Angular 6 app living in a subfolder on a D8 installation. I have no issues navigating to the app and the angular routing doing it's thing as I navigate the app. My issues is I get a Drupal 404 if I try to hit the sub routes directly in the address bar.
What do I need to add to my htaccess that will let Drupal ignore the folder and the angular routing still work?
Drupal: localhost:8888
Angular: localhost:8888/my-app
Things I've tried:
Adding: RewriteCond %{REQUEST_URI} !^/my-app/.*$ <- I get Not Found
Changed DirectoryIndex index.php index.html index.htm to DirectoryIndex index.php
Adding the following to httpd.conf:
<Location "/my-app">
 FallbackResource /my-app/index.html
</Location>
<Directory /my-app>
 FallbackResource disabled
</Directory>

Answer: You need to add the following rewrite in your Drupal .htaccess
RewriteCond %{REQUEST_URI} !/APP-FOLDER-NAME/.*

Then in the root of the angular application folder you need the following in your .htaccess
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^.*$ - [NC,L]

RewriteRule ^(.*) /APP-FOLDER-NAME/index.html [NC,L]

htaccess for Angular app in a subfolder


Maximum unique pairings between two lists of repeated items

Question: I'm trying to find a general approach to determining how many unique pairs I can form given two lists of items. The elements of each list are different. Within each list, items can be repeated. For example,
list 1     list 2
======     ======
1          A
2          A
2          A
3          A
3          A
3          B
4          B
5          B

Because of repetition, in this case there are only 7 unique pairs possible. ([3,A] must appear more than once).
If we pair the items in the order they are given, we get 6 unique pairs:
(1,A), (2,A), (3,A), (3,B), (4,B), (5,B) are unique [(2,A) and (3,A) would appear twice]

However, if we pair differently, we can get one more unique pair.
(1,A), (2,A), (2,B), (3,A), (3,B), (4,B), (5,A) are unique [(3,A) would appear twice]

Order doesn't matter (i.e., pair 1,A is the same as A,1).
Is there a general approach to computing the unique possible matchings? Any leads are appreciated.
Comment: Is a pair of elements from the same list allowed?
Comment: No, the pairs have to use exactly one item from list 1 and one item from list 2
Comment: What about the order? For example, are $(1,A)$ and $(A,1)$ considered two different pairs?
Comment: No, those would be the same pair. Order doesn't matter. Thanks.
Comment: Your example isn't clear to me. I see $5$ different elements on the first list and $2$ different elements on the second list, hence the total number of different pairs is $5\cdot2=10$. How did you get to $8$ or $7$???
Comment: And what do you mean by "$(3,A)$ must appear more than once"??? It doesn't work quite well with your question of "how many unique pairs I can make"???
Comment: Sorry, my phrasing may not be clear. I'm not talking about possible pairs, but rather actual pairings. Basically, given those lists, I want to how they can be matched to give the most unique pairs. Let me edit the question.
Comment: @barakmanos *either* 3-A *or* 3-B must appear more than once, anyway - pequod is that the meaning? So that no more than 7 distinct pairings appear in any pairing attempt, and maybe only 5 or 6?
Comment: @Joffan, yes that's right. Edited to clarify.
Answer: After looking at various research papers I am fairly comfortable with using the greedy algorithm outlined below, suspecting it always produces a maximum pairing.  In any case there is a quick check to determine when any pairing is a maximum (in linear time in number of vertices).  If this check finds a pairing is not a maximum, then it provides a way to construct a larger pairing.
Greedy algorithm
Take the list with the fewest distinct values (the 2nd list in the example) and pair the highest frequency value there (in this case A) with as many distinct values from the other list as possible.
In this case that produces five distinct pairs:
(1,A), (2,A), (3,A), (4,A), (5,A)

leaving the reduced problem of determining how many distinct matches we can make with B from the second list, and the remaining two distinct values in the first list:
(2,B), (3,B)

One way to think about this is the maximum number of edges in a labelled bipartite graph with (vertex) degree constraints, e.g.
1(1)
2(2)        A(5)
3(3)
4(1)        B(3)
5(1)

where the maximum allowed degrees are shown in parentheses next to the vertex "labels".
In the literature this type of problem is called the "degree constrained subgraph" problem as well as the "maximum cardinality b-matching" problem.  Here $b$ denotes a function on the vertices of a graph $G$, $b: V \to \mathbb{N}$, that gives an upper bound on the degree $\delta(v)$ that vertex would have in a subgraph (subset $M$ of edges of $G$).  Note that if $b\equiv 1$ at all vertices, this notion is simply that of a matching on $G$.
If $G=(X,Y;X\times Y)$ is a complete bipartite graph (as is the case here; any item on one list can be paired with any item on the second list), the problem of determining if the specified degree sequences $\delta(v) = b(v)$ can be attained in a subgraph of $G$ is called the bipartite realization problem, for which a polynomial time algorithm exists by consequence of the Gale-Ryser theorem.  When it succeeds in attaining all the degree bounds exactly, this is essentially the greedy algorithm above.
Formulation as a maximum network flow
The Ford-Fulkerson approach solves the bipartite realization problem, but also the more general problem posed here, of finding the maximum number of pairings (b-matching).
Given the two sets $X$ and $Y$ of distinct values in lists 1 and 2, respectively, define the complete bipartite graph $X\times Y$ and two additional nodes, source $s$ and sink $t$.  If each edge in the complete bipartite graph $X\times Y$ is given capacity 1, and if edges from $s$ to each vertex in $X$ are given capacity equal to the frequency it appears in the first list (degree constraint) and similarly edges from each vertex in $Y$ to $t$ given capacity equal to the frequency in the second list, then the maximum network flow from source to sink will be equal to the maximum cardinality of a b-matching in $G$.
I realize a picture will be helpful here:

This formulation works when $G$ is any bipartite graph, not necessarily complete.  The contribution of Ford and Fulkerson is to find a more efficient algorithm than merely solving the network flow by linear programming (i.e. maximizing flow out of source = flow into sink, subject to the sums of in/out flows cancelling at all internal nodes).
This more efficient algorithm uses the concept of augmenting paths, a path that alternates between edges of $G$ that are included in a current matching and edges that are not.  A b-matching is a maximum if and only if there is no augmenting path, but if there is an augmenting path, this can be used to increase the size of the b-matching.  The search for an augmenting path can be done in time linear to the number of vertices of $G$.
A fairly recent paper in this area is An $O(n^3)$ time algorithm for the maximum weight b-matching problem on bipartite graphs (Oct. 2014).
For a more historical introduction I recommend Edmonds, Matching, and the Birth of Polyhedral Combinatorics by William R. Pulleyblank, though it quickly moves beyond the treatment of bipartite graphs to matchings in general graphs by Jack Edmonds "blossom algorithm".
Comment: This is helpful, but I'm having trouble seeing how I would go from the picture (or the labeled graph you have in the table) to then compute the number of unique pairs? Can you say a little more?
Comment: I wasn't sure what your background with such graph algorithms might be, but I included some links to background material that is "classic" for this area.  Let me first try to find a case where the greedy algorithm fails, because Ford-Fulkerson, while polynomial, is more complicated to implement.
Comment: Awesome answer. Thanks so much. I've got an implementation that allows me to calculate max flow and so far it seems to be working! Need to do some more reading of the references you gave really wrap my head around what's happening.
Comment: Glad to be of service.  I'm going to add more about the Gale-Ryser thm. on bipartite realization and the degree constrained subgraph problem.  Just finished small fixes to the Wikipedia article on that.
Comment: Also, in the image, shouldn't there be two additional edges between the green and blue nodes? The middle section is not complete. The second from the bottom and second from the top, of the green nodes, only have one connection to blue?
Comment: Yes thanks; there should be. I used a Google code in-browser app to dash that off, and there are a couple of additions I should make besides fixing those two missing edges.
Comment: Cool! Thanks, just wanted to make sure I was following.


Select no NODE retorna vazio

Question: Quando executo essa query:
SELECT xml_nfe FROM NDD_COLD.COLD_PROD WHERE ide_id ='NFe42200602831172001104550000010390091233318028'

No plsql ele me retorna uma linha com informações, ai passo a mesma query no nodejs dessa forma:
const dbConfig = require("./conexaoDBoracle.js");
const oracledb = require("oracledb");

class ConsultaXML {
async consultaXML() {

let connection;

try {

  connection = await oracledb.getConnection(dbConfig)

  let result = connection.execute(`SELECT xml_nfe FROM NDD_COLD.COLD_PROD WHERE ide_id ='NFe42200602831172001104550000010390091233318028'`);

  console.log((await result))
  return (await result).rows;

} catch (err) {

  console.error(err);
}
}
}
module.exports = new ConsultaXML;

e ele me retorn isto:
{ metaData: [ { name: 'XML_NFE' } ], rows: [] }

Alguem tem ideia do que pode ser?
Comment: Voce chegou a tentar fazer essa query usando o `bind` pro `ide_id`?
Comment: apos a minha conexão adicionei o seguinte:
oracledb.fetchAsString = [oracledb.CLOB]

e o retorno do método assim:

const clob = (await result).rows[0][0];
return clob

Por causa do tamanho do retorno da query, conforme manual da oracle:

https://blogs.oracle.com/opal/node-oracledb-112:-working-with-lobs-as-string-and-buffer-connection-pinging
Answer: Resolvido, devido ao retorno da query ser muito extenso precisei fazer o seguinte
const dbConfig = require("./conexaoDBoracle.js");
const oracledb = require("oracledb");

class ConsultaXML {

async consultaXML(chave) {

let connection;

try {

  connection = await oracledb.getConnection(dbConfig)

  oracledb.fetchAsString = [oracledb.CLOB]

  let result = connection.execute(`SELECT XML_NFE AS XML
  FROM NDD_COLD.COLD_ENTR
  WHERE IDE_ID = '${chave}'

  UNION ALL

  SELECT XML_NFE AS XML
  FROM NDD_COLD.COLD_PROD
  WHERE IDE_ID = '${chave}'`);

  const clob = (await result).rows[0][0];

  return clob

} catch (err) {

  console.error(err);
}
}
}
module.exports = new ConsultaXML;

Select no NODE retorna vazio


How can I go to a certain view when the app is opened from push notification?

Question: How can I go to a certain view when the app is opened from push notification?  I am using storyboards.  I have some views that push and some that are modal.  I need to go to a certain view from anywhere in the app.  This view is pushed by a view that is normally only access from a settings type page that pushes this view.
Answer: You can add extra parameters into Notification Payload. In the Push Notification Programming Guide section The Notification Payload it states

Providers can specify custom payload values outside the Apple-reserved aps namespace.   Custom values must use the JSON structured and primitive types: dictionary (object), array, string, number, and Boolean. You should not include customer information as custom payload data. Instead, use it for such purposes as setting context (for the user interface) or internal metrics. For example, a custom payload value might be a conversation identifier for use by an instant-message client application or a timestamp identifying when the provider sent the notification. Any action associated with an alert message should not be destructive—for example, deleting data on the device.`

You can read this parameters in AppDelegate function application:willFinishLaunchingWithOptions:
There you can re-create UINavigationController instance.
Answer: If your app is running in background.In that case.
When app is opened from push notification below method call.
<code>-(void) application:(UIApplication *)application didReceiveRemoteNotification:(NSDictionary *)userInfo
</code>
Using userInfo you can set your way which view to push.
Suppose I wan't to go firstView that is depend on pushNotification In that condition I will add a value in APNS payload like view:firstView(that may be your class name) and using this value from userinfo(NSDictionary) you can navigate or push to particular view.
In that condition your APNS payload will be like this
{"aps":{"alert":"Your message","sound":"value","badge":1},"ViewName":"value"}
Comment: Sweet idea, passing in the name of the View I want.  Thanks!!


Why the trees generated via bagging are identically distributed?

Question: I have problem in intuitive understanding of following arguement:
"The trees generated via bagging are identically distributed, thus the expectation of the average of a set of trees is the same as the Expectation of a single tree"
1) What does really distribution of tree mean?
2)  Why, when there are generated via bagging are ID?
Answer: I think rapaio is conflating a couple key concepts and in doing so misinterpreted the OP's question.  Yes, the bootstrap samples utilized within a bagging algorithm are IID.  However, the bagging estimator is ID, NOT IID.
The bagging algorithm will generate B trees and the corresponding prediction estimates, $\{\hat{f}^b(X)\}_{b=1}^B$.  Since the tree estimator estimated each tree using draws from the same distribution, the identical distribution assumption will hold.  However, the independence assumption will not hold!!!  For example, imagine that there is one very strong predictor within the data.  In each tree this strong predictor will likely be the first split.  Therefore, the prediction of most trees will be similar.  Said another way, the predictions will be correlated (i.e. not independent).
Think about it, the bagging algorithm is taking a sequence of IID random variables (i.e. the bootstrap samples) and turning them into a sequence of ID random variables (by generating tree estimates)
The bagging algorithm is still helpful.  The bagging estimator is unbiased; bias is unaffected the lack of independence.  Therefore the average of $\hat{f}^b(X)$ will be the same as the expected value of any tree, i.e. $E(f^b(X)) = \frac{1}{B} \sum_{i=1}^B \hat{f}^b(X)$.  The variance of the bagging estimator will, however, be affected by non-independence i.e. remember $Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y)$.  It turns out that bagging estimator will have a smaller variance then a tree estimator (see pg 518 Elements of Statistical Learning).  However, we can further reduce estimator variance by attempting to decorrelate the trees.  This is where the notion of Random Forest comes from.  Again see pg 518 Elements of Statistical Learning or pg 319 Introduction to Statistical Learning for more.
Answer: Bagging technique uses bootstraps (random samples of the same length with replacement) to train each tree from the assembly. Thus, the samples used to build each individual tree comes from the same population as the original sample. This is why the input and target variables are called ID (identically distributed = same distribution).
More than that, because the samples are drawn randomly, the samples are also independent (knowing elements of a sample does not give hints on the elements of another sample). This is usually denoted as IID (independent and identically distributed).
The expectation of the mean is preserved because input and target variables are IID (samples are independent and are drawn from the same population). [see Law of Large numbers]
Because trees are basically a piece-wise constant approximation, what those trees can learn are constant averages on various regions. The trees only define input space regions (the leaf nodes), but on those regions approximates with an average.
Those constants are averages of some sort (mean, median) depending on the loss function. So what you can say about averages from the input and target variables, you can say about trees themselves (that they preserve the expectation of the average).
The bagging is used to reduce variance by averaging the models, while they preserve as much as possible the expectation of those variables.
I hope I was clear somehow, I will retry later when I will have the chance, to improve this, eventually.
Comment: "More than that, because the samples are drawn randomly, the samples are also independent (knowing elements of a sample does not give hints on the elements of another sample)"  -- I don't think this is entierly correct. The independency comes from the "drawn randomly" AND "with replacement." If you only drawn randomly, but without replacement, then you can't achieve independency.


how to subtract rows in a data frame in R depending on difference between values?

Question: I have a data frame that is in the following format
df <- data.frame(name=LETTERS[1:5], location=c(2000,2021,4532,1931,3457),
                 value=c(1,0,1,1,0))

name    location   value
A       2000       1
B       2021       0
C       4532       1
D       1931       1
E       3457       0

There are approximately a million rows in the data frame. How would I create a new dataframe that has the distance between every location if the locations are within 1000 of each other also checks to see if the values are both one for both locations?
For the above dataset, the dataframe would only have three rows in it with values of 21 (absolute value of 2000 - 2021), 69 (absolute value of 2000 - 1931), and 90 (abs. value of 2021-1931) because those are the only differences that are less than 1000. It would also have a column of 0 (because A and B values are not 1 and 1), 1 (because A and C values are 1 and 1), and 0 (because B and C are not 1 and 1). So it would look like:
21   0
69   1
90   0

I've tried using loops but since there are so many rows, it's inefficient. Is there some built in function that I should use to do this faster?
Thanks in advance.
Comment: @smci Not if there are many (>2) locations close to each-other. That actually happens in the example data.
Comment: I'm surprised that you don't seem to want any labels in the output. Do you not care to know the names corresponding to the rows in your output?
Comment: First, it seems obvious to sort (order) the dataframe by increasing `location`. Then the distance column is simply `diff(location)`. Then you can do the rest using a sliding-window of +/- 1000 on `location` column.
Comment: @Renu: it does make it simpler because we can use simple logical indexing: to create a window of +/- 1000 on `location`, excluding given row. Then we could use sapply/ dplyr / data.table aggregation over that window. I never said the window would contain only one member.
Comment: @smci I'm skeptical you'll get standard grouping tools to work well. If you have locations 2000, 2050, and 2120, you don't have a mutually exclusive grouper. Maybe `rollapply` has something for this? If I were to do an R solution I would emulate renu's answer with non-equi joins in data.table.
Comment: By the way, in your example you only want consecutive locations. So DA,AB,EC but not DB,DE,AE. But you said *"the distance between every location if the locations are within 1000 "*. So which is it?
Answer: library(sqldf)
sqldf("
select  a.location
        , b.location
        , a.location - b.location as locdiff
        , a.value*b.value as value
from    df a
        inner join df b
          on a.location - b.location between 1 and 1000
")

This gives
  a.location b.location locdiff value
1       2000       1931      69     1
2       2021       2000      21     0
3       2021       1931      90     0

Or with data.table. This is just @MKR's solution but adding a column to avoid a large join result. Not sure if it's possible to achieve this without creating a new column.
setDT(df)

df[, loc2 :=  location - 1000]

df[df
  , .( locdiff   = i.location - x.location
     , locationA = i.location
     , locationB = x.location
     , value     = x.value*i.value)
  , on = .(location >= loc2
          , location < location)
  , nomatch = 0]

gives
   locdiff locationA locationB value
1:      69      2000      1931     1
2:      90      2021      1931     0
3:      21      2021      2000     0

Comment: @Gregor Agree with comment that `sqldf` could be better option and its a great answer. In the meant time I worked on a `data.table` based solution in which I tried to minimize cartesian join by joining on `x.location > i.location`.
Comment: Great answer. SQL is much better for something like this than doing a million x million join in R and filtering down.
Answer: I agree with @Gregor comment where he mentioned sqldf to be better option to in above scenario in the sense that it avoid cartesian join of million records.
But I tried to optimize data.table based solution by first joining on x.location > i.location and then filtering on diff <=1000.
df <- data.frame(name=LETTERS[1:5], location=c(2000,2021,4532,1931,3457),
                 value=c(1,0,1,1,0))
library(data.table)
setDT(df)

df[df,.(name, diff = x.location - i.location, value = x.value*i.value),
         on=.(location > location), nomatch=0][diff<=1000]
#    name diff value
# 1:    B   21     0
# 2:    A   69     1
# 3:    B   90     0

Comment: If you add a column `df[, loc_max := location + 1000]` you can add `location < loc_max` to the `on` criteria instead of filtering post-hoc. I'm not sure quite how the internals work, but there's at least a good chance it will be more memory efficient.
Comment: @Arun mentions briefly the method `data.table` uses for joins ~10 mins into this video https://channel9.msdn.com/Events/useR-international-R-User-conference/useR2016/Efficient-in-memory-non-equi-joins-using-datatable
Comment: Thanks much for your comments. Let me perform some benchmark on these options. That can give us some useful results.
Comment: @Renu I think `non-equii` join would be efficient based on details from Arun in his video. Equii join could be faster but in above case there will be overhead to create a column.


Generalized Linear Model: Why do we insist on modeling the mean?

Question: A standard description of GLM is said to have three elements:

The linear predictor $\eta=X \beta$
The link function $g(.)$ that relates the linear predictor to the mean of the outcome variable $\mu=g^{-1}(\eta)=g^{-1}(X\beta)$
A distribution of the outcome $y$ with its mean $\mathbb E(y|X)=\mu$

See Wikipedia and Gelman et al.'s Bayesian Data Analysis 3rd edition for some examples.
My question is why is the mean of the distribution so special. Instead of modeling $\mathbb E(Y|X)=g^{-1}(X \beta)$, why can't we model other parameters of the distribution? That is $\text{(other parameters)}=g^{-1}(X \beta)$?
For example, the parameters of the Weibull distribution do not correspond to its mean, but I see models that look like this:
$$
\begin{aligned}
y & \sim Wei(v, \lambda)  \\
\ln(\lambda) &= X \beta
\end{aligned}
$$
As another example, the mixture density network models the parameters of a mixture of Gaussians.
Nevertheless, every book I read about GLM says that we model the mean $\mathbb E(Y|X)=g^{-1}(X \beta)$. It makes me wonder that I am missing something important.
Comment: You can fit other parameters directly; that is just not a generalized linear model.
Comment: Think about semiparametric models such as the proportional odds model which model whole distributions and things like odds ratios on P(Y > y | X).  From the fitted whole distribution you can estimate mean, median, other quantiles, variance, and just about anything else you want (covariate-specific).  So there are many ways to not "start with the mean".
Comment: For alternatives to (or extensions of) glm's where you can model also ther parameters, see for instance [tag:gamlss],   https://cran.r-project.org/web/packages/brms/vignettes/brms_distreg.html
Comment: I believe there are special properties of models which model the expected value of an exponential dispersion family with fixed dispersion instead of vice versa. It would be interesting to see an answer get into that.
Comment: I think I get it now - the model can be called GLM only when we model the mean with $g^{-1}(X\beta)$ but it is still a valid model if we model other parameters of the distribution. But is this distinction for historical reasons or are there any special properties for GLM?
Answer: Generalized linear models are generalization of linear regression model. Linear regression minimizes squared error for fitting the regression line. Minimizing squared error is equivalent to maximizing Gaussian likelihood. GLMs generalize this idea to other distributions, so we are estimating conditional means of other distributions.
So it's a little bit like asking "what's the fuss with bikes having two wheels", well they're kind of vehicles that do. As others noticed in the comments, there are many other models that not necessarily focus on means. GLMs are just a family of models that do.


Assigning a mysql field value to a PHP variable

Question: I am trying to assign the value of the my below database field (sname) to a php variable (name).but I am getting below error.Can someone help
Parse error: syntax error, unexpected T_ENCAPSED_AND_WHITESPACE, expecting T_STRING or   T_VARIABLE or T_NUM_STRING in /home/a4363282/public_html/pune/upload/upload_file.php on line 6

include './connection.php';
$query = "select * from shops WHERE city='pune' AND Ref=1;
$row = mysql_query($query);
$name=$row['sname'];
echo $name;

Comment: The error is causing by a missing closing `"` in `$query = "select * from shops WHERE city='pune' AND Ref=1;` which should read as `$query = "select * from shops WHERE city='pune' AND Ref=1";`
Answer: You are missing double quote at the end of query, Update this,
include './connection.php';
$query = "select * from shops WHERE city='pune' AND Ref=1";
$row = mysql_query($query);
$name=$row['sname'];
echo $name;

Comment: minus voter be aware explain reason?
Comment: i have balanced the vote score because the error in question is relates to syntax error which you have defined ,don't know why people dv without leaving a comment
Comment: @user1153551 Just run your code, then you'll know the reason for the minus.
Comment: thanks everyone.Apologies I missed the double quote here.
Answer: Please update you're code to MySQLi or PDO.
<?php
include './connection.php';
$query = "select * from shops WHERE city='pune' AND Ref=1";
$result = mysql_query($query);
while($row = mysql_fetch_assoc($result))
{
  echo $row['sname'];
}
?>

Answer: Avoid using mysql_* statements. Change your code with -
$mysqli = new mysqli("localhost", "my_user", "my_password", "world");

if (mysqli_connect_errno()) {
    printf("Connect failed: %s\n", mysqli_connect_error());
    exit();
}

$query = "SELECT * FROM shops WHERE city=? AND Ref=?";

if ($stmt = $mysqli->prepare($query)) {

    $stmt->bind_param("si", 'pune', 1);

    $stmt->execute();

    $stmt->bind_result($col1, $col2);

    $array = array();
    while ($fetch = $stmt->fetch()) {
        $array[] = $fetch;
    }
    print_r($array);

    $stmt->close();
}

$mysqli->close();

Answer: I don't know what are you trying to do with this code because it's wrong
Attempting to print $result won't allow access to information in the resource
$row = mysql_query($query);

here the $row is resource
so, One of the mysql result functions must be used. so your code would be
include './connection.php';
$query = "select * from shops WHERE city='pune' AND Ref='1'";
$row = mysql_query($query);

 while($ansArray = mysql_fetch_assoc($row)){

   echo $name = $ansArray['sname'];
 }


How to prove: Every affine set can be expressed as the solution set of a system of linear equations

Question: How can I prove that every affine set can be expressed as the solution set of a system of linear equations?
Please note that set $C  \subseteq \textbf{R}^n$ is said to be affine if for any $x_1 , x_2 \in C$ and $\theta \in \textbf{R}$, we have $\theta x_1 + (1-\theta)x_2 \in C$.
Comment: What is your definition of an affine set?
Answer: Fix an element $x_0 \in C$.
Claim: The set $K = \{x-x_0: x \in C\}$ is a linear subspace of $\Bbb R^n$.
Claim: There exists a linear map $T$ whose kernel is precisely $K$.
Claim: $C$ is the solution to the linear system $Tx = Tx_0$ on $x$.
Comment: thanks @omnomnomnom. But I really don't know how to prove the second section "There exists a linear map $T$ whose kernel is precisely $K$." ?!
Comment: It depends what you can assume.  Worst case, here's the full outline: begin with a basis $v_1,\dots,v_k$ of $K$, extend this to a basis $v_1,\dots,v_n$ of $\Bbb R^n$. We can define the map $T$ by
$$
T(a_1 v_1 + \cdots + a_n v_n) = a_{k+1}v_{k+1} + \cdots + a_{n}v_n
$$


Difficulties in grasping the proof of the Heine-Borel theorem

Question: I'm finding it difficult to grasp a few nuances of the proof of the Heine-Borel theorem as laid out in my textbook. Here is the passage that is giving me problems:
"Suppose a set S is closed and bounded. Let F be an open cover of S. For each x∈R, let Sx=S∩(∞,x] and let B={x: Sx is covered by a finite subcover of F}. Since S is closed and bounded, S has a minimum (say, d). Then Sd={d} and this is certainly covered by a finite subcover of F. Thus, d∈B and B is nonempty. If we can show that B is not bounded above, then it will contain a number z greater that the supremum of S. But then Sz=S, and since z∈B, we can conclude that S is compact."
Below are a few points that I wish to have clarified.
i) When the author writes B={x: Sx is covered by a finite subcover of F}, does he mean to say that 'a finite subcover of F' is some family of open sets H where H⊆G are both open covers of F, or that H⊆F?
ii) I would assume that S is compact because the author has found an open cover of S that has no finite subcover. In saying this, is he speaking of F or of B as being this particular open cover of S without a finite subcover?
Comment: $F$ is the open cover, $F = \{ U_\alpha : \alpha \in A\}$, where $A$ is some index set, and $U_\alpha$ is an open set, and $S \subset \bigcup\limits_{\alpha\in A}U_\alpha$. A finite subcover of $F$ is a finite subset $F_0 \subset F$ that also covers $S$ (resp., since in the proof the sets $S_x$ are considered, covers $S_x$). So $B$ is the set of all $x$, such that $S_x$ is contained in the union of some finite subfamily of $F$.
Answer: (1) A finite subcover of an open cover $F$ of $S$ is a finite family $H \subseteq F$ whose union contains $S$. (I'm not sure what $G$ was supposed to be in the OP, but what I said is correct.)
(2) The question seems confused. The author wants to show that every open cover $F$ of $S$ has a finite subcover. Anyway, $F$ is the open cover in question. $B$ on the other hand is not a collection of open sets; it is a collection of elements of $\mathbb{R}$.


Making your .NET language step correctly in the debugger

Question: Firstly, I apologize for the length of this question.
I am the author of IronScheme. Recently I have been working hard on emitting decent debug info, so that I can use the 'native' .NET debugger.
While this has been partly successful, I am running into some teething problems.
The first problem is related to stepping.
Due to Scheme being an expression language, everything tends to be wrapped in parenthesis, unlike the major .NET languages which seems to be statement (or line) based.
The original code (Scheme) looks like:
(define (baz x)
  (cond
    [(null? x)
      x]
    [(pair? x)
      (car x)]
    [else
      (assertion-violation #f "nooo" x)]))

I have on purpose laid out each expression on a newline.
The emitted code transforms to C# (via ILSpy) looks like:
public static object ::baz(object x)
{
  if (x == null)
  {
    return x;
  }
  if (x is Cons)
  {
    return Builtins.Car(x);
  }
  return #.ironscheme.exceptions::assertion-violation+(
     RuntimeHelpers.False, "nooo", Builtins.List(x));
}

As you can see, pretty simple.
Note: If the code was transformed into a conditional expression (?:) in C#, the whole thing would just be one debug step, keep that in mind.
Here is IL output with source and line numbers:
  .method public static object  '::baz'(object x) cil managed
  {
    // Code size       56 (0x38)
    .maxstack  6
    .line 15,15 : 1,2 ''
//000014:
//000015: (define (baz x)
    IL_0000:  nop
    .line 17,17 : 6,15 ''
//000016:   (cond
//000017:     [(null? x)
    IL_0001:  ldarg.0
    IL_0002:  brtrue     IL_0009

    .line 18,18 : 7,8 ''
//000018:       x]
    IL_0007:  ldarg.0
    IL_0008:  ret

    .line 19,19 : 6,15 ''
//000019:     [(pair? x)
    .line 19,19 : 6,15 ''
    IL_0009:  ldarg.0
    IL_000a:  isinst [IronScheme]IronScheme.Runtime.Cons
    IL_000f:  ldnull
    IL_0010:  cgt.un
    IL_0012:  brfalse    IL_0020

    IL_0017:  ldarg.0
    .line 20,20 : 7,14 ''
//000020:       (car x)]
    IL_0018:  tail.
    IL_001a:  call object [IronScheme]IronScheme.Runtime.Builtins::Car(object)
    IL_001f:  ret

    IL_0020:  ldsfld object
         [Microsoft.Scripting]Microsoft.Scripting.RuntimeHelpers::False
    IL_0025:  ldstr      "nooo"
    IL_002a:  ldarg.0
    IL_002b:  call object [IronScheme]IronScheme.Runtime.Builtins::List(object)
    .line 22,22 : 7,40 ''
//000021:     [else
//000022:       (assertion-violation #f "nooo" x)]))
    IL_0030:  tail.
    IL_0032:  call object [ironscheme.boot]#::
       'ironscheme.exceptions::assertion-violation+'(object,object,object)
    IL_0037:  ret
  } // end of method 'eval-core(033)'::'::baz'

Note: To prevent the debugger from simply highlighting the entire method, I make the method entry point just 1 column wide.
As you can see, each expression maps correctly to a line.
Now the problem with stepping (tested on VS2010, but same/similar issue on VS2008):
These are with IgnoreSymbolStoreSequencePoints not applied.

Call baz with null arg, it works correctly. (null? x) followed by x.
Call baz with Cons arg, it works correctly. (null? x) then (pair? x) then (car x).
Call baz with other arg, it fails. (null? x) then (pair? x) then (car x) then (assertion-violation ...).

When applying IgnoreSymbolStoreSequencePoints (as recommended):

Call baz with null arg, it works correctly. (null? x) followed by x.
Call baz with Cons arg, it fails. (null? x) then (pair? x).
Call baz with other arg, it fails. (null? x) then (pair? x) then (car x) then (assertion-violation ...).

I also find in this mode that some lines (not shown here) are incorrectly highlighted, they are off by 1.
Here are some ideas what could be the causes:

Tailcalls confuses the debugger
Overlapping locations (not shown here) confuses the debugger (it does so very well when setting a breakpoint)
????

The second, but also serious, issue is the debugger failing to break/hit breakpoints in some cases.
The only place where I can get the debugger to break correctly (and consistantly), is at the method entry point.
The situation gets a bit better when IgnoreSymbolStoreSequencePoints is not applied.
Conclusion
It might be that the VS debugger is just plain buggy  :(
References:

Making a CLR/.NET Language Debuggable

Update 1:
Mdbg does not work for 64-bit assemblies. So that is out. I have no more 32-bit machines to test it on. Update: I am sure this is no big problem, does anyone have a fix? Edit: Yes, silly me, just start mdbg under the x64 command prompt :)
Update 2:
I have created a C# app, and tried to dissect the line info.
My findings:

After any brXXX instruction you need to have a sequence point (if not valid aka '#line hidden', emit a nop).
Before any brXXX instruction,  emit a '#line hidden' and a nop.

Applying this, does not however fix the issues (alone?).
But adding the following, gives the desired result  :)

After ret, emit a '#line hidden' and a nop.

This is using the mode where IgnoreSymbolStoreSequencePoints is not applied. When applied, some steps are still skipped :(
Here is the IL output when above has been applied:
  .method public static object  '::baz'(object x) cil managed
  {
    // Code size       63 (0x3f)
    .maxstack  6
    .line 15,15 : 1,2 ''
    IL_0000:  nop
    .line 17,17 : 6,15 ''
    IL_0001:  ldarg.0
    .line 16707566,16707566 : 0,0 ''
    IL_0002:  nop
    IL_0003:  brtrue     IL_000c

    .line 16707566,16707566 : 0,0 ''
    IL_0008:  nop
    .line 18,18 : 7,8 ''
    IL_0009:  ldarg.0
    IL_000a:  ret

    .line 16707566,16707566 : 0,0 ''
    IL_000b:  nop
    .line 19,19 : 6,15 ''
    .line 19,19 : 6,15 ''
    IL_000c:  ldarg.0
    IL_000d:  isinst     [IronScheme]IronScheme.Runtime.Cons
    IL_0012:  ldnull
    IL_0013:  cgt.un
    .line 16707566,16707566 : 0,0 ''
    IL_0015:  nop
    IL_0016:  brfalse    IL_0026

    .line 16707566,16707566 : 0,0 ''
    IL_001b:  nop
    IL_001c:  ldarg.0
    .line 20,20 : 7,14 ''
    IL_001d:  tail.
    IL_001f:  call object [IronScheme]IronScheme.Runtime.Builtins::Car(object)
    IL_0024:  ret

    .line 16707566,16707566 : 0,0 ''
    IL_0025:  nop
    IL_0026:  ldsfld object
      [Microsoft.Scripting]Microsoft.Scripting.RuntimeHelpers::False
    IL_002b:  ldstr      "nooo"
    IL_0030:  ldarg.0
    IL_0031:  call object [IronScheme]IronScheme.Runtime.Builtins::List(object)
    .line 22,22 : 7,40 ''
    IL_0036:  tail.
    IL_0038:  call object [ironscheme.boot]#::
      'ironscheme.exceptions::assertion-violation+'(object,object,object)
    IL_003d:  ret

    .line 16707566,16707566 : 0,0 ''
    IL_003e:  nop
  } // end of method 'eval-core(033)'::'::baz'

Update 3:
Problem with above 'semi-fix'. Peverify reports errors on all methods due to the nop after ret. I dont understand the problem really. How can a nop break verification after a ret. It is like dead code (except that it is NOT even code) ... Oh well, experimentation continues.
Update 4:
Back at home now, removed the 'unverifiable' code, running on VS2008 and things are a lot worse. Perhaps running unverifiable code for the sake of proper debugging might be the answer. In 'release' mode, all output would still be verifiable.
Update 5:
I have now decided my above idea is the only viable option for now. Although the generated code is unverifiable, I have yet to find any VerificationException's. I dont know what the impact will be on the end user with this scenario.
As a bonus, my second issue has also be solved. :)
Here is a little screencast of what I ended up with. It hits breakpoints, does proper stepping (in/out/over), etc. All in all, the desired effect.
I, however, am still not accepting this as the way to do it. It feel overly-hacky to me. Having a confirmation on the real issue would be nice.
Update 6:
Just had the change to test the code on VS2010, there seems to be some problems:

The first call now does not step correctly. (assertion-violation ...) is hit. Other cases works fine. Some old code emitted unnecessary positions. Removed the code, works as expected. :)
More seriously, breakpoints fail on the second invocation of the program (using in-memory compilation, dumping assembly to file seems to make breakpoints happy again).

Both these cases work correctly under VS2008. The main difference is that under VS2010, the entire application is compiled for .NET 4 and under VS2008, compiles to .NET 2. Both running 64-bit.
Update 7:
Like mentioned, I got mdbg running under 64-bit. Unfortunately, it also have the breakpoint issue where it fails to break if I rerun the program (this implies it gets recompiled, so not using the same assembly, but still using the same source).
Update 8:
I have filed a bug at the MS Connect site regarding the breakpoint issue.
Update: Fixed
Update 9:
After some long thinking, the only way to make the debugger happy seems to be doing SSA, so every step can be isolated and sequential. I am yet to prove this notion though. But it seems logical. Obviously, cleaning up temps from SSA will break debugging, but that is easy to toggle, and leaving them does not have much overhead.
Comment: I think his question is along the lines of "why isn't my language stepping correctly?"
Comment: What's the question?
Comment: If you don't pass peverify you wont be able to run in partial trust situations, like from a mapped drive or in many plugin hosting setups. How are you generating your IL, reflection.Emit or via text + ilasm? In either case you can always put in the .line 16707566,16707566 : 0,0 '' yourself that way you dont have to worry about them putting a nop after the return.
Comment: Any ideas welcome, I will try them out and append the results to the question. First 2 ideas, try mdbg and write the same code in C# and compare.
Comment: @Hippiehunter: Thanks. Unfortunately without the `nop`s, the stepping fails (I will verify this again to be sure). It is a sacrifice I guess I would have to make. It is not like VS can even run without administrator rights :) Btw using Reflection.Emit via the DLR (a very hacked early branched one).
Comment: Are you using the default expression evaluator? Also have you tried applying the line directive to the ret instead of a nop?
Comment: Also according to this http://blogs.msdn.com/b/clrcodegeneration/archive/2009/05/11/tail-call-improvements-in-net-framework-4.aspx It looks like you should be able to put a nop after the tail call before the ret, thus giving you somewhere to put the hidden line directive
Comment: FYI: http://programmers.stackexchange.com/questions/100417/how-does-the-dlr-compare-to-c-4-0
Comment: I don't see any mentions of pdb's... I thought the pdb was what told the debugger how to map the IL onto the original code?
Comment: @justin.m.chase: PDB's are generated and I currently have to rely on its sequence points (see `IgnoreSymbolStoreSequencePoints` which I disable). If the debugger uses the IL sequence points (iow applying `IgnoreSymbolStoreSequencePoints`), it goes off by 1.
Comment: @Hippiehunter: I will try that, but my code will need some modification, and given the strictness of .NET 4, it might not be verifiable. Still not sure it will solve the stepping issue. Currently .NET 4 (and 2) seems to be at least happy with having unverifiable code in unreachable locations. Thanks for your feedback.
Comment: Here's another mention of IronScheme: http://programmers.stackexchange.com/questions/107368
Comment: @Robert Harvey: Thanks for the ping :)
Answer: I am an engineer on the Visual Studio Debugger team.
Correct me if I am wrong, but it sounds like the only issue left is that when switching from PDBs to the .NET 4 dynamic compile symbol format some breakpoints are being missed.
We would probably need a repro to exactly diagnose the issue, however here are some notes that might help.

VS (2008+) can-to run as a non-admin
Do any symbols load at all the second time around? You might test by breaking in (through exception or call System.Diagnostic.Debugger.Break())
Assuming that symbols load, is there a repro that you could send us?
The likely difference is that the symbol format for dynamic-compiled code is 100% different between .NET 2 (PDB stream) and .NET 4 (IL DB I think they called it?)
The 'nop's sound about right. See rules for generating implicit sequence points below.
You don't actually need to emit things on different lines. By default, VS will step 'symbol-statements' where, as the compiler writer you get to define what 'symbol-statement' means. So if you want each expression to be a separate thing in the symbol file, that will work just fine.

The JIT creates an implicit sequence point based on the following rules:
1.      IL nop instructions
2.      IL stack empty points
3.      The IL instruction immediately following a call instruction
If it turns out we do need a repro to solve your issue, you can file a connect bug and upload files securely through that medium.
Update:
We are encouraging other users experiencing this issue to try the Developer Preview of Dev11 from http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=27543 and comment with any feedback. (Must target 4.5)
Update 2:
Leppie has verified the fix to work for him on the Beta version of Dev11 available at http://www.microsoft.com/visualstudio/11/en-us/downloads as noted in the connect bug https://connect.microsoft.com/VisualStudio/feedback/details/684089/.
Thanks,
Luke
Comment: Thanks a lot. I will try make repro, if needed.
Comment: As for the confirmation, yes you are correct, but I would still prefer to solve the stepping issues without breaking verifiability. But as said, I am willing to sacrifice that if I could possibly prove that it will never throw a runtime `VerificationException`.
Comment: As for point 6, I simply made it 'line-based' for this question. The debugger does in fact step correctly in/out/over expressions like I intend it to work :)  The only problem with approach is setting breakpoints on the 'inner' expressions if an 'outer' expression covers it; the debugger in VS tends to want to make the most outer expression as the breakpoint.
Comment: I think I have isolated the breakpoint issue. While running under CLR2 the breakpoints seems to be re-evaluated when running the code again, albeit new code. On CLR4, the breakpoints only 'stick' to the original code. I have made a small screencast of the CLR4 behavior @ http://screencast.com/t/eiSilNzL5Nr. Notice that the type name changes between invocations.
Comment: I have filed a bug on connect. The repro is quite easy :) https://connect.microsoft.com/VisualStudio/feedback/details/684089/
Comment: Marking as answer in the meanwhile.
Comment: Thanks for filing the bug. We'll take a look at it internally, however it might be a few days.
Comment: No problem, I suspect an issue in the debugger 'section' as MDBG showed the same behavior.
Comment: Just to close this thread so you know I didn't disappear... The Connect ticket will be updated with future findings. We have found the actual issue, but it's not from my team, so there could be some delay with the Connect bug being updated.
Comment: If you can, I'd now recommend you try the developer preview build http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=27543. Thanks.
Comment: Thanks, I did, but the new 'remote debugger' (using pseudo remote) is too slow to even start my app without debug symbols. This is targeting .NET 4.5. :( I'll wait for the RC :) (Can you change the default back to the 'old debugger'?)
Comment: Interesting. Unlike this .NET issue, the Remote Debugger is something I actually work on. I am interested as to why it is slow for you, especially considering this is pseudo remote. I'll try to follow up through your CodePlex contact tomorrow about the performance issue. In the meantime, perhaps you could try x86?
Comment: IIRC, the hosting debugger process, was taking up all the CPU resources, leaving nothing for the app. I will create a repro tonite. Thanks :)
Answer: I am an engineer on the SharpDevelop Debugger team :-)
Did you solve the problem?
Did you try to debug it in SharpDevelop?  If there is a bug in .NET, I wonder if we need to implement some workaround.  I am not aware of this issue.
Did you try to debug it in ILSpy?  Especially without debug symbols.  It would debug C# code, but it would tell us if the IL instructions are nicely debugable.  (Mind that ILSpy debugger is beta though)
Quick notes on the original IL code:

.line 19,19 : 6,15 ''  occurs twice?
.line 20,20 : 7,14 ''  does not start on implicit sequence point (stack is not empty).  I am worried
.line 20,20 : 7,14 ''  includes the code for "car x" (good) as well as "#f nooo x"  (bad?)
regarding the nop after ret.  What about stloc, ldloc, ret?  I think C# uses this trick to make ret a distinct sequence point.

David
Comment: +1 Thanks for the feedback, as the first point, unfortunate side-effect from the code generator, but seems harmless. Second point, this is exactly what I need, but I see your point, will look into it. Not sure what you mean by the last point.
Comment: Third point was that .line 22,22 : 7,40 '' should be before IL_0020.  Or there should be something before IL_0020, otherwise the code still counts as  .line 20,20 : 7,14 ''.
The fourth point is "ret, nop" might be replaced with "sloc, .line, ldloc, ret".  I have seen the pattern before, maybe it was redundant, but maybe it had a reason.
Comment: I cant do the stloc, ldloc before ret, as I will loose the tail call. Ah I get you, you were referring to the original output?


Object reference not set to an instance of an object - MVC

Question: Estou carregando uma tabela, e fazendo a depuração, ela trás os dados perfeitamente, porém quando vou pegar os dados da tabela referencia na chave estrangeira, ela está me retornando o seguinte erro:

NullReferenceException: Object reference not set to an instance of an object.

Aqui é como está na minha model:
  public int Id { get; set; }
    public Empresa EmpresaProduto { get; set; }
    public int EmpresaID { get; set; }

Carregando no controller:
  var produtoempresa = await db.ProdutosEmpresas.Where(p => p.ProdutoID == id).ToListAsync();

E aqui é minha tabela:

                            @foreach (var item in Model.ProdutosEmpresas)
                            {

                                    @item.EmpresaProduto.RazaoSocial
                                    @item.QtdAtual

                            }

Eu tenho uma outra tabela, que foi montada da mesma forma, e carrega os dados da tabela referenciada normal, porém nesta eu não consigo.
Answer: Você não está dando o include: altere a linha de carregamento das entidades para:
var produtoempresa = await db
        .ProdutosEmpresas
            .Include(p => p.ProdutosEmpresas)
        .Where(p => p.ProdutoID == id)
        .ToListAsync();

Comment: Assim dá certo, mas o estranho, é que eu faço a mesma coisa em outra tabela, e não me retorna este erro, deve ser algo no banco, rs Obrigado.
Comment: Pode ter várias razões Mariana, se a propriedade estiver como virtual, vai carregar por Lazy Loading, caso não esteja marcada com virtual irá carregar somente com o Eager Loading (uso do Include). Aconselho a ler a documentação do EF.


Uma função de armazena elementos específicos de uma lista em outra lista

Question: Quero fazer uma função que recebe uma lista de números ordenados e dois números(um inferior e um superior) e que retorna uma lista nova com os elementos da primeira lista entre os dois números.
Exemplo:
lista inicial=[12,14,15,16,18,20,24,26,28,32,34,38]
limite inferior=13
limite superior = 26
lista exibida: [14,15,16,18,20,24,26]

O que eu tentei fazer:
def mudalista(l,li,ls):
    sublista=list()
    for el in l:
        if el>=li:
            pos_el1=l.index(el)
        elif el>=ls:
            pos_el2=l.index(el)
        sublista=l[pos_el1:pos_el2]
    print(l[subslista])
    return

Tentei começar a função com "for el in l" e quando(if) o elemento fosse maior que o numero menor, armazenasse em uma variável, fazendo a mesma coisa com o superior(elif), mas quando vou tentar fazer a segunda lista, ele dá erro.
Traceback (most recent call last):
  File "<pyshell#19>", line 1, in <module>
    mudalista([12,14,15,16,18,20,24,26,28,32,34,38],13,26)
  File "C:\Users\G1820583\Downloads\exercicio lista.py", line 11, in mudalista
    sublista=l[pos_el1:pos_el2]
UnboundLocalError: local variable 'pos_el1' referenced before assignment

Comment: Guilherme, conhece [teste de mesa](https://pt.stackoverflow.com/q/220474/5878)?
Comment: Não... vou ler sobre
Comment: Estude os métodos de lista do Python, mais precisamente como pegar segmentos dela e, claro, o método `index()`.
Answer: TL;DR
import bisect

def intervalo(lista, i, j):
    left = bisect.bisect_left(lista, i)
    right = bisect.bisect(lista, j)

    return lista[left:right]

Explicação
Repare neste trecho do seu código:
if el >= li:
    pos_el1 = l.index(el)
elif el >= ls:
    pos_el2 = l.index(el)
sublista = l[pos_el1:pos_el2]

O código acima tem 3 possibilidades:

el >= li então pos_el1 é criado, pois entrou no 1º if;
el >= ls então pos_el2 é criado, pois entrou no 2º if;
el é menor que ambos, li e ls, e nenhuma variável é criada pois não entrou em nenhum if e você apenas cria eles dentro destes ifs.

Note que em nenhum fluxo possível deste código as variáveis pos_el1 e pos_el2 podem coexistir, ou apenas uma existe, ou apenas a outra, nunca ambas são criadas no mesmo fluxo.
Isso faz com que a linha posterior: sublista = l[pos_el1:pos_el2] obrigatoriamente dê erro, pois você está usando uma das variáveis que obrigatoriamente não será criada.
Possíveis soluções:

List comprehension (#Docs)
A mais simples de todas poderia ser usar uma list comprehension filtrando apenas os itens que cumprirem sua condição. Como sua lista têm os valores ordenados, essa solução funciona bem:
def intervalo(lista, i, j):
    return [item for item in lista if i <= item <= j]

lista_inicial = [12,14,15,16,18,20,24,26,28,32,34,38]
intervalo(lista_inicial, 13, 26)
# [14, 15, 16, 18, 20, 24, 26]

Se você ainda não conhece o conceito de comprehensions, uma versão equivalente usando for ficaria assim:
def intervalo(lista, i, j):
    resultado = []

    for item in lista:
        if if i <= item <= j:
            resultado.append(item)

    return resultado

lista_inicial = [12,14,15,16,18,20,24,26,28,32,34,38]
intervalo(lista_inicial, 13, 26)
# [14, 15, 16, 18, 20, 24, 26]

Slice (#Docs)
Pode-se usar slices, como você mesmo fez, mas é preciso tratar quando os valores procurados não existem na lista, pois se os valores que limitam seu intervalo não existirem na lista você vai ter que selecionar o índice mais próximo. Uma solução poderia ser:
def intervalo_slices(lista, i, j):
    left, right = 0, len(lista)

    for index, item in enumerate(lista):
        if item < i:
            # se for menor que o mínimo mostra do próximo em diante
            left = index + 1
        if item <= j:
            # Se for maior que o máximo mostra até o próximo índice
            right = index + 1

    return lista[left:right]

O algoritmo é simples:

Eu crio left e right com valores padrão 0 e o tamanho total da lista (len(lista)), dessa maneira se o fluxo não entrar em nenhum dos ifs o resultado será lista[0:len(lista)] que é apenas uma cópia da lista original;
Uso enumerate para ter os índices dos itens que estou iterando e realizar os cálculos (semelhante ao i que vemos em laços for de outras linguagens);
Testo em todas as iterações se item é menor que o mínimo.
Caso a condição seja verdadeira, significa que o limite inferior da nossa lista é o item seguinte, pois vai ser igual ou maior que i;
Testo em todas as iterações se item é menor ou igual ao máximo, se sim, o índice que eu quero também é o próximo, pois slices não incluem o valor da direita (ex: [0, 1, 2, 3][:2] resulta em [0, 1]);
Por fim retorno o slice com os valores corretos

Se você der um print dentro dos if verá que left e right recebem valores várias vezes até que as condições deixem de serem verdadeiras. Não é uma solução muito otimizada, mas é um algoritmo mais "feito na mão" e eu queria mostrar o que deu errado com seu exemplo com uma possível solução para a sua linha de raciocínio.
Para uma solução mais otimizada nessa mesma linha de raciocínio, veja a próxima sessão.

Módulo bisect (#Docs)
Outra maneira de resolver o problema "qual índice da lista devo fazer slice" é usar um algoritmo de pesquisa binário para encontrar em qual posição da lista deve-se fazer o slice, mesmo que o valor não exista nesta lista.
O módulo bisect foi feito para trabalhar com sequências ordenadas e pesquisa binária, então podemos tirar proveito de um módulo nativo do python para resolver o problema com uma complexidade de O(Log2 n) ao invés de O(n) como os problemas anteriores.
Para isso usaremos as funções bisect e bisect_left. Ambas têm o mesmo propósito e funcionamento, elas servem para descobrir em qual posição de uma sequência ordenada um valor deve ser inserido para que seja mantida sua ordenação.
Por exemplo:
from bisect import bisect

lista = [0, 2, 4, 5, 8, 10]
index = bisect(lista, 3)
# 2

lista.insert(index, 3)
# lista = [0, 2, 3, 4, 5, 8, 10]
#                └── posição retornada por bisect

Veja que bisect retornou o índice 2, que é exatamente onde eu posso inserir o valor 3 na minha lista para que seja mantida a ordenação da mesma.
Mas pra que serve o bisect_left?
A funções bisect e bisect_left se diferenciam apenas quando o valor procurado já existe na sequência, ou seja, no exemplo anterior poderia ser usado bisect ou bisect_left pois o resultado seria o mesmo.
Quando procuramos o índice de inserção de um valor já existente na sequência ficamos com duas opções:

Inserir no mesmo índice do valor encontrado (bisect_left).
Para inserir o valor 1 na lista [0, 1, 2, 3]:
#      ┌── valor encontrado
[0, 1, 1, 2, 3]
#   └── novo valor

Inserir no próximo índice do valor encontrado (bisect).
Para inserir o valor 1 na lista [0, 1, 2, 3]:
#   ┌── valor encontrado
[0, 1, 1, 2, 3]
#      └── novo valor

Em resumo, a diferença é se o índice retornado é o mesmo do valor encontrado ou o próximo. O código abaixo ilustra os exemplos citados acima:
from bisect import bisect, bisect_left

lista = [0, 1, 2, 3]
indice = bisect(lista, 1)
# 2
indice_left = bisect(lista, 1)
# 1

Código final usando bisect
Agora que você entendeu o conceito, bastaria usar o bisect_left para pegar corretamente o índice mínimo para usar no slice e bisect para pegar o índice máximo.
import bisect

def intervalo(lista, i, j):
    left = bisect.bisect_left(lista, i)
    right = bisect.bisect(lista, j)

    return lista[left:right]

Conclusão
Estas são algumas formas de fazer o que você quer, apresentei o módulo bisect pois além de ser performático e interessante, acho importante para o apredizado termos exemplos de uso prático.
Criei este Repl.it com todos os exemplos rodando.
Answer: Esse erro que você apresentou ocorre porque a variável pos_el1 só é criada caso entre no if el>=1. Ou seja, se o programa não entrar nesse bloco, você tentará utilizar uma variável que não existe.
Há também alguns outros erros de lógica como por exemplo o bloco if el>=li. Caso o elemento seja maior que li o menor elemento será ele. Só que isso significa que o maior elemento da lista também será o menor, já que o maior elemento é maior que li.
Há também um problema ao utilizar slice para obter uma nova lista de números. Lembre-se que a posição final é uma posição de parada e o valor nessa posição não será retornado. Logo você deverá fazer com que a posição final seja a posição do maior elemento +1.
Veja esse código abaixo que eu fiz:
def mudaLista(lista,li,ls):

    menor_elemento = None
    maior_elemento = None

    for elemento in lista:

        if not menor_elemento and elemento >= li:
            menor_elemento = elemento

        if elemento <= ls:
            maior_elemento = elemento

    if menor_elemento and maior_elemento:

        pos_menor = lista.index(menor_elemento)
        pos_maior = lista.index(maior_elemento)

        return lista[pos_menor:pos_maior+1]

    else:
        return []

lista = [num for num in range(100)] #Gera uma lista com números de 1 à 100
li = 43
ls = 57
print(mudaLista(lista,li,ls)) # Saída: [43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57]


How can i measure the size of objects in an photo taken by iPhone without ARKit?

Question: we'll be using the United States quarter as our reference object and throughout all examples

I need to determine our "pixels per metric" ratio, which describes the number of pixels that can "fit" into a given number of inches, millimeters, meters, etc.
I need output should look something like the following:
Answer: You can find code on this blog to find size of objects.
Pixels per metric -
It is defined as Number of pixels per metric (mm, cm, mtr). First you need to find this ratio for a single object ( US Quarter in your case ). You will then use this ratio to find sizes of other objects. Now to find pixels per metric in your case -
1) Filter the image and find contours.
2) Sort the contours from left to right
3) Find the Corners of the first Contour (US quarter)
4) Find the Distance between any two adjacent corners of the object in pixels
5) pixelspermetric = Distance_between_corners_in_pixels / Distance_between_corners_in_cm
For example
Suppose the distance between two adjacent corners of US Quarter is 200 pixels and actual width of coin is 0.955 inches. So,
PixelsPerMetric = 200/0.955 = 209.4240
Now you can find size of other object as -
size = length_in_pixels/PixelsPerMetric
This ratio remains constant for a certain height (distance between object and camera). You need to calculate the ratio again if height changes.


iptables question for port forwarding

Question: I have a web server running on my server on port 7000. I want it to be accessible on port 80. I've read that this could be easily done with iptables port forwarding.
What would the correct command would be to achieve that?
E
Answer: You can try something along the lines of this:
<code>iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 7000
</code>

iptables question for port forwarding


How to make two widgets attached to each other and resizing each other in Qt Creator

Question: How to create two widgets(QList or QTab for example) attached to each other and draggable line between them so they can be resized together(one gets bigger and the other smaller and the other way around)?
Answer: http://doc.qt.io/qt-4.8/qsplitter.html
This can be done in the form editor with the layout options (shown below).

This will allow you to drag back and forth in between your widgets.
Comment: This option is always disabled for me. I am using the same items.
Comment: QSplitter *splitter = new QSplitter(parent);
    QListWidget *listwidget = new QListWidget;
    QTabWidget *tabwidget = new QTabWidget;

    splitter->addWidget(listwidget);
    splitter->addWidget(tabwidget);


Unexpected behavior of getline function

Question: I have this loop in my program tha reads a phone number as a string:
while (loop) {
    cout << "Telefone (0 para sair):" << endl;
    string phone;
    getline(cin, phone);

    // .. some logic
    while (loop2) {
        // .. more logic
        getline(cin, option);
        // .. more logic
    }
}

when the loop finishes the logic and returns to the begining, it doesn't ask for the phone again. It just goes through the logic again. Any ideas?
If I had used cin >> phone instead, it doesn't catch phone numbers with blank spaces.
Comment: @Nowayz, I don't see any problems with it apart from maybe the implied using directive.
Comment: @LuizEduardoF., What would really help is a [minimal, but complete example](http://stackoverflow.com/help/mcve). At this point, we can only guess.
Comment: @Nowayz No, that's the member function `std::istream::getline()`, not the global `std::getline()`.
Comment: What are the loop conditions?  You might want to use getline or another function which returns a stream as the loop condition.
Comment: @Nowayz, No, there's just one. The OP stated that if `getline` was not used, the input could not contain spaces, so `getline` must be used. It's sort of like an attempted solution, but seems like more of a thought-about solution that wasn't worth trying.
Comment: @Nowayz, It sounds a lot like one of those "I use `getline`, but it just gets skipped over" questions, which is why I asked about the other logic. Anyway, the `gets` is a non-solution. It will most likely suffer the same problem and it's being removed as of C++14 because of how horrible a function it is. The reason for `cin >> phone` not working is presumably a desire to include spaces in the input when reading instead of stopping at them.
Comment: @LuizEduardoF., That sounds *exactly* like what would happen if there's a `cin >> something;` in that logic you haven't shown, or if the stream is failing (e.g., if it reached EOF).
Comment: possible duplicate of [Why does std::getline() skip input after a formatted extraction?](http://stackoverflow.com/questions/21567291/why-does-stdgetline-skip-input-after-a-formatted-extraction)
Comment: Does that unshown logic include `cin >> something`?
Comment: Please spend a day or three on this website looking for similar problems; this issue has been debated thousands of times.
Comment: @chris I'll complete one more portion of the loop.
Comment: Isn't getline called std::cin.getline(std::string,size)
Comment: I guess that would work nevermind, I'm just getting confused by the question in general.  Of course its not asking for the phone again if loop2 never evaluates to false
Comment: Are there two questions here?  One about how to include spaces in the getline and another about why its not asking for the phone number again?
Comment: I still can't remotely comprehend what he's asking with "when the loop finishes the logic and returns to the begining, it doesn't ask for the phone again. It just goes through the logic again, any ideas?" assuming that his solution with getline worked fine
Comment: @LuizEduardoF. If you wanted an alternate solution you could just use a char* buffer and call gets instead..  If you're just asking why using cin >> phone doesn't work it's because that's just the way it's designed.  It splits input up between spaces
Comment: In the first step, the program does everything ok. But when loop2 becomes false and it go back to while(loop){} to ask a new phone number, it shows: `Telefone (0 para sair):` `Client not recognized. New Client:`. The program doesn't ask a new phone number and that's the question here.
Comment: Horrible solution for a horribly written question.  He could just link the clib statically, lol.
Answer: Your claim that the following loop doesn't ask for phone input is highly indicative that std::getline() failed to produce characters because of leading whitespace. My suspicion is that you performed some formatted extraction in your real code that left a newline in the stream. The unformatted extractor std::getline() delimits input upon the aquisition of the newline character, which is the reason you're experiencing this behavior.
Invariably, the solution to this problem is the clear the leading whitespace. This is done using the manipulator std::ws:
std::getline(std::cin >> std::ws, phone);
//           ^^^^^^^^^^^^^^^^^^^

Note: I talk about this in detail in my answer here.
You should also be checking if your input suceeded. This means enclosing the extraction within an if statement so you can ascertain if it successfully produced characters:
if (std::getline(std::cin >> std::ws, phone))
{
    // ...
}

Answer: Perhaps you need to use a cin.clear() or a cin.ignore() before trying to input again.
More explanation:
That's just from my experience, but according to jrok in this thread: getline() does not work if used after some inputs
"The extraction operations leave the trailing '\n' character in the stream. On the other hand, istream::getline() discards it. So when you call getline after an extraction operator, '\n' is the first character it encounters and it stops reading right there. Put this after before getline call extraction: cin.ignore()"
Comment: Perhaps you can explain why?
Comment: That's just from my experience, but according to jrok in this thread: http://stackoverflow.com/questions/12691316/getline-does-not-work-if-used-after-some-inputs, "The extraction operations leave the trailing '\n' character in the stream. On the other hand, istream::getline() discards it. So when you call getline after an extraction operator, '\n' is the first character it encounters and it stops reading right there. Put this after before getline call extraction: cin.ignore()"
Comment: I meant to explain why in your answer. This only gives the solution, not the explanation. Your comment is good, you can add that in to your answer.

Unexpected behavior of getline function


How to access monitoring web UI when running spark on AWS

Question: I'm running spark from AWS EC2. As the spark documentation said

"Each driver program has a web UI, typically on port 4040, that displays information about running tasks, executors, and storage usage. Simply go to http://:4040 in a web browser to access this UI"

My question is: what is this "driver-node"?
I thought it's the public IP of the driver node xx.xx.xxx.xx. But I kept getting page not found error when I user http://xx.xx.xxx.xx:4040
Please help.
Answer: Remember to create a security group and assign to it and open the inbound port such as 0.0.0.0/0 with port 4040
Then you should be fine to access it with its public IP, such as:
http://54.xx.xx.xx:4040

0.0.0.0/0 is opened to all, you can change it to your own IP or other ip ranges to limit the traffic.
Answer: I had the exact same question as you. Eventually, the  should be replaced by your IP address.
But not the IP address after you google "what's my IP address". You should execute ipconfig in command line and use that IP address in the URL.
And if you don't have a valid sparkContext (sc) running, the UI will return error
Answer: Alternatively

ssh tunnel to the remote/ec2 port, e.g.,
ssh -i "c:\path\to\keypair.pem" -N -L \
  4040:ec2-##-###-###-##.us-east-2.compute.amazonaws.com:4040 \
  ec2-user@ec2-##-###-###-##.us-east-2.compute.amazonaws.com

Then just connect your browser to http://localhost:4040/


Why is one of the two approximations in the bootstrap worse than the other?

Question: My statistics text has the following diagram:
$$\mathbb{V}_F(T_n) \overbrace{\approx}^{ \text{not so small}  } \mathbb{V}_{\hat{F}_n}(T_n) \overbrace{\approx}^{ \text{small}  } v_{\text{boot}}$$
Where $T_n$ is a statistic, $F$ is the true CDF and $\hat{F}_n$ is the empirical distribution function, $\mathbb{V}$ denotes variance, and $v_{\text{boot}}$ is the variance of the statistic that we got from bootstrap replications.
Why is one "small" and the other is "not so small"?
Answer: The second approximation is obtained by montecarlo simulation. Hence you can make it as small as you wish taking $B_{boot} \to \infty$ where $B_{boot}$ is the number of bootstrap replications.
The first approximation is due to estimating $F$ by $F_n$. By Glivenko Cantelli theorem this is small if $n$ is sufficently large. The problem is that you can not control it unless you can get as many samples as you wish i.e  make $n\to \infty$.
Note that there are others ways of using bootstrap that do not use sampling from the empirical distribution. In those cases the same explanation still hold if $\hat F$ is a consistent estimator of $F$.


CardView doesn't comes as Overlapping content

Question: Followed this link Overlap scrolling view with AppBarLayout but overlapping doesn't seems to work well.Here i'm attaching my layout files with the output that i got.
activity_main.xml
<android.support.design.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <android.support.design.widget.AppBarLayout
        android:layout_width="match_parent"
        android:layout_height="192dp"
        android:theme="@style/ThemeOverlay.AppCompat.Dark.ActionBar">

        <android.support.design.widget.CollapsingToolbarLayout
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            app:layout_scrollFlags="scroll|enterAlways">

            <android.support.v7.widget.Toolbar
                android:layout_width="match_parent"
                android:layout_height="?attr/actionBarSize"
                app:layout_collapseMode="pin" />
        </android.support.design.widget.CollapsingToolbarLayout>
    </android.support.design.widget.AppBarLayout>

    <android.support.v4.widget.NestedScrollView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:behavior_overlapTop="64dp"
        app:layout_behavior="@string/appbar_scrolling_view_behavior"
        android:elevation="5dp"
        >

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:orientation="vertical">

            <android.support.v7.widget.RecyclerView
                android:id="@+id/days_list_5"
                android:layout_width="match_parent"
                android:layout_height="match_parent">

            </android.support.v7.widget.RecyclerView>

        </LinearLayout>
    </android.support.v4.widget.NestedScrollView>
</android.support.design.widget.CoordinatorLayout>

custom_layout.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:background="@android:color/white"
    android:orientation="vertical">

    <android.support.v7.widget.CardView
        android:id="@+id/cardNews"
        android:layout_width="match_parent"
        android:layout_height="100dp"
        android:background="@color/colorAccent"
        android:layout_margin="16dp">

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:orientation="vertical">

            <TextView
                android:id="@+id/lblNewsTitle"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"

                android:textColor="@color/colorAccent"
                android:textSize="16dp" />
        </LinearLayout>

    </android.support.v7.widget.CardView>

</LinearLayout>

Please have a look and give me your suggestion where i'm wrong.
Comment: Try to set `app:layout_scrollFlags="scroll|exitUntilCollapsed">` in your `CollapsingToolbarLayout`
Answer: By removing android:background="@android:color/white" from my custom layout and changing the CollapsingToolBar as app:layout_scrollFlags="scroll|enterAlwaysCollapsed",I resolved the issue.