pam_tally2 or pam_faillock account lockout with ssh

Question: I'm running RHEL 6.2.  Most users will be using SSH to login using passwords.  Some might have keys.  All accounts are local.
I need to lock users out after N failed password logins.
The examples in man pam_tally2 and pam_faillock do not lock a user using ssh out.  The pam_tally2 example can lock a telnet (I enabled to test) login & subsequently will lock an ssh user out.  ssh cannot trigger it.
/etc/ssh/sshd_config has:
PasswordAuthentication yes # setting to no doesn't allow login with a password!
UsePAM yes
UseLogin no # setting to yes doesn't allow putty logins

Man page examples should work on an unaltered system.
Comment: This doesn't answer your question directly, but, try [fail2ban](http://www.fail2ban.org). (It's in EPEL.)
Answer: If you enable PasswordAuthentication then the SSH daemon handles passwords itself and not using PAM. You actually want to disable this in order to force it to use PAM:
PasswordAuthentication no
UsePAM yes
ChallengeResponseAuthentication yes

That won't catch users using keys however (although personally I think that's fine). If you do you'll probably have to use something like fail2ban which looks for authentication failures in the logs and adds iptables rules to block future attempts.
Answer: You'll need to add the following lines to /etc/pam.d/sshd:
auth       required     pam_tally2.so deny=6 onerr=fail unlock_time=1800
account    required     pam_tally2.so

Add them on lines 3 and 6 as indicated below:
#%PAM-1.0
auth       required     pam_sepermit.so
auth       required     pam_tally2.so deny=6 onerr=fail unlock_time=1800
auth       include      password-auth
account    required     pam_nologin.so
account    required     pam_tally2.so
account    include      password-auth
password   include      password-auth
# pam_selinux.so close should be the first session rule
session    required     pam_selinux.so close
session    required     pam_loginuid.so
# pam_selinux.so open should only be followed by sessions to be executed in the user context
session    required     pam_selinux.so open env_params
session    required     pam_namespace.so
session    optional     pam_keyinit.so force revoke
session    include      password-auth

Also make sure UsePAM yes is set in /etc/ssh/sshd_config
This will lock an ssh user out for 30 minutes after six failed authentication attempts.

If we follow the official RHEL 6 Security Guide, we can accomplish this without changing /etc/pam.d/sshd.
We edit both /etc/pam.d/system-auth and /etc/pam.d/password-auth, replacing
auth        sufficient     pam_unix.so nullok try_first_pass

with
auth        required       pam_faillock.so preauth silent audit deny=3 unlock_time=600
auth        sufficient     pam_unix.so nullok try_first_pass
auth        [default=die]  pam_faillock.so authfail audit deny=3 unlock_time=600

And, in both files, we add this line to the top of the "account" section:
account     required      pam_faillock.so

This will provide account lockout functionality to console users, screensaver users, and so on.
If you examine /etc/pam.d/sshd you can see it uses password-auth and therefore ssh users will experience the same lockout functionality.
Comment: I follow the RHEL one, and do get locked out when using SSH, but message does not display that I am locked out, but when I locally try to lock myself out I see the message.. I don't understand why.
Answer: I had to edit '/etc/pam.d/sshd '
with
Auth required pam_tally2.so Deny=5 Unlock_time=1200 Even_deny_root Root_unlock_time=10
and it works now.
Answer: Discrepancy in the behavior of unlock_time in pam_faillock when compared with pam_tally. (unlock_time is not drifted according to the last failed attempt)
Configure pam_faillock in system-auth and password-auth with deny=3 and unlock_time=300, Now try to login with any non-root user and enter invalid password 3 times after which the account gets locked as expected, say the current time is 1300 hrs. The account locks out and you will not be allow to login with correct password until 13:05 hrs, If the user again tries to login again at 13:02, system denies as account is locked-out.
For the same scenario, In case of pam_tally, as user attempts to login at 13:02, the user will be unlocked only after 13:07 instead of 13:05. unlock_time is drifted after failed attempt and adjusted here.


Batch File - Find string and write filename to another file

Question: i have 100 text files, and i need to find a string in all the 100 text files and if the string found in any of the file then i want to write that filename in a log, and i have many strings to be found in a separate text file, kindly provide a batch code to achieve this. strings are in a text file like below example,
my_strings.txt
test1
test2
test3
please help to achieve the above using a batch file.
Comment: StackOverflow is not a code-writing service. Please read the [guide to asking a good question](http://tinyurl.com/so-hints) and show [what you have tried](http://whatyouhavetried.com).
Answer: Here a few points to get you started to solving this problem yourself.
Searching Files and Printing File names see Findstr
Type findstr /? and enter at a command line.
Pay special attention to the /M option flag.
SS64 Technet
Capturing Output see >> Redirection
Rob


How to interpret result of recvfrom (raw socket)

Question: I'm developing a custom packet sniffer in Python3.
It does not have to be platform independant. I'm using Linux.
The method I use is to recvfrom() from a socket (AF_PACKET, SOCK_RAW).
It works fine but I have problem with info returned by recvfrom().
recvfrom() returns a tuple with 5 components.
Example: ('eno1', 2054, 0, 1, b'\x00!\x9b\x16\xfa\xd1')
How do I interpret the last 4 components?
Where is it documented?
I prefer not to use libpcap or scapy.
OK! here's a code fragment:
    s = socket.socket( socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))    ...    packet,pktAdr = s.recvfrom(65565)
    print( 'pktAdr:'+str(pktAdr))
Thanks!
Comment: Where is it documented? How about on docs.python.org? Also, you need to show your code if you want help.
Answer: It's not documented on  docs.python.org so I did some research.
I'm now in a position to answer to myself.
The tuple returned by recvfrom is similar to a sockaddr_ll structure returned by the Linux kernel.
The tuple contains 5 components:
 - [0]: interface name (eg 'eth0')
 - [1]: protocol at the physical level (defined in linux/if_ether.h)
 - [2]: packet type (defined in linux/if_packet.h)
 - [3]: ARPHRD (defined in linux/if_arp.h)
 - [4]: physical address
The example provided in  the question can be decoded into:
 - 'eno1'
 - ARP protocol (0x806)
 - Incoming packet
 - Ethernet frame
 - MAC address
In case of a WiFi interface in monitor mode, the [3] element would be 803 (meaning 'IEEE802.11 + RadioTap header').
Hope this will help somebody
Comment: how did you get 5 elemets tuple if it should return (bytes, address)?
Comment: Sockets with SOCK_RAW option are poorly documented (in Linux and Python). Here's my understanding (beware I'm not a kernel specialist). In SOCK_RAW mode, the packet returned by kernel is prefixed with a SLL structure (also known as Linux Cooked Interface). In Python, the SLL is removed from the packet and the SLL information is decoded and provided in an array (described in my answer). Thus your Python program will receive 2 elements (packet and SLL array[5]).


Are there compare-based sorting algorithms that are faster than n lg(n) by some constant factor?

Question: Is it possible to have algorithms that sort using, say, $1/2 * N*lg(N)$ compares?
Essentially, I am confused because I have seen the $N * lg(N)$ lower bound on sorting written in both tilde and big-Omega notation.
Thanks.
Comment: Can you give some examples of where you saw these?
Comment: Related: https://stackoverflow.com/questions/67493308/why-is-it-impossible-to-find-a-specified-value-in-a-sorted-array-faster-than-ol/67493406#67493406
Answer: The complexity notation is defined only up to a constant factor. AFAIK the constant factor is almost never used in analytic complexity analysis. Most of modern sorting algorithms have complexity $K * N * \log N$ where $K$ is some constant number. This number is typically hard to compute analytically, so empirical approaches are used instead. More precisely, the complexity of $N \log N$ is only an approximation for a general case. Many algorithms (e.g. quick sort) have special cases where they perform significantly worse than average. So in reality $K$ is not really a constant, but effectively a constant as long as the data stays in a certain regime. In practice, people select data regime (e.g. random data, or almost sorted data, or reverse-sorted data), and evaluate performance empirically using multiple runs. From such runs, you can fit the curve and get approximate values for $K$ for each algorithm, and thus conclude that some algorithms are better than others. But, in general, there is no one constant for an algorithm. All state of the art sorting algorithms target very specific data regimes and perform significantly worse if not used for the right job


What was the first team to use the 1-3-1 Forecheck

Question: I'm a big hockey guy, but over the last 2 or so seasons I've seen the Tampa Bay Lightning use the incredibly boring 1-3-1 forecheck. As you see in the picture below, the Tampa Bay Lightning (blue) is using the 1-3-1 forecheck against the Philadelphia Flyers (orange).
I was wondering, who was the first team / coach that used the 1-3-1 forecheck?
Comment: I don't know alot on Hockey but I recommend you the read the following post - http://sports.yahoo.com/nhl/blog/puck_daddy/post/What-s-the-mystery-behind-the-Tampa-Bay-Lightnin?urn=nhl-270470
Comment: Thanks for the reply. But that explains how the Lightning are using it. I was just wondering who came up with this, in my opinion, lazy forecheck.
Comment: According to this [article](http://www.dispatch.com/content/stories/sports/2010/05/23/bouchers-system-is-unusual-effective.html), Guy Boucher's 1-3-1 forecheck "is believed to stand alone in modern professional hockey." Additionally, former Dallas Stars coach [Marc Crawford](http://en.wikipedia.org/wiki/Marc_Crawford) uses a 1-3-1 zone, but with the two outside defenders closer to the middle of the ice (unlike Boucher).
Comment: You are correct when you say that The Tampa Bay Lightning uses it right now, however they were not the team to first use it. 3 seasons ago when Guy Boucher implementedthe 1-3-1 into Tampa's defense, there was an outcry among the league. This happened because this forecheck is a very "sit and wait" boring way to play. Also is has not been used in a **VERY** long time. I know the answer is out there!!! But regardless, +1 for finding out that Crawford used it.
Answer: The New Jersey Devils used a similar setup back in the mid 90's. It was so revolutionary that it knocked fans out of their seats -- and into their beds ;)
The 1-3-1 is a type of neutral zone trap, which many teams will employ situationally at different points in the game. It is a natural evolution of the single forechecker on a penalty kill, and is often done late in games to preserve comfortable leads. As a result, I am almost certain you can see 1-3-1 variants at various points of games as far back as you can find professional hockey footage; as such, it is hard to figure out who used it first.
But as for using it as a general game strategy, probably go back to 1995 NJ Devils, or so.
Edit, for more detail:
The resurgence of the 1-3-1 can be linked to the removal of the two-line pass rule. Prior to the two-line pass removal, teams had to carry the puck at least over their own blue line before attempting to connect with another player near the offensive zone. Two forecheckers could successfully jam the breakout at the blue line.
However, with two-line pass permissible, many teams (cf. Blackhawks 2009) will position a winger just before the offensive blue line. During the breakout, a player will fire a hard pass towards this winger, who will tip it deep, thus avoiding an icing. Then, the remaining winger and center will attack the offensive zone with speed, which gives them an advantage over the defenders, who have to transition to the puck.
This causes a lot of "dump and chase" hockey that favors the offense -- the fast moving offence can either get to the puck first, or come into the defender hard.
The counter to this is to jam the passing lane through the neutral zone by stacking three defenders deep. The back defender can still get back to jam the puck in the corner if necessary.
Consequently, the 1-3-1 saw little use pre-lockout in the NHL, because it wasn't a useful counter strategy. You saw it in College hockey somewhat, which didn't have a two-line pass rule.
Comment: @Zack NJD played 1-3-1 a little different in that era. Check this video around the 0:48 and 1:10 marks: http://www.youtube.com/watch?v=PxtXCaIYjPE
It's hard to see, but you can see that they position the 3 neutral zone defenders just before the red line. Contrast that to your image above, where the 3 are just before their defensive blue line. Also notice how in your image above, Philly, the breakout team, has men on the boards at the blue line -- this is for the tip and chase scheme. TB is essentially canceling that. So the 1-3-1 has evolved after the 2004 lockout. Not all 1-3-1s are the same.
Comment: As for why more teams don't use 1-3-1... it varies. This strategy is defensive, so it limits scoring opportunities. TB leans heavily on its top line to generate offense. I'd bet that you see TB using less 1-3-1 when the Stamkos line is on the ice (no evidence to support, just a hunch). Also, TB hasn't had many stud defenders. Compare that to other top Eastern Conference teams: Boston has Chara and Seidenberg, Ottawa has Karlsson and Alfredson, Philly (had) Pronger, Pittsburg has Letang, etc. While some are defensive minded, and some offensive minded, a strong D-corps allows for other schemes.
Comment: AWESOME response! My only question is this. You stated that the 1995 New Jersey Devils were the first to master it, and now that I think of it, I agree (accepted answer!). But you also said that teams started using the 1-3-1 more after the two-line pass rule was removed, which was in **2004-2005**. Is this true? The only team that I have seen use this forecheck as their "main" defense. following this rule change is the Tampa Bay Lightning.
Answer: The Flyers went with the one forechecker against the Soviet Red Army team in 1976. Their zone defense clogged up both sides of the ice in the neutral zone. This helped to defend against a team of superior skaters, who were constantly in motion--passing and weaving their way up the ice.
Fred Shero later said the idea was to funnel the Soviets up the middle, and force them to rely on the lone puck carrier. If he were going to score--again, as Shero said--he'd have to first get by the center (Clarke, for example), somehow get past two defensemen, and then finally have to beat the goalie.
The Flyers must have really bought into the strategy, just like the Devils must have in the 90s. I assume it takes a certain group of guys to actually make these kinds of systems work.
I may be late to the game here. But I figured this was relevant, and now almost 40 years in the past.
Comment: The second part of this sounds like a quote. Can you source this? Otherwise, nice answer!
Comment: I remember that game - the Soviets has no clue what to do, and could not free-wheel around as they were accustomed to. Great strategy to deal with them. The NHL did not use similar tactics at the time, so Shero did not use it in normal games.
Answer: Years late. But I think the Devils' 90s trap was more of a passive 1-2-2 that would morph into an even more passive 1-4 when ahead. Julien uses a similar system.
If I recall correctly, the Flyers played a 1-4 against the Red Army, essentially lining 4 players in front of their own blue line.


Can't set field names with Given users behat call

Question: When I run the following behat test:
Scenario: Logs in to the site
  Given users:
  | name     | pass        | mail                  | roles         | status | field_fir | field_la |
  | someuser | sompassword | email@example.com | administrator | 1      | Some      | User     |
  Given I am on "/user"
  When I fill in "Username" with "someuser"
    And I fill in "Password" with "sompassword"
    And I press "Log in"
  Then I should see "Some User"
  Then I visit "user/logout"
  Then I should be on the homepage

I get the following error:
Given users:                                # Drupal\DrupalExtension\Context\DrupalContext::createUsers()
  | name     | pass        | mail                  | roles         | status | field_fir | field_la |
  | someuser | sompassword | email@example.com | administrator | 1      | Some      | User     |
  Warning: explode() expects parameter 2 to be string, array given in /var/www/xmd8/vendor/drupal/drupal-extension/src/Drupal/DrupalExtension/Context/RawDrupalContext.php line 316

The weird thing is if I remove field_fir and change Then I should see "Some User" to Then I should see "User, it works.  For some strange reason it doesn't like field_fir. I verified that field_fir and field_la have exactly the same settings.
Answer: I didn't think this was the issue but I recently upgraded to PHP 7.  I downgraded to PHP 5.6 and the issue went away.  I created an issue on Drupal.org.
Turns out that I was using an older version of drupal-extension.  Upgraded to and the issue went away:
"drupal/drupal-extension": "^3.2",


NT SQL Connecting

Question: how can i connect win nt sqlexpress?
i am using this code to connect.it works with any version and kind of sql.
internal string GetConnectionString()
{
      return "Data Source=" + "MyPC\SQLExpress" +
              ";Initial Catalog=Master;User ID=" + username +
              ";Password=" + password;
}

when connectiong with express ===>>>
Login failed for user 'NT AUTHORITY'. The user is not associated with a trusted SQL Server connection.
or
Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection.
or
Login failed for user 'sysadmin'. The user is not associated with a trusted SQL Server connection.
Comment: Have a look at the SqlConnectionStringBuilder ( http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnectionstringbuilder.aspx ). This is a better way of programmatically generating Connection Strings.
Answer: You need to add the following to the connection string:
Trusted_Connection=yes;

If you are trying to use windows credentials you will need to set:
Integrated Security=true;

This will use Windows authentication rather than SQL authentication. If you don't spesify a user name and password, the currently logged in user will be used implicitly.
See http://www.connectionstrings.com/sql-server-2005 for more details.

NT SQL Connecting


Prevent Squid Proxy to change the outgoing request's IP address as it's own

Question: We are trying to forward the traffic of our virtual machine to Squid Proxy. When the requests are bumped by Squid Proxy and leaves the proxy, all the request's source Ips are changed from the client's IP to the Squid Proxy's server's IP address.
Our requirement is to ensure the requests are ssl-bumped, but at the same time, we want the client IP to be preserved and not altered by Squid Proxy. Is there a way to add any tags over the config file(squid.conf), to prevent the client IP modification.
We need to client IP for evaluation of rules separately, and while we need to ssl-bump, we also need the client source IP as the outgoing request's signature.
Answer: Proxy servers work by accepting connections from clients and sending requests on behalf of the clients to destination servers.
The following happens at TCP level:
<code>Client --> TCP connection --> Squid --> TCP Connection --> destination
</code>
Client opens the TCP connection to Squid using its own IP address. Squid opens the TCP connection to destination server using its own IP address.
This means that the TCP connections from the proxy server always have its own IP address. This is fundamental operating mode of TCP/IP protocol and one cannot work around it.
Answer: I think you want an x-forwarded-for header... You then read the header on the next hop and that header will contain the original IP address rather than the squid server. It's always going to replace the source as its own... But you can have it write the original as a header you can pull later. it's hwat you do with web proxies etc.


Unpublish node instead of deleting it

Question: In my web application, I would like to prevent accidental or unauthorized deletions or modifications (e.g. a student steals teacher's credentials and deletes or changes bad marks). But, at the same time, I'm trying to keep the interface as easy and straightforward as possible.
I've already enabled node revisions, so if a mark is changed we can restore the previous state, but I also hid the revision message, because it confuses non-techie teachers.
The next step would be transparently prevent mark deletion. The user should see and use the "Delete" button and the deletion confirmation page, but when confirming, Drupal should not delete the node and just unpublish it. Is this possible to do?
(I would then create a view to see/administer the revised/unpublished nodes, to check unauthorized modifications)
Answer: Use hook_node_delete . This hook is being invoked right before node deletion.
function YOURMODULE_node_delete($node) {
  if ($node->type == 'NODETYPE') {
    $node->status = 0;
    node_save($node);
    unset($node);
  }
}

Comment: So, `unset($node);`, by unsetting the node variable, prevents the node being removed from the database?
Comment: yes, this way the argument  (object $node) is not being passed to hook_entity_delete() which makes actual deletion, so nothing actually happends
Comment: I speaked too early. This method doesn't work, the node will still be deleted, so I'm reopening the question!
Answer: I fixed the unset() problem using a redirect

$node->status = 0;
node_save($node);
drupal_set_message(t('The node has been deleted.'));
drupal_goto('node/');
exit();

Unpublish node instead of deleting it


Is there a slowest rate of divergence of a series?

Question: $$f(n)=\sum_{i=1}^n\frac{1}{i}$$
diverges slower than
$$g(n)=\sum_{i=1}^n\frac{1}{\sqrt{i}}$$
, by which I mean $\lim_{n\rightarrow \infty}(g(n)-f(n))=\infty$. Similarly, $\ln(n)$ diverges as fast as $f(n)$, as $\lim_{n \rightarrow \infty}(f(n)-\ln(n))=\gamma$, so they 'diverge at the same speed'.
I think there are an infinite number of 'speeds of divergence' (for example, $\sum_{i=1}^n\frac{1}{i^k}$ diverge at different rates for different $k<1$). However, is there a slowest speed of divergence?
That is, does there exist a divergent series, $s(n)$, such that for any other divergent series $S(n)$, the limit $\lim_{n \rightarrow \infty}(S(n)-s(n))=\infty$ or $=k$? If so, are there an infinite number of these slowest series?
Comment: The answer is no. If you would like to see this yourself, given any arbitrary divergent series, try to come up with another divergent series that diverges more slowly.
Comment: See, for example, the third paragraph of this question: http://mathoverflow.net/questions/49415/nonexistence-of-boundary-between-convergent-and-divergent-series
Comment: There you go, I was searching for you the following nice paper: http://math.depaul.edu/mash/worstconv.pdf
Comment: @LordSoth I thought so, but how do I know for sure there will always be a slower one?
Comment: I've added the tag "asymptotics" because this seems to be what you are getting at - the order of growth of such series, and related questions is a huge subject, and there is a whole set of notation ("Big O" and "little o") around such comparisons. Someone may post some better links to basic material. But note, for example, that $f(n)$ grows like $\ln (n)$.
Comment: See also (for example) http://math.stackexchange.com/questions/417178/lim-n-rightarrow-inftyn-ln-na-n-0-implies-sum-a-n-converges/417222#417222
Answer: The proof in the paper ``Neither a worst convergent series nor a best divergent series exists" by J. Marshall Ash that I referenced above is so nice that I wanted to reproduce it below before it gets lost on the Internet.
$\bf{Theorem: }$ Let $\sum_{n=1}^{\infty} c_n$ be any convergent series with positive terms. Then, there exists a convergent series $\sum_{n=1}^{\infty} C_n$ with much bigger terms in the sense that $\lim_{n\rightarrow\infty} C_n/c_n = \infty$. Similarly, for any divergent series $\sum_{n=1}^{\infty} D_n$ with positive terms, there exists a divergent series $\sum_{n=2}^{\infty} d_n$ with much smaller terms in the sense that $\lim_{n\rightarrow\infty} \frac{d_n}{D_n} = 0$.
$\bf{Proof: }$ For each $n$, let $r_n = c_n + c_{n+1}+\cdots$ and $s_n = D_1 + \cdots + D_n$. Letting $C_n = \frac{c_n}{\sqrt{r_n}}$ and $d_n = \frac{D_n}{s_{n-1}}$, then $\lim_{n\rightarrow\infty} \frac{C_n}{c_n} = \lim_{n\rightarrow\infty} \frac{1}{\sqrt{r_n}}=\infty$ and $\lim_{n\rightarrow\infty} \frac{d_n}{D_n} = \lim_{n\rightarrow\infty} \frac{1}{s_{n-1}} = 0$, so it only remains to check $\sum C_n$ converges and that $\sum d_n$ diverges. To see that this is indeed the case, simply write $C_n = (1/\sqrt{r_n})(r_n-r_{n+1})$ and $d_n = 1/s_{n-1}(s_n-s_{n-1})$; observe that $\int_0^{r_1} 1/\sqrt{x}dx<\infty$ and $\int_{s_1}^{\infty} 1/xdx = \infty$; and note that the $n$th term of series $\sum C_n$ is the area of the gray rectangle in Figure 1a, while the $n$th term of series $\sum d_n$ is the area of the gray rectangle in Figure 1b.
Comment: Ash's paper is on JSTOR  reference being The College Mathematics Journal, Vol. 28, No. 4 (Sep., 1997), pp. 296-297
with the stable URL being http://www.jstor.org/stable/2687153
Answer: Just to expand on Raymond Manzoni and to partially answer your question with one of the coolest things I ever learned in my early childhood (as a math undergrad),
consider the family of series,
$$\begin{eqnarray}
\sum \frac{1}{n}&=&\infty \\
\sum \frac{1}{n\ln(n)}&=&\infty \\
\sum \frac{1}{n\ln(n)\ln(\ln(n))}&=&\infty
\end{eqnarray}$$
and so on. They all diverge and you can use the integral test to easily prove this. But here is the kicker. The third series actually requires a googolplex numbers of terms before the partial terms exceed 10. It is only natural that if natural log is a slowly increasing function, then log of log diverges even slower.
On the other hand, consider the family,
$$\begin{eqnarray}
\sum \frac{1}{n^2}&<&\infty \\
\sum \frac{1}{n(\ln(n))^2}&<&\infty \\
\sum \frac{1}{n\ln(n)(\ln(\ln(n)))^2}=38.43...&<&\infty
\end{eqnarray}$$
all converge which can be easily verified easily again using the integral test. But the third series converges so slowly that it requires $10^{3.14\times10^{86}}$ terms before obtaining two digit accuracy presented. Talking about getting closer to the "boundary" between convergence and divergence. Using this you can easily make up a series to converge or to diverge as slow as you want. So to answer your question, no there is no such thing as "the slowest diverging series". Any slowly diverging series you pick, we can come up with one diverging even slower.
Reference:
Zwillinger, D. (Ed.). CRC Standard Mathematical Tables and Formulae, 30th
ed. Boca Raton, FL: CRC Press, 1996.
Comment: "arbitrarily slowly": I see that each converges more slowly than the last, but I don't know what "arbitrarily slowly" means in the context of the question. Does it mean that given an arbitrary divergent series $\sum\limits_{n=1}^\infty a_n$ of positive numbers, there exists a series in your set of examples that diverges more slowly?
Comment: "So to answer your question, no there is no such thing..." That is shown in answers of Clement C. and Lord Soth.  The examples in this answer are interesting, but I do not see how they exactly support the assertion.
Comment: I am just giving a set of examples which include series converging and diverging arbitrarily slowly. This isn't really a proof, like I said, just examples.
Comment: To answer to the first comment, this reply totally helped me to understand the choosen answer by providing a way to construct such 'arbitrary close' series.
Answer: The Cauchy condensation test allows to find an infinity of slower divergent series :

$\displaystyle \sum \frac 1n\;$ behaves like $\;\displaystyle \sum 2^n \frac 1{2^n}=\sum 1\;$ which diverges
$\displaystyle \sum \frac 1{n\ln(n)}\;$ behaves like $\;\displaystyle \sum 2^n \frac 1{2^n\;n}=\sum \frac 1n\;$ which diverges
$\displaystyle \sum \frac 1{n\ln(n)\ln(\ln(n))}\;$ behaves like $\;\displaystyle \sum 2^n \frac 1{2^n\;n\ln(n)}=\sum \frac 1{n\ln(n)}\;$ which diverges... QED.

(from Chaitin's book 'Algorithmic Information Theory')
Comment: Interesting example of something, but not an answer to the question.
Comment: @JonasMeyer: Well yes it shows just a rather neat family at the 'border of divergence' (and there is a corresponding family on the other side...).
Comment: From this simple point of view the different series proposed will approach $0$ from the negative side and the replacement of any 'exterior' term $n, \ln(n), \ln(\cdots\ln(n)\cdots)$ by the same term to a power larger than $1$ bring it on the positive convergent side. But I can't be satisfied with this $1D$ picture nor with the infinite (countable) dimensional view of all the possibles elementary functions of $n$ ; at least a second dimension with the value of the limit would make the 'thing' nicer but oblige me to define a finite limit for all divergent series. Fine I'll have to read Hardy !
Comment: RaymondManzoni: I agree it is neat.  What exactly is meant by 'border of divergence'?
Comment: @Jonas: It was just a picture. Formalizing this would ask for much more work and start with something like "Consider the space of elementary functions of $n$ from the point of view of differential algebra that is the logarithmic, exponential and algebraic extensions of (say) the field of rational functions over $\mathbb{Q}$ (still not considering all the series and omitting the even/odd $n$ sequences and so on). Each of these series should have a 'speed' (to be defined...) of convergence or divergence with $0$ between the two sets.
Comment: I just spent a while trying to write this up for a different question and got bogged down in whether it mattered that $\log\log\log\log n$ isn't defined for $n < e^e$ :/
Comment: @BenMillwood: Well we are interested only by convergence or divergence here so that any finite lower bound doesn't matter compared to the infinite series. Anyway (like Chaitin) I didn't indicate the bounds to let the reader take care of the details...
Answer: If you have $(a_n)_{n\in\mathbb{N}}$ a divergent series:
$$\sum_{n=0}^N a_n \xrightarrow[N\to\infty]{}\infty$$ with $a_n > 0$ for all $n\in\mathbb{N}$), consider the series with term  $$b_n\stackrel{\rm{}def}{=}\frac{a_n}{\sum_{k=0}^n a_k}.$$
One can show that the series $\sum b_n$ diverges.
Comment: Mmmh... there should be a lot of answers to this on the site, but essentially, you can first see that is $b_n\not\to 0$, it's settled; so assume $b_n\to 0$. Then $b_n\sim -\ln(1-b_n)$, and writing further:
$$-\ln(1-b_n) = -\ln(1-\frac{a_n}{A_n}) = -\ln\frac{A_{n-1}}{A_n} = \ln A_n - \ln{A_{n-1}}$$ where $A_n=\sum_{k=0}^n a_k$. Now, since $\sum a_n$ diverges, $\ln A_n\to\infty$, and by telescoping the terms this means that the series $\sum(-\ln(1-b_n)) = \sum ( \ln A_n - \ln A_{n-1} )$ diverges; therefore, since $b_n\sim -\ln(1-b_n)$, the series $\sum b_n$ diverges as well.
Comment: I don't know how you could begin showing that (I know very little about asymptotics). Could you hint or provide a link?
Answer: Suppose there is a slowest-diverging series. Call it $(a_n)$.
Define $A(n)=\sum a_n$. Thus, $A(n)$ is a slowly-diverging function.
Define $B(n)=\ln A(n)$. Thus, $B(n)$ diverges even slower than $A(n)$.
Define $b_n=B(n)-B(n-1)$. Thus, $(b_n)$ is an even slower diverging series. Contradiction.
Comment: @Gary You're probably right. (I wrote this eight years ago.)
Comment: By "series" you mean "sequence", right?
Comment: I don't think so - $\sum b_n$ is what goes to infinity here, not $b_n$.
Comment: I guess it is an abuse of notation, e.g., $(a_n) $ should be $\sum_{n=0}^\infty a_n$ and $A(n) $ should be $\sum_{k=0}^n a_k$.
Answer: Let $A(n)$ be an Ackermann function, let $1_{A}$
be its indicator function (i.e $1_{A} (x)=1$ if $A(n)=x$ for some $n$; $0$ otherwise). Then $\sum 1_{A}$ diverges. A one that diverges faster but more interesting is $s(0)=1$, $s(n+1)= s(n)+ \frac{1}{p_{n}s(n)}$   where $p_{n}$ is the nth prime.
Comment: How does this address the question given by the OP?
Comment: Thinking outside the box. One is the slowest concrete example posted. It also shows the futility of such search. The other also is slow and uses some neat math.
Comment: Don't presume that I gave you the downvote. I didn't.
Comment: I didn't; thanks for the feedback.
Comment: Upvote (+1)! The answer makes sense. I'd never heard of the Ackermann function. Where does this come up?
Comment: @user139964 This is interesting. Can one construct an infinite sequence of series, each term diverging slower than the last, using the Ackermann function?


KoGrid JSON Dynamic widgets, with nested server calls

Question: I am working on a dashboard where I am using KOGrid, my idea is to dynamically create a bunch of widgets and bind data into the kogrid for each widget. I have two controllers (MVC4), where the fist returns a list of widget names and the second returns results (datatable) for each widget in JSON format. I am using JSON.Net JsonConvert to convert the results to json format then passing as JSON content result.
I am getting  "Uncaught Error: The argument passed when initializing an observable array must be an array, or null, or undefined".
Is it the nested JSON calls a problem? What is the most reliable way to handle Arrays of complex objects and binding them to multiple grids. Does it help if I use knockout.mapping?
var widgets = {};
var Widget = function (id, data) {
    this.id = ko.observableArray(id || []);
    this.data = ko.observableArray(data || []);
};

var ViewModel = function (data) {
    var self = this;

    self.widgets = ko.observableArray(
                        ko.utils.arrayMap(data, function (i) {
                            return new Widget(i);
                        }));

    $.getJSON('Widgets/Get', null, function (data) {
        $.each(data, function (index, item) {
            $.getJSON('Home/GetWidgetDetails?widgetName=' + item.WIDGET_NAME,
                    function (result) {
                        self.widgets.push(new Widget({ id: item.WIDGET_NAME, data: result }));
                    });
        });
    });

};

 ko.applyBindings(new ViewModel(widgets));

Comment: The error was due to id being oservablearray instead of just observable. The error is gone after fixing it but I still don't see the grid populated, no idea what is wrong in the HTML below?

Answer: I'm guessing you are getting this error on the data prop in the Widget class? If so, then yes it is probably the JSON structure of the data array. If they are complex objects themselves, you should repeat the same process you did for Widgets.

Make an object definition for these items. Perhaps WidgetItem.
use ko.utils.arrayMap to map the data array onto WidgetItem.
Success.

Without seeing the structure of data though, I can't give you anything more specific.
Comment: Each widget is different, say one is customers and the other is orders. I have a generic MVC controller which accepts widget name and it will automatically identify the sql behind it, executes it and returns the results in JSON format (using newtonsoft JsonConvert). KOGrid is an awesome control to bind anything on its way but I think it is knockout which is blocking. I can share the code somewhere if that helps.
Comment: It might. But, regardless, if you want to place these different widgets in an observableArray, you are going to want to have a type that represents then. Try to figure out common properties, and maybe make some of the names generic enough to handle your different widgets. I can help you with this if you give me some concrete examples.
Comment: I fixed it. https://github.com/sergun/Knockout-MultiModels.         $.each(data, function (index, item) {var li = "";div.append(li);
$("#" + item.WIDGET_NAME + "_li").append(" ");
$.getJSON('Home/GetWidgetDetails?widgetName=' + item.WIDGET_NAME,function (data) {ko.attach(item.WIDGET_NAME + "Tittle", {Title: ko.observable(item.WIDGET_NAME + "Tittle"),myObsArray: ko.observableArray(data) });});});


Change significant figures reporting for default floats

Question: Is there anyway to set the "default" number of significant figures for python floats or numpy floats?
I hate wrapping the 100+ print statements I have with custom rounding code, that sucks.
Comment: Instead of changing the default printing style, you could replace the `print` function with your custom version that handles floats differently.
Comment: Print controls for Python floats and numpy arrays are different.
Answer: There are few ways to approach the problem.

You can write a definition where you will pass your output statements and round the numbers to floating points you need in your definition.
As mentioned in the comment above by  GZ0 and answered here
Set global output precision python
you can change your print function

def print(args):
builtins.print(("%.2f" % a if isinstance(a, float) else a
for a in args))


javascript : parse float with string prefix in both start and end

Question: how to convert this string to float?
i want result 900.50
case-1: var convertThis = 'any string here 900,50 also any string here';
case-2: var convertThis = 'any string here 900.50 also any string here';
How to do this?
Answer: You could do this :
var num = parseFloat(convertThis.replace(/[^\d\.,]/g,'').replace(/,/,'.'));

But be aware that this would break as soon as you have more than one number or dot in your text. If you want something reliable, you need to be more precise about what the string can be.
Supposing you'd want to extract all numbers from a more complex strings, you could do
var numbers = convertThis.split(/\s/).map(function(s){
    return parseFloat(s.replace(',','.'))
 }).filter(function(v) { return v });

Here, you'd get [900.5]
Answer: Try following code:
var text = 'any string here 900,50 also any string here';
var matched = text.match(/\d+[,.]\d+/)[0].replace(',', '.');
var num = parseFloat(matched, 10);
console.log(matched);
console.log(num);

prints:
900.50
900.5

Answer: var myFloat = +(convertThis.match(/\d+[,\.]?\d+/)[0].replace(",","."))
Comment: then use the parseFloat(myFloat) ?
Comment: Nope, no need for that, the `+` before the paranthesis already "casts" the string to a number


Find largest prime factor for the number 600851475143L

Question: I am getting output for 13195L, 24L, and 23L. But I am not getting output for 600851475143L. The system is going into infinite loop. Please help me identify what is wrong with my code.
package problem3;

public class problem3_version1 {
    public static void main(String[] args) {
        long dividend = 600851475143L;
        // long dividend=13195L;
        // long dividend=24L;
        // long dividend=23L;
        int num_of_divisors = 0;
        for (long i = dividend - 1; i >= 2; i--) {
            System.out.println("i =" + i);
            int count = 2;
            for (long j = 2; j < i; j++) {
                if (i % j == 0)
                    count++;
                if (count == 3)
                    break;
            }
            if (count == 2) {
                if (dividend % i == 0) {
                    num_of_divisors++;
                    System.out.println("Highest factor is " + i);
                    break;
                }
            }
        }
        if (num_of_divisors == 0)
            System.out.println("The number is prime");
    }
}

Comment: Your program is not buggy.  Strictly speaking, it does not contain an infinite loop.  The problem is that it is exceedingly inefficient.  It will do something approaching `600851475143L * 600851475143L` divisions ... if the number has no factors.  That is a very, very large number of operations.  Solution: do some research on *efficient* ways to test for primality.
Comment: I would look at your `for (long j = 2; j < i; j++) {` loop; you are incrementing `j` but seeing if it less than `i`.
Comment: Or just, *please*, do some **research**, e.g. a web search for [`Find largest prime factor`](https://www.google.com/search?q=Find+largest+prime+factor), which will give you a ton of articles on the topic.
Comment: I am finding if "i" is prime or not. So not considering 1 and i itself in the loop. And if its not prime checking if its a factor of the dividend.
Comment: Yes, got it. Thank you for the input.
Answer: try this solution :
public class LargestPrimeFactor
{

    public static int largestPrimeFactor(long number) {
        int i;

        for (i = 2; i <= number; i++) {
            if (number % i == 0) {
                number /= i;
                i--;
            }
        }

        return i;
    }

    public static void main(String[] args) {
        System.out.println(LargestPrimeFactor.largestPrimeFactor(600851475143l));
    }

}

The problem was because you have nested loop with very big number, and that what made the loop
Comment: Not an infinite loop.  The loop would terminate ... eventually.


gvim: delay after hit-enter-prompt

Question: Sometimes after hitting Enter when the hit-enter-prompt appears, gvim hangs several seconds before redrawing the screen. Precisely, 1 second after hitting Enter, the cursor appears on the screen (i.e. inside the buffer window), after 3 additional seconds the screen is redrawn.
This happens if I jump to a tag (with Ctrl-]), but not if I execute a command like :!ls. Both actions show up the hit-enter-prompt, but only the former one leads to the delay.
What is the reason for this delay?
Edit:
I noticed that the problem appears only when jumping to a tag and the line "tag 1 of 2 or more" does not appear. Instead a message like

src/module.cxx 483L, 15688C
  src/module.cxx 483L, 15688C
  Press ENTER or type command to continue

is shown, where I don't know what the numbers mean (these are not the line numbers of the tags or of the line from where I jump). The filename is one of the files in which the tags are found. The other tag-containing file (src/module.h) does not appear.
Unfortunaltely, I don't know how to provoke this situation.
Comment: Not sure if it's relevant, but is `lazyredraw` set? Assuming no scripts are executing, that shouldn't cause a problem.
Comment: `lazyredraw` is not set here.
Answer: It might be due to a setting or script you have installed. You can debug your scripts by disabling some plugins to figure out what's happening.
Use default vim settings
gvim -u NONE -U NONE

Issue still occurs: likely happens in a default vim install. May be a bug.
Issue is fixed: it's an installed config setting or plugin. Try disabling plugins.
Disable all plugins
gvim --noplugin

Issue still occurs: config setting (vimrc or gvimrc). Try commenting out parts of your rc files to see if it still happens.
Issue is fixed: it's an installed plugin. Try disabling plugins until it's fixed.
Other notes
Some Linux distributions may include custom settings for vim (such as debian.vim) that may make your results different from others.
You might also get slightly different results if you remove your vimfiles. I'm not sure. You can easily stash up all of your vimfiles:
mkdir ~/bak
mv .*vim* ~/bak

Comment: I have `set insertmode` in my .gvimrc. If I start gvim (with only a filename as parameter), then deactivate `insertmode`, then jump to a tag, the hit-enter-prompt is not shown, although there are 2 tags. When `insertmode` is active, the hit-enter-prompt is shown on the same action (same source file, same tags file, same tag). The same is true for `showmode`: when activated, the hit-enter-prompt disappears, when deactivated, the hit-enter-prompt appears.
Comment: Does that behaviour still occur with `gvim -u NONE -U NONE --cmd "set nocp insertmode" Somefile.java` (vim defaults except turn on insertmode)? If I do that and have some java source files with the same method name reused and type `` to jump to a tag, vim jumps and says "tag 1 of 2 or more". I see no delay.
Comment: Also, I don't think showing all matching tags is the default behaviour. See `:help tag`. Do you have a map to `:tselect` or have `:set cscopetag`?
Comment: One thing to check that might be the cause of your delay is a map for hitting Enter. Type `:map ` after you see the delay. If `` is mapped in a chord, then vim will wait for `timeoutlen`. (So `:map a` will cause vim to wait to see if you're going to type `a`.) [See also this answer](http://stackoverflow.com/q/2158516/#2158610) that mentions delays related to esckeys (which you're using for the arrow keys in insertmode).
Comment: With `gvim -u NONE -U NONE --cmd "set nocp insertmode"` I see no hit-enter-prompt, i.e. no "tag 1 of 2 or more" message. I also see no delay in this case, as the delay only appears when the hit-enter-prompt is shown.
Comment: `cscopetag` is NOT set here. Also, I have no map to `:tselect` (as far as I can tell, using the `:map` and `:imap` commands).
Comment: No, I have no mapping to . Also, setting `timeoutlen` to a small (10 ms) or big (10000 ms = 10 s) value does not change anything, so I guess there is no mapping problem.
Comment: Found new fact: if I get a "tag 1 of 2 or more" message, hitting Enter does NOT lead to a delay. But in certain circumstances I get a message like "src/module.cxx 483L, 15688Csrc/module.cxx 483L, 15688C" (don't know how to insert a newline in this comment). I don't know what the numbers mean (these are not the line numbers where the tag is located, nor the line numbers from where I jumped), but the filename names the file to where I jump. Also, the line "tag 1 of 2 or more" is NOT shown in the message. The delay seems to only appear in this case, but I don't know at all how to provoke it.
Comment: That would be good to add to the question. I assume none of your source files have 15k columns, so maybe there's something wonky about your tags file. How big is it? Are you using a recent version of exuberant ctags? Can you replicate your problem if you only build tags for a few files?
Comment: My tags file is 115k in size. I use Ctags 5.8, and used the switch -n. If I build the tags only for a few files (tag file = 22k), I get the same problem. Omitting -n doesn't help. I assume that there is no problem with ctags.
Comment: I just realized that my tselect/cscope comment isn't what you're seeing. You never see a list of tags so of course you aren't using those. What text is causing the Press Enter prompt? Is it always only the double print of `src/module.cxx`?
Comment: As far as I can tell, yes. When I position the cursor over a class name and hit  I see that dubious 2 lines (note that in fact, there are 2 tags - the class and its constructor). Do you have an idea what that output means or why it is printed at all? I don't understand it. Perhaps when I knew what it means I could make further investigations...
Comment: I just realized that `483L, 15688C` means that your file src/module.cxx is 483 *lines* and contains 15688 *characters*. Are those values correct?
Comment: You could try grabbing the [latest vim code](http://www.vim.org/mercurial.php) and see if it was a bug that's been fixed. If that doesn't help, [the vim_use mailinglist](http://groups.google.com/group/vim_use) might have some better ideas for troubleshooting. See also `:help bugs` for reporting issues.
Comment: Yeah, you're right: 483 lines and 15688 characters. And I'll try the latest version at home next days (I'm @work here and should not invest too much time ^^). Thanks for your help so far. If I found a solution, I will post it here.

gvim: delay after hit-enter-prompt


Convert sql function to c#

Question: Hi all I am having an sql function where some part of it is used to convert varchar to datetime:
IF @dt IS NULL
    SET @dt = GETDATE();

SET @PDate = CONVERT(DATETIME, @PValue + '-' + CONVERT(NVARCHAR(4),YEAR(@dt)));
IF @PDate > @dt
    SET @Year = YEAR(@dt) - 1;
ELSE
    SET @Year = YEAR(@dt);

RETURN (SELECT CONVERT(DATETIME, @PValue + '-' + CONVERT(NVARCHAR(4),@Year)));

Can some one help me?
My value of @PValue is 01-APRIL.
I just declare Datetime as follows in my C# code:
DateTime? dt=null;

if(!(dt.HasValue))
    dt=DateTime.Now;

But after this I was confused to do the remaining.
Comment: If I am not wrong `SET @PDate = CONVERT(DATETIME, @PValue + '-' + CONVERT(NVARCHAR(4),YEAR(@dt))); ` will display `01-APRIL-YEAR`
Answer: You could do with a year field in order to convert properly, I can't see 01-APRIL being a valid date. 01-APRIL when? Once you have a valid structured datetime, your C# code will read:
string pValue = @"01/04";
DateTime dt = new DateTime();
DateTime now = DateTime.Now;
pValue += string.Format(@"/{0}", now.Year);
int year = 0;
if (DateTime.TryParse(pValue, ref dt))
{
  if (dt > now)
    year = dt.Year -1
  else
    year = dt.Year;
}

Comment: Hi my pvalue is `01-APRIL` this I need to convert to datetime
Comment: @Learner you can't have that as `DateTime` not even a `Date`, not reliably anyway because there isn't a year.
Comment: @Learner I see that you are getting your `Year` from `DateTime.Now` so you date isn't `01-APRIL` it is `01-APRIL-2013`
Answer: If you want to convert a string to a date, then you should use the DateTime.Parse method. You should probably declare the parameter as a string, not DateTime?.
Comment: Hi for the function I will pass dateTime value that is not the issue here as per said my value will be `01-APRIL` how can I convert this datetime in C# as per the query in SQL
Answer: Just call DateTime.Parse or DateTime.ParseExact, never parse dates yourself because you will have a bug in your implementation. It's one of life's certainties, like death and taxes.
If you use Parse and not ParseExact be sure to specify a CultureInfo because 03/06/99 will be parsed differently if the current locale is English (United States) as opposed to English (United Kingdom).
Answer: something like below
DateTime? dt = null;
        int pYear = 2012; // assuming pYear is getting populated from somewhere
        int finalYear = 1900; // initilizing some value to it;
        if (!(dt.HasValue))
            dt = DateTime.Now;
        var year = dt.Value.Year;
        var pValue = "1-Apr";
        if (pYear > year)
            finalYear = pYear - 1;
        else
        {
            finalYear = year;
        }

        return pValue + finalYear.ToString();

Convert sql function to c#


Linear Algebra Dimension

Question: Let  $L(U,V)$ = $\{T:U\rightarrow V\ :\ T\ \text{linear}\},$ and dim $(U)=n$, dim $(V)=m$. Then show that $$
\dim L(U,V) = mn.
$$
I don't know how to begin and I already searched the internet to find something, but  could not. Please if anybody could help.
Comment: http://math.stackexchange.com/questions/551031/if-v-k-is-a-finite-dimensional-vector-space-then-the-space-of-all-linear-tr
Comment: Are you familiar with the fact that linear mappings are in one-to-one correspondence with matrices? That helps a lot here.
Answer: Hint. Let $e_1,\ldots,e_n$ be a basis of $U$ and $f_1,\ldots,f_m$ a basis of $V$.
Set $T_{ij}:U\to V$ to be the linear transformation such that
$$
T_{ij}(c_1e_1+\cdots+c_ne_n)=c_if_j.
$$
Then $T_{ij}$, $i=1,\ldots,n$, $j=1,\ldots,m$ form a basis of $L(U,V)$.

Linear Algebra Dimension


How to specify many default values for several classes, functions in Python?

Question: This is not my code but I have a similar problem:
https://gist.github.com/songrotek/0c64d10cab5298c63cd0fc004a94ba1f
There are many parameters I want to edit and change in one central place (in the most semantic way possible) and then use to construct classes, run the main program etc..
Because I don't want to put everything in one file (it's too big, I need modules and different namespaces) and declare global variables.
Having to edit the code for setting some parameters doesn't seem to be a good solution.
class Example:
    def __init__(self, *initial_data, **kwargs):
        for dictionary in initial_data:
            for key in dictionary:
                setattr(self, key, dictionary[key])
        for key in kwargs:
            setattr(self, key, kwargs[key])

Using constructors like this for being able to use dictionaries for init doesn't seem to be transparent enough.
What are possible solutions for this problem? It's still code mostly used by me and rather small, so it should be a good compromise between lightweight/appropriate for the size and still easy to use and comfortable.
Comment: You can pass a dictionary into `**kwargs`.  For example, if your dictionary was called `d` you could pass `**d`.  Is that what you mean?
Comment: Not sure whether I understand your concern correctly, but does it go into the direction of what's been discussed here: [http://stackoverflow.com/questions/6198372/most-pythonic-way-to-provide-global-configuration-variables-in-config-py]
Answer: A common way to handle this is a configuration file that is being read by ConfigParser.
Here is an example:
# configuration.ini
[Example1]

constant1: 1
constant2: 2

[Example2]

c1: 'one'
c2: 'two'

# test.py
from ConfigParser import ConfigParser

conf = ConfigParser()
conf.read('configuration.ini')

def configure_instance(instance):
    for i in conf.items(type(instance).__name__):
        setattr(instance, i[0], i[1])

class Example1(object):
    def __init__(self):
        configure_instance(self)

class Example2(object):
    def __init__(self):
        configure_instance(self)

e1 = Example1()
e2 = Example2()
print vars(e1)
print vars(e2)

# {'constant1': '1', 'constant2': '2'}
# {'c2': "'two'", 'c1': "'one'"}

Comment: The drawback is that with the ini - file approach, all variables are strings. You could switch this to YAML (http://pyyaml.org/wiki/PyYAML) which autodetects the variable types.


NSTableView: Renaming one row results in renaming two rows at once

Question: My Setup: I have a cell based NSTableview. The columns are bound  to values of an NSArrayController (key-value-binding in Interface Builder). The NSArrayController contains a set of NSManagedObjects (CoreData entity (Device)). The tableview's sortDescriptor is bound the the one of array controller.
My Issue: When I sort the tableview by name and then rename a row, the NSArrayController rearranges it's content and the nstableview changes the order of the rows. I want that. However, instead of just renaming one row, it renames two rows. This only happens if the order of the rows has to change because of its sort descriptor.
I.e. The table view looks like the left row below. If I rename f to b it changes to the row on the right
<code>a         ->           a
c         ->           b
d         ->           b
e         ->           c
f         ->           d
</code>
I set a breakpoint at the setter of the name property to see what sets the name a second time. However I don't get much smarter by looking at the stack.
This is the first time the setter is being called:

This is the second time:

I have build a sample app, however I cannot reproduce this behavior. I must be doing something wrong somewhere, but I can't find what. Tried everything all day long. Does anyone have an idea, what is going wrong here? Thanks :)
Comment: A cell based tableview binds content, selectedIndexes and sortDescriptors automatically.
Comment: The tableview is calling endEditing/makeFirstResponder twice. It might be something with `editedRow`. Did you subclass NSTableView? Which delegate and datasource methods did you implement?
Comment: @Willeke Are you sure? If I remove the bindings for selectedIndexes and sortDescriptors, the array controller's selection does not change. Nor is the tableview sortable. No Subclass. I removed the delegate and datasource, still the same. The tableViews's data is the content property set to arrayController.arrangedObjects.
Comment: I have found the source of the issue. Thanks for your time Willeke. :)
Answer: I have found the source of the issue:
I am not sure what the exact reason ist, but there were two rings I did wrong.

I bound the NSTableView's content property the NSArrayController's arrangedObjects. I removed that. That was when I noticed something else must be wrong, since now no data at all has been visible in the NSTableView:
The NSArrayController was not yet ready when the NSTableView was
being setup with the bindings. So the columns have been bound to
nil. I am now binding the column values programmatically to the NSArrayController's arrangedObjects as soon as arrangedObjects is ready.
Comment: The content of the arraycontroller can be nil when the bindings are set up. That's the fun of bindings, they observe changes.
Comment: Somehow they didn't. Hmm.


Exercise: Should we trust our naive bayes or a human "guru"?

Question: This is from an exercise about the cost of missclassifications using a Naive Bayes classifier. We know that for some example which has three binary features $x_1,x_2,x_3$ the posterior is $P(Y = 1 | X_1 = x_1, X_2 = x_2, X_3 = x_3) = p$. There are two classes, y $\in$ {0,1}.The cost for misclassication is $c_m$. For each prediction we can either trust our naive bayes classifier, or ask a human expert with cost $c_h $ < $c_m$, who is 100% accurate.
The question is which of the two procedures would  minimize the expected cost?
Edit: I think part of the confusion we are having is this: We have two possible outcomes. So normally, if y = 1 is true, but we decide on state y = 0, i.e. a = 0, the cost should be $p*c_m$. If in contrast  y = 0 is true, but we decide on state y = 1, i.e. a = 1, the cost should be $(p-1)*c_m$. Would choosing the optimal action a* not imply by itself then that we are deciding based on the higher posterior?
BUT, the the answer below suggests that we already should take into account that the classifier goes with the maximal posterior before searching for optimal action a*: In this scenario, the cost would always be $c_m*min(1-p,p)$. For instance, if p = 0.45, the Naive Bayes decides on y = 0 (because of the higher posterior for y=0), so the cost is $0.45*c_m$, but even if p = 0.55, the cost would be $0.45*c_m$ since now Naive Bayes is choosing y =1.
Comment: Can you explain your reasoning behind the expected cost for naive Bayes?
Comment: The expected cost of the naive Bayes classifier should be the cost of misclassification times the probability of misclassification, which will expand out into the cost of misclassification times the sum of the probability of a false positive and the probability of a false negative. So two things: 1. These are *not* $p$ and $1-p$ in the equation above. 2. If instead we call the probability of a false positive $f$, it is not necessarily true that the probability of a false negative is $1-f$. So you won't necessarily get the canceling-out that you're seeing.
Comment: No. From your question's first paragraph, it seems like you are using $p$ to define the model's probability output for a given observation. This is not really related to the probability of misclassification, which is what should be in these expected cost equations. edit: of course, $p$ is related to the probability of misclassification insofar as the value of $p$ (combined with a cutoff threshold to make the model a classifier) determines the prediction of the model. But I think you're confusing the model's output with the predictive accuracy of that output.
Comment: Ahh..so you mean $p(y|x)((c_m)*(1-p)+p*(c_m))$? And there are only two classes y in this task! Then it would be true? If these are correct, I will edit shortly
Comment: Ah I think I understand now: So you say that p is just the likelihood for a certain event, and not the probability for the posterior?
Comment: Is there any more info provided for the question? For instance, it looks like we would need to know the underlying proportions of positive and negative cases. We need to arrive at the *probability of misclassification*, which would be $P(C_1 | R_0) * P(R_0) + P(C_0 | R_1) * P(R_1)$ where $C_x$ is the event that the classifier outputs x and $R_x$ is the event that the real class is x. We need to know the overall chance of a 0 case $R_0$ and a 1 case $R_1$, in addition to the respective probabilities of misclassification. Note that $P$ here is not your posterior, but denotes probability
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/63338/discussion-between-testguest-and-eithompson).
Answer: From my reading of the original post, the question boils down to "what is the relationship between $E[Cost_H]$ and $E[Cost_M]$?" i.e. the "expected cost of consulting the human" and the "expected cost of trusting the machine."
The first part of this is easy: the expected cost of consulting the human is the term $c_h$ given in the question. More formally:
$\begin{align}E[Cost_H] &= (Cost_H|Correct)\times P(Correct) + (Cost_H|Incorrect)\times P(Incorrect) \\ &=(c_h \times 1) + (c_h \times 0) \\ &=c_h \end{align}$
The second part is a bit more involved to derive, but is easiest to think of if we look at the confusion matrix which considers the true class, $Y$, against the predicted class, $y$.

Again we can derive an expression for the expected cost of trusting the machine (using the terms $a$, $b$, $c$, and $d$ from the confusion matrix to declutter):
$\begin{align}E[Cost_M] &= (Cost_M|Correct)\times P(Correct) + (Cost_M|Incorrect) \times P(Incorrect) \\ &= 0 \times P(Correct) + c_m \times P(Incorrect) \\ &=c_m \times(P(y=1|Y=0)P(Y=0)+P(y=0|Y=1)P(Y=1))\\ &=c_m \times ((c\times (1-p))+(b \times p))\end{align}$
Finally, as asked in our question, we compare the two expected costs. We can say that the trade off point between the two methods occurs when $E[Cost_H] = E[Cost_M]$:
$\begin{align}E[Cost_H] &= E[Cost_M] \\ c_h &=c_m \times ((c\times (1-p))+(b \times p)) \\ \frac{c_h}{c_m} &= c\times (1-p)+(b \times p) \end{align}$
So determining which approach will minimize the expected cost depends on $b$, the probability that the machine predicts class 0 given that the true class is 1, $c$, the probability that the machine predicts class 1 given that the true class is 0, and $p$, the probability of class 1 occurring. For example:
$\text{For the case where }p = (1-p) = 0.5 \text{ we have:}\\\text{If }\frac{c_h}{c_m} > \frac{c+b}{2} \text{, then trusting the machine minimizes the expected cost;}\\\text{If } \frac{c_h}{c_m} = \frac{c+b}{2} \text{, then the approaches have the same expected cost; and}\\\text{If }\frac{c_h}{c_m} < \frac{c+b}{2} \text{, then consulting the human minimizes the expected cost.}$
NB: If $p \neq (1-p)$ then the comparison is of the same form with the terms $c$ and $b$ appropriately weighted.
Comment: Thanks for the beautiful answer. Why however do you include b and c? "the probability that the machine predicts class 0 given that the true class is 1" -- what is the reason you would not set that to p?
Comment: @TestGuest Simply put, $b$ and $c$ are included as separate terms because they are different concepts than $p$. As a quick illustration, consider the case at the end of my answer where $p=(1-p)=0.5$. If $b$ and $c$ were forced to be equal to $p$, then this would mean that your classifier would be wrong half the time; it would have no discriminating power. A Naïve Bayes classifier is not equivalent to flipping a coin simply because the likelihoods of the two classes is the same, so the equivalencies $b=p$ and $c=p$ are "absurd" in the style of *reductio ad absurdum*.
Comment: I was thinking along these terms at first, and I believe it makes sense if we can somehow know b and c. For instance, we could get estimates of b and c if we had a separate holdout test set against which to measure our model's performance. However, the question seems to want the answer only in terms of p. Given that p is our best estimate of the probability that Y = 1, it in itself gives us the model's confidence in the prediction. In other words, p = 0.75 implies that we have a 0.25 chance of a [c] error.
Answer: What we arrived at in chat: we assume the naive Bayes classifier goes with the maximum a posteriori class, which gives us a p = 0.5 cutoff in the two class case.
Suppose p > 0.5. Our model thus predicts class 1, and we expect to be wrong with probability 1 - p. So, expected cost of going with the classifier is the cost of misclassification times 1 - p. Similarly, the expected cost of going with the classifier when p < 0.5 is the cost of misclassification (same for either type of error in this problem) times p.
If the expected cost thus calculated is less than the cost of the guru, then go with the model's prediction. Otherwise, consult the guru.
There is a simple equation which can express our decision rule no matter what p is. I'll leave this unanswered. Hint: p being further from 0.5 in either direction means we should be less likely to consult the guru.


Error in upgrading report server project in Visual Studio 2017

Question: I recently have attempted to migrate a solution created and developed in Visual Studio (VS) 2012 to VS 2017.
I've managed to migrate all projects of my solution except for a Report Server Project. When I try to open my solution in VS 2017 I get the following message:

I click on Yes and VS automatically upgrades the project file (this is a file having the extension <code>.rptproj</code>)
When I now try to build the project I get the following error message:

Error     Copying file Reports\Report\Project1\MyReportProject.rptproj.user to
  obj\Debug\AspnetCompileMerge\Source\Reports\Report
  Project1\MyReportProject.rptproj.user failed. Could not find file
  'Reports\Report Project1\MyReportProject.rptproj.user'

I thought the upgrade would be performed transparently. Am I missing something? Is there any extra step I should take in order to do the upgrade?
Note: I've already downloaded and installed the latest version of Microsoft Reporting Services Projects .vsix package.
Comment: 15.5 should be available to all editions.
Comment: I'd suggest contact MS on their forum or emailing  email@example.com as this would be something they would be interested in dealing with
Comment: update VS2017 to update 15.5 and look if it fixes the issue
Comment: Unfortunately upgrading from 15.4.4 to 15.5 did not solve the issue.
Comment: Exclude that file form your project
Comment: @ErikEJ This file is automatically generated and added to the project by Visual Studio. Therefore I don't know how to exclude it from the Report Services project.
Answer: You can try the latest msbuild.exe for SSRS is here.
This includes steps of up-gradation of SSRS for both VS15 and VS17.
Once you install the latest update, depending on which version of Visual Studio you're using, the new files enabling MSBuild for your projects will be installed in different folder path:
In Visual Studio 2017, it'll will be a nested folder in your Visual Studio folder hierarchy. For example, the location with the Community Edition is in the Community folder:

Edit 1:
Try these steps:
1) Close Visual Studio - 17
2) Open VS-17 Installer
3) Try to install the SSDT workflow as:
Comment: I've already gone through the procedure of installing the Microsoft Reporting Services Projects package. Please check the note in the OP.
Comment: Your error is not about upgrade, its visual studio internal error to open SSRS.
Comment: Thanks for your edit. I've already installed SQL Server Data Tools from [this](https://learn.microsoft.com/en-us/sql/ssdt/download-sql-server-data-tools-ssdt#ssdt-for-visual-studio-2017) link.


A Geometry problem from OMK 2017

Question: The following problem appeared in the National Mathematical Olympiad, or known short as OMK in Malaysia.
Let ABCD be a convex quadrilateral. Let points M and N be the midpoints of BC and CD respectively. Triangle AMN divides quadrilateral ABCD into 4 non-overlapping sections, such that the areas of the sections are of consecutive integers. (If the area of the smallest section is E, then the 2nd smallest section is E+1, the 3rd smallest is E+2, and the largest is E+3.) What is the maximum possible area of triangle ABD? (Note:The problem is edited slightly to prevent certain confusion)
I have seemingly found a solution to the problem:
Let E denote the area of the smallest of the 4 sections. Thus, the total area of quadrilateral is 4E+6. Note that:
$$Area of triangle ABD=Area of quadrilateral ABCD-Area of triangle BCD$$
If the area of triangle ABD should be maximum, the area of triangle BCD must be minimum. Finally, notice that triangle BCD is similar to triangle MCN, and since triangle MCN is one of the 4 sections, the smallest area of triangle MCN must be E. Thus, the minimum area of triangle BCD is $E×2^2=4E$ and thus the maximum area of triangle ABD is 6.
Questions:
1) Does my solution contain any absurdity or wrong calculations somewhere?
2) Is there an alternative solution? (Preferably an elementary solution)
3) Any improvement suggestions? (Improvements in explanation, term usages, etc.)
Comment: I think your solution is very good. It also shows that the area of $ABD$ is always $6$, *if a solution exists*. But such existence problems are rarely of concern in Olympiad type questions.
Answer: Your solution is perfect, however I tried – just for fun – to see if such a quadrilateral can indeed exist.
In general, we can construct a quadrilateral $ABCD$ where the areas of triangles $CMN$, $ABM$ and $ADN$ are respectively $n$, $n+a$ and $n+b$. To this end, choose a segment $MN$ at will and a point $C$ such that its distance $CH$ from $MN$ satisfies ${1\over2}MN\cdot CH=n$. Points $B$ and $D$ are then given by the request that $M$ and $N$ be the midpoints of $CB$ and $CD$.
Draw then two lines $EK$ and $FI$, parallel to $CD$ and $CB$, such that their distances $CK$ and $CI$ from $C$ satisfy ${1\over2}CM\cdot CI=n+a$ and ${1\over2}CN\cdot CK=n+b$. Let $E$ and $F$ be the intersections of those lines with the line through $C$ parallel to $MN$. If $A$ is the intersection point of $EK$ and $FI$, then triangles $ABM$ and $ADN$ have areas respectively $n+a$ and $n+b$.
By similar triangles you have then $CF=(1+a/n)MN$ and $CE=(1+b/n)MN$. But triangle $AEF$ is also similar to $CMN$, so the distance from $A$ to line $MN$ is $(1+a/n+b/n)CH$ and the area of triangle $AMN$ turns out to be $n+a+b$.
In our case we need $a+b=3$, then we can choose $a=1$ and $b=2$ (or the other way around) to obtain areas $n$, $n+1$, $n+2$ and $n+3$. That works for every $n$.

EDIT.
The area $S_{ABD}$ of triangle $ABD$ can then be computed as $2(a+b)$ and the areas of the four triangles are consecutive numbers if $0$, $a$, $b$, $a+b$ are consecutive numbers.
The solution outlined above, $\{a,b\}=\{1,2\}$, gives $S_{ABD}=6$, but there is another possible configuration where the areas of the four triangles are consecutive numbers: $\{a,b\}=\{-1,2\}$, giving $S_{ABD}=2$.
Values of $a$ and $b$ leading to $S_{ABD}\le0$ must be discarded, for in that case the resulting quadrilateral is not convex. Hence we can confirm that the maximum value of $S_{ABD}$ is $6$, while the minimum is $2$.

A Geometry problem from OMK 2017


Is algebraic multiplicity of a root of a polynomial always smaller than the characteristic of a field?

Question: I'm trying to prove a criteria for algebraic multiplicity of a root of a polynomial.
Let $F$ be a field and $f\in F[X]\setminus\{0\}$.
Let $r$ be the algebraic multiplicity of $c$ as a root of $f$.
Then, there exists $g\in F[X]$ such that $f=(X-c)^r g$ and $g(c)\neq 0$.
Then, $(D^rf)(c)=g(c)r!$. However, does this gurantee thay $(D^rf)(c)$ is nonzero even though the characteristic of $F$ is not zero?
(I'm not really familiar with abstract algebra language, so i hope you explain this in relatively simple words.. Thank you in advance)
Comment: No. The polynomial $x^k$ has $0$ as a root of multiplicity $k$, for any positive number $k$.
Comment: The confusion may be about trying to still use vanishing of the derivative to characterize multiplicity of the root in positive characteristic.
Answer: I'm not sure what you are asking. But it seems that you want to know under what conditions the following holds: If $c$ is a root of a polynomial $f$ with (algebraic) multiplicity $r$, then $(D^r f)(c) \neq 0$. Well, using your formula $(D^r f)(c) = g(c) r!$ and $g(c) \in F^*$, this holds if and only if $r! \in F^*$, which holds if and only if the characteristic of $F$ is either zero or a prime number larger than $r$.
Answer: No, if $r\ge p$ in your field of characteristic $p$, then $r!=0$ and the right side is zero regardless of $f(c)$. For example, if $p = 2$, then every factorial is even after the first, so all higher derivatives vanish.
Answer: I'm not sure that this answers your question either. Consider characteristic $p$.
The polynomial
$$
(x-1)^{p^2}=x^{p^2}-1
$$
has $x=1$ as a zero of multiplicity $p^2$, which is certainly larger than the characteristic. Note that already the first (formal) derivative of this polynomial vanishes. Thus this also shows that vanishing of derivatives is not as closely related to the multiplicity of the root as it is in characteristic zero.


saving content containing html tags and single double quotes into database

Question: i've table with fields X:headline,content ,time.
through htmlform i upload data into this table.
html is:
<form action="process2.php" method="post" enctype="multipart/form-data">
Headline: <input name="headline" type="text" size="100" />

Content:<textarea name="content" cols="100" rows="10" placeholder="Content here">
</textarea>

</form>

part of process2 php script:
$headl = mysql_real_escape_string($_POST['headline']);
$headline=htmlspecialchars("$headl", ENT_QUOTES);

$cont=$_POST['content'];
$cont = str_replace("<", "<", $cont);
$cont = str_replace(">", ">", $cont);
$content=htmlentities($cont,ENT_QUOTES);

$sql="INSERT INTO X(`headline`, `content`, `date`) VALUES ('$headline','$content',NOW())";

$query = mysql_query($sql)or die(mysql_error());

problem:
Headline and date are getting inserted.
whereas content remains empty...
i've tried mysql_real_escape_string on content also but no use.
why content is empty? ...any solution?
Comment: why aren't you using mysql_real_escape_string on content too?
Comment: Use a prepared statement.
Comment: try echo `$content` and see what it gives. maybe its empty before inserting to database
Comment: @echo_me   ya its empty before entering into database
Comment: @ÁlvaroG.Vicario how sql injection came inbetween... this html and php script is for admin, it wont be accessible to user..
Comment: then echo `$cont` and see also , you should know how to debug by yourself
Comment: @user2320375 - If you say so then you don't fully understand what SQL injection is. I'll put an analogy: you don't let your code divide by zero just because it's for internal use, do you?
Comment: you found `$cont` the problem was there ?
Answer: Please try this, it should work in your situation:
// htmlspecialchars Must with ENT_QUOTES, it actually make mysql_real_escape_string useless, but mysql_real_escape_string for safe anyway.

$headline = mysql_real_escape_string(htmlspecialchars($_POST['headline'], ENT_QUOTES));
$content = mysql_real_escape_string(htmlspecialchars($_POST['content'], ENT_QUOTES));

$sql = "INSERT INTO X(`headline`, `content`, `date`) VALUES ('{$headline}', '{$content}', NOW())";

$query = mysql_query($sql) or die(mysql_error());

To display, you can load out the data from mysql, and perform htmlspecialchars_decode for each.
And more simple way is, don't do htmlspecialchars when saving them, just like this:
$headline = mysql_real_escape_string($_POST['headline']);
$content = mysql_real_escape_string($_POST['content']);

After mysql_real_escape_string, data should be safe for mysql, because all known inject trick will be filtered. You only need to care about the html content it self. Oh, and the length of the both content must not break the char limit of your fields.


Como fazer um filtro baseado em uma condição?

Question: Tenho um data.frame com duas colunas (CLASSWK e IND) e gostaria de aplicar um filtro baseado em uma condição, mas não estou conseguindo fazer isso.
Tenho uma coluna CLASSWK, que quero manter apenas os valores que forem iguais a 1 ou 2. Mas isso só se aplica se a minha outra coluna, IND, for diferente de 0.
Então, basicamente, se IND == 0, eu apenas desconsidero meu filtro, mas se meu IND for qualquer outro valor, então eu tenho que aplicar o filtro na coluna CLASSWK e retirar os valores que são diferentes de 1 ou 2.
Meu dput:
structure(list(CLASSWK = c(1, 2, 1, 3, 5, 1, 2, 6, 7, 3), IND = c(1,
2, 3, 4, 0, 6, 0, 0, 8, 4)), class = "data.frame", row.names = c(NA,
-10L))

Código que tentei usar:
library(dplyr)

x %>%
  filter(case_when(IND != 0 & CLASSWK == 1 & CLASSWK == 2))

Answer: Eis uma maneira com o pacote dplyr.
Como CLASSWK pode tomar vários valores usa-se %in% e não a igualdade.
library(dplyr)

x %>% filter((CLASSWK %in% 1:2 & IND != 0) | IND == 0)
#  CLASSWK IND
#1       1   1
#2       2   2
#3       1   3
#4       5   0
#5       1   6
#6       2   0
#7       6   0

Em R base, qualquer das seguintes maneiras também resolve o problema.
subset(x, (CLASSWK %in% 1:2 & IND != 0) | IND == 0)
x[(x$CLASSWK %in% 1:2 & x$IND != 0) | x$IND == 0, ]

Comment: mas em ambos os casos eu estou retirando da base os valores onde `IND == 0` e eu gostaria de mantê-los também.
Comment: @AlexandreSanches Feito, veja agora.
Comment: Muito obrigado!


Testing count post sampling

Question: Some automated sampling is done (by a computer) on a collection of $N$ identical items. The sampling process consists in keeping each item with a constant probability $\alpha$ and discarding it with a probability $1-\alpha$. Call $X$ the number of items that have been kept at the end of the process.
The expected value of $X$ is $\alpha N$. More precisely, $X$ has a binomial distribution $B(N,\alpha)$ given $N$. Most often $N$ is large enough to use a normal approximation $B(N,\alpha)\approx\mathcal{N}(\alpha N,\alpha(1-\alpha)N)$.
The data are events recorded on the Internet. $N$ is from a few hundreds to a several millions. I know the sampling rate is exactly $\alpha=\frac{1}{10}$.
Now two sampling processes are done independently from two populations $N_A$ and $N_B$ with the same known sampling rate $\alpha=\frac{1}{10}$. After sampling we have $X_A$ and $X_B$ items being kept respectively. For example, we may get something like:
$X_A=1234, X_B=1322$. I want to test if the original populations are the same: I want to test the null hypothesis $N_A=N_B$.
How can I do ?
All I can observe is $X_A$ and $X_B$. I know for sure the sampling processes are independent (I'm not even sure this is important). I know the rate $\alpha$ is perfectly known.
Unlike most tests I've heard of (Kolmogorov-Smirnov, Fisher, $\chi^2$...) this is a test on raw counts and not on proportions (probabilities). That's why I struggle to find a standard technique.
Comment: Yes. And thanks for your comment. I edited the question to make all these points clearer.
Comment: Could you please explain what "X items left" means?  Is it the count of items in the sample, as suggested by your distributional assertions?  And shall we also deduce that "sampling" means each item is independently selected with a constant probability of $\alpha$?
Answer: I have found a solution that is satisfying for me.
Even though it's not a $\chi^2$ test we can use a method very close to a $\chi^2$ test. Let's define :
$$S=\frac{(X_A-X_B)^2}{(1-\alpha)(X_A+X_B)}$$
Assume the null hypothesis is true that is : $N_A=N_B$. Simply write them $N$. If $N$ is large enough, we can use approximations :
First about the numerator. $X_A$ has a normal distribution $\mathcal{N}(\alpha N,\alpha(1-\alpha)N)$. Same for $X_B$. Since $X_A$ and $X_B$ are independent, $X_A-X_B$ has a normal distribution $\mathcal{N}(0,2\alpha(1-\alpha)N)$. It is $\sqrt{2\alpha(1-\alpha)N}$ multiplied by a variable having a distribution $\mathcal{N}(0,1)$. Thus $(X_A-X_B)^2$ is $2\alpha(1-\alpha)N$ multiplied by a variable having a $\chi^2$ distribution with one degree of freedom.
For the denominator, I used a rough approximation. Because of the law of large numbers, $X_A+X_B$ is approximatively a constant $2\alpha N$. Note that I could have used $2X_A$ or $2X_B$ instead of $X_A+X_B$ but I wanted something symmetrical. Finally the denominator is approximated by $2\alpha(1-\alpha)N$. I guess this approximation needs $N$ to be larger than for the first approximation, but I don't really know.
Thus, when you assume the null hypothesis, $S$ has a $\chi^2$ distribution with $1$ degree of freedom.
I can use a classical $p$-value from the $\chi^2$ table or this nice calculator : http://www.danielsoper.com/statcalc/calculator.aspx?id=11. With my example $X_A=1234, X_B=1322$ I get $S=3.36$ and $p=0.065$. Under the null hypothesis, such a difference can be observed with probability 6.5%. This does not lead to reject the null hypothesis with usual limits (say 1% or 5%).
I ran this test on the data, which was actually hundreds of thousands of population pairs : hundred of thousands of these tests. Thus I chose a small limit for the $p$-value : around $10^{-6}$. I found flawed populations (null hypothesis rejected) leading to identify true bugs in the event recording process : $p$-values for buggy population pairs were actually extremely small : smaller than $10^{-20}$. After fixing them, all the tests became green (null hypothesis accepted).
Answer: In your example, $X_A = 1234$. No doubt after 1370 items, you will take 137 (1/10 of 1370) as sample, and 1233 were left. Next time, if you wait until other 10 items ready to take another one as sample, then $X_A = 1234$ means after you take 137th item for sample 1 more item comes in and you are waiting. So $N_A = 1370 + 1 = 1371$. If you do not waiting for 10 items and take the first one after 1370 items, then $N_A = 1370 + 1 (\mathrm{sample}) + 1 (\mathrm{left}) = 1372$. Anyway $N_A = 1371$ or $1372$ and no third value.
Similarly $X_B = 1322 \rightarrow N_B = 1460 + 8(\mathrm{left }) (+1(\mathrm{sample})) = 1468$ or $1469$, where (+1) mean may or may not add 1, see above.
Conclusion: Reject null hypothesis that $N_A = N_B$ because $N_A = 1371$ or $1372$ and $N_B = 1468$ or $1469$ and there is no overlap.
Comment: Yes. Want I meant is what whuber understood. The sampling is based on random independent trials. I updated the question to make it clearer.
Comment: This obviously is not a hypothesis test, and your expressions for the counts $N_A$ and $N_B$ are not integral, so it's doubtful that it correctly answers the question.  Could you explain how you're interpreting the problem?

Testing count post sampling


TypeError: expected bytes, Descriptor found

Question: How do I fix the following error traceback regarding tf.record?
(tensorflow1) c:\tensorflow1\models\research\object_detection>python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record --image_dir=(image directory)
Traceback (most recent call last):
  File "generate_tfrecord.py", line 17, in <module>
    import tensorflow as tf
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\python\__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\core\framework\graph_pb2.py", line 15, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\core\framework\node_def_pb2.py", line 15, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\core\framework\attr_value_pb2.py", line 15, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\core\framework\tensor_pb2.py", line 15, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "C:\Users\Dell-Oguz\Anaconda3\envs\tensorflow1\lib\site-packages\tensorflow\core\framework\resource_handle_pb2.py", line 91, in <module>
    __module__ = 'tensorflow.core.framework.resource_handle_pb2'
TypeError: expected bytes, Descriptor found

Comment: i am facing the same error. I figured it out by reading this site:
https://github.com/tensorflow/tensorflow/issues/35208 Hope it helps!
Answer: Try this, worked for me
pip install protobuf-py3
pip install --upgrade protobuf

Comment: Worked for me but gave me another error. Referring to this answer to solve the problem I'm currently facing https://stackoverflow.com/questions/60423758/keras-is-not-importing-typeerror-can-only-concatenate-str-not-list-to-str
Answer: I get the same error when I installed tensorflow with conda install -c anaconda tensorflow-gpu over existing tenforflow.
Remove existing environment and create a new one. Then run conda install -c anaconda tensorflow-gpu .
Comment: Hi
I did "conda uninstall -c anaconda tensorflow-gpu", then I run "conda install -c anaconda tensorflow-gpu" but I get same error.
Could you give detail?
Comment: Remove environment with `conda remove --name myenv --all` then create new environment `conda create -n mynewenv python=3.6` .

Then run `activate mynewenv` and `conda install -c anaconda tensorflow-gpu`
Comment: I am not sure why it is not fixed. You can try install cuda, cudnn and keras step by step. Check below links.
first install cuda;
https://developer.nvidia.com/cuda-downloads  (I recommend 10.0 or 10.1)
https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html#install-windows
after that try pip install tensorflow-gpu
Answer: Probably it's due to unmatched tensorflow and tensorboard versions.
I solved it after creating a separate conda environment for my code.
Answer: In my case,
pip install --ignore-installed --upgrade tensorflow-gpu==1.15.0
did the job :).


Proof about the difference between right and left ideals in a ring

Question: I have tried get a version of the proof stating that a left ideals of a ring is not, in general, a right ideal, and viceversa. Is my formulation right?
Comments and corrections are welcome. I have tried to understand some German papers on the topic I came across.......I hope it is fine......The proof takes the form of a reductio ad absurdum......
Thanks in advance:
The proof goes as follows: let us assume M to be a left module over the ring R with the external operation R × M → M.  Then we write the product with elements from R, proceeding from right to left.  For that purpose, the symbol ◦ is used.  Let then α·x := x◦α, ∀α ∈ R, x ∈ M. Now, generally speaking, the left module M with the external product ◦ : M × R → M is no right module whatsoever. This contradicts the Associative Law. According to the initial assumption M is a left module, that is, it is precisely the Associative Law that comes into play for scalar multiplication of a ring element from the left side.  We then suppose that M with the operation ◦ is also a right module. Assuming, without loss of generality, that all  other conditions apply as usual, it can be seen that instead of the Associative Law (M3) the following holds: (αβ) · x = x ◦ (αβ) = (x ◦ α) ◦ β = (α · x) ◦ β = β · (α · x)⊥, which constitutes a contradiction to the Associativity of the left module.
Comment: IMO a counterexample is the only proof you can have. Why would you not call the existence of a counterexample a proof that the fact is not always true? An example is never a proof, but a single counterexample is always a disproof (=proof that the claim is wrong).
Comment: Are you trying to prove that left modules can never be right modules? This is, of course, not true. Maybe you should try to find an explicit example of a left module that is also a right module, and a left module that is not a right module.
Comment: No, I am trying to prove that they are not automatically right modules. Of course I know they are sometimes so. The point is, they are not always such a thing. I provide examples in some other place. Here I am just concerned with the proof.
Answer: It is IMHO never a good idea to try and prove that some structure is not some other structure using the axioms only. After all, in many cases the said structure also IS that other  structure, so a proof "generally speaking" is doomed to fail. Here you cannot rule out the possibility that, in addition to what you describe as a substitute to the associativity, the usual associativity may still hold also. For example, if $R$ happens to be commutative every left ideal is also a right ideal. Therefore any argument that works in any ring is automatically wrong.
Furthermore, you cannot just add the assumption that $R$ is not commutative to your proof, and expect the readers to be convinced. This is because there are non-commutative rings were every left ideal is also a right ideal. The ring of Hamilton's quaternions is an example of such a ring.
No. When you are given the task to show that something is not always true, the first reaction should be to look for a counterexample. Here your task is to find a ring $R$ and its left ideal $L$ such that $L$ is not a right ideal of $R$. Hint: Look at the ring of $2\times2$ matrices.
Students often want to answer questions like this by starting out with one of the proofs from the textbook/lecture notes, and then point out a step that does not work. This gives me the teacher pimples. Such an answer simply allows us to conclude that this line of reasoning did not work. It leaves open the possibility that another argument still might.
Sorry about the rantish answer. I just feel that learning to appreciate the role of counterexamples is a very important part of the education of a mathematical mind. Hence the emphasis.
Comment: I don't understand. To make things clearer to me can you (for example) give a formal proof that a continuous function is not always differentiable.  Or a formal proof that a group is not always abelian. Then I would better understand what you mean. A counterexample is a proof by contradiction: We claim that a left ideal is not always a right ideal. Assume contrariwise that a left ideal is always a right ideal. Look, here's a left ideal that is not a right ideal. A contradiction. Q.E.D.
Comment: Why is that not a formal proof? AFAICT the point of that German paper is pedagogical and it only shows that the process described there does not allows one to turn a left module into a right module unless that identity happens to hold for all the elements of the module and the ring. I think it is more than a bit misleading to call it "Beweis."
Comment: @JavierArias I think someone has already pointed out to you in your proof that if the $R$ in the proof is commutative, it will be proving a commutative ring has a left ideal that isn't a right ideal, which is absurd. The strategy of beginning with "Let $M$ be a module over a ring $R$..." and then proving not all left ideals are right ideals is doomed to failure for that reason.
Comment: yes, you should use reasoning along those lines. I might say "Note that while over a commutative ring, left modules can be thought of as right modules (and vice versa), in general this is not the case. A counterexample is provided in section..."
Comment: I was trying to formulate in English what I found in German on oages 12-13 of this paper....http://www.mathematik-netz.de/pdf/Moduln.pdf . I do not know whether I succeeded
Comment: I have already provided the ring of 2x2 matrices as counterexample in some other section of my paper.....I am here just concerned with the formal proof.
Comment: By formal proof a mean a purely syntactical proof, with no resort to empirical examples.
Comment: @JavierArias I think Jyrki is right. If a proposition is true (has no counterexamples) then you can try to prove it. If a proposition is false, then it has a counterexample, an no proof exists for the proposition. Here, you are just working with the proposition "every right ideal is a left ideal" Maybe there is something linguistic going on here, but I think you should spend some time seeing it from the point of view of this solution no matter what.
Comment: So you mean the German text I provided is no proof whatsoever?
Comment: @JavierArias Of course it could be trying to explain heuristically why it is not the case, but that is not really a formal proof. Similarly if you had the statement "aliens don't exist" and you wanted to show that this statement is wrong, then you can wave your hands all day with axioms showing that they could possibly exist, but that isn't convincing. What would really be convincing is to produce an alien as a counterexample to the claim.
Comment: @rschwieb Would it be then better if I just write in a footnote something along these lines? "The specific case dealt with a little later in section §5 constitutes a perfect counterexample against the claim that a left module can be turned into a right module under any circumstances." (The section referred deals with 2x2 matrices and the example of those matrices with bottom line zero. -
Comment: @Jyrki Lahtonen Would it be then better if I just write in a footnote something along these lines? "The specific case dealt with a little later in section §5 constitutes a perfect counterexample against the claim that a left module can be turned into a right module under any circumstances." (The section referred deals with 2x2 matrices and the example of those matrices with bottom line zero. -


Question regarding Euler's form

Question: I know about the famous identity $$ e^{ix}=\cos x + i\sin x $$
I have proved the above identity by comparing the Maclaurin series of $e^{ix}$,$\cos x$ and $\sin x$. But I want to know about other proofs of this identity. I have no idea about any such proofs. Any help would be appreciated.
Comment: I hope that you mean $e^{ix}$  instead of just $e^x$
Comment: What do you think of this? https://en.wikipedia.org/wiki/Euler%27s_formula#Proofs
Comment: Yes Matti P. I just edited
Comment: You can't prove that $e^{i\theta} = \cos(\theta) + i\sin(\theta)$ any more than you can prove that $(i^2) = -1.~~$ $e^{i\theta} = \cos(\theta) + i\sin(\theta)$ is an *artificial contrivance* that extends the exponent rule $(e^a \times e^b) = e^{(a+b)}$ rule to complex numbers, based on the easily provable idea that $[\cos(a) + i\sin(a)] \times [\cos(b) + i\sin(b)] = [\cos(a+b) + i\sin(a+b)].$
Comment: @Matti P. Not so good proofs on wikipedia.
Comment: @user2661923 Please explain in detail.
Comment: I will post an answer, giving more details.
Comment: @user2661923 if you use infinite series as the definition of $e^x$ For complex $x$  then you can .
Comment: I'd like to propose a proof shown on 3 Blue 1 Brown, which basically shows that $z(t) = e^{it}$ parameterizes a path in the complex plane where the derivative is a 90 degree rotation of the position ($z'(t) = ie^{it}$) and $z(0) = 1$, and these two conditions can only be met if the path is the unit circle.
Comment: See also https://math.stackexchange.com/questions/72113/has-anyone-talked-themselves-into-understanding-eulers-identity-a-bit.
Answer: First, see my comment, following the query.
It was desired to facilitate solving problems involving complex numbers.  Based on the Taylor series,
it is know that for Real $x$
$$ e^x = \lim_{n\to\infty} \sum_{k=0}^n \frac{x^k}{k!}.\tag1$$
This formula, as is, (prior to 1700) would have been considered nonsensical, if $iy$ was substituted for $x$, where $i = \sqrt{-1}$ and $y$ is a real number.  That is, how can you possibly have a real number taken to a non-real exponent.  Just as  $2^3$ represents $2 \times 2 \times 2$, it does make sense to have any real number (or at least any positive real number) taken to any real exponent.
However, Euler noticed that if he broke the rule that equation (1) above is limited to Real numbers, and artificially decided to also apply it to imaginary numbers, or Complex numbers, that it would facilitate solving problems.  He also noticed that upon breaking the rule, that it would make sense to regard $e^{i\theta}$ as $\cos(\theta) + i\sin(\theta)$, based on the Taylor series for the sine and cosine functions.
As my comment indicated, re trig identities, Euler found that artificially extending equation (1) above, to Complex numbers, would preserve the law of exponents.  This means that you could have more compact syntax, which would facilitate solving problems.
Actually, $e^{i\theta}$, where $\theta$ is real, is nothing more than syntactic sugar for $[\cos(\theta) + i\sin(\theta)].$

Question regarding Euler's form


Why does PriorityQueue do (k - 1) >>> 1?

Question: I was studying PriorityQueue in Java.
When I looked into offer method, couldn't find out why using (k - 1) >>> 1 in siftUpComparable method.
Here is the source code:
public boolean offer(E e) {
    if (e == null)
        throw new NullPointerException();
    modCount++;
    int i = size;
    if (i >= queue.length)
        grow(i + 1);
    size = i + 1;
    if (i == 0)
        queue[0] = e;
    else
        siftUp(i, e);
    return true;
}

private void siftUp(int k, E x) {
    if (comparator != null)
        siftUpUsingComparator(k, x);
    else
        siftUpComparable(k, x);
}

private void siftUpComparable(int k, E x) {
    Comparable<? super E> key = (Comparable<? super E>) x;
    while (k > 0) {
        int parent = (k - 1) >>> 1; // what's the purpose of this line?
        Object e = queue[parent];
        if (key.compareTo((E) e) >= 0)
            break;
        queue[k] = e;
        k = parent;
    }
    queue[k] = key;
}

Answer: Since it's guaranteed that k > 0, that line could just as well have been (k - 1) >> 1. The >> and >>> operators shift right by the number of bits given as the right-hand operand. Shifting by 1 is exactly the same as dividing by 2, so this is computing parent = (k - 1) / 2 (with integer truncation), but doing it a little faster.
The difference between >> and >>> is what happens with negative numbers: >> fills on the left with the current value of the sign bit while >>> always fills with zero.
Comment: Thanks for your explanation, it gave me a clear understanding of how `offer()` works
Answer: It is getting the index of the parent element of the k-th element in the binary heap.
It is bit-shifting way for saying
int parent = (k-1)/2