clustering evaluation for a special case

Question: In my dataset each point comes from one of 3 classes, so the true labels are like [0,1,0,0,0,2,1....]. I have to cluster them in 200 clusters. I want each cluster to contain more points from the same class. So,
a cluster such as [0,0,0,1] (C1) is a good cluster, whereas [1,2,1,0] (C2) is not.
I am trying to come up with an evaluation strategy for this. The harmonic mean of class frequencies seems like a starting point. For C1, it would be (1/3)+(1/1)+(1/(0+eps)), for C2: (1/2)+(1/1)+(1/1). Taking eps=0.1, obviously, C1 has a higher score.
But this would be affected by cluster size. Is there a way to eliminate that? Or better, does any evaluation protocol exist in the literature for this problem?
EDIT
I am trying to create a codebook of visual words from images. I have N images from 3 classes. These 3 classes are the true labels.
From each image, F number of d dimensional feature descriptors are created. So I have a data matrix of N*F rows and d columns. These are clustered in 200 clusters and the cluster centers are combined to create the codebook. I believe This is a standard procedure for this problem (https://en.wikipedia.org/wiki/Bag-of-words_model_in_computer_vision).
I expect the clusters to have some kind of homogeneity, i.e., in each cluster, most data points should from one class of image.
Comment: It's unclear what you mean by a "cluster". For example, why not create classes of all zeros, all ones, and all twos? What constraints, if any, do you have on the sizes of the classes? What distinction are you trying to make with the phrase "true labels"?
Comment: It looks like your data is a single vector of class labels.  Is that correct, or are there other variables that can be considered "input" for which the label is the "output"?
Comment: I have added some clarification, please see if that makes it more clear.
Answer: This is not a special case.
The standard evaluation measures such as ARI (Adjusted Rand Index) should work for you.
If you study the Rand index there is a trivial modification to make it asymmetric, too.
Comment: In the documentation for scikit-learn it says "ARI is symmetric, so labelings that have pure clusters with members coming from the same classes but **unnecessary splits are penalized**" ( http://scikit-learn.org/stable/modules/generated/sklearn.metrics.adjusted_rand_score.html ). Isn't that different from my use case?
Comment: Well, a solution that has only pure clusters won't score 1.0 but that doesn't mean it is useful. Also "such as ARI" - there are maybe 40 evaluation measures, I'm pretty sure one of them will suit your needs better...


Question on rules of conditional probability

Question: This is a somewhat trivial question, but I can't find anything anywhere to tell me how to solve it.
I have the values of the following:
P(A|B)
P(C|A)
P(C|not A)
P(A)

I need to find P(A|B,C)
I know that P(A,B|C) = P(A|C) P(B|C)
Would it also be correct to assume
P(A|B,C) = P(A|B) P(A|C) ?
since I already have P(A|B) and can easily derive P(A|C) using the information I have (bayes theorem).
(We also know the B and C occuring are independent of eachother)
Comment: The data does not suffice to compute `P(A|B,C)`.
Answer: Per Bay's Rule
$$P( A B C ) = P( B C ) * P( A | B C ) = P( B C | A ) * P( A )$$
$$P( A | B C ) = \frac{ P( B C | A ) * P( A ) }{ P( B C )}$$
Assuming B and C are conditionally independent on A
$$P( A | B C )  = \frac{ P( B | A ) * P( C | A ) * P( A )}{P( B ) * P( C )}$$
Also per Bay's
$$P( B | A ) = \frac{P( A | B ) * P( B )}{P( A )}$$
So
$$P( A | B C ) = \frac{ P( A | B ) * P( B ) * P( C | A ) * P( A )}{ P( A ) * P( B ) * P( C )} $$
Cancelling
$$P( A | B C ) = \frac{ P( A | B ) * P( C | A )}{ P( C )} $$
Rewriting P( C )
$$P( A | B C ) = \frac{ P( A | B ) * P( C | A )}{ P( C | A ) + P( C | \overline{A} )} $$
Comment: "Since B and C are independent ... P( BC | A )  = P( B | A ) P( C | A )" This is frankly wrong, because independence does not imply conditional independence. Counterexamples abound and are customarily given in first courses on stochastic independence.
Comment: Ahh, yes, I did assume conditional independence as well.  If that is not the case, then it is wrong.  @BKS, can you clarify?


Drupal Facets Block Show everywhere (in non search page)

Question: I use facets search and work well but i want to show block everywhere in website please for your help.
I try to create new views with my search index and add facets block in views but i get wrong url ?f[0]=field_places%3A72
Answer: Depends on what module you are using with facetapi. If you are using search_api, then when creating the view you can add a view display of type "Facets Block" (like you would add a page, block or panel display).
Using that display you can add a facet block anywhere on your site, even on non search pages. And you can also set the search_path you want to redirect to when using the facet block.


Cadastrar dados passando por View diferentes

Question: Sou iniciantes em asp net mvc e gostaria de saber como posso cadastrar dados de uma entidade passando por View diferentes de um mesmo Controller
Obrigado !!
Comment: Você quis dizer, passando por *Actions* de um mesmo *Controller*, certo? *Controllers* não possuem *Views*. Qual o objetivo disso?
Comment: Acredito que sim... Como uma entidade tem muitos atributos, quero separar o cadastro desses atributos em Wiews diferentes...
Comment: Você está usando Bootstrap?
Comment: Estou sim @CiganoMorrisonMendez
Answer: O caminho para formulários muito longos é usando tabs. A View pode ter vários campos e o usuário vai preenchendo cada aba com os campos categorizados.
Há também o jQuery Steps, se quiser fazer com cara de Wizard. Tem NuGet.


Which test player had 99.9 average in his test career?

Question: Which test player has 99.9 average in his test career and in last match of his career he was out on 0 ? As if he had just one score in his last inning then his average would have became 100.
Answer: Sir Donald Bradman, an Australian batsman was the player who finished his playing career with 99.94 average in 52 matches and 80 innings.
And he needed just four runs from his final-ever Test match innings to finish with an average of exactly 100.

"...the great Australian needed just four runs from his final-ever Test match innings against England at the Oval in 1948 to finish with an average of exactly 100, only to miss a googly from Eric Hollies and be bowled for a two-ball duck."

Last match report.


Why is iptables not blocking an IP address? (LB/proxy version)

Question: WARNING: Long. Lots of info here.
3 years ago someone asked Why is iptables not blocking an IP address? and it turned out the reason was because the servers were behind CloudFlare which made it impossible to block IP addresses directly they way they wanted to unless you use it differently.  Any reverse proxy or load balancer would cause the same thing.
Similarly we have setup fail2ban with a rule to ban any bots which attempt to brute-force their way into the administrative login or spam xmlrpc.  The site is sitting behind a load balancer so obviously we can't directly ban the IP address but iptables is supposed to be accepting the connection and pattern matching the packet data to ban specific traffic.
This is fail2ban jail.conf config:
[wp-auth]
enabled = true
filter = wp-auth
action = iptables-proxy[name = lb, port = http, protocol = tcp]
         sendmail-whois[name=LoginDetect, firstname.lastname@example.com, firstname.lastname@example.com, sendername="Fail2Ban"]
logpath = /obfuscated/path/to/site/transfer_log
bantime = 604800
maxretry = 4
findtime = 120

This is the simply pattern match for wp-login requests:
[Definition]
failregex = ^<HOST> .* "POST /wp-login.php
ignoreip = # our ip address

This is our fail2ban iptables action which is supposed to be able to block these bots but for the most part doesn't seem to.  It is from the CentOS site Tips section for fail2ban behind a proxy. For the sake of brevity I've left only the section header comments in place.
# Fail2Ban configuration file
#
# Author: Centos.Tips
#

[INCLUDES]

before = iptables-blocktype.conf

[Definition]

# Option:  actionstart
actionstart = iptables -N fail2ban-<name>
              iptables -A fail2ban-<name> -j RETURN
              iptables -I <chain> -p <protocol> --dport <port> -j fail2ban-<name>

# Option:  actionstop
actionstop = iptables -D <chain> -p <protocol> --dport <port> -j fail2ban-<name>
             iptables -F fail2ban-<name>
             iptables -X fail2ban-<name>

# Option:  actioncheck
actioncheck = iptables -n -L <chain> | grep -q 'fail2ban-<name>[ \t]'

# Option:  actionban
actionban = iptables -I fail2ban-<name> 1 -p tcp --dport 80 -m string --algo bm --string 'X-Forwarded-For: <ip>' -j DROP

# Option:  actionunban
actionunban = iptables -D fail2ban-<name> -p tcp --dport 80 -m string --algo bm --string 'X-Forwarded-For: <ip>' -j DROP

[Init]
# Default name of the chain
name = default

# Option:  port
port = http

# Option:  protocol
protocol = tcp

# Option:  chain
chain = INPUT

So as I mentioned the site is on a pair of servers behind an elastic load balancer and seems to work in test.  We can add any of our own IP addresses and we cannot reach the site.  Despite this bots seem to be able to get through.
[root:~/] iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N fail2ban-SSH
-N fail2ban-lb
-A INPUT -p tcp -m tcp --dport 80 -j fail2ban-lb
-A INPUT -p tcp -m tcp --dport 22 -j fail2ban-SSH
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5666 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 3306 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 24007:24020 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A fail2ban-SSH -j RETURN
-A fail2ban-lb -p tcp -m tcp --dport 80 -m string --string "X-Forwarded-For: 188.8.131.52" --algo bm --to 65535 -j DROP
-A fail2ban-lb -p tcp -m tcp --dport 80 -m string --string "X-Forwarded-For: 18.104.22.168" --algo bm --to 65535 -j DROP
-A fail2ban-lb -p tcp -m tcp --dport 80 -m string --string "X-Forwarded-For: 184.108.40.206" --algo bm --to 65535 -j DROP
-A fail2ban-lb -p tcp -m tcp --dport 80 -m string --string "X-Forwarded-For: 18.104.22.168" --algo bm --to 65535 -j DROP
-A fail2ban-lb -j RETURN

Port 80 is the only port open to all.  All others are ACL'd via AWS Security Groups.  IPtables appears to be processing in the correct order and should therefore be blocking these IPs based on their X-Forwarded-For header.  There is a Firefox plugin which allows you to send these headers with initial requests and we get blocked as a result with any of these bot IPs as well.
The source IP address does not appear to be forging the X-Forwarded-For header as we've been playing with as the ELB rewrites them anyway.  tcpdump does not show any extra information on the packet at the server level.
22:07:14.309998 IP ip-10-198-178-233.ec2.internal.11054 > ec2-10.4.8.71.http: Flags [P.], seq 2545:3054, ack 19506, win 166, options [nop,nop,TS val     592575835 ecr 2772410449], length 509
E..1..@.@..9
...
f.p+..P.Nz.
20............
#Q.[.?.QPOST /wp-login.php HTTP/1.1
host: www.thiswebsite.com
Accept: */*
Accept-Language: zh-cn
Cache-Control: no-cache
Content-Type: application/x-www-form-urlencoded
Referer: http://www.thiswebsite.com/wp-login.php
User-Agent: Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)
X-Forwarded-For: 18.104.22.168
X-Forwarded-Port: 80
X-Forwarded-Proto: http
Content-Length: 21
Connection: keep-alive

These requests are all being logged in the transfer_log.
When we do the same thing and forge the X-Forwarded-For we get caught by iptables before ever reaching Apache. tcpdump also shows our extra IPs.
20:10:25.378873 IP ip-10-198-178-233.ec2.internal.11054 > ec2-10.4.8.71.http: Flags [P.], seq 3157:3860, ack 124583, win 267, options [nop,nop,TS     val 526293643 ecr 2507283790], length 703
E...Tf@.@.[.
...
f.p,O.P...GU........m.....
.^...r.QPOST /wp-login.php HTTP/1.1
host: www.thiswebsite.com
Accept: /
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Cache-Control: no-cache
Cookie: __utma=190528439.16251225.1476378792.1478280188.1478289736.3; __utmz=190528439.1476378792.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);     _icl_current_language=en; __utmc=190528439; __utmb=1905284220.127.116.118289736; __utmt=1
Pragma: no-cache
Referer: http://www.thiswebsite.com/
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:49.0) Gecko/20100101 Firefox/49.0
X-Forwarded-For: 188.8.131.52, <our ip address>
X-Forwarded-Port: 80
X-Forwarded-Proto: http
Connection: keep-alive

I also have the ELB access log here which I expect to see an entry for, just not the Apache transfer logs.
2016-11-07T22:07:14.309917Z mLB 188.8.131.52:60407 10.4.8.71:80 0.000079 1.99244 0.000091 200 200 21 3245 "POST http://www.thiswebsite.com:80/wp-login.php HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; 125LA; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)" - -

So the IP address (at least according to the ELB) does not appear to be forced at the X-Forwarded-For level.  Why is traffic from it not being blocked?  The IP address also shows up constantly in the fail2ban log with the usual:
fail2ban.actions[11535]: INFO [wp-auth] 22.214.171.124 already banned

Comment: *"The site is sitting behind a load balancer so obviously we can't directly ban the IP address*" - why not? Have you considered that fail2ban rather than calling iptables one can also use the `actionban=` to call another script/program/API function to get your loadbalancer/firewall to block te client at the ip-level? You're not rstricted to using the host based firewall on the server running fail2ban.
Comment: Did you manage to figure this out? I'm having the same problem. I've added a bounty.
Comment: How is apache writing it's logs for fail2ban to pick up?  Normally x-forwarded-for is a list of addresses and the last one in the list is the client.  It's possible that fail2ban is picking up the wrong ip to ban from the log.
Comment: Haven't tried it but does this help? https://centos.tips/fail2ban-behind-a-proxyload-balancer/
Comment: Also:  http://www.loadbalancer.org/blog/apache-and-x-forwarded-for-headers
Comment: Apache writes to access.log and Fail2ban is picking up the right IP address and adding it to iptables. You can see in the question iptables `DROP`s the packet but users are still able to browse/login to the site.
Comment: The article you linked to is the one being used and linked to in the question.
Comment: It was mentioned I could script a call to the ACL in the AWS Load Balancer but as it happens this only works with a load balancer with classic mode enabled serving servers within a VPC.  It does not appear that you can use the ACL on VPC load balancers and connect to Classic Instance servers.  The load balancer specifically setup for classic instance servers does not have an ACL capability attached to it.  It would be best if we just reprovision the servers under a VPC instead so we could do that but "budget" blah blah blah...etc. So unfortunately this issue is still unresolved.
Comment: It would be interesting to look at the size of the incoming packets to see if the bots are trying to use small packets/fragmentation to avoid detection.  A logging rule in the fail2ban-lp chain might also be helpful.
Comment: @BrandonXavier I'm testing this myself and I'm not sending small packets, plus iptables have flagged this.
Comment: @Abs Fair enough.  My comments were based on the tcpdump from the original poster which does not appear to be coming from the same client.  How is your iptables "flagging" it?  The rules appear to be correct yet apache still receives packets that should be dropped.  IMHO time to roll up your sleeves, grab some packet captures  and start manually tinkering with your rules to find out exactly where and how it's breaking down.
Comment: Tcpdump captures all port traffic, including dropped. Are you sure that the packet you captured from the intruder is the same packet that iptables passed? I could imagine trying to defeat this setup by sending cookies with increasingly longer strings until one of them resulted in a boundary break that splits the header with IP address into two packets. Is it possible you caught a packet without cookie that was dropped but there are others with cookies that were not dropped?
Answer: Your iptables rules look fine. One can't tell for sure what they're passing, however, without logging accepts. Tcpdump won't tell you this because it runs on incoming before iptables runs. Since you have need of a load balancer, iptables-accept logs for port 80 would likely produce very large files that you'd need to manage carefully (for disk use) and you'd need scripts or other tools to analyze them. However, that's what you'd need to do to find out what's getting through.
What I can tell you from the above, though, is that there's an inherent leak problem in using network-packet string matching for application-level filtering. Packet boundaries do not respect application boundaries, so in a high-attack environment, some of these requests will leak through. This is a statistical effect. With enough requests directed at your system, the probability that some of them will be split into more than one packet increases. That alone can account for the leaks. Hackers can tilt the odds in their favor by inserting headers that increase packet size. But volume alone is enough.
For thorough filtering you need to filter at the application level. Apache provides several mechanisms for that. You can block users identified by fail2ban with a .htaccess rule for X-Forwarded-For. You can also filter in your Location directive. I don't see an option for doing this within fail2ban, nor a standalone utility that does this, but a custom script to sync fail2ban jail-ees with an apache filter would be one way to implement an application-level filter.
One more consideration: You posted rules for IPv4. If you're accepting IPv6 connections on port 80, you'll want to make sure those rules are also maintained.
Answer: It sounds like fail2ban isn't actually making the new rules you think it is.  When you see the "already banned" message, do you do an "iptables -L" and see the desired rule?  An "already banned" message indicates that your detection is working, but your firewalling is not.  If you see the rule, but it's not working, that means something about the rule doesn't work, and you need to re-think the rule.
Your failregex and ignoreip tells me you just want to block everyone who isn't from your ip addresses.  That sounds a lot simpler than using fail2ban.
In fact, does your admin console even need a load balancer?  99% of the traffic to that site must be baddies trying to crack in.  If you know what addresses to allow, and your failregex and ignoreip indicates you do, you could just make the admin console accessible directly, but only to allowed IPs.
Oh, hey, this isn't over SSL is it?  If it is, iptables can't read the X-Forwarded-For header.
I really like HBruijn's idea about using cloudflares' API.  They should be doing the blocking, not you.
Finally, if you decide to do the filtering at the apache level, mod_rewrite has rewrite maps which are great for this.  The dbd and prg features enable you to change the mapping without restarting apache.
Hope some of that helps
-Dylan


When is appropriate to combine two treatment groups from one study into a composite whole for a meta-analysis?

Question: The treatment groups I would like to combine appear to be virtually the same, at least by generic label for each group, e.g., groups with the same treatment but implemented at different sites or stratified into subgroups that can be recombined into a sensible whole.  Lipsey and Wilson (2001) recommend that if this is the case, you can combine the treatment groups into a composite whole if appropriate statistics are available.  Which statistics are they referring to?  How can I go about appropriately combining these groups?
Comment: Can you be a little more specific about the application and perhaps provide a more elaborate example otherwise you risk an answer that states: "It depends on your situation"
Comment: Trials with multiple sites are often combined using the Mantel Haenszel method (http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704-EP713_Confounding-EM/BS704-EP713_Confounding-EM7.html) rather than just simply adding the sums together.
Answer: I am trying to answer despite this being an old question.
The key premise is whether the study you are talking about is a randomized trial, and these are separate randomization arms. If this is the case, then you can safely pool the groups (remember though that this must be clinically sound).
If the study is not randomized, or the groups are not generated through randomization, then it depends on the comparison/effect estimate of interest. You can use stratification (eg the CMH approach recommended by abousetta), or meta-regression. However, inferences will be weak and the main scope of your work will be hypothesis generating.


Is there a shift and copy cpu instruction that can be accessed from c#?

Question: I need to take an 8 bit number on a 64 bit cpu and shift it to the right 8 times. Each time I shift the number I need to shift the same 8 bit number in behind it so that I end up with the same 8 bit number repeating 8 times. This would end up being shift, add 8, shift add 8... etc. which ends up being 40+ cycles (correct me if I'm wrong).
Is there a way to do this operation (of shift and copy) in 1 cycle so that I end up with the same value in the end?
long _value = 0;
byte _number = 7;
for (int i = 0; i < 8; i++) {
    _value = (_value << 8) + _number;
}

EDIT: I'm trying to compare a stream of chars to detect keywords. I can't use string.contains because the string value may be across the boundary of the buffer. Also, the application has to run on embedded ARM cpus as well as desktop and server CPUs. Memory usage and cpu cycles are very important.
Comment: @leppie OK, I meant `_number * 0x101010101010101` which does give the same answer.
Comment: I don't think this kind of function exists. C# is a high level language. If you need these kind of optimizations, use a low level language like C.
Comment: @Chris Unsafe allows you to use pointer not to use new instructions.
Comment: "ends up being 40+ cycles" - x86/x64 cycles are as virtual as MSIL. Very hard to predict how this will work out in pipelined RISC instructions.
Comment: Is that really faster than _value * 0x101010101010101? I'd hope the hardware multiply would be efficient here.
Comment: @Rup, I was thinking about using some sort of multiply instruction but I just don't know the math :( How did you arrive at "101010101010101"?
Comment: @Rup: `_value * 0x101010101010101 = 0`
Comment: So it's a toss up between this and lookup tables.
Comment: @kurtnelle it's 10101... because that's the first bit set on each of eight bytes - or put a different way it's the result of `_number = 1` and you can get the result for any other `_number` by multiplying up. But you're right, lookup tables are a good idea too.
Comment: @Rup: I'll add some timing for that too, once I get home (so the results are comparable) :)
Answer: Nowadays, there is no direct connection between number of instructions performed and the number of cpu cycles which are needed to execute them.
You also seem to assume that a statement in C# corresponds to a single assembly/cpu instruction which also is wrong.
Your code seem correctly do what your algorithm description says (note that long is signed, use ulong for unsigned behaviour).
If you want to use specialized cpu extensions (like mmx,sse,whatever) that can do add-shift-assignment in one instruction, you need to use assembly code. But I am not sure if such a specific  instruction exist. This may depend on the type of CPU you have.
You cannot use assembly code directly together with c#, but you can use assembly together with c (either as linked object file use make it inline assembly). The compiled c-code can be used from c#/.net with interop.
But the first and important question for you should be: What are you trying to accomplish?
I doubt that performance is this critical for your app, and even if, you should honestly ask yourself if c# is the best language for your goal.
Comment: @codymainx: There are a few places in the proposed algorithm that I;m trying to optimize. Since this is intended to work in long running application, any performance gain (without to much developer overhead) would be a plus.
Answer: Another idea would be to precompute all for all values of byte a lookup table.
var lu = new long[256];
// init
var n = 7;
var v = lu[n];

Update
Some benchmark results (in ms per 100000000 iterations):

Loop: 272
Unrolled: 207
Unsafe: 351
Lookup: 250
HenkH: 216

The unrolled version is:
long _value = 0;
byte _number = 7;

_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;
_value = (_value + _number) << 8;

The unsafe version is:
long _value = 0;
byte _number = 7;

byte* p = (byte*)&_value;

*p++ = _number;
*p++ = _number;
*p++ = _number;
*p++ = _number;
*p++ = _number;
*p++ = _number;
*p++ = _number;
*p++ = _number;

Sadly not performing :(
The lookup is just a read to an array.
All compiled for x64/release.
Comment: At least 1 thing is confirmed: You can't tell from just looking at the code ):
Comment: Leppie, there are a few tweaks you might try: `long` should have been `ulong` all along I wonder if `|` instead of `+` would be noticeable.
Comment: Won't the lookup process for the array end up taking longer? Unless it's a binary tree lookup it may perform worse than the loop?
Comment: @kurtnelle: You would have to initialize the table, but with it being so small, it would hardly take time. Also, you can just fill it with constants. And I did give the timing based on getting a value from that initialized array.
Comment: Since there are only 265 items in the array it should be best to pre compute and look up.
Comment: @kurtn: strangely enough the benchmarks say it's not. I would have guessed them fastest too.
Answer: When you want it to be fast you could at least unroll your loop:
ulong _value = 0;
byte _number = 7;

_value = _number;
_value = (_value <<  8) + _value;
_value = (_value << 16) + _value;
_value = (_value << 32) + _value;

This would have fewer branches too.
Comment: This is not the faster: you should write just one expression. The MSIL will compose the formula as RPN, avoiding several (useless) operations.
Comment: I was under the impression that the compiler / jit did a lot of things like this for you. It makes sense in this case, but I hesitate to arbitrarily unroll loops for 'performance' reasons. Its worth a try, but I would definitely benchmark it to see the actual performance gain.
Comment: The main point was using the intermediates to reduce the number of steps. I would expect/hope that the Jitter would optimize these statements as a group, otherwise @Mario is right. I would not expect the Jitter to reduce it from 8 to 4 steps.
Comment: My apologize for writing "faster" instead of "fastest".
I made some benchmarking (nothing serious though), and it is much faster. The inline formula appears to be the fastest.
I agree that the result may change on other platforms.
I wonder why the guy is looking for such a strange thing, though...
Comment: @Henk Holterman: Added benchmarks to my answer, yours comes close to the 'naive' unrolled version.
Comment: I thought about unrolling the loop, but yes the performance gains are minimal.


Runner on third tags up on caught ball, the other two do not, does the run score?

Question: Does the run count in the following situation?
The bases are loaded with no outs. The batter hits a line drive to the pitcher. The runner on third base tags up by touching third base, runs home and touches home plate. The runner on second does not tag up, the pitcher throws the ball to second base to get the out. The runner on first base also does not tag up, and they throw the ball to first base to get him out. The outs were made after the runner on third touched home plate.
Comment: Possible duplicate of [Does a run scored on a tag out on a forced runner?](http://sports.stackexchange.com/questions/5770/does-a-run-scored-on-a-tag-out-on-a-forced-runner)
Comment: It's not a duplicate. The question you are referring to has nothing to do with the ball being caught. That play is tagging the runner, I'm referring to 'tagging up'. Touching the bag after the batted ball was caught before touching the ground.
Answer: First off, the situation you described makes those runners seem like very poor base-runners.  A runner should never tag up from 3rd on a line-drive back to the pitcher (let alone any line-drive hit within the infield) ESPECIALLY with 0 outs.  The correct action to take is for the baserunners to freeze until they see the whole play.. if the ball is caught go back to the bag, if the ball is dropped then either run or stay.
Actually, I am editing my initial answer... I didn't read the whole scenario (the last sentence).. if the runner who touched home beat out the pitcher turning a triple play.. then this rule is not applied and the run should count.
A run is not scored if the runner advances to home base during a play in which the third out is made (1) by the batter-runner before he touches first base; (2) by any runner being forced out; or (3) by a preceding runner who is declared out because he failed to touch one of the bases.
(1) the batter-runner never got a chance to touch first and was indeed out before the runner who was on 3rd touched home, but the batter-runner wasn't the 3rd out..
(2) none of those runner's were forced to go the next base once the ball was caught by the pitcher
(3) I am assuming that all runners touched all of the bases (well at least the bases that they got to, and the preceding ones)
So I would say that run did count.
Comment: Ok, brings me another question...

Runners on 1st and 2nd with 1 out.
Line drive to the outfield.
Runners both score, but batter then is tagged in a rundown.
Then they notice the runner from 2nd didn't touch 3rd and appeal.

The appealed out came third... but didn't it truly come second, since he missed the bag before the tag out?  So the run scores?

And if the fielding team reverses the order, making the appeal, then tagging the runner?

(Oddly Wikipedia seems to suggest you can't make a play on another runner during appeal, or you lose it, even though play has to still be live????)
Comment: Yes, it was horrible base running. I'm guessing the runners thought the ball hit the ground though.
Comment: @KKendall Let's just say if I was the coach, they would have permanent pinch-runners if this scenario were to be true and the pitcher turned a TRIPLE PLAY
Comment: Doesn't "preceding runner" refer to a runner in front of the runner in question? I think (3) applies in the following situation: with runners on first and second and 2 outs, the batter gets a hit (let's say a double). The base runners both score, but on appeal the lead runner (who started at 2nd base) failed to touch 3rd base; the runner starting at 1st touched all the bases. On appeal at 3rd, the lead runner is declared out; as that is the third out, no runs score. (I'm not sure what happens if the trailing runner misses 3rd; I think the lead runner's run still counts.)
Comment: Likewise, it's odd to think of the more straightforward but very rare strategy moment where runners on 2nd and 3rd, 1 out... runners take off early, but the runner from 2nd trips as the ball is grounded into the infield.

You've gotta tag the runner from 2nd first before forcing the guy from 1st.  Straightforward, even usually in the situation where you may not get the guy at 1st in time by waiting.

But can't say I'd think to do it in that order!
Answer: The run scores.
Batter runner is out on the catch MLB 5.09a1. The other runners are no longer forced but are out because they failed to tag up and appeal plays are made MLB 5.09b5. The lead runner scores according to 5.08a since he crossed the plate after he tagged up and the other two outs do not meet the exception of MLB 5.08a. Since the last two outs are not forced they are commonly referred to as timed plays. That is, what happened before them counts.
MLB refers to the Major League Baseball Official Rules 2016
Yes, the runners who were on first and second made base running errors. The pitcher should also have been able to throw home to get the lead runner, so a mental error. Sounds like a Little League game to me.
Answer: The runners who failed to tag are force outs because they must return to their base to tag up before advancing. So all three outs are force outs and the runner on third does not score, regardless of when in the timeline he crosses home plate.
Comment: Welcome to Sports SE, please add references to your answer before you attract downvote.


OpenVPN: TLS error

Question: I am trying to connect a laptop running Linux Mint to a FreeBSD server running OpenVPN with no luck. I keep seeing TLS errors like this in the server log:
Tue Sep 17 23:14:51 2013 us=127496 Authenticate/Decrypt packet error: packet HMAC authentication failed
Tue Sep 17 23:14:51 2013 us=127570 TLS Error: incoming packet authentication failed from [AF_INET] \\my IP number\\

I have tried using TCP instead of UDP, authenticating by password, and various other options, all to no avail. Here is the server config:
daemon
port 1194
proto udp
dev tun
ca      /usr/local/etc/openvpn/keys/ca.crt
cert    /usr/local/etc/openvpn/keys/server.crt
key     /usr/local/etc/openvpn/keys/server.key
dh      /usr/local/etc/openvpn/keys/dh2048.pem
server 192.168.100.0 255.255.255.0
tls-server
tls-auth /usr/local/etc/openvpn/keys/ta.key 0 # This file is secret
keepalive 10 120
duplicate-cn
user nobody
group nobody
persist-key
persist-tun
status                  /var/openvpn-status.log
log-append              /var/log/openvpn.log
verb 5

I generated the ca.crt and ca.key files on the server and copied them to the client along with ta.key, after which I created the necessary client certificates.
I am using Gnome Network Manager to configure the OpenVPN client. Not sure where it is writing its settings.
Any thoughts about what might be going wrong here?
TIA
Answer: As per @DerfK comment. You need to apply correct key-direction.
Edit your VPN connection in your Network Manager.
Go to VPN tab.
Click Advanced.
Go to TLS Authentication tab
Change the Key Direction (at the bottom) to 1
Answer: Have you transferred the TA.key file to the client?  This file is a shared secret.  It must be available on the client, and referenced by the configuration.
That feature isn't required though, you could disable it on the server.
Comment: Specifically, it needs to be configured as `tls-auth ta.key 1` (instead of 0) on the client side.  Details [here](http://openvpn.net/index.php/open-source/documentation/howto.html#security)
Comment: @neuhaus it's amazing how little documentation there is w.r.t. having the key block withini the .ovpn file... I was struggling with this and the key-direction did the trick. Thanks!
Comment: If you want to use the unified .ovpn file format with the `` element, use the option `key-direction 1`.

OpenVPN: TLS error


Compile a list of unique objects using LINQ

Question: Having the pseudo code
class Bar {int BarId;}

class Foo {
  List<Bar> Bars;
}

Using Linq, how should I select all avaiable unique bars from the foo list ?
List<Foo> foos = GetFoos(); // anything

List<Bar> allBars = foos.Select(f=>f.Bars)...?

Answer: You can use SelectMany() Linq method like below:
List<Bar> allBars = foos.SelectMany(f => f.Bars).ToList();

If you need distinct instance of Bar in your list so add Distinct() like below:
List<Bar> allBars = foos.SelectMany(f => f.Bars).Distinct().ToList();

Source Microsoft Documentation:

If you want to return distinct elements from sequences of objects of
  some custom data type, you have to implement the IEquatable generic
  interface in the class.
Comment: distict should use a comparer then?
Comment: I just want to say, distict by BarId, what is the need to build an interface for this?
Comment: @Serge you don't need to define a new interface but Bar class should implement IEquatable that all.
Comment: @Serge you can use `.GroupBy(bar => bar.Id, (k, g) => g.FirstOrDefault())` isntead of `Distinct`. Alternatively, you can use `DistinctBy(bar => bar.Id)` from https://www.nuget.org/packages/MoreLinq


Unnecessary Certificate Authority in Domain

Question: I have an organization with a file server that is a secondary domain controller(2003 R2), an Exchange server(2008 R2), and a primary domain controller(2008 R2).  We also have what used to be the primary domain controller, but has been demoted to be the third domain controller(2003).
I am wanting to remove this last one entirely, but am worried it will cause issues.  Many years ago, someone set this server up as a Certificate Authority, I believe trying to do something with email encryption that never panned out.  It currently only has 3 valid certificates issues, one to each of the domain controllers.
I am not at all familiar with the CA role.  Can I simply revoke the three issued certificates and then remove the role, or will that cause some kind of unforeseen problem?
Comment: Explain some more about how you think you demoted the 3rd DC to its present role.
Comment: One of the other servers has been made the Global Catalog Server and the FSMO roles have been transferred to it.  The domain itself is healthy.  I'm just unsure about the Certificate Authority role and what repercussions there might be by removing it.
Comment: I'm making a point about your terminology which has been out-of-use since NT 4.0 - PDC Emulator is only one FSMO role and it's holder is not the "primary domain controller". The FSMO roles can be spread out amongst multiple servers. Any, or all, servers can be GC servers. None of those things are "demotions", so please don't say that.
Comment: What I meant is that it is no longer the primary DC.  Do you have any helpful advice in relation to my question?
Comment: I have done so. When you're operating in an environment of professionals, it's a good idea to not use terminology that is both old and wrong - it makes it look like you don't know your technology as well as might be hoped for. It's a learning opportunity, that you don't want to turn your nose up at.
Answer: If you aren't certain whether those issued certs are used by anything, then you can do this in a phased approach.
Revoke the certs, ensure that the DCs know that they've been revoked (using the Certificates MMC snap-in on each DC), and then wait a few weeks.
If nothing breaks, then go ahead and remove the CA role.
Comment: More advice can be found in the following places: http://support2.microsoft.com/kb/889250 http://technet.microsoft.com/en-us/library/cc771494.aspx


roles for non-authenticated users?

Question: I'm looking at making a site that provides a different 'experience' (to use the marketing term) depending on who the visitor says they are, without authentication. The functionality I'm thinking of using is to programmatically change the role from anonymous to something else.
So the from the launch page, client wants the site to look different depending on whether the visitor clicks "I'm a floor manager", "I'm a purchaser", "I'm a decision maker" and a perhaps few other choices. They are going to see different promotional material and copy depending on which link they choose, and the client wants it to appear that the whole site is geared towards them -- in other words, menus, pages, and promotional material not 'for' that user will be absent when they make that choice. It should look like several different sites, after they click out of the initial page.
So I thought that a way to do this would be to make all the content, menus, images, pdfs,  and then assign access by role. That way, if you're looking at the site as that role, you only see the things relevant to you. But we don't want the users logging in -- they just click on a button to declare what role they are, and if they are tricking us, we really don't care.
Will I be able to do this in the way I am imagining?
Answer: Roles can be added to users who are authenticated users, which means they logged in. Anonymous users are only anonymous users; no other role can be added to them.
If you are just trying to show a page differently basing on the what selected from the anonymous users, then you can use a cookie to store a value that is used to alter the user interface. In Drupal 7 there are two helper functions that can be used to delete and save a cookie containing values for an anonymous user: user_cookie_delete(), and user_cookie_save().
Comment: That would be a worse headache than changing the shown page basing on the content of a cookie. `:-)` There is just a session per each user; you cannot have multiple sessions per user, on Drupal. As far as I know, there isn't a module for that already, but you could create your own custom module that changes the page content basing on the values of one or more cookies.
Comment: This sounds like it would be a lot of work to have things (pages, menu items) display or not display, based on a user cookies. Would I accomplish this through a module, or how?
Comment: Could I have one user for each role and then just log in programmatically the appropriate user, with the wanted role, when they click on the appropriate link? Can there me multiple simultaneous sessions for the same user?
Answer: There are a couple of ways you could approach this without programmatically logging people in. A cookie is one. Another would be by taxonomy term - you can use the Contexts, Panels and/or ThemeKey modules to build out a custom look or even change the theme for a specific path or term.
Another way you could do this is through Organic Groups. Novell uses OG to define "user communities" (which sounds a lot like what you are talking about). Check out http://www.novell.com/communities/ and you'll see each group can have quite different information and look/feel.
If you anticipate these groups of users will eventually connect with the company online using OG may be the best choice.
Answer: I've tackled something similar to this in a different way using Domain Access module. Set up subdomains for each variation of the site you want, have a splash screen to let the visitor make their initial choice which then directs them to the appropriate URL. Then use content/block visibility to control what is displayed on each subdomain.

roles for non-authenticated users?


Should I sub-sample very large datasets to run the Kolmogorov-Smirnov (KS) test?

Question: I have two lists with many millions of values (one has 21,410,024,757 values and the other 10,561,427 values). When I run the two-sample KS test the p-value is 0, but many posts suggest that for such large datasets the KS test isn't reliable, e.g:
Kolmogorov-Smirnov test statistic interpretation with large samples
Is it a good approach to sub-sample the two lists of values randomly and uniformly?
Comment: You misinterpret the posts.  The KS test is *extremely* reliable with such large datasets: and because no two huge datasets have the same distribution (except when artificially generated), the result is a foregone conclusion.  That is, the KS test is *superfluous* with large datasets because you already know the result.
Comment: Why do you want to subsample, to raise the p-value? Why?
Comment: The KS test does not exaggerate anything; but it is true that it detects differences anywhere along the quantiles.  It sounds like what you are looking for is not a test, but a way to examine and compare distributions in detail.  Have you considered probability plots?  The immediate objection is that you don't want to make a plot of billions of points--but there are shortcuts.  The simplest shortcut is to subsample the data, but for more ideas see : see https://stats.stackexchange.com/questions/35220.
Comment: @whuber thanks, I understand this point. But in terms of how different are two distributions, my understanding is that the KS test would exaggerate the dissimilarity because it takes the max difference instead of the overall difference.
Comment: @Vasilis If you subsampled and got the same max difference between the CDFs, how would you interpret that?
Comment: @Dave I'd sub-sample 1000 times and take the average p-value
Comment: The p-value is a separate issue. Your previous comment was talking about the maximum vertical distance between the CDFs, not the p-value. // The p-value is separate from practical significance.
Comment: @Dave Sorry, I mean I'd take the average max. difference
Comment: So why not use the max distance of your full data set? That totally avoids the issue of the KS test becoming very powerful with millions of observations, as you are able to say that, despite the tiny p-value, the distributions are just barely different (much as you seem you would do with the average vertical distance).
Answer: In effect what you are asking for is a p-hack.
The KS test is not "unreliable" with large $n$. Quite the opposite. It is overly powered. What does the KS tell you? If two distributions are different. Lots of data means lots of power. And you now have evidence that the first population with a sample of 21 million is different than the second population with a sample of 10 million. I don't think we should be surprised by this.
The problem is that you didn't think about this before running the test. If you now set the alpha level to some very low value, you are forced to decide whether to set this value to call the test significant or non-significant, you already know the p-value. It begs the question why run the test at all?
Also a note about reporting p-values. A p-value is never 0. Even when the sample is identical in the two arms, as a formality, you should report the p-value as p < 0.001 (or whatever significant figure you choose to report p-values); declaring something to be impossible can lead to a serious diagnosis of foot-in-mouth syndrome.
I find that tests are not useful summaries of distributional differences. It's surprising how informative graphics can be to summarize these values instead. Consider simply showing two overlayed estimates of the density using a filtering process. That was if the "difference" leading to a significant result is some small, ignorable facet, then the graph tells us there is no practical difference to mind.
Comment: This is not what "p-hacking" means, and what the OP is considering is not p-hacking.  What the OP is looking for is a way to decide whether the observed differences in the distribution are important enough to be concerned about.  The proposed solution of subsampling is a bad use of statistics, but not every bad use of statistics is "p-hacking".
Comment: @Nobody if we want to get down to brass tacks, p-hacking doesn't mean anything at all. It's just another vaguery that emerged in the era of "data science". The general problem is the same: you modify an automated procedure to get a result that depends on "personal preference"; (many people do KS to show distributions are the same, two sins at once). However, even an elementary statistics education should lead you to understand that you can't decide the parameters of the test after running it. Also subsampling from a SRS is no different that resetting the level of a test.
Comment: If "p-hacking" has no meaning, then we probably shouldn't be saying it at all, since it only serves to obfuscate, rather than elucidate, right?  In fact, I think the behavior commonly described as "p-hacking" is both prevalent and important enough to have a term to describe it.  What is happening with the OP has a different motivation AND a different remedy from what we normally call "p-hacking"; therefore, it isn't useful to use the same word to describe it.
Comment: @Nobody if you're so convinced, why not answer how, if at all, subsampling actually provides a "solution" to the "problem" of having "too much data".
Comment: If you read my answer below, my answer is to consider what question you want to ask of the data, and run a test that answers that question.  It really is worth taking a moment to understand the difference in these two situations. In the case of (true) p-hacking, you are asking a question and trying to make the test give you the answer you want (usually because you need that answer in order to publish). The important thing is that in these cases there is no legitimate problem to solve; you need to just accept that your data don't show what you'd hoped they would.  In the OP's case...
Comment: ... the test is giving an answer that, though correct, is not helping the OP accomplish their goals.  In this case, there IS a legitimate problem to solve.  Subsampling the data is the wrong remedy for this problem, but accepting the results of the K-S test at face value would be equally wrong.  The OP needs to figure out what question they really wanted to ask of the data and run the appropriate analysis to answer that question.
Answer: Can you say the source of the data and what practical goal you hope to reach doing a K-S test on these two huge samples.
in particular:

With such large datasets, you should be able to see some obvious differences between the the two samples, unless they are essentially from the same distribution,

Why would a P-value of $\approx 0$ unsatisfactory for your purposes?

A major complaint about the K-S test is that it sometimes detects differences not of practical importance when sample sizes are very large.

Consider the following, using R:
<code>set.seed(2021)
x1 = rgamma(10^6, 3, .1)
x2 = rgamma(10^5, 3, .1)
x3 = rgamma(10^5, 3, .101)
ks.test(x1, x2)

        Two-sample Kolmogorov-Smirnov test

data:  x1 and x2
D = 0.003331, p-value = 0.2654
alternative hypothesis: two-sided

ks.test(x1, x3)

        Two-sample Kolmogorov-Smirnov test

data:  x1 and x3
D = 0.008329, p-value = 6.656e-06
alternative hypothesis: two-sided

mean(x1); mean(x2); mean(x3)
[1] 30.00744
[1] 30.0714
[1] 29.70087
boxplot(x1,x2,x3, col="skyblue2", horizontal=T)
</code>
Comment: While that major complaint is made, it is a feature, not a bug, of hypothesis testing. (You know this, but not every reader will.)
Comment: Thanks for the reply. At least when plotting the two distributions, they look very similar so I think it's your third point, i.e. a case of exaggerating unimportant differences. So my idea was that this sub-sampling (repeated multiple times) would remove the impact of these small differences
Comment: The source of the data is number of items purchased in a single transaction in a very large e-shop. The one dataset contains data only for customers in a single country, while the other dataset contains data from all the other countries together.
Comment: Thanks for info. Based on that I'm not surprised there are some differences between the two samples. Maybe methods of time series would help you find meaningful components of difference.
Answer: The problem with your proposed solution is that there is no good way to determine how much downsampling you should do to get the "right" answer.  Reducing the sample size will reduce the power of the test, but you have no idea how much reduction is needed.  The deeper problem is that the K-S test is telling you whether there is statistical evidence for a difference between the distributions, but what you want to know is, whether the difference is important enough to be concerned about it.  The K-S test can't tell you this.
The first thing you are going to have to do to answer that question is to decide what kinds of differences you care about.  Do you want to know if the means are different?  The variances?  The 95th percentiles?  Statistics can't answer this for you.  You have to do some thinking about what kinds of differences would cause you to do something different in your business.  Then devise tests for those differences specifically.
What form the tests you are looking for are going to take depends a little on what kinds of differences you are looking for, but generally speaking you are going to want to avoid hypothesis tests and look for estimates of confidence intervals for the population parameters you are interested in.  So, for example, if you are interested in whether the mean of one distribution exceeds the other by some particular margin (which you have predetermined is relevant to your business use), you might start with the standard error of the means of the distributions and use them to estimate the standard error of the difference between the means.  If you're interested in something more esoteric, you might have to fit some parametric distributions to the data, or use a bootstrap Monte Carlo.  Whatever you method you pick, you will need to keep its assumptions and limitations in mind and make reasonably certain that your data doesn't violate them.
The most important thing, however, is to start by thinking about what it is you are really looking for.  Why do you care whether the distributions are different?  Trying to run statistical tests before you've thought about that is futile.
Answer: I wonder, how you have such a huge data. Any way, as I understand, your primary interest is to know whether the distribution pattern is same in both the data or not?
I think, it is better, if you resort to some sort of sampling.
it is not always possible to enumerate the entire population or huge population and as is not cost-effective, sampling is always advocated.  I suggest that you have relatively a small random sample of the data ( say 10,000 each) and then go for the analysis.
Form a frequency distribution for both the sampled data and then go for K-S test or the Chi-square test, as desired.
Comment: This answer is incorrect, because the p-values for a subsample will be wrong.
Comment: Whenever data is huge, researcher resort to sampling. 1% sample registration scheme, shortly known as  1% SRS is used to estimate various fertility parameters in India and released from time to time. So, adopting sampling using proper method is quite cost effective and time saving. Of course, sampling has its own limitations also. One has to compare the gain versus the loss in going for the sampling.


Implementação relacionamento n para n no Delphi

Question: Talvez seja algo simples para a maioria, mas gostaria de saber como implementar na pratica um relacionamento n para n entre tabelas do Mysql utilizando Delphi.
Criei o diagrama de meu BD no MySQL Workbench, criei o DB, tudo certo.
Minha dúvida é: as tabelas criadas automaticamente quando é feito um relacionamento n para n eu preciso criar uma rotina para salvar os dados nesta tabela na hora de cadastrar novos registros, ou é possível fazer isso automaticamente?
Complementando a postagem:
O sistema que estou fazendo é para pesquisa de satisfação.
Terei as tabelas:
Cliente: Dados do cliente pesquisado:
Pesquisa: Cadastro da pesquisa.
Perguntas_pesquisa: tabela n:n juntando a pesquisa com as perguntas que fazer parte da pesquisa
Perguntas: Perguntas de todas as pesquisas
Perguntas_opções _resposta: tabela n:n que junta a resposta as opções de resposta da pergunta (são todas resposta de multipla escolha para cada pergunta)
Opções_respostas: Todas as opções de resposta
Respostas: Tabela da pesquisa em si, que junta a pergunta, a opção de resposta
Resposta_cliente: Liga o cliente as respostas dadas.
Para o cadastro de perguntas e resposta, acho que tranquilo pois tenho que cadastras primeiro as perguntas e respostas, para depois ligar as duas na tabela n:n, o mesmo no cadastro de opções de resposta.
Minha dúvida fica mesmo por conta da tabela  Respostas_cliente, pois na hora que cadastro uma nova resposta, já vou ter que registrar nesta tabela. Queria saber se tem algo automático para salvar estes dados relacionados.
No Delphi vou utilizar os componentes padrão para acesso a databases, ou mesmo o Zeos Access.
Abaixo o diagrama.
Talvez como eu não conheço muito bem desta parte de modelagem de DB, pode ser que exista uma forma mais simples de criar este sistema, mantendo a flexibilidade de criação de novas pesquisas, perguntas e opções de resposta.
Comment: Olá, Conseguiu resolver teu problema?
Comment: Da uma olhada nestes links, talvez encontre algo que possa ajudar. http://www.devmedia.com.br/master-detail-simples-e-descomplicado-para-iniciantes-delphi/20370 http://www.devmedia.com.br/quick-tips-mestre-detalhe-em-delphi/15676
Answer: O delphi não faz isso de forma automática. O que você pode fazer é implementar um evento afterpost da vida e gravar os registros na N to N.
Comment: Obrigado Edgar. E tem algum sistema que faça de forma automática?
Comment: @user3771516 É possível fazer isso com Hibernate, mas não creio que tenha isso para delphi. Você teria que fazer um **webservice** em **java** ou **.net** e invoca-lo usando Delphi.


R: Findest nearest smaller value/date from another dataframe

Question: I have two data frames:
Ratings:
CUSIP   Date       Rating
BAEU    01.01.2014  A+
BAEU    30.01.2015  A
BAEU    28.02.2017  BB
BAEU    28.03.2018  BB
CUOD    01.03.2010  BBB
CUOD    02.03.2012  BB
CUOD    03.03.2016  AA
CUOD    04.03.2018  C
BBAE    20.06.2009  A
BBAE    21.06.2012  A+
BBAE    22.11.2015  B-
BBAE    23.06.2016  BBB

Output:
Date        CUSIP
01.05.2014  BAEU
01.01.2015  BAEU
01.02.2015  BAEU
01.01.2017  BAEU
02.01.2017  BAEU
15.03.2018  BAEU
01.05.2010  CUOD
02.08.2012  CUOD
01.01.2016  CUOD
04.05.2018  CUOD
20.06.2010  BBAE
21.01.2012  BBAE
23.11.2015  BBAE
01.01.2016  BBAE
23.06.2016  BBAE

I'd like to add a column to the data frame "Output" that is called "Rating". This column comes from data frame "Ratings" and takes the rating, based on the CUSIP and the rating that is valid on that respective date.
The result should look as following:
Output_II (column Rating to be generated):
Date        CUSIP   Rating (to be generated, based on data frame Rating)
01.05.2014  BAEU    A+
01.01.2015  BAEU    A+
01.02.2015  BAEU    A
01.01.2017  BAEU    A
02.01.2017  BAEU    A
15.03.2018  BAEU    BB
01.05.2010  CUOD    BBB
02.08.2012  CUOD    BB
01.01.2016  CUOD    BB
04.05.2018  CUOD    C
20.06.2010  BBAE    A
21.01.2012  BBAE    A
23.11.2015  BBAE    B-
01.01.2016  BBAE    B-
23.06.2016  BBAE    BBB

I have already tried with dplyr and zoo. It looked something like this:
library(dplyr)
library(zoo)
Output_II = Output %>%
  group_by(cusip, date) %>%
  mutate(...)

However, I did not find a way to finishing the code.
Comment: It's not evident in the data that your date column is a `Date` or `POSIXct` class. Is it currently a `character` column?
Comment: That's right. It's actually a Date (Format). I.e. "2009-01-30"
Comment: merge.xts should be able to work well for these things but I just spent 15 min pulling my hair (and I did the data camp course like 2 days ago :(). It seems it can only join by index, so you'd have to split both tables first, then convert to xts, then join, then reassemble the table, then reconvert is a data.frame if you want this layout with dates unsorted, not so pretty... I'll upvote a good xts solution.
Answer: You could fully join the dataframes, arrange on CUSIP and Date, and use the na.locf() function from zoo to carry the last Rating observation forward to fill the NAs. Since you've arranged on CUSIP and Date, the missing Ratings will be replaced by the appropriate Rating observations. At last, you need to filter the resulting dataframe so that it only contains the rows from the original Output dataframe. A safe method for that last step could be a right_join with the original Output, which also assures Output is arranged in the same order as originally.
library(dplyr)
library(zoo)

Output %>%
  full_join(Ratings) %>%
  arrange(CUSIP, Date) %>%
  mutate(Rating = na.locf(Rating)) %>%
  right_join(Output)

         Date CUSIP Rating
1  2014-05-01  BAEU     A+
2  2015-01-01  BAEU     A+
3  2015-02-01  BAEU      A
4  2017-01-01  BAEU      A
5  2017-01-02  BAEU      A
6  2018-03-15  BAEU     BB
7  2010-05-01  CUOD    BBB
8  2012-08-02  CUOD     BB
9  2016-01-01  CUOD     BB
10 2018-05-04  CUOD      C
11 2010-06-20  BBAE      A
12 2012-01-21  BBAE      A
13 2015-11-23  BBAE     B-
14 2016-01-01  BBAE     B-
15 2016-06-23  BBAE    BBB

DATA:
Ratings <- read.table(text = "CUSIP   Date       Rating
                      BAEU    01.01.2014  A+
                        BAEU    30.01.2015  A
                      BAEU    28.02.2017  BB
                      BAEU    28.03.2018  BB
                      CUOD    01.03.2010  BBB
                      CUOD    02.03.2012  BB
                      CUOD    03.03.2016  AA
                      CUOD    04.03.2018  C
                      BBAE    20.06.2009  A
                      BBAE    21.06.2012  A+
                        BBAE    22.11.2015  B-
                        BBAE    23.06.2016  BBB", h = T )

Output <- read.table(text = "Date        CUSIP
01.05.2014  BAEU
                     01.01.2015  BAEU
                     01.02.2015  BAEU
                     01.01.2017  BAEU
                     02.01.2017  BAEU
                     15.03.2018  BAEU
                     01.05.2010  CUOD
                     02.08.2012  CUOD
                     01.01.2016  CUOD
                     04.05.2018  CUOD
                     20.06.2010  BBAE
                     21.01.2012  BBAE
                     23.11.2015  BBAE
                     01.01.2016  BBAE
                     23.06.2016  BBAE", h = T)

Ratings$Date <- as.Date(Ratings$Date, "%d.%m.%Y")
Output$Date <- as.Date(Output$Date, "%d.%m.%Y")

Based on comments, below code could be safer in case not all CUSIPS in Output have a Rating in Ratings from before the first CUSIP date in Output. Also, the data I used is slightly modified to show what happens:
Ratings <- read.table(text = "CUSIP   Date       Rating
                      BAEU    01.01.2014  A+
                        BAEU    30.01.2015  A
                      BAEU    28.02.2017  BB
                      BAEU    28.03.2018  BB
                      CUOD    01.03.2010  BBB
                      CUOD    02.03.2012  BB
                      CUOD    03.03.2016  AA
                      CUOD    04.03.2018  C
                      BBAE    20.06.2009  A
                      BBAE    21.06.2012  A+
                        BBAE    22.11.2015  B-
                        BBAE    23.06.2016  BBB
                      TEST 01.01.2018 AAA", h = T )

Output <- read.table(text = "Date        CUSIP
01.05.2014  BAEU
                     01.01.2015  BAEU
                     01.02.2015  BAEU
                     01.01.2017  BAEU
                     02.01.2017  BAEU
                     15.03.2018  BAEU
                     01.05.2010  CUOD
                     02.08.2012  CUOD
                     01.01.2016  CUOD
                     04.05.2018  CUOD
                     20.06.2010  BBAE
                     21.01.2012  BBAE
                     23.11.2015  BBAE
                     01.01.2016  BBAE
                     23.06.2016  BBAE
                     01.01.2017 TEST
                     01.01.2019 TEST", h = T)

Ratings$Date <- as.Date(Ratings$Date, "%d.%m.%Y")
Output$Date <- as.Date(Output$Date, "%d.%m.%Y")
library(dplyr)
library(zoo)

Output %>%
  full_join(Ratings) %>%
  arrange(CUSIP, Date) %>%
  group_by(CUSIP) %>%
  mutate(Rating = na.locf(Rating, na.rm = F)) %>%
  right_join(Output)

         Date  CUSIP Rating
       <date> <fctr> <fctr>
 1 2014-05-01   BAEU     A+
 2 2015-01-01   BAEU     A+
 3 2015-02-01   BAEU      A
 4 2017-01-01   BAEU      A
 5 2017-01-02   BAEU      A
 6 2018-03-15   BAEU     BB
 7 2010-05-01   CUOD    BBB
 8 2012-08-02   CUOD     BB
 9 2016-01-01   CUOD     BB
10 2018-05-04   CUOD      C
11 2010-06-20   BBAE      A
12 2012-01-21   BBAE      A
13 2015-11-23   BBAE     B-
14 2016-01-01   BBAE     B-
15 2016-06-23   BBAE    BBB
16 2017-01-01   TEST   <NA>
17 2019-01-01   TEST    AAA

Comment: I agree to that, surely, and you are right a group_by() function could solve this potential hazard. :)
Comment: Or just do it in 1 chain like I suggest in above comments: `Output %>%
  group_by(Date, CUSIP) %>%
  slice(1) %>%
  full_join(Ratings) %>%
  arrange(CUSIP, Date) %>%
  group_by(CUSIP) %>%
  mutate(Rating = na.locf(Rating, na.rm = F)) %>%
  right_join(Output)
`
Comment: Nice method, using `zoo::na.locf` on a full join. +1
Comment: you should really add a group_by, this is not safe as it is now
Comment: What I meant is that, if you look at your output just after the `arrange` call, you're "lucky" that you have a value from first df as a first rating value for each group, else your `na.locf` would go too far and fill values that should stay NAs (still upvoted :) )
Comment: It's very possible that by design the dates from first df expand further so there won't be issues, but using `na.locf` without isolating the groups from each other is dangerous. So I would have added a `group_by(CUSIP)` between `arrange` and `mutate`
Comment: BTW, in that case you need to add na.rm = F in the na.locf function. I have added it in my answer.
Comment: Thank you Len for your suggestion. The problem is that not all of the CUSIPs in data.frame "Ratings" are used in data.frame "Output". I guess that leads to the result/problem that I now have more rows in the final data.frame "Output_II" than in the "original" data.frame "Output". Is there a way to only add the ratings for those CUIPS that are part of data.frame "Output"?
Comment: If you end with `%>% right_join(Output)` that should not be happening. If it does, please provide some data where that happens.
Comment: Is the problem probably that not all the CUSIPs from data.frame Output are included in the data.frame "Ratings"? But this can't be my problem either...
Comment: no, because with the `right_join()` you filter on only the rows that were in Output
Comment: hi len. I have found the problem, however I am struggling in finding a solution. In my dataframe Output, I have some lines (rows) that have the same date. And then the code somehow does not work anymore. Would you mind looking again into the code?
Comment: Do you have same dates for different CUSIPs or within the same CUSIPs?
Comment: I have same dates for the same CUSIP.
Comment: I don´t see why it would not be working in that case without sample data. But you might consider to start the code with: `Output %>%   group_by(Date, CUSIP) %>%
  slice(1) %>%` and then continue with the `full_join`
Comment: let's say we change in your sample data as following: Output <- read.table(text = "Date        CUSIP
01.05.2014  BAEU
                     01.01.2015  BAEU
                     01.01.2015  BAEU
                     01.01.2015  BAEU
                     02.01.2017  BAEU
                     15.03.2018  BAEU
                     01.05.2010  CUOD
                     01.05.2010  CUOD
                     01.01.2016  CUOD
                     04.05.2018  CUOD
                     01.01.2019 TEST", h = T). In that case I have to much rows in the Output_II...
Comment: I see. That is caused by the final `right_join`. Do you have any other important variables in Output? Otherwise I'd suggest to remove the duplicate rows from Output before running this operation.
Comment: In case you do have other important variables in Output, please create a temporary Output2 before hand like this: `Output2 <- Output %>%
  group_by(Date, CUSIP) %>%
  slice(1)`  Then you can run in the code in my answer again, with one slight modification: start with `Output2` instead of `Output`
Comment: Wow, this is it Len!! I appreciate your support. For all that have the same problem, this code from Len works greatly: Output %>% group_by(Date, CUSIP) %>% slice(1) %>% full_join(Ratings) %>% arrange(CUSIP, Date) %>% group_by(CUSIP) %>% mutate(Rating = na.locf(Rating, na.rm = F)) %>% right_join(Output)


How do I create hierarchical taxonomy terms on import nodes?

Question: I use drupal 7 and the feeds module.
I have an excel file that contain 10 fields,So I want to import these 10 fields to specified content type as node and  for 3 fields of those create hierarchical taxonomy terms on importing file.
For example My excel file have 10 columns.

column 1 (cars name)

BMW
Chery
MVM

column 2 (cars model)

BMW 110
Chery tigo 5
MVM 110s

column 3 (color)

red
black
silver

column 4

...
... . . . .

I want to import this file and after importing have 3 hierarchically taxonomy like following that assigned to nodes of content type.
Taxonomy terms is not added by me to content type yet and I want to create taxonomys and terms automatically and assign automatically to nodes.
I have created 3 taxonomy vocabulary in my drupal site (but not added any terms to those):
1- Car name (taxonomy vocabulary 1)
2- Car model (taxonomy vocabulary 2)
3- Car color (taxonomy vocabulary 3)
I want to have taxonomy terms such as:
-car name :
  -BMV
       - BMV 110
           - red
  - Chery
       - tgo5
          - black
Please Help me for importing some fields of my file as taxonomy terms and sub terms .
I Can't type english verywell.sorry
thank you
Answer: First make 3 taxonomy feeds importers for each of the tree taxonomy vocabularies separately

Import the terms from the same source file you got - you need only to specify Term Name and matching the right columm under Mapping for "Taxonomy term processor"
In case you want to import taxonomy with the hierarchy, use Taxonomy CSV import/export to put the taxonomy terms.
Sample import file with terms&hierarchy should look like this:
BMW;BMW 5;
BMW;BMW 7;
Audi;R8;
Audi;A1;
After you run these  imports, make the final node importer.
Change the settings to node processor. Remember under Mapping for Node processor when you map taxonomy terms to have "Search taxonomy terms by: Term name" checked on these three columns
Comment: thank you @laniol. but ni this case taxonomy terms imported to site.I want to import hierarchicaly terms.
Comment: added paragraph about importing taxonmy tree.


How to correctly use dplyr verbs inside a function definition in r?

Question: I want to use filter and summarise from dplyr  inside my function. Without a function it works like following:
library(dplyr)
> Orange %>%
+     filter(Tree==1) %>%
+     summarise(age_max = max(age))
  age_max
1    1582

I want to do the same inside a function, but following fails:
## Function definition:

df.maker <- function(df, plant, Age){

  require(dplyr)

  dfo <- df %>%
    filter(plant==1) %>%
    summarise(age_max = max(Age))

  return(dfo)
}

## Use:
> df.maker(Orange, Tree, age)

 Rerun with Debug
 Error in as.lazy_dots(list(...)) : object 'Tree' not found

I know that similar questions have been asked before. I've also gone through some relevant links such as page1 and page2. But I can't fully grasp the concepts of NSE and SE. I tried following:
df.maker <- function(df, plant, Age){

  require(dplyr)

  dfo <- df %>%
    filter_(plant==1) %>%
    summarise_(age_max = ~max(Age))

  return(dfo)
}

But get the same error. Please help me understand what's going on. And how can I correctly create my function? Thanks!
EDIT:
I also tried following:
df.maker <- function(df, plant, Age){

  require(dplyr)

  dfo <- df %>%
    #filter_(plant==1) %>%
    summarise_(age_max = lazyeval::interp(~max(x),
                                          x = as.name(Age)))

  return(dfo)
}

> df.maker(Orange, Tree, age)
 Error in as.name(Age) : object 'age' not found

Comment: Does [this](http://stackoverflow.com/questions/40017629/using-dplyr-within-a-function-non-standard-evaluation/40018601#40018601) answer your question?
Comment: @Axeman, I tried that, as shown in my edit. But it still doesn't work. I think it has something to do with environment.
Answer: Either supply character arguments and use as.name:
df.maker1 <- function(d, plant, Age){
  require(dplyr)
  dfo <- d %>%
    filter_(lazyeval::interp(~x == 1, x = as.name(plant))) %>%
    summarise_(age_max = lazyeval::interp(~max(x), x = as.name(Age)))
  return(dfo)
}
df.maker1(Orange, 'Tree', 'age')

  age_max
1    1582

Or capture the arguments with substitute:
df.maker2 <- function(d, plant, Age){
  require(dplyr)
  plant <- substitute(plant)
  Age <- substitute(Age)

  dfo <- d %>%
    filter_(lazyeval::interp(~x == 1, x = plant)) %>%
    summarise_(age_max = lazyeval::interp(~max(x), x = Age))
  return(dfo)
}
df.maker2(Orange, Tree, age)

  age_max
1    1582

Comment: Thanks a lot! I didn't know about `substitute()`.


How to fill a textfield with appropiate username based on avatar selection

Question: I have constructed the following method fetchLastFiveLogins and the method successfully adds the users avatar to the _avatarButton.  However, when a user taps the avatar it calls the fillUserName method.  That part is working, but I can't figure out how to fill the _textfieldUsername with the username associated with the avatar.  Below is a snippet of the fetchLastFiveLogins method.
// sort / filter "results" to display the last five "lastLogin(s)"
    NSSortDescriptor *sortDescriptor = [[NSSortDescriptor alloc] initWithKey:@"lastLogin" ascending:NO];
    NSArray *sortDescriptors = [NSArray arrayWithObjects:sortDescriptor, nil];

    [request setFetchLimit:5];

    [request setSortDescriptors:sortDescriptors];

    //     fetch records and handle error
    NSError *error;
    NSArray *results = [_managedObjectContext executeFetchRequest:request error:&error];

    if (!results) {
        // handle error
        // also if there is login data handle error so app doesn't crash
    }
    Account *anAccount;

    // create a stock image
    UIImage *btnImage = [UIImage imageNamed:@"HomeBrewPoster1.jpg"];

    NSMutableArray *avatars = [NSMutableArray arrayWithCapacity:5];
    for ( anAccount in results) {
        NSLog(@"results%@",anAccount.lastLogin);
        if(anAccount.lastLogin) {
            NSLog(@"anAccount.lastLogin = %@, by:%@",anAccount.lastLogin,anAccount.username);
            if (anAccount.avatar != nil) {
                UIImage *avatarImg = [UIImage imageWithData:anAccount.avatar ];
                [avatars addObject:avatarImg];
            }
            else {
                [avatars addObject:btnImage];
            }
        }
    }
    NSLog(@"avatars array%lu",(unsigned long)avatars.count);
    for ( NSInteger i = 0; i < 4; i++) {
        // Check that we have enough logins
        if (i < results.count) {
            NSLog(@"avatars =%@",avatars[i]);

            CGFloat staticX = 0;
            CGFloat staticWidth = 80;
            CGFloat staticHeight = 80;
            CGFloat staticPadding = 5;

            _avatarButton = [UIButton buttonWithType:UIButtonTypeRoundedRect];

            // the last two values control the size of the button
            _avatarButton.frame = CGRectMake(0, 0, 80, 80);
            [_avatarButton setFrame:CGRectMake((staticX + (i * (staticHeight + staticPadding))),5,staticWidth,staticHeight)];

            // make corners round
            _avatarButton.layer.cornerRadius = 40; // value varies -- // 35 yields a pretty good circle.
            _avatarButton.clipsToBounds = YES;

            // assign method / action to button
            [_avatarButton addTarget:self action:@selector(fillUserName) forControlEvents:UIControlEventTouchDown];

            [_avatarButton setBackgroundImage:[avatars objectAtIndex:i] forState:UIControlStateNormal];

            NSLog(@"avatarImage = %@",[_avatarButton backgroundImageForState:UIControlStateNormal]);
            [_avatarScroll addSubview:_avatarButton];
        }
    }
}

Comment: avatars array does not contain lastLogin and username. How can you access those without having them in avatars array ?
Comment: @harshayarabarla the `results` array contains the `lastLogin`, `username`, and `avatar` attributes from the Core Data datamodel.
Answer: Step 1: Make the fillUserName method take the parameter fillUserName:(id) sender, so that the method will be passed the button that was pressed. You'll want to tweak the addTarget so that it uses @selector(fillUserName:) (with the colon at the end).
Step 2: You'll need to be able to distinguish which of the buttons was pressed. Often people use the tag property of UIButton (and other controls) to distinguish the buttons. If you set the tag to be the array index of the result, then the button with tag 0 is the 0th entry in your results array. See this question: How do i set and get UIButtons' tag? for some code samples.
Step 3: Ensure that you've held on to references to the results array, so that you can access it from the fillUserName: method.
Update: As I mention in my comment, below, a solution that makes use of blocks seems like a much better solution, and some folks have written some nice code to enable that.
Comment: I confess that I've never really loved this heuristic in Objective-C. But your question motivated me to find this answer, which was interesting: http://stackoverflow.com/questions/3908003/uibutton-block-equivalent-to-addtargetactionforcontrolevents-method


Is there any way to compile C++ code by pressing f5 or any other key in VScode?

Question: Is there any shortcut key in vscode to compile C++ code , like vim or atom has f5 ? Press the key and you have a black dialog window . I use vscode for competitive programming and I compile code using "g++" syntax on command prompt . Any method faster than this will work .
Comment: https://code.visualstudio.com/docs/cpp/config-msvc
Comment: Could be useful: [Execute a bat file on keypress in VsCode](https://stackoverflow.com/a/59849230/2752075)
Answer: You need to install the extension called "Code Runner". To do that :

Click on the four squares with one square not align icon on the left.
Type "Code Runner", find the extension and install it.
Also install the "C/C++" extension.
When both extensions are installed, to compile and run, press Ctrl + Alt + n.

Small tip :
If you want to use VS Code for competitive programming, you might want to change where the code runs to the terminal. As default, the c++ code will get executed in the "output" window but that is a output-only window, so you will not be able to test inputs. To run the code in the terminal (in which you can output/input) follow these steps :

Go to the Code Runner extension settings
Search for "run in terminal" and click in the small square.
Comment: Please do make use of the text formatting options available.
Comment: Do you mean I should use paragraphs or lists to make my answer clearer or someting more involved like colors or images ?
Comment: I've put lists now, is it better or do you have other propositions to make my answer better ?
Comment: @Brooke Jackson I have a problem with thi cmd. When I running it nothing happens, only this line `[Running] cd "c:\git\Cprata\" && gcc test.c -o test && "c:\git\Cprata\"test` apper in OUTPUT ?
Answer: There's the Tasks: Run Build Task VS Code command, which you can run in the command palette. It is bound by default to ctrl+shift+b, but you can rebind it to whatever you want. If you configure a default build task, it will run the default build task. Otherwise, it will open a menu for you to pick one from the available options.
The C/C++ extension should detect what compilers you have installed on your machine and provide generated build tasks in that menu. See also https://stackoverflow.com/a/77652228/11107541.
The CMake Tools extension contributes the commands CMake: Build and CMake: Build Target. For keybinding, see the command IDs cmake.build, cmake.buildAll, cmake.buildWithTarget, and cmake.buildNamedTarget.


Systems of linear equations

Question: I am I right on the following:
A 2X2 system of equation with no solution will look like this:
x+3y=3
2x+6y =-8
1/2 + 3/6 ≠ 3/8

A 2X2 system of equations with in finite solutions will look like this:
x +2y=4
2x+4y=8
1/2 + 2/4 =4/8

A 2X2 system of equations with unique solution will look like this
x  -y= 1
3x +2y=12

This is because they are multiples of each other?
Thanks!
Comment: Why do you say 4x4 when all your examples are 2x2?
Comment: Oops, i should change that. 10 hours of just linear algebra can really get to your head :/
Comment: How do you get $1/2 + 3/6 \neq 3/8$ and $1/2 + 2/4 =4/8$? And the latter is wrong.
Answer: An $m\times n$ system of $m$ linear equations in $n$ unknowns has no solutions when the system can be transformed into an equivalent system in which one of the equations is false. In this case, the system is called inconsistent. A system is inconsistent iff there is no solution.
A system has infinitely many solutions when, after simplification to remove redundancies, for example with row reduction of the equivalent matrix, you are left with less equations than unknowns. This is sometimes called underdetermined (such systems have free variables, which can be used to parametrize the solution space).
When you are left with the same number of reduced (or independent) equations as unknowns, the system has a unique solution.
The process of reducing a system of equations to eliminate redundancies can be carried out systematically, in fact with matrices, using columns to represent each variable or the constant on the RHS. The corresponding process with matrices is called row reduction or Gaussian elimination.
The $2\times2$ system in your first example
$$
\matrix{
x&+&3y&=&3\\
2x&+&6y&=&-8\\
}
$$
can be simplified/reduced by subtracting a multiple of the first equation from the second, to get the equivalent system:
$$
\matrix{
x&+&3y&=&3\\
0x&+&0y&=&-14\\
}
$$
Since the second equation $0=-14$ is false, the system has no solutions.
Note that if you divided the two original equations, you would get
an equivalent falsehood (all falsehoods are equivalent):
$$
\frac12=\frac{x+3y}{2(x+3y)}=\frac{x+3y}{2x+6y}=\frac3{-8}=-\frac38,
$$
however you cannot distribute the division over each term!
If your second example were
$$\matrix{x&+&2y&=&4\\2x&+&4y&=&8,}$$
this would be a good example of an underdetermined, but consistent system with infinitely many solutions, since the second equations is a multiple of the first, therefore redundant (contains the same information). The solutions can be parametrized as $x=4-2y$ for $y\in\mathbb{R}$ free. Or you could say $x=4-2t,~y=t$ for $t\in\mathbb{R}$ free. Again, here, you cannot divide the like parts of the first two equations to obtain another equivalent equation.
Your third example is fine. But these are all $2\times2$ examples -- two equations, and two unknowns.
Comment: Overdetermined is the wrong word.  A system with more equations than unknowns is called overdetermined, even if it does have solutions.  What you want is "inconsistent".  And to  clarify slightly: the other two cases require first of all that the system is not inconsistent.  It doesn't matter how many equations and unknowns you have if the equations are inconsistent.
Comment: @RobertIsrael: Thanks.

Systems of linear equations


How do you push an element onto a list dynamically in Redis/Ohm?

Question: This is a follow-up to this question: Setting a dynamic field in Ohm / Redis
I'm unable to find the method that I can use with the send() method, to dynamically be able to add an object to an unknown list.  I tried adding this method to the Ohm::Model class:
def add_to_list(name, obj)
    send((name.to_s + '<<').to_sym, obj)
end
h.add_to_list(:player_ids, OhmSeat.create(seat_number: 5, value: 6))

But I get
undefined method `player_ids<<'

There is a rpush method, but I can't seem to call it directly. and this doesn't work:
h.player_ids.rpush(OhmSeat.create(seat_number: 5, value: 6))

Comment: Is `player_ids<<` a valid method name? You aren't meaning `foo.player_ids.<<`, are you?
Answer: As @Andrew Grimm mentioned, you should do:
def add_to_list(name, obj)
  send(name) << obj
end

or just do:
h.player_ids << OhmSeat.create(...)


How can I calculate the limit $\lim\limits_{n \to \infty} \frac1n\ln \left( \frac{2x^n}{x^n+1} \right)$.

Question: I have the following limit to find:
$$\lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg ( \dfrac{2x^n}{x^n+1} \bigg)$$
Where $n \in \mathbb{N}^*$ and $x \in (0, \infty)$.
I almost got it. For $x  > 1$, I observed that:
$$\lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg ( \dfrac{2x^n}{x^n+1} \bigg) = \lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg ( \dfrac{2x^n}{x^n(1 + \frac{1}{x^n})} \bigg) = \lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg ( \dfrac{2}{1+\frac{1}{x^n}} \bigg)$$
Because $x>1$, we have that $x^n \rightarrow \infty$ as $n \rightarrow \infty$, so that means that we have:
$$\dfrac{1}{\infty} \cdot \ln \bigg ( \dfrac{2}{1+\frac{1}{\infty}} \bigg ) = 0 \cdot \ln 2 = 0$$
The problem I have is in calculating for $x \in (0, 1]$. If we have that $x \in (0, 1]$ that means $x^n \rightarrow 0$ as $n \to \infty$, so:
$$\lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg( \dfrac{2x^n}{x^n + 1} \bigg ) = \lim\limits_{n \to \infty} \dfrac{\ln \bigg( \dfrac{2x^n}{x^n + 1}\bigg )}{n}  $$
And I tried using L'Hospital and after a lot of calculation I ended up with
$$\ln x \lim\limits_{n \to \infty} \dfrac{x^n + 1}{x^n}$$
which is
$$\ln x\cdot \dfrac{1}{0}$$
And this is my problem. Maybe I applied L'Hospital incorrectly or something, I'm not sure. Long story short, I do not know how to calculate the following limit:
$$\lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg( \dfrac{2x^n}{x^n+1} \bigg )$$
when $x \in (0, 1]$.
Comment: In the second part, $x\in (0,1)$ open.
Answer: No L'hopital needed - you just have to use the fact that $\ln(xy) = \ln(x) + \ln(y)$ and break up the limits.
$\lim\limits_{n \to \infty} \dfrac{1}{n} \ln \bigg( \dfrac{2x^n}{x^n + 1} \bigg ) = $
$\lim\limits_{n \to \infty} \dfrac{\ln (2) + \ln(x^n) - \ln(x^n + 1)}{n} = $
$\lim\limits_{n \to \infty} \dfrac{\ln (2)}{n} + \lim\limits_{n \to \infty} \dfrac{n\cdot \ln(x)}{n} -  \lim\limits_{n \to \infty} \dfrac{\ln(x^n + 1)}{n} = $
$ 0 + \ln(x)+ \lim\limits_{n \to \infty}\dfrac{\ln(x^n + 1)}{n} = \ln(x) $
Answer: The term reduces to
\begin{align*}
\dfrac{1}{n}\log\left(2-\dfrac{2}{x^{n}+1}\right)&=\dfrac{1}{n}\log 2+\dfrac{1}{n}\log\left(1-\dfrac{1}{x^{n}+1}\right).
\end{align*}
We do the L'Hopital to the second term, it becomes
\begin{align*}
&\lim_{n\rightarrow\infty}\dfrac{\dfrac{1}{1-\dfrac{1}{x^{n}+1}}\dfrac{1}{(x^{n}+1)^{2}}x^{n}\log x}{1}\\
&=\lim_{n\rightarrow\infty}\dfrac{x^{n}+1}{x^{n}}\dfrac{1}{(x^{n}+1)^{2}}x^{n}\log x\\
&=\lim_{n\rightarrow\infty}(x^{n}+1)^{-1}\log x\\
&=\log x.
\end{align*}
Answer: Hint:
$(1/n)\log 2 +(1/n)\log x^n-(1/n)\log (x^n+1)=$
$(1/n)\log 2 + \log x -$
$(1/n)\log (x^n+1)$.
Answer: Since $\frac{d}{dn}x^n=x^n\ln x$, $\frac{d}{dn}\frac{2x^n}{x^n+1}=-2\frac{d}{dn}\frac{1}{x^n+1}=\frac{2x^n\ln x}{(x^n+1)^2}$ and $\frac{d}{dn}\ln\frac{2x^n}{x^n+1}=\frac{\ln x}{x^n+1}$. So you want $\lim_{n\to\infty}\frac{\ln x}{x^n+1}$, regardless of $x$.
Answer: You can use Cesàro theorem to obtain
$$\lim_{n\to\infty} \frac{\ln 2 +n\ln x-\ln(x^{n}+1)}n = \lim_{n\to\infty}\left(\ln x-\ln\frac{x^{n+1}+1}{x^{n}+1}\right)\xrightarrow{n\to\infty} \ln x$$
since $\frac{x^{n+1}+1}{x^{n}+1} \to 1$ for $x \in (0,1]$.


Determine why logs were purged

Question: We had some issues with a user connecting through OWA. This user used to be a server admin.
Upon checking today, all IIS logs have been removed up to and including sept 30 (the day the issues were diagnosed). We do have a back-up and of course all the passwords on the servers and accounts have been changed, and I've checked the system logs - there doesn't seem to be an IP matching that particular user's address.
The "security"-logs seem to have been purged, but there is no log event that the logs have been purged. There are also several other event logs (such as RDP) not showing the IP and dating well back into August. It seems like those logs are actually hitting their max size of 20MB and then doing some form of logrotate.
Of course, attempting to be an as good sysadmin as I can be, I'm a total controlfreak. Can anyone explain to me whether it's possible that Windows automatically purges logs every 3 months? Or can this be done webbased? (we only have OWA/ECP for Exchange 2013).
I also noticed that the physical server is running low on disk... Could that be a reason?
Comment: Or the reason could be that the annoyance consisted of your ex-admin doing some unspecified unauthorized things and removing the evidence... This is a very good reason to send at least auth logs off to another server instead of storing them all locally.
Comment: possible duplicate of [How do I deal with a compromised server?](http://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server)
Comment: If the (virtual) server had been accessed it would've been visible in the logs of the physical server (since someone had to log on through RDP there if they wanted to access it) - not the case. Furthermore, there's no erased RDP logs on the server where the logs were pruned either. Because of this I'm strongly believing that the server hasn't been compromised.
Comment: `This user used to be a server admin.` as in used to be and still employed there or as in used to be and no longer works there?  If they still work there, ask them point blank.  If they don't they should have never had access to do what they did once they stopped being an employee (including even logging into OWA).
Comment: Used to be an employee. Logging on to the OWA happened through a shared mailbox (so no files were accessed). The shared mailbox's password has been changed as well, obviously.
Comment: Man, there are way better ways to do shared mailboxes in exchange than shared passwords.
Comment: I know, but that was the situation once I got there. This isn't even my full-time occupation on the job... Though I'm stressing several issues towards management.
Answer: Logs are automatically purged (on a time- or size-based rotation) because disk space is finite. Keeping every log entry forever would fill even the largest of disks quickly, and every operating system does some kind of log pruning.
While that answers your question, it doesn't solve your problem: Someone who should not have access to your servers clearly still does (at least through OWA).
I suggest reviewing How do I deal with a compromised server? as others have suggested, and deciding how to proceed based on the feedback given there.
I also strongly advise forcing a password change for all users - If this person is logging in to things as an ex-employee who knows what other shady bits they may be retaining (copies of everyone's passwords, mayhaps?).
After that you can start thinking about a real security policy for your company so the next time you fire someone you can be sure all access is properly revoked...
Comment: Thank you for your answer; I hadn't found anything about logs being automatically purged (at least not by IIS). On the matter of passwords: I have asked everyone to change their passwords indeed (and confirm it towards me, and yes i do test if they don't use default passwords anymore). I was, indeed, planning on a security policy. The company is getting better slowly but surely.
Comment: @QuintenVK I'm not sure how log rotation is configured in Windows (I'm sure someone around here knows though) - it's also possible the logs WERE maliciously cleared by this individual. When in doubt I think you're justified in assuming the worst, especially with no logs proving otherwise....
Comment: Let me explain. The exchange server runs on hyper-V. There are event logs on the hyper-V machine, which don't show any records for RDP sessions around the time of pruning (note: the server is behind a firewall). The same goes for event logs on all the virtual machines. There are literally no traces to be found at all, while this guy isn't the most subtle type.

Determine why logs were purged