Entity Framework code first Object reference not set to an instance of an object

Question: I get the following error when I try to use EF code first,
Object reference not set to an instance of an object.

I get this when I try to run ANY query on a collection like e.g. db.Surveys.ToList(). I have also tried to run the Enable Migrations command, gives the same error.
My web app is asp.net web forms and I am using EFv6.1.1 (latest).
I have also added a connection string with the context name in my web.config.
I have been using Entityframework for ages and never had this issue, any help is appreciated
Stacktrace
System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Web.UI.ParseChildrenAttribute.GetHashCode()
   at System.Collections.Generic.ObjectEqualityComparer`1.GetHashCode(T obj)
   at System.Collections.Generic.HashSet`1.InternalGetHashCode(T item)
   at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
   at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
   at System.Collections.Generic.HashSet`1..ctor(IEnumerable`1 collection, IEqualityComparer`1 comparer)
   at System.Data.Entity.ModelConfiguration.Utilities.AttributeProvider.GetAttributes(Type type)
   at System.Data.Entity.ModelConfiguration.Utilities.AttributeProvider.<>c__DisplayClass4.<GetAttributes>b__3(PropertyInfo pi)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at System.Data.Entity.ModelConfiguration.Utilities.AttributeProvider.GetAttributes(PropertyInfo propertyInfo)
   at System.Data.Entity.ModelConfiguration.Conventions.PropertyAttributeConfigurationConvention`1.<.ctor>b__0(ConventionTypeConfiguration ec)
   at System.Data.Entity.ModelConfiguration.Conventions.TypeConvention.ApplyCore(Type memberInfo, ModelConfiguration modelConfiguration)
   at System.Data.Entity.ModelConfiguration.Conventions.TypeConventionBase.Apply(Type memberInfo, ModelConfiguration modelConfiguration)
   at System.Data.Entity.ModelConfiguration.Configuration.ConventionsConfiguration.ApplyModelConfiguration(Type type, ModelConfiguration modelConfiguration)
   at System.Data.Entity.ModelConfiguration.Conventions.Convention.ApplyModelConfiguration(Type type, ModelConfiguration modelConfiguration)
   at System.Data.Entity.ModelConfiguration.Configuration.ConventionsConfiguration.ApplyModelConfiguration(Type type, ModelConfiguration modelConfiguration)
   at System.Data.Entity.ModelConfiguration.Mappers.TypeMapper.MapEntityType(Type type)
   at System.Data.Entity.DbModelBuilder.<>c__DisplayClassd.<MapTypes>b__7(Type type)
   at System.Linq.Enumerable.WhereListIterator`1.MoveNext()
   at System.Data.Entity.Utilities.IEnumerableExtensions.Each[T](IEnumerable`1 ts, Action`1 action)
   at System.Data.Entity.DbModelBuilder.MapTypes(EdmModel model)
   at System.Data.Entity.DbModelBuilder.Build(DbProviderManifest providerManifest, DbProviderInfo providerInfo)
   at System.Data.Entity.DbModelBuilder.Build(DbConnection providerConnection)
   at System.Data.Entity.Internal.LazyInternalContext.CreateModel(LazyInternalContext internalContext)
   at System.Data.Entity.Internal.RetryLazy`2.GetValue(TInput input)
   at System.Data.Entity.Internal.LazyInternalContext.InitializeContext()
   at System.Data.Entity.Internal.LazyInternalContext.get_ModelBeingInitialized()
   at System.Data.Entity.Infrastructure.EdmxWriter.WriteEdmx(DbContext context, XmlWriter writer)
   at System.Data.Entity.Utilities.DbContextExtensions.<>c__DisplayClass1.<GetModel>b__0(XmlWriter w)
   at System.Data.Entity.Utilities.DbContextExtensions.GetModel(Action`1 writeXml)
   at System.Data.Entity.Utilities.DbContextExtensions.GetModel(DbContext context)
   at System.Data.Entity.Migrations.DbMigrator..ctor(DbMigrationsConfiguration configuration, DbContext usersContext, DatabaseExistenceState existenceState, Boolean calledByCreateDatabase)
   at System.Data.Entity.Migrations.DbMigrator..ctor(DbMigrationsConfiguration configuration)
   at System.Data.Entity.Migrations.Design.MigrationScaffolder..ctor(DbMigrationsConfiguration migrationsConfiguration)
   at System.Data.Entity.Migrations.Design.ToolingFacade.ScaffoldRunner.Run()
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)
   at System.Data.Entity.Migrations.Design.ToolingFacade.Run(BaseRunner runner)
   at System.Data.Entity.Migrations.Design.ToolingFacade.ScaffoldInitialCreate(String language, String rootNamespace)
   at System.Data.Entity.Migrations.EnableMigrationsCommand.<>c__DisplayClass2.<.ctor>b__0()
   at System.Data.Entity.Migrations.MigrationsDomainCommand.Execute(Action command)

Comment: Please add the code that throws this error
Comment: @DanielKelley I can't see the connection?
Comment: did you also run **update-database**?
Comment: @jbutler483 No didn't bother to because the add migration  command failed
Comment: possible duplicate of [What is a NullReferenceException and how do I fix it?](http://stackoverflow.com/questions/4660142/what-is-a-nullreferenceexception-and-how-do-i-fix-it)
Comment: @DanielKelley your suggestion has absolutely nothing to do with the question. The exception here isn't thrown by my code but by EF whose code I do not have access to and it is as a result of the absence of a property (albeit in my code), how EF uses that property to instantiate other classes internally is a different kettle of fish don't you think?
Comment: I love how I am running into the same "duplicate of What is a NullReferenceException and how do I fix it" comments on questions about EF Migrations. Do you guys even read these questions before firing your "this is a duplicate" load?
Answer: I found the answer.
I missed adding the public int Id {get; set; } field on one of my objects and apparently Entity Framework doesn't like that.
Comment: you should create a base class that has the Id and have all your entities inherit from it
Comment: Which would be how I'd normally do it if I wasn't in a hurry to get the bleeding prototype out. Thanks


What's wrong if I fit the auto-regression with OLS?

Question: I am doing auto-regress by usual linear regression package.
e.g.
$y_t=φx+ε_t$ with $x =y_{t-1}$
My reason is that,
Auto-regression does assumes iid errors, same for linear regression.
Linear Regression doesn't have assumption on independent variables.
What's different is merely that the independent variables is replaced with lagged dependent variable y.
But I see AR always is fitted by specific methods, I am afraid I'm doing wrong.
Comment: Actually, Hamilton ["Time Series Analysis"](http://press.princeton.edu/titles/5386.html) says explicitly on p. 123 that OLS is the common method of estimation of AR processes.
Answer: To answer the title question, fitting an AR($p$) model using OLS will yield biased estimates. The reason is that for unbiasedness, the model errors should be uncorrelated with past, current and future values of regressors, which is not the case in autoregressive models. For example, in case of AR(1)
$$ y_t=\varphi y_{t-1}+\varepsilon_t $$
(assuming zero mean for simplicity). Lag this by 1 to obtain
$$ y_{t-1}=\varphi y_{t-2}+\varepsilon_{t-1}. $$
Note that $\varepsilon_{t-1}$ enters the model of $y_{t-1}$; hence, the regressor $y_{t-1}$ will be correlated with lagged error $\varepsilon_{t-1}$. The argument is given (without proof) e.g. in this lecture note, p. 5-6.
On a positive note, OLS gives consistent estimators for an autoregressive model (see the same lecture note, p. 4-5)
Also, in my experience OLS is quite popular for fitting AR models, and is pretty standard for fitting multivariate AR, i.e. VAR, models.
Comment: Nice answer, upvoted. Unfortunately the link appears to be broken?
Comment: @DanLewis3264, I should have included a full title back then... Now it is gone.
Answer: If you fit by regressing $\mathbf{y}_{p+1:n}=(y_{p+1},...,y_n)^\top$ on its lags $X=[\mathbf{y}_{p:n-1},\mathbf{y}_{p-1:n-2},...,\mathbf{y}_{1:n-p}]$ the lags of that on you're going to be conditioning on the first $p$ values (for an AR(p)). If you fit by say maximum likelihood you're able to incorporate the likelihood for the first $p$ values.
If $n$ is not very large relative to $p$ is can sometimes make a substantial difference.
Comment: I thought that estimating by OLS would amount to effectively cutting the sample by $p$ observations so that lagged regressors could be obtained -- instead of appending the data with some arbitrary values for the negative lags. As such there would be no conditioning on the first $p$ values.
Comment: @RIchardHardy you just *exactly* described why it is conditioning on the first $p$ values. They're removed from $y$, but they're all still in $X$ in the regression (and in regression, $y$ is conditioned on $X$). ergo the likelihood you maximize is one conditioning on the first $p$ y-values. (You can also show it algebraically by decomposing the likelihood, it's quite straightforward. The likelihood for the AR can be split via the prediction error decomposition into two components, that for $y_{p+1},...,y_n$ and $y_1,...,y_p$, ...ctd
Comment: ctd... if you write the likelihood conditional on $y_1,...,y_p$ the second term drops out and you're left with the regression model).
Comment: Thanks for the explanation! Apparently, my confusion is in terminology. Also, I did not make the connection between OLS and maximum likelihood (I only thought in terms of OLS), which apparently explains conditioning. It still gives me some thought to digest.
Comment: @RIchard I was just googling for something to link you to for the algebraic version. [This](http://faculty.chicagobooth.edu/ruey.tsay/teaching/uts/lec8-08.pdf) document in section 2 shows it for the AR(1) case; the AR(p) case works the same way.
Comment: Thank you for the material. Now I see what I was missing. In an AR(1) example without intercept, I did not consider the contribution of $y_1$ to the likelihood -- I did not realize that it was informative. Intuitively, I thought that the process we started observing at time $t=1$ did not actually launch as a zero-mean realization of a random variable independent of the past; I thought of an infinite past (t=0,-1,-2, $\dotsc$) that we cannot observe. But I failed to realize that given the structure of the process, $y_1$ contributes with its unconditional density to the likelihood. Thanks again!


Cent OS - OpenVPN client connects but can't access internet

Question: I set up openVPN 2.3 on a centOS 6.5 system I could connect with my windows client to the server. client gets IP address and DNS entries but when I like to access the internet with my client it does not work.
I have set the "net.ipv4.ip_forward = 1" in "/etc/sysctl.conf". And added the firewall rule "iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE".
A "tracert 22.214.171.124" ony my client ends into a timeout.
Any hints or suggestions I can try? Thanks
Please find below my IPTable onfiguration
<code># Generated by iptables-save v1.4.7 on Sat Feb 13 16:18:46 2016
*mangle
:PREROUTING ACCEPT [2300:167838]
:INPUT ACCEPT [2300:167838]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1911:1150640]
:POSTROUTING ACCEPT [1911:1150640]
COMMIT
# Completed on Sat Feb 13 16:18:46 2016
# Generated by iptables-save v1.4.7 on Sat Feb 13 16:18:46 2016
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [15:1699]
:fail2ban-SSH - [0:0]
:fail2ban-sasl - [0:0]
:fail2ban-submission - [0:0]
-A INPUT -p tcp -m tcp --dport 25 -j fail2ban-sasl
-A INPUT -p tcp -m tcp --dport 587 -j fail2ban-submission
-A INPUT -p tcp -m tcp --dport 2022 -j fail2ban-SSH
-A INPUT -p tcp -m tcp --dport 25 -j fail2ban-sasl
-A INPUT -p tcp -m tcp --dport 587 -j fail2ban-submission
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 587 -m state --state NEW,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 110 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 995 -j ACCEPT
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 2022 -j ACCEPT
-A INPUT -p udp -m udp --dport 1194 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 1194 -j ACCEPT
-A INPUT -p tcp -m tcp --sport 1024:65535 --dport 3306 -m state --state     NEW,ESTABLISHED -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -j DROP
-A FORWARD -i tun0 -o eth0 -j ACCEPT
-A FORWARD -i eth0 -o tun0 -j ACCEPT
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 21 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 20 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -o eth0 -p tcp -m tcp --sport 3306 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 25 -j ACCEPT
-A OUTPUT -p icmp -m icmp --icmp-type 0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A fail2ban-SSH -j RETURN
-A fail2ban-sasl -j RETURN
-A fail2ban-submission -j RETURN
COMMIT
# Completed on Sat Feb 13 16:18:46 2016
# Generated by iptables-save v1.4.7 on Sat Feb 13 16:18:46 2016
*nat
:PREROUTING ACCEPT [88:5707]
:POSTROUTING ACCEPT [29:2145]
:OUTPUT ACCEPT [29:2145]
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
COMMIT
</code>
Completed on Sat Feb 13 16:18:46 2016
Answer: I could get it running. I had to add additional Forwarding rules
<code>iptables -I FORWARD 1 --source 10.8.0.0/24 -j ACCEPT
iptables -I FORWARD -i p4p1 -o tun0 -j ACCEPT
iptables -I FORWARD -o p4p1 -i tun0 -j ACCEPT
</code>
where "p4p1" is my ethernet interface. Please get the correct name of your interface from th "ifconfg" command.


Graceful shutdown on CF for a stateful app

Question: I have a game app, which is stateful, and it is deployed on Cloud Foundry. If I update the app, I need shutdown gracefully, i.e. the old version should not be stopped until all running games are finished.
According to the CF docs, when issuing cf stop, the app is only given 10 seconds to shut down after the SIGTERM is sent, before the app gets killed using SIGKILL. This is not working in my case.
I thought about pushing state into sth like a DB or Redis, and then hot-swap the running games to the new app instance, but since I am heavily using Web Sockets, this seems to create more problems than it solves, cause it would also break the existing connections.
Another solution would be to not send a cf stop, but instead add an operations endpoint to my app, like POST /api/admin/stop, which makes the app stop accepting new games, and then shutdown itself after all running games have finished.
I third option could be to change the design entirely and use sth like WebRTC as protocol, which means the app will only server the static resources, but has no active role any more in running games, cause all clients then connect to each other directly instead of through the server. I am not experienced in WebRTC though and wonder whether the solution works reliably, e.g. if some users use a VPN.
Right now I am in favor of the second option. But is it supposed to be that way that a CF app can terminate itself? And if yes, how to do it cleanly?
Or are there any other options? What's the best solution?
Answer:
According to the CF docs, when issuing cf stop, the app is only given 10 seconds to shut down after the SIGTERM is sent, before the app gets killed using SIGKILL. This is not working in my case.

This is the expected behavior. If you're not seeing that, you'd want to look into things more closely. You didn't mention the language/frameworks you're using, and some languages make it easier and some make it harder to handle signals (some even handle them by default).
I'd suggest the following:

You'd want to look into the specifics of your application's language and see how to handle signals. Makes sure that you're implementing that correctly in your application. Even in languages that handle signals automatically, you will still need to tie into some sort of shutdown hook so that you can execute your custom shutdown code and save your state.

You need to catch the signal and handle it as quickly as possible. You only get 10s before that SIGKILL is sent and then your app terminates.

I would suggest writing a trivial sample app that handles signals, you can log when the signal is caught and play around with pausing for different durations after to see how things are handled by the platform. This is also useful because 10s is just the default value for how long the platform will wait. Your platform operator can change that value, so it could potentially be shorter or longer. Doing a test like this would allow you to test and find the exact value set by your operator.

I thought about pushing state into sth like a DB or Redis, and then hot-swap the running games to the new app instance, but since I am heavily using Web Sockets, this seems to create more problems than it solves, cause it would also break the existing connections.

I don't have a lot of comments on your app's architecture as I don't know enough about what you're doing to make educated comments besides the following note.
When running on CF, you want to try and utilize services for as much state as you possibly can. Don't write to the local disk & try to make sure you have things like session state stored in a durable cache (like Redis).
Application instances can be shut down for a number of reasons. One of those is if you run cf stop, but they can also be shut down if your platform operations team is doing updates or if a Diego Cell (where your app containers live) crashes. Some of the reasons your app will be shut down are not things you can control, so you have to architect with that in mind.
The key things to keep in mind are a.) if you have multiple instances, the platform will guarantee that you always have at least one running and b.) given a.) you need to be able to balance requests across multiple app instances. If you can manage those two items, your app should run pretty well on CF.


Movieclip not orbiting correctly

Question: I'm current a student doing a flash animation. I'm new to it and algorithms.
The problem with my current project is that the movieclip I'm animating is not orbiting correctly, I followed the video properly, and the Moon, Earth, Sun thing works. However using that particular set of codes, I applied to my project, but it seems that the center is way off from my desired center.
Video link: http://www.youtube.com/watch?v=HO2YvR1r-Bo
The bottom code is called by using a timer instead of ENTER_FRAME
    function rocketMoving(){
var earthGravityX:Number = earth.x + (earth.width * .5) / 2;
var earthGravityY:Number = earth.y + (earth.height * .5) / 2;
    sinrocket += .001;
    cosrocket += .001;
    rocket.x = Math.sin(sinrocket * orbitSpeed1) * earthOrbitDiameter - earthGravityX ;
    rocket.y = Math.cos(cosrocket * orbitSpeed1) * earthOrbitDiameter - earthGravityY ;
    rocket.rotation = (Math.atan2(rocket.y-earth.y, rocket.x-earth.x) * 180 / Math.PI);

}

Answer: It sounds like you've set your registration point wrong.
At 3:00 in the video you mentioned, he sets the registration point for the sun to be top left.
He also sets the registration point for the earth to top left, and he sets the sets the registration point of the moon to the top left.
So be sure those are the same.
Looking at your code...
var earthGravityX:Number = earth.x + (earth.width * .5) / 2;
var earthGravityY:Number = earth.y + (earth.height * .5) / 2;

Beware of order of operations here!
Going by the idea that earth has top left as registration point, and earth is at 100,100, with a size of 10x10...
var earthGravityX:Number = 100 + (10 * .5) / 2;//earth.x + (earth.width * .5) / 2;
var earthGravityY:Number = 100 + (10 * .5) / 2;//earth.y + (earth.height * .5) / 2;

Turns into...
var earthGravityX:Number = 100 + 5 / 2;//earth.x + (earth.width * .5) / 2;
var earthGravityY:Number = 100 + 5 / 2;//earth.y + (earth.height * .5) / 2;

var earthGravityX:Number = 100 + 2.5;//earth.x + (earth.width * .5) / 2;
var earthGravityY:Number = 100 + 2.5;//earth.y + (earth.height * .5) / 2;

equals 102.5. That's off center.
I can't find the correct order of operations for you, however. Halving the earth's width and adding it to the X gets you the center... but what's the division by 2 for?
What happens if you remove the division?

Movieclip not orbiting correctly


how to use wso2 esb connector in proxy

Question: I download esb connector from git and follow steps to deploy foo-connector. These operation are all ok.
After I tried to create a proxy to try on this connector, and invoke by fiddler with url
"http://t00012-laptop:8010/services/foo". ESB always give response 202.
The esb proxy configuration is following:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="foo"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
        <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="BOOLEAN"/>
         <foo.foo-operation>
         </foo.foo-operation>
      </inSequence>
      <outSequence>
         <log/>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>

Answer: Have a look at foo connector source code :

foo-operation is a template that just log the message payload + the
value of a parameter named "generated_param" and then call a class org.wso2.carbon.connector.foo
This class print Hello WORLD in std
output with the value of "generated_param"

If you change your proxy def like this :
<proxy xmlns="http://ws.apache.org/ns/synapse" name="foo" transports="https,http" statistics="disable" trace="disable" startOnLoad="true">
    <target>
        <inSequence>
            <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="STRING"/>
            <foo.foo-operation>
                <generated_param>ROJ</generated_param>
            </foo.foo-operation>
        </inSequence>
    </target>
    <description/>
</proxy>

call http://t00012-laptop:8010/services/foo : you will receive an empty response with http status code 202 and will see in std output from the ESB :
[2014-09-19 08:20:44,640]  INFO - LogMediator To: /services/TestFOO, MessageID:
urn:uuid:874f494e-86fa-49b5-b0b5-2b5f781eca43, Direction: request, template_para
m = ROJ, Envelope: <?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xmlns
:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body></soapenv:Bod
y></soapenv:Envelope>
Hello WORLD foo !!! : paramter :ROJ

That's all...
Comment: it works like you describe, my following question is how to let this connector to return response to caller? The http status 202 is really not good.
Comment: You can build your own response message (remove property FORCE_SC_ACCEPTED in this case), have a look there for exemple : http://miyurudw.blogspot.fr/2012/08/how-to-create-simple-mock-services.html
Answer: If you want to return the response of endpoint to caller just add response tag inside inSequence as shown below.
 <?xml version="1.0" encoding="UTF-8"?>
    <proxy xmlns="http://ws.apache.org/ns/synapse"
           name="foo"
           transports="https,http"
           statistics="disable"
           trace="disable"
           startOnLoad="true">
       <target>
          <inSequence>
            <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" type="BOOLEAN"/>
             <foo.foo-operation>
             </foo.foo-operation>
             <respond/>
          </inSequence>
          <outSequence>
             <log/>
             <send/>
          </outSequence>
       </target>
       <description/>
    </proxy>

how to use wso2 esb connector in proxy


Expectation of the Sum of K Numbers without replacement

Question: Given $n$ numbers, where the value of each number is different, denoted as $v_1, v_2, ..., v_n$, and the probability of selecting each number is $p_1, p_2, ..., p_n$, respectively.
Now if I select $K$ numbers based on the given probabilities, where $K \leq n$, what is the expectation of the sum of those $K$ numbers? Note that the selection is without replacement, so that the $K$ numbers cannot involve duplicate numbers. I understand that if the selection is with replacement, the expectation of the sum of the $K$ numbers equals $K \times E(V)$, where $$E(V) = v_1 \times p_1 + v_2 \times p_2 + ... + v_n \times p_n.$$
Furthermore, what about the expectation of the variance of those $K$ numbers?
I am a CS PhD student who is working on a big data problem, and I don't have any statistics background. I expect that someone can give me a formula as the answer. However, if the answer is too complicated to be described by a formula or intensive computation has to be involved, an approximate answer is totally acceptable.
You can assume $n$ here is quite large, and the probability can vary a lot. In practice, the values of those probabilities come from a query log, which records a series of aggregation queries. The point is that the frequency of each number involved in the queries can be quite skew, i.e., some are rarely queried, while some are queried very frequently. You can assume the probability distribution is normal distribution, zipf distribution or any other reasonable alternatives.
The value distribution is only a contiguous subset of any possible distribution. In other words, if you have a histogram that represents a certain distribution, the all the numbers involved in this problem are the numbers all within a single bucket.
In terms of the value of K, you may assume it is always less than the number of frequently queried elements.
Comment: The expectation of the variance of the sum will be different without replacement; you'll need a finite population correction factor if there's no replacement. (To see this intuitively, note that if K=n the variance of the sum is zero, because it will always be the same number; so as K approaches n the variance of the sum will be lower.)
Comment: This question might be trickier than it may look.  Consider the case $n=2$ and $(v_1,v_2)=(0,1)$. The expected sum of two values drawn with replacement is $2p_2$ which is twice the expected sum of one value of course; but the expected sum of two values drawn without replacement obviously is $v_1+v_2=1\ne 2p_2$ except when $p_1=p_2=1/2$.
Comment: @zbicyclist Perhaps I didn't state the problem clearly. In my scenario, if K = N, then the variance of those K numbers will be the variance of the general population rather than 0.
Comment: (1) This doesn't look like a [tag:self-study] question to me: it looks like a genuine applied problem in probability. (2) How large might $n$ be? Exact solutions look impracticable except when all subsets can be enumerated. (3) If $n$ could be much greater than $20$ or so, precluding rapid enumeration, what can you say about the $p_i$? For instance, could they vary or will they all be quite close to $1/n$? This could inform efforts to find approximate answers.
Comment: Thanks for the edits. The more you can tell us about $N$, $K$, the $v_i$, and the $p_i$, the better.  For instance, if $K\max(p_i)\ll 1$ then formulas for sampling with replacement ought to be good approximations (because very few values, if any, would be selected more than once). I believe the hardest cases are where there is a wide range of values of the $p_i$--so that you cannot just replace most of them with zeros and yet with $p_i\gt 1/K$ for an appreciable number of $i$--and $K\approx N/2$.
Comment: I don't want to challenge your intuition, but I think this community could validate it or extend it if you would explain more fully what your probability model is and what you are trying to achieve. It would be a big accomplishment to avoid having to solve the problem as you have stated it! Your references to updating a probability and estimating sums already indicate that your somewhat abstract description of what you need likely does not correspond to what you are actually doing.
Comment: It's an interesting question, but I am wondering whether you need it answered. Are you really sure that sampling *without* replacement is an appropriate model for your situation? Note the tricky nature of the sampling without replacement: after $v_i$ is selected, the remaining $v_j,$ $j\ne i,$ are selected with *new* probabilities given by $p_j/(1-p_i)$, and so on until $K$ have been selected. That's why a formula is hard to come by.
Comment: @whuber I strongly agree with our analysis. The relative probability needs to be updated after each selection, so does the expectation of the remaining values. This is the most difficult part. I am wondering if there is any possible solution other than a brute force enumeration. In my case, I wish to estimate the sum of k values, and my intuition tells me that no duplicate value should be involved in a single summation.
Comment: Sorry for the confusion. My previous comment was actually consistent with your analysis. Although all the probabilities are constants, if one value is selected, the expectation of the other K-1 values should be different from E(V). The purpose of my program is that, given a K, I wish to estimate the sum of the k selected elements. Isn't it the expectation of the sum of K numbers?
Answer: This is probably in the nature of an answer that, while accurate, is
probably not that useful. Horvitz and Thompson (1952) provide results that cover this situation in general.  These results are given in terms of the
combinatorial expressions one might expect.
To keep consistent with their notation, and also to correspond better with more widely used notation, let me redefine some quantities. Let $N$ be the number of elements in the population and $n$ be the sample size.
Let $u_i$, $i=1,...,N$, represent the $N$ elements of the population, with given values $V_i$, $i=1,...,N$ and probabilities of selection $p_1,...,p_N$.  For a given sample of size $n$, let the observed values in the sample be $v_1,..., v_n$.
What is desired are the mean and variance of the sample total
$$\sum_{i=1}^n v_i.$$
As mentioned in the comments, the probability of selecting a particular sample $s = \{u_i, u_j, ..., u_t\}$ drawn in that order is
$$\textrm{Pr}(s) = p_{i_1}p_{j_2}\cdots p_{t_n},$$
where the initial probability  $p_{i_1}$ of drawing $u_i$ is given by $p_i$, the second probability $p_{j_2}$  of drawing $u_j$ is conditional on having removed $u_i$ from the population, and so forth. So each subsequent unit drawn results in a new probability distribution for the next unit (hence, the choice of different indicial letters, because each represents a different distribution.)
There are
$$S^{(i)} = n! \binom{N-1}{n-1}$$
samples of size $n$ that contain $u_i$ out of the entire population.  Note that this takes into account the $n!$ permutations of the sample.
Let $s_n^{(i)}$ denote a specific sample of size $n$ which includes $u_i$.
Then, the probability of selecting element $u_i$ is given by
$$P(u_i) = \sum  \textrm{Pr}(s_n^{(i)}),$$
where the summation is over the set of size $S^{(i)}$ of all possible samples $s_n^{(i)}$ of size $n$ that contain $u_i$.  (I changed the notation a little from the paper since it seemed confusing to me.)
Similarly, define
$$S^{(ij)} = n! \binom{N-2}{n-2}$$
as the number of samples containing both $u_i$ and $u_j$.  Then we can define the probability of a sample containing both as
$$\textrm{P}(u_i u_j) = \sum \textrm{Pr}(s_n^{(ij)}),$$
where the summation is over the set of size $S^{(ij)}$ of all possible samples $s_n^{(ij)}$ of size $n$ that contain $u_i$ and $u_j$.
The expected value is then derived as
$$E \left( \sum_{i=1}^n v_i \right) =
\sum_{i=1}^N \textrm{P}(u_i) V_i.$$
Although the variance is not derived explicitly in the paper, it could be obtained from expections of the $q$th moment
$$E \left( \sum_{i=1}^n v_i^q \right) =
\sum_{i=1}^N \textrm{P}(u_i) V_i^q$$
and the cross-products
$$E \left( \sum_{i \ne j}^n v_iv_j \right) =
\sum_{i \ne j} \textrm{P}(u_i u_j) V_i V_j.$$
In other words, it looks as though one would would need to go through all possible subsets to do these calculations.  Maybe this could be done for smaller values of $n$, though.
Horvitz, D.G. and Thompson, D.J. (1952) A generalization of
sampling without replacement from a finite universe. Journal of the
American Statistical Association 47(260): 663-685.


Parametrization of a circle (3d) with not radius explicitly given

Question: I have continuously searched for this question to get an understanding of what is going on, but I seem to only find problems where the radius is given or there are only 2 coordinates.
Parametrize the circle of radius R centered at (7,1,-2) on the plane x+2y+3z=3.
Comment: What ways do you have to get started? Hint: $R$ isn't really a variable. Try it with a number first.
Answer: You need to find a vector normal to the plane. One such is obtained using the coefficients of x,y,z so that $n=(1,2,3)$ is normal to the plane. Now you also need a pair of unit vectors each normal to $n$ and also normal to each other. This can be done using say $(2,-1,0)$ for one of them and then cross product for the third, and to make them unit vectors divide each by its length. Call these $u,v.$
Then a parametrization would be $(x,y,z)=(7,1,-2)+R(\cos t)u +R (\sin t) v.$


Is it possible to do if-else checks in Ansible?

Question: Currently, my spot EC2 provisioning play looks like this:
- name: Provisioning Spot instaces
  ec2:
    spot_price: 0.50
    spot_wait_timeout: 300
    assign_public_ip: no
    aws_access_key: "{{ aws_id }}"
    aws_secret_key: "{{ aws_key }}"
    region: "{{ aws_region }}"
    image: "{{ image_instance }}"
    instance_type: "{{ large_instance }}"
    key_name: "{{ ssh_keyname }}"
    count: 3
    state: present
    group_id: "{{ cypher_priv_sg }}"
    vpc_subnet_id: "{{ private_subnet_id }}"
    wait: true
    instance_tags:
      Name: Cypher-Worker
    #delete_on_termination: yes
  register: ec2

So, is it possible to do the following:
While provisioning a (spot)EC2 instance, I want to check incrementally (like (40%. 50%, 60%, 70% .. )) And if all fails, then create an on-demand instance. How do I do this?
Can I make use of the Blocks feature here?  If yes, then how?
Comment: Check what incrementally?
Comment: @MattSchuchard The `spot_price` value
Comment: Can you use some kind of `async/poll/register` and/or `begin/rescue` to do this? I am speaking theoretically here.
Comment: @MattSchuchard Yeah, there is the [Blocks](http://docs.ansible.com/ansible/playbooks_blocks.html) feature, but it doesn't take more than two `rescue` blocks. So, there's that :(
Answer: How about this:
 - name: Create ec2 instance
   ec2:
    key_name: "{{ key }}"
    group: "{{ group }}"
    instance_type: "{{ itype }}"
    image: "{{ image }}"
    region: "{{ region }}"
    wait: yes
    wait_timeout: 500
    volumes:
     - device_name: /dev/xvda
       volume_type: "{{ voltype }}"
       volume_size: "{{ volsize }}"
    monitoring: no
    vpc_subnet_id: "{{ subnetid }}"
   register: system

 - name: Wait for ssh
   wait_for: host={{ item.public_ip }} port=22 state=started
   with_items:
    - "{{ system.instances }}"

 - name: Name the new instance
   ec2_tag: resource={{ item.id }} region=eu-central-1 state=present
   args:
    tags:
     Name: "{{ name }}"
   with_items:
    - "{{ system.instances }}"

You (try to) create your instance and store its facts in system. If you need to check whether system actually contains what you need, you can simply add when: system.instances.id[0] is defined, or go even further - by checking whether your new instance is accessible on port 22, as in the above example.
You can use those conditional checks for another tasks, implementing a if-not logic - if the instance creation failed and it's not accessible by ssh, create an on-demand instance.
 - name: Wait for ssh
   wait_for: host={{ item.public_ip }} port=22 state=started
   with_items:
    - "{{ system.instances }}"
   register: result

Next step:
   when: '"ERROR" in result'

Answer: Example:
shell: /some/command
register: result
when: ( result is not defined ) or ( result.rc != 0 )
with_items:
  - 40
  - 50
  - 60
  - 70
  - 100 # on-demand

This will run the task with all listed items until it is successful with one of them.
UPD: This is example for shell module, I don't know ec2 module output structure, do not use it. For shell module it works fine. Why the downvotes?
Comment: Getting this error: `The error was: error while evaluating conditional (( ec2 is not defined ) or ( ec2.rc != 0 )): 'dict object' has no attribute 'rc'`
Comment: @Dawny33 for ec2 module you should use apropriate check for it's output structure. I posted example for shell module. And you should check registered variable, not module itself.


Insert query depending on value previously inserted

Question: I'm using MySQL in node.js (MySQL module) to manage my database. I have two tables related to one main table as follow:
Table 1 (Main table): Flights
+-----+-------------+-----------------+
| Id  | CreatedDate |      Notes      |
+-----+-------------+-----------------+
| 1   | 2015-01-01  | No reservations |
| 2   | 2015-02-15  | Without places  |
| ... | ...         | ...             |
+-----+-------------+-----------------+

Table 2: FlightPassengers
+----------+------------+--------+
| IdFlight | Passenger  | Arrive |
+----------+------------+--------+
| 1        | 125        | true   |
| 2        | 115        | false  |
| ...      | ...        | ...    |
+----------+------------+--------+

Table 3: PassangerParameters
+----------+-------------+-------+
| IdFlight |  Parameter  | Value |
+----------+-------------+-------+
| 1        | Age         | 12    |
| 2        | Civil status| S     |
| 2        | Age         | 25    |
| 4        | Civil status| M     |
| 4        | Age         | 40    |
| ...      | ...         | ...   |
+----------+-------------+-------+

Right now, when I want to add a flight, I have to make the INSERT INTO Flights SET ? MySQL query first and then get the Flight Id to make this others querys:
INSERT INTO FlightPassengers VALUES (?,?,?)
INSERT INTO PassangerParameters VALUES (?, ?, ?), (?, ?, ?)

to insert the values in the other tables. On this way I feel is very inefficient and uncomfortable, specially for the PassangerParameters table.
How can I make this query more efficient? There's a method that I can take to make this properly?
Comment: Executing those queries one after the other is actually the standard way of doing it. You could use PL/SQL to do it all inside the database engine, but you would still need to execute the queries sequentially. Also, storing logic in the database is usually not a great idea, so I would definitely recommend to just do it in node.
Comment: How are you accessing the database? Most APIs will return the last insert ID when executing an insert query.
Comment: @jwueller I'm using mysql module [link](https://www.npmjs.com/package/mysql) . I already get the last insert Id to update my other tables but I want to know if there's an efficient method to do this instead using last insert Id
Answer: You could use a TRIGGER to automatically insert into other tables when you insert into Flights
CREATE TRIGGER insertPassInfo
    AFTER INSERT ON Flights
    FOR EACH ROW
    BEGIN
        INSERT INTO FlightPassengers VALUES (NEW.Id, ?, ?);
        INSERT INTO PassengerParameters VALUES (NEW.Id, ?, ?), (NEW.Id, ?, ?);
    END

Comment: That's a good idea, but how do i handle the fact that `PassengerParameters `'s rows could vary in diferents requests? i.e. sometime I have 5, 3 or 1 parameter to set of a passanger


Como agrupar resultados de uma query?

Question: Tenho a seguinte consulta que traz o nome da receita e os ingredientes:
$query = ("
SELECT r.receita, i.ingrediente
FROM ptp_receitas r, ptp_receitas_ingredientes ri, ptp_ingredientes i
WHERE r.id = ri.idreceita
AND ri.idingrediente = i.id
AND r.id = ".$rid);
$db -> setQuery($query);
$results = $db -> loadObjectList();
foreach($results as $row){
    echo '
'.$row->receita.'
';
    echo '
Lista de Ingredientes:
';
    echo $row->ingrediente;

}

O problema é que desse jeito ele repete o nome da receita em todas as linhas do ingrediente, por exemplo:
Receita de Arroz
arroz
Receita de Arroz
sal

Eu queria que viesse:
Receita de arroz
arroz
sal

Answer: Faça a seguinte lógica com uma variável auxiliar que irá resolver seu problema:
$query = ("
SELECT r.receita, i.ingrediente
FROM ptp_receitas r, ptp_receitas_ingredientes ri, ptp_ingredientes i
WHERE r.id = ri.idreceita
AND ri.idingrediente = i.id
AND r.id = ".$rid);
$db -> setQuery($query);
$results = $db -> loadObjectList();

$count = 0;
foreach($results as $row){
    if($count != $row->idreceita){
        echo '
'.$row->receita.'
';
        echo '
Lista de Ingredientes:
';
        $count=$row->idreceita;
    }
    echo $row->ingrediente;

}

Dessa forma, irá imprimir
 Nome da Receita
 e a linha do
 apenas 1x por receita.
Comment: Perfeito, só alterei o idreceita por id e coloquei o r.id no SELECT. Valeu!!!
Comment: Beleza! Que bom que deu certo ;)

Como agrupar resultados de uma query?


Solutions of the equation $x^x=\frac{1}{256}$

Question: Find the solutions of the equation
$$x^x=\frac{1}{256}$$
I know that the function $f(x)=x^x$ is defined for $x>0$, so the solutions, if they exists, must be $>0$; as far as I know, this is because to maintain the formal properties of power, we must impose that $x>0$ or it is easy to get contradictions like $-1=(-1)^{\frac{2}{2}}=[(-1)^2]^{\frac{1}{2}}=1^{\frac{1}{2}}=1$.
So, since the function $x^x$ has an absolute minimum at $x=\frac{1}{e}$ and it is $\left(\frac{1}{e}\right)^{\frac{1}{e}}>\frac{1}{256}$, it follows that there isn't an $x>0$ such that $x^x=\frac{1}{256}$ and it follows that the equation hasn't real solutions.
However, Wolfram|Alpha says that the equation has the integer solution $x=-4$ and it is indeed a solution, as one can check by substitution. However, for what I said before, the function $x^x$ doesn't exist for $x<0$ and so it can't be evaluated for that value.
Why is this happening? Has it something to do with complex numbers? Or finding a solution of that equation is different to consider the function $x^x$ involved? Thank you.
Comment: $x^{x}$ is defined for $x < 0$, so I don't understand the issue. $-4$ is in fact a solution as you mentioned. Your "minimum" is only a minimum for when $f$ is strictly defined for only x $> 0.$
Comment: $x^x$ is in general not real-valued for $x<0$. However, it is real-valued whenever $x$ happens to be an even integer: If $x=-2n$, then $x^x=(-2n)^{-2n}=(2n)^{-2n}$.
Comment: https://math.stackexchange.com/questions/1551470/domain-of-xx  Does this answer your question??
Comment: *Mathematica* "solves" with $$x=-\frac{\log 256}{W(-\log 256)}$$ which is $x\approx -1.05228+2.28451 i$
Answer: The problem is here:

the function $x^x$ doesn't exist for $x < 0$

This is wrong. Or, if you take it this way, then $x = -4$ cannot be and is not a solution, for then $(-4)^{-4}$ would have to be undefined as well.
You see, whenever you talk about functions, you cannot forget the domain - and in this case, the domain of $x \mapsto x^x$ for $x$ real is a bit tricky, and your question solution lies right in the tricky part. Everyone can agree that $(0, \infty)$ is part of it, but not everyone necessarily agrees on what the case is for other values of $x$. When you say

one can check by substitution [that it is indeed a solution]

you are presuming a larger domain than the one you just gave in order to do that "substitution", and that changes the answer to the question, and indeed it changes results like "that there is no solutions below $x = \frac{1}{e}$".
Now Wolfram does, indeed, presume a larger domain. In fact, it takes the domain to be all of $\mathbb{R}$, and moreover takes the codomain to be $\mathbb{C}$, so you are right that complex numbers play a role here.
Even more in fact, given that the domain is part of the definition of the function, you could say you and Wolfram are working with two different functions, that both happen to be denoted by the ambiguous notation $x^x$. Hence there is no surprise that there will be two different answers as to what the solution set is.
Answer: Wolfram seems to take as $$x^x = \frac{1}{4^4} = \frac{1}{(-4)^4} = (-4)^{-4}$$
Answer: Yes, this does have to do with complex numbers. As you correctly observed; blindly taking powers like $(-1)^{\frac12}$ will lead to seeming contradictions.
Thus in complex-analysis, we make sure to be very specific in what we mean with $(-1)^{\frac12}$. This is done by defining the complex log, where we need to pick a branch. This way we can consider $x^{x}$ on a suitable space, determined by the choice of the branch. We can then vary the branch, to check all of $\mathbb{C}$.
Hope this was enough to show you that the contradictions can be avoided and the $x<0$ cases can be considered.


Jboss - "Unable to get managed connection for java:/AppDS"

Question: Estão ocorrendo as seguintes exceções na minha aplicação:
10:52:34,506 ERROR [br.com.app.dao.NotificacaoDAO] (http--0.0.0.0-8080-77) java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,509 ERROR [br.com.app.dao.UsuarioDAO] (http--0.0.0.0-8080-55) java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,513 ERROR [br.com.app.business.NotificacaoBS] (http--0.0.0.0-8080-77) br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,513 ERROR [br.com.app.dao.MensagemDAO] (http--0.0.0.0-8080-6) java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,520 ERROR [br.com.app.business.MensagemBS] (http--0.0.0.0-8080-6) br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,522 ERROR [br.com.app.controller.serversent.ListarNovasNotificacoesController] (http--0.0.0.0-8080-77) br.com.app.exceptions.BusinessException: br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,512 ERROR [br.com.app.business.UsuarioBS] (http--0.0.0.0-8080-55) br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,530 ERROR [br.com.app.services.UsuarioService] (http--0.0.0.0-8080-55) br.com.app.exceptions.BusinessException: br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,534 ERROR [br.com.app.controller.serversent.ListarNovasMensagensController] (http--0.0.0.0-8080-6) br.com.app.exceptions.BusinessException: br.com.app.exceptions.DAOException: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS
10:52:34,585 ERROR [br.com.app.dao.UsuarioDAO] (http--0.0.0.0-8080-66) java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/AppDS

Em um minuto ela disparou várias exceções como essa /\
Sempre acontece uns 5 dias após iniciar a aplicação e ela ficar 24h online.
Eu já pesquisei sobre, mas realmente não encontrei o problema, talvez poderia ser alguma conexão aberta, mas eu fecho todas as conexões.
Alguém sabe o porque?
Obs: Eu uso o JBOSS 7.1.1
Answer: A causa mais comum para esse tipo de erro é que o banco não tem mais conexões para distribuir. Se você está liberando as conexões após o uso, pode ser que alguma outra parte do seu programa ou componente esteja abrindo conexões.
Pode ser um problema de carga também. Talvez você tenha que configurar o banco para suportar mais conexões abertas ao mesmo tempo. Mas como você diz que o problema ocorre depois de certo tempo, parece algo cumulativo.
O que você pode fazer para solucionar o problema é verificar quantas conexões estão abertas (e mesmo monitorar isto, se necessário). Isso pode ser feito pelo JBoss CLI ou através de ferramentas de monitoração. Não irão solucionar o problema mas irão mostrar se há um vazamento de conexões e lhe dar recursos para tentar resolver o problema.
Aqui tem informações de como obter estatísticas de datasources através do JBoss CLI (command line interface): https://docs.jboss.org/author/display/AS71/CLI+Recipes#CLIRecipes-Readstatisticsofactivedatasources
Ele vai retornar várias propriedades que você pode usar para descobrir o que está acontecendo. Veja os valores que estiverem nestas aqui:

"ActiveCount" => "50" - Total de conexões disponíveis e ativas no
momento
"AvailableCount" => "50" - Total de conexões disponíveis
para uso
"CreatedCount" => "30" - Total de novas conexões criadas
"DestroyedCount" => "0" - Total de conexões destruídas
"MaxUsedCount" => "50" - Total de conexões usadas

Aqui há alguns exemplos de monitoração de dados de datasources no JBoss: https://community.jboss.org/thread/175957?tstart=0
Answer: Amigo, como o próprio stacktrace diz... ocorreu um erro fazer um lookup do "java:/AppDS" dentro da sua classe NotificacaoDAO.
Provavelmente você está com algo assim: (voce não esclareceu muito bem os detalhes)
public class NotificacaoDAO {

@Resource(lookup = "java:/AppDS")
private Datasource ds

...
}

A definição do endereço JNDI no caso do JBoss aconselho o segunite padrão(Para Datasources):
java:jboss/datasources/MEU_DS

Oque acontece também é queessa DS não está corretamente registrado OU você não tem o driver instalado corretamente:
Na DOCUMENTAÇÃO está bem claro esse processo.


Paginaçao com EF 7 Asp.net 5 MSQL 2008

Question: Bom dia gostaria de saber se alguém tem alguma solução para paginação usando EF - Entity Framework para versões anteriores ao MSQL 2012
Answer: Sim, o PagedList.Mvc.
Recomendo também instalar o PagedList.EntityFramework para suporte a operações assíncronas.

Paginaçao com EF 7 Asp.net 5 MSQL 2008


Bonferroni inequality proof

Question: Is this proof for $P(\bigcup_{i=1}^n A_i)\le\sum_{i=1}^nP(A_i)$ correct?
Pf. By induction. For $n=2$, $$P(A\cup B)=P(A)+P(B)-P(A\cap B)\le P(A)+P(B)$$
Assume that the statement is true for $n-1$, that is: $$P(\bigcup_{1}^{n-1}A_i)\le\sum_{i=1}^{n-1}P(A_i)$$ Let $B=\bigcup_{1}^{n-1}A_i$,
$$P(B\cup A_n)\le P(B)+P(A_n)\le \sum_{i=1}^{n-1}P(A_i) + P(A_n)=\sum_{i=1}^{n}P(A_i)$$
Answer: Yes it's correct. Maybe it would be better to point out that you use the induction assumption in the second inequality, while the first uses the result for $n=2$.
Comment: Thanks! Any ideas for extending to a countably infinite union $$P(\bigcup_1^\infty A_i)$$?
Comment: For each $n$, $\mathbb P(\bigcup_{j=1}^nA_j)\leqslant \sum_{j=1}^\infty \mathbb P(A_j)$. Then use and show the fact that $\mathbb P(\bigcup_{j=1}^nA_j)\uparrow \mathbb P(\bigcup_{j=1}^\infty A_j)$.
Comment: What is this arrow notation?
Comment: Convergence and the sequence is non-decreasing

Bonferroni inequality proof


Help with user interface translation

Question: Our portals are using User Interface Translation to do the translations. These copies/texts are located in the front-end as part of the content, or sometimes are pop ups.
This tool is not translating. That has to be done manually and the tool only provides a box where to input the translation.
Questions:

Does Drupal allow to automatically translate interface strings by just selecting the desired target language?
Is it possible to mass-upload translations in Drupal? How would we achieve it?
Comment: Have a look into the translation management module (https://www.drupal.org/project/tmgmt). It can use external services (e.g. Google Translate API) for translation jobs. And - configured properly - may come close to what you're looking for.
Answer: This tool used to be called "Translate interface" in Drupal 7, and it seems they should have stuck to the name.

No, this tool doesn't actually translate the interface, it only stores the translations you enter there yourself, it allows you to search through existing strings and enter new translations.
As suggested in the comment you may find outside services that do translations for you.
This tool has Export and Import options (you can see them as tabs in your screenshot), you can use them to export into the usual .pot file format (common in translating) and use an application to translate through there, then import the edited file into Drupal.
There are tools you can use to convert the .pot files into CSV for example, although it's better to keep to the standards. PO and POT files

Drupal Localization project is where anyone can contribute translations for Drupal core and contributed projects. You can do your translations there and let the whole community benefit: Translate to Thai
If you have any custom modules on your site you'll have to translate yourself through that tool, Export/Import or some third party service as mentioned.

Help with user interface translation


Model averaging

Question: We have the equation:
$P(C|X)=\sum_{i}P(C|X,M_{i})P(M_{i}|X)$
I should technically be able to prove that it works even when the total number of models is 1, so I go from the right hand side of the equation
$P(C|X,M_{1})P(M_{1}|X)=\frac{P(C,X,M_{1})P(X,M_{1})}{P(X,M_{1})P(X)}=P(C,M_{1}|X)$
But I should be able to get P(C|X). Am I missing something?
Comment: Try not removing the index $i$ when doing the calculcation. What does it mean when $i=1$? Think about it.
Comment: I don't get it? i=1 means that I am considering only the first model (and in my case the only model).
Comment: If you break a conditional probability into cases, then the union of all the cases you are breaking into must give you back the whole space.
Answer: You made a mistake in your calculation. You assumed $i=1$ but forgot that in order to use the law of total probability, you must have a countable partition of the sample space.
If $i=1$, then you must have $M=\Omega$ (here $\Omega$ is representing the sample space). Assuming that the collection $\{M_i\}_i$ is a partition of $\Omega$:
$$
\begin{align}
\sum_i P(C|X,M_i)P(M_i|X) &= \sum_i \frac{P(C,X,M_i)}{P(X,M_i)}\cdot\frac{P(M_i,X)}{P(X)}\\
&=\sum_i \frac{P(C,M_i,X)}{P(X)}\\
&=\sum_i P(C,M_i|X)\\
&=P(C|X)
\end{align}
$$
where the last equality follows from the partition.
Comment: Hi, I just figured this out from your comment before you posted haha. Ultimately, it means that since there is just one model, C intersection M must just return C. Is that right?
Comment: Yes. There is only one model.
Answer: Note: The  posterior probablity:
$P(M_i|X)= \displaystyle\frac{P(X|M_i)P(M_i)}{\sum_kP(X|M_K)P(M_K)}$
So when i=1 we get that it's equal to 1, so if there is only one M, then it will happen for sure.
More inf on model averaging: http://www.stat.colostate.edu/~jah/papers/statsci.pdf

Model averaging


Supermicro H8DGU-F hangs with CentOS7

Question: I try to fix our old failing blade replacing nodes with old Supermicro 1U servers that are cheap now. I do it for my own money so price matters (company doesn't want to buy new equipement). Servers contain Supetmicro PSU with good caps from good brands, not Ablecom (I checked values and ESR of 3 of them and they are good), Supermicro H8DGU-F boards (SR5670 + SP5100 chipset), 2x Opteron 6238 12 core CPUs, 2 Intel Kawela LANs. Infiniband ConnectX2 or ConnectX3 card is inserted in the only PCIE slot of this board (infinibands are different, problem persists). We use CentOS-7 from 2019 autumn as an operating system but the shop where I bought servers say that the problem is also appeae in Windows. They say they selected best items they have and made some tests but today I faced this nasty problem again on this hardware...
The problem is that an operating system hangs spontanously mainly when CentOS starting (while initializing hardware, before Welcome... text) or when the system is under load (scientific calculation, all cores). Machine becomes inaccessible vis ssh, screen is blank, no reaction on keyboard or mouse. If it hangs while loading OS and you didn't press Esc to show messages, the bottom bar continues to move sometime, than stops. If you press Esc you may see that it hang while cheching HDD or initializing Infiniband... BIOS is updated to newest 3.5c, CMOS cleared, optimal defaults loaded. I monitored temperarures with IPMI and just with my finger, nothing bad. Voltages in IPMI and BIOS are good. Ripple at high load at 12 V rail is maximom 200 mV, I don't think it can cause hangs, and there ere different supplies.
I bought 4 servers and 6 H8DGU-F boards. Two of boards with 1.01 revision hang identically (after 2 days of load, after hours of load, or when the calculation started, or during boot), one of 1.01 revision from the same stock worked for 7 days under maximum load and was successifully rebooted about 10 times, one of 2.00 revision has all memory slots of CPU2 dead (it's not relevant, replacement was sent), one of 2.00 revision worked successifully for 9 days and started sucsessifully approximately 10 times.
What can be the reason? I can't believe serverboards are that bad. It's really frustrating. They are expensive when new, shouldn't they be reliable and durable according to their price?
Can someone, please, suggest what can be the reason?
(Sorry, it's about IPMI version of the board, so I corrected topic)
Comment: I'm not sure But I had similar issues with a DL380 I solved it by respecting exact pairs of ram & cpu  then setup bios / then setup of the SAS disk controllers & only after all of that centos did boot properly and I was able to run the installation to the end and use the server for my own trainning
Comment: I had similar but not exact problem with H8DGi-F that has onboard Adaptec and external PCI LSI RAID controllers. CentOS setup crashed when initializing drive partitions (at the very beginning of the main installation process, before formatting). To avoid it, just use Rescan drives (or so) button in setup GUI before modifying partitions, and it's better to clean the beginning of the system drive with dd before installation if you don't need existing partitions (data). But it's different issue. It's crash but not hang and it's reproducable while hangs are sporadic and seem not to depend on OS.
Answer: Looks strange that there are no answers, maybe just too old board. SO answering (partially) my question. I had a talk with one of companies I bought rhese board in. They say that this model (rev. 1.01, 2.00) is problematic, other shop confirms for rev. 1.01. From 4 boards with 2.00 revision one doesn't see memory in CPU2 slots, one reboots and has network problems, two are being tested under full load now. From 1.01 revision two boards hang (while booting, immediately after high load starts, after hours or ~2 days of high load), one board worked for 2.5 weeks under high load, one board worked for ~2 days under high load (usually, 15...20 CentOS-7 boot tests are done before puttinig the server in operation because it sometime helps to identify the hanging earlier that just putting under load).
So avoid H8DGU-F, they seem to be very undurable/unreliable, although cheap. In my situation I can't see other option because of high prices of other Opteron 6000 boards so I will check if there are 3...4 good boards to use, may be with one H8DGi or H8DG6 (they are dual-chipset versions and has onboard RAID controller so they are 2...5 times more expensive).

Supermicro H8DGU-F hangs with CentOS7


Gröbner basis of ideals

Question: I have the following problem:
Let $I, J ⊂ S $ be ideals and $<$ a monomial order on $S$. Let
$G, G'$ be Gröbner bases of $I$, respectively $J$, with respect to $<$ . Prove that if
$\,\operatorname{in}_<(g)$ and $\,\operatorname{in}_<(g')$ are relatively prime for any $g ∈ G$, $g' ∈ G'$, then $G ∪ G'$
is a Gröbner basis of $I + J$.
I would be grateful if someone would help me with some advice.
Comment: If you prove the relatively prime part then note the Chinese remainder theorem for rings

http://www.math.clemson.edu/~macaule/classes/m20_math4120/slides/math4120_lecture-7-07_h.pdf

Two ideal $I$ and $J$ are coprime  (relatively prime) if $I+J =R$ (the ring)
Answer: The argument presented here is taken from the book Ideals, Varieties, and Algorithms by Cox, Little and O'Shea. As is often the case with this kind of problem, we will apply Buchberger's criterion:

Theorem (Buchberger's criterion).
Let $I$ be an ideal of $k[x_1,\dots,x_n]$. Then a basis $G = \{g_1,\dots,g_s\}$ of $I$ is a Gröbner basis if and only if $S(g_i,g_j) \rightarrow_G 0$ for all $i \neq j$.

Here, the notation $f \rightarrow_G 0$ means that $f$ has a standard representation with respect to $G$, i.e. we can write
\begin{equation*}
f = h_1g_1 + \dots + h_sg_s,
\end{equation*}
where $\deg(f) \geq \deg(h_ig_i)$ whenever $h_ig_i \neq 0$. Notice that if $G \subseteq H$, then $f \rightarrow_G 0$ implies that $f \rightarrow_H 0$.
Write $G = \{g_1,\dots,g_s\}$ and $G'= \{g_1',\dots,g_t'\}$. Without loss of generality, suppose that all the elements of $G \cup G'$ have leading coefficient $1$. Since $G$ is a Gröbner basis of $I$, we know that $S(g_i,g_j) \rightarrow_{G \cup G'} 0$ for all $1 \leq i,j \leq s$, and similarly for $S(g_i',g_j')$. It remains to consider $S(g_i,g_j')$. Writing $g_i = x^\alpha+p$ and $g_j' = x^\beta+q$, where $\text{in}_<(g_i) = x^\alpha$ and $\text{in}_<(g_j') = x^\beta$, we compute
\begin{equation*}
S(g_i,g_j') = x^\beta\cdot g_i-x^\alpha\cdot g_j' = (g_j'-q)g_i-(g_i-p)g_j' = p\cdot g_j'-q\cdot g_i.
\end{equation*}
We claim that $S(g_i,g_j') = p\cdot g_j'-q\cdot g_i$ is a standard representation. Indeed, the leading terms $\text{in}_<(p\cdot g_j') = \text{in}_<(p)\cdot x^\beta$ and $\text{in}_<(q\cdot g_i) = \text{in}_<(q)\cdot x^\alpha$ have different degrees (otherwise $x^\alpha$ and $x^\beta$ wouldn't be relatively prime), hence
\begin{equation*}
\deg(S(g_i,g_j')) = \max\{\deg(p\cdot g_j'),\deg(q\cdot g_i)\}.
\end{equation*}

Gröbner basis of ideals


IPv6 with DHCP6: where does the default route come from?

Question: My ISP provides native IPv6 with DHCP6-PD; my router is a Netgear router running OpenWrt. Everything works fine, but I want to understand how exactly, because there are no RAs whatsoever from the ISP's routers.
The traffic dump shows two RS messages from my router, followed by DHCP6 solicit. The ISP replies with DHCP6 advertisement, and DHCP6 request/reply follows. After this the ISP's host that replied to my DHCP6 requests sends an NS message for my router's link-local address, my router sends an NA with rtr,sol flags set; my router then sends an NS for the aforementioned host which responds with an NA with the same flags. Neither of the NS/NA messages contain any route/prefix information. And at this point a default route appears and traffic starts flowing. DHCP6, obviously, does not provide routing information.
How exactly does this work and how is the default route established? I tried to research but nothing came up so far.
Router advertisements appear on the external interface sometimes (maybe once in 2-4 hours) but they do not seem to be the source for the default route in my table as they happen well after the entry appears.
Answer: Disclaimer: I don't use OpenWRT so I'm taking some stabs here...
Looks like OpenWRT uses odhcp6c as the client.  Also looks like they might "fake" a default route to the DHCP interface ?
https://openwrt.org/docs/guide-user/network/ipv6/start
(Note the section Protocol dhcpv6 options concerning default route)
https://gitlab.labs.nic.cz/turris/openwrt/commit/28c47a66d9b988e242bbba36883f639c592941a5
Comment: Regular SOHO routers do not use odhcpd, yet they still work fine under these conditions (judging by the local forums), which is why I am wondering how exactly the gateway address is derived.


How to handle server failure in an n-tier architecture?

Question: Imagine I have an n-tier architecture in an auto-scaled cloud environment with say:

a load balancer in a failover pair
reverse proxy tier
web app tier
db tier

Each tier needs to connect to the instances in the tier below.
What are the standard ways of connecting tiers to make them resilient to failure of nodes in each tier? i.e. how does each tier get the IP addresses of each node in the tier below?
For example if all reverse proxies should route traffic to all web app nodes, how could they be set up so that they don't send traffic to dead web app nodes, and so that when new web app nodes are brought online they can send traffic to it?

I could run an agent that would update all the configs to all the nodes, but it seems inefficient.
I could put an LB pair between each tier, so the tier above only needs to connect to the load balancers, but how do I handle the problem of the LBs dying? This just seems to shunt the problem of tier A needing to know the IPs of all nodes in tier B, to all nodes in tier A needing to know the IPs of all LBs between tiers A and B.

For some applications, they can implement retry logic if they contact a node in the tier below that doesn't respond, but is there any way that some middleware could direct traffic to only live nodes in the following tier?
If I was hosting on AWS I could use an ELB between tiers, but I want to know how I could achieve the same functionality myself.
I've read (briefly) about heartbeat and keepalived - are these relevant here? What are the virtual IPs they talk about and how are they managed? Are there still single points of failure using them?
Answer: Your question is How do I deal with failures?
The answer is Redundancy, or more specifically

Create a set of nodes that can do the work you need done.

Make sure they have separate power and networking paths to your core.

If you need to tolerate a failure of a single node in a set, put the set behind a load balancer as you described.
If you need to tolerate the failure of your load balancer, give it a partner.

Same caveat about separate power and networking paths.

If you need to tolerate the failure of multiple nodes go for N+S redundancy
(multiple spares ready to jump in and take over).

You can do this with Amazon ELB (if you're on EC2), the pf firewall (or pfsense) with a round-robin virtual IP, or various software load-balancing tools like haproxy (which are probably the best choice as they come with some decent failure-detection capabilities, though they do require additional hardware).
There are also dedicated commercial load balancer solutions like Cisco's content switches or content switching modules if you have the cash.

Don't forget to simulate failures in your test environment to make sure things fail over the way you expect.
Comment: Yes, I will probably use internal ELBs between my tiers but I wanted to understand how they work. It was the stuff about the virtual IPs I didn't know. Thanks anyway
Comment: @andy yeah that part is important - otherwise when you lose one of your N machines you lose 1/N of the things behind it and start having a bad day :-)
Answer: an application load balancer like haproxy does this. for example, if it detects 5xx errors from a web server, it can mark the server as failed. also, if a server fails the three-way handshake, it can mark it as failed, plus try another server while the client continues to wait.
using keepalived and heartbeat, you can have a pair of haproxy servers. if one fails, the other takes over.
i use haproxy as an example here, but pretty much any application load balancer (a.k.a. layer 4/7 load balancers) have these characteristics.
Comment: you need to look at keepalived. you assign a single IP address for keepalived to use, and it assigns the right to use it to one server. if that server dies, keepalived assigns it to the other server.
Comment: So if I put haproxy between the reverse proxy and web app tiers, would I put the IPs of both haproxy instances into the reverse proxy configs? How do I handle haproxy failing (possibly catastrophically with the instance dying and losing the IP. When another comes up it'd have a new IP)?
Comment: thanks, this was the part I was missing - handling failover of LBs using a virtual IP
Answer: The LB should monitor the proxy tier and automatically remove hosts (i.e. redirect traffic to surviving nodes) that are gone.
The reverse-proxy should again use an LB that monitors the web apps. The web-apps should be able to take over sessions from other nodes.
The web-apps should connect via LB to the db-servers.


How to use puppets stored config & concat to generate a semicolon seperated list of ip adresses?

Question: puppet version 2.7.18
stored configs (not puppetdb)
I my case i have 3 couchbase nodes, which should be concated to an couchbase connection string which looks like that:
192.168.19.12;192.168.19.40;192.168.19.66

so on each couchbase server i do something like this:
@@concat::fragment { "foo": target => '/tmp/foo', content => "$ipaddress", order => 1, }

and on the app server, which should connect to the couchbase server, i want generate a yaml config file looking like this:
  couchbase:
    class:          MyCouchbaseStorage
    param:
      connection:   MyCouchbaseConnection
      connection_param:
        username:     myusername
        password:     mypassword
        bucket:       mybucket
        host:         192.168.19.12;192.168.19.40;192.168.19.66
        persist:      1

all except the host lines are no problem, but the host entry is really tricky
i concat the hosts by collecting them with:
Concat::Fragment <<| tag == 'mycbtag' |>> { target => '/tmp/database.yml' }

so now i have the problem, that i have no ";"
calling concat like this
@@concat::fragment { "foo": target => '/tmp/foo', content => ";$ipaddress", order => 1, }

will produce:
host:         ;192.168.19.12;192.168.19.40;192.168.19.66

calling concat like that
@@concat::fragment { "foo": target => '/tmp/foo', content => "$ipaddress;", order => 1, }

will produce:
host:         192.168.19.12;192.168.19.40;192.168.19.66;

so how to modify the collected content or how do i get the desired result?
host:         192.168.19.12;192.168.19.40;192.168.19.66

Comment: Just a though, but why not use an `inline_template()` with the ruby `join()` function like in http://serverfault.com/a/350418/984
Comment: inline templates with resource collector? how should this work?
Comment: Why is `@@concat::fragment { "foo": target => '/tmp/foo', content => "$ipaddress", order => 1, }` not giving you the desired result? - As an aside, it is advisable to prefer functions from the `stdlib` module over `inline_template` for performance reasons.
Answer: I achieve this for Zookeeper using PuppetDB to retrieve the nodes, and a custom plugin to join them. The details of using PuppetDB are in the answer to my original question, and the custom plugin looks like this:
require 'puppet/face'
module Puppet::Parser::Functions
    newfunction(:comma_join_nodes, :type => :rvalue) do |args|
        query = args[0]
        fact = args[1]
        q = Puppet::Face[:query, :current].facts(query)
        return q.each_value.collect { |facts| facts[fact] }.sort.join(',')
    end
end

This should allow you to create a string in your manifest like this:
$nodes = comma_join_nodes('Class[couchbase]', ipaddress)

Which will use PuppetDB to find all nodes assigned the couchbase class, and return their IP addresses.
Note that this shoots for eventual consistency - it will take a run for all the nodes to report to PuppetDB that they have the couchbase class assigned, and only on the second run will they all connect. This works fine for my Zookeeper class, but I suppose may not quite suit for the Couchbase one.
Comment: looks like a good solution, but in my case it is a legacy system which is using puppet 2.7 with stored configs.


How to make UIBezierPath to fill from the center

Question: How do I make a UIBezierPath that starts drawing from the center and expands to the left and right?
My current solution is not using UIBezierPath but rather a UIView inside another UIView. The children view is centered inside of it's parent and I adjust child's width when I need it to expand. However, this solution is far from perfect because the view is located inside of UIStackView and the stack view is in   UICollectionView cell and it sometimes does not get the width correctly.
   override func layoutSubviews() {
    super.layoutSubviews()
    progressView.cornerRadius = cornerRadius
    clipsToBounds = true
    addSubview(progressView)
    NSLayoutConstraint.activate([
        progressView.topAnchor.constraint(equalTo: topAnchor),
        progressView.bottomAnchor.constraint(equalTo: bottomAnchor),
        progressView.centerXAnchor.constraint(equalTo: centerXAnchor)
    ])
    let width = frame.width
    let finalProgress = CGFloat(progress) * width
    let widthConstraint = progressView.widthAnchor.constraint(equalToConstant: finalProgress)
    widthConstraint.priority = UILayoutPriority(rawValue: 999)
    widthConstraint.isActive = true
}

the progress is a class property of type double and when it gets set I call layoutIfNeeded.
I also tried calling setNeedsLayout and layoutIfNeeded but it does not solve the issue. SetNeedsDisplay and invalidateIntrinzicSize do not work either.
Now, I would like to do it using bezier path, but the issue is that it starts from a given point and draws from left to right. The question is: how do I make a UIBezierPath that is centered and when I increase the width it expands (stays in center)
Here is the image of what I want to achieve.
Comment: Nevertheless the way I would do this is just what you said at first, a view that draws a horizontal shape in its own width.
Comment: That would be for some other reason that you have not revealed in your question. "after scrolling a couple of times" suggests that you are not taking cell reuse into account. But it's hard to say, because you have not shown the relevant code.
Comment: The problem is that after scrolling a couple of times, it stops updating or for some reason gets it incorrectly because it does not call layoutSubviews every time. I tried forcing layoutSubviews but it does not work.
Answer: There are quite a few options. But here are a few:

CABasicAnimation of the strokeStart and strokeEnd of CAShapeLayer, namely:

Create CAShapeLayer whose path is the final UIBezierPath (full width), but whose strokeStart and strokeEnd are both 0.5 (i.e. start half-way, and finish half-way, i.e. nothing visible yet).
func configure() {
    shapeLayer.strokeColor = strokeColor.cgColor
    shapeLayer.lineCap = .round
    setProgress(0, animated: false)
}

func updatePath() {
    let path = UIBezierPath()
    path.move(to: CGPoint(x: bounds.minX + lineWidth, y: bounds.midY))
    path.addLine(to: CGPoint(x: bounds.maxX - lineWidth, y: bounds.midY))
    shapeLayer.path = path.cgPath
    shapeLayer.lineWidth = lineWidth
}

Then, in CABasicAnimation animate the change of the strokeStart to 0 and the strokeEnd to 1 to achieve the animation growing both left and right from the middle. For example:
func setProgress(_ progress: CGFloat, animated: Bool = true) {
    let percent = max(0, min(1, progress))
    let strokeStart = (1 - percent) / 2
    let strokeEnd = 1 - (1 - percent) / 2

    if animated {
        let strokeStartAnimation = CABasicAnimation(keyPath: "strokeStart")
        strokeStartAnimation.fromValue = shapeLayer.strokeStart
        strokeStartAnimation.toValue = strokeStart

        let strokeEndAnimation = CABasicAnimation(keyPath: "strokeEnd")
        strokeEndAnimation.fromValue = shapeLayer.strokeEnd
        strokeEndAnimation.toValue = strokeEnd

        let animation = CAAnimationGroup()
        animation.animations = [strokeStartAnimation, strokeEndAnimation]

        shapeLayer.strokeStart = strokeStart
        shapeLayer.strokeEnd = strokeEnd

        layer.add(animation, forKey: nil)
    } else {
        shapeLayer.strokeStart = strokeStart
        shapeLayer.strokeEnd = strokeEnd
    }
}

Animation of the path of a CAShapeLayer

Have method that creates the UIBezierPath:
func setProgress(_ progress: CGFloat, animated: Bool = true) {
    self.progress = progress

    let cgPath = path(for: progress).cgPath

    if animated {
        let animation = CABasicAnimation(keyPath: "path")
        animation.fromValue = shapeLayer.path
        animation.toValue = cgPath

        shapeLayer.path = cgPath

        layer.add(animation, forKey: nil)
    } else {
        shapeLayer.path = cgPath
    }
}

Where
func path(for progress: CGFloat) -> UIBezierPath {
    let path = UIBezierPath()
    let percent = max(0, min(1, progress))
    let width = bounds.width - lineWidth
    path.move(to: CGPoint(x: bounds.midX - width * percent / 2 , y: bounds.midY))
    path.addLine(to: CGPoint(x: bounds.midX + width * percent / 2, y: bounds.midY))
    return path
}

When you create the CAShapeLayer call setProgress with value of 0 and no animation; and
When you want to animate it out, then call setProgress with value of 1 but with animation.

Abandon UIBezierPath/CAShapeLayer approaches and use UIView and UIView block-based animation:

Create short UIView with a backgroundColor and whose layer has a cornerRadius which is half the height of the view.
Define constraints such that the view has a zero width. E.g. you might define the centerXanchor and so that it's placed where you want it, and with a widthAnchor is zero.
Animate the constraints by setting the existing widthAnchor's constant to whatever width you want and place layoutIfNeeded inside a UIView animation block.


Find point given a line and two angles

Question: Let's say I have two points $p_1=(x_1, y_1)$ and $p_2=(x_2, y_2)$, which are given as two points of a triangle $T$. And let's say I know the angles of $T$ at $p_1$ and $p_2$.  How do I find the third point $p_3=(x_3, y_3)$ of $T$?
Comment: Can't you ask the two angels? :)
Comment: @Vakey Do you mean you know the angles subtended by $(x_3,y_3)$,$(x_1,y_1)$ at $(x_2,y_2)$ and $(x_3,y_3)$,$(x_2,y_2)$ at $(x_1,y_1)$?
Comment: Have a look at any exposition on "angle side angle"; for example http://www.mathsisfun.com/algebra/trig-solving-asa-triangles.html
Comment: @Vakey Have a look at http://zonalandeducation.com/mmts/intersections/intersectionOfTwoLines1/intersectionOfTwoLines1.html
Comment: "Find point given a line and two *angles"
Comment: What do you mean by "the angles of these two points"?
Comment: I am trying to form a triangle. So those two points form one line. And lets say that the angle of point (x1, y1) is 45, and the angle of (x2, y2) is 45. Given this information, I want find the point (x3, y3).
Comment: @Marvis I do not know what a subtend is. My mathematical knowledge is very low.
Comment: @Fixee Good article. Although, that tells me how to get two missing sides. What I am looking for the x,y coordinates of missing the point (p3) that will complete the triangle.
Comment: @Vakey Once you have the two missing sides, you can write them as lines and find the unique point of intersection.  That is your $p_3$.
Comment: @Fixee Can you show me an example of how to find p3 given that you have those lines?
Comment: @Fixee Awesome. Thanks!
Answer: I believe that this is what you want.  We can use basic linear algebra.
We have the two sides of the triangle.  I write these sides as lines:
$$f(x)=m_1x+b_1=\frac{y_1-y_3}{x_1-x_3}x+b_1$$
$$g(x)=m_2x+b_2=\frac{y_2-y_3}{x_2-x_3}x+b_2$$
$m$ is the slope of the line.  I assume that the angles you are talking about are the angles of elevation or depression of the lines $f$ and $g$.  To turn an angle into slope, we use $\tan \theta$ where $\theta$ is the angle.  Thus
$$f(x)=\tan (\theta_1)x+b_1$$
$$g(x)=\tan (\theta_2)x+b_2$$
To find $b_1$ and $b_2$, solve for them in $f(x_1)=y_1$ and $g(x_2)=y_2$ respectively.
$$f(x_1)=y_1=\tan (\theta_1)x_1+b_1\implies b_1=y_1-\tan (\theta_1)x_1$$
$$g(x_2)=y_2=\tan (\theta_2)x_2+b_2\implies b_2=y_2-\tan (\theta_2)x_2$$
To find $x_3$, we know this is the unique point such that $f(x_3)=g(x_3)$. Thus
$$
f(x_3)=g(x_3)=\tan (\theta_1)x_3+y_1-\tan (\theta_1)x_1=
\tan (\theta_2)x_3+y_2-\tan (\theta_2)x_2 \implies
x_3=
\frac{\tan (\theta_1)x_1-\tan (\theta_2)x_2+y_2-y_1}{\tan (\theta_1)-\tan (\theta_2)}$$
I'm sure there are other ways to do this, but this is the first thing that comes to mind.
Comment: Can you please check line functions . They should be  $f(x)=\tan (\theta_1)x+b_1$ and
$g(x)=\tan (\theta_2)x+b_2$. should not?
Comment: Thanks. Though I am having trouble finding x3. I used an example where x1=0, y1=4, x2=4, y2=0, θ1 = 45, θ2 = 45. Pointing this out on a graph, I have found that x3=4 and y3=4. Using your formula above I have found that arctan(θ1)−arctan(θ2) = arctan(45)−arctan(45) = 1.5-1.5 = 0. And you can't divide by zero. Is my math wrong, or is this a special case?
Comment: @Mathlover Indeed it is $\tan$, my mistake.  I never remember which one to use!
Comment: @Vakey They are not both $45^\circ$.  One should be $-45 ^\circ=315 ^\circ$
Comment: @Argon Thanks! That works. However, how do you get y3?
Comment: @Vakey $y_3=f(x_3)$
Comment: Awesome! That works. Thanks a bunch!

Find point given a line and two angles


How to disable cache in my custom contextual filter code?

Question: I have created a custom module for authentication by using access_token. The authentication is working fine. So, I thought to create a custom contextual filter and use that as a filter by access token. Please find my code below.
namespace Drupal\simple_auth\Plugin\views\argument_default;

use Drupal\views\Plugin\views\argument_default\ArgumentDefaultPluginBase;

use Symfony\Component\HttpFoundation\JsonResponse;
use Drupal\Core\Site\Settings;

/**
 * Default argument plugin for the current user's carts.
 *
 * @ViewsArgumentDefault(
 *   id = "uid_from_simple_auth_access_token",
 *   title = @Translation("User ID from Simple Auth Acess Token")
 * )
 */
class UserFilterByAccessToken extends ArgumentDefaultPluginBase implements CacheableDependencyInterface {

 /**
   * {@inheritdoc}
   */
  public function getArgument() {
    return $this->extractUserIdFromAccessToken();
  }

  public function getCacheTags() {
    return [];
  }

  public function getCacheContexts(){
    return [];
  }

  public function getCacheMaxAge(){
    return 0;
  }

  /**
   * Extracts the user ID from the access token.
   */
  protected function extractUserIdFromAccessToken() {
    $request = \Drupal::request();
    $access_token = $request->headers->get('access-token');

    if (!$access_token) {
      return new JsonResponse([
          "error" => "Access token is required"
      ], 401);
    }

    $storedRefreshToken = hash('sha256', $access_token . Settings::get('simple_auth_access_secret_key'));

    $query = \Drupal::database()->select('simple_auth_access_tokens', 'sart');
    $query->fields('sart', ['uid', 'expiration_time']);
    $query->condition('sart.token', $storedRefreshToken);
    $result = $query->execute()->fetchAssoc();

    return $result['uid'];
  }
}

The above code have created my contextual filter in the view. But my exact issue here is,
for example,
  public function getArgument() {
    return 1;
  }

This filters the userid 1, when I check the view in browser (rest export view), and if I update the code like this
  public function getArgument() {
    return 2;
  }

I am getting the same result till clearing the cache. So I need to disable cache for this operation. I have tried many solutions but not getting what I have needed. I have tried adding cache tags and I cleared it using .module file, but it is not working.
Kindly help on this and if you need more info please reply.
Comment: More like add cache context.
Comment: Yes, add a user cache context. But first, refactor your code. The module is placing the authentication code in the wrong place, it needs to be in an authentication provider. See https://www.drupal.org/docs/drupal-apis/authentication-api/create-a-custom-authentication-provider
Answer: I don't know the function cacheTags(), I assume you mean getCacheTags().
And you are returning the same cache tag for all your requests. A cache tag is data dependent and therefore must vary by data. The typical Drupal schema for individual cache tag strings is entity_type:primary_key. So if your data varies by user, you'd need something like
<?php

use \Drupal\Core\Cache\CacheableDependencyInterface;

class UserFilterByAccessToken extends ArgumentDefaultPluginBase implements CacheableDependencyInterface {
  public function getCacheTags() {
    return ['user:' . $this->extractUserIdFromAccessToken()];
  }
}

Comment: I think OP also needs `extends ArgumentDefaultPluginBase implements CacheableDependencyInterface {`.
Comment: When OP searches their project for that  exact string they will find examples in core.
Comment: @hudri, Thanks for your answer. I have updated my code please check my code above. Now here is the new scenario. **Check 1** - I am passing wrong access token -> I am getting empty result. **Check 2** - I am passing correct access token -> I am getting the required data which is good. **Check 3** - I am passing wrong token -> Here I am getting same data from **Check 2**, seems cache not clearing once the data is received.


Regarding a warning message in JAGS for R

Question: I am experimenting with rjags r package while learning Bayesian. I fitted following multivariate model using tobacco data set in rrr package.

where \gamma is a random intercept which measures the correlation between the outcomes.  Also I purposefully make some values of the response as missing to explore the results with a unbalanced structure of the data. At the same time, I added AID variable which denotes the subject ids.
I fitted this model using rjags as follows.
library(rrr)
require(dplyr)
library(rjags)
data("tobacco")
tobacco <- as_data_frame(tobacco)

tobacco$AID=seq(1:25)
tobacco[4,1]=NA
tobacco[14,1]=NA
tobacco[8,1]=NA
tobacco[6,2]=NA
tobacco[1,2]=NA
tobacco[19,2]=NA
tobacco[21,2]=NA

N1=length(tobacco$Y1.BurnRate)
bayes_model_mul="model {
for(i in 1:N1){
Y1.BurnRate[i]~dnorm(mu1[i],tau1)
Y2.PercentSugar[i]~dnorm(mu2[i],tau2)
mu1[i]=beta1[1] + beta1[2]*X2.PercentChlorine[i] + beta1[3]*X3.PercentPotassium[i] + gamma[AID[i]]
mu2[i]=beta2[1] + beta2[3]*X2.PercentChlorine[i] + beta2[2]*X1.PercentNitrogen[i]+
beta2[4]*X3.PercentPotassium[i]+gamma[AID[i]]
gamma[i] ~ dmnorm(0,tau_u)

}
for (l in 1:3) { beta1[l] ~dnorm(0, 0.001) }
for (l in 1:4) { beta2[l] ~dnorm(0, 0.001) }
tau1 ~ dgamma(.01,.01)
sigma_tau1 = 1/tau1

tau2 ~ dgamma(.01,.01)
sigma_tau2 = 1/tau2

tau_u ~ dgamma(.01,.01)
sigma_tau_u = 1/tau_u
}"

model3 <- jags.model(textConnection(bayes_model_mul),
                     data = list(Y1.BurnRate=tobacco$Y1.BurnRate,
                                 Y2.PercentSugar=tobacco$Y2.PercentSugar
                                 ,X1.PercentNitrogen=tobacco$X1.PercentNitrogen,
                                 N1=N1,X2.PercentChlorine=tobacco$X2.PercentChlorine,
                                 X3.PercentPotassium=tobacco$X3.PercentPotassium,AID=tobacco$AID),
                     n.chains=1)
params <- c('beta1','sigma_tau1','sigma_tau2','beta2','sigma_tau_u','gamma')
samps.1 <- coda.samples(model3, params, n.iter = 10000)
burn.in=1000
summary.model.1=summary(window(samps.1, start = burn.in))

I didnt get any error. But I got following warning message.
Warning message:
In FUN(X[[i]], ...) : start value not changed

Can anyone help me to figure what is this error message is about ?
Thank you.
Answer: The window function is used to select a range of samples. It takes start and end parameters. The value that start takes should be greater than the the sum of the number of iterations passed to n.adapt in jags.models and n.iter in update, if explicitly setting a number of burnin samples.
You can have a look at the object retuned by jags.model to see how the number of iterations changes.
# Define the model
model3 <- jags.model(textConnection(bayes_model_mul), n.adapt=1234,
                     data = list(Y1.BurnRate=tobacco$Y1.BurnRate,Y2.PercentSugar=tobacco$Y2.PercentSugar,X1.PercentNitrogen=tobacco$X1.PercentNitrogen,N1=N1,X2.PercentChlorine=tobacco$X2.PercentChlorine,X3.PercentPotassium=tobacco$X3.PercentPotassium,AID=tobacco$AID),
                     n.chains=1)
# Look at the number of iterations (== n.adapt)
model3$iter()
#1234

# Add some burnin iterations: iterations are updated to n.adapt + burnin
update(model3, n.iter=1111)
model3$iter()
# 2345

# Draw more samples -> iterations updated to n.adapt + burnin + n.iter
samps.1 <- coda.samples(model3, n.iter = 10000, variable.names=c('beta1','sigma_tau1','sigma_tau2','beta2','sigma_tau_u','gamma'))
model3$iter()
# 12345

When we use window the start value is expected to be greater than n.adapt + burnin as these iterations have been discarded. If it is not then a start value is extracted from the coda.samples object (see mcpar(samps.1[[1]])) and your manual start value is not used.
So using w=window(samps.1, start=2345) gives the warning that you see

Warning message:
In FUN(X[[i]], ...) : start value not changed

But the following is okay as the start is greater than n.adapt + burnin
w=window(samps.1, start=2346)

Remember these are just warnings and in this case are not that important. But if you rerun coda.samples without rerunning jags.model this will again update the iterations which makes it a bit harder to track what values to pass to window (and can then throw errors)
Comment: Hi bob; Yes my previous comment was wrong ... a couple of years since i looked at this, sorry.  For the model at the link, if you run `model.fit$iter()` after the `jags.model` stage it returns `0`, which I assume there is no [adaptation](https://www.rdocumentation.org/packages/rjags/versions/4-13/topics/adapt) done as (probably) a conjugate model is fit. Hence, the start value will be greater than zero hence no warning.  ...
Comment: In [this](https://www.rensvandeschoot.com/tutorials/rjags-how-to-get-started/) tutorial, it uses a `burnin` of 500, `n.adapt` is unspecified so it should default to 1000, but `start` is only set to 500 - yet it does not produce the "start value not changed" warning. Why is this? (given you are saying that it expects `start` to be greater than `n.adapt + burnin`)
Comment: Hi @bob; they dont use the `update` function in that tutorial, so no explicit burnin. They just cut the first 500 iterations using the `window` function, so as `start=500` is less than "*n.adapt in jags.models and n.iter in update*" (e.g. 1000) there is no warning.
Comment: But because `start=500` is less than `n.adapt=1000`, there should be a warning, right? The `n.adapt` samples should have been discarded, thus needing to start at 1001 in order to avoid the warning.
Comment: ... (ps if you run `mcpar(model.samples[[1]])` ; the first value is `n.adapt(jags.model) + n.iter(udpate) + 1`, and the `second n.adapt(jags.model) + n.iter(udpate) + n.iter(coda.samples)` and the third in thin)
Comment: ... and a quick look at the samplers (after running the model) using [runjags](https://cran.r-project.org/web/packages/runjags/vignettes/quickjags.html) indicates that they re all conjugate
Comment: Thanks for circling back on this. Your response was insightful and helped a lot! What I understand is that even if I specify `n.adapt = 1000`, you're saying it ignores that because it realizes it's a conjugate model and thus doesn't need adaptation. Makes sense! Also didn't realize that `model.fit$iter()` is different before and after `coda.samples(model.fit ...)` is run (aka didn't realize `coda.samples()` changes the `model.fit` object directly.
Comment: If we were to not specify `n.adapt` as in this case (thus letting it default to 1000), would any prespecified `model.inits` get ignored? @user20650
Comment: what do you mean by `model.inits` please @bob ? Do you mean the `inits` argument in `jags.model`? If so, no it shouldn't be ignored.. Do you have a specific example in mind that is not playing the way you expect?
Comment: Yep, I mean the inits argument, sorry. When I run `model.fit$state()[[1]]` after the `jags.model()` call, it does not reflect my initial values. However I think this might be the iteration state _after_ adaptation, thus making sense that it's does not reflect the inits I specified. Is there any way to see the chain starting from my inits? Not seeing how to get it out of the object created from `jags.model()`.
Comment: Yes, it appears the values returned by  `model.fit$state()` are not equal to the starting values for a model that requires adaptation. I am not aware of how you could get those values; and they are likely not retuned as they aren't very useful ..> will not be a Markov chain.
Comment: you could ask at https://sourceforge.net/p/mcmc-jags/discussion/ . the author is responsive
Comment: Gotcha, will do! @user20650 - thanks!


How much correlation is significant enough to build a regression model?

Question: I am trying to build a regression model using two time series data in R. There is not much correlation between the two time series, so I am using trend part of both time series(using STL decomposition) for the model. The maximum correlation between these two is about 0.6 at a fixed lag. Is this sufficient to establish that one time series affects the other and build a regression model upon them.
Comment: as climate scientists have shown us there's no need to correlate too much to make grand conclusions :)
Answer: "The maximum correlation between these two is about 0.6 at a fixed lag. Is this sufficient to establish that one time series affects the other" Correlation does not imply causation. Just because to sets of data are correlated it does not mean one affects the other. You could take a look at the Granger Causality test. This might be more aligned with what you want to do
Comment: Apologies for mixing up my words. Your answer hopefully helps; wasn't aware of this test before.
Comment: while looking up the test you mentioned, i came up against another test called convergent cross mapping. It would be very helpful if you could explain the significance of this test too. Thanks in advance


Show that the equation $x^2+xy-y^2=3$ does not have integer solutions.

Question: Show that the equation $$x^2+xy-y^2=3$$ does not have integer solutions.
I solved the equation for $x$:
$x=\displaystyle \frac{-y\pm\sqrt{y^2+4(y^2+3)}}{2}$
$\displaystyle =\frac{-y\pm\sqrt{5y^2+12}}{2}$
I was then trying to show that $\sqrt{5y^2+12}$ can not be an integer using $r^2\equiv 12 \pmod{5y^2}$.
I got stuck here.
Comment: You were close to finished. We have $5y^2+12\equiv 2\pmod{5}$, but $2$ is not congruent to a square modulo $5$. For the squares modulo $5$ are in order, congruent to $0,1,4,4,1$.
Answer: Just note that $5y^2$ is a multiple of $5$ and hence it ends on $0$ or $5$. Thus, $5y^2+12$ ends in $2$ or $7$, but there are no perfect squares that have this endings.
Comment: This is assuming that what you did was correct. I didn't bother to check. Im just answering your last question.
Comment: Yeah, I'll check that. Thank you!
Answer: Note that
$$x^2+xy-y^2=(x-2y)^2+5(xy-y^2)=(x-2y)^2\qquad({\rm mod}\>5)\ .$$
But
$$0^2=0,\quad(\pm1)^2=1,\quad(\pm2)^2=-1\qquad({\rm mod}\>5)\ ,$$
which implies that $3$ is not a quadratic residue modulo $5$.
Answer: Generally, If $f(x) \equiv 0 \pmod{mn}$ has a solution, then that means $mn \mid f(x)$ for some $x$. But since $m \mid mn$ and $n \mid mn$ you can see $m,n \mid f(x)$, i.e. $f(x) \equiv 0 \pmod{m}$ and $f(x) \equiv 0 \pmod{n}$. So in your case you could conclude $r^2 \equiv 12 \pmod{5}$ and get rid of $y^2$ that is unknown. The rest is obvious as others said.


Divergence of $\sum_{n=1}^\infty\frac{1}{\log(1+e^n)}$

Question: I'm having difficulties proving the divergence of the series
$$\sum_{n=1}^\infty\frac{1}{\log(1+e^n)}.$$
Wolfram|Alpha suggest to use the Comparison test. I tried to look for a lower estimate to force it up to infinity but I couldn't come up with something useful.
Answer: Writing $1+e^n=e^n(1+e^{-n})$, we have
$$ \log(1+e^n)=n+\log(1+e^{-n})\leq n+\log 2\leq 2n $$
for all $n\geq 1$. Therefore
$$ \frac{1}{\log(1+e^n)}\geq \frac{1}{2n} $$
and hence $\sum_{n=1}^{\infty}\frac{1}{\log(1+e^n)}$ diverges by comparison with the harmonic series.
Answer: As it is a series with positive terms, you can use equivalents and it will be  very short:
\begin{alignat}{3}1+\mathrm e^n&\sim_\infty\mathrm e^n,\enspace\text{hence}&\qquad &&\frac1{\log(1+\mathrm e^n)}\sim_\infty \frac1{\log \mathrm e^n}=\frac1n,&\qquad\qquad&\end{alignat}
which diverges.


How do I add javascript to a module?

Question: I'm building a module with a basic form. This form has some js attached to it that I want to keep in the module, so that it's not cluttering my regular scripts file. How do I add it to my module?
I tried to add this is in my .module file but no luck.
function mymodule_init() {
drupal_add_js(drupal_get_path('module', 'mymodule') .'/myscriptfile.js');
}

What should I do?
Comment: https://drupal.org/node/756722 and https://drupal.org/node/542202 - there are many many ways, depending on what you want. But hook_init you should avoid. Go with info file (second link) if in doubt.
Answer: You can declare the script file inside mymodule.info with
scripts[] = myscriptfile.js

This way it's gonna be included in every page of your site, exactly the same result you obtain adding it manually to hook_init.
In case you want it only when you render a form, add this to the form declaration
$form['#attached']['js'][] = drupal_get_path('module', 'mymodule).'myscriptfile.js';

Comment: Ok. I don't really want it on every site though. I would like it to only appear on the page that the module block is rendered on.

The js contains some validation and other related stuff that only applies to the form that the module builds. So it's unnecessary to render it in other places. What's the best way to do this then?
Comment: In that case the solution is similar to the other answer
Answer: In a form you can set an #attached value. This value will allow you to attach javascript or css to the form.
I believe that is the best way. Here is an example below. (taken from drupal.org)
$form['#attached']['js'] = array(
  drupal_get_path('module', 'ajax_example') . '/ajax_example.js' => array(
  'type' => 'file',
  ),
);

How do I add javascript to a module?