como criar um bean que retorna um entityManeger

Question: estou usando springboot + ejb + cdi + wildfly10
já criei o datasource e configurei , fui na interface
do wildfly e o DS está conectando normal.
Dao
<code> @Repository
public interface PessoaDao extends JpaRepository<Pessoa, Long>{

}
</code>
Service
<code>@Stateless
@LocalBean
@Service
public class PessoaService {

  @Inject
   private PessoaDao p;

    public PessoaService() {
    }

    public void salvar(){
        Pessoa pp= new Pessoa();
        pp.setNome("x");
        p.save(pp);
    }
}
</code>
Controller
<code>@RestController
public class PessoaRsc {

    @Inject
    private PessoaService pService;

    @RequestMapping(value="/buscar", method=RequestMethod.GET,produces=MediaType.TEXT_PLAIN)
    public String salvar (){
        return "Salvo";
    }
}
</code>
e criei tb essa classe
<code>@Dependent
public class EntityManagerFactory {

        @PersistenceContext(name = "cdi")
        EntityManager entityManager;

        @Produces
        @RequestScoped
        public EntityManager createEntityManager() {
            return entityManager;
        }
    }
</code>
está gerando esse erro
<code>APPLICATION FAILED TO START
***************************

Description:

Field p in com.allancdi.service.PessoaService required a bean named 'entityManagerFactory' that could not be found.

Action:

Consider defining a bean named 'entityManagerFactory' in your configuration.

16:21:47,606 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 66) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./cdi: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./cdi: java.lang.RuntimeException: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pessoaService': Unsatisfied dependency expressed through field 'p'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pessoaDao': Cannot create inner bean '(inner bean)#4f327418' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4f327418': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: java.lang.RuntimeException: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pessoaService': Unsatisfied dependency expressed through field 'p'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pessoaDao': Cannot create inner bean '(inner bean)#4f327418' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4f327418': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:236)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:100)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
    ... 6 more
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pessoaService': Unsatisfied dependency expressed through field 'p'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pessoaDao': Cannot create inner bean '(inner bean)#4f327418' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4f327418': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
    at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:737)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:370)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:314)
    at org.springframework.boot.web.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:152)
    at org.springframework.boot.web.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:132)
    at org.springframework.boot.web.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
    at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:169)
    at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:186)
    at io.undertow.servlet.core.DeploymentManagerImpl$1.call(DeploymentManagerImpl.java:171)
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44)
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:234)
    ... 8 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pessoaDao': Cannot create inner bean '(inner bean)#4f327418' of type [org.springframework.orm.jpa.SharedEntityManagerCreator] while setting bean property 'entityManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4f327418': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:129)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1531)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1276)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1138)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1066)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:585)
    ... 39 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#4f327418': Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
    at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:634)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:448)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)
    ... 52 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'entityManagerFactory' available
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:687)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1207)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:284)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
    ... 60 more
</code>
ele diz que não existe esse bean, mas eu já criei uma classe que retorna o entiyManeger e ele não encontra.
Comment: Não é você quem tem que criar a classe EntityManagerFactory. Ela já é implementada pelo seu provedor de JPA, você só tem que configurar no seu arquivo xml ou java config para que a classe seja carregada corretamente.
Answer: Tente utilizar o Qualifier @Named na classe EntityManagerFactory. Desta forma o CDI terá um Bean com o nome entityManagerFactory para a injeção.


#include  in multiple files

Question: I hope this hasn't been asked before on this site. I wasn't able to find a solid answer from google.
What happens when you #include iostream in multiple files of a project? I always use #ifndef and #define in my header files. Does that prevent iostream from being included more than once?
Comment: The only real danger is it being included more than once in the same Translation Unit, not project, but the standard headers have include guards to my knowledge, yes.
Comment: Yes, this technique prevents multiple inclusion, and is known as [Include guard](http://en.wikipedia.org/wiki/Include_guard).
Comment: So, If I am using inheritance, is it a good idea to #include iostream only in the base class? Since the derived classes all depend on the base class, it should guarantee that it only gets included once right?
Comment: @Shan, That's mostly a matter of preference. I see lots of people who prefer including something once in a header that is included by the source for that header, and others, including me, who include every header directly used in every file that directly uses it. Keep in mind that you don't always need to include things in header files if you can either use a forward declaration or don't actually use it *in* the header.
Comment: @chris Cool. Thank you so much for your help. It's for a class assignment and I just wanted make sure so I don't get points taken off for multiple #includes.
Comment: Relevant question, was thinking the same thing myself
Answer: I had two interpretations of this question:

Can you include iostream multiple times in the same compilation unit?

Yes, safely. All standard headers have include guards so to allow this usage safely.

Will using include guards yourself in your code prevent iostream from be included multiple times?

Surely, as well it would prevent everything in between #ifndef and #endif (or in a header marked with #pragma once, in compilers that support it) from being seen by the compiler more than once... it is just what include guards does, right? But that is not really needed for #include <iostream>, because it is already guarded from multiple inclusions.
Answer: Like chris and Haroogan said, yes they will prevent that.
What the #ifndef and #define are are pre-processor instructions, and are translated in english to
#ifndef (if undefined)
#define (define)
So if you make a header, and #include "myheader.h" twice, then the file will not be included again because you encompassed it in an if statement, which will cause the file to only be included(and defined) when it has not yet been included(defined).
Comment: But there is built-in protection for standard headers?
Comment: @Shan, I can't imagine there would be in a sense of them being special compared to other headers. It sounds like a very unnecessary addition to the standard when compared with a rule that says all standard headers must protect against multiple inclusions.
Answer: regardless. you still attempt to include it multiple times even thow you use ifndef. because the multiple files will be included once. and all attempt to include <iostream> but if you open <iostream> it has
#pragma once

which basically means it will only ever be included once. it is not the advised way to avoid multiple includes i find in the industry. because its harder to know that it wont be included again. compared to seeing ifndef's on the top and bottom.
Comment: It doesn't have to, seeing as how `#pragma once` is non-standard.
Comment: I bet the primary reason is portability.
Comment: what do you mean it doesnt have to? i litually opened the file to check what type of include guards it had. and it has `#pragma once`
Comment: @Zanven, Doesn't have to doesn't mean it can't. All I'm saying is that the standard has no mention of a `#pragma once`, so it can't possibly mandate its use in standard headers. I can't find it, but I would imagine it has something stating that implementations must protect the standard headers from multiple inclusions, and how they do that is up to them, `#pragma once` being a possibility if the implementation supports it.
Comment: yeah i agree that `#pragma once` is a bad way to do it. this is straight from wiki

#pragma once is a non-standard but widely supported preprocessor directive designed to cause the current source file to be included only once in a single compilation. Thus, #pragma once serves the same purpose as #include guards, but with several advantages, including: less code, avoiding name clashes, and sometimes improved compile speed.
Answer: #ifndef __IOSTREAM_H
#include <iostream>
#endif

...

Seems that most standard headers have a sort of include guard as above.
Alternately you could make your own include guard by defining the required macro.

#include  in multiple files


Why does NFL have games on Thursdays?

Question: Players and specially coaches don't like playing on short weeks. It doesn't give the players enough time to heal from the previous game, and it makes it tough for the coaches to come up with game plans and do their analysis.
So what made the NFL hold night games to Thursdays?
Comment: According to the [Wikipedia article](https://en.wikipedia.org/wiki/Sports_Broadcasting_Act_of_1961#Exceptions) that @Philip referenced, the NFL has only played 8 games total on a Friday since 1978.
Answer: This is due to the Sports Broadcasting Act of 1961: the NFL is legally forbidden to televise any matches on a Friday night within 75 miles of any high school during the high school regular season, and similarly for Saturday nights and colleges. As virtually the whole of the USA is within 75 miles of a high school or college, this means they couldn't televise the match at all - and as the NFL's money comes from television broadcasts, that's a huge deterrent not to play.
Once the high school and college seasons end, the NFL is free to play on Fridays and Saturdays, which is why playoff games happen on a Saturday.

Why does NFL have games on Thursdays?


Do we use the SD of whole population or SD of just one sample to calculate SE of samples means in central limit theorem?

Question: I'm learning applications on Central Limit Theorem and got really confused with a few points. According to this tutorial, the procedure to apply CLT usually goes like this:

So if SD is the population standard deviation, how are we gonna get it?? Isn't the whole population standard deviation what we eventually calculate by applying CLT and analyzing a sample of the whole population? How come the population standard deviation become a prerequisite??
Please tell me this tutorial is wrong.
I think the SD actually refers to the standard deviation of a sample (of some size n), which we can actually get easily. For example:

sample the whole population with a sample size of n (e.g. randomly select 10000 users from the whole population of 10 billions)
calculate the mean of the 10000 measurements.
calculate SE = SD / sqrt(10000), where SD is the standard deviation of the 10000 measurements, instead of the standard deviation of the whole population of 10 billions.

This explanation would make much more sense.
Any thoughts?
Comment: (a) IMHO this part of the tutorial (even taken out of its context) seems to be poorly written. (b) The CLT requires existence of population mean $\mu$ and population variance $\sigma.$ Then even if the population is not normal, for large enough $n,$ the sample mean $\bar X$ is _approximately_ normal with $E(\bar X) = \mu$ and $SE(\bar X) = \sigma/\sqrt{n}.$ (c) If you don't know $\sigma$ and approximate it with sample SD $S,$ then you need to use methods involving Student's t distribution.
Comment: @BruceET thanks for making it clear. Just to confirm, if you check out this question: https://stats.stackexchange.com/questions/405157/central-limit-theorem-do-we-care-about-standard-deviation-within-one-sample-of,  we never care about the standard deviation within one sample, right?
Answer: I have looked at your other two questions. You never make it entirely clear
(to me anyhow) what your objective is. So, owing to your persistence, I will give you some examples, computations, and explanations that may be helpful. [Most of what is below is introduced in standard elementary textbooks on applied statistics. Such a textbook might be a better-organized guide than online tutorials and videos. Used copies of out-of-date editions of such books are pretty cheap on Amazon and elsewhere.]
Suppose you take a random sample of size $n = 1000$ and find sample mean
$\bar X = 247$ and sample standard deviation $S = 87.$
Your goal might (1) be to find a 95% confidence interval (CI) for the population mean $\mu$, or (2) to test the null hypothesis $H_0: \mu = 250$ against the
alternative $H_a: \mu < 250.$ Then (1) requires a t confidence interval and (2) requires a one-sample t test.
(1) The 95% CI for $\mu$ is of the form $\bar X \pm t^*S/\sqrt{n},$ where $t^* = 1.962$ cuts 2.5% from the upper tail of Student's t distribution with $n - 1 = 999$ degrees of freedom (which is very close to standard normal). My late-night computations, which you should verify, give the interval $(241.60, 252.40
).$
In Minitab statistical software, the output for a t confidence interval procedure is shown below; it agrees with my computation.
One-Sample T
   N    Mean  StDev  SE Mean        95% CI
1000  247.00  87.00     2.75   (241.60, 252.40)

(2) The test statistic for the t test is $T = \frac{\bar X - \mu_0} {S/\sqrt{n}} =\frac{247-250}{87/\sqrt{1000}} =  -1.090.$ For a  left-sided t test specified by $H_0$ and $H_1$ at the 5% level of significance, the 'critical value'  is $c = - 1.646.$ That is, you could reject $H_0$ at the 5% level
if $T < -1.646,$ but $T = -1.090$ so you cannot reject.
[The critical value cuts 5% from the lower tail of Student's t distribution
with 999 degrees of freedom.]
The 'P-value' of the test is the probability under
that distribution of a t statistic less than the observed $-1.090.$ You need
some sort of software to find the P-value, which turns out to be  $0.1380.$
Using the P-value as a criterion, you could reject $H_0$ if the P-value were smaller than 5% (which it is not).
The Minitab printout for this left-sided t test is shown below (slightly edited for relevance); it shows the same test statistic and P-value as in my computations above.
One-Sample T

Test of μ = 250 vs < 250

   N    Mean  StDev  SE Mean       T      P
1000  247.00  87.00     2.75   -1.09  0.138

Note: In one of your questions you mentioned a sample of size $n = 10,000.$
If you had the same sample mean $\bar X = 247$ and sample SD $S = 87$ a sample of that size, then the 95% CI would be $(245.295, 248.705)$ and the P-value of the one-sided test would have been smaller than $0.0005,$ leading to rejection of the null hypothesis.  Sample size matters.
Addendum about a CI for population SD:
For normal data: Because $$Q=\frac{(n−1)S^2}{σ^2} \sim \mathsf{CHISQ}(\text{df} = n−1),$$ one can use a printed table of chi-squared distributions or software to find quantiles .025 and .975, $L$ and $U,$ respectively, of that distribution to get $$P(L<Q<U)= \cdots = P\left(\frac{(n−1)S^2}{U}<σ^2<\frac{(n−1)S^2}{L}\right)=.95.$$
[Notice the 'reversal' of $L$ and $U,$ which results from taking reciprocals in solving
the inequality to 'isolate' $\sigma^2.]$
Hence a 95% CI for $σ^2$ is of the form
$$\left(\frac{(n−1)S^2}{U},\,\frac{(n−1)S^2}{L}\right).$$
Take square roots of endpoints to get 95% CI for $σ.$
For example, if a sample of size $n=50$ from a normal population has sample variance $S^2=34.5,$ then a 95% CI for the population SD $σ$ is $(4.91,\,7.32.).$ [Notice that the point estimate $S=5.87$ is contained within this CI, but not at its midpoint (because chi-squared distribution is skewed.] Computation from R:
v = 34.5;  sqrt(49*v/qchisq(c(.975,.025), 49))
[1] 4.906476 7.319376

Output from Minitab:
95% Confidence Intervals

                CI for        CI for
 Method          StDev       Variance
 Chi-Square  (4.91, 7.32)  (24.1, 53.6)

As you suggest, such intervals tend to get shorter with increasing $n.$ However, intervals can still be disappointingly long, even for moderately large $n.$ If the sample variance $S^2 = 34.5$ had resulted from a sample of size $n=500,$ then the resulting 95% CI for $\sigma$ would be $(5.531,\, 6.262).$ "Variances are very variable."
v = 34.5;  sqrt(499*v/qchisq(c(.975,.025), 499))
[1] 5.530786 6.262223

Comment: In both of my examples (CI & t test) $S/\sqrt{n}$ is the estimated standard error of the sample mean $\bar X.$ People often call this the 'standard error', omitting the word _estimated_ when the estimation is thought to be obvious.
Comment: Thank you so much!  To clarify, my confusion is on the comparison between the SD of whole population vs SD of one sample (size 1000 in your answer). e.g. Can we use SD of one sample to calculate the SE of samples mean? In your answer you made it clear that we use SD of one sample divided by sqrt(n) to calculate SE.  In many other sources, they use SD of the whole population.  So I was confused: how accurately can SD of one sample approximate the SD of whole population??
Comment: This makes much more sense now:  SD of one sample / sqrt(n) = estimated SE of sample mean;   SD of whole population / sqrt(n) = exact SE of sample mean.  An important follow-up questions is: How accurately can we approximate SD of whole population using SD of one sample? Of course the bigger the sample size, the better the approximation. But it must require some strict mathematics to support this approximation. Is there any?
Comment: The former comment in response to your last question was deleted here and put into an Addendum to my Answer.  (Computations shown, equations displayed, abbreviations spelled out, articles supplied.)
Comment: I'm a newbie in using this website. what does it mean? ((Computations shown, equations displayed, abbreviations spelled out, articles supplied.) )
Comment: It just means I had more room in Answer format than in Question format to do the explanation.  No need to omit computations, leave out 'the' and 'a', use 'dist'n' for 'distribution', etc.


remove required property from input field on form submit

Question: Model:
[Display(Name = "City"]
[Required]
[RegularExpression(@"^(?!\d$).*$"]
[StringLength(20,MinimumLength = 2]
public string City { get; set; }

Form:
@Html.LabelFor(x => x.City, new { @class = "control-label" })
@Html.TextBoxFor(x => x.City, new {id="city" })

Script:

I want to remove required property if some condition is met.Unable to do this this way.How can i achieve this?
Comment: With Jquery: $("#city").removeAttr("required"); or use document.getElementById("city").removeAttribute('required');
Comment: Are you using JqueyVal? If yes, removing the required isn't the only thing!
Comment: possible duplicate of [How to set HTML5 required attribute in Javascript?](http://stackoverflow.com/questions/18770369/how-to-set-html5-required-attribute-in-javascript)
Answer: JavaScript is case sensitive.
Use
document.getElementById("city").required = false;

Demonstration
Be careful that your code can't work as you try to access the element before it exists. Put your script after the element if you don't execute the code on an event :
<input type="text" id="city" required>

Note also that you can't change this in a submit function and expect your form to be submitted because it's too late : this event handler won't be called if the required field is not filled !
Comment: Form is still not submitting and i am doing this in form submit function
Comment: @dystroy Given that he's using MVC4, maybe exists a JqueryVal. So removing the required isn't enougth.
Comment: @user2137186 I've edited my answer to point another problem. It may be relevant or not, depending on how far the code you show is from what you really have.
Comment: I am call that script function on form submit function
Comment: But is your submit function called ? If required is true and the field empty, then it shouldn't be called. You must change it elsewhere, from example by binding to a button click event.
Comment: This does normally work but not with the data annotations.
Answer: You can use:
document.getElementById("city").removeAttribute("required");

or with jQuery
$('#city').removeAttr('required')

Comment: Form is still not submitting and i am doing this in form submit function
Comment: @user2137186 you can post, just! We will reflag it for mvc as well!
Comment: well i am using .net mvc4 ...so this should not be posted here
Answer: You shoud do this:
if(somecondition is true)
{
  var city = document.getElementById("city");
  city.removeAttribute('required');
}

Comment: Form is still not submitting and i am doing this in form submit function
Comment: You should post the entire function.
Comment: what do you mean by that?
Comment: You should post the entire form, and the function, theres something else wrong!
Answer: By the time the On Submit function is to remove the required attribute, the form would have already passed validation. The steps are first you have to disable validation then you will be able to submit the form with the required fields empty. Next remove the required attributes and finally manually invoke validation via $.validator.unubtrusive.parse(form) and this will now validate the form by invoking the other validation types such as string length and formatting, everything else except the required attributes. Check if form.valid() then submit() else do nothing.
Answer: In php its very easy, this will not validate your form while submitting,
<input type="submit" onclick="return confirm('Are you sure?')" value="Abort Test" formnovalidate>

Answer: Not sure if anyone has figured out a better way to do this yet and it might not be the most efficient way to do it but you could use PHP to get this done. If the condition you are looking to meet is known before the page loads then you can have this:
<input type="text" id="city"

<?php
     if (condition_is_met) {
          echo ">";
     } else {
          echo "required>";
     }
?>

Answer: If you put your text fields in a form, it is possible to overwrite the required field with a special attribute called novalidate. This is the syntax: <form novalidate> .. </form>
Here is an example:
<form novalidate>
  Zip code: <input type="text" name="zipcode" pattern="[0-9]{5}" required>

  <input type="submit">
</form>

The novalidate Attribute Here is a link with two live demos using novalidate attribute.
Answer: Or in the moment that form is submitted you can disabled it if it's null:
if($("#City").val().trim().length == 0)
    $("#City").attr("disabled", true);
Answer: This works for me with data annotations.  Use the "rules" function.
$("#FileUpload").rules("remove", "required");

and to add:
$("#FileUpload").rules("add", "required");

UPDATE: This will fix frontend validation but backend validation still fails.  To make the ModelState valid, I change the ValidationState of that input.
ModelState["FileUpload"].ValidationState = Microsoft.AspNetCore.Mvc.ModelBinding.ModelValidationState.Valid;

I do check to make sure that's the only error too:
if (ModelState["FileUpload"].Errors.Count == 1)
    if (ModelState["FileUpload"].Errors[0].ErrorMessage.Equals("The FileUpload field is required."))
        ModelState["FileUpload"].ValidationState = Microsoft.AspNetCore.Mvc.ModelBinding.ModelValidationState.Valid;


pyqt image mask not moving with image

Question: I'm trying to hide/show an image in PyQT using a mask. I'm able to obtain a nice result with the following code:
    self.picture = QLabel()
    self.pix = QPixmap('PyQt_Logo.png')
    self.picture.setPixmap(self.pix)
    self.picture.setMask(self.pix.createMaskFromColor(Qt.transparent, Qt.MaskOutColor))

But if I align the picture in the center of its layout container using:
    self.picture.setAlignment(Qt.AlignmentFlag.AlignHCenter | Qt.AlignmentFlag.AlignVCenter)

the mask is not moved and it is applied left-aligned. How to apply the image mask on the centered image?
Answer: The mask is always based in widget coordinates, but if you change the alignment of the image it won't translate the mask along with it.
A possible solution is to install an event filter on the label and update the mask whenever it's resized. In order to do so, we need to keep a persistent reference of the mask, but using a QRegion (which allows translation):
        self.baseMask = QRegion(self.pix.createMaskFromColor(Qt.transparent, Qt.MaskOutColor))
        self.picture.setMask(self.baseMask)
        self.picture.installEventFilter(self)
        # ...

    def eventFilter(self, obj, event):
        if event.type() == QEvent.Type.Resize:
            rect = self.picture.pixmap().rect()
            rect.moveCenter(self.picture.rect().center())
            self.picture.setMask(self.baseMask.translated(rect.topLeft()))
        return super().eventFilter(obj, event)

Alternatively, with a subclass:
class MaskLabel(QLabel):
    baseMask = None
    def setMask(self, mask):
        if isinstance(mask, QBitmap):
            mask = QRegion(mask)
        self.baseMask = mask
        self.updateMask()

    def updateMask(self):
        if self.baseMask and self.pixmap():
            rect = self.pixmap().rect()
            rect.moveCenter(self.rect().center())
            super().setMask(self.baseMask.translated(rect.topLeft()))

    def resizeEvent(self, event):
        self.updateMask()

Obviously, the above is only valid for centered images, if you need a different alignment, you need to verify the alignment direction and eventually translate the mask accordingly, based on the label rect().

pyqt image mask not moving with image


What is the effect of sequentially removing outliers on the p-value and can graphing the change be informative?

Question: What is the effect of sequentially removing outliers on the p-value of a t-test and can graphing changes in p-value provide useful information
In the data we compare soil pH and soil moisture data collected by students and teaching assistants. We begin by ranking student-collected data points based on IQR score. This score was the absolute value of the difference between a data point and the median divided by the interquartile change: |(ni - median)/IQR|. Once ranked, we run a t-test comparing student and teaching assistant data. After the initial test, we sequentially remove student outliers from the analysis. After each data point is removed, the t-test is re-run. The figures below chart p-value on the y-axis and the number of IQR-ranked data points removed from the analysis.

How I am interpreting these figures:
I would remove outliers that fit the following conditions:
A) the left side of the figure has a very steep slope and
B) that slope decelerates rapidly, observed as a sharp angle in the line. This appears to be the case in the soil moisture data on the left.
A curving negative slope without sharp bends on the left side of the figure (e.g. pH figure) indicates outliers that should not be removed from the analysis since they align with the rest of the dataset. While these points have high IQR values, there is nothing to indicate that they do not just represent tails on a typical distribution.
My questions:

Is this a valid qualitative analysis to address our question?
Is the interpretation of these figures accurate?
Is there anything else we can glean from these figures, particularly any patterns in the middle or right side of the x-axes?
Comment: This seems like a bizarre procedure. Why did you do this ? You ask *"Is this a valid qualitative analysis to address our question?"* but I don't even know what your question is. In what sense even is the analysis *"qualitative"* ?
Answer:
Is this a valid qualitative analysis to address our question?

I'm not sure what you question is, but I'm confident the answer is "absolutely not".  You're performing several tests and removing data without good reason except for "they are outliers" -- which is not a precise term in my own opinion anyway.

Is the interpretation of these figures accurate?

In all likelihood, no.  Your method seems very ad hoc and so it is difficult to comment on properties of this method of estimation.  I'm extremely dubious about any inferences made from this approach.  Can you simulate your approach to demonstrate some desirable properties?

Is there anything else we can glean from these figures, particularly any patterns in the middle or right side of the x-axes?

I would hesitate from inferring anything from these.  What you're doing sounds very much to me like data dredging if not p-hacking.
Why don't you begin by telling is your research question.  These students are collecting soil samples.  Why?  Why are you interested in comparing TA vs. Student data?  Are you not confident in the student's abilities?
Comment: @benxyzzy but just because one *can* doesn't mean one *should*
Comment: (+1) I don't get how such analyses are motivated.
Comment: @RobertLong They are motivated by the fact one can, and if one did the outcome would not appear any different from a more successful analysis that did not entail these practices
Comment: @RobertLong I agree, but the perverse incentive is clear
Comment: @benxyzzy it's not clear to my little brain. Please could you explain this incentive ?
Comment: @RobertLong I wonder why so cagey. Anyway it's obvious that to the extent a user of statistics would prefer a given answer (and rarely are they uninvested), they will seek to claim this answer is the case, unless there is some externally imposed checking. "Check" as in both to scrutinise, and to curtail. So if there is a method by which Type 1 error can be inflated without any external sign of this being done, the perverse incentive is clear. Perhaps you imagine Type 1 error inflation to be the natural check on such behaviour, but rarely does the bad statistician need to worry so far ahead


Show orginal dialog on notification click

Question: I have and sms app that when I receive an sms there is a dialog created to show the message and a notification created also in the notification bar. My problem is that when I click the notification it starts my activity and creates a new dialog on top of the original instead of just refocusing the one that has already been created.
   NotificationManager manger = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
   Notification notification = new Notification(R.drawable.sms,smsBody, System.currentTimeMillis());

   Intent notificationIntent = new Intent(this, SendSMSActivity.class);
   PendingIntent contentIntent =
           PendingIntent.getActivity(this, 0, notificationIntent, PendingIntent.FLAG_ONE_SHOT );

   notification.setLatestEventInfo(this, ContactName,smsBody,contentIntent);
   notification.flags = Notification.FLAG_AUTO_CANCEL;
   //notification.sound = Uri.parse("file:///sdcard/notification/ringer.mp3");

   notification.defaults = notification.DEFAULT_SOUND | notification.DEFAULT_VIBRATE;
   manger.notify(1, notification);

my activity code
public void onCreate(Bundle savedInstanceState) {
       super.onCreate(savedInstanceState);

        //wakeup screen
        PowerManager pm = (PowerManager) getApplicationContext().getSystemService(Context.POWER_SERVICE);
        WakeLock wakeLock = pm.newWakeLock((PowerManager.SCREEN_BRIGHT_WAKE_LOCK | PowerManager.FULL_WAKE_LOCK | PowerManager.ACQUIRE_CAUSES_WAKEUP), "TAG");
        wakeLock.acquire();

        final String sender = getIntent().getStringExtra("PhoneNumber");
        final String body = getIntent().getStringExtra("smsBody");
        final String contactId = getIntent().getStringExtra("contactId");
        final String SenderName = getIntent().getStringExtra("SenderName");

        new ContactNameAsyncTask().execute();//this start the create dialog

This is my dialog code
private void ShowMessage(){
   NotificationManager manger = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
   Notification notification = new Notification(R.drawable.sms,smsBody, System.currentTimeMillis());

   Intent notificationIntent = new Intent(this, SendSMSActivity.class);
   PendingIntent contentIntent =
           PendingIntent.getActivity(this, 0, notificationIntent, PendingIntent.FLAG_ONE_SHOT );

   notificationIntent.putExtra("NotificationMessage", "focused");
   notificationIntent.setData((Uri.parse("foobar://"+SystemClock.elapsedRealtime())));

   notification.setLatestEventInfo(this, ContactName,smsBody,contentIntent);
   notification.flags = Notification.FLAG_AUTO_CANCEL;
   //notification.sound = Uri.parse("file:///sdcard/notification/ringer.mp3");

   notification.defaults = notification.DEFAULT_SOUND | notification.DEFAULT_VIBRATE;
   manger.notify(1, notification);

    final Dialog dialog = new Dialog(this);
    dialog.requestWindowFeature(Window.FEATURE_NO_TITLE);
    dialog.setContentView(R.layout.custom);

    try {
        TextView text = (TextView) dialog.findViewById(R.id.text);
        text.setMovementMethod(ScrollingMovementMethod.getInstance());
        text.setText(smsBody);

        TextView stext = (TextView) dialog.findViewById(R.id.sender);
        stext.setText(ContactName);
    }catch(NumberFormatException nfe) {
        TextView text = (TextView) dialog.findViewById(R.id.text);
        text.setMovementMethod(ScrollingMovementMethod.getInstance());
        text.setText("message not found check your inbox");

        TextView stext = (TextView) dialog.findViewById(R.id.sender);
        stext.setText("unknown");
    }

    ImageView image = (ImageView) dialog.findViewById(R.id.image);
    image.setPadding(2, 2, 2, 2);
    image.setBackgroundColor(Color.GRAY);

    //get contact photo
    try {

        image.setImageURI(ContactPicUri);
    } catch(NumberFormatException nfe) {
        image.setImageResource(R.drawable.sender);
        //Toast.makeText(context,"Contact image not found "+ nfe,Toast.LENGTH_LONG).show();
    }

    //end get contact photo
    //button1
    Button dialogButtonReply = (Button) dialog.findViewById(R.id.dialogButtonReply);
    dialogButtonReply.setOnClickListener(new OnClickListener() {
        public void onClick(View v) {
             NotificationManager manger = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            manger.cancel(1);
                smsReply(PhoneNumber, smsBody);
                 goHome();
            }
    });
    //button1 end

    Button dialogButtonClose = (Button) dialog.findViewById(R.id.dialogButtonClose);
    dialogButtonClose.setOnClickListener(new View.OnClickListener() {
        public void onClick(View v) {
             NotificationManager manger = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            manger.cancel(1);
            //markMessageRead(context,PhoneNumber,smsBody);
            goHome();

        }
    });

    Button dialogButtonQuickReply = (Button) dialog.findViewById(R.id.dialogButtonQuickReply);
    dialogButtonQuickReply.setOnClickListener(new View.OnClickListener() {
        public void onClick(View v) {
             NotificationManager manger = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            manger.cancel(1);
            QuickReply();

        }
    });
    dialog.show();

}
Comment: I don't see anywhere that you are creating or invoking a dialog. Can you post the relevant code?
Answer: You should avoid using Dialog directly as per the Android docs. So for this reason I will point you to DialogFragment. But the code is fairly straight forward.
FragmentTransaction ft = getFragmentManager().beginTransaction();
Fragment prev = getFragmentManager().findFragmentByTag("dialog");
if (prev != null) {
    ft.remove(prev);
}
ft.addToBackStack(null);

// Create and show the dialog.
DialogFragment newFragment = MyDialogFragment.newInstance(mStackLevel);
newFragment.show(ft, "dialog");

If you take a look at the link, you will see that saying when you show() the fragment, you pass it in a tag, in this case calling it dialog. So this makes sure that no other Dialog is showing, meaning it won't show another one on top of the original. Naturally you have to extend the DialogFragment class to make your custom version, but its not so bad :)


Which test(s) for analyzing yes-no answers for control vs treated conditions

Question: To keep it simple, I have sets of data in which I have a control and a treated sample. The data I obtain is "yes" or "no" (0 or 1) for a particular behaviour. I have 150 data points for each condition (150 for control, 150 treated), obtained across 3 experimental runs, which I intend to pool. They are not paired in any way.
Which test would you recommend? I am currently thinking that binary logistic regression is the best choice, but I'm not sure. I am a biologist and not a mathematician, so please keep that in mind :)
Answer: A binary logistic regression would work, for the same reason that you can use a regression when you want to compare two sample means.
However, it may be more apparatus that is required.
With a small sample, one might consider a binomial test. In this case, your sample sizes are nice and big, so a straight out proportions test should be effectively indistinguishable from it and a little simpler to deal with.
However, since you have three experimental runs it might be worth including experimental run as a covariate (even though if all is well it should not have a covariate effect). In that case, you could use logistic regression to do teh comparison incorporating the experimental run variable, or you could look at a chi-square test to achieve basically the same thing (though if you want a one-tailed test, the first thing would be better).
Answer: Since your metric is a proportion, the hypothesis test for proportions is the best choice.
This pdf can help you
Comment: Certainly the general category of model/test is determined by what the question of interest is, but I think data type definitely comes into it -- after all, if the data were not proportions or counts, one wouldn't even *consider* a proportions test. The data type, therefore, must be at least part of the basis on which such selections are made.
Comment: @Glen_b Agreed.  What I was objecting to was the "logic:" namely, concluding that the data type determines the "best choice" of procedure.
Comment: The conclusion may be correct, but the logic is not: one does not select statistical tests based on the data type.
Comment: No dispute there. Indeed if I was reading this for the first time now, I'd probably have just assumed that was the intent.
Answer: A proportion test as zeferino suggest, or also a $\chi^2$ test or the Fisher exact probability test.
Comment: Which one would you recommend? Each comes with its own null hypothesis, and the sampling design suggests that at least one margin if fixed in the two-way table.
Comment: I think that with enough number of data the p-values are very similar in any case. With few data, it is clear that a Chi square must be avoided. (or use the Yates correction, but probably is better a Fisher test).
Comment: Thanks to you and Zeferino. I will look at the recommended tests and see if one will work best.


C++ wstringstream << NULL

Question: I'm having an issue with wstringstream. When I'm doing this
    std::wstringstream ss;
    wchar_t* str = NULL;
    ss << str;

Application crashes with error
Unhandled exception at 0x53e347af (msvcr100d.dll) in stringstr.exe: 0xC0000005: Access violation reading location 0x00000000.

For example this works good:
ss << NULL;
wchar_t* str = L"smth";
ss << &str;

Not always str has value, it could be NULL sometimes and when it is NULL I would like to put 0 into stream. How to fix it?
Answer: If it's null, don't output a null wchar_t pointer:
( str ? ss << str : ss << 0 );

Note that this won't work:
ss << ( str ? str : 0 )

since the implicit conditional operator return type is a common type to both its expressions, so it would still return a null wchar_t pointer.
Comment: @user1112008: The implicit conditional operator return type is a common type to both its expressions, so in your case it would still return a null `wchar_t` pointer.
Comment: @user1112008: What's wrong is that `const std::wstring` is not a pointer. Use regular function overloading to have a different implementation for pointers than non-pointers: `template std::string realToString(type real)` for the regular case, and `template std::string realToString(type* real)` for pointers.
Comment: ahhhhh, I was trying `ss<<(!str ? 0 : str);` and it wasn't working
Comment: one more question, what if we are using templates? `template  std::string realToString(type real)` and we to check if `real` isn't NULL?
Comment: @user1112008: Assuming `type` is a pointer, is still the same check.
Comment: `error C2440: '?' : cannot convert from 'const std::wstring' to 'bool'` something is wrong
Answer: Check before you output to the stringstream (as already suggested)
if (str == NULL) {
    ss << 0;
} else {
    ss << str;
}

C++ wstringstream << NULL


how to avoid the loops in a specific multiple conditional statement in R

Question: I am using R to make inference from electronic medical records(EMRs) in R. Actually I do write a loop command that can work, but the problem is the loop can be really slow when dealing with millions of EMRs. So could anyone transform my commands into a faster way (maybe a vector based computing or other possible ways) ?
My purpose is to figure out whether a set of commodities(in this case, they are from p324 to p9274) contains either of a set of characters(in this case, they are I25.2, I21. and I22.).
here is a sample of my data:
test <- data.frame(p324 = c("I24.001", "I10.x04", "I48.x02", "I48.x01", "I25.201", "I25.201", "I25.101", "I25.101", "NA", "I50", "I25.101", "I25.101", "I25.101", "I45.102", "I50.902"),
p327 = c("I20.000", "K76.000", "E11.900", "I44.200", "NA", "I49.904", "I45.102", "I50.910", "NA", "I10  05", "J98.402", "NA", "NA", "R57.0", "I10.x04"),
p3291 = c("I50.903", "K80.100", "N39.000", "I25.103", "NA", "I50.908", "NA", "I10  04", "NA", "I25.101", "I10  03", "NA", "NA", "I25.101", "I10.x05"),
p3294 = c("I10.x05", "K76.807", "J98.414", "K81.100", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "J43", "I10.x06"),
p3297 = c("NA", "I83.900", "E87.801", "NA", "NA", "I21.620", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "I10.x07"),
p3281 = c("K80.100", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "I10.x08"),
p3284 = c("K76.807", "I21.620", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "NA", "I10.x09"),
p3287 = c("I83.900", "I10.x3", "I10.x2", "I10.x1", "I10.x0", "I10.x1", "I10.x2", "I10.x3", "I10.x4", "I10.x5", "I10.x6", "I10.x7", "I10.x8", "I10.x9", "I10.x10"),
p3271 = c("I50.908", "NA", "I10.x1", "I10.x0", "I10.x1", "I10.x2", "I10.x3", "I10.x4", "I10.x5", "I10.x6", "I10.x7", "I10.x8", "I10.x9", "I10.x10", "I10.x11"),
p3274 = c("NA", "I10.x1", "I10.x0", "I10.x1", "I10.x2", "I10.x3", "I10.x4", "I10.x5", "I10.x6", "I10.x7", "I10.x8", "I10.x9", "I10.x10", "I10.x11", "I10.x12"))

Here is my code:
for (i in 1:15)
{
  if (any(
    c(
      substr(test$p324[i], 1, 5),
      substr(test$p327[i], 1, 5),
      substr(test$p3291[i], 1, 5),
      substr(test$p3294[i], 1, 5),
      substr(test$p3297[i], 1, 5),
      substr(test$p3281[i], 1, 5),
      substr(test$p3284[i], 1, 5),
      substr(test$p3287[i], 1, 5),
      substr(test$p3271[i], 1, 5),
      substr(test$p3274[i], 1, 5)
    ) %in% c("I25.2")
  ) |
  any(
    c(
      substr(test$p324[i], 1, 4),
      substr(test$p327[i], 1, 4),
      substr(test$p3291[i], 1, 4),
      substr(test$p3294[i], 1, 4),
      substr(test$p3297[i], 1, 4),
      substr(test$p3281[i], 1, 4),
      substr(test$p3284[i], 1, 4),
      substr(test$p3287[i], 1, 4),
      substr(test$p3271[i], 1, 4),
      substr(test$p3274[i], 1, 4)
    ) %in% c("I21.", "I22.")
  ))
  test$MI[i] = 1
  else
    test$MI[i] = 0
}

So, can anyone transform my commands, or give me some suggestions so that, it could operate efficiently and fast even when the cases are more than 1 million? Thanks so much.
Comment: I compared the codes that use `sapply & grepl()` by @Sixiang.Hu, `grepl()` by @David Arenburg, and `substr` by @David Arenburg, it appears that the `sapply` code has the best performance. Here are the results of my tests:
Comment: Please see my updated about my question. I timed each code and checked the results. It appeared that substr by @David Arenburg generates inconsistent results. I am not sure the reason why.
Comment: Probably because `grepl` finds more than just the exact matches. My guess you have some wrong matches there. I would check your results to see if you get the correctly. Also, your first variation doesn't match what was provided in the answer, for instance, there is no `as.data.frame(t(test))` there. Finally, you don't have to run `unlist(test)` twice, you could store it and reuse.
Comment: @David Arenburg Thanks for your benchmarking results. I tested these three methods based on my empirical dataset, with 523,441 observations, and I don't know how can I give this data to you to check the reasons for generating these NA values by the substr operator. I made some minor revisions to the sapply/grepl function for my dataset and I don't think it is the reason for outperforming the substr codes. Even though I stored `unlist(test)`, `sapply & grepl()` (5.062496 secs) still outperform `substr` (7.068666 secs) and `grepl()` (12.69934 secs) on my dataset. I am checking for the reasons.
Comment: @David Arenburg I think I have figured out the reason for these NA values. The problem is when the data contains NA values, the `substr` operation will generate NA values, while the `grepl` operation could generate the right results. I also updated the benchmark results based on my data and I think you are right, although in my result, it shows that    David(3.728264) > SixHu(4.323772) > SixHuVec(11.867062).
Answer: I'd suggest to use regular express and then sapply for vectorizing.
t_test <- as.data.frame(t(test))
chk <- function(x){
  grepl("I25\\.2|I21\\.|I22\\.",x)
}

sapply(t_test,chk)

The return result would be true or false based, and can be easily converted to 0 or 1.
Edit1:
My bad that didn't notice it is row-based check. Updated the code above.
Edit2:
Change the regression pattern:
1. escaping the . using \\. Otherwise, single . means matching any character
2. changing [] to |, given [] means if any character in it will give a TRUE.
Comment: Won't this apply column-wise, where the OP is working row-wise?
Comment: @Sixiang.Hu Thanks for your help! The performance improves a lot by sapply. I am wondering how the function "grepl" works. When I type `grepl("[H34.0][G45]","H34.023")` it returns TRUE. However, when I type `grepl("[H34.0][G45][G46][I60][I61][I62][I63][I64][I65][I66][I67][I68][I69]","H34.023")` it returns FALSE. This causes great trouble because sometimes I need to include many sets of characters.
Comment: @MiaoCai, responce edited, for further regular expression, you can search for "r regular expression" (e.g. http://stat545.com/block022_regular-expression.html).
Comment: The by row loop here is unnecessary. You could easily vectorize this using something like `res <- grepl("I25\\.2|I21\\.|I22\\.", unlist(test)) ; dim(res) <- dim(test) ; test$MI <- rowSums(res)`
Comment: @Sixiang.Hu Thanks for your help. I saw your C.V on linkedin, and it appears that we were from the same city. I graduated from Huazhong University of Science and Technology, next to Wuhan University.
Answer: If you are looking for performance improvement:

Don't run loops by row (sapply is also a loop)
Don't run vectorized operations in a loop (why are you running substr by row by column when you can just run it once?)
Avoid using regex- it is slow. Instead, if you are dealing with exact matches here, just use == or %in%.

Here is a simple vectorized possible solution for your problem
res <- (substr(unlist(test), 1, 5) == "I25.2") |
       (substr(unlist(test), 1, 4) %in% c("I21.", "I22."))
dim(res) <- dim(test)
test$MI <- rowSums(res)

This is basically ran substr(test, k, n) only once for each k/n combo (which is 2 in total) and compared against the values in interest. Then, (because %in% doesn't have a data.frame method) we transformed the resulting vector into the correct format again and summed the matches per row (in a vecotrized manner). The result is how many matches per row. If you like, it could be very easily transformed into binary (also in a vectroized manner)

Benchmarks
So OP mentioned benchmarks, so here are some benchmarks done on 10K/10 rows/columns

The grepl/sapply solution is about X10 slower than the vecotrized ones
My suggestion to vectorize the grepl solution improves performance by about X10 times
My own solution performs very similarly to the vectorized grepl solution, while I believe it will generalize much better as the regex will become more complex (for additional matches), while %in% has almost no marginal cost for additional mathces

Setup (using OPs test data)
set.seed(123)
big.df <- as.data.frame(matrix(sample(unlist(test, use.names = FALSE), 1e5, replace = TRUE), ncol = 10))

# sapply / grepl
SixHu <- function(df) {
  t_test <- as.data.frame(t(df))
  chk <- function(x){
    grepl("I25\\.2|I21\\.|I22\\.",x)
  }
  unname(colSums(sapply(t_test, chk)))
}

# Vectorized grepl
SixHuVec <- function(df) {
  res <- grepl("I25\\.2|I21\\.|I22\\.", unlist(df))
  dim(res) <- dim(df)
  rowSums(res)
}

# Vectorized substr
David <- function(df) {
  tmp <- unlist(df)
  res <- (substr(tmp, 1, 5) == "I25.2") | (substr(tmp, 1, 4) %in% c("I21.", "I22."))
  dim(res) <- dim(df)
  rowSums(res)
}

Validation
identical(SixHu(test), SixHuVec(test))
## [1] TRUE
identical(SixHu(test), David(test))
## [1] TRUE

Benchmark Results
microbenchmark::microbenchmark(SixHu(big.df),
                               SixHuVec(big.df),
                               David(big.df))
# Unit: milliseconds
#             expr       min         lq       mean     median        uq       max neval cld
#    SixHu(big.df) 989.55655 1021.17121 1047.63956 1041.94771 1062.7705 1151.4196   100   b
# SixHuVec(big.df)  67.52131   72.39233   84.61193   75.31462   85.5352  147.0646   100  a
#    David(big.df)  63.48242   68.20945   88.73896   75.19159  115.3958  147.0867   100  a

Comment: Thanks for your suggestions!Since I am just a beginner in R, loop is the most intuitive way for me to write codes. I run substr by row and by column because actually I have more than 200 variables and I just want to pick some of them. I compared the codes that use sapply & grepl() by Sixiang.Hu, grepl() by David Arenburg, and substr by David Arenburg, it appears that the sapply code has the best performance. Your code provided in this section, however, generates many NA values. Please refer to my updates to my question. Thanks.
Comment: @MiaoCai I've added some benchmarks, see my edit. I believe your benchmarks are wrong.
Comment: Thanks. I have figured out the reason why NAs were generated, and it is updated in the orginial question. I also made a benchmark based on my data. Interestingly, my benchmark results is different from yours. sapply is not as bad as that in your benchmark results.
Answer: UPDATE 1
I compared the codes that use "sapply & grepl()" by @Sixiang.Hu, "grepl()"by @David Arenburg, and "substr" by @David Arenburg, it appears that the sapply code has the best performance. "substr" by @David Arenburg code provided in this section, however, generates many NA values. Could any explain the reason why these NA values generate?
> # sapply & grepl()
> start.time <- Sys.time()
> test <- subset(I61, select = c("p324", "p327", "p3291", "p3294", "p3297", "p3281", "p3284", "p3287", "p3271", "p3274"))
> MIchk <- function(x){
+   grepl("I25\\.2|I21\\.|I22\\.",x)
+ }
> test1 <- sapply(test,MIchk)
> test$MI <- rowSums(test1)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 2.363007 secs
> table(test$MI,exclude = NULL)

     0      1      2   <NA>
254495   3523     15      0
>
> # grepl()
> start.time <- Sys.time()
> test <- subset(I61, select = c("p324", "p327", "p3291", "p3294", "p3297", "p3281", "p3284", "p3287", "p3271", "p3274"))
> res <- grepl("I25\\.2|I21\\.|I22\\.", unlist(test))
> dim(res) <- dim(test)
> test$MI1 <- rowSums(res)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 2.51223 secs
> table(test$MI1,exclude = NULL)

     0      1      2   <NA>
254495   3523     15      0
>
> # substr
> start.time <- Sys.time()
> test <- subset(I61, select = c("p324", "p327", "p3291", "p3294", "p3297", "p3281", "p3284", "p3287", "p3271", "p3274"))
> res <- (substr(unlist(test), 1, 5) == "I25.2") | (substr(unlist(test), 1, 4) %in% c("I21.", "I22."))
> dim(res) <- dim(test)
> test$MI2 <- rowSums(res)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 3.473388 secs
> table(test$MI2,exclude = NULL)

     0      1      2   <NA>
154897   2461     11 100664

UPDATE 2
The reason that the substr operation generates many NA value is that my dataset contains NA values. So I performed the following codes and then the above mentioned three operations yeild consistent results:
library(dplyr)
test %>% mutate_if(is.factor, as.character) -> test
test[is.na(test)]<-0

and then I performed the three codes:
> #=================================
> # sapply & grepl()
> start.time <- Sys.time()
> MIchk <- function(x){
+   grepl("I25\\.2|I21\\.|I22\\.",x)
+ }
> test1 <- sapply(test,MIchk)
> test$MI <- rowSums(test1)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 5.864876 secs
> table(test$MI,exclude = NULL)

     0      1      2   <NA>
520339   3081     21      0
> #=================================
> # grepl()
> start.time <- Sys.time()
> test1 <- subset(test, select = c("p324", "p327", "p3291", "p3294", "p3297", "p3281", "p3284", "p3287", "p3271", "p3274"))
> res <- grepl("I25\\.2|I21\\.|I22\\.", unlist(test1))
> dim(res) <- dim(test1)
> test$MI1 <- rowSums(res)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 17.20333 secs
> table(test$MI1,exclude = NULL)

     0      1      2   <NA>
520339   3081     21      0
> #=================================
> # substr
> start.time <- Sys.time()
> test2 <- subset(test, select = c("p324", "p327", "p3291", "p3294", "p3297", "p3281", "p3284", "p3287", "p3271", "p3274"))
> tmp <- unlist(test2)
> res <- (substr(tmp, 1, 5) == "I25.2") | (substr(tmp, 1, 4) %in% c("I21.", "I22."))
> dim(res) <- dim(test2)
> test$MI2 <- rowSums(res)
> end.time <- Sys.time()
> end.time - start.time
Time difference of 4.386484 secs
> table(test$MI2,exclude = NULL)

     0      1      2   <NA>
520339   3081     21      0

Finally, I also made a benchmark test, which shows that substr operation is slightly better than sapply/grepl and significantly better than vectorised grepl alone. Here are my codes and results:
#--------------------------------
SixHu <- function(df) {
  MIchk <- function(x){
    grepl("I25\\.2|I21\\.|I22\\.",x)
  }
  test1 <- sapply(df,MIchk)
  rowSums(test1)
}
#--------------------------------
# Vectorized grepl
SixHuVec <- function(df) {
  res <- grepl("I25\\.2|I21\\.|I22\\.", unlist(df))
  dim(res) <- dim(df)
  rowSums(res)
}
#--------------------------------
David <- function(df) {
  tmp <- unlist(df)
  res <- (substr(tmp, 1, 5) == "I25.2") | (substr(tmp, 1, 4) %in% c("I21.", "I22."))
  dim(res) <- dim(df)
  rowSums(res)
}
> microbenchmark::microbenchmark(SixHu(test),
                                 +                                SixHuVec(test),
                                  +                                David(test))
Unit: seconds
expr       min        lq      mean    median        uq       max neval cld
SixHu(test)  4.323772  4.598328  4.836165  4.760263  4.988194  5.801979   100  b
SixHuVec(test) 11.867062 12.826925 13.342357 13.243638 13.635339 18.705615   100   c
David(test)  3.728264  4.180152  4.389600  4.344938  4.519908  6.396018   100 a

Therefore, vectorized substr() by @David Arenburg is the best answer while sapply/grepl by @Sixiang.Hu, and significantly better than grepl()by @David Arenburg. Anyway, these three methods are all much better than the loops by the OP :(. Thanks guys! @David Arenburg @Sixiang.Hu


Mean change scores analysis for studying natural course of disease

Question: I investigating how vision (amongst other outcomes) changes over time, assessed at time 0, week 12 and week 24 in an observational cohort, and wanted to obtain the mean change values at each time point, with p-values comparing the change from baseline. There's no missing data at baseline. I am thinking of fitting some variation of the linear mixed effects model, something like the following:
(1) vision ~ Age * Time + baseline_vision * Time + 1|ID
Baseline vision is being adjusted for as a covariate as is the norm, though not entirely sure it is needed when studying natural course of the disease (observational cohort). I've also adjusted for age, so that I can obtain the mean change in vision at each time point in this cohort while holding age constant.
Therefore, in the outcome only week 12 and week 24 measurements are modelled. I understand that using the emmeans function you can get the adjusted mean values for vision at times week 12 and week 24, i.e. emmeans(model, ~ Time).
Is it possible to get the mean change values at week 12 and 24 from this model? Or do you need to fit the model with change from baseline as the outcome, so something like:
(2) change_from_baseline ~ Age * Time + baseline_vision * Time + 1|ID
Not sure if the model above should be fitted as ive read that modelling change scores is not a good idea, however, both should yield similar results when adjusting for the baseline value so I was wondering if it is possible to get the adjusted mean change values from model (1) ?
Answer: If I understand your question correctly, and if I'm correct that Time is a factor with 3 levels (0, 12, and 24), you can get the means via
emm <- emmeans(model, ~ Time)
emm

Then the changes from week 0 are obtained by
contrast(emm, "trt.vs.ctrl1")

That said, I am not sure what your model is doing. Are you excluding the baseline values from vision and instead putting it in as a covariate? What would be wrong with having vision measurements at all three times according to a factor Week having levels 0, 12, and 24, and Age a covariate, and a simpler model:
model2 <- lmer(vision ~ Age * Week + 1|ID)
(emm2 <- emmeans(model2, ~ Week | Age, at = list(Age = c(<ages of interest>))))
contrast(emm2, "trt.vs.ctrl1")

# and also compare the Age trends
(trnd2 <- emtrends(model2, "Week", var = "Age"))
pairs(trnd2)
```

Comment: @s.stats It just seems to me that since you want to estimate changes from baseline anyway, it is more sensible to include the baseline condition as a factor level so that you can directly estimate those changes. "Controlling for baseline" suggests that is a nuisance factor when in fact it is of interest.
Comment: Thanks! Yes thats right, I have removed the baseline vision values from the outcome and adjusted for it as a covariate. I am studying the trajectory of change in vision in my cohort who are all not receiving any treatment. The baseline values for vision were truncated at a value due to the inclusion criteria. Also, we expect there to be greater changes in vision for lower baseline vision values, so there may be a regression to the mean effect. I wanted to assess the mean change in vision at each time point, while controlling for baseline vision.


need to write htaccess rule for directory pages

Question: I am wondering if there is any way to do this;
If any page is not available in /x2/ directory then it will load same name page from /y2/  without changing the display url.
display url -> abc.com/x2/pen.html
fetch page -> abc.com/y2/pen.html
here page extension is also being rewritten in both directories means .html extension is loading existing .php page content.
Pls let me know if this is possible and how?
Many thanks
Answer: Inside /x2/ directory create this .htaccess:
RewriteEngine On

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /y2/$1 [L]


Creation of ROC curve of logistic regression classifier with rejection

Question: For a logistic regression classifier, I create a roc curve by variation of the threshold on the output probability.
Question: can I create an additional ROC curve with 5% rejection rate based on the classification probability, by rejection of the samples closest to the threshold? This means that every point on the ROC will be based on a different not rejected samples. If yes, where I can find a reference paper about it? If no, what is the proper procedure, and where I can read about it?
Lately, someone suggested that instead of rejecting by 5% of the testing set, I should reject by a threshold which is extracted for 5% of the training set. I am not sure that the difference is important but if it is a standard procedure, I would be happy to find a reference of it.
Comment: [This paper](https://dl.acm.org/doi/10.1016/j.patrec.2004.09.004) might be of interest, they use a three way split (training, validation, test) for training the model, obtaining thresholds and evaluating the models, respectively.
Comment: @APerson, thanks for the paper. It is related very closely to the topic and in fact, what it suggests is similar to what I have proposed. Nevertheless, I think that the writer did not do a very good job. Nevertheless, probably, it is as far as the literature goes...
Comment: What are you trying to accomplish with this re-drawn ROC curve? By my reading, the paper linked by @APerson uses the standard ROC curve to "reject" cases near the cutoff for further testing, based on relative costs of true and false classifications and the cost of the further testing. That doesn't seem to be exactly what you had in mind. In general, one worries about losing information by omitting data from the model, which is what would happen with a re-drawn ROC curve. What is the corresponding advantage that you expect to gain by this procedure?
Comment: what do mean exactly by the verb "reject"? in what context?
Comment: @EdM, I am trying to accomplish creation of a ROC curve with improved performance (say AUC) by the implementation of samples rejection. The link provided by APerson is the closest thing I have found, also not exactly what I looked for. I am not worried about losing information, as the rejection is done after the training and is mainly directed to the testing samples. The one thing that bothered me is creation of ROC with rejection but it seems that it has some reference in the literature. The other thing is the variation of the rejected samples from one point on the ROC curve to another...
Comment: @carlo, in the context of classification. Some of the samples will be classified and some will be rejected
Comment: I know it's not exactly what you want, but you can take a look at a paper about [accuracy-rejection curve](http://proceedings.mlr.press/v8/nadeem10a/nadeem10a.pdf)
Comment: I am familiar with it, thanks
Answer: I have found partial answer to my questions in Tortorella, Francesco. "A ROC-based reject rule for dichotomizers." Pattern Recognition Letters 26.2 (2005): 167-180. (suggested by @Aperson)
Here, for every point in the ROC curve, we define two thresholds, $t_1$, $t_2$ around the original threshold $t_{opt}$ and reject all the samples within the created region. This way allows improving the performance in all the point of the ROC curve.

Only question left is how to define those $t_1=f(t_{opt})$, $t_2=f(t_{opt})$? It seems that there are many different ways to define them but all the ways should be defined based on the training set and implemented on the testing set.
Comment: The paper shows how to choose $t_1$ and $t_2$ based on the relative benefits/costs of true/false positives/negatives, corrected for the costs of rejecting true negatives or positives. Relative costs should always be considered when choosing cutoffs; the usual default classification cutoff at p=0.5 implicitly assumes equal costs of either misclassification type. You might also consider different scoring rules or targeted maximum likelihood as ways to improve performance near a cutoff; see [this page](https://stats.stackexchange.com/q/440764/28500) and links.
Comment: Table 1 of the cited paper seems to have a misprint, with costs of rejecting positives/negatives, CRP and CRN, interchanged between the correct rows. The formulas in the text seem OK at first glance.
Comment: @EdM, I was only looking at the method, did not check the numbers/results.


How should I defend against my opponent's jump smash?

Question: I have played badminton for several years. When I play with advanced players, I have trouble defending because they have a very powerful jump smash. I try to defend against it but they smash very powerfully and to the edge of the line.
Is there any training or guidelines for defending against powerful jump smashes?
Answer: First of all, you should be aware that at professional levels, the (jump) smash is the shot to end a rally (or to prepare a kill shot). Especially in doubles, but also high-level (Men's) singles, players fight for the chance to smash downwards, i.e. try to avoid lifting as much as possible. Sometimes, the easiest way to defend against smashes is not lifting in the first place, and rather playing flat or short. This may necessitate taking shots earlier at the net then you currently are.
Another aspect to watch is the distance and angle of your lift. Analyze (e.g. via video or having a friend record) where the smash started. Smashes from the midcourt or even forecort are nearly impossible to defend against. In Men's singles/doubles, if your lift did not reach the doubles service line, then it is far too short. Additionally, look at the angle of the lift: A flat shot is both easier to hit and already has more energy. If possible, your lift should come down vertically.
The defensive technique begins with the proper footwork. On your forehand side, one step should be enough for sufficiently tall players. On your backhand side, you have two options: Either a small step with your non-racket-arm foot, or stepping over (i.e. body moved away from the net) with your racket-arm foot. For smashes toward the very line, you need the latter unless you are extremely tall.
You have three basic options for your shot: Short, flat, and high. Unless the shot is going far to your racket-arm side, defensive shots are easier with the backhand.
In singles it is usually a good idea to play a short shot, in order to gain the attack yourself, so that you can be the one smashing. However, beware of your opponent storming to the net in order to kill your defense. It is not a trick per se, but you should be able to hit longline as well as cross-court shots. The latter force your opponent to cover a great distance, and are thus usually best if you can consistently hit them.
In doubles, a short defense must be quite good in order to evade the partner of the smasher. Oftentimes, it is better to counter with a flat and fast defense that goes around the front player. If your defense is angled really flat, there's a good chance that you'll end up in a neutral or even attacking situation.
To train your defense, there are multiple options, depending on your level of play (and the level of skills of your coach):

Make sure your footwork technique is correct by doing the footwork for far-right / far-left defenses without a shuttle. Have a coach watch you and note details, such as unnecessary small steps or suboptimal racket preparation.
To get a feel for the shot, you can hit shuttles thrown to you from about 3 meters distance. The feeder should wear safety goggles, especially when you're training fast and flat defenses. You can start by defending shuttles thrown to your body and advance to shuttles thrown at the side lines.
Better, but harder than throwing, is hitting shuttles, typically from the net. The coach just pushes the shuttles downwards (like this). If your coach lacks the necessary skill to hit the shots downwards in this fashion, a machine can also be used. Again, you can start by defending shots to your body or the general vicinity, and graduate to shots going to the sidelines, as well as shorter/longer and with higher frequency.
Finally you simply defend shots in an exercise: You lift everything (or defend short to the net, after which your next shot will be a lift) and your partner smashes everything downwards. It is harder to control for frequency and accuracy, but it's the only way you'll learn to read your opponent's technique in order to anticipate where and how to defend.


Not sure if i am messing up the hover or the IF Then

Question: Ok as always in my mind this should work but i am new to javascript and trying a lot of things at once. This is a nav bar that shows and hides divs while change color of the nav when clicked or hover over. well it does not do that last part. The active ID is the one that is currently displayed. Sorry i now it is nothing simple.
$(function () {
    var active;

    $('.selection').hide();
    $('#homeDiv').show();

    $('.navlink').hover(

    function () {
        $(this).css("color", "#806ac7");
    },

    function () {
        if ($(this == active)) {
            $(this).css("color", "#961014");
        } else {
            $(this).css("color", "#000000");
        }
    });
    $('.navLink').click(function (e) {
        active == $(this);
        $('.navLink').css("color", "#000000");
        $(this).css("color", "#961014");
        $('.selection').hide();
        $('#' + this.id + 'Div').show();
    });
});

Comment: `if($(this == active)) {` Well... you're passing a boolean to `jQuery()`, which will return an object, which is always "truthy".
Comment: and inside your click event `active == $(this)` equates to `false`, then does nothing with `false`. `active` is still `undefined`
Comment: Also, there's no reason to do `$('#' + this.id + 'Div').show();`, when you can just do `$(this).find('div').show();` *(also notice the lowercase `d` in `div`)*.
Answer: In addition to what I noted in my comment under the question, you need to store the element, not a jQuery object, and that compare the elements.
If you try to compare the result of different calls to $(this), it will always be false, because they're unique objects.
$(function () {
    var active;

    $('.selection').hide();
    $('#homeDiv').show();

    $('.navlink').hover(

    function () {
        $(this).css("color", "#806ac7");
    },

    function () {
           // RIGHT HERE, compare the element
        if (this === active) {
            $(this).css("color", "#961014");
        } else {
            $(this).css("color", "#000000");
        }
    });
    $('.navLink').click(function (e) {

        // RIGHT HERE, store the element
        active = this;

        $('.navLink').css("color", "#000000");
        $(this).css("color", "#961014");
        $('.selection').hide();

         // Lowercase "D" in "div", and improved DOM selection
        $(this).find('div').show();
    });
});

Comment: active == this; shouldn't it be active = this ?
Comment: nothing explicitly wrong with storing the jquery object. Probably not as performant though.
Comment: `active == this;` should be `active = this;`
Comment: @Kostia: Except that it was being used for a comparison to `this` in another function. If we store the jQuery object, it will never compare equally to a different jQuery object. We would need to extract the element before the comparison is done.
Comment: @Huangism: Yes, thanks. Didn't notice that error in the question.
Comment: navlink <> navLink that was the main problem. it is always the small things.
Answer: Seems like you misplaced your closing bracket in this line: if ($(this == active)). It should actually be if (this == active)
Also, you are writing a boolean instead of setting a variable: active == $(this) which should be active = this;
Comment: Your `==` comparison will always be `false`, because objects are compared based on identity, so `$(this) == active` will always be comparing two different objects.
Comment: @CrazyTrain touche. Somehow thought jquery would cache the selector.
Comment: It does store the element at index `0`, so you could do `$(this)[0] == active`, but then you'd be creating a jQuery object, and immediately throwing it away. But the `==` just sees them as objects, and checks to see if they're the same object.
Answer: I think the best way to do what you want, is to add a class when the user clicks on your nav items. In your hover handler, check for the class and change the colour accordingly. Or you can simply use important in the css to set the active colour, so you don't need to check for active in your hover
$('.navLink').click(function (e) {
    var $this = $(this);
    $this.addClass('open').css("color", "#961014");
    $('.navLink').css("color", "#000000");
    $('.selection').hide();
    $('#' + this.id + 'Div').show();
});

$('.navlink').hover(

function () {
    $(this).css("color", "#806ac7");
},

function () {
    var $this = $(this);
    if ($this.hasClass('open') {
        $this.css("color", "#961014");
    } else {
        $this.css("color", "#000000");
    }
});

The solution above is using jquery to change the css colour but you can easily do it in css. Assuming the rest of your code works
Comment: I know a lot more CSS then i do javascript, so i did try this first. the problem is with 'active' selector only works with links not li, at least to my knowledge.
Comment: @user2428881 my solution does not involve the :active selector at all, instead it adds a class named active, of course this you can change to whatever you want. You define the important css on the active class not :active. I will change it to something that's less confusing. Changed active to open,this is just a css class
Answer: $(function () {
    var active;

    $('.selection').hide();
    $('#homeDiv').show();

    $('.navLink').hover(

    function () {
        $(this).css("color", "#806ac7");
    },

    function () {
        if (active == this || active == undefined) {
            $(this).css("color", "#961014");
        } else {
            $(this).css("color", "#000000");
        }
    });
    $('.navLink').click(function (e) {
        active = this;
        $('.navLink').css("color", "#000000");
        $(this).css("color", "#961014");
        $('.selection').hide();
        $('#' + this.id + 'Div').show();
    });
});

http://jsfiddle.net/n3heA/


Using High School Algebra to turn first equation into format of second equation.

Question: This question is really bothering me.  It is just for personal study, but I can't seem to turn this one equation into the other just using basic algebra(in the high school sense)(wolfram alpha clearly says the equations are equal but I'm struggling showing/turning the first one into the other).
$$\frac{(1+i)z+(3+i)}{(-1+i)z+(3-i)}\quad=\quad i\times\frac{2-i+z}{2+i-z}$$
Answer: Multiply numerator and denominator with $\color{blue}{1+i}$ and simplify:
$$\begin{align}\frac{(1+i)z+(3+i)}{(-1+i)z+(3-i)}
& = \frac{3+z+i(z+1)}{3-z+i(z-1)}\color{blue}{\frac{1+i}{1+i}} \\[6pt]
& = \cdots \\[6pt]
& = \frac{2+2iz+4i}{4-2z+2i} \\[6pt]
& = i\frac{ (2 - i) + z}{(2+i)-z}
\end{align}$$
Answer: A straight-forward, grind way is to set $z:= x + iy$ and make calculations to bring the numerator and denominator in the form $a+ib$ and then multiply the denominator with $a-ib$. It should 99.9% lead you to the result you got sooner or later ($a=Re\{denominator\}$ and $b = Im\{denominator\})$.
Comment: I tried it just now, and I honestly am not getting near the desired form at all.


Proof of Equivalence of Sets

Question: If A is a subset of the set of all functions $f : \mathbb{R}\rightarrow \mathbb{R}$ and let $g:\mathbb{R}\rightarrow\mathbb{R}$ be a bijective function. We use the notation $gAg^{−1} = \{g\circ f\circ g^{−1} : f \in A\}$. Prove that $A ≈ gAg^{−1}$.
I know to show equivalence I need to show a bijection from $A$ to $gAg^{−1}$, I'm not sure how to use the given information to start going about this.
Answer: A bijection from $A$ to $gAg^{-1}$ is, first of all, a function. Given an element $f$ of $A$, what would the obvious way to define an element of $gAg^{-1}$ be? This defines a function which is obviously surjective; then you need to check that it's injective, which shouldn't be too hard.

Proof of Equivalence of Sets


Are regression trees actually performing "regression?"

Question: I haven't used decision trees in the past, and I'm looking into them now.
With regression trees, I am wondering if we are technically performing classification instead of regression. We train our decision tree using training data that have continuous outputs, but the regression itself has a finite number of nodes and thus a finite number of outputs. So when you feed in an example into the tree, there's a finite number of predictions it can make. Doesn't that effectively make the regression problem a classification problem ?
Answer: Consider linear regression with a single parameter.  It will always predict one number, namely the sample mean of the outcome on which it was trained.  Is this regression even though it can only give one output?
If you subscribe to the definition of regression as a learning problem in which the output is quantitative rather than categorical (or perhaps more strongly, a learning problem in which the predictions could possibly belong to some infinte set rather than a finite set), then trees are performing regression.  They are capable of learning to output continuous values rather than prediction category membership (well...they can do that too, but assume we're only talking about regression trees).
Frankly, what we call what the tree does is not really important. If you want to insist it is classification, be my guest.  Though I suspect you'll see some eyebrows raised when you do.
Comment: The distinction you are making is between a linear function and a piecewise constant function -- but that's neither fundamental nor conceptually relevant to regression.  See the hits at https://stats.stackexchange.com/search?q=regression+definition for more about this.
Comment: @anonuser01 consider simple linear regression with single binary feature $y = \beta_0 + \beta_1 X + \varepsilon$. in such case, it can predict either $\beta_0$ for $X=0$, or $\beta_0 + \beta_1$ for $X = 1$, so also two outcomes. Regression is about predicting numerical outcomes.
Comment: @whuber The "piecewise constant function" part clicked for me.
Comment: I don't really get the linear regression analogy. For a single input, it clearly has a unique output, but linear regression itself can generate an infinite number of outputs $\in \mathbb{R}$ whereas a regression tree has an output $\in T$, where $|T|$ is the number of leaves in the tree.
Comment: @Tim In your example I agree that I would categorize that as a regression problem even though there are only 2 outcomes. But when you say regression is about prediction "numerical" outcomes, are you implying that classification isn't? Classification could also predict numerical outcomes.
Comment: @anonuser01 no it doesn't. It predicts categories, like "cat", "male", "high", etc. Sure, you could predict categories like "1" vs "2" vs "more", but classification *doesn't care* that those are numbers. Categories have no numerical meaning in classification.
Answer: The term "classification" is generally used to refer to situations where the output is from a small number of options that do not have any structure, such as an inherent order (the term is sometimes still used in cases where the output does have a structure, but usually it doesn't).
The definition of "regression" is often given as meaning that the output is continuous, but there are a finite number of values that a float variable can take, yet we consider it to be "continuous". There is a point at which a large enough number of possible outputs, with a clear numerical structure of order, etc., is considered to be "regression".
For instance, if SAT score is considered to be a predictive model of future college grades, it would generally be considered to be more of a regression than a classification model, even though there are a finite number of possible scores. Netflix's percentage match is regression, even though there are only 101 different possible percentages. Something along the lines of the Homeland Security Advisory System is debatable: it has a small number of categories, but those categories are ordered. Making it even more complicated, classification models often are derived from a regression model. For instance, if you're training a model to detect "cat" versus "noncat", you likely are going to calculate a "catlike" score, and output "cat" if it exceeds a threshold.


ASP.Net MVC Custom Authentication

Question:
Possible Duplicate:
Is it possible to create a Logon System with ASP.NET MVC but not use the MembershipProvider?

I'm looking to create a custom authentication model for my MVC app but I'm not sure where to implement my custom IPrincipal and IIdentity classes. I don't want to do this in the Global.asax on every request as not all the pages in the site will need authentication. So my question is where? Should I be doing this in my custom action filters for the actions that require the user to be logged in?
I don't wish to use the Membership Provider or Forms Authentication this needs to be completely custom and separated.
Comment: This should have been kept open. The listed duplicate link covers authorization, but not authentication.
Answer: Have you considered creating a seperate base class for your controllers?
You can override the OnActionExecuting event to check to see if the user is authenticated... something like:
<code>Public Class AuthenticatedPageController
    Inherits Controller

    Protected Overrides Sub OnActionExecuting(ByVal filterContext As ActionExecutingContext)
    // Code to check user is authenticated - if not chuck them to the log in page...

    End Sub

End Class
</code>
Then, any controllers for pages where the user needs to be authenticated, inherit from this class instead of the standard controller class.
Comment: This is kind of what I want but when I check the current context user object I want it to be my implementation of the IPrincipal object and not the one created by default.

ASP.Net MVC Custom Authentication


how to render content type's fields in Custom Module Form

Question: I'm new to Drupal 8 and I'm trying to create a form that allow to fetch my content types hierarchy and show its fields using ajax callback, but unfortunately none of the fields shows up eventhough the ajax works fine and callback function correctly fired?
The Fields Types in my ContentType is {body:text_with_summary, price:decimal, title:string}
As I Tried with debugging it seems that my code cannot define those types as form field types
here is my code in MyForm.php File:
public function level3SelectedCallback(array &$form, FormStateInterface $form_state) {
    $ajax_response = new AjaxResponse();
    $fields = MyController::contentTypeFields('MY_VOC');

    foreach ($fields as $field){
        $name = $field->id();
        $label = $field->label();
        $type = $field->getType();
        $desc = $field->getDescription();
        $form['fields_wrapper']['field_'.$label] = array(
            '#type' => $type,
            '#description' => $desc,
            //'#options' => '',
            '#attributes' => array(
                'class' => array(
                    'myField'
                ),
            ),
            );
        $ajax_response->addCommand(new HtmlCommand('#fields-wrapper', $form['fields_wrapper']['field_'.$label]));

    }

    $ajax_response->addCommand(new InvokeCommand('#fields-wrapper', 'change'));
    $form_state->setRebuild(True);

    return $ajax_response;
}

My Controller Function is:
static function contentTypeFieldsAsFields($contentType) {
    // Get the definitions
    $fields = array_filter(
            \Drupal::service('entity.manager')->getFieldDefinitions('node', $contentType), function ($field_definition) {
                return $field_definition instanceof \Drupal\field\Entity\FieldConfig;
            }
    );
    return $fields;
}

Answer: To get an empty form of a content type you would use code like this:
$node = \Drupal\node\Entity\Node::create(['type' => 'page']);
$form = \Drupal::service('entity.form_builder')->getForm($node);

To open this form with ajax, put the form in a controller:
public function contenttypeform(NodeTypeInterface $node_type) {
    $node = $this->entityTypeManager()->getStorage('node')->create([
      'type' => $node_type->id(),
    ]);
    return $this->entityFormBuilder()->getForm($node);
  }

Configure a route for this controller and open the form in an ajax modal like this example from ConfigSync.php:
$links['view_diff'] = array(
            '#type' => 'link',
            'title' => $this->t('View differences'),
            'url' => Url::fromRoute($route_name, $route_options),
            'attributes' => array(
              'class' => array('use-ajax'),
              'data-dialog-type' => 'modal',
              'data-dialog-options' => json_encode(array(
                'width' => 700
              )),
            ),
          );

Comment: Great But yes I want to do this in D8 specifically in ajax callback
Comment: I don't think that this can work, but you could use a modal dialog with ajax, see the edited answer.
Answer: Thanks @4k4 for your great help but I've solved it how to use it in drupal 8 here is my FormBuild code but unfortunately not all fields shown just the title and there is the warning shown in console:
Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.
VM3435:19 [CKEDITOR] Error code: editor-incorrect-destroy.
VM3435:19 [CKEDITOR] For more information about this error go to http://docs.ckeditor.com/#!/guide/dev_errors-section-editor-incorrect-destroy
and here is my Form Build Code:
$form['fields_wrapper'] = [
        '#type' => 'container',
        '#attributes' => ['id' => 'fields-wrapper', 'class' => ['hiddenselect']],
    ];

    if ($form_state->getValue('level1') != false) {
        $val = $form_state->getValue('level1');
        $this->fields = MY_Controller::contentTypeFieldsAsFields($val);
        $node = \Drupal\node\Entity\Node::create(['type' => $val]);
        $myform = \Drupal::service('entity.form_builder')->getForm($node);

        foreach ($this->fields as $field) {
            //$name = $field->id();
            $n = $field->getName();
            $form['fields_wrapper'][$n] = $myform[$n];
        }

and here is my ajax callback code:
public function level1SelectedCallback(array &$form, FormStateInterface $form_state) {
    // Instantiate an AjaxResponse Object to return.
    $ajax_response = new AjaxResponse();
    $val = $form_state->getValue('level1');

    foreach ($this->fields as $field) {
        $n = $field->getName();
        $ajax_response->addCommand(new HtmlCommand('#fields-wrapper', $form['fields_wrapper'][$n]));
    }
    $ajax_response->addCommand(new InvokeCommand('#fields-wrapper', 'change'));

    $form_state->setRebuild(True);

    return $ajax_response;
}

Answer: Use
\Drupal::entityManager()->getFieldDefinitions('entity_type','bundle');

It will give all fields related to content type.
EntityFieldManagerInterface::getFieldDefinitions()
Comment: I'm using it in my controller function as the last piece of code in my Question after edit it: