Repository - Filtro de Registros de acordo com permissão do usuário

Question: Pesquisando sobre como filtrar os registros com os repository, encontrei as anotações:

@PreAuthorize
@PreFilter
@PostFilter

Tem 2 Roles no sistema: ROLE_ADMIN e ROLE_USER sendo que o ROLE_ADMIN pode acessar todos os registros, o ROLE_USER pode acessar somente seus registros, com base em condições de colunas da tabela, o que recomendam?

Fazer um método diferente pra cada tipo de usuário e usar o controlador para gerenciar qual método usar?
Usar os filtros no repository?

Percebi que o @PostFilter retorna tudo e somente depois filtra, retornando tudo e depois filtrando, deixando a consulta mais lenta, e o @PreFilter apresenta a exceção abaixo:
@PreFilter("filterObject.id == 7")
java.lang.IllegalArgumentException: A PreFilter expression was set but the method argument typeclass java.lang.String is not filterable
at org.springframework.security.access.expression.method.ExpressionBasedPreInvocationAdvice.findFilterTarget(ExpressionBasedPreInvocationAdvice.java:82)
at org.springframework.security.access.expression.method.ExpressionBasedPreInvocationAdvice.before(ExpressionBasedPreInvocationAdvice.java:50)
at org.springframework.security.access.prepost.PreInvocationAuthorizationAdviceVoter.vote(PreInvocationAuthorizationAdviceVoter.java:72)
at org.springframework.security.access.prepost.PreInvocationAuthorizationAdviceVoter.vote(PreInvocationAuthorizationAdviceVoter.java:40)
at org.springframework.security.access.vote.AffirmativeBased.decide(AffirmativeBased.java:63)
at org.springframework.security.access.intercept.AbstractSecurityInterceptor.beforeInvocation(AbstractSecurityInterceptor.java:233)
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:65)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy127.findAllByIdentificadorRelacionamentoOcorrencia(Unknown Source)
at br.com.teste.controller.OcorrenciaRegistroFuncionarioControllerImpl.listar(OcorrenciaRegistroFuncionarioControllerImpl.java:36)
at br.com.teste.controller.OcorrenciaController.init(OcorrenciaController.java:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:311)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:134)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:409)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1620)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$2.getObject(AbstractBeanFactory.java:345)
at org.joinfaces.annotations.ViewScope.get(ViewScope.java:42)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:340)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1081)
at org.springframework.beans.factory.access.el.SpringBeanELResolver.getValue(SpringBeanELResolver.java:55)
at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
at org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:94)
at org.apache.el.parser.AstValue.getValue(AstValue.java:137)
at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:184)
at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:109)
at com.sun.faces.facelets.tag.TagAttributeImpl.getObject(TagAttributeImpl.java:356)
at com.sun.faces.facelets.tag.TagAttributeImpl.getBoolean(TagAttributeImpl.java:150)
at com.sun.faces.facelets.tag.jstl.core.IfHandler.apply(IfHandler.java:91)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:95)
at javax.faces.view.facelets.DelegatingMetaTagHandler.applyNextHandler(DelegatingMetaTagHandler.java:137)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:202)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:95)
at javax.faces.view.facelets.DelegatingMetaTagHandler.applyNextHandler(DelegatingMetaTagHandler.java:137)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:202)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.DelegatingMetaTagHandler.applyNextHandler(DelegatingMetaTagHandler.java:137)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:202)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:95)
at com.sun.faces.facelets.tag.ui.DefineHandler.applyDefinition(DefineHandler.java:106)
at com.sun.faces.facelets.tag.ui.CompositionHandler.apply(CompositionHandler.java:206)
at com.sun.faces.facelets.impl.DefaultFaceletContext$TemplateManager.apply(DefaultFaceletContext.java:395)
at com.sun.faces.facelets.impl.DefaultFaceletContext.includeDefinition(DefaultFaceletContext.java:366)
at com.sun.faces.facelets.tag.ui.InsertHandler.apply(InsertHandler.java:111)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:95)
at javax.faces.view.facelets.DelegatingMetaTagHandler.applyNextHandler(DelegatingMetaTagHandler.java:137)
at com.sun.faces.facelets.tag.jsf.ComponentTagHandlerDelegateImpl.apply(ComponentTagHandlerDelegateImpl.java:202)
at javax.faces.view.facelets.DelegatingMetaTagHandler.apply(DelegatingMetaTagHandler.java:120)
at javax.faces.view.facelets.CompositeFaceletHandler.apply(CompositeFaceletHandler.java:95)
at com.sun.faces.facelets.compiler.NamespaceHandler.apply(NamespaceHandler.java:93)
at com.sun.faces.facelets.compiler.EncodingHandler.apply(EncodingHandler.java:87)
at com.sun.faces.facelets.impl.DefaultFacelet.include(DefaultFacelet.java:312)
at com.sun.faces.facelets.impl.DefaultFacelet.include(DefaultFacelet.java:371)
at com.sun.faces.facelets.impl.DefaultFacelet.include(DefaultFacelet.java:350)
at com.sun.faces.facelets.impl.DefaultFaceletContext.includeFacelet(DefaultFaceletContext.java:199)
at com.sun.faces.facelets.tag.ui.CompositionHandler.apply(CompositionHandler.java:174)
at com.sun.faces.facelets.compiler.NamespaceHandler.apply(NamespaceHandler.java:93)
at com.sun.faces.facelets.compiler.EncodingHandler.apply(EncodingHandler.java:87)
at com.sun.faces.facelets.impl.DefaultFacelet.apply(DefaultFacelet.java:161)
at com.sun.faces.application.view.FaceletViewHandlingStrategy.buildView(FaceletViewHandlingStrategy.java:1006)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:99)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:219)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:659)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:115)
at org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:59)
at org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:90)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:108)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:474)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)

Answer: Uma possibilidade simples, que é baseada no usuário autenticado, não necessidando criar mais endpoint, é fazer um filtro no repository, conforme exemplo abaixo:
Controler
@RequestMapping(value = "/clientes",
    method = RequestMethod.GET,
    produces = MediaType.APPLICATION_JSON_VALUE)
@Timed
public ResponseEntity<List<Cliente>> getClientes() throws URISyntaxException {
    List<Cliente> clientes;
    if (SecurityUtils.isCurrentUserInRole(AuthoritiesConstants.ADMIN)) {
        clientes= clienteRepository.findAll();
    } else {
        clientes= clienteRepository.findByUserIsCurrentUser();
    }

    return new ResponseEntity<>(clientes, HttpStatus.OK);
}

Repository
public interface ClienteRepository extends JpaRepository<Cliente,Long> {

    @Query("select Clientes from Clientes clientes where clientes.user.login = ?#{principal.username}")
    List<Cliente> findByUserIsCurrentUser();

}

Util
public final class SecurityUtils {

    private SecurityUtils() {
    }

    public static boolean isCurrentUserInRole(String authority) {
        SecurityContext securityContext = SecurityContextHolder.getContext();
        Authentication authentication = securityContext.getAuthentication();
        if (authentication != null) {
            return authentication.getAuthorities().stream()
                .anyMatch(grantedAuthority -> grantedAuthority.getAuthority().equals(authority));
        }
        return false;
    }
}

Um ótimo exemplo opensource com a stack Spring Boot + Angular a ser estudado é o JHipster.
Comment: Obrigado Murillo, vou estudar.


SQL developer counting and showing info

Question: SELECT PERSONALINFO.FN,
   PERSONALINFO.SN
 FROM PERSONALINFO
 JOIN BOOKING ON BOOKED.EMAIL = PERSONALINFO.EMAIL
 HAVING COUNT(BOOKEDE.EMAIL) > 2

I am using sqldeveloper to try and count how many times an email comes up in the booking database, and if the email comes up more than twice it will get the personalinfo from the personalinfo database and show it, but I just can't seem to get it working at all,  am I am quite lost.
Answer: When you use an aggregate function, you need a GROUP BY clause
SELECT PERSONALINFO.FIRSTNAME,
       PERSONALINFO.SECONDNAME
FROM PERSONALINFO
JOIN BOOKING ON BOOKING.EMAIL = PERSONALINFO.EMAIL
GROUP BY PERSONALINFO.FIRSTNAME, PERSONALINFO.SECONDNAME
HAVING COUNT(BOOKING.EMAIL) > 2

Comment: Thanks man! any tips on how to order it by descending booking.email counts, I tried ORDER BY BOOKING.EMAIL and ORDER BY COUNT(BOOKING.EMAIl)
Comment: If you add COUNT(BOOKING.EMAIL) as a 3rd selected column,  you may be able to do ORDER BY 3 DESC.  I know that works in SQL Server, and I'm not sure if that's a SQL Server specific trick.  http://sqlfiddle.com/#!6/3b668/1/0
Answer: SELECT PL.FIRSTNAME, PL.SECONDNAME, PL.EMAIL FROM PERSONALINFO PL
  INNER JOIN BOOKING B ON B.EMAIL = PL.EMAIL
  WHERE COUNT(B.EMAIL)>2 ;

Answer: I think this is what you want:
SELECT  pi.FIRSTNAME ,
        pi.SECONDNAME ,
        pi.EMAIL
FROM    (select email
         from BOOKING b
         group by email
         having count(*) > 2
        ) b join
        PERSONALINFO pi
        ON b.EMAIL = pi.EMAIL;

Your initial query has several problems.  Some are syntactic, such as having a count() function in the where clause rather than in a having clause.  Some are logical, such as the fact that duplicate emails in PERSONALINFO will also cause "matches" to the query, even when there is only one booking.
This version fixes these problems by aggregating BOOKING first, extracting the emails of interest, and then joining to the other table.
Also, I added table aliases into the query (the pi and b).  This makes the query easier to read and maintain.

SQL developer counting and showing info


Making all methods static

Question: In my application there are many classes which contain methods for processing data which can be computation and data enrichment.
My question is - if the class doesn't have any class level variables, can I make all the methods in the class static?
I suppose that there won't be any issues with threading.
Are there any consequences? Are there any performance advantage as I don't have to instantiate the class?
A sample class:
Class A{
    public Object findTheCar(String id){
        Car car= new Car();
        //do something
        return car;
    }
}

I intend to change the above to static.
Comment: If you find you are wanting to write classes which contain many/all static methods, you are not thinking object-oriented; you aren't doing OO design.   You should consult a textbook such as http://www.amazon.com/dp/0672330164/?tag=stackoverfl08-20 for more info.
Comment: Oh, I dunno, perhaps your employer, professor, coworkers, future self, and Buddha?
Comment: @BadZen some of those seem valid ;)
Comment: http://stackoverflow.com/questions/2671496/java-when-to-use-static-methods
Comment: @GhostCat how to accept answer?
Comment: @blathur the LINK that I put behind "accept" in my comment tells you how.
Comment: And for the record: *you* decide which answer you intend to accept. If you want to accept any of the current one. But chances are that not much more will be coming in.
Answer: One rule-of-thumb: ask yourself "does it make sense to call this method, even if no Obj has been constructed yet?" If so, it should definitely be static.
So in a class Car you might have a method double convertMpgToKpl(double mpg) which would be static, because one might want to know what 35mpg converts to, even if nobody has ever built a Car. But void setMileage(double mpg) (which sets the efficiency of one particular Car) can't be static since it's inconceivable to call the method before any Car has been constructed.
(Btw, the converse isn't always true: you might sometimes have a method which involves two Car objects, and still want it to be static. E.g. Car theMoreEfficientOf( Car c1, Car c2 ). Although this could be converted to a non-static version, some would argue that since there isn't a "privileged" choice of which Car is more important, you shouldn't force a caller to choose one Car as the object you'll invoke the method on. This situation accounts for a fairly small fraction of all static methods, though.)
Though there are some valid reasons to use static methods:

Performance: if you want some code to be run, and don't want to instantiate an extra object to do so, shove it into a static method. The JVM also can optimize static methods a lot (I think I've once read James Gosling declaring that you don't need custom instructions in the JVM, since static methods will be just as fast, but couldn't find the source - thus it could be completely false). Yes, it is micro-optimization, and probably unneeded. And we programmers never do unneeded things just because they are cool, right?
Practicality: instead of calling new Util().method(arg), call Util.method(arg), or method(arg) with static imports. Easier, shorter.
Adding methods: you really wanted the class String to have a removeSpecialChars() instance method, but it's not there (and it shouldn't, since your project's special characters may be different from the other project's), and you can't add it (since Java is minimally sane), so you create an utility class, and call removeSpecialChars(s) instead of s.removeSpecialChars(). Sweet.
Purity: taking some precautions, your static method will be a pure function, that is, the only thing it depends on is its parameters. Data in, data out. This is easier to read and debug, since you don't have inheritance quirks to worry about. You can do it with instance methods too, but the compiler will help you a little more with static methods (by not allowing references to instance attributes, overriding methods, etc.).

You'll also have to create a static method if you want to make a singleton, but... don't. I mean, think twice.
Now, more importantly, why you wouldn't want to create a static method? Basically, polymorphism goes out of the window. You'll not be able to override the method, nor declare it in an interface. It takes a lot of flexibility out from your design. Also, if you need state, you'll end up with lots of concurrency bugs and/or bottlenecks if you are not careful.

So define static methods in the following scenarios only:

If you are writing utility classes and they are not supposed to be changed.
If the method is not using any instance variable.
If any operation is not dependent on instance creation.
If there is some code that can easily be shared by all the instance methods, extract that code into a static method.
If you are sure that the definition of the method will never be changed or overridden. As static methods can not be overridden.

Lets also discuss it in more details:
Advantages:
Static members/methods are used as in helper classes say like Math or in constants classes. which helps other objects to utilize Strings or useful functions for which you do not need to create object but invoked using Class name. Example – singleton objects are invoked using a static function.
Disadvantages:
Static members are part of class and thus remain in memory till application terminates and can't be ever garbage collected. Using excess of static members sometime predicts that you fail to design your product and trying to cop of with static / procedural programming. It denotes that object oriented design is compromised. This can result in memory over flow. Also there are certain disadvantages if you make any method static in Java for example you can not override any static method in Java so it makes testing harder you can not replace that method with mock. Since static method maintains global state they can create subtle bug in concurrent environment which is hard to detect and fix.
Things to remember:
The static variable will be part of the class definition rather than on the heap. However static variables are useful when you know there will be accesses to the object from multiple places. Access to static resources is not thread safe. You might get weird/unpredictable results in a threaded environment. But if your only reading the static value then using threads for it is fine.
How Static Breaks encapsulation:
The technical implementation of them is to allow state to be maintained across all instances of a class. The problem is that this is intrinsically not OOP because it disregards encapsulation. If a variable can be altered by any instance of a class then the fundamental principle behind encapsulation/information hiding is lost entirely: An object is no longer in complete control of its state. Its state now relies on variables which are essentially global. Which we know is bad. Even private static variables maintain state at a global level but simply limit its access. Any instance of the object can alter the static variable which causes ambiguity as individual instances of the object no longer have control over their own state. State changes can arbitrarily happen without knowledge of an object which relies on that state which is problematic because the object may not work correctly when this happens. Much as it's often said that "Inheritance breaks encapsulation" statics do this in a far more severe way: By not just exposing internal implementation but also by exposing internal state.

In Your Example Question:

As you mentioned, the method is going to be used in multithreaded invironment, think of the below problem (modified your code):
public Object findTheCar(String id) {
        Car car = null; //Line 2
        if (id != null) {
            car = new Car();
        }
        //Line 6
        // do something

        //
        //Line 10
        return car;
    }

In the above: if this executed from two threads, and 1st thread is at
  line 6, and second thread is at line 2, still this will be thread
  safe.

Because:
Local variables are stored in each thread's own stack. That means that local variables are never shared between threads. That also means that all local primitive variables are thread safe.
Local references to objects are a bit different. The reference itself is not shared. The object referenced however, is not stored in each threads's local stack. All objects are stored in the shared heap. If an object created locally never escapes the method it was created in, it is thread safe. In fact you can also pass it on to other methods and objects as long as none of these methods or objects make the passed object available to other threads.
Object members are stored on the heap along with the object. Therefore, if two threads call a method on the same object instance and this method updates object members, the method is not thread safe.
Thread safety check: If a resource is created, used and disposed within the control of the same thread, and never escapes the control of this thread,the use of that resource is thread safe.
From: http://tutorials.jenkov.com/java-concurrency/thread-safety.html
Comment: Thanks for your valuable time and a well covered answer. But my point here is as the class does not contain any class level variables how the class is supposed to have a state to alter? same with the garbage collection. And Why classify as utilty class? My class here is going to process concurrent requests.
Comment: @rahulk updated the answer to give you more clarity.
Comment: @ shridutt kothari I have doubt there in the updated answer, Both thread will be handling the method in their own context,never overlapping - is it not? Other wise this issue will occur for all the utilty classes.
Comment: @rahulk Yes, it will raise the issue in all static utility classes, static code blocks are more buggy in multithreaded environment.
Comment: In the case of method findTheCar, the local variable car is not shared between threads. Each thread has its own.
Comment: @KlitosKyriacou : yes you are right, local variable car will not be shared between threads, i updated the answer. Thanks for pointing the mistake!!
Comment: @Klitos Kyriacou methods will be in threads own stack..right?
Comment: @rahulk yes you and KlitosKyriacou, both are right, local variables will be in thread's own stack. Cheers!!!
Answer: If you have no class level variables, then yes, you can make all the methods on your class static. And it might even be slightly more efficient due to avoiding object instantiations. But in object oriented programming, objects have "state" and "behavior". Your objects have no state, only behavior. So are you really doing object oriented programming here? I would argue you are not (which isn't necessarily a bad thing). But maybe you might be more comfortable in a non-OO programming language.
Comment: What about Math class https://docs.oracle.com/javase/7/docs/api/java/lang/Math.html? Do you think this is an example of not OOP in java core?
Comment: @Pavlo. Yes. The Math class is most certainly not object oriented. It's a function library. It has a private constructor so objects are not even meant to be instantiated from the Math class.
Comment: @Asaph if there are no class level variables is there state to worry about?
Comment: @rahulk Correct. There is no state. My overall point is that if the majority of your classes have no state, you don't really have an OO design. Consider refactoring.
Answer: Class without properties is thread safe. So you can work with class instance or make static method. Both will work fine. This is the question of your application architecture. If this simple Java application, I would use static method here.
Comment: May be this is not a simple java application, But I am thinking like keeping OOPS aside, why not going for a rule like 'if no class level variables- go for static' :)
Comment: If you are concerned with thread-safety, then the way to go are immutable classes. Not static methods.
Answer: There are two issues that I think what most other answers completely ignored so far.
First of all, static is an anomaly in objected oriented programming. If you look at it, it is simply "procedural programming" in disguise. This could mean: if your code consists mainly of static methods that somehow massage certain data ... chances are: you didn't do a good job in proper OO modeling/design. Because, when you create OO designs, you look into creating abstractions; and bringing together "data and behavior" using reasonable encapsulation; and even more important by providing clever information hiding.
Then (almost as worse in my eyes): using static puts a huge burden on the testability of  your code. So, chances are: you are either not doing TDD/unit tests; or you already resorted to byte-code-manipulating frameworks such as PowerMock in order to test code that uses static methods. And both of those options are rather ... not so good.
So, long story short: The answer from shridutt gives you a good starting point to assess what could be static. But I wholeheartedly recommend you to first step back, and for example: show some of your code to experienced designers/coders and get their feedback regarding "how OO and testable is this stuff".
Your question sounds like you could benefit more from thinking a bit more about your code than from adding some "static"s here or there.
Comment: How does using static make it more difficult to test your code? You have a function, it takes arguments and returns a value. Seems very easy to test, assuming it doesn't have side-effects.
Comment: If you use static methods carefully, i.e., make them pure, IMO this is a good practice instead of an anomaly. Pure functions are easy to test and reason about.
Comment: Anyway I would definitely suggest thinking less about your code, coding more, and seeing where that takes you. Thinking more about your code leads to analysis paralysis. I've found more success in coding, testing, coding, testing, coding, testing, refactoring.
Comment: @dimadima If you are doing reasonable unit tests, you very often have to **control** the methods that your class under test is calling. You can't control static methods; only by using PowerMock. And as said: using PowerMock is a very bad idea.
Answer: You can definitely make all the methods static. Why not? If the language allows it, and it makes sense to you, do it. I've rarely benefited from doing something that didn't make sense to me just because I'd not seen that done before or because it didn't follow some design pattern (like "OO"—that means a lot of things to many people, and Java—especially Java 8—actually supports many paradigms in addition to OO). I definitely wouldn't add lines of code that aren't necessary (Obj a = new Obj(); a.do() vs Obj.do()).
In terms of prior art, such classes are often called "utility classes". But there's nothing that has to be utility about such a class. Personally, I use static methods when I see no need to capture values or references within a class instance. There are many times this arises for me, and I've seen this all over in other people's programming as well.
Comment: (1) Each static method, presuming it has no side effects, is a unit you can test. (2) OO is not some rigid concept that calls for all things to be objects.
Comment: Sure you can unit test a static method. But how do you unit test a method that calls a static method?
Comment: Hmm, because **static** is to **OO** like "Ike" to "Tina Turner"? Or maybe because **static** will render your code un-unittestable almost by default?
Comment: @dimadima Same thought here
Comment: I consider static methods as a way to get around the restriction of java, i.e., everything needs to be inside a class definition. Pure functions are good, as we have already seen from tons of functional programming languages.
Answer: It depends on need,if we want to make methods static or not. A good thumb rule is if the method is going to called (used) across the application for performing some standard operations it is usually ok to consider making them static.
Drawback of overusing static is related to disposing and releasing object since, static method will stay in memory till program is stopped (re-started), overuse of static can have some impact on memory consumption.
Comment: static method will stay in memory till program is stopped - the blueprint for the method is staying in memory which can be ignored -Am I right?
Answer: If the methods do not modify a particular instance, that is, if the method does not change the instance variables of the object then you can make the method static.If you're making all the methods static, then all the variables outside methods should be static as well.
About threading, as per my knowledge, the difference would be when you would want to synchronize your code.For instance methods, you synchronize or you lock on objects, however for static methods you synchronize on class variables.So at a particular time in your class, only one synchronized block will be executing.
(For instance methods, for n instances, n synchronized blocks could execute).
For performance
In case of static methods,the binding is at compile time and for instance methods the binding is dynamic, that is at run time.Also the compiler implicitly inserts a this for instance methods.So static methods are slighly faster than instance methods.
In inheritance, static methods cannot be overriden, but then you can always call the method directly with the class, so that's no issue.
So, if you're not instantiating the class, making the methods static and using it as a helper class is a good idea.

Making all methods static


Selenium Error loading page

Question: After recompiling geckodriver, and moving it into my /bin directory (if you need help with that look here)
I've ran into another problem, and I'm not quite sure how to fix it.
/usr/bin/python2.7 /home/lyend/Documents/python/tools/swagSearcher/swagSearcher.py
Traceback (most recent call last):
  File "/home/lyend/Documents/python/tools/swagSearcher/swagSearcher.py", line 8, in <module>
    web.get('google.com')
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 248, in get
    self.execute(Command.GET, {'url': url})
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Error loading page
I'm truly stuck, and it has nothing to do with my code. If you'd like to know the code, check below
web = webdriver.Firefox()
web.get('google.com')

OS: Linux Mint 17.3
Python: 2.7.6
Firefox Version: 49.0
I'd appreciate any help with this error.
Answer: web.get('google.com')

to
web.get('http://www.google.com')

solved this issue for me on a very similar build

Selenium Error loading page


Poisson Process Probability Question

Question: The following is question taken from a set of lecture notes,

Henry works for the council and is responsible for the up keep of speed cameras on roads in the area. Recently a particular camera has stop working, and Henry must go and repair it. However, to diagnose the issue and then complete the repairs, Henry must shut down the camera completely for 2 hours. The council tells Henry that the camera catches speeding drivers at a rate of 3 per hour.
Henry arrives at the camera location at 8:45AM. After waiting 15 minutes, he observes no speeding drivers and decides to start his repairs. What is the probability that any speeding cars will pass the camera while it is down?
You may assume that the passing of cars follows a Poisson process.

I'm having trouble trying to solve this problem. I believe it involves an exponential distribution since it asks for the probability any cars pass it in an interval of time.
I have,
Let $T$ be the number of hours between speeding cars. Let $T \sim \operatorname{Exp}(3)$.
$$\Pr( 9{:}00\text{ AM} < T < 11{:}00 \text{ AM}) = \Pr( 0.25 <T< 2.25) \text{ (where $8{:}45$ AM is $t=0$)}$$
I'm not really sure where to go next.
Answer: In Poisson processes the distribution of the waiting times is memoryless so all you need is to compute the probability of at least one arrival in the next 2 hours.
It is probably easiest to look at the Poisson process. An average of 3/h means an expected value of $\mu=6$ over 2 hours. You then need
$$
P(X\geq1)=1-P(X=0)=1-\frac{\mu^0 e^{-\mu}}{0!}=1-e^{-6}\approx0.9975.
$$
If you want to go the exponential route then consider that inter-arrival times follow an exponential distribution with mean $1/3$ (i.e., 20 minutes). You want to compute
$$
P(T<2)=1-e^{-\frac2{1/3}}=1-e^{-6}\approx0.9975.
$$
Comment: Rather than saying "Poisson processes are memoryless", one should say that the distribution of the waiting times in Poisson processes is memoryless. The Poisson distribution is NOT memoryless and one could be misunderstood as saying that it is.
Comment: Do you mean $1 - Pr(X = 0)$? Thanks for the response too.
Comment: @R.Evet Yes ! fixed my answer.

Poisson Process Probability Question


Bootstrap 3 dropdowns not working via javascript click action

Question: I have a large dropdown help menu that I'd like to open normally from the menu AND also open from a link outside of the dropdown menu structure.
JS Fiddle of broken scenario: http://jsfiddle.net/tmfzog1j/1/
When I run <code>$('.dropdown-toggle').dropdown("toggle");</code> from the browser console, it works fine, which makes me think it's some kind of event.propagation problem.
Answer: works
you have to stop propagation on that puppy, so you were totally right.
<code>$( document ).ready(function() {
  $('.remote').click(function(e){
      e.stopPropagation();
    $(".dropdown-toggle").dropdown('toggle');
  });
});
</code>
JSFIDDLE


Using procmail to remove backdated spam

Question: I started getting spam backdated by a few years. Is there an easy way to detect and divert these to /dev/null using procmail?
Answer: If the date is definitely present then something simple like:
 # The order of the recipes is significant
 #  header format is "Date: Sat, 13 Feb 2010"
 :0
 * ^Date: .*,.*200[0-9]
 /dev/null

Comment: Well, if date is not present, then the mail is not backdated, so it's correct your filter does not match in that case :-).
Answer: It's actually independent of procmail, but a cronjob that run at a fixed period could do this:
find /maildirectory -type f -mtime +7 -exec rm -rf {}\;

This would delete mail over a week old.
Comment: -1 The question is about deleting *backdated* mails (i.e. the mail arrives now, but with a date of like "2008"). Your solution would just delete mail that *arrived* more than 7 days ago :-(.

Using procmail to remove backdated spam


Contextual filter on Views Block file not found

Question: I am trying to use a views block with a contextual filter for taxonomy terms to create an index that can filtered based on terms in the URL.
This works fine with a views page: so if the URL is /my-index then I see all the sub pages. And if the URL is /my-index/apples I see all the sub pages tagged with apples.
Now I when I make a block the same as the page view described, I understand that the block does not have a source for the contextual filter, so I have to provide a filter value. I have tried all the settings, but the problem seems to be that when using a views block and the URL is /my-index/apples I get a file not found error. With /my-index I get the full list as expected.
The filter works in preview mode, and if say I set the Default Value to Fixed Value=apples, is works at URL /my-index (filtered for apples as expected), but when URL is /my-index/apples then again I get file not found error. Then if I change the filter value to 'Display all results for specified field' I get the same result i.e. /my-index shows all and /my-index/apples gets file not found.
It seems to me that when the URL is /my-index/my-term it is not getting to the /my-index page to run the views script in the block. So I cannot connect the page URL to the block because I am not getting to the page at all when I add my-term to the URL.
I have set up a new alias for the page and tried new terms in case there is some sort of clash somewhere but to no avail.
The answers provided at Contextual filter with a block view did not seem to help. I tried every setting in every combination several times.
Answer: I think you are just a bit confused about the structure of the page drupal creates so hopefully this might help to clarify it all.
When you create a page display in Views with a contextual filter, you basically are creating a menu item / path on your website. In this case, it's probably something along the lines of my-index/%, with the % being the contextual filter.
You then define that filter to be a tax term.  So, you can then type in  www.yourhost.com/my-index/foo and www.yourhost.com/my-index/bar and www.yourhost.com/my-index/baz and get the $content of those pages filled with the fields of the view you created filtered by foo, bar, or baz.
Or any other term a user puts in in place of the %.
This way you aren't creating umpteen individual views, eg, you don't have to create my-index/foo, my-index/bar, my-index/baz, etc etc page displays for each and every possible tax term.
Plus if you say show me everything if the contextual filter is missing, then you can also go to www.yourhost.com/my-index and get a list of everything.
Now to blocks.  These aren't the main $content of your page, but rather little boxes of additional information off to the side.  They can be placed pretty much on any page you want through the block admin pages, you just go in there and tell it what region you want and pages and off you go.
So, in this case, let's say you created a page display called my-index/% like the above.  In this example, let's also say that this view lists the content on your system sorted by Title.  So, in the main content area of this page you'd have that listing of your content tagged with that tax term sorted by title.
But let's say you wanted a block off to the side listing the very latest content tagged with that same tax term.
Here, you would create a block display of content sorted by creation date in descending order and limited to 1 item.  Here you would also add a contextual filter, but in the supply the default argument, you would tell it to take it from the url and specify where it is, in this case the 2nd argument.
Then you'd save everything and go to the block admin page and down in the list of blocks available you'd see this View Block.  Assign it to a region in your theme and then in the pages, restrict it to my-index/* and then it will show up on that page along with your main content.
Comment: Jimajamma, a big thank you for your rapid, detailed answer. I don't doubt that I am confused :). However I did understand everything you say, I think. I'm trying to use a block as I needed two parts on a page - one with a carousel, and underneath a list of the page links similarly filtered - I couldnt seem to get two sets of data in a single page view. So I was hoping to pull the tax terms from the url and use them in both blocks. Based on what you said I added a view block to a view page rather than a normal node and it does just what I want now.
Comment: I didn't mean the confused part as an result, more of 'you are *almost* there.'  In any case, glad I could help out!


Model attributes visibility inside admin panel

Question: Is there a way to manage model attributes visibility inside admin panel per action? Suppose I've generated Project API with name and slug attributes via the CLI. I want to hide slug from new and edit actions and use beforeCreate callback to generate it. Can I override the view or there are relative settings?
Answer: The logic used to populate admin fields is mainly contained in this file: https://github.com/wistityhq/strapi-generate-admin/blob/master/files/api/admin/public/src/strapi/explorer/explorer.service.js#L25
If you want to add specific behavior for some fields, please add custom code in this file. To do so:

cd api/admin/public/src/
npm install (node 4 required)
gulp serve
visit http://localhost:3002
edit the logic here: https://github.com/wistityhq/strapi-generate-admin/blob/master/files/api/admin/public/src/strapi/explorer/explorer.service.js#L25
finally, run gulp dist to save your updates

In a next Strapi version, admin fields display will be probably based on an external JSON file or on an adminOptions field located in each attribute of myModel.settings.json files.


Restrict Location per Domain in OpenLayers

Question: I have Domain Access and Open Layers running on a site. Think of it as a map-based Craigslist. Each city has it's own subdomain, offering location-based content.
Users create content with a location on each subdomain, and I need to make sure that content's location is inside a radius (say, within 50 miles of a centerpoint) for that subdomain.
How can I do this?
Answer: I think you can do this with:

GeoIP API
Context
Context GeoIP
Domain Context


Centralizando utilizando o position absolute

Question: Não estou entendo o seguinte código:
div{
    position: absolute;
    top: 0;
    bottom: 0;
    left: 0;
    right: 0;
    margin: auto;
    width: 100px;
    height: 100px;
    background: red;
}

Exemplo online
Eu sei que definir top: 0 e bottom: 0 e setar uma largura o elemento vai esticar ocupando height: 100% do elemento pai(body), o mesmo com left: 0 e right: 0 (me corrijam se eu estiver errado).
Mas oque eu não entendo é o margin: auto, alguém pode me explicar como ele vai centralizar o elemento no centro ?
Answer: Para centralizar algo no centro da pagina existe várias técnicas.
Explicando o margin sua função como observado é definir uma margem para algum elemento, e quando seu valor esta auto quer dizer que o navegador irá definir a margem automaticamente, segue exemplo:

.quadrado{
  background-color: red;
  display: block;
  height: 100px;
  margin: 0 auto;
  width: 100px;
}

Observe que estou utilizando margin: 0 auto o zero representa o margin-top e o margin-bottom, e o valor auto representa o margin-left e o margin-right resumindo é uma abreviação.
No meu exemplo as margens da direita e esquerdas estão automáticas e com isso o navegador defini automaticamente seu posicionamento, e quando o elemento é um bloco com largura definida ele centraliza horizontalmente.
Para alinhar horizontalmente e verticalmente eu utilizaria esta técnica:

body, .container, html{
  height: 100%;
}

.container{
  display: flex;
  align-items: center;
  justify-content: center;
}

.quadrado{
  background-color: red;
  display: inline-block;
  height: 100px;
  width: 100px;
}

Obs. Quando você defini margin: auto você esta atribuindo uma margin automática para top right bottom left
Answer: div{
    position: absolute;
    left: 50%;
    margin-left: -50px;
    width: 100px;
    height: 100px;
    background: red;
}

Este trecho de código deve resolver o seu problema.
Answer: Olá, eu estava com essa dúvida também e coloquei na div com absolute o left e o right com -50%
div{
    position: absolute;
    top: 0;
    left: -50%;
    right: -50%;
    width: 100px;
    height: 100px;
    background: red;
}

No meu caso, não coloquei o bottom: 0;, mas pode ser que no seu caso seja necessário.
Comment: Com esse código a div nem aparece na tela, vc testou isso onde? https://jsfiddle.net/twhxf5k4/ não funciona


JFrame não atualiza até função do botão terminar

Question: Quando eu clico no botão ele chama um método, e até esse método terminar a JFrame não atualiza
o programa está na imagem abaixo

private void jButtonBuscarActionPerformed(java.awt.event.ActionEvent evt) {
     jTextArea.setText("");
     AtualizeStats("Buscando Dispositivos");
     if (bioSignals.SearchDevice())
     {
         jButtonIniciar.setEnabled(true);
     }
 }

o método que realmente demora é o que está dentro da condição if, mas o que vem antes basicamente deveria aparecer instantaneamente, até porque o método AtualizeStats("Buscando Dispositivos") só muda o texto da caixa preta e printa no console:
public void AtualizeStats(String stats)
{
    System.out.println(stats);
    jTextArea.append(stats + "\n");
}

o que acontece é que no console aparece "Buscando Dispositivos" instantaneamente mas a JFrame espera até tudo o que está no listener do botão ser executado
Comment: E tá certo, tudo isso roda na mesma _thread_... Deve ter alguma coisa pra forçar a atualização da UI. Se esse tipo de coisa não for recorrente, dá pra usar algo assim.
Comment: Jovem, depois de `AtualizeStats("Buscando Dispositivos");` tente fazer `jTextArea.revalidate()` ou `repaint()`
Comment: nenhum dos dois funcionou, continua do mesmo jeito
Comment: o que eu acho mais estranho é que até o estado do botão fica travado, ele permanece clicado até terminar de executar
Comment: No site há exemplos de utilização do swingworker [aqui](https://pt.stackoverflow.com/questions/tagged/swingworker),
Answer: Para poder executar tarefas sem travar a interface gráfica, você precisa utilizar a classe SwingWorker, pois ela foi feita para paralelizar tarefas demoradas da EDT, que é a thread que gerencia o funcionamento da interface gráfica no swing.
Dentro dessa classe, você irá delegar a tarefa no método doInBackground(), que roda num pool separado, de forma que a tela continua funcionando, enquanto a tarefa executa. É possível também exibir progresso com as classes publish() e process(), e exibir algum aviso de termino através da classe done().
Não vou entrar nos detalhes de cada método pois não é o objetivo da resposta, mas deixarei alguns links para estudo no final. Segue um exemplo de uso do SwingWorker, semelhante ao que você está fazendo, há comentários em trechos relevantes:
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import javax.swing.*;
import javax.swing.border.EmptyBorder;
import java.util.List;

public class SwingWorkeSimpleExample extends JFrame {

    private static final long serialVersionUID = 1L;
    private JPanel contentPane;
    private JPanel northPanel;
    private JButton btnSearch;
    private JScrollPane scrollPane;
    private JTextArea textArea;
    private JProgressBar progressBar;

    public static void main(String[] args) {
        EventQueue.invokeLater(SwingWorkeSimpleExample::new);
    }

    public SwingWorkeSimpleExample() {
        setTitle("SwingWorker Test");
        initComponents();
        pack();
        setLocationRelativeTo(null);
        setVisible(true);
    }

    private void initComponents() {

        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        setPreferredSize(new Dimension(350, 300));
        this.contentPane = new JPanel();
        this.contentPane.setBorder(new EmptyBorder(5, 5, 5, 5));
        this.contentPane.setLayout(new BorderLayout(0, 0));
        setContentPane(this.contentPane);

        this.northPanel = new JPanel(new FlowLayout(FlowLayout.RIGHT, 5, 5));
        this.contentPane.add(this.northPanel, BorderLayout.NORTH);

        this.btnSearch = new JButton("Search...");
        this.btnSearch.addActionListener(new ActionWorker());
        this.northPanel.add(this.btnSearch);

        this.textArea = new JTextArea();
        this.textArea.setBackground(Color.BLACK);
        this.textArea.setForeground(Color.WHITE);

        this.scrollPane = new JScrollPane(this.textArea);
        this.scrollPane.setBorder(new EmptyBorder(2, 0, 2, 0));
        this.contentPane.add(this.scrollPane, BorderLayout.CENTER);

        this.progressBar = new JProgressBar(0, 10);
        this.contentPane.add(this.progressBar, BorderLayout.SOUTH);
    }

    private void textAreaAppend(String str) {
        this.textArea.append(str + System.lineSeparator());
    }

    class ActionWorker implements ActionListener {

        @Override
        public void actionPerformed(ActionEvent e) {

            SwingWorker<Void, Integer> worker = new SwingWorker<Void, Integer>() {

                @Override
                protected Void doInBackground() throws Exception {
                    //aqui tudo ocorrerá em paralelo a gui
                    //desativo o botão, adiciono uma mensagem inicial
                    btnSearch.setEnabled(false);
                    textAreaAppend("Searching...");
                    //laço que alimenta a barra de progresso apenas para exemplo
                    for (int i = 0; i < progressBar.getMaximum(); i++) {

                        if (i > 0 && i % 4 == 0)
                            textAreaAppend("Wait...");
                        Thread.sleep(500);
                        //esse metodo "publica" atualizações da tarefa
                        //para a que a barra de progresso seja alimentada
                        //conforme o incremento do laço
                        publish(i);
                    }

                    textAreaAppend("Done!");
                    return null;
                }

                @Override
                protected void process(List<Integer> chunks) {
                    //este métodor recebe o que foi "publicado"
                    // no doInBackground para que possamos atualizar
                    //o progresso na tela
                    progressBar.setValue(chunks.stream().reduce(Integer::sum).orElse(0));
                }

                @Override
                protected void done() {
                    //entrará aqui quando terminar a tarefa, ou
                    //ocorrer alguma exceção
                    progressBar.setValue(0);
                    btnSearch.setEnabled(true);
                }
            };
            worker.execute();
        }
    }
}

Executando:

Worker Threads and SwingWorker
https://pt.stackoverflow.com/questions/tagged/swingworker
Comment: Muito obrigado, funcionou muito bem


New installation showing HTML source instead of rendered HTML

Question: I just installed drupal 8.8.1 using Composer.
When starting installation it displays HTML source in browser window instead of rendering the HTML of the installation script. The code below is what is displayed:
<!DOCTYPE html>

    <meta charset="utf-8" />
<meta name="Generator" content="Drupal 8 (https://www.drupal.org)" />
<meta name="MobileOptimized" content="width" />
<meta name="HandheldFriendly" content="true" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="robots" content="noindex, nofollow" />
<link rel="shortcut icon" href="/core/misc/favicon.ico" type="image/vnd.microsoft.icon" />

    <title>Choose language | Drupal</title>
    <link rel="stylesheet" media="all" href="/core/assets/vendor/normalize-css/normalize.css?0" />
<link rel="stylesheet" media="all" href="/core/misc/normalize-fixes.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/ajax-progress.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/align.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/autocomplete-loading.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/fieldgroup.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/container-inline.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/clearfix.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/details.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/hidden.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/item-list.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/js.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/nowrap.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/position-container.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/progress.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/reset-appearance.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/resize.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/sticky-header.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/tabledrag.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/tablesort.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/components/tree-child.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/system.admin.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/stable/css/system/system.maintenance.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/base/elements.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/base/typography.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/base/print.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/layout/layout.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/action-links.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/breadcrumb.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/button.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/collapse-processed.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/container-inline.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/exposed-filters.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/field.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/form.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/icons.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/inline-form.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/item-list.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/link.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/links.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/menu.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/more-link.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/pager.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/tabledrag.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/tableselect.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/tablesort.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/tabs.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/textarea.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/ui-dialog.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/classy/css/components/messages.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/admin-list.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/content-header.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/container-inline.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/container-inline.module.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/breadcrumb.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/buttons.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/details.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/messages.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/dropbutton.component.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/entity-meta.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/field-ui.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/form.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/help.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/menus-and-lists.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/modules-page.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/node.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/page-title.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/pager.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/panel.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/skip-link.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/tables.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/search-admin-settings.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/tablesort-indicator.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/system-status-report-general-info.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/system-status-report.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/tabs.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/system-status-report-counters.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/system-status-counter.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/components/views-ui.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/theme/colors.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/theme/maintenance-page.css?0" />
<link rel="stylesheet" media="all" href="/core/themes/seven/css/theme/install-page.css?0" />

      Skip to main content

        Drupal
                  8.8.1

Installation tasks

    Choose language
     (active)

    Choose profile

    Verify requirements

    Set up database

    Install site

    Configure site

Choose language

    <form class="install-select-language-form" data-drupal-selector="install-select-language-form" action="/core/install.php" method="post" id="install-select-language-form" accept-charset="UTF-8">

      Choose language
        <select data-drupal-selector="edit-langcode" id="edit-langcode" name="langcode" class="form-select"><option value="af">Afrikaans</option><option value="ast">Asturianu</option><option value="az">AzÉ™rbaycanca</option><option value="id">Bahasa Indonesia</option><option value="jv">Basa Java</option><option value="bs">Bosanski</option><option value="ca">Català </option><option value="cy">Cymraeg</option><option value="da">Dansk</option><option value="de">Deutsch</option><option value="et">Eesti</option><option value="en" selected="selected">English</option><option value="es">Español</option><option value="eo">Esperanto</option><option value="eu">Euskera</option><option value="fil">Filipino</option><option value="fr">Français</option><option value="fy">Frysk</option><option value="fo">Føroyskt</option><option value="ga">Gaeilge</option><option value="gl">Galego</option><option value="gd">Gà idhlig</option><option value="hr">Hrvatski</option><option value="it">Italiano</option><option value="sw">Kiswahili</option><option value="ht">Kreyòl ayisyen</option><option value="ku">Kurdî</option><option value="lv">LatvieÅ¡u</option><option value="lt">Lietuvių</option><option value="xx-lolspeak">Lolspeak</option><option value="hu">Magyar</option><option value="mg">Malagasy</option><option value="nl">Nederlands</option><option value="nb">Norsk, bokmÃ¥l</option><option value="nn">Norsk, nynorsk</option><option value="oc">Occitan</option><option value="pl">Polski</option><option value="pt-br">Português, Brasil</option><option value="pt-pt">Português, Portugal</option><option value="ro">Română</option><option value="gsw-berne">Schwyzerdütsch</option><option value="sco">Scots</option><option value="sq">Shqip</option><option value="en-x-simple">Simple English</option><option value="sk">SlovenÄina</option><option value="sl">SlovenÅ¡Äina</option><option value="fi">Suomi</option><option value="sv">Svenska</option><option value="se">Sámi</option><option value="vi">Tiếng Việt</option><option value="tr">Türkçe</option><option value="is">Ãslenska</option><option value="cs">ÄŒeÅ¡tina</option><option value="el">Ελληνικά</option><option value="be">БеларуÑкаÑ</option><option value="bg">БългарÑки</option><option value="ky">Кыргызча</option><option value="mk">МакедонÑки</option><option value="ru">РуÑÑкий</option><option value="sr">СрпÑки</option><option value="tyv">Тыва дыл</option><option value="uk">УкраїнÑька</option><option value="mn">монгол</option><option value="kk">Қазақ</option><option value="hy">Õ€Õ¡ÕµÕ¥Ö€Õ¥Õ¶</option><option value="he">עברית</option><option value="ug">ئۇيغۇرچە</option><option value="ur">اردو</option><option value="ar">العربية</option><option value="ms">بهاس ملايو</option><option value="fa">Ùارسی</option><option value="ne">नेपाली</option><option value="mr">मराठी</option><option value="hi">हिनà¥à¤¦à¥€</option><option value="bn">বাংলা</option><option value="pa">ਪੰਜਾਬੀ</option><option value="gu">ગà«àªœàª°àª¾àª¤à«€</option><option value="ta">தமிழà¯</option><option value="ta-lk">தமிழà¯, இலஙà¯à®•à¯ˆ</option><option value="te">తెలà±à°—à±</option><option value="kn">ಕನà³à²¨à²¡</option><option value="ml">മലയാളം</option><option value="si">සිංහල</option><option value="th">ภาษาไทย</option><option value="lo">ພາສາລາວ</option><option value="bo">བོད་སà¾à½'་</option><option value="dz">རྫོང་à½</option><option value="my">ဗမာစကား</option><option value="ka">ქáƒáƒ თული ენáƒ</option><option value="am">አማርኛ</option><option value="km">ភាសាážáŸ'មែរ</option><option value="ja">日本語</option><option value="zh-hans">简体中文</option><option value="zh-hant">ç¹é«"中文</option><option value="ko">한국어</option></select>

Translations will be downloaded from the Drupal Translation website. If you do not want this, select English.

<input autocomplete="off" data-drupal-selector="form-i-o33hif7d1m-uxztfnjrb8op-dmeqk-qza61jsd-mm" type="hidden" name="form_build_id" value="form-i_o33hif7D1m_uxztfNjRB8op-DMeqK-qZA61Jsd-mM" />
<input data-drupal-selector="edit-install-select-language-form" type="hidden" name="form_id" value="install_select_language_form" />

<input data-drupal-selector="edit-submit" type="submit" id="edit-submit" name="op" value="Save and continue" class="button button--primary js-form-submit form-submit" />

</form>

I rightly assumed that the server isn't sending Content-Type header. I open ../core/install.php and add:
header("Content-Type: text/html");

to install.php and all starts working as expected.
Is this a bug in this version of Drupal setup or am I missing something in this server's setup? Not a huge deal but I'd like to fix things properly.
Here is what the server is sending out (which includes pertinent info about environment)
Cache-Control: must-revalidate, no-cache, private
Connection: close
Content-Type: text/html;charset=UTF-8
Date: Fri, 27 Dec 2019 16:17:01 GMT
ETag: "1577463421"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Fri, 27 Dec 2019 16:17:01 +0000
Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips PHP/7.1.27
X-Content-Type-Options: nosniff
X-Powered-By: PHP/7.1.27

Comment: updated with info about environment
Comment: Probably server. But to say with more certainty we need more info. Like how did you install, on what server, with what PHP and so on.
Comment: Is Drupal's .htaccess in the webroot?
Answer: It looks like you are using Apache webserver on Red Hat. The advice on fixing clean urls on Drupal 8 will probably solve this problem.
First, check to confirm there is a .htaccess file in your document root.
After that, there are 2 solutions to get those rules to be followed.
One solution is to enable Apache to follow the .htaccess rules that Drupal provides. Quoting from that article:

You will need to have the Allow Override directive set in your Apache configuration file (this will allow local .htaccess overrides on your site):
AllowOverride All
AccessFileName .htaccess

The other solution is simply to copy the configuration from the .htaccess into your webserver's configuration.
Be sure to restart the apache server after changing the configuration. There are many more bits of advice in this document that are worth reading.
Comment: It should be noted that
AccessFileName .htaccess
Needs to go in the  configuration and
AllowOverride All
goes in the  config

Simply pasting the code in the answer in httpd.conf or in any section will break apache.

This solution also doesn't work  to solve this problem....
Comment: Good point, the excerpt alone is not enough to solve the problem. That document is very long and I pasted one piece of it. All of it is relevant.
Answer: Adding the following to .htaccess in ../web solved issue:
<IfModule mod_headers.c>
  <Files ~ "\.(php)$">
    Header set Content-Type "text/html; charset=utf-8"
  </Files>
</IfModule>

As long as you explicitly set Content-Type headers when your php is outputting files (such as .pdf, .csv etc) which you should be doing anyway, this won't break your code.


Poor model training- what next?

Question: I am trying to train models to predict if somebody will get breast cancer- it is a binary classification problem, using limited features that replicate data a patient's primary care physician will have on them.
I have the following steps:

Fill in missing values - 'mean' for continuous columns, '10' for discrete columns (discrete columns go up to 0,1,2,3,4,5,6)
Feature selection (correlation based, whole dataset)
Test train split (20/80)
SMOTENC to balance classes (previously was 4000 vs 50,000) on train data only
One hot encode the discrete columns in test and train
Standard scale in test and train
Then --> 10 fold cross validation to determine best hyperparameters for KNN (1-30), and Logistic Regression Classifier (['solver'] ,['penalty'] ,['C'] )
I am getting decent AUROC on the train data (0.7 on LR, 0.85 on KNN) but it seems to have completely overfit, as I am getting 0.5 AUROC on test data.

I believe the probelem is SMOTE.I have very class imbalanced data (7:93), and using SMOTE seems to make it overfit. Without SMOTE, they are just poor models and I get about 52% AUROC on test and train.
I am at a loss as to what I can do next or change about what i've done. Any suggestions, pages to read, or advice in general would be greatly appreciated. I am relatively new to ML and this is for a large university project, and I cannot work out whether it is my data that is the problem- that it just won't train perhaps due to the minimal features I have, or something I am doing.
I am currently training a Random Forest model as I have read that deals with imbalanced data better, and also plan to have a look into XGBoost.
Comment: Welcome to Cross Validated! [Are you sure that your class imbalance is problematic?](https://stats.stackexchange.com/questions/357466)
Answer:
Imputation should be used with care. If you fill in the mean value for any missing one, you simulate knowledge you simply don't have. Consider simply keeping a NA value if your model can deal with it, or try deleting all cases with missing values and see how your model performs then.
Select your features only based on the training data. Doing selection first and splitting later leaks data.
As Dave writes, class imbalance is usually not a problem, and SMOTE will not solve a non-problem.
Doesn't your software do the one-hot encoding by itself?
Remember to scale based only on the training data, and then applying this scaling separately to the test data. Scaling all data at once again leaks data.
kNN is a hard classifier. I would strongly recommend using a probabilistic classifier instead, especially for rare targets - see here.

Finally, you may be interested in How to know that your machine learning problem is hopeless?

Poor model training- what next?


Linear regression : from parameters with standard error to prediction interval

Question: I have the slope and intercept of my model as well as their respective standard error but not the data from which it was estimated (I retrieved it from the literature). I would like to plot the predictions a*x+b with the prediction interval around the curve, to know if my own data falls into it. Do I have enough information to do this, and if yes, how (in R or python)?
Comment: No, you don't have enough information. You are at least missing the covariance between the parameter estimates. Also, your data is not expected to fall into the confidence band. Maybe you are confusing that with the prediction band.
Comment: If you  want to see if your *data* lie within an interval, it's not a confidence interval you seek. A confidence interval is an interval for the population mean, not a sample. If the data is not from the same sample as was used to generate the fit you want a prediction interval.
Comment: @alpagarou Yes, you could use the bootstrap method, but you'd need the original data for bootstrapping. You could also assume two independent normal distributions for intercept and slope using the estimates and standard errors, but such an assumption of independence is not justified. And you need the covariance to simulate from a multivariate normal distribution.
Comment: Is this something you want? https://tomholderness.wordpress.com/2013/01/10/confidence_intervals/
Comment: Note also that requests for code are generally regarded as off-topic here (not that you're responsible for it ending up on crossvalidated). The remainder of your question is on topic however.
Comment: Oh indeed I think I need a prediction interval, sorry. But I don't have the covariance between the parameters estimates ...
Comment: Isn't there some kind of bootstrap method that could achieve prediction interval by drawing pairs of slope/intercept ...?
Answer: As @Roland points out, you also need the covariance of the parameters. If you have those, for example, you have a used curve_fit in python...
from pylab import *
from scipy.optimize import curve_fit

# fake data
x = arange(0, 30)
y = x + 5*np.random.normal(size=len(x))

# fit
f = lambda x, *p: polyval(p, x)
p, cov = curve_fit(f, x, y, [1, 1])

... you could assume that the fit (p, cov) represents a normal distribution and sample from it, selecting the lower and upper 2.5% quantiles:
# simulated draws from the probability density function of the regression
xi = linspace(np.min(x), np.max(x), 100)
ps = np.random.multivariate_normal(p, cov, 10000)
ysample = np.asarray([f(xi, *pi) for pi in ps])
lower = percentile(ysample, 2.5, axis=0)
upper = percentile(ysample, 97.5, axis=0)

# regression estimate line
y_fit = poly1d(p)(xi)

# plot
plot(x, y, 'bo')
plot(xi, y_fit, 'r-')
plot(xi, lower, 'b--')
plot(xi, upper, 'b--')

Again as @Roland points out, the data doesn't, in general, fall between the confidence intervals of the fit. Also, to see that the covariances are important, you can diagonalize the covariance matrix with cov = np.diag(np.diag(cov)) and re-draw.
There's a more analytical solution specific to the linear fit here (with very slightly different results for some mathy reason).


Google assistant on Raspberry pi B3 won't start

Question: I have tried building the Google Assistant on my Raspberry pi model b3 following this walkthrough
https://developers.google.com/assistant/sdk/prototype/getting-started-pi-python/run-sample
Everything went well until I tried starting it with
google-assistant-demo

and I get the following stacktrace
    Traceback (most recent call last):
  File "/home/pi/env/bin/google-assistant-demo", line 11, in <module>
    sys.exit(main())
  File "/home/pi/env/local/lib/python2.7/site-packages/google/assistant/__main__.py", line 66, in main
    with Assistant(credentials) as assistant:
  File "/home/pi/env/local/lib/python2.7/site-packages/google/assistant/library/assistant.py", line 86, in __init__
    self._event_queue = IterableEventQueue()
  File "/home/pi/env/local/lib/python2.7/site-packages/google/assistant/library/event.py", line 172, in __init__
    super(IterableEventQueue, self).__init__(maxsize=32)
TypeError: must be type, not classobj

I don't even know where to start and have not tried any solutions. What could be causing the exception?
Comment: You didn't configure the venv properly and now it's trying to run on Python 2 instead of 3.
Answer: As pointed in the official documentation:
The Google Assistant library currently require Python > 3.4.
You can configure a virtual environments using the following commands:
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools
source env/bin/activate

Comment: While this did not exactly solve my problem it pointed me in the right direction. The call to 'google-assistant-demo' did invoke python 2 methods. I went through the setup from beginning to end and in each place where a 'python' command was used I replaced it with 'python3' thanks for your help!


jQuery multiple get() from database to array - good idea?

Question: I'm trying to make a "slideshow" viewer for a number of books with chapters in them. The "slides" are chunks of HTML data stored in a database. A chapter can consist of 50-100 slides, but they are quite small (1-2 kB each and pictures in some). I want the user to have buttons to click for the previous and next slide and minimise loading times.
What I have now is a PHP page called ContentFetcher.php that only queries the database to get the contents of one slide (section).
<?php
require_once 'Database.php';
echo Database::getInstance()->getSectionText($_GET["bookID"], $_GET["chapter"], $_GET["section"]);
?>

My Javascript runs a loop and fetches each section with $.get() for the current chapter:
        $(function() {
            <? $db = Database::getInstance(); ?>

            var chapterSize = <?= $db->getChapterSize($this->bookID, $this->chapter) ?>;
            var chapterContent = []; // this is where I want to store the contents
            var currentSection = <?=$this->section;?>;

            for (var i = 1; i <= chapterSize; i++) {
                $.get("ContentFetcher.php", {
                    bookID  : <?= $this->bookID  ?>,
                    chapter : <?= $this->chapter ?>,
                    section : i
                }, function(data) {
                 chapterContent.push(data);
             });
           }

        $("#slides").html(chapterContent[currentSection]); // show the current slide

     });

So is this a good idea performance-wise? If so, my understanding of JS closures isn't that great yet - how can I access the same chapterContent inside the closure?
Answer: This idea means you'll do between 50 and 100 HTTP requests to load one chapter ; and those will all be done almost at the same time.
So, each time someone tries to load a chapter, your webserver will get 100 requests in a couple of seconds -- might not be such a good idea.

In such a situation, I would probably try to implement something like this :

When the user opens the chapter, do a single HTTP request, which will return the first 10 slides
When the user arrives on slide 8, do a single HTTP request, to load the slides 11 to 20
When the user arrives on slide 18, do a single HTTP request, to load the slides 21 to 30
And so on

Of course :

If a user arrives on slide 56 by typing its URL (if that's possible) or using some "goto slide X" feature, you must do an HTTP request to load slides 51 to 60.
And it means users might have to wait for a little bit if they go from slide to slide too quick.

It will make both your client and server-side code a bit more complex, but it's probably one of the best solution :

You won't load too much data at a time,
You won't pre-load too much data,
You you won't do too many HTTP requests,
And those will be done a bit in advance, so the user doesn't have to wait to see new slides arrive.
Comment: I second that. The HTTP requests would create a huge overhead (for both server and client). If one page is really about 1-2 kB you could as well load a whole chapter in one go (and the pictures dynamically as you approach a particular page).
Answer: No, this is definitely not a good idea. I'd recommend you just do one $.get() request to receive all the records in a JSON array.
Regarding closures, you must watch the excellent Douglas Crockfords' videos "The JavaScript Programming Language" (4 parts). Yes, the closures can be used this way, they are just great.


Strings in python 3.7

Question: How to count sub-strings in a string?
Example: findSubstrings("foxcatfox","fox") # should return 2
Comment: This isn't really a problem one would typically use recursion for. Recursion is great for things with depth (so graphs/trees/etc) but here you just have a string so a loop would be more suitable.
Comment: what about [str.count()](https://docs.python.org/3/library/stdtypes.html#str.count)? like `"foxcatfox".count("fox")` Why the recursion? And do you want to count overlapping hits or not?
Answer: If recursion is really a must, you can try dividing the problem first.
Say if you found a matching substring at position i, then the total number of substring is 1 + findSub(string[i+1:], sub), so you can write something like this:
def findSubstringsRecursive(string, substring):
    counter = 0
    substringLength = len(substring)
    for i in range(len(string)):
        if string[i] == substring[0]:
            end = i + substringLength
            sub1 = string[i:end]
            if substring == sub1:
                return 1 + findSubstringsRecursive(string[i+1:], substring)
    return 0

Comment: This is not a purely recursive approach though. The matching substring is found in an iterative manner. The recursion is then kind of forced in :)
Comment: @schwobaseggl but iteration can almost always be rearranged into recursion, no?
Comment: Yeah, the point of such an obviously academic exercise most often is to see the aesthetic and implementational simplicity of recursive solutions. I think that point is kind of lost in this mixture. The lack of simplicity can be seen as you did not even get rid of the obsolete `counter` ;)
Answer: The following pure recursive approach is simple enough (apart from the bool->int coercion):
def findRec(s, pat):
    if len(s) < len(pat):  # base case should be obvious
        return 0
    return (pat == s[:len(pat)]) + findRec(s[1:], pat)  # recurse with smaller size

>>> findSubstrings('foxcatfox', 'fox')
2
>>> findSubstrings('foxcatfox', 'foxc')
1
>>> findSubstrings('foxcat', 'dog')
0

I should note that this counts overlapping occurrences which may or may not be desired. One might also add protection against or define behaviour for an empty substring.
Comment: This is going to exceed max depth (default 1000) if the string itself is too long.
Comment: @iBug Which is the case for the recursive approach to any problem if the problem size is too big...
Comment: OP's code looks for overlapping occurrences, so yours should be fine to go with.

Strings in python 3.7


Pygame Rect not appearing on display

Question: I know this must have been answered a number of times, but unfortunately, the topic is difficult to search and I am truly at a los, as Python is not throwing an error.
I am following a Python Crash Course tutorial (Eric Matthes) and am programming a game along the lines of space invaders. The below Module is intended to control the bullets:
import pygame
from pygame.sprite import Sprite

class Bullet(Sprite):
"""A class to manage bullets fired from the ship"""

def __init__(self, ai_settings, screen, ship):
    """Create bullet at the ships current positio"""
    super().__init__()
    self.screen = screen

    # Create a bullet rect at (0, 0) and then set the correct position.
    self.rect = pygame.Rect(0, 0, ai_settings.bullet_width,
        ai_settings.bullet_height)
    self.rect.centerx = ship.rect.centerx
    self.rect.top = ship.rect.top

    # Store the bullet's position as a decimal value.
    self.y = float(self.rect.y)

    self.color = ai_settings.bullet_color
    self.speed_factor = ai_settings.bullet_speed_factor

def update(self):
    """Move the bullet up the screen."""
    # Update the decimal position of the bullet.
    self.y -= self.speed_factor
    # Update the rect position.
    self.rect.y = self.y

def draw_bullet(self):
    """Draw the bullet on the screen."""
    pygame.draw.rect(self.screen, self.color, self.rect)

The actual game screen is printing the bullet count, so I know the bullets are on the screen and also vanish again, as the move towards the edge of the screen, but they are not displayed.
The game itself looks as follows:
import sys

import pygame
from pygame.sprite import Group
from settings import Settings
from ship import Ship
import game_functions as gf

def run_game():
    # Initialize game and create a screen object.
    pygame.init()
    ai_settings = Settings()
    screen = pygame.display.set_mode((ai_settings.screen_width, ai_settings.screen_height))
    pygame.display.set_caption("Alien Invasion")

    # Make a ship.
    ship = Ship(ai_settings, screen)

    # Make a group to store the bullets in.
    bullets = Group()

    # Start main loop for the game

   while True:
   #Watch for keyboard and mouse events.
       gf.check_events(ai_settings, screen, ship, bullets)
       ship.update()
       gf.update_bullets(bullets)

       gf.update_screen(ai_settings, screen, ship, bullets)

       # Redraw Screen during each pass through.
       screen.fill(ai_settings.bg_color)
       ship.blitme()

       # Make most recently drawn visible
       pygame.display.flip()

run_game()

The settings are there and no, the bullets are not the same color as the screen;-)
Can anyone help me find my error in thinking?
My impression is, the pygame.draw.rect function should make them display in combination with the gf.update_bullets(bullets) call.
Many thanks and best regards,
Simon
Additional File added:
game_functions.py
import sys

import pygame
from bullet import Bullet

def check_keydown_events(event, ai_settings, screen, ship, bullets):
    """Respond to keypresses"""
    if event.key == pygame.K_RIGHT:
        # Move the ship to the right.
        ship.moving_right = True
    elif event.key == pygame.K_LEFT:
        ship.moving_left = True
    elif event.key == pygame.K_SPACE:
        fire_bullet(ai_settings, screen, ship, bullets)

def fire_bullet(ai_settings, screen, ship, bullets):
    # Create a new bullet and add it to the bullets group.
    if len(bullets) < ai_settings.bullets_allowed:
        new_bullet = Bullet(ai_settings, screen, ship)
        bullets.add(new_bullet)

def check_keyup_events(event, ship):
    """Respond to keypresses"""
    if event.key == pygame.K_RIGHT:
        # Move the ship to the right.
        ship.moving_right = False
    elif event.key == pygame.K_LEFT:
        ship.moving_left = False

def check_events(ai_settings, screen, ship, bullets):
    """Respond to keypresses and mouse events."""
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            sys.exit()

        if event.type == pygame.KEYDOWN:
            check_keydown_events(event, ai_settings, screen, ship, bullets)
        elif event.type == pygame.KEYUP:
            check_keyup_events(event, ship)

def update_screen(ai_settings, screen, ship, bullets):

    # Redraw all bullets behind the ship and aliens
    for bullet in bullets.sprites():
        bullet.draw_bullet()

def update_bullets(bullets):
    """Update position of bullets and get rid of old bullets"""
    # Update bullet position
    bullets.update()

    # Get rid of bullets that have disappeared.
    for bullet in bullets.copy():
        if bullet.rect.bottom <= 0:
            bullets.remove(bullet)
    print(len(bullets))

    """Update images on the screen and flip to the new screen."""

Comment: I can't see any call to the draw_bullet() method.  Is that done in ship.blitme()?
Comment: Hi Alkanen, many thanks for your quick feedback.
this is actually in another "game_functions" file - uploading here as well:
Comment: I see @sloth gave you the answer I was fishing for.  I didn't see your draw, so I wasn't sure you actually performed a draw in update_screen() or if that only moved things on the screen to be drawn later =)
Answer: Check the order in which you draw things to the screen surface.
You call gf.update_screen(ai_settings, screen, ship, bullets), and after that, you actually erase the whole screen with screen.fill(ai_settings.bg_color).
Make sure to call screen.fill(ai_settings.bg_color) first, then draw your other stuff.

There are some other issues with your code, e.g.

you use the Sprite class, but your sprites don't have an image attribute, which makes using Sprite kind of pointless
you already use the Group class for your sprites, but draw them manually with a for loop and the draw_bullet function. Just give your bullets an image and call bullets.draw(screen) instead
you check if bullets are outside the screen in update_bullets. The the bullet class handle it itself, by simply using something like this:
if not self.screen.rect.contains(self.rect):
    self.kill
your entire game_functions.py file makes your code hard to read
Comment: Many thanks Sloth, the order of screen.fill and gf.update_screen did the trick. I also appreciate the comment - as it's a tutorial I guess I need to stick to the book for the moment, but I'll try that out once it's all working. Many thanks here.

Pygame Rect not appearing on display


Inserting a record in coulmn name "domain" in access

Question: I am trying to insert a record in a column named as "domain" in MS Access db from excel using VBA. But it gives an error(Automation error) but with the same code if i change the column name as domai or do it works fine.
The code is simple insert statement
code:
'Insert into tblnm(domain) values('anything')'

Answer: Reserved words need to be enclosed in []
insert into tblnm([domain]) values('anything')

http://www.msoffice.us/Access/Access20010Reserved_Words_Special_Characters.htm