Understanding $du \, dv$ or $(du)^2$ being dropped in Itô calculus

Question: I will phrase my question in terms of a concrete example, which involves the proof presented for Feynman-Kac formula, where one is interested in solving the following PDE:
\begin{gather}\frac{\partial u}{\partial t}(x,t) + \mu(x,t) \frac{\partial u}{\partial x}(x,t) + \tfrac{1}{2} \sigma^2(x,t) \frac{\partial^2 u}{\partial x^2}(x,t) -V(x,t) u(x,t) + f(x,t) = 0, \end{gather}
defined for all $x \in \mathbb{R}$ and $t \in [0, T]$, subject to the terminal condition
\begin{gather}
u(x,T)=\psi(x),
\end{gather}
where $\mu, \sigma, \psi, V, f$ are known functions, $T$ is a parameter and $ u:\mathbb{R}\times[0,T]\to\mathbb{R}$ is the unknown. The proof about the form of the solution starts as follows:

Let $u(x, t)$ be the solution to the above partial differential
equation.  Applying the product rule for Ito's processes to the
process:
\begin{gather} Y(s) = e^{-\int_t^s V(X_\tau,\tau)\, d\tau} u(X_s,s)+
> \int_t^s e^{-\int_t^r V(X_\tau,\tau)\, d\tau}f(X_r,r) \,
> dr\end{gather}
one gets
\begin{align} dY = {} & d\left(e^{-  \int_t^s V(X_\tau,\tau)\,
> d\tau}\right) u(X_s,s) + e^{-  \int_t^s V(X_\tau,\tau)\,
> d\tau}\,du(X_s,s) \\[6pt] & {} + d\left(e^{-  \int_t^s
> V(X_\tau,\tau)\, d\tau}\right)du(X_s,s) + d\left(\int_t^s e^{-
> \int_t^r V(X_\tau,\tau)\, d\tau}  f(X_r,r) \, dr\right) \end{align}
Since :\begin{gather}d\left(e^{-  \int_t^s V(X_\tau,\tau)\,
> d\tau}\right) =-V(X_s,s) e^{-  \int_t^s V(X_\tau,\tau)\, d\tau}
> \,ds,\end{gather}  the third term is $O(dt \, du)$ and can be
dropped.

My question is mathematically (aka rigorously) what does it mean to just drop $O(dt \, du)$?? I know the question has been raised before for this, but it seems OP there was convinced with yet another convergence argument, but my question is about the equality that is not based on asymptotic arguments. I assume in a rigorous sense $dt$ is a differential form. Can somebody in that language explain why such a cancellation takes place?
Comment: It's unfortunate that you are not getting answers. Try adding one tag such as "stochastic processes", maybe removing "derivatives" or "differential geometry". You want to attract the attention of people with a good grasp of Ito's formula, rather than calculus or PDE. As far as I understand, your question is a standard one in the theory of stochastic processes, and I even vaguely remember it being answered in the lecture notes on SDE of Craig Evans. I think he made a book out of them.
Comment: @GiuseppeNegro thank you so much for the constructive feedback and extensive comment. maybe I'll look into the course material of Evans. unfortunately i stopped studying math heavily after getting my bachelor in it. so your comment is especially helpful. hopefully the current formulation helps it getting attention
Answer: First, it's important to clarify what those differentials mean. If you are given the SDE $$dX_t = \mu(t,X_t) dt + \sigma(t,X_t) dB_t$$
where $B$ is a Brownian motion, then this is just shorthand for $$X_t - X_0 = \int_0^t \mu(s,X_s) ds + \int_0^t\sigma(s,X_s) dB_s$$
In this integral equation, the first integral is a pathwise Riemann integral and the second integral is an Itô integral. In particular, the differentials in this context have no intrinsic meaning.
Second: the reason why you can treat $du \, dv = (du)^2 = 0$ and $(dW_t)^2 = dt$ is because these formal products make Itô's formula work. In the proof of Itô's formula, these play the role of covariation, so that in particular:
$$\int_0^t (du)^2 = \lim_{n\to \infty} \sum_{i \in \pi^n} (u_i- u_{i-1})^2 \leq \lim_{n\to \infty} \sup_{i \in \pi^n} |u_i - u_{i-1}|  \sum_{i \in \pi^n} |u_i- u_{i-1}| = 0$$
And,
$$\int_0^t(dW_s)^2 = \lim_{n\to \infty} \sum_{i \in \pi^n} (W_{t_i} - W_{t_{i-1}})^2 = t = \int_0^t ds$$
Where $\pi = (\pi^n)$ is a refining partition whose mesh vanishes as $n\to\infty$.


Proving continuity of $x^2$

Question: I need to prove the continuity for:
$$f:\mathbb{R}\to\mathbb{R}, x\mapsto x^2$$
What I've got so far:
$$|x-x_0|<\delta \Rightarrow |x^2-x_0^2|<\epsilon$$
$$|x^2-x_0^2|=|x+x_0||x-x_0|<|x+x_0|\delta=|x-x_0+2x_0|\delta\leq(|x-x_0|+|2x_0|)\delta=(\delta+|2x_0|)\delta$$
but how do I continue, I am stuck here.
Thanks in advance.
Comment: Let $\delta = \min (\frac {\epsilon}{|2x_0| + 1},1)$
Comment: In this case using "min" is not really necessary.  But it is a common enough technique that it is wroth knowing.  You need to show that a $\delta$ exists for a given $\epsilon.$   Rarely is there need to find the largest possible delta.  Since we expect $\delta$ to be small, sometimes it is easier if we establish an upper bound.  If we say that $\delta \le 1$ then  $\delta^2\le 1$ which will allow us to replace $\delta^2$ with $1$ and the inequality will still be true.
Comment: @DougM I'd appreciate it, if you could elaborate, how the $\delta$ disappears, and where the $1$ comes from, maybe I could understand then why we use the $min$ here
Answer: If you now had $(\delta+2x_0)\delta = \epsilon$, you'd be done. Since you are free to choose $\delta$ based on $x_0$ and $\epsilon$, manipulate this equation so it gives you a choice for $\delta$.
Comment: @user3125470 We can always choose a smaller $\delta$ than we need to, so we can assume $\delta <1$, so $\delta^2 < \delta$. Does this help?
Comment: that's the point I am struggling at, probably because of my lack in inequalities
Comment: Yes, explains everything now. Thanks

Proving continuity of $x^2$


Spring Websocket with jetty broke mvc tests

Question: I include a websocket to my project and then i have error wherever i use @WebAppConfiguration. So I try to find a problem and get that ServletContext is mocked and not instance of ContextHandler.Context
So how can i fix my MVC tests and use websockets in one project?
example of MVC test
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = {WebAppConfig.class})
@WebAppConfiguration
public class PlayerControllerTest {

  private MockMvc mockMvc;

  @Mock
  private PlayerService playerService;

  @InjectMocks
  private PlayerController playerController;

  @Test
  public void getById()
  throws Exception {

    when(playerService.find(0L)).thenReturn(region);

    mockMvc.perform(get("/player/{playerId}", 0L))
        .andExpect(status().isOk())
        .andExpect(content().contentType("application/json;charset=UTF-8"))
        .andExpect(jsonPath("playerName").value("Russia"))
        .andExpect(jsonPath("id").value(0));

    verify(playerService, times(1)).find(0L);
    verifyNoMoreInteractions(playerService);
  }

  @Before
  public void setUp() {
    MockitoAnnotations.initMocks(this);
    this.mockMvc = MockMvcBuilders.standaloneSetup(playerController).build();
  }
}

Websocket config
@Configuration
@EnableWebSocketMessageBroker
public class WebSocketConfig extends AbstractSecurityWebSocketMessageBrokerConfigurer {

  @Autowired
  ServletContext servletContext;

  @Override
  public void registerStompEndpoints(StompEndpointRegistry registry) {
    registry.addEndpoint("/supportChat")
        .setHandshakeHandler(handshakeHandler())
        .setAllowedOrigins("*")
        .withSockJS().setClientLibraryUrl("//cdn.jsdelivr.net/sockjs/1.1.2/sockjs.min.js");
  }

  @Bean
  public DefaultHandshakeHandler handshakeHandler() {

    WebSocketPolicy policy = new WebSocketPolicy(WebSocketBehavior.SERVER);
    policy.setInputBufferSize(8192);
    policy.setIdleTimeout(TimeUnit.DAYS.toMillis(1000));

    servletContext.setAttribute(DecoratedObjectFactory.ATTR, new DecoratedObjectFactory());
    return new DefaultHandshakeHandler(
        new JettyRequestUpgradeStrategy(new WebSocketServerFactory(servletContext, policy)));
  }

  @Override
  public void configureMessageBroker(MessageBrokerRegistry config) {
    config.enableSimpleBroker("/queue", "/topic");
    config.setApplicationDestinationPrefixes("/app");
  }

  @Override
  protected void configureInbound(MessageSecurityMetadataSourceRegistry messages) {
    messages.simpDestMatchers("/app/supportChat/**", "/supportChat/**")
        .PermitAll();
  }

  @Override
  protected boolean sameOriginDisabled() {
    return true;
  }

  @Override
  public void configureClientOutboundChannel(ChannelRegistration registration) {
    registration.taskExecutor().corePoolSize(8);
    registration.setInterceptors(presenceChannelInterceptor());
  }

  /**
   * Use for debug of STOMP over WebSocket.
   * @see PresenceChannelInterceptor
   */
  @Bean
  public PresenceChannelInterceptor presenceChannelInterceptor() {
    return new PresenceChannelInterceptor();
  }

  @Override
  protected void customizeClientInboundChannel(final ChannelRegistration registration) {
    registration.setInterceptors(new SubscribeInterceptor());
  }

  @Override
  public boolean configureMessageConverters(List<MessageConverter> messageConverters) {
    messageConverters.add(new GsonMessageConverter());
    return true;
  }
}

Exception
java.lang.IllegalStateException: Failed to load ApplicationContext

    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
    at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83)
    at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:189)
    at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:131)
    at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:228)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:287)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:289)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:247)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.context.ApplicationContextException: Failed to start bean 'stompWebSocketHandlerMapping'; nested exception is java.lang.IllegalStateException: Unable to start Jetty WebSocketServerFactory
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:176)
    at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
    at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:346)
    at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:149)
    at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:112)
    at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:879)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545)
    at org.springframework.test.context.web.AbstractGenericWebContextLoader.loadContext(AbstractGenericWebContextLoader.java:134)
    at org.springframework.test.context.web.AbstractGenericWebContextLoader.loadContext(AbstractGenericWebContextLoader.java:61)
    at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:108)
    at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:251)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116)
    ... 24 more
   Caused by: java.lang.IllegalStateException: Unable to start Jetty WebSocketServerFactory
    at org.springframework.web.socket.server.jetty.JettyRequestUpgradeStrategy.start(JettyRequestUpgradeStrategy.java:126)
    at org.springframework.web.socket.server.support.AbstractHandshakeHandler.doStart(AbstractHandshakeHandler.java:204)
    at org.springframework.web.socket.server.support.AbstractHandshakeHandler.start(AbstractHandshakeHandler.java:198)
    at org.springframework.web.socket.sockjs.transport.handler.WebSocketTransportHandler.start(WebSocketTransportHandler.java:91)
    at org.springframework.web.socket.sockjs.transport.TransportHandlingSockJsService.start(TransportHandlingSockJsService.java:163)
    at org.springframework.web.socket.sockjs.support.SockJsHttpRequestHandler.start(SockJsHttpRequestHandler.java:103)
    at org.springframework.web.socket.server.support.WebSocketHandlerMapping.start(WebSocketHandlerMapping.java:65)
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
    ... 36 more
Caused by: java.lang.NullPointerException
    at org.eclipse.jetty.websocket.server.WebSocketServerFactory.doStart(WebSocketServerFactory.java:320)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.springframework.web.socket.server.jetty.JettyRequestUpgradeStrategy$ModernJettyWebSocketServerFactoryAdapter.startFactory(JettyRequestUpgradeStrategy.java:303)
    at org.springframework.web.socket.server.jetty.JettyRequestUpgradeStrategy$WebSocketServerFactoryAdapter.start(JettyRequestUpgradeStrategy.java:276)
    at org.springframework.web.socket.server.jetty.JettyRequestUpgradeStrategy.start(JettyRequestUpgradeStrategy.java:123)
    ... 43 more

Comment: Try specifying webenvironment to use a random port. Something like this  .@SpringBootTest(classes = TestApp.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
Answer: I too faced a similar issue
The problem was with the web application context using a mocked port which failed to create required beans for jetty to operate.
For a spring boot application
Try annotating
@SpringBootTest(classes = TestApp.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)

against your test class.
By default web environment uses a mocked port


Usando Broadcast Receiver com DownloadManager

Question: Estou aprendendo a fazer o uso do DownloadManager com BroadcastReceiver para saber quando o download está concluído.
O problema: vamos supor que vou fazer o download de 2 PDFs em uma página, eu registro 2 Receiver, ou seja, um para cada PDF, depois coloco cada um deles na fila de download (veja a chamada dm.enqueue). O que acontece é que para cada download concluído (status == 8) eu recebo 2 vezes essa notificação.
No aplicativo quando a página possui apenas 2 PDFs a fazer download o dm.enqueue(request); é chamado apenas duas vezes como esperado.
Eu pensei: a notificação de download concluído não deveria ser enviada apenas uma vez para cada arquivo?
    DownloadManager.Request request = new DownloadManager.Request(uri);
    request.setDescription(getString(R.string.download_description));
    request.setTitle(getString(R.string.download_title));
    request.allowScanningByMediaScanner();
    request.setNotificationVisibility(DownloadManager.Request
            .VISIBILITY_VISIBLE_NOTIFY_COMPLETED);
    request.setDestinationInExternalPublicDir(Environment
            .DIRECTORY_DOWNLOADS, uri.getLastPathSegment());
    dm = (DownloadManager) getSystemService(Context
            .DOWNLOAD_SERVICE);
    BroadcastReceiver receiver = new BroadcastReceiver() {
        public void onReceive(Context ctxt, Intent intent) {

            Log.d(TAG, "onReceive called");

            // http://stackoverflow.com/a/13322285/3697611
            Bundle extras = intent.getExtras();
            DownloadManager.Query q = new DownloadManager.Query();
            q.setFilterById(extras.getLong(DownloadManager
                    .EXTRA_DOWNLOAD_ID));
            Cursor c = dm.query(q);
            if (c.moveToFirst()) {
                int status = c.getInt(c.getColumnIndex
                        (DownloadManager.COLUMN_STATUS));

                // se estiver completo
                if (status == DownloadManager.STATUS_SUCCESSFUL) {
                    String uri = c.getString(c.getColumnIndex
                            (DownloadManager.COLUMN_URI));
                    String local_uri = c.getString(c.getColumnIndex
                            (DownloadManager.COLUMN_LOCAL_URI));

                    //TODO: bug, estas mensagens estão aparecendo 2
                    // vezes para cada download
                    display(getString(R.string.completed) + uri);
                    display(getString(R.string.saved) + local_uri);

                    try {

                        // quando completar o download devemos
                        // pegar o texto
                        getText(new File(new URI(local_uri)));

                    } catch (URISyntaxException e) {
                        printStackTrace(e);
                    }
                }

            }

        }
    };
    registerReceiver(receiver, new IntentFilter(DownloadManager
            .ACTION_DOWNLOAD_COMPLETE));
    receivers.add(receiver);
    dm.enqueue(request);

O código completo está aqui.
P.S.: esse é só um aplicativo bem simples que fiz para desenvolver meu conhecimento sobre o Android, atualmente ele busca PDFs na página onde publicam o diário oficial do município da cidade onde moro.
Nota: o app foi testado em um Alcatel Pixi 4'6 com Android 5.1.
Depurando
Quando coloco um logd eu vejo as mensagens assim:
09-04 15:10:28.395 30024-30252/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: dm.enqueue register a request
09-04 15:10:28.422 30024-30252/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: dm.enqueue register a request
09-04 15:10:33.288 30024-30024/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: onReceive called
09-04 15:10:33.312 30024-30024/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: onReceive called
09-04 15:10:34.593 30024-30024/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: onReceive called
09-04 15:10:34.609 30024-30024/com.example.android.officialdiary D/com.example.android.officialdiary.MainActivity: download: onReceive called

Percebemos que foram registrados dois requests (nesse exato momento tem 2 PDFs no portal), mas para cada um deles o onReceive foi acionado duas vezes (total deu 4/2 = 2).
Obs.: se você quer testar o código com mais rapidez basta iniciar a seção de depuração do Android Studio e quando o App começar desmarque "Extrair texto" e marque "Limpar dados" (vai apagar os PDFs baixados e fazer o download novamente mas não vai extrair o texto dos PDFs).
Answer: O BroadcastReceiver está a ser registado no método download().
Cada vez que o método download() é chamado o BroadcastReceiver é registado.
Quando o download terminar o BroadcastReceiver será chamado as vezes que foi registado.
Deverá criar e registar o BroadcastReceiver uma só vez no método onCreate().
Nota: Vi que você está a usar o construtor da Activity para criar o array de recievers, não o use, faça isso no onCreate().
Comment: Você tem de ter em conta que o método `onDestroy()` pode não ser chamado e nesse caso os BroadcastReceiver não são "desregistados". Desinstale a aplicação e volte a testar.
Comment: obrigado pelo comentário @ramaral, veja a seção "depurando" e me diz o que acha
Comment: alterei o código, deixei de usar o construtor para inializar o array de receivers, obrigado @ramaral


Establish the convergence and find the limit of $a_n$

Question: I've just been introduced to the idea of series and I'm currently stuck on this problem:
Establish the convergence and find the limit of $a_n$ if
$$a_1 = 1, a_{n+1} = \frac{2(2a_n + 1)}{a_n+3}, n \in \mathbb N$$
I figured that I'm suppose the prove that the series is increasing and bounded but I'm not sure how to prove either.
Answer: Note that all terms will be positive. A limit $l$ would have to satisfy $l = \frac{2(2l + 1)}{l+3}$ which has positive solution $l=2$. We can use that to find an easy solution by considering the series of terms $a_n - 2$ and proving this series tends to $0$.
We have $ a_{n+1}-2 = \frac{2}{a_n+3}(a_n - 2) $. Now $\frac{2}{3}>\frac{2}{a_n+3}>0$ and so $ |a_{n+1}-2|< \frac{2}{3}|a_n - 2| $. The required result is now obvious with $|a_2 - 2| $ less than  $\frac{2}{3}$, $|a_3 - 2| $ less than  $\frac{4}{9}$, etc.
Comment: Thank you! Can you briefly explain what method/idea you used here $$ \frac{2}{3} > \frac{2}{_+3} > 0 $$ and how you were able to deduce this: $$ |_{+1}−2|< \frac{2}{3}|_−2|$$
Comment: Since $a_n$ is positive, the fraction $\frac{2}{a_n+3}$ has to be positive and less than $\frac{2}{3}$.  Then we combine this result with  $|a_{n+1}-2| = \frac{2}{a_n+3}|a_n - 2|$.
Answer: Note that $$a_{n+1} = \frac{2(2a_n + 1)}{a_n+3} =4-\frac {10}{a_n+3}$$
Thus the sequence is increasing and bounded by $4$ thus it is convergent with a limit of $l$
The limit satisfies $$l=\frac {4l+2}{l+3}$$
Solving for $l$ and choosing the positive answer  results in $l=2$
Comment: How where you able to get $4 - \frac{10}{a_n + 3}$ from the given sequence? I'm still new to this topic so excuse my naiveness.
Comment: @BandSome Divide the numerator $4a_n+2$ by the denominator $a_n+3$ you get the quotient of $4$ and remainder of $-10$


compound interest program problem in dev c++ using c

Question: I am trying to write a program to print simple interest and compound interest in dev c++ in c programming but it is not printing the correct compound interest can anybody tell why?
here is my code:
<code>#include<stdio.h>
#include<math.h>
int main()
{
int p,r,t,si,ci;
printf("enter the principle:");
scanf("%d",&p);
printf("enter the rate:");
scanf("%d",&r);
printf("enter the time:");
scanf("%d",&t);
si=p*r*t/100;
ci=p*pow((1+(r/100)),t);
printf("simple interest:%d",si);
printf("\ncompound interest:%d",ci);
int a=getch();
return 0;
}
</code>
Comment: For instance.. `r/100` will give the result 0 (zero) when `r` has a value below 100. Probably not what you want. Consider using the type `double` instead.
Comment: Seems you are doing integer calculations for something that requires floating point calculations
Comment: Are you sure all of the variables should be integers?
Comment: Don't forget to check the return of `scanf()` and there is no `getch()` in C unless you include the non-standard ancient DOS header `conio.h`. (use `getchar()` instead)
Comment: And when you change to `double` use `%lf` instead of `%d`
Answer: You have given int instead of float.
<code>#include<stdio.h>
#include<math.h>
int main(){
    float p,r,t,si,ci;
    printf("enter the principle:");
    scanf("%d",&p);
    printf("enter the rate:");
    scanf("%d",&r);
    printf("enter the time:");
    scanf("%d",&t);
    si=p*r*t/100;
    ci=p*pow((1+(r/100)),t);
    printf("simple interest:%d",si);
    printf("\ncompound interest:%d",ci);
    int a=getch();
    return 0;
}
</code>


Changing the elements in a set changes the 'equals' semantics

Question: Imagine that we have this piece of code.
public class HashAddAfter {
    private class A {
        public int value;
        public A(int value) {
            this.value = value;
        }
        public void setValue(int value) {
            this.value = value;
        }
        // Code for hashCode()...
        // Code for equals()...
    }

    Set list1 = new HashSet();
    Set list2 = new HashSet();

    public static void main(String[] args) {
        HashAddAfter x = new HashAddAfter();

        A e1 = x.new A(1);
        A e2 = x.new A(1);

        x.list1.add(e1);
        x.list2.add(e2);

        System.out.println(x.list1.equals(x.list2));   // true

        e1.setValue(4);
        e2.setValue(4);

        System.out.println(x.list1.equals(x.list2));   // false
    }
}

I didn't put the code for hashCode() and equals() due to space constraints, but it's the one generated from Eclipse.
The problem is that before changing the elements in the two sets, the sets are equal. After changing their values (each to the same value), the sets are not equal anymore, although e1.hashCode() == e2.hashCode() and e1.equals(e2).
I'm guessing that when comparing two HashSets, Java uses the original hashCode of the element (the one at the moment of insertion). So, changing the elements after insertion changes their original hashCode, and therefore contains() will return false.
In my opinion this is a very unintuitive behaviour.
What do you think?
Answer: This is exactly the expected behavior.  There is no possible way for a Set implementation to be aware that the hashCode of an element has changed, so there's nothing it can do to defend against that possibility.
From the Set Javadoc:

Note: Great care must be exercised if mutable objects are used as set elements. The behavior of a set is not specified if the value of an object is changed in a manner that affects equals comparisons while the object is an element in the set. A special case of this prohibition is that it is not permissible for a set to contain itself as an element.


Reparametrization trick with non-Gaussian distributions?

Question: I am thinking about the re-parameterization trick in a variational auto-encoder.
I know that it can be used with normal distribution. Can the re-parameterization trick be applied to other distributions like Bernoulli? What are the limitations of its application?
Comment: what's the trick?
Answer: Reparameterization of a VAE can be applied to any distribution, as long as you can find a way to express that distribution (or an approximation of it) in terms of

The parameters emitted from the encoder
Some random generator. For a Gaussian VAE, this is a $\mathcal{N}(0,1)$ distribution because for $z \sim \mathcal{N}(0,1)$ means that $z\sigma + \mu =x\sim\mathcal{N}(\mu,\sigma^2)$. Other distributions might employ a different random number generator. A Dirichlet VAE uses a $\mathcal{U}(0,1)$ distribution, an approximation to a gamma distribution, and the ratio of independent gamma random variables to yield a Dirichlet distribution.

This transformation needs to be differentiable in order to use back-propagation. That's why we use re-parameterization.
The choice of re-parameterization can influence how the encoding works -- ideally, we want all components of the model to be used in encoding and decoding the data.
This article provides more details in the context of a Gaussian and Dirichlet VAE.
Weonyoung Joo, Wonsung Lee, Sungrae Park, Il-Chul Moon by "Dirichlet Variational Autoencoder"

This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a Dirichlet prior for a continuous latent variable that exhibits the characteristic of the categorical probabilities. To infer the parameters of DirVAE, we utilize the stochastic gradient method by approximating the Gamma distribution, which is a component of the Dirichlet distribution, with the inverse Gamma CDF approximation. Additionally, we reshape the component collapsing issue by investigating two problem sources, which are decoder weight collapsing and latent value collapsing, and we show that DirVAE has no component collapsing; while Gaussian VAE exhibits the decoder weight collapsing and Stick-Breaking VAE shows the latent value collapsing. The experimental results show that 1) DirVAE models the latent representation result with the best log-likelihood compared to the baselines; and 2) DirVAE produces more interpretable latent values with no collapsing issues which the baseline models suffer from. Also, we show that the learned latent representation from the DirVAE achieves the best classification accuracy in the semi-supervised and the supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models show better performances in most cases.
Comment: Isn't there going to be a differentiability problem with a random node that has a Bernoulli distribution? (The point of the reparametrisation trick being to make gradient computation easier)
Comment: @ThomasLumley Yes, that's why we use reparameterization.
Comment: I don't mean just any random node. Specifically, if $y=f(U, p)$ where $U$ is the randomness and $y$ is Bernoulli and $p$ is the Bernoulli mean, how do you make $y$ differentiable wrt $p$?
Comment: I didn't make a claim that it's possible in the specific case of a Bernoulli. The question is framed in general terms, and the Bernoulli distribution is introduced as an example. My reading is that if OP is specifically interested in a VAE for a Bernoulli distribution, they would have said so, instead of introducing a Bernoulli distribution as an example of a VAE for a non-normal distribution.
Comment: Ok. I thought you'd found some clever way to do it for the Bernoulli and I was not getting the explanation
Comment: On the other hand, Googling `Bernoulli VAE` turns up some promising results, such as https://davidstutz.de/bernoulli-variational-auto-encoder-in-torch/ -- basically, exploit the re-parameterization trick.
Comment: The reparameterization trick for Bernoulli itself seems straightforward. Collect some frozen samples from uniform in x∈[0,1]. Then, have the network predict the Bernoulli probability p∈[0,1]. Create a unit that calculates Heaviside[p-x], and it should work. The bigger issue is what to do about backpropagation through this non-differentiable unit. There are standard tricks like softening the Heaviside or using surrogate gradient, but based on these links, I suppose even more clever approaches have emerged.
Comment: @MRule This is why I wrote "This transformation needs to be differentiable in order to use back-propagation. That's why we use re-parameterization."


The matrix obtained by the product of two orthogonal vectors from two different spaces

Question: Let them be $${u_1,u_2,u_3,u_4}$$ and $${v_1,v_2,v_3,v_4,v_5,v_6}$$ orthonormal vectors in $\mathbb{R}^4$ and $\mathbb{R}^6$ , and $$A=v_1u_1^{T} + v_2u_2^{T}$$
How to determine the solution system $$Ax=2v_1+v_2$$
Answer: HINT: What if you look at all vectors $x\in\Bbb R^4$ with $x\cdot u_1 = 2x\cdot u_2$? (To solve this, you can rewrite the equation in homogeneous form easily enough.) Note, in particular, that any vector in the span of $u_3$ and $u_4$ automatically satisfies the equation.


Using differential calculus to prove a fact on exponents

Question: I have done some calculations recently on squares and cubes and $4^{th}$ powers, and so on, and I found out something. It goes like this:
$$
\begin{align}
f(x)&=x^2\\
f'(x)&=2x^1=2x\\
f"(x)&=2
\end{align}
$$
This is only for the rate of change in the squares. For the rate of increase, you get:
$$\begin{align}
f(x)&=x^2+(x^2+x+1)+(x^2+x)\\
f'(x)&=2x+(2x+1)+(2x+1)\\
f''(x)&=2+(2)+(2)\\
&=6
\end{align}
$$
To make any noticeable pattern appear, we have to add another term or two. the cubes' differentiation would be this:
$$
\begin{align}
f(x)&=x^3+(x^3+x^2+x+1)+(x^3+x^2)\\
f'(x)&=3x^2+(3x^2+2x+1)+(3x^2+2x)\\
f''(x)&=6x+(6x+2)+(6x+2)\\
f'''(x)&=6+(6)+(6)\\
&=18
\end{align}
$$
Notice the pattern yet? No? Here's one more:
$$
\begin{align}
f(x)&=x^4+(x^4+x^3+x^2+x+1)+(x^4+x^3)\\
f'(x)&=4x^3+(4x^3+3x^2+2x+1)+(4x^3+3x^2)\\
f''(x)&=12x^2+(12x^2+6x+2)+(12x^2+6x)\\
f'''(x)&=24x+(24x+6)+(24x+6)\\
f''''(x)&=24+(24)+(24)\\
&=72
\end{align}
$$
Do you notice the pattern yet?  It is this:
$$\text{if} f(x)=x^z+(x^z+x^{z-1}+x^{z-2}+\cdots+x^0)+(x^z+x^{z-1}), \\
\text{then} f''{^{(\text{z times})}}''(x)=3(z!)$$
Basicially, in the equation $x^z+(x^z+x^{z-1}+x^{z-2}+\cdots+x^0)+(x^z+x^{z-1})$, which is the rate of increase in a cube or polytope, the $z^{th}$ derivative (or, in calculus terms, you could put it as $\frac{d^zf}{dx^2}$)is always $3\times z!$
My question is this: is there any practical use for this? My friend said it may have something to do with the rate that the exponential function increases (as in $3z! $ added to a number added to a number and so on z times until you get $x^z$), but I am not sure. Is it for that? If not, where is it useful? Thanks in advance.
Comment: You could show this without having to guess a pattern - you have $3x^z+2x^{z-1}+x^{z-2}+...+z^0$. Taking derivatives $z$ times leaves you with $3z!$. I don't understand what is special about this function though.
Comment: just a remark on notation: the $n$-th derivative of a function $f$ is commonly denoted as $f^{(n)}(x)$ as [shown here](https://en.wikipedia.org/wiki/Notation_for_differentiation#Lagrange.27s_notation).
Comment: @Dando18 that is interesting...   i'll use that next time, if it pops up again
Answer: Let $f(x) = x^z + (x^z + x^{z-1} + \cdot\cdot\cdot + x^0) + (x^z + x^{z-1})$.
First, rewrite this as $f(x) = 3x^z + (x^{z-1} + x^{z-2} + \cdot\cdot\cdot + x^0) + (x^{z-1}) $ Now we know that the $n$-th derivative of $x^n$ is $n!$ and it's trivial that the $n$-th derivative of $x^k$ is $0$ for any integer $0 < k < n$.
Since all the terms after $3x^z$ become $0$ as we differentiate, we only have to concern ourselves with the first term.
$$ \frac{d^z}{dx^z}\left[ 3x^z \right] = 3 \cdot \frac{d^z}{dx^z}\left[ x^z \right] = 3z!$$
Addendum: Why is $\frac{d^n}{dx^n} [x^n] = n! \,$ ?
As you did in your post, write out the first couple derivatives to notice the pattern.
$$ \begin{align*}
f(x) &= x^n \\
f'(x) &= n x^{n-1} \\
f''(x) &= n (n-1) x^{n-2} \\
f'''(x) &= n (n-1) (n-2) x^{n-3} \\
f^{(k)}(x) &= \frac{n!}{(n-k)!}x^{n-k} \\
f^{(n)}(x) &= \frac{n!}{(n-n)!} x^{n-n} = \frac{n!}{1} x^0 = n!
\end{align*} $$


Law of total expectation and how prove that two variables are independent

Question: I am confused in understanding the formal definition for independence and how does this differ from the law of total expectations.
I thought that $E[u|x]=E[u]$ formally means independence of $u$ and $x$.
However, when I have $E[u|x]=a$ (const) and use the law of total expectations,
$E[u]=E[E[u|x]]=a$
I obtain $E[u]=a$
Don't I have then $a=E[u|x]$ and $a=E[u] \implies a=E[u|x]=E[u] \implies \text{independence between x and u}$?
Something does not add up, as this would mean that we can always establish independence between two variables. Where is my error in thinking?
Answer: It's impossible to read minds, so to locate any error in thinking let's help you work through as simple an example as possible using the most basic possible definitions and axioms of probability.
Consider a sample space $\Omega = \{au, bu, cu, av, bv, cv\}$ ("$au$" etc. are just names of six abstract things) where all subsets are considered measurable.  Define a probability $\mathbb{P}$ on $\Omega$ in terms of its values on the atoms via
$$\mathbb{P}(au) = \mathbb{P}(cu) = p,\ \mathbb{P}(bu) = r-2p;\quad \mathbb{P}(av) = \mathbb{P}(cv) = q,\ \mathbb{P}(bv) = 1-r-2q$$
where $p,q,r$ are any numbers for which all six probabilities are positive.  For instance, we may take $p=1/6$, $q=1/8$, and $r=1/2$.  Because the sum of the six given probabilities is unity, this defines a valid probability measure.
Define the random variables $U$ and $X$ as
$$U(\omega)=-1, 0, 1$$
depending on whether the initial letter in the name of $\omega$ is $a$, $b$, or $c$ respectively; and
$$X(\omega) = 0,1$$
depending on whether the final letter in the name of $\omega$ is $u$ or $v$ respectively.
This can be neatly summarized in a $3$ by $2$ table of probabilities, headed by values of $U$ and $X$, whose interpretation I trust is evident:
$$\begin{array}{r|cc}
  & \text{X=0} & \text{X=1} \\
\hline
 \text{U=-1} & p & q \\
 \text{U=0} & r-2 p & 1-r-2 q \\
 \text{U=1} & p & q
\end{array}$$
It is then easy to compute the following:

$\mathbb{P}(X=0) = \mathbb{P}(\{au,bu,cu\}) = p + r-2p + p = r$ (sum the left column in the table).
$\mathbb{P}(U=-1) = \mathbb{P}(\{au,av\}) = p+q$ (sum the top row in the table).

Independence means nothing other than probabilities multiply.  This would imply, among other things, that
$$p = \mathbb{P}(au) = \mathbb{P}(X=0, U=-1) = \mathbb{P}(X=0)\mathbb{P}(U=-1) = r(p+q)$$
(investigate the top left entry in the table).  But this is rarely the case; for instance, $1/6 \ne (1/2)(1/6 + 1/8)$.  Therefore $X$ and $U$ are not independent (except for some special possible combinations of $p$, $q$, and $r$). However,
$$\mathbb{E}(U) = \mathbb{P}(U=-1)(-1) +  \mathbb{P}(U=0)(0) + \mathbb{P}(U=1)(1) = 0$$
That is, the expectation of $U$ is zero. (This should be obvious from the symmetry: the chance that $U=1$ balances the chance that $U=-1$ regardless of the value of $X$.)  However,
$$\mathbb{E}(U|X=0) = \mathbb{P}(U=-1, X=0)(-1) + \cdots + \mathbb{P}(U=1, X=0)(1) = 0,$$
and similarly $$\mathbb{E}(U|X=1) = 0.$$  That is, the conditional expectation of $U$--which is a function--has the constant value zero.  In symbols, we have found that
$$\mathbb{E}(U|X) = 0 = \mathbb{E}(U)$$
and we have a simple, explicit example showing why constant conditional expectation does not imply independence.
Comment: (+1, awhile ago) My favorite example of this is $(X,Y)$ distributed uniform on $\{(1,0), (0,1), (-1,0), (0,-1)\}$. In that case you get $\mathbb E(X \mid Y) = \mathbb E(Y \mid X) = 0 = \mathbb E X = \mathbb E Y$ but $X$ and $Y$ are not independent.
Comment: Thanks @Zen: you are correct that the conditional notation should be used at first and then that should be equated with the joint probabilities.
Comment: @Cardinal I began with that one but somehow--through some error--felt early on that I could not make a counterexample of it. What you see here was the result of the next step.  Sorry about the extra complexity...
Comment: ...also, no need to apologize for the "extra complexity". I'm glad it's there because it shows that simple one-off examples like the one I mentioned aren't just parlor tricks.
Comment: (+1) Nice answer. A minor typo: in the computation of $\mathbb{E}(U\mid  X=0)$, it should be $P(U=-1\mid X = 0)$, etc (the conditional, not the joint). Doesn't change the result, of course.
Comment: oh wow, great answer! So independence stems from independence in probability... Can I ask why classical regression models claim u and x are independent? Is this because when they assume u and x being jointly normal distributed and we have  strict exogenity, this would imply their independence in probability?
Comment: Majte In most regression models $x$ is not a random variable. It is a value set by the experimenter who observes a response; *for simplicity,* variations in the response are assumed not to depend on $x$, *until proven otherwise*: there's no "claim" made. In so-called "mixed" models $x$ is random. If the errors are thought to depend on $x$, that has to be included in the model in a *quantitative* way. Many such models exist; for instance, heteroscedastic responses are modeled in this way, as well as correlations induced by grouping, repeated measures, etc.
Comment: Fine, @whuber. Sorry for nitpicking. It is cool to think how many times this answer will be hit by people googling **constant conditional expectation does not imply independence**. Probably, forever.
Comment: To follow up on my original question and associated confusion, so when we have $E[u_i|x_i]=E[u_i]=0$ it does NOT imply (assuming conditional homoskedasticity) that $V[u_i|x_i]=V[u_i]=σ^2$. But if $u_i$ and $x_i$ were truly independent as within the probability space or as in their joint density being equal to the product of their marginal densities, we could assert this claim of $V[u_i|x_i]=V[u_i]$? In other words, independence implies $E[u_i|x_i]=E[u_i]$ and $V[u_i|x_i]=V[u_i]$ but usually never the other way around?
Comment: I don't understand the first part of that comment, Majte (because I'm missing some of the context), but the statement following "in other words" is correct.
Comment: What I meant is, once we established independence, it ought to work for all moments, e.g. $E[u_i^d|x_i]=E[u_i^d]$. However, if we have just the constant conditional expectation, we cannot assert such claim? (Perhaps read my previous duplicate post, which should now make more sense)
Comment: Quick note: after just a few hours, Google already ranks it first when I search for **constant conditional expectation does not imply independence**.
Comment: Does this mean I am getting famous for my stupidity and I don't have to even undress for that? :-)
Comment: @whuber: I like the example, in particular, because a simple rigid transformation (45 degree rotation) preserves all four of the expected values but results in two independent random variables. Most people respond with disbelief initially; those that think about it further tend to come away with a better understanding of these matters.


solution to mixture CDF / inverse CDF of finite mixture

Question: I am currently numerically solving the follwing equation, which is a convex combination/finite mixture of two marginal CDFs $F(x)$ and $G(x)$ (actually they are joint CDFs but all arguments but x are fixed) on the left hand side, for x:
$$ωF(x)+(1−w)G(x)=a$$
where $a$ is fixed but I want to obtain solutions fo different $w∈[0,1]$. Furthermore are both $F(.)$ and $G(.)$ strictly increasing (because I am working with continous CDFs) and mapping from [0,1]. In my current example it seems that the solution $x0$ satisfies: $x_0=x_{0,ω=0}+ω(x_{0,ω=1}−x_{0,ω=0})$ where $x_{0,ω=0}$ is the solution for $ω=0$, that is the solution to $G(x)=a$, and $x_{0,ω=1}$ is defined accordingly. Both solutions exist.
Is my observation true in general for the given assumptions? I have tried to show $ωF(x_0)+(1−w)G(x_0)=a$ but could not figure out how to. Alternatively, if I had an expressing for the inverse CDF of the mixture I could try to show that this equals $x_0$ for $a$.
Answer: Your relation $x_0=x_{0,ω=0}+ω(x_{0,ω=1}−x_{0,ω=0})$ will not hold in general.  If $F(x)$ and $G(x)$ were linear over the same domain, but were different (which can not happen with a Uniform due to difference in domain if distributions aren't identical), then it would be true for all parameter values.  The relationship might happen to work in particular instances of $F(x)$, $G(x)$, and $a$. What is your current example for which you believe the relation holds?
Here's a simple counterexample:
Let $F(x)$ be exponential with parameter value = 1, i.e., $F(x) = 1-exp(-x)$ for $x\ge 0$.
Let $G(x)$ be exponential with parameter value = 2, i.e., $G(x) = 1-exp(-2 x)$ for $x\ge 0$.
Let $a$ = 0.5
When $ω$ = 1, the solution is 0.6931.
When $ω$ = 0, the solution is half of that, namely 0.3466.
When $ω$ = 0.5, the solution is 0.4812, which is not halfway between the
$ω$ = 0 and $ω$ = 1 solutions. This contradicts your assumed relation (as I am interpreting it).
Comment: Thanks! Do you have an idea under which conditions it woould holds (at least approximately?)
Comment: @InfiniteVariance , the closer to linearity the CDFs, and therefore the inverse CDFs, are in the region of the solution, the better the approximation will hold.


RE: Remove ask question in answer colums

Question: I have recommended removing some question asked in a column of the answer. I have checked my history still it in the answer column.  moderator, please check and remove the question.
https://magento.stackexchange.com/review/low-quality-posts/145833
https://magento.stackexchange.com/review/low-quality-posts/145887
Answer: Rama Chandran M, Thanks for this question.
The posts are removed Now
Comment: Thanks for removed :)


How to save an indexedrowmatrix to a local file orderly in spark?

Question: I have a blockmatrix and want to save it on my local disk in a single file.
val filepath = "hdfs://*********************"

object RowOrdering extends Ordering[IndexedRow] {
    def compare(a:IndexedRow, b:IndexedRow) = a.index compare b.index
}

blockmatrix.toIndexedRowMatrix.rows.map(x => x.vector.toDense).coalesce(1, true)(RowOrdering).saveAsTextFile(filepath)

But after I reloading the file
sc.textFile(path).take(1)(0)

It doesn't return the first line of the matrix. so, how can I save such a matrix while the order is still kept? thanks a lot!
Comment: What did you already try to attempt?
Comment: Try 'repartition(1)'
Answer: I have figure it out. just use the command
blockmatrix.toIndexedRowMatrix.rows.sortBy(x => x.index).map(x => x.vector.toDense).saveAsTextFile(filepath)


Add block programmatically in the right region

Question: I have followed the instruction to create a new block here which is basically the same as here on SE. My block is indeed created but the region is wrongly set to "none" instead of "Content Post".
In my module events.module file:
function events_block_info () {
    return [
        'events_list' => [
            'info'       => "Events list",
            'region'     => 'content_post',
            'visibility' => BLOCK_VISIBILITY_NOTLISTED
        ]
    ];
}

function events_block_view ($delta = '') {
    $result = false;
    if ($delta == 'events_list') {
        /* ... do my stuff .... */
    }

    return $result;
}

This creates a block but the region is set to "none". The drawback is I have to go on each environment and change the region manually back to "Content Post". It is inefficient.
I also find it extremely weird that the creation of a block is done this way. I would expect instead that the creation takes place in the module.install file in the hook_install() function. Maybe there is way?
Comment: You have a pluralisation bug (module is called `event`, but you're implementing hooks for a module called `events`). Other than that your method is correct, there's no way to provide blocks through an install file (it's not how Drupal's architecture works yet, in Drupal 8 you don't use the block hooks). `region` is per-theme by the way, and has some extra constraints, the variable desc in the function docs themselves should let you know if those apply to you
Comment: Oops, typo I corrected the events.module in my post here. In my code it was already ok.
Comment: @Clive About "regions", when the `hook_block_info()` is executed, it will add a block for each enabled theme. But in my case all of them are set to the "none" region...
Comment: are you sure the machine name for the region is correct?
Answer: I think you need to set 'status'.
eg.
function events_block_info () {
    return [
        'events_list' => [
            'info'       => "Events list",
            'status'     => TRUE,
            'region'     => 'content_post',
            'visibility' => BLOCK_VISIBILITY_NOTLISTED
        ]
    ];
}

Comment: Is there any way in drupal 8 ?


How can i execute a certain function on each sheet of a xlsx file having more that 100 sheets in a parallel way?

Question: I have an xlsx file  File.xlsx, it has more than 100 sheets.
I need to perform a certain function f() on each sheet's data and finally return a list of each sheet's operated appended together.
I tried using pandas, reading each sheet's data one by one and then applying the function on it and appending in a list. Which is taking a lot of time. Need to the reduce the operation time.
How can I make the sheet wise execution in parallel? Can i use DASK aor anything else?
Note: need to jsonify each sheet's data so using df.to_json()
Answer:
How can I make the sheet wise execution in parallel? Can i use DASK aor anything else?

This is a trivial usage of Dask:
import dask
import pandas as pd

@dask.delayed
def get_sheet(filename, sheet_index=0):
    return pd.read_excel(filename, sheet_name=sheet_index)

@dask.delayed
def process(df: pd.DataFrame) -> pd.DataFrame:
    """
    Inputs
    ------
    df : pd.DataFrame
        A Pandas DataFrame. For this example, this DataFrame represents on sheet.

    Returns
    -------
    out : pd.DataFrame
        A new dataframe that makes some modifications on the original sheet.
    """
    out = df.copy()
    out["foo"] = "bar"
    return out

if __name__ == "__main__":
    # Get the sheets of the Excel file (test.xlsx has two sheets)
    future_dfs = [get_sheet("test.xlsx", sheet_index=i) for i in [0, 1]]

    # Process the sheets
    processed_dfs = [process(df) for df in future_dfs]

    # Now that we've detailed the computation, start the computation.
    dfs = dask.compute(processed_dfs)

The function process should not modify the input DataFrame because functions should be pure and not modify the inputs. See the docstring of dask.delayed for more detail: https://docs.dask.org/en/latest/delayed-api.html#dask.delayed.delayed


Reasons to turn off/on unreachables messages?

Question: I see on this question (How can I stop Linux from sending ICMP "Destination Unreachable" responses?) that there was a lot of discussion pointing to the fact that you shouldn't turn off ICMP unreachable messages. I am wondering why and when you should? I too want to know how to do it. I know it breaks MTU path discovery but what else?
On cisco devices you can turn this on and off, there must be a reason. In their documentation it just says that turning it off is supposed to be for increased security as in it's harder to get information about your network? This is what the cisco documentation says. I need to implement the ability to turn this on and off on a switch for my company so I am learning about it. Regardless of the why's I still have to do it, but I'd like an informed answer on why to do it or not to give to others.
When I want to turn off ICMP redirects I do this:
echo 0 > /proc/sys/net/ipv4/conf/all/accept_redirects

Is there something similar for unreachables?
The user on the other thread did it like so:
iptables -A OUTPUT -p icmp --icmp-type destination-unreachable -j DROP

is this a good way, then i could turn it on again by stopping this drop?
Should I be telling people that they don't need this feature?
EDIT:
Online I see this:
An attacker could gather information's about your network when scanning it,
like unused IP's and networks. When working with (interface-) Access-Lists,
a deny statement triggers an ICMP Type 3     Code 9/10 message
(Network/Host is Administratively Prohibited). When disabling ICMP    unreachables
on the interface where the ACL is applied, the deny statement
acts like a 'drop' and does not reply.

Answer: From a very thorough and well-written answer about the same subject over at security.SE, which I highly recommend you to read:

At its core ICMP was designed as the debugging, troubleshooting, and
  error reporting mechanism for IP. This makes it insanely valuable so a
  lot of thought needs to into shutting it down. It would be a bit like
  tacking >/dev/null 2>&1 to the end of all your cron entries.

Source Quench / Redirect is pretty much obsolete and removed from modern networking devices. IPv6 requires ICMP to fully function.
The bottom line: Don't block anything unless you fully understand the implications. If I were to give you one advice, it would be to block icmp echo in your external firewall, and leave everything else open. But that is just my opinion.
Comment: I don't block any ICMP at all, as I don't believe it really helps. Security through obfuscation had never really worked..
Comment: Thanks I agree it is silly to turn it off, however now I have to do it anwyay as customers want it. I will just do: iptables -A OUTPUT -p icmp --icmp-type destination-unreachable -j DROP. Hmm I wonder can I do it per vlan interface
Comment: +1 Good summary and reference link. I don't agree with blocking *echo*, and apparently Google doesn't either. But that's probably the smallest offense you could commit in blocking some part of ICMP.
Answer: Preventing your server from sending "destination unreachable" basically makes it mute to a majority of port scans, which increases security. However, not by much. It does break path discovery and there's no real reason to block it. There are many many ways to scan a server and that ICMP response is just one of them.
Comment: This is worthless for preventing port scans from giving useful information. Worse, it lets me probe the general outlines of your firewall, which is probably not what you want.
Comment: I see, there are other ways to see the information so this command is fairly useless. Still I have to implement the ability to turn it on and off. Thanks!
Comment: +1 If your router support rate limiting it's a good idea to limit ICMP response so you aren't the unwitting participant in a reflection attack. Otherwise, the information gleaned from ICMP is of little real value, and an attacker could find the information other ways anyway.
Comment: Not replying to ICMP doesn't mute you from port scans, it just takes longer to find something that will answer..
Answer: I think you have some mistakes. ICMP Redirect and ICMP Destination Unreachable is different.
ICMP Redirect should be disabled, because it is extremely easy to fake and an attacker can forge ICMP redirect packets basically.
ICMP Destination Unreachable is need for your host to do path MTU discover, disabled it can make some false positve with you host communication. But an attacker can also use it to discover your network topology.
The simple solution for you, is make your firewall deny all new incomming ICMP packet, only accept packets with ESTABLISHED state.
iptables -A INPUT -i eth0 -p icmp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUPUT -i eth0 -p icmp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -e eth0 -p icmp -j DROP

So your host can still do path MTU discover, and is almost "invisible" with some automatic scanner.
Comment: @NathanC Is there a way to turn them off per interface? It says I can't use output with -i
Comment: nvm got it in the end
Comment: ICMP is stateless so it wouldn't work in this case.
Comment: Are you kiddy? Have you ever try this?
Comment: I really disagree with dropping ICMP in general. It does so much more than just discovering paths and that sort of thing. ICMP handles states differently - so you'd also need `--state RELATED` for the unreachable messages to even work.
Comment: Hi I know they are different, I was just wondering was there a similar way to turn them both off. What is this related part mean @NathanC? Anything wrong with just dropping destination-unreachable?
Comment: You can just switch it off then back on if you encounter issues.


Click element by element in the list

Question: How can I click element by element in the list where the locator looks like this:
id=${NameFirst.replace(" ", ""}_${NameLast.replace(" ", "")}

    *Keywords*
    | Get all first and last names
    | | ${Name} | Query Master Database
    | | ... |
    | | [Return] | ${Name[0][0]} | ${Name[0][1]}

    *Test Case*
    | Click element by element
    | | :FOR | ${NameElement} | IN | @{NameElementList}
    | | ${NameFirst} | ${NameLast}= | Get all first and last names
    | | set suite variable | ${NameFirst}
    | | set suite variable | ${NameLast}
    | | Click Element | id=${NameFirst.replace(" ", ""}_${NameLast.replace(" ", "")}

When I run test case it clicks the first element only
INFO ${Name} = [['AAA', 'BBB'], ['CCC', 'DDD']]
INFO    ${NameFirst} = AAA
INFO    ${NameLast} = BBB

Comment: You are not using `${NameElement}` anywhere in the FOR loop which makes the loop pretty much useless.
Answer: First, it appears that your loop isn't formatted correctly. The body of the loop needs an extra level of indentation. The following example illustrates:
*Test Case*
| Click element by element
| | :FOR | ${NameElement} | IN | @{NameElementList}
| | | ${NameFirst} | ${NameLast}= | Get all first and last names
| | | set suite variable | ${NameFirst}
| | | set suite variable | ${NameLast}
| | | Click Element | id=${NameFirst.replace(" ", ""}_${NameLast.replace(" ", "")}

Here the lines belonging to forloop needs be be spaced out leaving one cell. PFA

Click element by element in the list


While com JavaScript pegando imagens dentro de uma pasta

Question: Como fazer para pegar imagens dentro de uma pasta com laço de repetição em javascript e inserir juntamente com a estilização do Materialize Css
Observanndo que as imagens dentro da pasta já estão todas com numeração sequencial.

01 Apartamentos Cabine Elétrica

Comment: Vc quer inserir as imagens na div com os mesmos atributos via JavaScript?
Comment: Sim, perceba que  repete a cada linha e a img tem class="responsive-img card materialboxed" que também deve repetir. entendeu ?
Answer: Em vez de usar while, você poderia usar for e inserir o HTML na div. No for abaixo, o valor 10 é a quantidade de imagens que você deseja inserir:

document.addEventListener("DOMContentLoaded", function(){ // aguarda o DOM ser carregado
   // seleciona a div com a classe .row
   var container = document.querySelector("#abate > .row");
   var html = '';
   for(var x=1; x <= 10; x++){
      html += `

`;
   }

   container.innerHTML = html;

});

01 Apartamentos Cabine Elétrica


MongoDB ifNull on lookup

Question: I am trying to figure out how to do the following:
I currently use the following script to see if an advert matches a listing.
<code>'$lookup': {
                'from': 'advertisers',
                'localField': 'title',
                'foreignField': 'title',
                'as': 'adsys'
            }
</code>
but some of our listings don't have adverts and when it is currently returning it is showing adsys:[]
Instead I would like it to ether show adsys:NULL or remove the adsys completely. I have looked at
<code>$ifNull
</code>
However I am unsure how I attach it to the adsys if it's in the lookup.
I tried this:
<code> {
            '$lookup': {
                'from': 'advertisers',
                'localField': 'title',
                'foreignField': 'title',
                'as': 'adsys'
            }
        },
        {
                '$project': {
                   'title': 1,
                   'adsys': { '$ifNull': [ "$adsys", "Unspecified" ] }
                }
             }
</code>
and it returned this:
<code>{"data":[{"track":[{"_id":"5e1a96c3081a7a70d5c8971f","title":"Out of Mind","adsys":[]}]}]}
</code>
what I would like it to do is return this
<code> {"data":[{"track":[{"_id":"5e1a96c3081a7a70d5c8971f","title":"Out of Mind","adsys":NULL}]}]}
</code>
Comment: If `title` is an object of `track` which is in an object of `data` then how is that `title:1` working in `$project` ?
Answer: We can set null value using $unwind keyword after checking value of adsys and set null value if adsys is not defined.
<code>db.tableName.aggregate(
    [
        {
            '$lookup': {
                'from': 'advertisers',
                'localField': 'title',
                'foreignField': 'title',
                'as': 'adsys'
            }
        },
        { "$unwind": { path: "$adsys", preserveNullAndEmptyArrays: true } },
        {
            '$project': {
                'title': 1,
                'adsys': { '$ifNull': ["$adsys", "NULL"] }
            }
        }
    ]);
</code>
Comment: well this would work, but there can be issues in few cases(which might be most cases) - 1) If the result has huge dataset `$unwind` is not a good option, 2) If `title` in `listing` document has multiple matches in `advertisers` then `adsys` will have more than one object(n num of matching objects) - So doing `$unwind` & then grouping those back to form the original result document will be tedious. Basically this will help if result data is less & `adsys` has either `[]` or exactly one object `[{}]` :-)
Comment: Also in `$project` you don't need this `'adsys': { '$ifNull': ["$adsys", "NULL"] }` as after `$unwind` you won't see `adsys` if it's `[]` as it will be removed -O/P's another ask..
Answer: You can try to add this as the end stage :
<code>{$project : {'title' :1, 'adsys' : {$cond : [{$gt:[{$size: '$adsys'}, 0]}, '$adsys' , null ] }}}
</code>
So your query would be :
<code>{
    '$lookup': {
        'from': 'advertisers',
        'localField': 'title',
        'foreignField': 'title',
        'as': 'adsys'
    }
}, { $project: { 'title': 1, 'adsys': { $cond: [{ $gt: [{ $size: '$adsys' }, 0] }, '$adsys', null] } } }
</code>
** Sample Data :**
<code>/* 1 */
{
    "_id" : ObjectId("5e1aa8d5627ef782369ef777"),
    "title" : "Out of Mind",
    "adsys" : []
}

/* 2 */
{
    "_id" : ObjectId("5e1aa8e1627ef782369ef854"),
    "title" : "Out of Mind",
    "adsys" : [
        {
            "abc" : 1
        }
    ]
}
</code>
Result :
<code>/* 1 */
{
    "_id" : ObjectId("5e1aa8d5627ef782369ef777"),
    "title" : "Out of Mind",
    "adsys" : null
}

/* 2 */
{
    "_id" : ObjectId("5e1aa8e1627ef782369ef854"),
    "title" : "Out of Mind",
    "adsys" : [
        {
            "abc" : 1
        }
    ]
}
</code>

MongoDB ifNull on lookup


Regex issue in number of occurrences

Question: I have a requirement validate parameters. My parameters can take any characters with spaces and other symbols, but the number of substrings should be limited, like {1,128}
My regex pattern is:
^*[a-zA-Z0-9~@#$^*%()_+=\{}\[\]|\\,.?: -]*${1,5}

Test input is 'My name is Rani and Age 58'
Though I need to use maximum five words I can put any number of words without errors.
Comment: You need to define what is "word delimiter" and what constitute a "word", then you can write the regex by using the form: `^(++){0,}+$`
Comment: @user2622132 This seems to be one of those `not-so-well-defined-problems`. As asked in previous comment you really need to define what you consider a `"word"` here and provide some sample input/output values.
Answer: Try,
String str = "My name is Rani and Age 58";
if(str.split("\\s").length>0 && str.split("\\s").length<129)
    System.out.println("Accepted");
else
    System.out.println("Rejected");

Regex issue in number of occurrences


How to program the deleted d-jack-knife

Question: I have a dataset containing 45 observations. I want to sample times from this dataset, but with sample size equal to 35 each time. So each time I want to delete 10 datapoints from the original dataset. In total there are ${45}\choose{10}$ possible ways to delete d points from the sample. I want all these possibilities to be exploited during the resampling.
Can someone help me to program this? I must add that the code ${45}\choose{10}$ seems not to work in R because it exceeds the memory capacity.
Many thanks in advance!
Comment: I use the code combn(45,10) which gives following error: Error: cannot allocate vector of size 23.8 Gb
In addition: Warning messages:
1: In vector("list", count) :
  Reached total allocation of 4095Mb: see help(memory.size)
2: In vector("list", count) :
  Reached total allocation of 4095Mb: see help(memory.size)
3: In vector("list", count) :
  Reached total allocation of 4095Mb: see help(memory.size)
4: In vector("list", count) :
  Reached total allocation of 4095Mb: see help(memory.size)
Comment: But probably there is a way to program the situation I described in anthoer way without using that code
Comment: Well probably I'll just restrict it to 1000 samples. I want to estimate the standard error of of an estimate based on the deleted-d-jackknife bootstrap. Concretely my question is how to program a deleted-d-jackknife bootstrap. Surprisingly it is easy to program a jackknife bootstrap, but not a deleted-d-jackknife bootstrap
Answer: It is rare in statistical analysis that such code is truly needed (and it is not needed for the Jackknife), but here (for the record) is a solution.

A reliable method to generate a non-duplicating collection of $k$-subsets of $n$ things is to associate a unique combination with each integer $x$ in the set $\{0, 1, \ldots, \binom{n}{k}-1\}$.  This can be done by walking upwards in Pascal's Triangle back to its apex, exploiting the relation $\binom{n}{k} = \binom{n-1}{k-1} + \binom{n-1}{k}$, and picking elements at each row where you move left (from $k$ to $k-1$).
Start with the triple $(x, n, k)$ with $0 \le x \lt \binom{n}{k}$ and $0 \le k \le n$.  When $x \lt \binom{n-1}{k-1}$, select the first of $n$ elements, decrement $x$ to $$x^\prime = x - \binom{n-1}{k-1},$$ and proceed to select $k-1$ out of $n-1$ elements based on the triple $(x^\prime, n-1, k-1)$.  Otherwise do not select the first of $n$ elements, do not change $x$, and proceed to select $k$ out of $n-1$ elements based on the triple $(x, n-1, k)$.  In any event, if $n = k$, then pick all $k$ elements.
The proof that this works depends on some easily established invariants:

At the outset, $0 \le x \lt \binom{n}{k}$.
Exactly $k$ elements will be picked.

You can check that the following code satisfies these invariants.
choose.int <- function(x, n, k) {
  if(n <= k) return(rep(TRUE, k))
  u <- choose(n-1, k-1)
  pick <- x < u
  if (pick) y <- choose.int(x, n-1, k-1) else y <- choose.int(x-u, n-1, k)
  return(c(pick, y))
}

The proof that it generates all such subsets, without duplication, can be accomplished with an inductive argument on $n$.

Thus, to select (say) $1000$ unique $10$-subsets of $45$ things, obtain a random sample without replacement from the set $\{0, 1, \ldots, \binom{45}{10}-1\}$.  Using choose.int, convert each value $x$ in this set into a vector indicating which of the $45$ elements to select:
n <- 45; k <- 10
sample <- sapply(sample.int(choose(n, k), 1000)-1, choose.int, n=n, k=k)

Timing indicates about $10,000$ such subsets can be generated per second (because R is not efficient with recursive functions).
Because this is a largish array, let's illustrate with smaller values.  How about picking $6$ of the $10 = \binom{5}{3}$ three-subsets of five things:
n <- 5; k <- 3
sapply(sample.int(choose(n, k), 6)-1, choose.int, n=n, k=k)

The output will vary with the random seed, but in one case it was
      [,1]  [,2]  [,3]  [,4]  [,5]  [,6]
[1,]  TRUE  TRUE FALSE  TRUE  TRUE FALSE
[2,]  TRUE FALSE  TRUE  TRUE FALSE  TRUE
[3,] FALSE FALSE  TRUE  TRUE  TRUE  TRUE
[4,]  TRUE  TRUE FALSE FALSE FALSE  TRUE
[5,] FALSE  TRUE  TRUE FALSE  TRUE FALSE

Each column indicates which elements to include in the sample.  They all have $3$ TRUE values for the elements picked.  No two of the columns are the same.
Answer: Delete-d jackknife is not efficient method for this kind of cases. As you noticed, the number of possible combinations is ${n}\choose{d}$, and that is in your case 3190187286 possibilities. Function combn returns error since you ask it to generate a huge matrix of possible combinations and it does not fit the memory. You could try some workarounds like not generating the whole matrix of combinations but creating one combination at a time, computing some summary statistic and not preserving all the intermediate values. Notice however that also in this case you would have to loop through all the combinations and it would take a very long time. This is certainly not efficient. Why not trying bootstrap instead? Bootstrap generally needs much less iterations.

How to program the deleted d-jack-knife


Samba authentication problem when attempting to connect from Windows client

Question: I've got a Linux server running Ubuntu and Samba. I've created two shares in Samba that point to directories that are owned by the user "cameron".
When I attempt to connect to these shares on Windows 7 is connects and allows me to see the files but they are read-only. This is the desired action for guest users but not for authenticated users.
My user on the Windows client is "Cameron" and has the same password as the Linux user "cameron".
I don't think my Windows user has authenticated against the Linux user. I even created a users.map file to map the user cameron (linux) to Cameron (windows) but still it does not work.
Here is my samba config file (UPDATED):
[global]
    server string = %h server (Samba, Ubuntu)
    map to guest = Bad User
    passwd program = /usr/bin/passwd %u
    passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
    username map = /etc/samba/users.map
    syslog = 0
    log file = /var/log/samba/log.%m
    max log size = 1000
    os level = 65
    preferred master = Yes
    dns proxy = No
    wins support = Yes
    usershare allow guests = Yes
    panic action = /usr/share/samba/panic-action %d
    valid users = cameron
    write list = cameron

[www]
    path = /usr/local/apache2/htdocs
    write list = @www-data
    force group = www-data
    guest ok = Yes

[cameron]
    path = /home/cameron
    write list = @www-data
    force group = www-data
    guest ok = Yes

Comment: One thing to note is this file is generated via the Samba web interface swat.
Answer:
Are there authentication errors in the Samba logs?
Do the files on disk have the proper permissions?  If a file is read only on disk, it doesn't matter what you tell samba.

On a side note, if your user name in Windows and Linux is the same, then you don't need to add the user to the user map file.
Comment: I can see the shares and browse them from Windows. The shares are owned by "cameron" and group "www-data". They have the following permissions: `rwxrwxr--` but when I attempt to create a file I get a "You need permission to perform this action" dialog.
Comment: The reason for the map file was because the case was different and I was not sure if Linux is case-sensitive to usernames.
Comment: Just removed the read-only lines and it appears to work now. I'm a little confused as I thought that adding my user to the write list would override this setting.
Answer: I solved the problem at my server by noting the properties of the folder I was sharing. I had admin as the group and Bj as the user.
So, I entered into the configuration file in the section I wanted to share the following:
force group = admin
force user = Bj
Now all computers on my network can read and write to that folder.
Answer: You haven't told Samba how to do authentication.  If this is a small server for yourself, you can use user security.
Add security = user and encrypt passwords = yes to the global section, then run 'smbpasswd cameron' to set your password for Samba.  The password does not have to match your unix or windows password, though matching your windows password would make life easier.
Then restart samba and you should be able to login.
Comment: Under swat I've selected USER security which is what I want. I think the reason it's not in the above config is because it's the default for Samba hence why swat has left it out. It's the same with encrypt passwords. Both are defaults. Do I need to logout and then back in again for Windows to attempt to authenticate?
Answer: The problem resolved itself when I removed the read-only option. Though from reading the documentation as my user was in write-list it should have overridden this setting.
Answer: In my case what did help was to setup Samba-related password using command:
sudo smbpasswd -a username where "username" must be a login from the Linux box. Then on Windows box you use this login and the password set by this command and it works.
Related documentation: https://ubuntu.com/tutorials/install-and-configure-samba#4-setting-up-user-accounts-and-connecting-to-share


If I have a holomorphic function on the unit disc, do I know anything about the radius of convergence of its series expansion about zero?

Question: I'm looking at a proof that assumes only that
$f : \mathbb{D} \rightarrow \mathbb{D}$ is holomorphic with $f(0) = 0$
The first step in the proof is to "expand $f$ in a power series centered at $0$ and convergent in all of $\mathbb{D}$."
In other words, $f(z) = a_0 + a_1 z + a_2 z^2 + ...$ inside the disc. Is this a valid step? How do I know that the function's series expansion is valid in the whole disc?
Comment: I believe [this](http://en.wikipedia.org/wiki/Analyticity_of_holomorphic_functions) might be useful.
Answer: Following Tunococ's link, I found the following: "the fact that the radius of convergence is always the distance from the center a to the nearest singularity; if there are no singularities (i.e., if ƒ is an entire function), then the radius of convergence is infinite. Strictly speaking, this is not a corollary of the theorem but rather a by-product of the proof."
Since there are no singularities, then the series is valid for the whole disc!
Comment: This  is incorrect: you may not  assume  that $f$ has no singularity. Counter-example: $f(z)=\frac {1}{3}+\frac {1}{z-3}$.
Comment: Sorry, I meant that there are no singularities within the disc.


Filter view using taxonomy name in url

Question: I have a view that shows a thumbnail of the node and the teaser of the node for all nodes of a certain content type. What I want to do is to only show nodes tagged with a certain word, using the url
My website taxonomy page has this format /category/tags/beauty
How can I do this?
Answer: Add an Argument to your view with Taxonomy: Term ID (with depth). You may want to use %2 for the title (or %3?). At the bottom of the argument config form you'll see "Set the breadcrumb for the term parents."
Answer: You can achieve that by using Taxonomy: Term ID as argument and than Validator: Taxonomy term, Argument type: Term name/synonym converted to Tern ID as following:

Create view (as per the requirement) and the the required fields.
Set the path of page display: category/tags/%. (Remember % is at third place, this we will be using in contextual filter).
Add the argument: Taxonomy: Term ID.
Configuring the added argument by adding Validator options: Taxonomy term.
Select Argument type: Term name/synonym converted to Tern ID.
Save the view and clear the cache to check the changes.

Filter view using taxonomy name in url


Media Queries - combining AND/OR

Question: I am trying to create a media query which, in pseudo code, reads as the following:
@media min-aspect-ratio: 4/3 AND (min-width: 550px OR min-height: 550px)

But I cannot work out the correct syntax. So far, the closest I can think of is:
@media all and (min-aspect-ratio: 4/3) and ((min-width: 550px), all and (min-height: 550px))

Since, in media queries, a logical or is the equivalent to comma separated values, hence the min-width or min-height.
Can someone assist please
Answer: Since
A and (B or C)

could be expanded as
(A and B) or (A and C)

then you could transform the expression into
@media (min-aspect-ratio: 4/3) and (min-width: 550px),  // A and B
       (min-aspect-ratio: 4/3) and (min-height: 550px)  // A and C

the comma (,) is equivalent to the or condition
Comment: Don't forget the brackets around min-aspect-ratio.
Comment: Thanks! Makes sense now - and quite simple now you mention it :)

Media Queries - combining AND/OR


Elementary subgroup question

Question: List all the elements of the subgroup generated by the subset $\{2,3 \}$ of $\mathbb{Z}_{12}$
The solution said $\langle \gcd(2,3,12)\rangle = \langle 1\rangle = \mathbb{Z}_{12}$
Could someone explain to me why are we taking the gcd?
What exactly does it mean for $\{2,3 \}$ to generate things? I am not seeing any connection or picture here
Comment: If you write \gcd rather than gcd (with no backslash) then not only will the letters not get italicized like variables, but proper spacing will appear in expressions like $a\gcd(b,c)$.  Also, I change $<1>$ to $\langle 1 \rangle$ (coded as \langle 1 \rangle).  That's also standard usage.
Answer: The subgroup generated by a set $A$ is the smallest subgroup that contains all elements in $A$. So you just put all possible combinations of elements in $A$. But for abelian groups like $\mathbb{Z}_{12}$ it is easier, the subgroup generated by $\{2,3\}$ is the collection $\{n\cdot 2+m\cdot 3:n,m\in\mathbb{Z}\}$.
This is also when gcd comes into play. Because $\operatorname{gcd}(2,3)=1$, we know there are $x,y\in \mathbb{Z}$ such that \begin{equation}
x\cdot 2+y\cdot 3=1,
\end{equation}so $1$ is in the subgroup generated by $\{2,3\}$. But $1$ is one generator of $\mathbb{Z}_{12}$, so once you have $1$, you have everything. That is why the subgroup is actually the entire $\mathbb{Z}_{12}$.
Answer: If $G$ is a group and $S$ is a subset of $G$, $\langle S\rangle$, the subgroup generated by $S$, is simply the smallest subgroup of $G$ that contains every element of $S$. It can be defined as $\bigcap\{H\leqslant G:S\subseteq H\}$, the intersection of all subgroups of $G$ containing $S$ as a subset. However, it's probably more instructive to look at it a bit differently.
Consider the subset $S=\{4,6\}$ of $\Bbb Z_{12}$. $S$ is not a subgroup of $\Bbb Z_{12}$: it's not closed under addition, it doesn't contain the identity element, and it doesn't contain the additive inverse of $4$. What, at a bare minimum, do we have to add to $S$ to turn it into a subgroup of $\Bbb Z_{12}$? We certainly have to throw in $-4=8$, though $-6=6$ is already in $S$; that expands $S$ to $\{4,6,8\}$. We also have to close the set under addition, so we have to throw in $4+6=10$, $4+8=0$, and $6+8=2$, bringing the set up to $\{0,2,4,6,8,10\}$. This is a subgroup of $\Bbb Z_{12}$, so nothing more need be added: $$\big\langle\{4,6\}\big\rangle=\{0,2,4,6,8,10\}\;.$$
$S=\{4,6\}$ generates the subgroup $\{0,2,4,6,8,10\}$ in the sense that we can work out from $S$ as I did above, adding elements as we discover that they're needed if we're to have a group, until we've added just enough to get a group, and the process ends.
Finding $\langle S\rangle$ when $S$ is a subset of one of the cyclic groups $Z_n$ turns out to be especially easy. That's where the greatest common divisor comes in. Notice that in my example the subgroup of $\Bbb Z_{12}$ generated by $\{4,6\}$ turned out to be the set of multiples of $2$ in $\Bbb Z_{12}$, which is easily seen to be the subgroup of $\Bbb Z_{12}$ generated by the single element $2$: once you have $2$, you must have $2+2=4$, $4+2=6$, $6+2=8$, $8+2=10$, and $10+2=0$, and these six elements do indeed form a subgroup of $\Bbb Z_{12}$. It's not an accident that $2=\gcd\{4,6,12\}$; in general is you have a subset $S\subseteq Z_n$, you'll find that $\langle S\rangle$ is just the set of multiples (in $\Bbb Z_n$) of the gcd of $n$ and the members of $S$. This is a matter of elementary number theory; I suspect that it's been proved either in class or in your text.
Answer: It means to take all the integer multiples of $\{2,3\}\,$ and their sums, since the operation in $\,\Bbb Z_{12}\,$ is additive. If it were multiplicative it'd mean to take all the powers and products of those elements.
In this case:
$$(-1)\cdot 2+1\cdot 3=1\Longrightarrow \forall\,k\in\Bbb Z_{12}\,\,,\,k=k\cdot 1=(-k)\cdot 2+k\cdot 3$$
so $\,k\,$ belongs to the subgroup generated by $\,\{2,3\}\,$ and thus this subgroup is, in fact, the whole group.

Elementary subgroup question


Forming a consistent estimator for the area under the regression line

Question: I am trying to solve the following problem:

Take the following simple linear regression model, where $x_i \in \mathbb R$:
$y_i=\beta_0 + x_i \beta_1 + \epsilon_i$
Given that:

$\mathbb E[\epsilon_i]=0$
$\mathbb E[\epsilon_i|x_i]=0$
$\beta_0 >0$
$\beta_1 <0$

Let $\theta_0$ represent the area under the regression line. Propose a consistent estimator of $\theta_0$.

I have began by finding the integral of $y_i$ with respect to $x_i$. That is,
$\theta_0= \int \beta_0 + x_i \beta_1 + \epsilon_i$ $dx_i=\beta_0x_i + {x_i^2\over{2}}{\beta_1} + \epsilon_ix_i$
I am considering proposing an MLE and proceeding to find the derivative of the log-likelihood of this expression. However, since this expression is rather complicated and I foresee the MLE computation turning incredibly thorny, I suspect I may be approaching this incorrectly. Any thoughts?
Comment: The regression line is defined over $\mathbb{R}$; presumably you intend it to apply over some specific interval. It would be good to make it explicit what interval is intended (do you mean just the part where the regression function is positive in the first quadrant? That is over $0<x,<-\beta_0/\beta_1$?)
Comment: Yes, it would be over the first quadrant I believe
Answer: The indefinite integral you have provided, $\theta_0$, involves two parameters $\beta_0$ and $\beta_1$ and the random term $\epsilon_i$.  Hence $\theta_0$ is random.  Then you say you want to find the MLE or a consistent estimator of $\theta_0$.  You cannot find the MLE of an random quantity, you can only find the MLE of parameters (unknown constants) in a model.  So you would have to add $\epsilon_i$ as an unknown parameter in the model, and you would find that its MLE is zero.  It makes much more sense to find a consistent estimator for the mean of the area under the regression curve.  In fact, the two solutions are the same since the only reasonable estimate of $\epsilon_i$ is 0 and then $\theta_0$ equals the mean area under the regression curve.
Finally, you can estimate the MLEs of $\beta_0$ and $\beta_1$ using simple linear regression (least squares) and replace them with their population values to obtain the MLE of the mean area under the regression curve.


MongoDB: nested array count + original document

Question: I have the following document structure which contains an array of votes:
    { _id: ObjectId("6350e2c1a15e0e656f4a7472"),
      category: 'business',
      votes:
       [ { voteType: 'like',
           userId: ObjectId("62314007da34df3f32f7cfc0") },
         { voteType: 'like',
           userId: ObjectId("6356b5cbe2272ebf628451b") } ] }

What I would like to achieve is to add for each document the sum of votes for which voteType = like, while keeping the original document, such as:
    [ [{ _id: ObjectId("6350e2c1a15e0e656f4a7472"),
          category: 'business',
          votes:
           [ { voteType: 'like',
               userId: ObjectId("62314007da34df3f32f7cfc0") },
             { voteType: 'like',
               userId: ObjectId("6356b5cbe2272ebf628451b") } ] }, {sum: 2, voteType: "like"} ], ...]

At the moment, the only workaround that I found is through an aggregation although I cannot manage to keep the original documents in the results:
     db.getCollection('MyDocument') .aggregate([   {
            $unwind: "$votes"   },   {
            $match: {
              "votes.voteType": "like",
            }   },   {
            $group: {
              _id: {
                name: "$_id",
                type: "$votes.voteType"
              },
              count: {
                $sum: 1
              }
            }   },
          { $sort : { "count" : -1 } },    {$limit : 5}
        ])

which gives me:
    { _id: { name: ObjectId("635004f1b96e494947caaa5e"), type: 'like' },
      count: 3 }
    { _id: { name: ObjectId("63500456b96e494947cbd448"), type: 'like' },
      count: 3 }
    { _id: { name: ObjectId("63500353b6c7eb0a01df268e"), type: 'like' },
      count: 2 }
    { _id: { name: ObjectId("634e315bb7d17339f8077c39"), type: 'like' },
      count: 1 }

Answer: You can do it like this:

$cond with $isArray - to check if the votes property is of the type array.
$filter - to filter votes based on voteType property.
$size - to get the sized of the filtered array.

db.collection.aggregate([
  {
    "$set": {
      "count": {
        "$cond": {
          "if": {
            "$isArray": "$votes"
          },
          "then": {
            "$size": {
              "$filter": {
                "input": "$votes",
                "cond": {
                  "$eq": [
                    "$$this.voteType",
                    "like"
                  ]
                }
              }
            }
          },
          "else": 0
        }
      }
    }
  }
])

Working example
Comment: Thank you! I also have some cases where votes in the document is null, would there be a way to filter them through the query? Otherwise I got an error: "MongoServerError: PlanExecutor error during aggregation :: caused by :: The argument to $size must be an array, but was of type: null"
Comment: I updated my answer. Can you check it again?


Example of two matrices such that $LM+ML=0$

Question: I am looking for some examples of $n\times n$ complex matrices $L$ and $M$ satisfying $LM=aML$, for some fixed $a\in \mathbb{C}$. In particular if we take $n>2$ and $a=-1$ then we can see that matrices of the form $AB=0=-BA$ gives the required result. But I need some non trivial examples. Please give some hints so that I can proceed.

Should I look some different rings other than the matrix rings?
Comment: It should be noted $\det(aML)=a^n\det(M)\det(L)=a^n\det(LM)$, so for most $a$-s you need a non-invertible matrix.
Comment: Also, $\operatorname{tr}(aML)=a\operatorname{tr}(LM)$ so for $a\ne 1$ we need $\operatorname{tr}(ML)=0$ too.
Comment: What about$$A=B=\begin{bmatrix}0&1\\0&0\end{bmatrix}?$$
Comment: But you wrote $AB=0=BA$. Why do you say now that the "product should be non zero"?
Comment: @JoséCarlosSantos It will not work, product should be non zero
Comment: @MANI But is it possible? If we have $A, B\in\mathbb R^2$ with $AB+BA=O$, then $\det(AB) =\operatorname{tr}(AB) =0$, i.e. $AB$ is similar to $\begin{bmatrix}0&1\\0&0\end{bmatrix}$ or to $O$. I don't believe that the first case would appear, but it's only a guess.
Comment: @Jochen Sir, i have edited the question, can we look for marices of order greater than $2$.
Comment: @JoséCarlosSantos I am saying that matrices satisfying the relation $AB-0=BA$ will give the  required result, but can we have some examples of the form $AB=-BA$ with  the product $AB$ is non zero matrix, now I am not very precise for $n$, I have edited the question.
Comment: @JoséCarlosSantos can i get this property $xy=-yx\neq 0$ in some other ring?
Comment: Either $|a|=1$, or $LM$ (and thus $ML$) is nilpotent. If $|a|=1$ then $a$ must be a root of unity (of order at most the dimension of $LM$ and $ML$), and the non-zero eigenvalues of $LM$ and $ML$ are also roots of unity. This is not difficult to prove from the fact that the non-zero eigenvalues of $ML$ and $LM$ must coincide, combined with $LM=aML$.
Answer: Take $M = \begin{pmatrix} 0 & 1 & 0 & \cdots & 0 \\
0 & 0 & 1 & \cdots & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
0 & 0 & 0 &\cdots & 1\\
0 & 0 & 0 &\cdots & 0\end{pmatrix}$
Then for any $a \neq 0 \in \mathbb{C}$, the matrix $aM$ is nilpotent of order $n$, so it is similar to $M$. So there exists $L \in \mathcal{GL}_n({\mathbb{C}})$ such that $aM = LML^{-1}$, so $LM=aML$.
Answer: We can do the $a = -1$ case in your title using Clifford algebras. Namely, the Clifford algebra $\text{Cl}(\mathbb{R}^2)$ can be given a presentation with two generators $i, j$ subject to the relations
$$i^2 = j^2 = -1, ij + ji = 0.$$
This is exactly the algebra of quaternions $\mathbb{H}$ and in particular is $4$-dimensional, with basis $\{ 1, i, j, ij \}$. The quaternions act on themselves by left multiplication and this gives a $4$-dimensional faithful representation of $\text{Cl}(\mathbb{R}^2)$ which is also its unique simple module. In this representation $i$ and $j$ act via the $4 \times 4$ matrices
$$L_i = \left[ \begin{array}{cccc} 0 & -1 & 0 & 0 \\
 1 & 0 & 0 & 0 \\
 0 & 0 & 0 & 1 \\
 0 & 0 & -1 & 0 \end{array} \right]$$
$$L_j = \left[ \begin{array}{cccc} 0 & 0 & -1 & 0 \\
 0 & 0 & 0 & -1 \\
 1 & 0 & 0 & 0 \\
 0 & 1 & 0 & 0 \end{array} \right].$$


HAProxy - ssl client ca chain cannot be verified


placing divs in a circle

Question: I have the following code which places divs around a circle.  However, I want to rotate the circle to change the sequence of divs from the top of the circle.
function drawCircle(selector, center, radius, angle, x, y)
    {

    var total = $(selector).length;
    var alpha = Math.PI * 2 / total;

    $(selector).each(function(index)
    {
        var theta = alpha * index;
        var pointx = Math.floor(Math.cos( theta ) * radius);
        var pointy = Math.floor(Math.sin( theta ) * radius );

        $(this).css('margin-left', pointx + x + 'px');
        $(this).css('margin-top', pointy + y + 'px');
    });

   }

  $(document).ready(function()
  {

   drawCircle('.box', 0, 250, 0, 500, 500);

  });

Answer: var theta = Math.PI/2 - alpha * index;

That was hard :p
Comment: Excellent..so how can I now flip the circle vertical (mirror in x axis)?
Comment: This is what I have now, but the div at the bottom needs to be at the top.  So flipping the circle vertically will  get the sequence of divs right (not rotating by 180 degrees though...) http://jsfiddle.net/xMdaf/
Comment: `Math.PI/3*2 + alpha * index`
Comment: Nope, that doesn't work.  I'll go back to my original question of how I can rotate the circle of divs by entering a number for the angle.
Comment: Sorry, I meant `Math.PI/2*3`
Answer: Why not rotate everything with CSS? Just use the transform property.

placing divs in a circle


Is it possible to parallelize python code using Pari via cypari2?

Question: I'm having trouble trying run a few loops in parallel when employing Pari via cypari2. I'll including a couple of small working examples along with the Tracebacks in case anyone has some insight on this.
Example 1 -- using joblib:
from cypari2 import Pari
from joblib import Parallel, delayed

def AddOne(v):
    return v + pari.one()

pari = Pari()
vec = [pari('x_1'), pari('x_2')]
print(vec)

#works
newVec = Parallel(n_jobs=1)(delayed(AddOne)(i) for i in vec)
print(newVec)

#doesn't work
newVec2 = Parallel(n_jobs=2)(delayed(AddOne)(i) for i in vec)
print(newVec2)

The output:
[x_1, x_2]
[x_1 + 1, x_2 + 1]
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/joblib/externals/loky/backend/queues.py", line 150, in _feed
    obj_ = dumps(obj, reducers=reducers)
  File "/usr/lib/python3/dist-packages/joblib/externals/loky/backend/reduction.py", line 247, in dumps
    dump(obj, buf, reducers=reducers, protocol=protocol)
  File "/usr/lib/python3/dist-packages/joblib/externals/loky/backend/reduction.py", line 240, in dump
    _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
  File "/usr/lib/python3/dist-packages/joblib/externals/cloudpickle/cloudpickle_fast.py", line 538, in dump
    return Pickler.dump(self, obj)
  File "stringsource", line 2, in cypari2.pari_instance.Pari.__reduce_cython__
TypeError: no default __reduce__ due to non-trivial __cinit__
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "min_jake_joblib.py", line 16, in <module>
    newVec2 = Parallel(n_jobs=2)(delayed(AddOne)(i) for i in vec)
  File "/usr/lib/python3/dist-packages/joblib/parallel.py", line 1016, in __call__
    self.retrieve()
  File "/usr/lib/python3/dist-packages/joblib/parallel.py", line 908, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/usr/lib/python3/dist-packages/joblib/_parallel_backends.py", line 554, in wrap_future_result
    return future.result(timeout=timeout)
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
_pickle.PicklingError: Could not pickle the task to send it to the workers.

Seems to be a problem with pickling the Pari objects, but is there any way around it?
Example 2 -- using multiprocessing:
from cypari2 import Pari
import multiprocessing

def AddOne(v):
    return v + pari.one()

pari = Pari()
vec = [pari('x_1'), pari('x_2')]
print(vec)

#doesn't work
if __name__ == '__main__':
    pool = multiprocessing.Pool(processes = 2) ## doesn't matter how many I use
    newVec = pool.map(AddOne, (i for i in vec))
    print(newVec)

It seg faults, but doesn't completely exit automatically, so I have to use Ctrl^C to kill it. The output:
[x_1, x_2]
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 576, in _handle_results
    task = get()
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
  File "cypari2/gen.pyx", line 4705, in cypari2.gen.objtogen
  File "cypari2/gen.pyx", line 4812, in cypari2.gen.objtogen
  File "cypari2/convert.pyx", line 557, in cypari2.convert.PyObject_AsGEN
cysignals.signals.SignalError: Segmentation fault
^CProcess ForkPoolWorker-1:
Process ForkPoolWorker-2:
Traceback (most recent call last):
  File "min_jake_multiprocessing.py", line 14, in <module>
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 356, in get
    res = self._reader.recv_bytes()
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
  File "src/cysignals/signals.pyx", line 320, in cysignals.signals.python_check_interrupt
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
  File "src/cysignals/signals.pyx", line 320, in cysignals.signals.python_check_interrupt
KeyboardInterrupt
KeyboardInterrupt
    newVec = pool.map(AddOne, (i for i in vec))
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 765, in get
    self.wait(timeout)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 762, in wait
    self._event.wait(timeout)
  File "/usr/lib/python3.8/threading.py", line 558, in wait
    signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.8/threading.py", line 302, in wait
    waiter.acquire()
  File "src/cysignals/signals.pyx", line 320, in cysignals.signals.python_check_interrupt
KeyboardInterrupt
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/util.py", line 300, in _run_finalizers
    finalizer()
  File "/usr/lib/python3.8/multiprocessing/util.py", line 224, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 695, in _terminate_pool
    raise AssertionError(
AssertionError: Cannot have cache with result_hander not alive

I suppose someone will tell me to use sympy or some other symbolic algebra package instead, but the symbolic algebra I need to do is quite complex and Pari can handle it extremely well. However, in the end I'd like to be able to process a queue of class objects that contain Pari objects in parallel. Any thoughts/suggestions are appreciated.
Comment: I've tried your second example in the isolated environment provided by `python:3.8-slim` docker image. It works fine for Python 3.8.12, pari-gp 2.13.2, and cypari2==2.1.2 on Debian GNU/Linux 11.
Answer: Well, this isn't a full answer, but it works for me so I wanted to share in case anyone else runs into this issue.
The first issue appears to be that the versions of libpari-dev and pari-gp on the apt repository were too old. The apt repository contains version 2.11 whereas the version on Pari's git repository is version 2.14. Uninstalling and following the instructions from here to install from source fixed most of my problems.
Interestingly, I still needed to install libpari-gmp-tls6 from the apt repository to get things to work. But, after that I was able to get the test examples above to run. The example using multiprocessing ran successfully without modification, but the example using joblib required the use of the "threading" backend in order to run.