SMTP, IMAP, and Web Issues on SBS 2003

Question: We just started having issues as of last night. Basically, all was good until I tried to add a users account to someones phone. It wasn't working and then noticed some activesync errors on the box. Per a KB article, it mentioned to add some reserved ports into the registry to resolve, and if the events continue to occur to remove the patch 951746 or 951748. Also, I can't get on the web whatsoever, but I can ping google and get a response.
In any case, I'm having issues now in that since last night around this time SMTP and IMAP will not start. Both give an error like "The service could not bind instance 1.  The data is the error code.".
I checked netstat, but I don't see anything using or listening on SMTP or IMAP. I also tried to telnet to port 25, but don't get any connection.
There are no firewalls or AV enabled. Any help greatly appreciated while I wait on MS to give me a call back after opening this ticket.
Comment: Here is the article of the initial issue: http://blogs.technet.com/b/sbs/archive/2008/07/17/some-services-may-fail-to-start-or-may-not-work-properly-after-installing-ms08-037-951746-and-951748.aspx
Answer: Without having any knowledge of the Patches you mentioned, I would check to make sure DNS is running on the server.  It sounds to me like somehow the bindings got screwed up and caused the system services (DNS, SMTP, IMAP etc) to either go offline or refuse to start.  All things being equal, check that you didn't update the Network Card Driver. And verify that it has the correct address.
If SMTP isn't starting then trying to telnet to it will not work.  But it is usually a good test to try it locally on the server back to itself using the loopback address (127.0.0.1).  If that works, then it is probably just a setup issue.  If it doesn't, then there is probably a service failure or it is set to NOT respond on 127.0.0.1.
I did have one phone (one version of Motorola Android) that decided to slam the ActiveSync service.  Something happened with the Mobile Device Partnership that caused it to repeatedly hit port 443.  So might be worth taking a look at your web logs as well.  (but that shouldn't have anything to do with SMTP and IMAP)
Comment: The SMTP service in services does start. The SMTP Virtual Server in Exchange does not. However, there still is nothing using port 25 it seems from netstat and telneting to port 25 on the localhost fails.
Comment: What does the event log say about the SMTP Virtual Server?
Answer: It turns out there is an issue in SBS 2003 after applying certain patches. The DNS service eats up many available ports. Unfortunately, they were not showing as being used via netstat or TCPview. After shutting down the DNS Server service I was able to start the other services. I then restarted DNS and it was okay.
I've still got a ticket open looking into a real fix, but this is a valid workaround.

SMTP, IMAP, and Web Issues on SBS 2003


Formula to copy Column A to Column B without blanks and sorted?

Question: I have the following column, A
C
R

H

Z

T

I would like a formula that automatically copies the data from A to B so that the data has no empty elements and it is sorted.
Column B should then look like:
C
H
R
T
Z

After that, if I were to add an element such as F to column A, such as:
C
R

H

Z
F
T

Column A would automatically update as:
C
F
H
R
T
Z

How can this be done with formulas (non-VBA code)? I'd like it to happen automatically (meaning i don't have to keep going to filter, sort every time a new piece of data is added to column A)
Thanks
Comment: To do it with formulas you will require a helper column.
Comment: @ScottCraner That's totally okay. How would I do it even with helper columns?
Answer: You will need to create a helper column, with this array formula:
=SUMPRODUCT(IFERROR(CODE(UPPER(MID(A1,ROW(INDIRECT("1:"& MAX(LEN($A$1:$A$9)))),1))),10)*100^(MAX(LEN($A$1:$A$9))-ROW(INDIRECT("1:"& MAX(LEN($A$1:$A$9))))))

Being an array formula it needs to be confirmed with Ctrl-Shift-Enter instead of Enter when exiting edit mode.  IF done correctly then Excel will put {} around the formula.

Then you would use this formula to sort and filter:
=IFERROR(INDEX(A:A,AGGREGATE(15,6,ROW($B$1:$B$9)/((AGGREGATE(15,6,$B$1:$B$9/($A$1:$A$9<>""),ROW(1:1))=$B$1:$B$9)*($A$1:$A$9<>"")),1)),"")

One note, this may implode with large strings.
Comment: Wow that formula is insane. I'd like to see if there's a simpler way but can you explain what exactly you're doing in those formulas?
Comment: @Hatefiend the first converts the letters to numbers.  It goes character by character and creates a number based on the code for each letter.  Then the second uses that code to sort and filter.  The AGGREGATES are doing a SMALL(IF()).  I hope you find an easier way but I doubt it,  If the strings were always single character then it would be a lot easier and not need the helper column, but I assumed the example was an over simplification.
Comment: So aggregate is what is sorting the numerical values lowest to highest?
Comment: Yes,  It is a way to use the SMALL formula with conditions without the need to use Ctrl-Shift-Enter.
Comment: Ah so for the final argument of aggregate, you use `ROW` meaning... get me the 1st smallest, 2nd smallest, etc?
Comment: Yes, it finds the first row where the condition is met and returns that row number to the INDEX.
Answer: You can use this array formula and drag it down, A1 your first Data in column A, in B1 write helper column to sort values:
=IF(COUNTIF($A$1:$A$21,"<="&A1)>0,COUNTIF($A$1:$A$21,"<="&A1),999999999)
and drag it down
In C1 write   =IF(INDEX(A:A,MATCH(SMALL(IF(B:B<>0,B:B,999999999),ROW(A1)),B:B,0))=0,"",INDEX(A:A,MATCH(SMALL(IF(B:B<>0,B:B,999999999),ROW(A1)),B:B,0)))
Press Ctrl+ Shift+Enter at the same time
IF will check for non empty cells and return the corresponding row
Small will find the upper non empty cell and move with the row,in row1 Small(,1)
in row10 small(,10)
INDEX(A:A) is your initial column
IFERROR in case the cells in A are empty it will return "" empty in column B
You have to drag the formula down even if you have empty cells in A, in that case when you write in the cells the formula will autofill in column B
9999999 it will not affect the small function
Without helper column you will need to sort the Data or use it unsorted but remove blank
You can use in B1:
=IFERROR(INDEX(A:A,SMALL(IF(A:A<>"",ROW(A:A),999999999),ROW(A1))),"")
Also array formula
Press Ctrl+ Shift+Enter at the same time
and drag it down
Answer: Another idea assuming your column contains only single characters,

Formula in column B,
=IFERROR(CODE(A:A),"")
Formula in column C,
=IFERROR(CHAR(SMALL(B:B,ROW())),"")
When you enter the formula you should not be doing it cell by cell or by dragging it. Select the entire column, paste the formula in formula box and press CTRL + ENTER. This will apply the formula for all cells in that column.


How to apply some function to each line of file in Haskell

Question: Let's say I have a parseMessage function, and I want it to apply to every line of file and get array of results. How can I do that?
parseMessage :: String -> SpecificDataStructure

type Path = String
parseFile :: Path -> [SpecificDataStructure]
parseFile what_do_I_need_here?

Answer: First, you have to recognize that parseFile has to operate in the IO monad, since it involves file I/O. This means extracting the contents from the IO action produced by readFile, then packaging the results back into an IO action. Here it is first in two steps, separating the pure computation from the IO actions:
parseContents :: String -> [SpecificDataStructure]
parseContents contents = map parseMessage (lines contents)

parseFile :: String -> IO [SpecificDataStructure]
parseFile fileName = readFile fileName >>= return . parseContents

and here it is written as one function
parseFile :: String -> IO [SpecificDataStructure]
parseFile fileName = do
                   contents <- readFile fileName
                   let parsed = map parseMessage (lines contents)
                   return parsed

Answer: You might be looking to do something like this:
parseFile = do
    f <- readFile "myfile.txt"
    map myFunc $ lines f

lines splits a file into a list of Strings, and map lets you apply a function (supplied in the first argument) to each element of the list. The result is a list of values. (:t map in GHCi yields map :: (a -> b) -> [a] -> [b]).
Answer: First, get the contents of the file using readFile
contents <- readFile "filename"

Then, split the file into lines using the function appropriately named function
let fileLines = lines contents

Finally, convert each line to your structure
let objects = map parseMessage fileLines

This will be an array of structures representing the original file.


Find MLE and CI for $\psi = \mathbb{P}_\theta(X_1>0)$ when $X_1,...,X_n \sim \text{IID } \mathcal{N}(\theta,1)$

Question: Question: Let $X_1,\cdots,X_n \sim \text{IID }\mathcal{N}(\theta,1)$ where $\theta\in\mathbb{R}$ is unknown and let $\psi = \mathbb{P}_\theta(X_1>0)$. Find the maximum likelihood estimator $\hat{\psi}$ of $\psi$ and find a 95% confidence interval for $\psi$.

Comments: I think I understand the first part of the two part question. From the log-likelihood function, it can be shown that $\hat{\theta}_{MLE}=\bar{X}_n=:\sum_{i=1}^n X_i/n.$ Then, with the invariance property of the MLEs, we have
$$\hat{\psi}_{MLE} = \mathbb{P}\left(\mathcal{N}(\bar{X}_n,1)>0\right).$$
My question is how do we construct the confidence interval? I am having difficulty calculating the any of the moments $\mathbb{E} \hat{\psi}_{MLE}$ and $\mathbb{E} \hat{\psi}_{MLE}^2$ and hence $\mathbb{V}(\hat{\psi}_{MLE}).$
Since $\bar{X}_n\sim \mathcal{N}(\theta,1/n)$, letting $f_{\bar{X}_n}(x)$ be the density function, I have tried using Tonelli's to switch the order of integration on the following expression
$$\mathbb{E} \hat{\psi}_{MLE}=\int_{\mathbb{R}} \mathbb{P}\left(\mathcal{N}(x,1)>0\right) f_{\bar{X}_n}(x)dx=\int_{\mathbb{R}}\int_0^\infty \frac{1}{\sqrt{2\pi}}e^{-(y-x)^2/2}dy \frac{1}{\sqrt{2\pi n}}e^{-(x-\theta)^2/(2\sqrt{n})}dx,$$
but Tonelli's seems to not be the correct route. How do I obtain the variance for the confidence interval?
Edit: Further, I am interested in finding the (nondegenerate) asymptotic distribution of $\hat{\psi}_{MLE}$.
Answer: $\psi$ is a smooth function of $\theta$, so the delta-method says
$$\sqrt{n}(\psi(\hat\theta)-\psi(\theta))\stackrel{d}{\to} N(0, \sigma^2)$$
where $\sigma^2$ is the limiting value of $n\psi'(\theta)^2\mathrm{var}[\hat\theta]$
You know $\mathrm{var}[\hat\theta]=1/n$, so you just need $\psi'(\theta)$, which is $\phi(0-\theta)$ by the fundamental theorem of calculus (or something very close to it).
And a check
<code>> theta<-1
>  n<-50
>  barx<-rnorm(100000,m=theta,s=1/sqrt(n))
>  psi<-function(theta) pnorm(0,theta,1)
> psi(1)
[1] 0.1586553
> mean(psi(barx))
[1] 0.1609813
> dnorm(0,1,1)^2
[1] 0.05854983
> var(psi(barx))*50
[1] 0.05881804
</code>
Comment: Yes! I keep forgetting to check for the delta method. However, to construct the confidence interval for fixed $n$, do we not need to find the standard error for fixed $n$? I understand that it will converge to the limiting variance from the delta method but how does one get an expression for the confidence interval for fixed n?
Comment: The fixed-$n$ variance is approximately $1/n$ times the asymptotic variance -- that's how my simulation check worked, since I was comparing $n$ times the simulation variance to the asymptotic variance.
Comment: Thanks for the suggestion. Any ideas on getting an exact expression?
Comment: I would be a bit surprised if there were a closed-form expression for the finite-sample variance.


Which definition of dimension came first?

Question: In my algebraic geometry class, the dimension of an affine variety $X=V(I)$ was defined as the supremum of the length of chains of prime ideals in the coordinate ring $R=k[x_1,\ldots,x_n]/\sqrt{I}$, or, the Krull dimension of $R$. A few minutes later, we were given the definition in terms of transcendence degree: the dimension of $X$, if $X$ is integral, is the transcendence degree of $K(R)$ over $k$.  I have also seen the dimension defined in terms of the degree of Hilbert polynomials (Theorem C, page 225 in Eisenbud).
My question: which of these came first and why? Why were the other definitions developed? What did they allow us to do more easily or effectively that previous definitions did not?
Comment: Actually, if you have Eisenbud at hand, you can read chapter 8, which gives a short history of dimension in algebraic geometry.
Comment: Search this - http://www.encyclopediaofmath.org/index.php/Commutative_algebra - page for the word "dimension" - it containts a lot of interesting commutative algebra history. In particular, it seems like dimension were originally defined as the transcendence degree of $K(R)/k$.
Comment: I think you're right: "Beginning with Kronecker and Lasker's dimension, defined then as the transcendence degree of the quotient ring corresponding to a prime ideal in a ring of polynomials, the modern combinatorial definition of dimension was subsequently proposed by W. Krull."
Comment: @Andrew Well this is embarrassing: I cited Eisenbud in my question but didn't think to turn back a few pages. Thanks. If you convert this to an answer I'll accept it.
Answer: Chapter 8 of Eisenbud has a short history of dimension in algebraic geometry, even giving axioms for a theory of dimension. The historical order seems to be transcendence degree (think meromorphic functions on a Riemann surface), Krull dimension, then Hilbert functions. In particular, Eisenbud mentions that, though one might suspect differently, the most powerful computation method today is via the Hilbert function, through applications of Groebner bases.
I also tracked down a quote that first came to mind regarding the Zariski tangent space, from Eisenbud-Harris, p.28: "[...] this was some years after Krull had introduced the notion of a regular local ring to generalize the properties of polynomials rings, one of the rare cases in which the algebraists beat the geometers to a fundamental geometric notion."


ID sendo criado e não começando com 0 - Oracle 11g

Question: Estou estudando Banco de Dados, usando Oracle 11g e fazendo as tabelas no SQLPLUS.
Eu crio a tabela:
CREATE TABLE COMPRAS (ID NUMBER PRIMARY KEY, VALOR NUMBER, DATA DATE,
OBSERVACOES VARCHAR2(30), RECEBIDO CHAR CHECK (RECEBIDO IN (0,1)));

E logo após, crio uma sequência:
CREATE SEQUENCE ID_SEQ;

Com isso, eu vou inserir os dados com este comando:
INSERT INTO COMPRAS (ID, VALOR, DATA, OBSERVACOES, RECEBIDO) VALUES
(ID_SEQ.NEXTVAL, 100, '12-MAY-2007', 'Compras de Maio', '1');

E para ver os dados inserido na tabela, eu rodo:
SELECT * FROM COMPRAS;

Comment: Provavelmente ocorreram dois erros antes de você acertar o comando INSERT. Mesmo quando ocorre um erro a sequencia é atualizada e por isso podem existir buracos.
Comment: Ola @Vagner, não altere sua pergunta para indicar que você conseguiu achar uma solução, fico feliz em saber que você quer ajudar outros com o mesmo problema, nesse caso a melhor opção é você responder sua própria pergunta, mas tarde o sistema vai liberar você para aceitar sua própria resposta. Dica: como você achou a solução na documentação vale a pena você adicionar o link da documentação na sua resposta. =D --- Vou reverter sua edição, você pode ver-la [aqui](https://pt.stackoverflow.com/posts/403968/revisions)
Answer: A solução que resolve este problema é que a documentação do Oracle 11g é diferente das demais pelo visto, dando uma pesquisada nas documentações eu encontrei que não é só preciso criar a sequência com este comando abaixo:
CREATE SEQUENCE ID_SEQ;

Na documentação do Oracle 11g, é preciso fazer desta forma, segue abaixo:
CREATE SEQUENCE ID_SEQ START WITH 1 INCREMENT BY 1 NOCACHE NOCYCLE;


When does a closed contour integral vanish?

Question: I've had some issues with Cauchy's Thorem for a little while now. Namely, when does a closed contour integral nrcessarily equal 0.
The statement I have learned is:
If $f $ is analytic on a simply-connected domain, $E $, and $C $ is a piecewise smooth closed curve on $E $ then:
$$\oint_C f(z)dz = 0$$
My trouble comes up with the simply-connected criteron, so I was hoping that someone could give me an answer to this:

If $f $ is analytic on a connected domain $E $ with a hole centered at $z_0$, and $C$ is a closed curve which contains $z_0$, but does not intersect it, can we still say that
  $$\oint_C f (z)dz = 0$$

Personally, I see no reason to think this is untrue. It seems to me that Cauchy's Theorem requires simply connected domains just so that you can say every closed curve vanishes. But I suspect that a closed contour integral should still equal $0$ if the integrand is analytic and the curve stays completely within the domain. Is this true, or am I looking at things incorrectly?
Comment: Is $\{z:z\ne0\}$ a connected domain with a hole centred at $0$ and is the unit circle a closed curve that contains $0$ but does not intersect it?
Comment: @LordSharktheUnknown: Yes, that would be an example of what I am talking about. Would the integral of any analytic function along the unit circle be $0$ in such a domain?
Comment: Short answer:no! I don't have time to write a longer answer, but look at residues.
Comment: @Henrik: Thanks, I'll look into that (I haven't learned about residues as of yet)
Answer: You are looking at things incorrectly. The integral
$$\int_{\{z:|z|=1\}}\frac{1}{z}$$ does not vanish
(Aassume $f(z) =\frac{1}{z}$ to be defined on $\mathbb{C}\backslash \{0\}$).
Comment: Assuming you're taking the domain to be something which does not contain $0$ here, I think that cleared things up a bit now. So in reality, we could take a function that was analytic at all points on the interior of the closed curve, and *this* closed contour integral would vanish, even if the domain wasn't simply connected, because we could *extend* the domain to be something which was simply connected (with $f $ analytic on it). Is that correct?
Comment: For example, integrating $f (z) = z $ on the same closed curve in your example would be $0$, even if the domain we were integrating on didn't necessarily contain all points inside the curve (i.e. wasn't simply connected)
Comment: I'm not sure I got your statement correctly, but in fact, if $f$ is defined e.g. on the unit disc without a point and can be analytically continued to the domain including the point, then the integral along any closed curve would vanish. This is because you cannot distinguish between $f$ and its continuation along the curve, and the statement is true for the continuation.
Comment: Yes, that is exactly what I meant! Thank you for clearing that up for me!


Can I use a 3DES key within google KMS HSM?

Question: I have been looking across the different HSM provides alike Azure, AWS, IBM and Google but I haven't find any documentation about google HSM explicitely supporting 3DES keys.
However I can see the type
<code>"CRYPTO_KEY_VERSION_ALGORITHM_UNSPECIFIED"
</code>
in https://cloud.google.com/kms/docs/reference/rest/v1/CryptoKeyVersionAlgorithm
This type is used to create a key in the KMS.
Does Google KMS HSM support 3DES?
Answer: I recommend that you upvote the existing feature request for 3DES here: https://issuetracker.google.com/issues/123697325 (simply click the star).
Next, could you explain whether 3DES would be sufficient to satisfy your use case?  I would expect that there may be other requirements beyond 3DES such as DUKPT that you may require for your use case.
Answer: It does not support 3DES as a key type you can create and use at this time. We'd be interested to know more about why you would want 3DES.
Comment: We need 3DES encryption for POS PIN translation. A process similar to this: [link](https://madrock.net/tag/dukpt/)

So we need a HSM service that cyphers the PIN with 3DES.
So much thanks for answering.
Comment: If there is any way for getting 3DES encription from GCloud KMS HSM please let us know.
Comment: Thanks for the info on your use case! If you don't absolutely need the key protected by an HSM, you can do the 3DES in software. Alternately, we can discuss how to rack your own HSM close to Google Cloud so you can use it to secure the key.


How to close modal after form submit?

Question: I am building an entity reference widget, which displays the reference link, next to the input field, to the target entity's edit form.
I make it to open in a modal, with the following attributes on the link:
 'class' => ['use-ajax'],
          'data-dialog-type' => 'modal',
          'data-dialog-options' => json_encode([
            'height' => 700,
            'width' => 900,
          ]),

That works perfectly. However when I click submit on the modal form, I got redirected to the referenced entity view page, instead of the modal being closed and stay on the original form, where this widget is being used.
Is there any special attributes (like "data-dialog-type") which tells to the modal form's submit button to close the modal instead of redirecting to the original view?
UPDATE
Adding this to the URL actually pretty much solves the problem:
    'query' => [
      'destination' => $this->requestStack->getCurrentRequest()->getRequestUri(),
    ],

This redirects the to the original page, with the potential values being updated (referenced item title), which is actually a pretty good thing!
That being said, I am still interested in the "ajax submit" version!
Answer: You need to implement and use your own EntityEditForm - is this the case ?
Then skip to the last code block :D
Otherwise you'll need to accomplish this first.
See this answer (Overriding an entity add/edit form through alterroute) for an approach to 'inject' a custom form. Here the crucial code to it (adapted to NodeForm):
modules/custom/form_overwrite/form_overwrite.services.yml
services:
    route_subscriber:
        class: Drupal\form_overwrite\Routing\RouteSubscriber
        tags:
 - {name: event_subscriber }

modules/custom/form_overwrite/src/Routing/RouteSubscriber.php
 <?php

/**
 * @file
 * Contains \Drupal\form_overwrite\Routing\RouteSubscriber.
 */

namespace Drupal\form_overwrite\Routing;

use Drupal\Core\Routing\RouteSubscriberBase;
use Symfony\Component\Routing\RouteCollection;

/**
 * Listens to the dynamic route events.
 */
class RouteSubscriber extends RouteSubscriberBase {

  /**
   * {@inheritdoc}
   */
  protected function alterRoutes(RouteCollection $collection) {
     if ($route = $collection->get('entity.node.edit_form')) {
  $route->setDefault('_entity_form', '\Drupal\form_overwrite\Form\NewEditForm');
     }

   }
}

modules/custom/form_overwrite/src/Form/NewEditForm.php
<?php

namespace Drupal\form_overwrite\Form;

use Drupal\node\NodeForm;

/**
 * Form handler for the node edit forms.
 */
class NewEditForm extends NodeForm {}

There you make sure the #ajax is set, like:
<?php

public function form(array $form, FormStateInterface $form_state) {
   ...
   $form['actions']['DESIRED_EXISTING_ACTION'] = array(
      ...
      '#type' => 'submit',
      '#ajax' => array(
        'callback' => '::ajaxSubmit',
        'event' => 'click',
      ),
      ...
    );
   ...
}

and overwrite the ajaxSubmit function, adding the CloseModalDialogCommand, like:
public function ajaxSubmit(array &$form, FormStateInterface $form_state) {
    $response = new AjaxResponse();

    if ($form_state->getErrors()) {
      ...
    }
    else {
      ...
      $response->addCommand(new \Drupal\Core\Ajax\CloseModalDialogCommand\CloseModalDialogCommand());
    }

    return $response;
  }

Comment: I want to use general entity forms, like node, and custom forms. Am not able to alter their submit (via hook) to rather make ajax response, since the modal somehow using it's own. Any suggestion on that?
Comment: You could fe. alter the route of the entity type your dealing with and set there your 'custom' form. see the newly linked answer.
Comment: In the link there is a specific entity type. Here this is dynamic, so I won't able to extend from an exact Form class. Probably I will have to use composition here? Could I somehow alter the modal window process, which takes the form submit fields and ajaxifies them?
Comment: You'll need to do it for every Form class .. but most of them could be reused, and just the routing part will blow up .. May be a better way thoe ...
Comment: Well, I don't want to write a code, which will need to be extended each time there is a new content entity type: \
Comment: well with the nodeEdit form you'll have them all - no ? (if you mean node type only by content entity type)
Comment: True, bad wording by me: I want to make it usable by any entity type , not just node!
Comment: So actually, this is not the answer I was looking for.
Answer: I solved this by only using a event subscriber in a custom module.
modalCloseSubscriber.php
<?php

namespace Drupal\custom_module\EventSubscriber;

use Drupal\Core\Ajax\AjaxResponse;
use Drupal\Core\Ajax\CloseDialogCommand;
use Drupal\Core\Session\AccountProxyInterface;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpKernel\Event\FilterResponseEvent;
use Symfony\Component\HttpKernel\KernelEvents;

/**
 * Custom module event subscriber.
 */
class ModalCloseSubscriber implements EventSubscriberInterface {

  /**
   * Current logged in user.
   *
   * @var \Drupal\Core\Session\AccountProxyInterface
   */
  protected $currentUser;

  /**
   * Constructs event subscriber.
   *
   * @param \Drupal\Core\Session\AccountProxyInterface $current_user
   *   Current logged in user.
   */
  public function __construct(AccountProxyInterface $current_user) {
    $this->currentUser = $current_user;
  }

  /**
   * Kernel response event handler.
   *
   * @param \Symfony\Component\HttpKernel\Event\FilterResponseEvent $event
   *   Response event.
   */
  public function onKernelResponse(FilterResponseEvent $event) {
    $response = $event->getResponse();
    if (!$response instanceof AjaxResponse) {
      return;
    }
    if ($response->getCommands()) {
      $commands = $response->getCommands();
      if (empty($commands)) {
        return;
      }
      elseif (!empty($commands)) {
        foreach ($commands as $command) {
          if ($command['command'] == 'insert' && $command['selector'] == '#layout-builder') {
            // leave blank for default modal window to close.
            $response->addCommand(new CloseDialogCommand());
          }
        }
      }
    }

  }

  /**
   * {@inheritdoc}
   */
  public static function getSubscribedEvents() {
    return [
      KernelEvents::RESPONSE => ['onKernelResponse'],
    ];
  }

}

This closes the modal after i create a block with layout builder.
Answer: I achieved it by hook_form_alter into the form, check if the current request is an Ajax request and then add a callback to fire the CloseDialogCommand().
/**
 * Implements hook_form_alter().
 */
function MYMODULE_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id) {
  if ($form_id === 'MY_FORM_ID') {
    $request = \Drupal::request();
    // Check if Ajax request.
    if ($request->isXmlHttpRequest()) {
      $form['actions']['submit']['#ajax']['callback'] = '_MY_MODULE_MY_FORM_submit_ajax_callback';
    }
  }
}

/**
 * Custom Ajax callback.
 */
function _MY_MODULE_MY_FORM_submit_ajax_callback(array &$form, FormStateInterface $form_state) {
  // If form is valid then close the dialog.
  if (!$form_state->hasAnyErrors()) {
    $response = new \Drupal\Core\Ajax\AjaxResponse();
    $response->addCommand(new \Drupal\Core\Ajax\CloseDialogCommand());
    return $response;
  }
  // Otherwise call the default #ajax callback.
  $form_object = $form_state->getFormObject();
  $response = $form_object->submitAjaxForm($form, $form_state) ;
  return $response;
}

Comment: I tried your solution. The node is saved properly but your custom ajax callback throws me the following error:
`Error: Call to undefined method Drupal\node\NodeForm::submitAjaxForm() in cs_modals_submit_ajax_callback() (line 42 of .../public_html/modules/cs_modals/cs_modals.module)
`
Comment: @Apoc – Ah maybe because I was on a Webform. If everything works for you except this error, simply remove the last three lines of code?!

How to close modal after form submit?


Is 'file path' required in hook_menu()?

Question: I just started learning Drupal 7 Module development. In a tutorial I watched, the person said that

"In hook_menu(), if we have a 'file' attribute then the 'file
  path' is a must and drupal wont work if it is not declared in drupal 7."

But when I read a book, it said that

The 'file_path' is not required in drupal 7 provided that the file
  is present in the same directory.

As you can see, its a bit confusing. So my questions are
1) Is 'file path' required in Drupal 7 ?
2) When to use 'file path' ?
I've posted a snippet of the hook_menu implementation :
function modulename_menu() {
  $items['...'] = array(
    ....
    ....
    'file' => 'modulename.admin.inc',
    'file path' => drupal_get_path('module','modulename'),
  );
  return $items;
}

Note : Assume that the file is present in the same modules folder
Answer: The first sentence you quoted is wrong. "file path" is not a must, if you set "file"; in fact, by default Drupal will take its value is the directory containing the module implementing the menu. If you want to use a different directory, such as drupal_get_path('module', 'mymodule') . '/includes'), then you can set "file path."
_menu_router_build() contains the following code.
// Calculate out the file to be included for each callback, if any.
if ($item['file']) {
  $file_path = $item['file path'] ? $item['file path'] : drupal_get_path('module', $item['module']);
  $item['include file'] = $file_path . '/' . $item['file'];
}

$item['module'] is set by menu_router_build() using the following code.
  foreach (module_implements('menu') as $module) {
    $router_items = call_user_func($module . '_menu');
    if (isset($router_items) && is_array($router_items)) {
      foreach (array_keys($router_items) as $path) {
        $router_items[$path]['module'] = $module;
      }
      $callbacks = array_merge($callbacks, $router_items);
    }
  }

A module could implement hook_menu_alter() to alter $item['module'].
Comment: Note for future self: clear caches in-between altering file path.
Answer:
1) Is 'file path' required in Drupal 7 ?

Required if file being included is not in module path. Its not mandatory if the file is included in the module itself.
file path: The path to the directory containing the file specified in "file". This defaults to the path to the module implementing the hook.

2) When to use 'file path' ?

When you want to include a file outside your module.

Is 'file path' required in hook_menu()?


Script compilation error at declaring repository on gradle

Question: I am trying to declare some repositories in my build.gradle.kts, but whenever I do gradle bootRun I get a script compilation error that I cannot get rid of.
 * What went wrong: Script compilation error:

   Line 13:             url "https://plugins.gradle.org/m2/"
                            ^ Unexpected tokens (use ';' to separate expressions on the same line)

The lines causing the error are these:
buildscript {
    repositories {
        mavenCentral()
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
        classpath(kotlin("gradle-plugin", version = "1.3.61"))
    }
}

I cannot understand what is wrong.
I'm running this from a Powershell.
Gradle -version output:

Gradle 6.6.1

Build time:   2020-08-25 16:29:12 UTC Revision:
f2d1fb54a951d8b11d25748e4711bec8d128d7e3
Kotlin:       1.3.72
Groovy:       2.5.12
Ant:          Apache Ant(TM) version 1.10.8 compiled on May 10 2020
JVM:          1.8.0_252 (Amazon.com Inc. 25.252-b09)
OS:           Windows 10 10.0 amd64
Answer: Solved changing the conflicting line to
maven {
    url = uri("https://plugins.gradle.org/m2/")
}

Syntax changed due to using Kotling DSLs instead of Groovy, something I wasn't aware of.


Get Action arguments similar to webapi in mvc action filter

Question: I have an API method that accept a Dictionary<string, object> that is returned from HttpActionExecutedContext.ActionContext.ActionArguments property (used within a WebApi action filter). I need to construct a similar dictionary from ActionExecutedContext in a MVC action filter so that I can use same API call from the mvc filter as well.
I tried several approaches but they are not very straitforward. If there is easy way to construct the actionArguments dictionary from ActionExecutedContext   please let me know.
    //webapi - working fine
    public override void OnActionExecuted(HttpActionExecutedContext context)
    {
        Call_A(context.ActionContext.ActionArguments)
    }

    //mvc- need to find a way to get argument dictionary
    public override void OnActionExecuted(ActionExecutedContext context)
    {
        Call_A(context.?)
    }

Answer: Not sure if this exists for OnActionExecuted, but it definitely exists for OnActionExecuting.   If you do need it on Executed (if it doesn't exist, I don't think it does) then you can store it in the filter temporarily, eg:
public IDictionary<string, object> actionParams { get; set; }

public override void OnActionExecuting(ActionExecutingContext context)
{
    this.actionParams = context.ActionParameters;
}

public override void OnActionExecuted(ActionExecutedContext context)
{
    foreach (var param in this.actionParams) {
        // param.Key
        // param.Value
    }
}

Comment: Thanks for the tip. I was aware about the ActionParameters in ActionExecutingContext. But did't have any idea how to use it. Your approach sounds reasonable.
Comment: I believe that this code is bugged because your class level variable actionParams is shared by all requests/threads. By the time request 1 enter OnActionExecuted, request 2 may have already changed the actionParams variable.
Comment: Now at least in Web API 2 you have the ability to simply use actionExecutedContext.ActionContext.ActionArguments
Comment: Only 1 instance of the class is created; not 1 instance per service call. I just confirmed this by calling an action method twice simultaneously.
Comment: `private Dictionary dict;

        public override void OnActionExecuting(HttpActionContext actionContext)
        {
            dict = actionContext.ActionArguments;
        }
        public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
        {
            _logger.Debug(dict.Values.First());
        }`

First request is "A", second is "B". But "B" gets logged twice.


Interpretation of an I(2) process?

Question: I know that an ARIMA(0,0,0) process is white noise and ARIMA(0,1,0) is a random walk,
Is there an interpretation of what an ARIMA(0,2,0) process is?
Comment: You have a second order random walk.  This means that the expected value of second order differences (analogous to the second derivative) has expectation zero.  Hence, the sample path "is expected" to be smooth.  Therefore, second order random walks are useful building blocks for modelling smooth functions.
Comment: Related: [Interpretation of an I(3) process?](https://stats.stackexchange.com/q/314010/1352)
Answer: One interpetation is that the rate of change is random walk.
It's like a free fall where the gravitational force is stochastically changing.
If you drop the body on earth, it's moving according to the law:
$$\frac{d^2}{dt^2}x=g$$
where $g$ is a gravitational constant. This means that the body's acceleration is always the same and equal to $g$. In a discretized version this would be $$\Delta^2x_t=g$$
If you replace the constant with a random variable, you get I(2) process:
$$\Delta^2x_t=\varepsilon_t$$
It means that the acceleration is random, not constant anymore.
Here's a demo. We're shooting a canon at 45 degree angle to the right, the initial velocity is 100, gravitational constant is 9.81. The red line show the constant acceleration, and the black lines are of stochastic acceleration i.e. I(2):

Python code:
<code>import matplotlib.pyplot as plt
import numpy as np
import math

dt = 0.1
g = 9.81
angle = math.pi/4
v = 100

vx = math.acos(angle)*v
vy = math.asin(angle)*v

x, y = 0, 0
while (y>=0):
    plt.plot(x,y,'r.')

    dx = vx*dt
    dy = vy*dt
    dvy = -g*dt

    x = x + dx
    y = y + dy
    vy = vy + dvy

# I(2)

np.random.seed(0)
vx = math.acos(angle)*v
for i in range(4):
    g0 = g
    x, y = 0, 0
    vy = math.asin(angle)*v
    while (y>=0):
        plt.plot(x,y,'k.')

        g = g0 + np.random.randn()*20

        dx = vx*dt
        dy = vy*dt
        dvy = -g*dt

        x = x + dx
        y = y + dy
        vy = vy + dvy

plt.show()
</code>

Interpretation of an I(2) process?


Request headers are not parsing correctly using Nginx

Question: I just created a new Ubuntu instance on AWS. After installing Nginx request headers are not parsing correctly whenever I'm requesting using a device(Android/iOS).
Using postman I am able to get headers on node app but whenever I try with the device it's showing undefined. Using public DNS I'm able to get headers so I'm expecting that issue is with Nginx configuration.
Here are the Headers which I'm receiving on my Node js app from postman and android device respectively:
// using Postman
{ host: '<Host>',
 'x-real-ip': '<id-address>',
 connection: 'upgrade',
 'cache-control': 'no-cache',
 'postman-token': <token>,
 authorization: <auth_token>,
 'content-type': 'application/json',
 'user-agent': 'PostmanRuntime/6.2.5',
 accept: '*/*',
 'accept-encoding': 'gzip, deflate' }

// using android device
 { host: '<host>',
  'x-real-ip': '<ip-address>',
  connection: 'upgrade',
  'accept-encoding': 'gzip',
  'user-agent': 'okhttp/3.10.0' }

As you can see in device headers authorization is not present so it is showing undefined.
Nginx configuration:
upstream backend {
    server 127.0.0.1:8000;
}
server {
if ($host = <host>) {
    return 301 https://$host$request_uri;
} # managed by Certbot

listen [::]:80;
listen 80;
server_name <host>;
return 404; # managed by Certbot
location / {
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_pass  http://backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_next_upstream error timeout http_502 http_503 http_504;
}
}

Please help me with this.
Headers coming from iOS device
http://<request-url>/?is_new_device=false&device_id=A2F6C479-2C52-4A43-ADB6-D86771431B5A
 ▿ url : Optional<URL>
  ▿ some : http://<request-url>/?is_new_device=false&device_id=A2F6C479-2C52-4A43-D86771431B5A
   - _url : http://<request-url>/?is_new_device=false&device_id=A2F6C479-2C52-4A43-D86771431B5A
 - cachePolicy : 0
 - timeoutInterval : 60.0
 - mainDocumentURL : nil
 - networkServiceType : __C.NSURLRequestNetworkServiceType
 - allowsCellularAccess : true
 ▿ httpMethod : Optional<String>
  - some : "GET"
 ▿ allHTTPHeaderFields : Optional<Dictionary<String, String>>
  ▿ some : 2 elements
   ▿ 0 : 2 elements
    - key : "Authorization"
    - value : "Bearer google-oauth2 <token>"
   ▿ 1 : 2 elements
    - key : "User-Agent"
    - value : "Impact/3.5.16 iPhone9,3 iOS/13.3 CFNetwork/1121.2.2 Darwin/19.2.0"
 - httpBody : nil
 - httpBodyStream : nil
 - httpShouldHandleCookies : true
 - httpShouldUsePipelining : false

Comment: Can you add log from android device with request headers? Or, just add nginx log and print all request headers in it.
Comment: Can you add more explanation of your flow, are you saying you are not using jnginx when request via node app?
Comment: Also I think you should change `proxy_set_header Host $http_host;` to `proxy_set_header Host $host;`
Comment: @Sohan yes.. every request is coming through Nginx to Node app.. Changed with $host.. not working
Comment: You are not using any proxy_pass right? Also there are no custom hearders in response?
Comment: What is your proxy_pass url? why i cannot see that when you are using above headers
Comment: yes, I'm using the proxy pass and authorization headers
Comment: Why I cannot see in nginx configuration?
Comment: @Sohan backend {
    server 127.0.0.1:;
}
Comment: @Talkerbox I added headers coming from iOS device in the question. Please check
Answer: My issue is solved by removing the below code from the Nginx configuration. Headers are not parsing when I redirect requests from Http to https.
if ($host = <host>)
{ return 301 https://$host$request_uri; } # managed by Certbot


Ao selecionar o chekbox preciso que a frase fique tachada. Segue img exemplo

Question:

                  Design a nice theme

                   2 mins

                  Make the theme responsive
                   4 hours

                  Let theme shine like a star
                   1 day

                  Let theme shine like a star
                   3 days

                  Check your messages and notifications
                   1 week

                  Let theme shine like a star
                   1 month

                  <input type="checkbox" value="">

                  Design a nice theme

                   2 mins

                  <input type="checkbox" value="">
                  Make the theme responsive
                   4 hours

                  <input type="checkbox" value="">
                  Let theme shine like a star
                   1 day

                  <input type="checkbox" value="">
                  Let theme shine like a star
                   3 days

                  <input type="checkbox" value="">
                  Check your messages and notifications
                   1 week

                  <input type="checkbox" value="">
                  Let theme shine like a star
                   1 month

Answer: Se baseando na sua marcação, pode-se usar a regra input[type=checkbox]:checked ~ .text para definir a estilização do elemento com classe text quando o checkbox próximo estiver checado:

input[type=checkbox]:checked ~ .text {
  color: #ccc;
  text-decoration: line-through
}

    <input type="checkbox" value="">

    Design a nice theme

     2 mins

    <input type="checkbox" value="">
    Make the theme responsive
     4 hours

    <input type="checkbox" value="">
    Let theme shine like a star
     1 day

    <input type="checkbox" value="">
    Let theme shine like a star
     3 days

    <input type="checkbox" value="">
    Check your messages and notifications
     1 week

    <input type="checkbox" value="">
    Let theme shine like a star
     1 month

Answer: Também dá para fazer pelo JavaScript manipulando a propriedade textDecoration

document.getElementById('texto').style.textDecoration = "line-through";
Esse texto vai ser riscado!!!

No seu exemplo:

document.querySelectorAll('input[type=checkbox]').forEach((elemento) => {
  elemento.addEventListener('change', (e) => {
    var span = e.target.parentNode.querySelector('.text');
    span.style.textDecoration = (e.target.checked) ? "line-through" : "";

  });
});

    <input type="checkbox" value="">

    Design a nice theme

     2 mins

    <input type="checkbox" value="">
    Make the theme responsive
     4 hours

    <input type="checkbox" value="">
    Let theme shine like a star
     1 day

    <input type="checkbox" value="">
    Let theme shine like a star
     3 days

    <input type="checkbox" value="">
    Check your messages and notifications
     1 week

    <input type="checkbox" value="">
    Let theme shine like a star
     1 month

Comment: Executei com o js e funcionou muito bem. Grato!


Top X-posts of the past week? SQL/PHP [figured it out]

Question: How can I get the top X posts of the past week? I have two tables: td_table - holds info about the post; vote_table - holds the info about the votes that have been made for the posts. The following gives me the top three posts (those that have most votes for), but it gives me the top three of all times.
$query = 'SELECT t.id, t.content, t.userid, t.time FROM tb_table t,
(SELECT *, count(*) AS num FROM vote_table GROUP BY voted_id ORDER BY num desc) u
WHERE u.vote_id=t.id LIMIT 3';

I tried using something like this:
SELECT * FROM vote_table WHERE t.time > date_sub(date(now()), interval 7 day)

But whenever I add this kind of WHERE condition to the above query it either dies (if I add it to the select from vote_table) or it loads an empty page (if it's and AND condition of the last where)...
So... how can I combine those? Call all the information from the query, but only those voted_ids that have been made for the past week?
Answer: Figured it out!
using_time >= (UNIX_TIMESTAMP() - ((60*60*24)*7))


Composer downgrading module version

Question: I wish to move the bootstrap theme from the repository
located in the folder theme/custom to be a dependency in composer.json
I've checked that the current version drupal/bootstrap is 3.16
I perform rm -rf web/theme/custom/bootstrap
composer require drupal/bootstrap:8.x-3.16
^^-> wrong syntax and have decided to just composer require drupal/bootstrap
Now I need to downgrade the version of drupal/bootstrap in composer.json
and perform composer update --lock
But I got an error:
Nothing to modify in lock file
Unable to find a compatible set of packages based on your non-dev requirements alone.
Your requirements can be resolved successfully when require-dev packages are present.
You may need to move packages from require-dev or some of their dependencies to require.

  Problem 1
    - Root composer.json requires drupal/bootstrap 3.16, found drupal/bootstrap[3.23.0] but it does not match the constraint.

Any hints would appreciate.
Comment: Does this answer your question? [How can I downgrade a module with Composer?](https://drupal.stackexchange.com/questions/297861/how-can-i-downgrade-a-module-with-composer)
Answer: What that I have done to fix it is obvious from the question
remove drupal/bootstrap from composer.lock and from composer.json
composer require drupal/bootstrap:3.16

Composer downgrading module version


How are the targets generated in deep Q-learning that the network can train on?

Question: I think I have a reasonable understanding of how neural networks and Q-learning work as separate concepts. But when combining the two to form deep Q-learning I struggle with understanding what exactly is used as a target when training the NN.
As I understand it the neural network takes a state/action pair as input and produces an estimation of the Q-value. So far so good. But in order to improve the estimation the network needs to compare its estimation with the 'true' value and then backpropagate on the error between them. But where does the true Q-value come from?!
I have read several blogs/papers about deep RL and they all swoosh past the details of the backpropagation as if the concept is obvious to everyone who's reading which is greatly annoying. So if someone could explain that specific part to me I would be grateful.
Answer: If I'm correct, you are struggling with the concept of the 'true' Q-value? I'm not an expert on back propagation, but I might be able to shine some light on the whereabouts of this 'oracle' Q-value!
The Q(s,a) describes the total expected reward starting in state s, performing action a, receiving reward r and subsequently following the current policy until termination.
In TD learning, one calculates Q(s,a), and subsequently (using the same approximator, like the same NN), the value of Q(s',a'), in which (s',a') are one time-step later than (s,a). Using a perfect Q-function, the difference between Q(s,a) and Q(s',a') should be exactly the direct reward received in going from s to s' by action a: $Q(s,a) = Q(s',a') +r$. Therefore, $\delta$ is computed as $\delta = Q(s',a')+r-Q(s,a)$, neglecting discount rate $\gamma$.
The trick here is that the 'real' $Q$ was never known, but the real reward $r$ is. Using $\delta$ as a measure for the error, the weights of your approximator can be adjusted, usually using a gradient descent type approach.
Did this help you at all?
Comment: Just came across this blog post, which addresses your issue directly: https://danieltakeshi.github.io/2016/12/01/going-deeper-into-reinforcement-learning-understanding-dqn/
Comment: Thank you very much. With this together with some other reading I think I'm slowly starting to get how deep RL works now!


Checking users current password when changing account settings

Question: I am building a form for a logged in user to change their account settings (email and password). To do this, they need to be able to confirm their current password. This is how I'm building my form in settings.ctp:

Your images

Account settings

Modify your account settings

        <?php echo $this->Session->flash(); ?>
        <?php
            echo $this->Form->create('User');
            echo $this->Form->input('currentPassword', array('type' => 'password'));
            echo $this->Form->input('username', array('disabled' => 'disabled', 'value' => $username));
            echo $this->Form->input('email');
            echo $this->Form->input('password', array('type' => 'password'));
            echo $this->Form->end('Update');
        ?>

and this is my controller action:
public function settings() {
    $this->set('title_for_layout', 'Account Settings');

    $this->User->id = $this->Auth->user('id');
    if ($this->request->is('post')) {
        if($this->Auth->user('password') == $this->request->data['User']['currentPassword']) {
            if ($this->User->save($this->request->data)) {
                $this->Session->setFlash('Your details have been saved');
                $this->redirect(array('action' => 'settings'));
            } else {
                $this->Session->setFlash('Your details could not be updated. Try again.');
            }
        } else {
             $this->Session->setFlash('Invalid password. Try again.');
        }
    }
}

However, the if() for a password check always evaluates to false and always shows the "invalid password" message. I'm guessing I'm not checking the password correctly, but I don't know the correct way to do it.
Also, I don't want the user to be able to change their username. I know I've set the form field to disabled, but if a user sends a post request to my settings action they could change their username. How do I stop the username from being updated with $this->User->save()?
EDIT: Looks like two problems, one, I wasn't hashing the password before the comparison and two, $this->Auth->user('password') is actually NULL. Which raises another question, how do I get the users hashed password from the database in order to compare it?
Comment: Is the password stored in your database using plaintext? If not, that would cause your validation check to fail.
Comment: It's using Cake's default hashing algorithm, which I believe is sha1?
Comment: you'll need to hash the `currentPassword` as well so that it matches whatever `Auth::user('password')` is
Comment: also note that even though the `username` is disabled; a determined user could modify it - so I would `unset` it before you save to prevent the username from being overwritten accidentally/on purpose.
Answer: If the password isn't stored in your database as plaintext then the password check will fail since you are not encoding / encrypting the data you are passed from the form. An easy way to check that (only on a test machine please) would be to replace $this->Session->setFlash('Invalid password. Try again.'); with
$this->Session->setFlash('The password should be ' . $this->Auth->user('password') . 'but it is actually ' . $this->request->data['User']['currentPassword']);

or something similar.
Hope it helps
edit:
If the password is stored using sha1, you can check it like this
if($this->Auth->user('password') == sha1($this->request->data['User']['currentPassword']))
{
     //code for successful password here
}

Comment: you might want to validate the user based on something else, like a session ID? The Auth component is probably in a Session object which would be too easy for a malicious user to hijack.
Comment: After doing that, I get this: `The password should be but it is actually password`. After doing a `var_dump` on `$this->Auth->user('password')` I see that it's NULL. Looks like Cake doesn't store the users password in the Auth component like it stores the username and user id. :(
Comment: I should add that I haven't used Cake before, just plain PHP.
Comment: If it doesn't show you password, use: $variable = $this->Auth->user(); debug($variable);.
Answer: You're correct about the Auth component not holding the password in it's data. Try this code out:
public function settings() {
    $this->set('title_for_layout', 'Account Settings');

    if ($this->request->is('post')) {
        $user = $this->User->findById($this->Auth->user('id'));
        if($user['User']['password'] == AuthComponent::password($this->request->data['User']['currentPassword'])) {
            if ($this->User->save($this->request->data)) {
                $this->Session->setFlash('Your details have been saved');
                $this->redirect(array('action' => 'settings'));
            } else {
                $this->Session->setFlash('Your details could not be updated. Try again.');
            }
        } else {
             $this->Session->setFlash('Invalid password. Try again.');
        }
    }
}

And my suggestion for stopping them from changing their username is to just discard that input altogether? You could simply echo their username for them rather than have a disabled input box.
Comment: I didn't consider how the save() function handles saving, unset() is exactly what you should be using in that case. Also, if you'd like to optimize the query you could use containable to make sure it only returns the data you need.
Comment: Thanks, works. I could disable the input but the user will still be able to change their username by sending a POST request to that page with a `username` parameter, because of the way Cake handles user saving. I've opted to use `unset()` instead, to make sure even if the `username` field is sent it's discarded.


TypeScript recursively convert object keys from PascalCase to camelCase format

Question: I need to create a function to modify keys of object from PascalCase to camelCase format. Ie
const input = {
    FirstName: 'John',
    LastName: 'Smith'
};

const expectedOutput = {
    firstName: 'John',
    lastName: 'Smith'
};

// function looks as follows
function pascalToCamelCase<T>(input: T): CamelCaseKeys<T> {
    if (typeof input !== 'object' || input === null) {
        return input;
    }

    // do conversion....
    return output;
}

But I don't know how to modify keys accordingly via CamelCaseKeys
Answer: Well, I learned something today! We can use the new typescript 4.1 template string literal features and utility methods to achieve this in a typesafe way.
Capitalize<T> will capitalise a string.
Uncapitalize<T> will uncapitalise a string (what you're after).
More info here on these.
From these two we can build a helper type UncapitalizeObjectKeys<T>:
type UncapitalizeKeys<T extends object> = Uncapitalize<keyof T & string>;

type UncapitalizeObjectKeys<T extends object> = {
  [key in UncapitalizeKeys<T>]: Capitalize<key> extends keyof T ? T[Capitalize<key>] : never;
}

Notes:

Uncapitalize<keyof T & string> - we intersect with string to only get the keys of T which are strings, as we can't capitalise numbers or symbols
We have to uncapitalise the keys in [key in UncapitalizeKeys<T>] - and then re-capitalise them to actually pull the proper value out of T with T[Capitalize<key>]. The conditional part Capitalize<key> extends keyof T  is just checking if the capitalised, uncapitalised key still is assignable to keyof T, as TS isn't able to maintain this relationship (...yet?).

We can then pull a couple of parts out of @spender's answer - replacing the runtime type checking as TS should be able to assert these (assuming these objects aren't coming from IO:
type UncapitalizeObjectKeys<T extends object> = {
  [key in UncapitalizeKeys<T>]: Capitalize<key> extends keyof T ? T[Capitalize<key>] : never;
}

type UncapitalizeKeys<T extends object> = Uncapitalize<keyof T & string>;

export const lowerCaseKeys = <T extends object>(obj: T): UncapitalizeObjectKeys<T> => {
    const entries = Object.entries(obj);
    const mappedEntries = entries.map(
        ([k, v]) => [
            `${k.substr(0, 1).toLowerCase()}${k.substr(1)}`,
            lowerCaseKeys(v)]
    );

    return Object.fromEntries(mappedEntries) as UncapitalizeObjectKeys<T>;
};

We now get the output we're after:

Playground link
Comment: Unfortunately I have TS 3.2. So I need some 'older' approach
Comment: Not possible in that case sadly
Comment: methodologySaga.ts:59 RangeError: Maximum call stack size exceeded
Answer: I will put more modern answer here just for visibility.
Though the implementation in the other answers is correct, I think this type is more elegant. playground
type UncapitalizeObjectKeys<T> = {
  [key in keyof T as Uncapitalize<key & string>]: T[key] extends Object ? UncapitalizeObjectKeys<T[key]> : T[key]
}

Works since 4.1
Comment: Thanks for your input. Can you please add playground link as well
Comment: @Timothy sure. I also updated it to be recursive over fields
Comment: Very neat solution. I love it mate
Answer: Here's a recursive solution that I've used to good effect:
export const lowerCaseKeys = (obj: any): any => {
    if (typeof obj !== 'object') {
        return obj;
    }
    if (Array.isArray(obj)) {
        return obj.map(lowerCaseKeys);
    }
    if (obj === null) {
        return null;
    }
    const entries = Object.entries(obj);
    const mappedEntries = entries.map(
        ([k, v]) => [
            `${k.substr(0, 1).toLowerCase()}${k.substr(1)}`,
            lowerCaseKeys(v)] as const
    );
    return Object.fromEntries(mappedEntries);
};

Playground link
Comment: turns out you can actually type this now with TS 4.1 https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-1.html
Comment: thank for that... this part I have, I don't have TypeScript part... The most important part I need implement is: (obj: any): any
Comment: check my answer for the TS part @Timothy
Answer: This particular form works better with intellisense, and handles arrays
export type UncapitalizeObjectKeys<T> = T extends Array<infer U>
  ? Array<UncapitalizeObjectKeys>
  : T extends object
  ? {
      [K in keyof T as Uncapitalize<string & K>]: UncapitalizeObjectKeys<T[K]>
    }
  : T

type Movie = {
  Title: string
  Year: number
}

type BibliographyPascalCase = {
  DirectorName: string
  Movies: Movie[]
}

type BibliographyCamelCase = UncapitalizeObjectKeys<BibliographyPascalCase>

/*
type BibliographyCamelCase = {
  directoName: string
  movies: {
    title: string
    year: number
  }[]
*/


Tuning a keyboard

Question: Is there any software in which I can connect my casio keyboard (mk-210) to use as input and select the tuning in cents or frequency for each note manually to listen to as an output so I can try different tunings as much as I want?
Edit: in particular I'd like a tunning that isn't the same on every octave, so that's not so easy to get.
Comment: Don't know about that k'bd., but there are many out there with many different temperament options.
Comment: Do you want to tune the keyboard, or do you want to generate a MIDI file and tune that?  If the former, I'm skeptical that any digital keyboard will allow tuning different octaves differently.  As with pipe organs, get yourself multiple keyboards to do that.
Comment: I thought the keyboard itself wasn't really important. That there should be a software made for midi in general for which the keyboard served as input and nothing else. I posted the model just in case. It does have some temperaments like pythagorean, pure minor and major and some others, but I needed something specific. Thanks for the idea though. :)
Comment: You need: (1) MIDI interface for your computer, unless the Casio has USB connectivity that does this without dedicated MIDI connections (2) _host_ application for virtual instrument plugins, plugin types VST on Windows, VST or AU on Mac, examples of such host applications are Reason, Reaper, Garage Band, Logic, Cubase, Ableton, Cakewalk, to name a few, (3) a VST or AU instrument plugin that supports _microtuning_.
Comment: I know some admins might go to on hold this question as off topic because OP asks for a software. To prevent such intensions I'd like to point out that this question and eventually answers might have a great impact of music theory and practice.
Answer: I can't speak to the logistics of connecting your particular keyboard.  But one piece of software that is useful for this (and has been around forever) is Scala.  Depending on the abilities of your equipment, it can export tuning files in various formats or use MIDI pitch bends to allow real-time performance on an instrument that doesn't natively support such detailed tuning features.
And it can definitely do scales without octave equivalence as well as all sorts of other exotic tuning things.  It does have a bit of a learning curve, and it's a little old-fashioned and clunky in terms of its interface.  But it's probably one of the most flexible things out there for manipulating and building tunings.
Comment: Ok, so I tried Scala, and after some struggling, it ended up doing exactly what I needed, so many thanks. I even managed to play the new sounds directly from the keyboard with no lag which is a luxury. But I don't know how to do it again..
Answer: Cubase has its Micro Tuner in the MIDI Effects section.
I'm not clear if different tunings can be set in different octaves.  From the description I suspect not, but I'll check when I get to my 'big' computer where Cubase lives!
https://steinberg.help/cubase_plugin_reference/v9/en/_shared/topics/plug_ref/micro_tuner_r.html

Tuning a keyboard


Maximum rate of change along which curve?

Question: The temperature $T(x,y)$ at points in the $xy$-plane is given by $T(x,y)= x^2 -2 y^2$. An ant wishes to cool off as quickly as possible. Along what curve through $(2,1)$ should the ant move in order to continue to experience maximum rate of cooling?
$$∇T = 2x \hat i-4y \hat j$$
I think the tangent of the ant's path has to be parallel to the negative of the gradient of $T$ at all times, but I don't know how to go on.
Answer: The tangent to the ant path at time $t$ is directed along the vector $(x'(t),y'(t))$ and the gradient of $T$ at every point $(x(t),y(t))$ is directed along the vector $(x(t),-2y(t))$ hence one asks that the vectors $(x'(t),y'(t))$ and $(x(t),-2y(t))$ are colinear at all time, that is, that $$\left|\begin{array}{cc}x'(t)&x(t)\\ y'(t)&-2y(t)\end{array}\right|=0$$ or, equivalently, that $$2x'(t)y(t)+y'(t)x(t)=0$$ Multiplying the LHS by $x(t)$ yields $$2x'(t)x(t)y(t)+y'(t)x(t)^2=(x(t)^2y(t))'$$ hence the ant moves along a path of equation $$x(t)^2y(t)=c$$ If $(x(0),y(0))=(2,1)$, one gets $c=$ [to be completed by the OP] hence the curve has equation [to be completed by the OP].
Answer: This plot has a saddle point:

and its contour plot:

i thought this plot can help you.


Dúvida validação de cpf antes de inserir no banco

Question: Na minha classe SocioDAO eu tenho o metodo salvarSocio() que irá salvar no banco de dados. Na minha classe DadosUsuario eu tenho as informações que precisam ser gravadas
no banco no caso cadastrarUsuario(). Como proceder para validar CPF?
Não queria colocar a validação dentro do meu método cadastrarUsuario() pois não o mesmo não deveria ficar muito grande.
public class SocioDAO {

    public void salvarSocio(Socio socio) throws SQLException {
        StringBuilder sql = new StringBuilder();
        sql.append("INSERT INTO socio ");
        sql.append("(nome, telefone, ddd, email, cpf) ");
        sql.append("VALUES (?, ?, ?, ?, ?) ");

        Connection conexao = ConexaoFactory.conectar();

        PreparedStatement comando = conexao.prepareStatement(sql.toString());

        comando.setString(1, socio.getNome());
        comando.setInt(2, socio.getTelefone());
        comando.setInt(3, socio.getDdd());
        comando.setString(4, socio.getEmail());
        comando.setString(5, socio.getCpf());

        comando.executeUpdate();

    }

public class DadosUsuario {

    static Scanner scan = new Scanner(System.in);

    public void cadastrarUsuario() {

        Socio cadastrarUser = new Socio();

        System.out.println("Informe um nome: ");
        cadastrarUser.setNome(scan.nextLine());

        System.out.println("Informe um telefone: ");
        cadastrarUser.setTelefone(scan.nextInt());

        System.out.println("Informe o DDD: ");
        cadastrarUser.setDdd(scan.nextInt());

        System.out.println("Informe o email: ");
        cadastrarUser.setEmail(scan.next());

        System.out.println("Informe o cpf: ");
        cadastrarUser.setCpf(scan.next());

        SocioDAO dao = new SocioDAO();

        try {
            dao.salvarSocio(cadastrarUser);
            System.out.println("USUÁRIO CADASTRADO COM SUCESSO.");
        } catch (SQLException e) {
            System.out.println("ERRO AO CADASTRAR USUÁRIO.");
            //e.printStackTrace();
        }

    }

Answer: Já que você não tem uma camada de negócio e acessa o DAO diretamente, você deveria colocar a validação dentro do DAO, que encontrando algum problema, lançar uma excecao. Colocando dentro do DAO, qualquer lugar que você venha a instanciar ele, garantirá que o dado estará sendo validado.
Em uma aplicação mais séria, nunca acessamos o DAO diretamente. Sempre passamos por uma camada de negócio que serve justamente para fazer esta validação, ficando Service (com sua regra de negócio e validação) -> DAO
Aproveite para dar uma olhada nesta resposta também, o assunto pode ser bem longo.
edit: exemplo simples com camada de negócio
public class SocioService { // Tem pessoas que preferem SocioBusiness

    private SocioDAO socioDAO;

    public SocioService() {
        socioDAO = new SocioDAO();
    }

    public void inserir(Socio socio) throws ServiceException { // inserir ou adicionar deixa claro que é "criação" de registro.

        if (socio.getNome() == null || socio.getNome().trim().length()) {
            throw new ServiceException("Nome não pode estar vazio");
        }

        // ... outras validacoes

        socioDAO.salvarSocio(socio); // cuidado com o adjetivo salvar, ele pode deixar a pessoa que chama em duvida, se estiver trabalhando em equipe, a pessoa iria querer ver o fonte para saber se aqui está sendo inserido ou atualizando um registro.
    }
}

public class ServiceException extends Exception {

    public ServiceException(String mensagem) {
        super(mensagem);
    }
}

Comment: E como eu poderia criar essa outra camada... Poderia me dar um toque?
Comment: embora não seja o escopo da sua pergunta, adicionei. Se você não tem também como se recuperar de uma SQLException, crie uma exceção de runtime (não checada) sua, e lance-a de dentro do DAO. Tem outros detalhes importantes também de como não manipular transacoes dentro de um método do DAO, mas agora não vem ao caso :o)


Prolog: How to create all possible combinations without repetitions

Question: I am trying to create a predicate that finds all possible combinations without repeating same numbers. I tried using permutation predicate, but it found duplicated lists. For example:
permutation([0,1,1], L).
L = [0,1,1];
L = [0,1,1];
L = [1,0,1];
L = [1,1,0];
L = [1,0,1];
L = [1,1,0];

What I need:
newPermutation([0,1,1], L).
L = [0,1,1];
L = [1,0,1];
L = [1,1,0];

Can someone please help me with that? Thanks a lot...
Comment: `?- distinct(permutation([1,1,0],L)).`
Comment: I want to help but you should first do the footwork and figure out what you really need. Start by reading the [Wikipedia page about "permutation"](https://en.wikipedia.org/wiki/Permutation). You don't have a set, you have a list, representing possibly a multiset, and anyway you say that you need "combinations without repetitions" but then you use a predicate called "permutation".
Comment: @GuyCoder: `distinct(permutation([X,Y],Xs)), X = Y` still gives a redundant answer.
Answer: The repetition-free permutations of [0, 1, 1] are the possible interleavings of the lists [0] and [1, 1]:
?- list_list_interleaving([0], [1, 1], Interleaving).
Interleaving = [0, 1, 1] ;
Interleaving = [1, 0, 1] ;
Interleaving = [1, 1, 0] ;
false.

We can define this as:
list_list_interleaving([], Ys, Ys).
list_list_interleaving([X | Xs], [], [X | Xs]).
list_list_interleaving([X | Xs], [Y | Ys], [X | Interleaving]) :-
    list_list_interleaving(Xs, [Y | Ys], Interleaving).
list_list_interleaving([X | Xs], [Y | Ys], [Y | Interleaving]) :-
    list_list_interleaving([X | Xs], Ys, Interleaving).

For more than two distinct elements, we need the ability to interleave all the lists in a list:
lists_interleaving([Xs], Xs).
lists_interleaving([Xs, Ys | Lists], Interleaving) :-
    lists_interleaving([Ys | Lists], Interleaving0),
    list_list_interleaving(Xs, Interleaving0, Interleaving).

For example:
?- lists_interleaving([[a, a], [b], [c, c]], Interleaving).
Interleaving = [a, a, b, c, c] ;
Interleaving = [a, b, a, c, c] ;
Interleaving = [a, b, c, a, c] ;
Interleaving = [a, b, c, c, a] ;
Interleaving = [b, a, a, c, c] ;
Interleaving = [b, a, c, a, c] ;
Interleaving = [b, a, c, c, a] ;
Interleaving = [b, c, a, a, c] ;
Interleaving = [b, c, a, c, a] ;
Interleaving = [b, c, c, a, a] ;
Interleaving = [a, a, c, b, c] ;
Interleaving = [a, c, a, b, c] ;
Interleaving = [a, c, b, a, c] ;
Interleaving = [a, c, b, c, a] ;
Interleaving = [c, a, a, b, c] ;
Interleaving = [c, a, b, a, c] ;
Interleaving = [c, a, b, c, a] ;
Interleaving = [c, b, a, a, c] ;
Interleaving = [c, b, a, c, a] ;
Interleaving = [c, b, c, a, a] ;
Interleaving = [a, a, c, c, b] ;
Interleaving = [a, c, a, c, b] ;
Interleaving = [a, c, c, a, b] ;
Interleaving = [a, c, c, b, a] ;
Interleaving = [c, a, a, c, b] ;
Interleaving = [c, a, c, a, b] ;
Interleaving = [c, a, c, b, a] ;
Interleaving = [c, c, a, a, b] ;
Interleaving = [c, c, a, b, a] ;
Interleaving = [c, c, b, a, a] ;
false.

The key observation here is that interleaving is not the same as just inserting elements into a list at an arbitrary position: Interleaving keeps the relative order of the elements of the lists. So the first occurrence of a will always precede the second occurrence of a. We can see this more clearly if we label the elements:
?- list_list_interleaving([a1, a2], [b1, b2], Interleaving).
Interleaving = [a1, a2, b1, b2] ;
Interleaving = [a1, b1, a2, b2] ;
Interleaving = [a1, b1, b2, a2] ;
Interleaving = [b1, a1, a2, b2] ;
Interleaving = [b1, a1, b2, a2] ;
Interleaving = [b1, b2, a1, a2] ;
false.

a1 always precedes a2, b1 always precedes b2.
So we can do what we need if our input is separated into such a list of lists. This is a multiset of the elements of the original list. We can compute multisets like this:
list_multiset([], []).
list_multiset([X | Xs], Multiset) :-
    list_multiset(Xs, Multiset0),
    (   ClassX = [X | _],
        select(ClassX, Multiset0, MultisetWithoutClassX)
    ->  Multiset = [[X | ClassX] | MultisetWithoutClassX]
    ;   Multiset = [[X] | Multiset0] ).

For example:
?- list_multiset([a, b, c, a, c], Multiset).
Multiset = [[a, a], [b], [c, c]].

So then the distinct permutations (combinations, whatever) are the interleavings of a list's multiset representation:
distinct_permutation(List, Permutation) :-
    must_be(ground, List),
    list_multiset(List, Multiset),
    lists_interleaving(Multiset, Permutation).

This works:
?- distinct_permutation([0, 1, 1], Permutation).
Permutation = [0, 1, 1] ;
Permutation = [1, 0, 1] ;
Permutation = [1, 1, 0] ;
false.

It's much faster than slaggo's solution, but so far only works on ground lists:
?- time(aggregate_all(count, distinct_permutation([1,1,1,2,2,2,3,3,3,3,4,4,4,4,4],P), C)).
% 63,090,949 inferences, 3.958 CPU in 3.958 seconds (100% CPU, 15941609 Lips)
C = 12612600.

It remains to handle lists containing variables. The heavy lifting in all of this is done by select/3. All we need is to "just" implement a reified select_t/4 similarly to memberd_t/3. Unfortunately I haven't managed to do this so far. Suggestions are very welcome, or for someone to take this approach and run with it.
Edit: And now with fully pure support for arbitrary lists
I was thinking too complicated above: select/3 is not needed, nor any reified version of it. The above version uses select/3 for a relation that (operationally) adds an element to a multiset: If there is already an equivalence class containing X, it is extended by another X element, whereas if there isn't such a class, a new class [X] is added.
But we can write this much more directly as well:
list_multiset([], []).
list_multiset([X | Xs], Multiset) :-
    list_multiset(Xs, Multiset0),
    multiset_elem_inserted(Multiset0, X, Multiset).

multiset_elem_inserted([],                  X, [[X]]).
multiset_elem_inserted([[X|Xs] | Classes],  X, [[X,X|Xs] | Classes]).
multiset_elem_inserted([[Y|Ys] | Classes0], X, [[Y|Ys] | Classes]) :-
    dif(X, Y),
    multiset_elem_inserted(Classes0, X, Classes).

This handles variables correctly, enumerating on backtracking all possible ways of constraining any pair of terms in the list with =/2 or dif/2:
?- list_multiset([X, Z, X, Y], Multiset).
X = Z, Z = Y,
Multiset = [[Y, Y, Y, Y]] ;
X = Y,
Multiset = [[Y, Y, Y], [Z]],
dif(Z, Y) ;
Z = Y,
Multiset = [[Y, Y], [X, X]],
dif(X, Y),
dif(X, Y) ;
X = Z,
Multiset = [[Y], [Z, Z, Z]],
dif(Z, Y),
dif(Z, Y),
dif(Z, Y) ;
Multiset = [[Y], [X, X], [Z]],
dif(X, Y),
dif(X, Y),
dif(Z, Y),
dif(Z, X) ;
false.

And this carries over to the distinct permutations too (we can now remove the must_be from distinct_permutation):
?- distinct_permutation([X, Y], Permutation).
X = Y,
Permutation = [Y, Y] ;
Permutation = [Y, X],
dif(X, Y) ;
Permutation = [X, Y],
dif(X, Y) ;
false.

?- distinct_permutation([X, Y], Permutation), X = Y.
X = Y,
Permutation = [Y, Y] ;
false.

?- distinct_permutation([X, Y], Permutation), dif(X, Y).
Permutation = [Y, X],
dif(X, Y),
dif(X, Y) ;
Permutation = [X, Y],
dif(X, Y),
dif(X, Y) ;
false.

Comment: You're right that `ground` is a bit coarse. For example, the code would work correctly for `[f(X), g(Y)]` I believe, as would `[f(X), f(X)]` -- list elements containing variables are OK if they don't unify with any other list elements introducing new bindings. Not sure I'd call that "much better", it's a somehow even more artificial criterion. Maybe you're thinking of something else.
Comment: But anyway, this is somewhat moot as I have now updated my answer with what I believe is a fully general version above. Like so often with Prolog, it's simpler than my original thinking.
Comment: s(X). A general observation: It would help a lot to produce honest instantiation errors in all those cases where you currently are unable to produce correct answers. The English "...only works on ground lists" just doesn't execute on the Prologs I have access to.
Comment: The entire answer doesn't execute on the Prologs you have access to. You must be able to read and understand the text and take out the Prolog bits that you need and use them in the intended context. Nonetheless, I added a `must_be` guard.
Comment: I have the suspicion that you could give a much better (more general) criterion, even now without any further improvement. It is so important to get it also automated because otherwise your code can only be tested by some hand-crafted test cases (where someone read the text) and not by exhaustive runs which beat by far any other testing approach.
Comment: Please consider the next time to produce different answers.
Answer: For ground lists you may do what @GuyCoder suggested: distinct(permutation([1,1,0],L)).
For arbitrary lists you may enumerate all distinct solutions with the help of dif/2:
permutation_no_dup([], []).
permutation_no_dup(L, PL):-
  same_length(L, PL),
  length(L, Len),
  numlist(1,Len, RLMax),
  reverse(RLMax, LMax),
  length(LCur, Len),
  maplist(=(1), LCur),
  permutation_no_dup(LCur, L, LMax/LCur-L, [], PL).

permutation_no_dup([], _, _, PL, PL).
permutation_no_dup([], _, LMax/LCur-L, PL, PL1):-
  dif(PL, PL1),
  next(LCur, LMax, NLCur),
  permutation_no_dup(NLCur, L, LMax/NLCur-L, [], PL1).
permutation_no_dup([Take|LCur], L, Info, PL, PL1):-
  nth1(Take, L, Item, L1),
  permutation_no_dup(LCur, L1, Info, [Item|PL], PL1).

next([Cur|LCur], [Max|_], [NCur|LCur]):-
  Cur < Max,
  succ(Cur, NCur).
next([Cur|LCur], [Cur|LMax], [1|NLCur]):-
  next(LCur, LMax, NLCur).

same_length([],[]).
same_length([_|Xs], [_|Ys]) :-
   same_length(Xs, Ys).

Sample run:
?- permutation_no_dup([0,1,1], L).
L = [1, 1, 0] ;
L = [1, 0, 1] ;
L = [0, 1, 1] ;
false.
?- permutation_no_dup([X,Y], L), X=Y.
X = Y,
L = [Y, Y] ;
false.

Update:
With the above code, I get this output with SWI 8.0.2 which is obviously wrong:
?- permutation_no_dup([x,y,Z,Z],P), P=[x,y,z,z].
false.

?- P=[x,y,z,z], permutation_no_dup([x,y,Z,Z],P).
P = [x, y, z, z],
Z = z ;
false.

but rearranging the call to dif/2 in the second clause of permutation_no_dup/5 so it now reads:
permutation_no_dup([], _, _, PL, PL).
permutation_no_dup([], _, LMax/LCur-L, PL, PL1):-
%      dif(PL, PL1),    % <-- removed dif/2 from here
  next(LCur, LMax, NLCur),
  permutation_no_dup(NLCur, L, LMax/NLCur-L, [], PL1),
  dif(PL, PL1).         % <-- Moved dif/2 to here
permutation_no_dup([Take|LCur], L, Info, PL, PL1):-
  nth1(Take, L, Item, L1),
  permutation_no_dup(LCur, L1, Info, [Item|PL], PL1).

Now we get:
?- permutation_no_dup([x,y,Z,Z],P), P=[x,y,z,z].
Z = z,
P = [x, y, z, z] ;
false.

?- P=[x,y,z,z], permutation_no_dup([x,y,Z,Z],P).
P = [x, y, z, z],
Z = z ;
false.

Comment: Fixed as [8.5.3](https://swi-prolog.discourse.group/t/ann-swi-prolog-8-5-3/4688).
Comment: I forgot to put `next/3`, which gives the next item in the enumeration. And I added a clause for the corner case  `permutation_no_dup([], L).`
Comment: `permutation_no_dup([x,y,Z,Z],P), P=[x,y,z,z].` fails. It should (trivially) succeed.
Comment: In SICStus it works.
Comment: In SWI 6.6.4, 7.6.4, 8.4.0 it does not work. Which version are you using? (Note that I removed the other case which failed only in 6.6.4)
Comment: I am using SWI 8.0.2. In SICStus `permutation_no_dup([x,y,Z,Z],P), P=[x,y,z,z].` succeeds or fails as with my SWI version ?
Comment: @false: Rearranging the call to `dif/2` as shown in the updated answer fixes your sample. Still not sure why...
Comment: Obviously, atvs are still broken in SWI.
Comment: In SICStus `permutation_no_dup([x,y,Z,Z],P), P=[x,y,z,z].` correctly succeeds, to see that this is in fact correct, just consider: `P=[x,y,z,z], permutation_no_dup([x,y,Z,Z],P)` which succeeds in SICStus but also in all versions of SWI.
Comment: Reported as [#105](https://github.com/SWI-Prolog/issues/issues/105).


underscore - transform an array with _.map()

Question: I have a multi-dimensional array of objects that I need to transform into a different array. I'm convinced that _.map() is what I need. Not having used it before, I'm having trouble traversing the array to extract the correct values. Given this simplified example:
[
   {
      "08/25/2015":[
         {
            "source":"someSource0",
            "name":"someName0",
            "stuff":"-6.728479",
            "stuffValue":"14.862200",
            "amount":"-100.00"
         },
         {
            "notNeeded0":-100,
            "subtotal":"-100.00"
         }
      ]
   },
   {
      "08/26/2015":[
         {
            "source":"someSource1",
            "name":"someName1",
            "stuff":"-9.496676",
            "stuffValue":"10.530000",
            "amount":"-100.00"
         },
         {
            "notNeeded0":-100,
            "subtotal":"-100.00"
         }
      ]
   },
   {
      "08/27/2015":[
         {
            "source":"someSource2",
            "name":"someName2",
            "stuff":"-9.469697",
            "stuffValue":"10.560000",
            "amount":"-100.00"
         },
         {
            "notNeeded0":-100,
            "subtotal":"-100.00"
         }
      ]
   },
   {
      "08/28/2015":[
         {
            "source":"someSource3",
            "name":"someName3",
            "stuff":"-1.731841",
            "stuffValue":"10.570000",
            "amount":"-18.24"
         },
         {
            "source":"someSource4",
            "name":"someName4",
            "stuff":"-2.628939",
            "stuffValue":"31.100000",
            "amount":"-81.76"
         },
         {
            "notNeeded0":-100,
            "subtotal":"-100.00"
         }
      ]
   },
   {
      "notNeeded1":-400,
      "notNeeded2":"-400.00"
   }
]

I needed to transform it to something structured like this:
[
   {
      "date":"08/27/2015",
      "detail":[
         {
            "source":"someSource2",
            "name":"someName2",
            "stuff":"-9.469697",
            "stuffValue":"10.560000",
            "amount":"-100.00",
            "subtotal":"-100.00"
         }
      ]
   },
   {
      "date":"08/28/2015",
      "detail":[
         {
            "source":"someSource3",
            "name":"someName3",
            "stuff":"-1.731841",
            "stuffValue":"10.570000",
            "amount":"-18.24",
            "subtotal":"-100.00"
         },
         {
            "source":"someSource4",
            "name":"someName4",
            "stuff":"-2.628939",
            "stuffValue":"31.100000",
            "amount":"-81.76",
            "subtotal":"-100.00"
         }
      ]
   }
]

Please feel free to ask questions if needed. Thanks for your assistance.
EDIT: This code is not for web-based use. So any reference to web and browser compatibility is not necessary. And, I use the underscore library to simplify tasks like looping that can get pretty ugly using pure JS. Hope that clarifies the intent.
Comment: you can use javascript's own *array.[filter](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter)(keep_only_dates).[map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map)(transform)* - unless you need to support IE8
Comment: `.map` is pretty straightforward: The callback is invoked for every element in the array and the return value is added to the new array. The documentation even has examples: http://underscorejs.org/#map .
Comment: @FelixKling i've read the doc first. it's simplified one-level example didn't help me much which is why I posted this question here. It would probably have helped if I had mentioned more about what I've already done. I'll remember that. Thx.
Comment: Oftentimes, I fail to decompose the problem into manageable bits. this example is no different. one solution a colleague proposed is to break out the details (and a totals node not in the OP) and roll them into the final object as evidenced in this updated gist. https://gist.github.com/mjstelly/b7cbfdfc3efda5a90636
Answer: using built-in .filter, .map and .splice:
var modified = original.filter(function only_nested_arrays(obj) {
  var keys = Object.keys(obj)
  return keys.length === 1 && Array.isArray(obj[keys[0]])
}).map(function transform(obj) {
  var date = Object.keys(obj)[0],
      inner_array = obj[date],
      subtotal = inner_array.splice(-1)[0].subtotal
  inner_array.forEach(function(obj_inner) {
    obj_inner.subtotal = subtotal
  })
  return {date: date, detail: inner_array}
})

console.log(JSON.stringify(modified, null, 2))

But watch out for browser compatibility (see the links for a polyfill to make it work on IE8 or lower). Runnable code that alerts the result if you want to test:

"use strict"
var original = [
  {
    "08/25/2015":[
      {
        "source":"someSource0",
        "name":"someName0",
        "stuff":"-6.728479",
        "stuffValue":"14.862200",
        "amount":"-100.00"
      },
      {
        "notNeeded0":-100,
        "subtotal":"-100.00"
      }
    ]
  },
  {
    "08/26/2015":[
      {
        "source":"someSource1",
        "name":"someName1",
        "stuff":"-9.496676",
        "stuffValue":"10.530000",
        "amount":"-100.00"
      },
      {
        "notNeeded0":-100,
        "subtotal":"-100.00"
      }
    ]
  },
  {
    "08/27/2015":[
      {
        "source":"someSource2",
        "name":"someName2",
        "stuff":"-9.469697",
        "stuffValue":"10.560000",
        "amount":"-100.00"
      },
      {
        "notNeeded0":-100,
        "subtotal":"-100.00"
      }
    ]
  },
  {
    "08/28/2015":[
      {
        "source":"someSource3",
        "name":"someName3",
        "stuff":"-1.731841",
        "stuffValue":"10.570000",
        "amount":"-18.24"
      },
      {
        "source":"someSource4",
        "name":"someName4",
        "stuff":"-2.628939",
        "stuffValue":"31.100000",
        "amount":"-81.76"
      },
      {
        "notNeeded0":-100,
        "subtotal":"-100.00"
      }
    ]
  },
  {
    "notNeeded1":-400,
    "notNeeded2":"-400.00"
  }
]

var modified = original.filter(function only_nested_arrays(obj) {
  var keys = Object.keys(obj)
  return keys.length === 1 && Array.isArray(obj[keys[0]])
}).map(function transform(obj) {
  var date = Object.keys(obj)[0],
      inner_array = obj[date],
      subtotal = inner_array.splice(-1)[0].subtotal
  inner_array.forEach(function(obj_inner) {
    obj_inner.subtotal = subtotal
  })
  return {date: date, detail: inner_array}
})

alert(JSON.stringify(modified, null, 2))
Comment: I like the JSON `stringify` formatting. I'd forgotten it could do that. It makes it much easier to read.
Comment: caveat: I didn't actually test this on my code. As often happens in business, the development changed directions and we no longer need this algorithm. I wanted to give credit for the assistance. That has intrinsic value to our community. Thanks, @aprillion
Answer: Using _.filter, _.map & _.chain here is what it would look like.
var transformed = _.chain(data).filter(function (obj) {
    var keys = _.keys(obj);
    // only allow where obj has only one key & the value is an array
    // consider replacing this with a check to see if the key is a date which should be more appropriate
    return keys.length === 1 && _.isArray(obj[keys[0]]);
}).map(function (obj) {
    var date     = _.keys(obj)[0];
    var details  = obj[date];
    var subtotal = details.splice(-1)[0].subtotal; // get the last obj from array destructively
    details      = _.map(details, function (detail) {
        detail.subtotal = subtotal; // append the subtotal to all the remaining detail objects
        return detail
    });
    return {
        date   : date,
        details: details
    }
}).value();

I'd suggest that you use the native functions as shown in Aprillion's answer so that you can avoid the whole chain() & then value() usages for getting the actual value from the wrapped object. You can use underscore when the requirements are a bit more complicated.


Adding the impact of hurricanes to a demand forecasting model

Question: I am currently working on a project in which I am trying to create a better forecast using extraneous variables, in particular a dummy variable representing whether a hurricane makes landfall to the US in a given month.
For example, let's say that I am given a monthly forecast (from a software package that uses a univariate approach taking into account 36 months of data and the item's seasonality and trends). For whatever reason, on this item the impact of a hurricane in a particular month is not picked up by the software's forecast.
With that being said, I would like to figure out the impact that a hurricane has on the demand for this item, and then add that result to the forecast. To do so, I would run a regression on the actual demand and lags of a hurricane dummy  (lag 3 is the last lag to be statistically significant in this example) and find the impact at each lag. Then add these effects to the software's forecast as necessary when a hurricane occurs on a given month.
My question is, would this be valid way to attack this problem? If not, what are some other options for doing so? Any help is greatly appreciated!
Side note: from a practical perspective, from past business knowledge, it is known that a hurricane doesn't have impact on current months demand. Only future months. This would mean that for the purposes of my project, I don't need to predict whether a hurricane will occur in a month or not, just what impact this hurricane would have on future months after it has occurred.
Additionally, I know that there are a lot of other factors at play here (category of hurricane, where it makes landfall, etc.), but for now I just want to use the dummy variable mentioned above.
Thanks again!
Comment: @AJKOER Weather is not climate. I am well aware of the unusual weather this year. Climate is a long term average. I also recall a bad article that showed that for a dozen years there was no increase in temperature, that is also weather, not climate. Climate changes constantly and has for billions of years, not one year and not dozens. Over the last several hundred million years there have been periods of time with much higher CO2 and temperature. We are now in a period (long term) of increasing temperature and about half-way between the historical extremes of CO2 in the atmosphere.
Comment: So just use the landfall as the start of a clock, and see what happens thereafter.
Comment: Not sure if I believe the comment: "Side note: from a practical perspective, from past business knowledge, it is known that a hurricane doesn't have an impacted on current months demand. Only future months" as many people affected by hurricanes (now much more frequent) have learned from not having an ample supply of things like toilet paper, water,...,  on hand. This change in inventory policy has likely created an artificial rise in demand (at least one time) for many products that were in short supply during a hurricane or other natural catastrophe events.
Comment: @AJKOER There is no consensus that hurricanes are becoming more frequent. There is some consensus that they are becoming stronger. Some say they are stronger but less frequent. This latter makes sense when you consider that a stronger storm covers a much larger area. That is, common sense dictates that fewer big marbles fit in the same jar as a larger number of small marbles.
Comment: @Carl: This hurricane season alone the number of named storms consumed the entire generated list of names. In fact, the contingency in such a case is to use the Greek alphabet (like Eta,...) and a good # of those. To quote a source: "The 2020 Atlantic hurricane season has truly been one for the record books. With 30 named storms, it has set the record for the most named storms in a single season."  at https://www.scientificamerican.com/article/in-2020-record-breaking-hurricanes-arrived-early-and-often1/#:~:text=The%202020%20Atlantic%20hurricane%20season,miles%20per%20hour%20or%20higher.)
Answer: Can you please provide a small example of you data?
It is not clear how your data looks.
what is item in your data?
what are "lags of a hurricane dummy"?
how the 3 lags are built in the data?
do you mean something like: lag_1, lag_2,lag_3 which represents 3 last month, and 1 or 0 if
hurricane was forecasted?
please provide an example so I can take better look.
Comment: Hurricanes have no reliable distribution so I am not sure how you would build them into time series. When they come, where, and how powerful they are would all matter. It is not, there is a hurricane, there is not.
Comment: The data is historical data consisting of the following: the forecast for the demand of an  item at a given month, the actual demand for that item in the given month, hurricane_dummy being whether a hurricane occurred in that month (either one or zero - even if multiple hurricanes occurred in a month, the value would remain 1). lag_1 being a dummy variable of whether a hurricane occurred in the prior month (if a hurricane occurred in January there would be a 1 for this variable in February) , etc. for lag_2 and lag_3.  Does that help Arkady?


can I get rid of this unknown constant this way?

Question: Given
$y=Ke^{mx}+J$
where K and J are both unknown constants, can I do this:
$\frac{y}{K}=e^{mx}+\frac{J}{K}$
Since J and K are both unknowns, I can just combine them and call the combined unknown K.
$\frac{J}{K}=K$
$\frac{y}{K}=e^{mx}+K$
$y=K(e^{mx}+1)$
I'm really trying to get rid of J altogether.

The differential equation this relates to:
$ay''+by'+cy=0$
set $c=0$
$ay''+by'=0$
set $y'=w$
$aw'+bw=0$
solve this:
$e^{\frac{b}{a}x}w=K$
$w=Ke^{\frac{-b}{a}x}$
$y=K\int e^{\frac{-b}{a}x}$
$y=-K \frac{a}{b}e^{\frac{-b}{a}x} + J$
$y=Ke^{\frac{-b}{a}x} + J$
Comment: Absolutely not!, in the first passage you get $J/K$, not $K$.
Comment: The first step should presumably read $\frac{y}{K}=e^{mx}+\frac{J}{K}$. The $J$ is still around, and won't go away. A usual way to get rid of $J$ is if you know $y$ at some specific $x_0$.
Comment: Yeah but $\frac{J}{K}=K$, since they're both unknowns.  If If I have J+K, I can just rename the conglomerate unknown either J or K.  Can't I do this with multiplication too?
Comment: If you want to rename $J/K$, you cannot use the same name of another already used variable.
Comment: @KorganRivera In some occasions, you could replace $J/K$ with $K$, though you should be clear you are doing this by writing it out.  HOWEVER, in this case that is not okay because you still have another $K$ in the equation and thus your two $K$'s are different.  You can not all of the sudden declare that they are the same.
Comment: Can you explain what you mean by wanting to get rid of $J$?  Do you just want to have two unknowns?  Then you'll need some sort of information like the value of $y$ at some value $x$ (or a couple such points).
Comment: I thought that if two unknown values combine, it doesn't matter what name you give it, it's still unknown.  So, given that both Ks are unknown, it can't be proven that they're not the same thing.
Comment: @Graphth When you have a second-order differential equation, and the coefficient of $y''$ is zero, you can solve and get something of the form of my first equation.  But I always end up with J and that complicates things.
Comment: How about you give us an example of such a second-order differential equation you are starting with.
Comment: @Graphth  Well I suppose so.  I'll append it to the question.
Comment: Okay, so now that you have added to your question, the new question is, why do you think you can just set $c = 0$?  By the way you wrote it, $c$ is some fixed constant, possibly 0 but possibly not 0.  Then, you set it equal to 0.
Comment: What??  Is it against the law?
Comment: @KorganRivera Yes, and you are under arrest.
Comment: Well I'm wasting my time here, forget I asked.
Comment: You're wasting our time too, don't forget that.  Either $c$ is some unknown fixed constant, or it's 0.  If it's 0, then there's no reason to write it in the first place.  If it's not assumed to be 0, then you're solving a different differential equation when you all of the sudden decide you want it to be 0.  So, you can set $c = 0$ if you want, but you're not solving the original differential equation you wrote.  You're just solving $ay'' + by' = 0$.  That's all I'm trying to make clear here.
Comment: Why would you use the same variable name for two different variables? Why?
Comment: @SiliconCelery   Really?  Ok listen.  Here's the deal here.  Unknown constants are neat.  If I have two unknown constants A and B, I can say that A+B = A, if I want to. That way, I'm only using one symbol.  I could call it Z, I could call it Jennifer if I wanted to.  In any case, it's an unknown value.  So long as I have some symbol in there that represents 'some unknown value'.  It's similar to $\infty+\infty=\infty$.
Comment: @KorganRivera Except, you're doing something different that's not actually allowed.  This isn't two constants of integration that you just add together.  One is in front of $e^{mx}$ and one is out by itself.  It's not at all the same situation as you just described.  And, it has nothing to do with $\infty + \infty = \infty$.
Answer: No, your starting function is $y = Ke^{mx} + J$ and your last is $y = Ke^{mx} + K$.  Unless $K = J$, these are clearly different functions.
When you divided both sides by $K$, you should get
$$\frac{y}{K} = e^{mx} + \frac{J}{K}.$$
Korgan, since you still don't understand, here are all the details.  Let's start at the equation just above.  If we want to replace $\frac{J}{K}$ with some other constant, let's use a different letter so it's less confusing.  So, let $I = \frac{J}{K}$.  This gives
$$\frac{y}{K} = e^{mx} + I$$
but we still have $K$ in the denominator, and never have we found any justification to know that $K = I$ so we can't just replace $K$ with $I$.  After all, $I = \frac{J}{K}$.  We never said anything about $I = K$.  If you want to solve for $K$ to get rid of it, you have to solve $I = \frac{J}{K}$ for $K$ which gives $K = \frac{J}{I}$.  So, our new equation reads
$$\frac{y}{\frac{J}{I}} = e^{mx} + I$$
Now, if you really want to call the new constant $K$, now would be a less confusing time to change it to $K$, so we would have
$$\frac{y}{\frac{J}{K}} = e^{mx} + K$$
Multiplying both sides by $\frac{J}{K}$ to clear the denominator gives
$$y = \frac{J}{K} e^{mx} + J$$
Notice, it STILL does not give
$$y = K e^{mx} + K$$
This would only happen in the specific case where $\frac{J}{K} = K$ and $J = K$, which would lead to $J = K = \pm 1$.  But, we don't know $J$ or $K$ so we can't assume their values to make things simpler.  They're unknown, yet fixed.  They're not waiting around for us to simply pick their value.
And, also notice that where we end up is actually more complicated than where we started.
Comment: They're not functions.  Like I said, they're unknown constants.
Comment: @KorganRivera $K$ and $J$ are unknown constants, $y = Ke^{mx} + J$ is a function, where you input a value of $x$ and get out a value of $y$.  The function depends on the fixed values of the unknown constants $K, J, m$.
Comment: Oh, I thought you were saying that J and K were functions.
Comment: Thanks for the answer.  What about renaming $-K\frac{a}{b}$ as just K?  Can that work?
Comment: @KorganRivera As I showed in this example above, you can do such a thing, but you have to be very careful about the other variables that are around and make sure you account for all of them.  So, instead, there would be much less chance for confusion if you renamed it using a new variable name.