How to group/cluster this information about consumers?

Question: I have several millions of rows similar to those below, describing consumers(data has been modified):

city
road
house_number
postcode
state

fort worth
sycamore school rd
5421
76123
texas

fort worth
sycamore school road
4750
76133
tx

I want to group/cluster this data, so that people who live closer stay in the same group/cluster.
My idea was to join in cell by row, creating a string, e.g. for row 1 we would have "fort worth sycamore school rd 5421 76123 texas" and then use a similarity deep learning similarity model to vectorize each sentence/row, and then apply a clustering method to those vectors.
However, I found that deep learning similarity models usually use high-dimensional embeddings of 300 dim or more, and that for a clustering method seems to be highly prohibitive... Also, the similarity models that I found weren't capable of understanding that "tx" is "texas".
What would you advise me to do?
Answer: I recommend against attempting to one-hot-encode this data, or any machine learning embedding, because your data can already be mapped to spatial coordinates using geospatial databases. Rather, I would first look at mapping these data to latitudes and longitudes.
With the latitude and longitude of each point, distances can be computed as arcs on the surface of a sphere between such points. You can use this to occupy a distance matrix.
With a distance matrix, you could perform distance-based clustering using one of a variety of algorithms. There are a lot of clustering algorithms to choose from that make tradeoffs. The Scikit-Learn package nicely visualizes this for some common algorithms.

The rows of the above grid of diagrams are some easy-to-generate data sets with varying statistical, geometric, and (persistent) homological properties. The same documentation tabulates some of the aspects you might like to consider. Fortunately your (latitude, longitude) values can be readily plotted and colour-coded by the partitioning algorithm which can help you evaluate how sensible the output is.

Method name
Parameters
Scalability
Usecase
Geometry (metric used)

K-Means
number of clusters
Very large n_samples, medium n_clusters with MiniBatch code
General-purpose, even cluster size, flat geometry, not too many clusters, inductive
Distances between points

Affinity propagation
damping, sample preference
Not scalable with n_samples
Many clusters, uneven cluster size, non-flat geometry, inductive
Graph distance (e.g. nearest-neighbor graph)

Mean-shift
bandwidth
Not scalable with n_samples
Many clusters, uneven cluster size, non-flat geometry, inductive
Distances between points

Spectral clustering
number of clusters
Medium n_samples, small n_clusters
Few clusters, even cluster size, non-flat geometry, transductive
Graph distance (e.g. nearest-neighbor graph)

Ward hierarchical clustering
number of clusters or distance threshold
Large n_samples and n_clusters
Many clusters, possibly connectivity constraints, transductive
Distances between points

Agglomerative clustering
number of clusters or distance threshold, linkage type, distance
Large n_samples and n_clusters
Many clusters, possibly connectivity constraints, non Euclidean distances, transductive
Any pairwise distance

DBSCAN
neighborhood size
Very large n_samples, medium n_clusters
Non-flat geometry, uneven cluster sizes, outlier removal, transductive
Distances between nearest points

OPTICS
minimum cluster membership
Very large n_samples, large n_clusters
Non-flat geometry, uneven cluster sizes, variable cluster density, outlier removal, transductive
Distances between points

Gaussian mixtures
many
Not scalable
Flat geometry, good for density estimation, inductive
Mahalanobis distances to  centers

BIRCH
branching factor, threshold, optional global clusterer.
Large n_clusters and n_samples
Large dataset, outlier removal, data reduction, inductive
Euclidean distance between points

Bisecting K-Means
number of clusters
Very large n_samples, medium n_clusters
General-purpose, even cluster size, flat geometry, no empty clusters, inductive, hierarchical
Distances between points

I cannot tell you apriori which of these algorithms will be the most suitable for your data. But hopefully this gives you a sense of some of the algorithms that are out there, and that they are non-equivalent in the partitions they recommend.

Here is an afterthought. The main purpose of hard clustering, as opposed to fuzzy clustering, is to suggest a partition. You can also choose partitions based on other data. For example, you could simply partition your data (spatially) by the city they live in.

Here is another afterthought. If you wanted to cluster by distances, you have some choice in what space you are calculating distances within. Above I recommended using arcs on a sphere. You could also use distances along paths in a transportation network (if that is relevant to your research question).

String mapping such as $\text{tx} \rightarrow \text{texas}$ can be done using traditional programming; no machine learning required.
Comment: Galen which algorithms would you suggest I try for distance-based clustering?


Como remover todas as ocorrências de "/" de uma string usando Javascript

Question: Olá, estou tendo problemas para aplicar um simples regex com Javascript. Assim:
var str = "/Date(1421287200000-0200)/";
console.log(str.replace('/\//g',''));             //não funciona, mesmo estando certo
console.log(str.replace('/[/]/g',''));            //não funciona, mesmo envolvendo "/"
console.log(str.replace('/','').replace('/','')); //funciona, mas não tem lógica

Como resolver este problema apenas com regex no Javascript?
NOTA: os dois primeiros exemplos foram testados no site Regexr como no exemplo aqui: http://regexr.com/3eo6t
Answer: No javascript uma regex é delimitada / não precisa de aspas simples.
Mude:
console.log(str.replace('/\//g',''));

Para:
console.log(str.replace(/\//g,''));

Comment: @LeandroLUK depende da linguagem, algumas precisa das aspas e dos delimitadores.
Comment: ja estava respondendo a minha própria pergunta com isso mesmo rsrs, viajei aqui colocando como string um regex
Comment: foi por isso que eu errei, to aprendendo uma pancada de linguagem na faculdade, me embolei todo kkk
Answer: Uma opção é quebrar pela / e depois juntar.

var str = "/Date(1421287200000-0200)/";
str = str.split("/").join("");
console.log(str);
Comment: mas ainda não é a melhor opção lógica. fazendo isso você executa 2 métodos, ao invés de apenas 1
Comment: É pra ser uma opção diferente do que já foi respondido, não a melhor, para você ou quer procurar saber que existe.


Elements in partial ordered set.

Question: Let $(S, ≤)$ be a partial order with two minimal elements $a$ and $b$, and a maximum element $c$. Let $P: S → \{$True, False$\}$ be a predicate defined on $S$. Suppose that $P(a) =$ True, $P(b) =$ False and $P(x) ⇒ P(y)$ for all $x, y ∈ S$ satisfying $x≤y$, where $→$ stands for logical implication. Which of the following statements CANNOT be true?

$P(x) = $True for all $x ∈ S$ such that $x ≠ b$
$P(x) = $False for all $x ∈ S$ such that $x ≠ a$ and $x ≠ c$
$P(x) = $False for all $x ∈ S$ such that $b ≤ x$ and $x ≠ c$
$P(x) = $False for all $x ∈ S$ such that $a ≤ x$ and $b ≤ x$

My attempt :

This can be true, because making all elements true trivially satisfy formula $P(x) ⇒ P(y)$.
This can also be true, if all elements are connected from $b$, then , they can be anything, in particular, they all can be false.
This can also be true, infact, it is exactly the case we saw in option $(2)$
This can't be true, because if an element has an edge from both $a$ and $b$, then it has to true.

Can you explain more formally , please?
Answer: Let $G$ be the partial order consisting only of the elements $a,b$ and $c$ thus $a\leq c$ and $b\leq c$ but $a$ and $b$ are unrelated. As $P(a)$ hold and $a\leq c$ necessarily $P(c)$ has to hold. Thus we know that
$$P(a)=true\>\>\>\> P(b)=false \>\>\>\> P(c)=true$$

is true, since each element $x\in G$ such that $x\neq b$ then $x = a$ or $x=c$ but then $P(x)=true$.
is also true, since if $x\neq a$ and $x\neq c$ then $x=b$ thus $P(x)=false$
is also true since each $x$ such that $x\geq b$ and $x\neq c$, the only option avalible for $x$ is $x=b$ and thus we know that $P(x)=false$.

On the other hand
4. can be true if we do not have any elements which is greater than both $a$ and $b$, something which is not necessary in a partial order. So for instance choose the order consisting of three disjoint elements which are unrelated to eachother. If we however assume that $c$ is a greatest element, then indeed it can not be true.
Comment: Thanks for nice explanation.

Elements in partial ordered set.


REGEX - Regular Expression pattern matching

Question: I've got a regex pattern that I've got to match. It should match the first three characters exactly and any other integer w/ alphabet combination like this:
ch_1q2ew34sdfsadf334f43tf4y6y, or ch_1343ggrgr or asdfasdf2234234

I've got this so far, it works but I want to make sure I'm doing this right:
/([c])+([h])+([_])+[\w\d\._%+-]+/

Is this right?
Answer: Your logic is correct but small changes in the syntax:
/ch_[\w\d._%+-]+/

No need to put single character in [ ] and + here means match one or more characters.. i dont think you want to put that..
No need to escape . inside character set..
( ) is a capture group and content captured inside a capture group can be back referenced with \1, \2, etc.. Use if you want to capture and dont if not required..
Comment: This regex did it **@karthik**! thanks for the help!


How do I pass different values depending on the imported data in React?

Question: I want to take data from js files classified as categories such as 'Delivery' and 'Cafe' and deliver different data to different pages.
I thought about how to import it using map(), but I keep getting errors such as 'products' is not defined.'
It must be done, but it is not implemented well with javascript and react weak. If you know how to do it, I'd appreciate it if you could let me know.
Products.js
export const Product = [
  {
    Delivery: [
      {
        id: '101',
        productName: '허니랩',
        summary: '밀랍으로 만든 친환경 식품포장랩 허니랩.',
        description:
          '~~',
        images: ['3k7sH9F'],
        companyName: '허니랩',
        contact: '02-6082-2720',
        email: 'email@example.com',
        url: 'https://honeywrap.co.kr/',
      },
      {
        id: '102',
        productName: '허니포켓',
        summary: '밀랍으로 만든 친환경 식품포장랩 허니랩. 주머니형태.',
        description:
          "~~",
        images: ['4zJEqwN'],
        companyName: '허니랩',
        contact: "02-6082-2720",
        email: "firstname.lastname@example.com",
        url: "https://honeywrap.co.kr/",
      },
],
},
{
    HouseholdGoods: [
      {
        id: '201',
        productName: '순둥이',
        summary: '아기용 친환경 순한 물티슈',
        description:
          '~',
        images: ['4QXJJaz'],
        companyName: '수오미',
        contact: '080-000-3706',
        email: 'email@example.com',
        url: 'https://www.suomi.co.kr/main/index.php',
      },
      {
        id: '202',
        category: ['HouseholdGoods'],
        productName: '순둥이 데일리',
        summary: '친환경 순한 물티슈',
        description: '품질은 그대로이나 가격을 낮춘 경제적인 생활 물티슈',
        images: ['OMplkd2'],
        companyName: '수오미',
        contact: '080-000-3706',
        email: 'email@example.com',
        url: 'https://www.suomi.co.kr/main/index.php',
      },
],
},
];

Delivery.js
(The file was named temporarily because I did not know how to classify and deliver data without creating a js file separately.)
import React from "react";

function Delivery(
  productName,
  companyName,
  contact,
  email,
  url,
  summary,
  description
) {
  return (

{productName}

{companyName}

Contact: {contact}

Email: {email}

URL: {url}

{summary}

{description}

  );
}

export default Delivery;

Category.js
import React from "react";
import Delivery from "./Delivery";
import { Product } from "./Products";

class Category extends React.Component {
  render() {
    state = {
      products: [],
    };
    this.setState(_renderProduct());

    return
{products ? this._renderProduct() : "nothing"}
;
  }

  _renderProduct = () => {
    const { products } = this.state;
    const renderProducts = products.map((product, id) => {
      return (
        <Delivery
          productName={Product.productName}
          companyName={Product.companyName}
          contact={Product.contact}
          email={Product.email}
          url={Product.url}
          summary={Product.summary}
          description={Product.description}
        />
      );
    });
  };
}

export default Category;

Sorry and thank you for the long question.
Answer: There are quite a few different problems I've found.
First is that you call setState inside render in the Category component, this causes an infinite loop. Instead call setState inside a lifecycle method like componentDidMount or use the useEffect hook if using functional components.
Another problem is that state in Category is also defined inside render. In class components you would normally put this in a class constructor outside of render.
In your setState call you refer to _renderProduct(), this should be this._renderProduct() instead.
Now the main problem here is the structure of your data / how you render this structure.
Products is an array of objects where each object either has a Delivery or HouseholdGoods property which is an array of products. I would advise you to change this structure to something more like this:
export const Product = {
  Delivery: [
    {
      id: "101",
    },
    {
      id: "102",
    },
  ],
  HouseholdGoods: [
    {
      id: "201",
    },
    {
      id: "202",
    },
  ],
};

or this:
export const Product = [
  { id: "101", productType: "Delivery" },
  { id: "102", productType: "Delivery" },
  { id: "201", productType: "HouseholdGoods" },
  { id: "202", productType: "HouseholdGoods" },
];

I personally prefer the second structure, but I've implemented the first as this seems to be what you were going for:
class Category extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      products: null,
    };
  }

  componentDidMount() {
    this.setState({ products: Product });
  }

  render() {
    const { products } = this.state;
    return (

        {products
          ? Object.keys(products).map((productKey) => {
              return (

                  {products[productKey].map((product) => {
                    return (
                      <Delivery
                        key={product.id}
                        productName={product.productName}
                        companyName={product.companyName}
                        contact={product.contact}
                        email={product.email}
                        url={product.url}
                        summary={product.summary}
                        description={product.description}
                      />
                    );
                  })}

              );
            })
          : "no products"}

    );
  }
}

We need a nested loop here, because we need to map over each property key and over the array of objects inside each property. If you use the other structure for Product I've shown, you can simply map over Product without needing two loops.
Now the last important problem was that you weren't destructuring the props inside your Delivery component, instead you should do something like this:
function Delivery({
  productName,
  companyName,
  contact,
  email,
  url,
  summary,
  description,
}) {
  return (

{productName}

{companyName}

Contact: {contact}

Email: {email}

URL: {url}

{summary}

{description}

  );
}

Example Sandbox


Distribution of errors

Question: I am struggeling with a basic question and would be happy to get some pointers. I am trying to evaluate an algorithm $f$ which maps some sample $x$ onto a scalar $y\in\mathbb{R}$, i.e. $f(x) = y$.
For each each sample I have a ground-truth version of this scalar $\hat{y}$ available and I want to estimate the performance of the algorithm with respect to $z=y-\hat{y}$.
Preliminary evaluations have shown that the algorithm is largely unbiased with the mean of $z$ being close to zero and errors are distributed normally with $z \sim \mathcal{N}(\varepsilon, \sigma)$ for $\varepsilon$ close to zero.
However, I am struggling how specifically to evaluate the performance, preferably with an error bound as e.g. a confidence interval. I could

Work with the signed errors $z$. However, in this case I only see how to do a static evaluation of the accuracy ("95% between thresholds ...").
Work with the unsigned errors $\vert z \vert$. In this case I could estimate the mean absolute error. However, as I understood I would need to work with folded or a half-normal distributions. Here I am struggling to find references on how to compute confidence intervals (apart from bootstrapping), making me question if this is indeed the canonical way to deal with my problem.

What is the recommended procedure to tackle this problem? As mentioned, I would prefer if this method would allow me to bound some unsigned version of the errors (squared, absolute values, ...). Additionally, I want to estimate the necessary sample size to bound the error in a confidence interval of prescribed width.
Answer: You say you have sufficient proof that $Z$ is normal $\mathcal{N}(\varepsilon,\sigma^2)$.
Evaluate the goodness of fit
You can use the log-likelihood of the data $\mathcal{D}= \{z_1, z_2, \dots, z_n\}$ as a measure of $f$'s quality
$$\mathcal{L}(\mathcal{D}) = \sum_{i=1}^n\log p(z_i\mid \varepsilon, \sigma)$$
Where $p(z\mid \varepsilon, \sigma) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp(-\frac{(z-\varepsilon)^2}{2\sigma^2})$ the normal pdf.
This would be almost the same as computing the sum of squared residuals.
Confidence region
A confidence region at level $\alpha$ is defined by $P(Z\in C_\alpha) = \alpha$.
Signed errors (Normal case)
In the normal case you can use the normal CDF $\Phi$ as follows: if $Z$ is $\mathcal{N}(\varepsilon,\sigma^2)$ then $\tilde{Z}=(Z-\varepsilon)/\sigma$ is standard normal. A $95\%$ confidence interval can be found by inverting the relationship
$$ P(-\delta \leq \tilde{Z} \leq \delta) = 2P(\tilde{Z}\leq \delta) = 2\Phi(\delta) = \alpha$$
because the normal pdf is symmetric about $0$. By inverting, you get
$$ \delta = \Phi^{-1}(\alpha/2)$$
This can be done in python3 using scipy.stats.norm.ppf (which is $\Phi^{-1}$).
Once you have $\delta$, your confidence region is simply
$$-\delta \leq \tilde{Z}\leq \delta \iff \varepsilon -\sigma\delta \leq Z \leq \varepsilon + \sigma\delta$$
Unsigned errors (FoldedNormal case)
Use the same definition for a confidence region, I'll assume you want an interval of the form $[0,\delta]$. Normalize again to get $\lvert\tilde{Z}\rvert$. Then the relationship is
$$ P(0 \leq \lvert\tilde{Z}\rvert \leq \delta) = F_{\mathrm{FN}}(\delta) = \alpha$$
where $F_{\mathrm{FN}}$ is the FoldedNormal CDF. Invert this to get
$$ \delta = F_{\mathrm{FN}}^{-1}(\alpha)$$
In order to compute $\delta$ from $\alpha=0.95$ you can use any implementation of the FoldedNormal inverse cdf such as scipy.stats.foldnorm.ppf.
Comment: Alright here I've included the computations :)
Comment: You said your errors $Z$ had a standard deviation of $\sigma$. You have to check for yourself what $\sigma$ is in your specific case
Comment: Compute $\delta$ using the folded normal with $c=0$, then scale it back using $\sigma$. The $[0,\delta]$ interval for $\lvert\tilde{Z}\rvert$ becomes the interval $[0, \varepsilon + \sigma\delta]$ for $\lvert Z \rvert$.
Comment: You should estimate $\varepsilon, \sigma$ from the whole dataset $\mathcal{D}$ and then plug them in the formulas I gave
Comment: Please create a new question if you need anything more specific
Comment: You are absolutely right, computing the SSD definitely would be a canonical approach! However, I am just realizing that my original questions did not emphasize enough that I would like to establish bounds on some unsigned version of the error (preferably a confidence interval, questions updated). Would I be able to do this with this approach?
Comment: Once you have the PDF and CDF you can compute confidence regions. The unsigned variables are HalfNormal and you can compute a confidence region using the CDF. I'll expand on that later.
Comment: Thank you so much for your detailed input! Am I correct to assume that (at least for the normal case), by $\sigma$ you mean the standard error and not the standard deviation? Does this relationship hold for the FoldedNormal as well?
Comment: Sorry, I just realized that `scipy.stats.foldnorm.pdf` models $p_{\text{FN}}(c, 1)\vert$ and thus requires an additional parameter `c`. However, I have found a way to recover $\mathcal{N}(\mu_\text{N},\sigma_\text{N}^2)$ from my errors $Z$, including confidence intervals for $\mu_\text{N}, \sigma^2_\text{N}$. Does this help somehow? The only way I can think of is by using the bounds of those CIs to calculate bounds on the CIs of $\mu_\text{FN}, \sigma^2_\text{FN}$ based on direct equations for the mean/var of the folded normal. However, this seems quite some back-and-forth.
Comment: Sorry for the confusion: $\varepsilon>0$ is the mean of the *underlying normal distribution*, the mean of the associated folded normal is thus strictly larger (and far away from 0).  Are you sure that the 95% confidence interval for the folded normal can have the expression $[0, \delta]$? Shouldn't the interval be able to shrink if $n\to\infty$? In this case this expression could only be valid if the mean of the folded normal is zero.
Comment: If $\varepsilon$ is the mean of $Z$, then $Z-\varepsilon$ has mean zero. No matter the sample size, the new variable always has mean zero (compute \mathrm{E}[Z-\hat{\varespilon}]$ to verify this). The absolute value of a standard normal is a folded normal with parameters $0,1$ can be modelled by setting $c$ to $0$ in the scipy method
Comment: The folded normal has zero mass for negative values, I chose an interval starting at zero but you can choose another interval if you so desire.
Answer: This sounds like a nonlinear regression problem.  Your notation for the problem essentially reverses many of the standard statistical conventions (e.g., you are using a hat for the true value rather than the estimator), so I will show you how to write it out in the standard way, and then measure the performance of your method.
Given $n$ sample pairs you can write your problem using the standard model form:
$$y_i = f(x_i) + \varepsilon_i
\quad \quad \quad \varepsilon_1,...,\varepsilon_n \sim \text{IID N}(0, \sigma^2).$$
In this model form the function $f$ is the true regression function (that you are trying to estimate) and the error terms $\varepsilon_1,...,\varepsilon_n$ represent deviations of the observed response values from their true expected value.  You have developed a method to estimate the function, so let's denote your estimator as $\hat{f}$ and the resulting response estimates and residuals as:
$$\hat{y}_i = \hat{f}(x_i)
\quad \quad \quad
r_i = y_i - \hat{y}_i.$$
This is the standard notation for the problem, which is essentially the reverse of the notation you are using in your question.  Note that the "hats" on the variables refer to the estimates of the underlying true quantities, which are denoted without hats.  Consequently, the direction of the residuals is positive when the true value is higher than the estimated value.  I strongly recommend that you adopt the standard notational convention, so that you do not confuse people with your analysis.

Now that we have framed your problem in standard notation, let's get to the substance of your question.  Since this is a nonlinear regression problem, you can use standard measures of "goodness-of-fit" to evaluate the fit of your model.  I will not go into detail here, since there are entire books written on regression goodness-of-fit, but I will note that many of the standard statistics make use of the residual sum-of-squares:
$$SS_\text{Res} = \sum_{i=1}^n (y_i - \hat{y}_i)^2.$$
A lower value for the residual sum-of-squares constitutes a better "fit" for the model and the estimation method.  If your function estimator $\hat{f}$ is highly accurate then it will tend to give response predictions that are nearer to the true value, which will tend to reduce the magnitude of the residuals and reduce the residual sum-of-squares.
Since there are existing estimation methods for nonlinear regression problems, one obvious thing to do here would be to compare your algorithm with other estimation methods and see how they perform against one another (e.g., as measured by the residual sum-of-squares or some other goodness-of-fit statistic).  Ideally, to avoid overfitting you should use a set of training data to train each method and then use a separate set of test data to compare their performance.
Comment: Oh wow, I can't believe I missed the connection to regression! At first glance confidence and/or prediction bands seem to be the most relevant concepts for my application. However, my data is high-dimensional and my model non-differentiable. Have I understood correctly that this would make calculation of confidence bands a nuisance since those rely on the derivatives of the model w.r.t. the parameters? Also thank you for cleaning up my notation, you have spotted correctly that my background is indeed not in statistics.
Answer: In my final solution for my problem I closely followed the strategies detailed in [1]. As described in my question, my targets were two-fold:

Calculate a confidence-interval
Estimate the necessary sample size to determine a confidence interval of given width.

Among the strategies outlined in [1], the simplest one is based on the central limit theorem which states that for samples $\{x_1, \dots, x_n\}$ from an arbitrary distribution with mean $\Psi$ and variance $\Theta^2$ it holds that
$$
\frac{\overline{x} - \Psi}{\Theta/\sqrt{n}} \overset{n\to\infty}{\rightarrow} \mathcal{N}(0, 1),
$$
where $\overline{x} := 1/n\sum_{i=0}^n x_i$. If $n$ is large enough, a confidence-interval for the mean of any distribution can thus be created by
$$
 \left(\overline{x} - z\frac{\sigma(x)}{\sqrt{n}}, \; \overline{x} + z\frac{\sigma(x)}{\sqrt{n}}\right)
$$
since sample mean and standard deviation $\overline{x}$ and $\sigma(x)$ are unbiased estimators of $\Psi$ and $\Theta$ respectively.
Since this expression additionally easily allows to estimate the sample size $n$ for a prescribed confidence-interval width, this method was theoretically perfectly suitable for my purposes. However, the question left was: Is $n$ large in my application?
To check this, I wrote a small script to estimate the coverage of the so-determined confidence interval for various parameter combinations $(\Psi,\Theta,n)$ in the vicinity of the expected parameters of my application. Turns out, coverage for the 95% confidence interval is between 93% and 97%, which is completely sufficient for what I wanted to achieve.
[1] Pek, Jolynn & Wong, Augustine & Wong, Octavia. (2017). Confidence Intervals for the Mean of Non-Normal Distribution: Transform or Not to Transform. Open Journal of Statistics. 07. 405-421. 10.4236/ojs.2017.73029.

Distribution of errors


Postgresql execute procedure on update (value changed)

Question: I have the following trigger which calls the update_recipe_timestamp_proc function which updates the modified_on column.
If the same values gets inserted/updated the trigger is still triggered although there were no new values supplied!
Trigger
CREATE TRIGGER update_recipes_timestamp_r
BEFORE UPDATE OF "identifier", "name", "description"
ON recipes
FOR EACH ROW EXECUTE PROCEDURE update_recipe_timestamp_proc();

Function
CREATE OR REPLACE FUNCTION update_recipe_timestamp_proc()
RETURNS TRIGGER AS $$
BEGIN
    NEW."modified_on" = now();
    RETURN NEW;
END;
$$ language 'plpgsql';

Do I have to program the logic in my function and return NULL or NEW based on the result or is there a simple solution?
Answer: An UPDATE is performed no matter if the new values are different from the old ones, and the same applies to triggers.
Three possible solutions:

As suggested by Sevanteri's comment, use
CREATE TRIGGER update_recipes_timestamp_r
BEFORE UPDATE OF "identifier", "name", "description"
ON recipes FOR EACH ROW
WHEN (OLD IS DISTINCT FROM NEW)
EXECUTE PROCEDURE update_recipe_timestamp_proc();

Write the trigger procedure like this:
IF (OLD IS DISTINCT FROM NEW) THEN
   NEW."modified_on" = now();
   RETURN NEW;
END IF;

make the UPDATE conditional:
UPDATE recipes
   SET "identifier" = val1, "name" = val2, "description" = val3
WHERE ...
  AND "identifier" <> val1 OR "name" <> val2 OR "description" <> val3;

Comment: There is also the `WHEN` condition when creating the trigger. `WHEN (OLD.* IS DISTINCT FROM NEW.*)`
Comment: Right, that's even better!
Comment: @LaurenzAlbe I do not understand how I must use your solution. Maybe you could edit your or my answer. Thanks!
Comment: Is this better?
Answer: Solution with WHEN to only update the modified_on column on specific column changes.
CREATE TRIGGER update_recipes_timestamp_r BEFORE
UPDATE ON recipes FOR EACH ROW
WHEN (
OLD."name" <> NEW."name" OR
OLD."description" <> NEW."description" OR
OLD."identifier" <> NEW."identifier")
EXECUTE PROCEDURE update_recipe_timestamp_proc();


How do I nest parameter expansions for uppercasing and substitution in Bash?

Question: I have two bash string built in commands that work fine independently but when nested generate an error message no matter what I try. Here's the two individual commands that work:
$ A="etc/.java"
$ echo $A
/etc/.java
$ B="${A//$'\057\056'/$'\057'}"
$ echo $B
/etc/java
$ B="${A^^}"
$ echo $B
/ETC/.JAVA

Now trying to combine the two commands together I get errors:
$ B="${${A^^}//$'\057\056'/$'\057'}"
bash: ${${A^^}///.//}: bad substitution
$ B="${ ${A^^}//$'\057\056'/$'\057'}"
bash: ${ ${A^^}///.//}: bad substitution
$ B="${ ${A^^} //$'\057\056'/$'\057'}"
bash: ${ ${A^^} ///.//}: bad substitution
$ B="${"${A^^}"//$'\057\056'/$'\057'}"
bash: ${"${A^^}"//'/.'/'/'}: bad substitution
$ B="${ "${A^^}" //$'\057\056'/$'\057'}"
bash: ${ "${A^^}" //'/.'/'/'}: bad substitution
$ B="${${A^^} //$'\057\056'/$'\057'}"
bash: ${${A^^} ///.//}: bad substitution

Simplified examples are presented above so one can copy and paste to their own terminal. Piping or redirection would be complicated because my real world code is this:
  while [[ $i -lt $DirsArrCnt ]] ; do
    DirsArr[$i]=false
    CurrNdx=$i
    CurrKey="${DirsArr[$(( $i + 1 ))]}"
    # ^^ = convert to upper-case
    # ${Variable//$'\041\056'/$'\041'} = Change /. to / for hidden directory sorting
    if [[ "${"${CurrKey^^}"//$'\041\056'/$'\041'}" > \
          "${"${LastKey^^}"//$'\041\056'/$'\041'}" ]] || \
       [[ "${"${CurrKey^^}"//$'\041\056'/$'\041'}" = \
          "${"${LastKey^^}"//$'\041\056'/$'\041'}" ]] ; then
        LastNdx=$CurrNdx
        LastKey="$CurrKey"
        i=$(( $i + $OneDirArrCnt))
        continue
    fi

Comment: This? http://stackoverflow.com/questions/917260/can-var-parameter-expansion-expressions-be-nested-in-bash
Comment: You can't nest expansions like this. On another note, wouldn't `"${A//\/.\/}"` be simpler than the octal escapes?
Comment: @BenjaminW. Well I cheated a bit to make the example simpler because I had to remap `/` to `!` because `-` is lower on the ASCII table which means directory names with `-` sort lower than subdirectories with `/` path dividers. Thanks for stating it's impossible to nest. If you want to post an answer showing setting an intermediate variable inbetween, I'll accept and upvote.
Comment: Pretty sure this is a duplicate actually, let me check before answering ;)
Comment: I searched for an existing Q&A before posting... but I'm kind of search challenged so it may be there.
Comment: can you try to convert first in two steps (or whatever number of steps) before comparison?
Comment: Oh, I remember something. Do you always want to uppercase that variable? If so, there is a possibility to do it.
Comment: @CharlesDuffy Can you reopen this? In this very case it's possible with a solution not mentioned in the canonical.
Comment: @BenjaminW., done. Would you consider editing the question's title to narrow its scope to what your answer fits?
Comment: @CharlesDuffy Will do!
Comment: @BenjaminW. The upper case is needed because when sorting the directories `/etc/X11` appears before `/etc/acpi` when you set `LANG=C` which you need to do so that `/` sorts before `-`.
Answer: In the special case of one of the expansions being upper casing, it can be done in a single expansion, using declare -u (introduced in Bash 4.0). declare -u converts to uppercase on assignment.
Combining upper casing and substitution then becomes this:
$ declare -u A='/etc/.java'
$ echo "${A//\/./\/}"
/ETC/JAVA

There is the analogous -l for lower casing and the (undocumented) -c for title casing, but these are the only cases where you can do "nested" parameter expansion.
Comment: Works like a charm :). Thank you.


Statsmodels: How does statsmodels' VAR(p ) / VAR(p) in general works?

Question: I'm trying to use statsmodels' VAR(p) to forecast some data points in the future.
Below is a snapshot of what my data looks like:

What I understand about VAR(p) equation in general is that when p = 3, VAR(p) will use 3 lags time to predict the future. What I'm not sure is whether it uses 3 lags time consecutively or 3 lags time at a specific time.
For example (using my data):
A. p = 3, means 2016-08-15 00:20:00, 2016-08-15 00:15:00, 2016-08-15 00:10:00
OR
B. p = 3, means 2016-08-15 00:20:00, 2016-08-14 00:20:00, 2016-08-13 00:20:00
(same hour at different days)
Then, when it forecast the future, does it forecast the future consecutively or at same hour different days?
For example (to predict 3 data points), does VAR(p) outputs forecasted points on the following dates?
A. 2016-08-15 00:25:00, 2016-08-15 00:30:00, 2016-08-15 00:35:00
OR
B. 2016-08-16 00:20:00, 2016-08-17 00:20:00, 2016-08-18 00:20:00
Maybe I'm just way off...I'm not sure, that's why I'm asking for clarification.
Thanks for helping.
Answer: In general (not specifically about statsmodels), a VAR(p) model is built according to your option "A".
You are correct in that the value of a series the previous day at the same time may be useful in forecasting. Generally this is referred to as "seasonality" (because if you have monthly data, the season within the year may have some predictive power...this is the same, but about the time within the day).
I don't know if your specific package will do it, but generically you can build a seasonal version of a VAR(p) model by adding a lag at the seasonal period (12 x 24 = 288 five minute intervals). For example, in your case, you could build something like:
$Y_t = A_1 Y_{t-1} + A_{288} Y_{t-288} + e_t$
This model would depend both on the value 5 minutes ago and on the value at this hour yesterday.


How to change existing folder's icon via Command Line? (Windows 10)

Question: I have a folder on my desktop and 12 different icons. I want to create a scheduled task with Task Scheduler that runs as long as my PC is on and changes the icon of the folder every 15 minutes.
I have done my research and I found this code:
`attrib -h -r c:\test\desktop.ini
echo [.ShellClassInfo] >C:\test\desktop.ini
echo IconFile=%SystemRoot%\system32\shell32.dll>>C:\test\desktop.ini
echo IconIndex=0 >>C:\test\desktop.ini
attrib +h +r c:\test\desktop.ini
attrib +r c:\test`

However, I have no clue on how to implement/amend this to execute the task that I want to.
I would not like a Batch file appearing on my Desktop.
Any help would be appreciated.
Thanks.
Comment: I did ask a question.
Comment: You forgot to ask a question.
Answer: I was able to make a working model here:
Set DriveLetter=C
Set Pathing=test
Set IconPath=users\username\desktop\icon.ico
attrib -s -h -r %DriveLetter%:\%Pathing%\desktop.ini
echo [.ShellClassInfo] >%DriveLetter%:\%Pathing%\desktop.ini
echo IconFile=%DriveLetter%:\%IconPath%>>%DriveLetter%:\%Pathing%\desktop.ini
echo IconIndex=0 >>%DriveLetter%:\%Pathing%\desktop.ini
attrib +s +h +r %DriveLetter%:%Pathing%\desktop.ini
attrib +s +r %DriveLetter%:\%Pathing%
pause

Please note the following:
This doesn't require the need for "DriveLetter" I just like adding that to my scripts (Though having the drive letter specified is required, you just don't need the extra option it can be merged in with the path[Pathing] Option)
DriveLetter= The drive you want it on
Pathing= the path to the folder in question you wish to change
Iconpath= the full path to the icon's location
The result is I have a folder located @ E:\test\ that has the icon in question
you should be able to run that code and get output by only editing the top 3 lines, or you could write it in manually.
Comment: This isn't quite working for me. I can create the desktop.ini file and make it match exactly what is created if I use the UI to set the icon, but it never works. It changes the icon to some other default looking folder, but not the actual default. I don't understand.
Answer: You can use below script that I found it here.
If [%1] == [] goto :eof
ECHO [.ShellClassInfo] >%1\desktop.in
ECHO IconResource=C:\icon.ico,0 >>%1\desktop.in
move %1\desktop.in %1\desktop.ini
attrib +S +H %1\desktop.ini
attrib +R %1


Functions satisfying the below functional equations

Question: Find all functions from $f :\mathbb{R} \rightarrow \mathbb{R}$ such that it satisfies $|f(x)| = |x|$ , $f(f(-y))= -f(y)$ and $f(f(x))= f(x)$ $\forall$ $x,y \in \mathbb{R}$ .
What I did was: say $f(x) = x \forall x \in \mathbb R$ then it satisfies all conditions given ,so its a possible solution.  If we have $f(x) = -x$ for some part in the domain of $f(x)$, we will get that by the second and third condition that $f(-x) = f(x)$ and $f(f(x))= -f(x)$ but now I am not getting how to show whether this is possible or not.
Answer: Let $f$ be a solution of all three conditions. We get, for $x \in \mathbb{R}$:
$$f(x) = f(f(x)) = f(f(-(-x))) = - f(-x) $$
As such, $f$ is odd.
Suppose that, for a given $x \in \mathbb{R} \setminus \{0\}$, $f(x) \neq x$. Then, by using our conditions: $f(x) = - x$.
Thus:
$$f(x) = f(f(x)) = f(-x)$$
Since $f$ is odd: $$f(-x) = - f(x)$$
And so: $$-x = f(x) = 0$$
Absurd, which gives: $$\forall x \in \mathbb{R}, \quad f(x) = x$$
And $f$ is the identity.


android I/O Error: An existing connection was forcibly closed by the remote host

Question:
error :I/O Error: An existing connection was forcibly closed by the remote host

I tried killing and resetting ADB but no resolution.(Error happened in Samsung galaxy S5).
Answer: Finally Fixed : I had to update Android SDK .....
Comment: what do you mean you had to update Android SDK, can you provide a bit more to that answer?


Problemas com posicionamneto com colunas no Bootstrap 3

Question: Estou com um problema de posicionamento com as grids do bootstrap estou fazendo uma loja virtual onde o layout tem que ficar assim:

Porem com minha codificação ele ficou assim:

Percebam que nessa imagem os vinhos não formam as três colunas e ficam um em baixo do outro segue meu código:
O HTML:

Os melhores vinhos você encontra aqui

Vinho tinto

Vinho branco

Espumante

Vinho do porto

filtros

Vinho tinto Touriga

R$ 150,00

                        comprar

Vinho tinto Touriga

R$ 150,00

                        comprar

Vinho tinto Touriga

R$ 150,00

O CSS:
.title-category{
    font: italic 400 2em @font-family-serif;
}

.categories{
    font-size: 1.5em;

    .filter-category{
        background-color: @lightGray;

        .category{
            padding: 20px 40px;
            width: 100%;
            background-color: @black;
            display: block;
        }

        :hover{
            background-color: @gold;
        }
    }
}

.gallery{
    padding-top: 50px;

    .gallery-list{
        display: table;

        .wine-gallary{
            background-color: @gray;
            padding: 25px 85px;
            width: 100%;
        }

        .hover-shop{
            background-color: @gold;
            padding: 15px;
            width: 100%;
            display: block;
            font-size: 1.2em;
        }

        .description-wine{
            font: italic 400 1.5em @font-family-serif;
        }

        .price{
            color: @gold;
            font-size: 1.2em;
        }
    }
}

Observação: Estou usando less a pagina esta toda correta somente a parte dos vinhos que esta "zuada" vale lembrar que na versão mobile eles tem que ficar um em baixo do outro mesmo.
Answer: Antes de qualquer coisa, tente dar uma olhada a mais nesse link: http://getbootstrap.com/css/#grid ele mostra como trabalhar com a estrutura do bootstrap, pelo próprio bootstrap. A seguir, tenta essa estrutura abaixo:
...

Vinho tinto

Vinho branco

Espumante

Vinho do porto

 ... filtros aqui ....

 ... lista com produtos ....

Agora consegui postar aqui uhauha
Answer: Tudo indica que o problema está nos tamanhos das colunas do seu grid. A largura sem deve totalizar 12, se ultrapassar irá haver quebra de linha.

...

...

...

Somando as larguras das colunhas col-sm-N: 4 + 8 + 8 = 20
Como é mais que 12, haverá quebra de linha e o layout não ficará como esperado.
Ajuste para algo assim, deve funcionar.

...

...

...


What strings for classical guitar you advise to beginer?

Question: Recently, I found an old classical guitar(1989) in my cupboard and now I want to tune it up! But I need to change the strings.

Which parameter of tension is better for beginners?
Does the brand of strings matter?
What do the numbers in the description like 28-43, 29-46 mean?
Which strings will you recommend for an absolute beginer?
Comment: Welcome to Musical Practice & Performance.
Answer: Measure to find out whether your guitar is a full-sized guitar or a shorter-scale guitar. On a full-sized guitar, the length of the strings, from the bridge saddle to the nut, is 648mm.
For a beginner, if you have a full-scale guitar, just use a package of nylon classical guitar strings labeled "standard tension" or "normal tension". Any brand will do.
The numbers indicate the diameter of each string, in fractions of an English inch, or in millimeters. Some musicians prefer a particular diameter of string for each of the six string positions, so they want to know the exact specifications.
If you have a shorter-scale guitar, you may need to find a special set of strings (all of which will be thicker in diameter) designed to provide the proper tension and tuning for a short-scale instrument.


Dados não foram alterados no banco - laravel 5.7 eloquent

Question: Boa tarde pessoal, estou tentando fazer um update porem os valores inseridos nos campos não salva no banco, segue arquivos para verificarem.
Index.blade.php
@extends('layouts.app')
@section('content')
<?php
if (session('message')):?>

        {{ session('message') }}

 <?php
  else:?>

 <?php
      endif;
       ?>

      Cadastrar uma nova aula

    Ritmo
    Dia da Semana
    Horário
    Status
    Alterar
    Excluir

<?php
$RowsCount = $Aulas->count();

if($RowsCount >= 0 ):

      foreach($Aulas as $models):
      ?>

        <?php echo $models['Ritmo']; ?>
        <?php echo $models['DiaSemana']; ?>
        <?php echo $models['Horario']; ?>
        <?php echo $models['Status']; ?>

         Id)}}">A

              <form method="POST" action="{{ route('Aulas.destroy', $models->Id) }}">
                @csrf
                @method('DELETE')
                <button type="submit" name="btn-deletar" class="btn btn-warning btn-outline-warning btn-icon" data-type="success" data-from="top" data-align="right">E</button>
              </form>

       <?php
      endforeach;
      else: ?>

        -
        -
        -
        -

       <?php
      endif;
       ?>

@stop

route/web
<?php

/*
|--------------------------------------------------------------------------
| Web Routes
|--------------------------------------------------------------------------
|
| Here is where you can register web routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| contains the "web" middleware group. Now create something great!
|
*/

Route::get('/', function () {
    return view('welcome');
});

Auth::routes();
Route::group(['middleware' => 'web'], function () {

    Route::get('/home', 'HomeController@index')->name('home');

    Route::get('/ListAula', 'CadastrosBasicos\AulasController@Index')->name('aulas');

    Route::get('/CadAula', 'CadastrosBasicos\AulasController@create')->name('create');

     Route::get('/edit', 'CadastrosBasicos\AulasController@edit')->name('edit');

    Route::post('/store', 'CadastrosBasicos\AulasController@store')->name('store');

    Route::post('/update', 'CadastrosBasicos\AulasController@update')->name('update');

    Route::resource('Aulas','CadastrosBasicos\AulasController');

});

Controller
<?php

namespace App\Http\Controllers\CadastrosBasicos;

use Illuminate\Foundation\Auth\AuthenticatesUsers;
use App\Models\CadastrosBasicos\Aulas;
use Illuminate\Routing\Controller;
use Illuminate\Http\Request;

class AulasController extends Controller
{
   public function edit(Request $request, $Id )
    {
         $Aula = Aulas::find($Id);
         return view('CadastrosBasicos.Aulas.Editar', compact('Aula'));
    }

    public function update(Request $request, $Id)
    {
      $AulaDb = Aulas::find($Id);
      $AulaDb->Id = $Id;
      $AulaDb->Ritmo = $request->get('Ritmo');
      $AulaDb->DiaSemana = $request->get('DiaSemana');
      $AulaDb->Horario = $request->get('Horario');
      $AulaDb->Status = $request->get('Status');
      $AulaDb->save();

       return redirect() ->route('aulas')
                       ->with('message', 'Aula Alterada com sucesso!');
    }
}

Moldes
<?php

namespace App\Models\CadastrosBasicos;

use Illuminate\Database\Eloquent\Model;

class Aulas extends Model
{
    protected $table = 'Aulas';
    public $timestamps = false;

    protected $fillable = ['Id','Ritmo','Horario','DiaSemana','Status'];

}

Comment: Faltou colocar o formulário que está usando para atualizar.
Comment: as suas rotas não tem padrão, e ruim se o projeto for crescendo, os nomes também deveria seguir um nomenclatura, tudo isso é bom para não ter choque de rotas.
Answer: Eai Claudio blz?, seguinte cara, no seu method update, vc informa que está passando um ID, e com esse ID, vc está procurando pela Aula. Mas, na sua rota vc não está passando um ID pra o method, vc deve padronizar a sua rota passando o ID assim:

Route::post('/update/{aulaId:[0-9]+}', 'CadastrosBasicos\AulasController@update')->name('update');

OBS: Lembre-se de passar o ID no button de update que acessa a rota, como vc não postou o código dela, não sei como está.
E no seu method, vc captura esse ID (Lembrando, vc não precisa inserir o ID novamente um linha que vc está fazendo o update):

public function update(Request $request, $aulaId)
{
  $data = $request->all();
  $AulaDb = Aulas::find($aulaId);
  $AulaDb->Ritmo = $data->Ritmo;
  $AulaDb->DiaSemana = $data->DiaSemana;
  $AulaDb->Horario = $data->Horario;
  $AulaDb->Status = $data->Status;
  $AulaDb->save();
  return redirect() ->route('aulas')
                   ->with('message', 'Aula Alterada com sucesso!');
}

Outra forma que vc pode também fazer é assim, caso, os campos que vc está passando no request sejam iguais aos do banco:
public function update(Request $request, $aulaId){
     $aula = Aulas::findOrFail($aulaId);
     $aula = $aula->fill($request->all());
     $aula->save();
}


A proof: Closure of a set of complex numbers is closed

Question: I'm assuming the proof should apply for complex, and real numbers. I've offered a proof for complex numbers, and I am not so sure about some of the steps (namely $a_n \in \bar S_c)$ I take so a second look/hints would be appreciated! The only other theorem I have is that a closed set implies every sequence in the set converges to a point in the set.
Let $\bar S = S \cup \partial S$ denote the closure of S, with $\partial S$ being the boundary. Also let $\bar S_c$ be the complement of the closure (sorry if that is unconventional).
Suppose $\bar S_c$ is closed. Consider $z \in \partial S$, then $D(z, r) \cap S_c \neq \emptyset$ for any $r > 0$. Take $a_n \in D\left(z, \frac{r}{n}\right) \cap S_c$ for $n = 1, 2, ...$ Then $a_n \in \bar S_c$ because $a_n \notin S, a_n \notin \partial S$. Clearly, $a_n \rightarrow z$ as $ n \rightarrow \infty$. However, $z \in \bar S$ and $a_n \in \bar S_c$ which means there is a convergent sequence in $\bar S_c$ converging to a point outside $\bar S_c \implies \bar S_c$ is open $\implies \bar S$ is closed.
Comment: What is the definition of $\partial S$ and what is the definition of "closed"?  Surely this is a matter of definition.
Answer: You can prove your result directly from the definition of closure.
And not that $\bar{S}$ contains all the limit points of $S$
Let $x \notin \bar{S}$
Then exists $r>0$ such that $B(x,r) \cap S=\emptyset$ and $B(x,r)$ does not contain any limit point of $S$ because if it had a limit point $y$ of $S$ then  from the definition of the limit point $B(x,r)$ as a neighborhood of $y$ would intersect $S$ which is absurd.
Thus $B(x,r) \subseteq \mathbb{C}$ \ $\bar{S}$.
So for every element $x$ in the complement of the closure of $S$ we find an open ball with center $x$ which is a subset of the complement.
So  $\mathbb{C}$ \ $\bar{S}$ is open.
Also $a_n$  need not to belong to  $\bar{S}^c$
Take for instance the interval $[0,1]$.
We know that $1 \in \partial{([0,1])}$
Take $$a_n=\begin{cases}\
          1-\frac{1}{n} & n=2k+1\\
          1+\frac{1}{n} & n=2k\\
          \end{cases}$$
and $a_n \to 1$
Comment: @danny...From the definition of the closure..remember $x \in \bar{A}$ if for all $r>0$ we have that $B(x,r) \cap A \neq 0$..Now if a point does not belong to the closure of $A$ ,what is the ''negation'' of this definition?
Comment: Yes haha that seems much more straightforward way to see it. Thank you!
Comment: How do we know such an $r$ exists?
Comment: To construct a sequence you apply the definition inductively for $r=1,\frac{1}{2}...\frac{1}{n}....$
Comment: I'm not sure I follow? The sequence I constructed was for a point in the boundary of S but here you are talking about a point in the complement of the closure?
Comment: @danny i don't understand..you talk about $a_n$?
Comment: I am saying I'm not following how we know that such an $r$ exists to satisfy $B(x, r) \cap S = \emptyset$?
Comment: In other words  every open ball with $x$ as its center  intersects $A$..Now if $x$ is not in the closure then exists a such a ball that does not intersect $A$
Comment: Thank you! I think I got it now
Answer: We will prove that $$\bar S =\bigcap_{\ \ \ S\subseteq F\\ F\ \text{closed}} F.$$
Not only will this give us that $\bar S$ is closed as intersection of closed sets, it will give us extremely useful characterization of closure (which I personally prefer to be definition): the closure of $S$ is the smallest closed set containing $S$.
Let $F$ be closed such that $S\subseteq F$. We want to prove that $\bar S = S\cup \partial S\subseteq F$ and by the choice of $F$, it is enough to prove that $\partial S\subseteq F$.
Assume the contrary, i.e. there exists $x\in\partial S$ such that $x\not\in F.$ Then, $x\in F^c$, which is open. Thus, there exists open ball $B(x,r)$ such that $B(x,r)\subseteq F^c$. But, since $x\in\partial S$, it means that $B(x,r)\cap S\neq\emptyset$. Let $y\in B(x,r)\cap S$. In particular, $y\in S\subseteq F$, so $B(x,r)\cap F\neq\emptyset$. But this is in contradiction with $B(x,r)\subseteq F^c$. Thus, $\partial S\subseteq F$ and consequently $\bar S\subseteq F$.
Since $F$ was arbitrary, it follows that
$$\bar S \subseteq\bigcap_{\ \ \ S\subseteq F\\ F\ \text{closed}} F.$$
Let us now prove the other inclusion. Let $$x\in\bigcap_{\ \ \ S\subseteq F\\ F\ \text{closed}} F$$ and assume that $x\not\in S$. We will prove that $x\in\partial S$ then, which will complete our proof.
If $x\not\in\partial S$, then there exists open ball $B(x,r)$ such that $B(x,r)\subseteq S^c$. Let $F = B(x,r)^c$. Then, $F$ is closed and $S\subseteq F$, so by our choice of $x$, $x\in F$. Contradiction. Thus, if $x\not\in S$, then $x\in\partial S$, which proves $x\in S\cup\partial S$.
Answer: A set is closed if and only if it contains its limit points.
$\partial S = \{\text{limit points  of } S\} \cap \{\text{limit points of } S^c \}$.
It suffices to prove:
Lemma:  If $y$ is a limit point of $\overline S = S\cup \partial S$ then $y$ is a limit point of $S$.
Thus every limit point $\overline S$ is in $\{\text{limit points  of } S\} \subset \overline S$.  So $\overline S$ is closed.
Proof of Lemma:
Suppose not.  Let $y$ be a limit point of $\overline S$ that is not a liit point of $S$.  Then there exist an $\epsilon > 0$ so that $B_{\epsilon}(y)$ contains no points (other than possible $y$ itself) that are in $S$.
But $y$ is a limit point of $\overline S$ so $B_{\epsilon}(y)$ contains a $w \in \overline S$.  But $w \not \in S$ so $w \in \overline S \setminus S = \partial S \subset \{\text{limit points of } S\}$.
So $w$ is a limit point of $S$.  Let $\delta = \min(d(w,y), \epsilon - d(w,y))$.
The there exists a point $z\in S$ that is in $B_{\delta}(w)$.
But $d(z,y) \le d(w,z) + d(w,y) < \delta + d(w,y) \le \epsilon - d(w,y) + d(w,y) = \epsilon$.  So $z \in B_{\epsilon} (y)$.
Which is a contradiction.
====
One graphical way of viewing limit  points is $z$ is a limit point of $X$ if $\inf \{d(x,z)|z \in X\} = 0$.
Which would mean if $x$ is a limit point of $\overline S$ means $\inf d(x,y):y\in \overline S = 0$ but $\inf d(y, s):s \in S = 0$ .  So $\inf d(x,s)= 0$.  So $x$ is a limit point of $S$.


Custom module with its own menu?

Question: I am in the process of writing my 3rd custom module and this one is my biggest challenge yet.
I know how to use hook_menu to generate menu items, and depending on the path you choose determines where they show up on the site (administration menu or main menu, etc).
My question is - how do I specify a NEW menu, so that the paths I create do not appear in "primary links" or "navigation" but in their own "custom module" menu?
Answer: You specify the menu_name parameter.
From the devel module:
<code>$items['devel/cache/clear'] = array(
  'title' => 'Empty cache',
  'page callback' => 'devel_cache_clear',
  'description' => 'Clear the CSS cache and all database cache tables which store page, node, theme and variable caches.',
  'access arguments' => array('access devel information'),
  'menu_name' => 'devel',
);
</code>
Comment: Note that you will need to create your menu first in hook_install() or so. The menu system doesn't automatically create a menu for you. See http://drupalcontrib.org/api/drupal/contributions--devel--devel.install/function/devel_install/6

Custom module with its own menu?


Display additional fields in the Inbox?

Question: I have added two new fields (a Viewfield and a simple text field), and I would like to display these fields in the Inbox (/messages) along with the existing Subject, Participants, and Last Updated data. How can I add those fields to that page ? Is there a tpl format I could use ?
Answer: /**
 * Implements hook_form_alter().
 */
function modulename_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'privatemsg_list') {
    $form['updated']['list']['#header']['bar_column_field']['data'] = 'Bar Column';
    $form['updated']['list']['#header']['bar_column_field']['field'] = 'bar_column_field';
    $form['updated']['list']['#header']['bar_column_field']['class'][] = 'privatemsg-header-bar-column';
    $form['updated']['list']['#pre_render'][] = '_pre_render_bar_column';
  }

/**
 * Custom #pre_render function.
 */
function _pre_render_bar_column($tableselect) {
  $i = 0;
  foreach ($tableselect['#options'] as $id => $options) {
    $data = 'Bar data in the table ' . $i++ . '';
    $tableselect['#options'][$id]['bar_column_field'] = array('data' => $data, 'class' => array('privatemsg-list-bar'));
  }
  return $tableselect;
}

You can change the $data variable with any field's value, after performing a query or something, and placing your actual data to the new column.
A field could be attached to messaging (admin/config/messaging/privatemsg/fields) or to any other available entity.

Display additional fields in the Inbox?


Lotus Notes version 5.0.8 database migration

Question: This is my first time ever working with Lotus Notes, where the client want to move its database  to mysql.
I have spent a week to read on Lotus Notes but going nowhere. When I search for some tool to migrate database, it does not support version 5.
It is a better idea to upgrade to latest version and than migrate...please advice...
Answer: Yes, you can update to latest IBM Notes/Domino version and database will still run without any change. That is one of the cool things about Notes...
Comment: As IBM Notes is a document oriented system there are no data tables like in relational database systems. A good way to find out about application's data structure is to look at the forms and views. In forms are defined (almost) all the fields. In views are listed documents created by forms. Other ways are using Notes-ODBC or to export/transform database design as XML (=DXL) at menue "Tools / DXL Utilities" in Notes Designer.
Comment: Hi, thanks...is there a way to find out number of tables and list of all field names.


Use column header and index values for operation

Question: I have two dataframes which are constructed from pivot tables of an elevation profile and they look like below. The second frame is a shifted version of the first time to the right by one column
Frame 1
Lat          32.063102  32.063194    ...      32.124676  32.124769
Lon                                  ...
-100.214676    617.143    617.167    ...        681.024    680.839
-100.214583    617.316    617.331    ...        680.916    680.730
-100.214491    617.495    617.497    ...        680.802    680.609
-100.214398    617.673    617.679    ...        680.690    680.499
-100.214306    617.862    617.862    ...        680.589    680.401

Frame 2
Lat          32.063194  32.063287    ...      32.124769  0.000000
Lon                                  ...
-100.214676    617.167    617.175    ...        680.839          0
-100.214583    617.331    617.337    ...        680.730          0
-100.214491    617.497    617.505    ...        680.609          0
-100.214398    617.679    617.687    ...        680.499          0
-100.214306    617.862    617.867    ...        680.401          0

Is it possible to use the column headers and index values for operation when I try to construct a new dataframe? For example, when I subtract the second dataframe from the first, can I use the sum of square of the respective column header and index as a denominator for every value?
I tried storing the row and column headers as separate lists but couldn't integrate them when iterating over both dataframes.
Any help is appreciated, thanks.
Comment: Please provide a [minimal, complete, verifiable example](https://stackoverflow.com/help/mcve) of your code. Also, please explain exactly what you want to do with the row and column indices. There are many ways to use these in sum-of-squares computations and it is not clear what you mean by _denominator_ as a subtraction has no denominator. E.g., do you want the first element of your resulting matrix to be `(617.143 - 617.167) / (32.063102**2 + (-100.214676)**2 + 32.063194**2 + (-100.214676)**2)`?
Comment: The first element should be the (617.143 - 617.467)/(32.063102-32.063194). Sorry for not being complete
Answer: You want to be able to do the computation by generic column indices (as you explained in your response to my comment). This adds an additional difficulty, since the standard behavior for computations with DataFrames (i.e. df1 - df2 or so) is to match row and column indices. This will in your case result in NaNs and useless results.
Consequently, you will have to replace the column indices by generic ones. However, if you put the original column indices in DataFrames of the same shape as the original, you can still use those in the computation.
Example:
import pandas as pd
import numpy as np

"""generate example data"""
df1 = pd.DataFrame(columns=[32.063102, 32.063194])
df1.loc[-100.214676] = [617.143, 617.167]
df1.loc[-100.214583] = [617.316, 617.331]
df1.loc[-100.214491] = [617.495, 617.497]

df2 = pd.DataFrame(columns=[32.063194, 0])
df2.loc[-100.214676] = [617.167, 0]
df2.loc[-100.214583] = [617.331, 0]
df2.loc[-100.214491] = [617.497, 0]

"""generate auxiliary dataframes with original column indices"""
df1i = pd.DataFrame()
df1i = df1i.append([df1.columns]*len(df1.index))
df1i.index = df1.index

df2i = pd.DataFrame()
df2i = df2i.append([df2.columns]*len(df2.index))
df2i.index = df2.index

"""replace column indices by generic ones to allow the computation by generic column index"""
df1.columns = np.arange(len(df1.columns))
df2.columns = np.arange(len(df2.columns))

"""do the computation"""
df3 = (df1 - df2) / (df1i - df2i)
print(df3)


How to check the Webform submission values in a twig template

Question: I have an Webform with an option element, (option_one), with 6 options.
Below this are 2 option elements with 5 and 2 options.
These 2 option elements have a conditional statement to display if the correct option is selected on the, (option_one), option element.
Added email handlers to the form for the site admin and the submitter.
Using the default email format, all display as per the Webforms default email output, with the correct option element hidden as per the coditional statements.
Goal
Build my own twig template to enable me to theme the email using inline styles.
Below is the code used to check for the selected options.
UPDATE
Question simplified:
How to compare a Webform token with a given value in a twig 'if' statement?
{% if webform_token('[webform_submission:values:airports_to_radio]', webform_submission, [], options) == Heathrow %}
Don't want or need to go and build a submit handler via hook_form_alter()
{% if webform_token('[webform_submission:values:airports_to_radio]') == 'Heathrow' %}

{{ webform_token('[webform_submission:values:heathrow_terminals_to]', webform_submission) }}

{% endif %}

{% if webform_token('[webform_submission:values:airports_to_radio]') == 'Gatwick' %}

{{ webform_token('[webform_submission:values:gatwick_terminals_to]', webform_submission) }}

{% endif %}

This works to a point. Both sections of the twig template do not display, so i assume that the syntax is correct, but not using the correct comparison.
Any help to get the correct syntax for the {{ if }} statement would be great, or any links to information on the web would also be great.
Webform source code below.
airports:
  '#type': fieldset
  '#title': Airports
  airports_to_radios:
    '#type': radios_other
    '#title': Airports
    '#options':
      Heathrow: Heathrow
      Gatwick: Gatwick
      Stansted: Stansted
      Southend: Southend
      'London City': 'London City'
      Luton: Luton
    '#options_display': buttons
    '#other__option_label': Hotel
    '#other__title': Hotel
    '#other__placeholder': Hotel
  heathrow_terminals_to:
    '#type': radios
    '#title': 'Heathrow Terminals'
    '#options':
      'Terminal One': 'Terminal One'
      'Terminal Two': 'Terminal Two'
      'Terminal Three': 'Terminal Three'
      'Terminal Four': 'Terminal Four'
      'Terminal Five': 'Terminal Five'
    '#options_display': buttons
    '#states':
      visible:
        ':input[name="airports_to_radios[radios]"]':
          value: Heathrow
  gatwick_terminals_to:
    '#type': radios
    '#title': 'Gatwick Terminals'
    '#options':
      'South Terminal': 'South Terminal'
      'North Terminal': 'North Terminal'
    '#options_display': buttons
    '#states':
      visible:
        ':input[name="airports_to_radios[radios]"]':
          value: Gatwick
  customer_email:
    '#type': email
    '#title': 'Customer Email'
    '#default_value': test@localhost

Comment: Try to make the question more clear please. I understand that you need to send an e-mail with a custom template after the webform is submitted? You want to do that because the in-built webform UI does not provide you enough customization level, am I right?
Comment: How to compare a Webform token with a given value in a twig 'if' statement?
Comment: I think I finally understood what you need and your current problem. Check my answer, I think the problem is with the type of data. Your code would work for text fields, not for select/radios/checkboxes.
Answer: Adding :raw to your webform submissions value has worked for me.
{% if webform_token('[webform_submission:values:airports_to_radio:raw]') == 'Heathrow' %}


Simulation of Fisher's exact test is underestimating power

Question: Summary: My simulations don't match my power calculation.
(This question has become rather long, so you might not read to the bottom. I think it's a sample size problem.)
I'm running some simulations to help design a study, and I'm running into something that's confusing me.
There are two groups, treated and untreated, and the outcome is crashing.
We have an estimated crash rate of 0.1 in the untreated group, and think we can bring this down to 0.08.  A small (and non-feasible) sample size is 100 per group.
So I run power.prop.test() to estimate the power.
    power.prop.test(p1=0.1, p2=0.08, n=100)

    power.prop.test(p1=0.1, p2=0.08, n=100)

         Two-sample comparison of proportions power calculation

                  n = 100
                 p1 = 0.1
                 p2 = 0.08
          sig.level = 0.05
              power = 0.07122973
        alternative = two.sided

 NOTE: `n` is number in *each* group

Then I ran a simulation, because I wanted to explore how often we'd make the wrong decision as to the best treatment.
    library(Exact)

    #Create a data frame called d, populate it with the numbers
    #above.
    set.seed(12345)
    nTreated <- 100
    nUntreated <- 100
    probTreated <- 0.1
    probUntreated <- 0.08

    d <- data.frame(id = 1:10000)
    d$nTreated <- nTreated
d$nUntreated <- nUntreated
    d$probTreated <- probTreated
d$probUntreated <- probUntreated

    #Generate some random results using rbinom()
    d$treatedCrashes <- apply(cbind(d$nTreated, d$probTreated),
    1,  function(x)  sum(rbinom(x[1], 1, x[2])))

    d$untreatedCrashes <- apply(cbind(d$nUntreated,
    d$probUntreated), 1, function(x)  sum(rbinom(x[1], 1,
                                                  x[2])))

    #Do fisher's exact test on each replication:
    d$fisher <- apply(cbind(d$nTreated - d$treatedCrashes,
                    d$treatedCrashes,
                            d$nUntreated - d$untreatedCrashes,
                            d$untreatedCrashes), 1,
                      function(x)  fisher.test(matrix(x,
nrow=2))$p.value)
    #test power
    mean(d$fisher < 0.05)

And I get 4.8% power, which is lower than the power.prop.test function said, AND is less than 0.05 - which seems kind of wrong.
        mean(d$fisher < 0.05)
    [1] 0.0478

Is this about the small sample approximation? Is it a stupid coding error? (I don't think it is, but I've often been wrong about that before). Is it something I haven't thought of?
In response to the suggestion that it's because the Fisher's exact test is conditioned on the margins, I reran the model with Barnard's test (in the Exact library). (But I reduced to 1000 replications, as this took 40 minutes).
    d$exact <- apply(cbind(d$nTreated - d$treatedCrashes,
                     d$treatedCrashes,
                             d$nUntreated - d$untreatedCrashes,
                             d$untreatedCrashes), 1,
                   function(x)  exact.test(matrix(x, nrow=2),
to.plot=F, cond.row=T)$p.value)
    d$exact <- lapply(d$exact, function(x) x[1][[1]])

    mean(d$exact < 0.05)

    mean(d$exact < 0.05)
    [1] 0.049

I have pretty much the same power.
However, I also ran an exact test for power using the power.exact.test() function, also in the Exact library, which gives a very similar level of power:
    power.exact.test(p1=0.1, p2=0.08, n1=100,
         n2=100,simulation=T, nsim=1000, method="Boschloo")
    $power
    [1] 0.045

    $alternative
    [1] "two.sided"

    $method
    [1] "Boschloo"

Which makes me think that it's a sample size problem.
Answer: Fisher's Exact Test is conservative (i.e. the false positive rate for the nominal 0.05 test is actually less than 0.05 when the null hypothesis is true). What you're finding there is no coincidence. To say that a test is "exact" does not mean that it is of the right size, but that the interpretation of the p-value in small samples is correct.
Here's a reference from university lecture notes on biostatistics
Applied Biostatistics by Scott S. Emerson, Professor of Biostatistics
University of Washington
Comment: This app https://tslumley.shinyapps.io/Exactly shows the rejection regions in 2x2 tables for a set of exact and non-exact tests, so you can see how the tests differ.  Fisher's exact test tends to be more conservative than the other exact tests (`power.prop.test` corresponds to 'Exact Wald', I think).
Answer: Fisher's test is conditional on both margins of the table; your simulation is only conditional on one margin.
[You're right, there's more to it:
(1) Your first calculation with power.prop.test uses the asymptotic approximation of the Binomial distribution to the Normal, so won't give exactly the same answer as an exact test.
(2) Any exact test will be conservative because there's only a finite number of possible contingency tables & you can't find a subset to form the rejection region with exactly 5% probability under the null hypothesis, so you have to settle for a bit less (@AdamO's answer).
(3) There are different exact tests formed by conditioning on the grand total, the row total, the column total, or both row & column totals. Choosing a different conditioning scheme means (a) asking a different question of the test, & (b) changing the number of possible contingency tables (so tests that condition most tend to be most conservative - e.g. Fisher's).
(4) There are also different test statistics that can be used, which do not necessarily give the same ordering of possible tables under the null hypothesis.
(5) And when you're simulating, there's simulation error to take into account (@Maarten's answer).]
Comment: Thanks for that, but I don't think it's the (whole) answer. I've added some details to the question, where I've used a Barnard's test, and it doesn't make much difference.
Answer: There is randomness in your Monte Carlo simulation (as there should be), so even if the true rejection rate is exactly .05 and there are no errors in your program you'll still be likely to find small deviations from that number. If you were to repeat your Monte Carlo simulation with 10,000 replications many times and the test functioned exactly as it should, then you would still expect to find that (approximately) 95% of these simulations would result in rejection rates between 4.6% and 5.4%. These bounds are based on the 2.5$^{\mathrm{th}}$ and 97.5$^{\mathrm{th}}$ percentiles of a binomial distribution with n=10,000 and p=.05. So the fact that you found a rejection rate of 4.8% in a simulation with 10,000 replications does not provide all that strong evidence against the Fisher's exact test. As AdamO pointed out, you'll in all likelihood find the rejection rate to be less than 5% even if you increase the number of replications, but the current simulation does not have enough replications to reliably find that phenomenon.


Reminder action for users

Question: I'm using drupal 7.
Simply i need to remind some user every 28th to do something.
What the best module to make a reminder for users?
Answer: You could use the Rules module to do this.

Reminder action for users


How to call classes from a variable variable?

Question: I'm not quite sure how to correctly put this question. I want to dynamically call functions that are contained in classes (I think this means they are called 'methods').
Here is an example of my code which I hope helps explain what I am trying to achieve.
In this instance $result returns all the different modules that are loaded. This then checks if the module's PHP file has been included with it's class, then if that class exists - trys to call the class directly.
foreach ($results as $result) {
    $moduleclass_name = 'TestClassName_' . $result->module_name . '::FunctionToCall';
    if (method_exists($moduleclass_name, 'FunctionToCall'))
        $VariableToRetrieve = $modulefunction_name($Parameter1, $Parameter2);
}

This returns an error

"Call to undefined function
  TestClassName_modulename::FunctionToCall()"

although the 'TestClassName' has been declared correctly.
Can someone tell me what I'm doing wrong?
Comment: Can you post the segment of code where you create the class and method ?
Comment: Also: What is the difference between `$moduleclass_name` and `$modulefunction_name`?
Answer: What you want is probably call_user_func_array().
The code would look similar to this:
call_user_func_array(array($classNameOrInstance, $functionName), array($arg1, $arg2, $arg3));

EDIT Also, in your example you seem to have included the function name in the class parameter for method_exists, too...
Answer: You may use call_user_func() as such to achieve what you are trying to do. Also, it is better to use is_callable() instead of method_exists() to validate if the method is callable (method may exist but its visibility may prevent it from being callable.
foreach ($results as $result) {
    $module_callback = array('TestClassName_' . $result->module_name,'FunctionToCall');
    if (is_callable($module_callback))
        $VariableToRetrieve = call_user_func($module_callback, $Parameter1, $Parameter2);
}

Answer:
I think it doesn't work because your syntax may not support "static method calls".
I suggest you give a try to Franz's method call_user_func().
I did a similar thing on a former project.

It was designed to call a class which implemented an interface, so method names where known.
I don't think it's difficult to modify this code in order to make it match with yours.
class CDispatcher {
    public static function GetDispatcher( $module = 'core' ) {
        $class_name = $module . 'Dispatcher';

        try {
            // looks for the file associated with the class
            // if the file is not found an exception is raised
            search_class( $class_name );
        } catch ( exception $e ){
            throw new UnkwownModuleException($module);
        }

        return new $class_name();
    }
}

// Then, you call this class :
$new_instance = CDispatcher::GetDispatcher( $my_module );


List Azure role assignments using Azure Logicapps

Question: I'm new using logic apps and I'm trying to get information about RBAC assignments to users. I was using graph to get information about AD user as you can see in this image below.

In this documentation I found how to list this information but using management.azure.com and I don't know what kind of permissions my logicapp needs and if this step is configured correctly.
Answer: You need to enable system managed identity in logic app and also need to add role assignment for logic app in subscription.
I have reproduced issue from my side and below are steps I followed,

Created logic app as shown below,

Enabled system managed identity in logic app,

When I run logic app got below error,
"code": "AuthorizationFailed",
"message": "The client 'XXXXX' with object id 'XXXXX' does not have authorization to perform action 'Microsoft.Authorization/roleAssignments/read' over scope '/subscriptions/XXXXX' or the scope is invalid. If access was recently granted, please refresh your credentials."

Added role assignment for the created logic app as a contributor role in Subscription access control,

5. After adding role assignment, logic app ran successfully,

Reference link


Show that there do not exist any integer $m,n$ such that $\frac{m}{n}+\frac{n+1}{m}=4$

Question: The question is
Show that there do not exist positive integers $$m,n$$ such that   $\frac{m}{n}+\frac{n+1}{m}=4$ . It is the problem.I tried to make a quadratic equation of m. And I found that we have to show that there do not exist positive integer $n$ such that $\sqrt{3n^2-n}$ .
But how?? Anyone have any idea!
Comment: http://www.artofproblemsolving.com/community/c3046h1046758__  http://www.artofproblemsolving.com/community/c3046h1046841___
Answer: Continuing your proof on showing that the discriminant $D$ is not a perfect square:
$D= 3n^2-n = n(3n-1)$
$\gcd(n, 3n-1) =1$ by elementary methods.
So we need both $3n-1$ and $n$ to be perfect squares. But $3n-1$ cannot be a perfect square taking mod 3.


Python Twisted JSON RPC

Question: Can anyone recommend some simple code to set up a simple JSON RPC client and server using twisted?
I found txJSON-RPC, but I was wondering if someone had some experience using some of these anc could recommend something.
Answer: txJSONRPC is great. I use it and it works. I suggest you give it a try.
SERVER:
from txjsonrpc.web import jsonrpc
from twisted.web import server
from twisted.internet import reactor

class Math(jsonrpc.JSONRPC):
    """
    An example object to be published.
    """
    def jsonrpc_add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b

reactor.listenTCP(7080, server.Site(Math()))
reactor.run()

CLIENT:
from twisted.internet import reactor
from txjsonrpc.web.jsonrpc import Proxy

def printValue(value):
    print "Result: %s" % str(value)

def printError(error):
    print 'error', error

def shutDown(data):
    print "Shutting down reactor..."
    reactor.stop()

proxy = Proxy('http://127.0.0.1:7080/')

d = proxy.callRemote('add', 3, 5)
d.addCallback(printValue).addErrback(printError).addBoth(shutDown)
reactor.run()

As a bonus, I will leave some alternative: amp.
http://amp-protocol.net
Comment: Worked fine except for two minor changes. I needed to change
class Math(jsonrpc.jsonRPC):
to
class Math(jsonrpc.JSONRPC):
in the server

and
the ports mismatch in the client and server code, have them agree on one and it works fine
Comment: It might be better to use `d.addCallback(printValue); d.addErrback(printError)` instead of `d.addCallbacks(printValue, printError)` in order to catch errors from `printValue()`.
Comment: @1337Rooster, @J.F. Sebastian: I've fixed the issues you guys mentioned for future readers. Thanks for the comments.
Comment: How do i add extra headers to the request like csrf tokens? For example Transmission's rpc server requires X-Transmission-Session-Id to be appended to the request header.
Comment: JSON-RPC  2.0 is not  supported  yet by txjsonrpc.
took  me a while to  figure out .
Answer: If you are looking for a framework-independent approach, this lib I pushed (using mixin) might be helpful:
Answer: Cyclone, a Tornado async web server implementation written using twisted, has a built-in json-rpc request handler that uses the python json/simplejson module.  Example server and client code is here.
Answer: wikipedia has a bunch of implementations listed for python: https://en.wikipedia.org/wiki/JSON-RPC#Implementations
That said, txjason feels like the one best integrated with twisted. It seems to support out of order responses out of the box for example. Most of it would be portable to python3 using six. The most horrible part is the parameter validation, which is not exposed in the normal public API anyway.
Answer: For me this worked  better  then  "libraries" , speaking  of client.
    TESTDATA = {'id': 1234,
                'method': 'getbalance',
                }
    URL = 'http://localhost:7777'

    d= getPage(URL,method="POST",postdata=json.dumps(TESTDATA))
    d.addBoth(lambda x :print(json.loads(x)))

Python Twisted JSON RPC


Python class attributes vs meta classes vs class variables

Question: I have a mongo wrapper with hooks to a timer class, basically, every time a collection is updated or saved it spawns a timer which in turn executes a given function when it expires. My question is, what would be the pythonic way to specify those functions? My thought was to simply add them to the collection wrapper like this:
class TestCollection(Collection):
    __name__ = 'test_collection'
    __database__ = 'test'
    __primary_key__ = 'field_1'

    post_delete = 'call_this_func_with_getattr_after_delete'
    expire = 'also_call_this_with_getattr_when_timer_expires'

    field_1 = Key()
    field_2 = Key()
    field_3 = Key()

Then I can just add the logic on my timer class to run the specified function when expired and the same for my mongo wrapper. This could also be achieved in different ways (class Meta, mangled attribute names, etc...) but I just wanted to know the general consensus when doing something like this.
Answer: Don't store names you have to look up when you cab just store references to callables directly. Any function, method, or an instance of a class with a __call__ method, is an object just like anything else, and can be stored in your expired attribute.


how can I add/remove security groups on running ec2 instances which is not included in VPC?

Question: I want to change the security group on running ec2 instance which is not included in VPC. I read that this is possible only if my instance is running in VPC.
I tried to create a VPC and include my running instance in the active VPC by creating new network interfaces, and this is not possible as well.
Do you have any ideas how can I add/remove security groups on running ec2 instances?
Answer: After you launch an instance in EC2-Classic, you can't change its security groups. The same does not apply to VPC (Amazon EC2 Security Groups).
You could try creating AMI from current instance and then launch new instance with appropriate security groups.
Comment: The restriction on changing an instance's security groups only applies to EC2-Classic, not VPC, per the link you cited.
Comment: yes, but downtime is not allowed, thanks anyway
Answer: If downtime is not allowed, you could clone your server instance by making an image of it, then relaunching the image with the correct security group.
You could then create an ElasticLoadBalancer, place both the original and the clone behind the ELB, switch DNS (with very low TTL) from the priginal server to the ELB endpoint and the wait until the load is distributed across both.
You would then drop your original server out of the ELB, using connection draining, and reverse your DNS change so that it now pointed to the ip address of the new server.
You can then stop/terminate your old server, leaving you with the new one in the correct Security Group.
Of course, this solution won't work with a server running a writable database, but if you've got a writable DB on a single server instance in EC2 Classic, you've got other problems.


Proof of Approximate / Exact Bayesian Computation

Question: The ABC algorithm is given as

Draw $\theta \sim \pi(\theta)$
Simulate data $X \sim \pi(x | \theta)$
Accept $\theta$ if $\rho(X, D) < \varepsilon$

where $\pi(\theta)$ is the prior, $\pi(x | \theta)$ is the likelihood, $\rho(\cdot | \cdot)$ is some distance measure, $D$ is the observed data and $\varepsilon$ is the tolerance that represents a trade off between accuracy and computability.
Generally, in papers that I have seen on this, a proof is given where it states we actually sample from $\pi_{\varepsilon} = \pi(\theta | \rho(X, D) < \varepsilon)$ and then if $\varepsilon \to 0$, this converges to the true posterior $\pi(\theta | D)$.
If in Step 3, we had
3*. Accept $\theta$ if $X = D$
I was wondering if anyone knew how to prove that in this new algorithm, we sample from the true posterior? So there is no $\varepsilon \to 0$ argument?
Answer: This case is the original version of the algorithm, as in Rubin (1984) and Tavaré et al. (1997). Assuming that $$\mathbb{P}_\theta(X=D)>0$$ the values of $\theta$ that come out of the algorithm are distributed from a distribution with density proportional to
$$\pi(\theta) \times \mathbb{P}_\theta(X=D)$$
since the algorithm generates the pair $(\theta,\mathbb{I}_{X=D})$ with joint distribution
$$\pi(\theta) \times \mathbb{P}_\theta(X=D)^{\mathbb{I}_{X=D}} \times
\mathbb{P}_\theta(X\ne D)^{\mathbb{I}_{X\ne D}}$$
Conditioning on $\mathbb{I}_{X=D}=1$ leads to
$$\theta|\mathbb{I}_{X=D}=1 \sim \pi(\theta) \times \mathbb{P}_\theta(X=D)\Big/\int \pi(\theta) \times \mathbb{P}_\theta(X=D) \,\text{d}\theta$$
which is the posterior distribution.
On the side, I gave this very proof in class a few hours ago!
Comment: Yes, indeed!,  this is actually the most rudimentary form of acceptance-rejection: simulate $X$ from $f_1$ and accept if the realisation of $Y\sim f_2$ is equal to the realisation of $X$.
Comment: Thanks Xi'an! If this was extended outside of the Bayesian context, so for arbitrary functions $f_{1}(x)$ and $f_{2}(x)$, does this work to find samples from $f \propto f_{1}f_{2}$?


How to get Panels to pass a correct date to Views (D7, Views/Panels3)

Question: I'm trying to get a date from a node to a view pane. When someone is viewing the node, they should see a view on the right that is just a list of other nodes with the same day in the "event_date_time" field.
Here's screenshots of how everything is set up:
https://i.stack.imgur.com/1mrac.jpg
1) First is the pane layout. The left is the node content with event_date_time. The right top is a view panel that has no filters, so you can see all the nodes and dates. Below it is the same view panel with a contextual filter.
2) the current results. You can see the right bottom only matches the two nodes that have the exact same year, month, day, hour and minutes.
3) the contextual filter setup. It's set with granularity of days, but doesn't seem to make any difference what granularity is set. Even if it is changed to "year", it still only returns the one exact matching item. If I change one of the two matching nodes from 4:00 to 4:15, they no longer match.
4) I'm asking for the date field from the Panel context in Views
5) And it's set up in Panels to send
6) The contextual filter returns the correct 4 events that match the day, if I manually pass it a YYYY-MM-DD format.
I looked through the context and relationship options in both panels and views and can't see any other way to send or format this date. I feel like I must be missing something basic in my understanding of how this is supposed to work.
(I get the same results whether I create the contextual filter using "Content: Event Date Time" or "Date: Date (node)", and whether they are set to "date range" or "only this field".)
This is Drupal 7.22, Panels & Views both current dev.
Comment: Please try creating a custom pane in Panels, find the token (keyword) that Panels uses to represent the time argument, and write it out in the pane. That way, you'll get to the the exact value Panels passes on to Views. Might be a start. (It's not a very elegant debugging method, but it's often been helpful for me)
Comment: That's a good idea, so at least I know what I'm dealing with. The token is %node:field_event_date_time, and panels spits it out as "2013-07-31 16:00:00", which is what I would expect. Using that as a manual filter on the views preview interface gives the same (incorrect) results as the panes do.
Answer: After struggling for some time, I've still not found any way that works via context from Panels. Other people have said Panels values were determined by the "token" content display mode, but that isn't the case, at least with dates. Of course you can put the value in the URL and have Views grab it there, but for my use that wasn't appropriate. It also seems more fragile since it makes all content dependent on the path structure.
The best option I found was setting the "Argument Input" on the View to "Input on Pane Config". Then in the Panel Pane, use keyword/token substitution. I created a custom date & time formatter that is CCYY-MM-DD, called "date only", and set the Panel option to "%node:field-my-date-field:value:date_only". That grabs just the first value from a date range. For a field without range, "%node:field-my-date-field:date_only" works.


What does Amp of a function mean?

Question: I am reading a proof of the Brouwer fixed point theorem where the author writes the following

"$w$ is continuous on $K$ and does not vanish, and therefore $\mathrm{Amp}
 \; w$ is defined everywhere".

The notation $\mathrm{Amp}$ is never given a meaning in the text however. What does it denote?
source
Comment: It would help enormously if you named / linked to the source.
Comment: Are you sure it isn't a typo for map?
Comment: I found [this](http://projecteuclid.org/DPubS/Repository/1.0/Disseminate?view=body&id=pdffirstpage_1&handle=euclid.tmj/1178244835) by Googling parts of the quote. From the context, it appears to be a synonym for "argument".   Further Googling reveals that sometimes "amplitude" is (or was) used in this way, but that is news to me.
Comment: @JonasMeyer, he included the source, Tohoku 1956. He does the case of two dimensions in the complex plane and is clearly using argument.
Comment: Maybe Amplitude?!
Answer: It is a synonym for argument in the complex plane. Not one any of us knew.

What does Amp of a function mean?


Random Forest Algorithm Where to start?

Question: I need to solve a real time classification problem using random forest algorithm.Can anyone suggest any book,website where I should start looking it ?
Comment: here under random-forest tag:) at least to further specify the question as to why your problem is so peculiar that the general RF references are not helpful.
Comment: Have you read [Wikipedia](http://en.wikipedia.org/wiki/Random_forest) and [Breiman's original article](http://link.springer.com/article/10.1023%2FA%3A1010933404324)? We will happily try to help you with more specific points that remain unclear to you. Please consider editing your question to be more specific.
Comment: Sorry for the unspecific nature of question.This is my first one and I am just starting over a project which requires me to classify  a realtime data using random forests of which I have no idea.I have started reading wiki and the breiman article(thanks for that).Will try to be more specific from now.
Answer: The RF algorithm is quite fast once built, so for most cases you won't have any trouble running data through your model in real time.  Just:

Build your model with data at hand, adjusting parameters to get a good predictive model.
Then have a look at the feature importances to see if you can trim down the set of variables you need for a good model.
Once you've gotten the minimum set of features and the best model, set up a system to generate those features from your real time data as it comes in.
Pump the generated features into your model and return the result.

If you are looking for an online version of RF, where the model itself gets updated as new data comes in, that is a different technique.  You can start with Online Random Forests and Streaming Random Forests if you are going that route.

Random Forest Algorithm Where to start?


How can I turn off the caching for a Paragraph type?

Question: I'm rendering an array whose URL is dependent on a $_GET variable.
Where the variable is present and valid, I want to serve an iframe with the _GET variable as part of the URL. At the moment the cache is capturing the variable and preventing different value being passed to the page.
In my module I have this:
function MY_MODULE_preprocess_paragraph__iframe(&$variables) {
    $query = @$_GET['e'];
    /*...*/
}

But the value $query is staying the same until the caches are cleared.
Is there a better way of capturing the $_GET variable, or a way of preventing the value from being cached?
Comment: You can't turn off caching for a paragraph (at least not in prod). You need a cache context for the query parameter. If you have a lot of variations you could add a placeholder with a lazy builder callback. Then the cache context doesn't create a new cache entry for the paragraph and the parent entity whenever the GET variable changes.
Comment: Yes, you could meet the auto-placeholdering conditions, for example max-age=0. Or set #create_placeholder, see https://drupal.stackexchange.com/a/271124/47547
Comment: What's a lazy builder callback? Is it implemented in a preprocess hook?
Comment: Lazy answer: https://www.drupal.org/docs/drupal-apis/render-api/auto-placeholdering
Answer: Declare the dependency on the GET parameter with a cache context:
/mymodule/mymodule.module
function mymodule_preprocess_paragraph(&$variables) {

  $foo = \Drupal::request()->query->get('foo');
  $variables['content']['foo'] = [
    '#markup' => "foo = $foo",
    '#cache' => ['contexts' => ['url.query_args:foo']],
  ];
}

More info https://www.drupal.org/docs/drupal-apis/cache-api/cache-contexts
However, if you have more than a few possible values of foo this is a problem with cache performance, especially for nested entities like paragraphs. Each value creates a new cache entry for the paragraph and its parent entities.
To avoid this put a placeholder in the paragaph:
/mymodule/mymodule.module
function mymodule_preprocess_paragraph(&$variables) {

  $variables['content']['foo_placeholdered'] = [
    '#lazy_builder' => ['\Drupal\mymodule\MymoduleLazyBuilders::getFoo', []],
    '#create_placeholder' => TRUE,
  ];
}

and build the content which depends on the GET parameter in a lazy builder:
/mymodule/src/MymoduleLazyBuilders.php
<?php

namespace Drupal\mymodule;

use Drupal\Core\Render\Element\RenderCallbackInterface;

class MymoduleLazyBuilders implements RenderCallbackInterface {

  public static function getFoo() {
    $foo = \Drupal::request()->query->get('foo');
    return  [
      '#markup' => "foo = $foo",
      '#cache' => ['contexts' => ['url.query_args:foo']],
    ];
  }

}

For time consuming calculations such placeholders can be delivered through BigPipe. So if you enable this module you can also improve page speed.