Can I sync .vscode folder?

Question: I use  cpptools extension. When I open folder in vscode, I have to copy . Vscode folder to new folder. It's a hassle. I know how to sync general setting by github, but I'm not sure it syncs .vscode folder in project directory.
So, can I load . vscode when I opened a new folder?
Comment: `.vscode` is your workspace settings, it's recommended to commit into git repo with the project, and it shouldn't be shared across projects.
Answer: The <code>.vscode</code> folder is intended for workspace-specific things. If you have settings there that you need in multiple workspaces, I'd recommend moving them to your user settings.

Can I sync .vscode folder?


is there a direct way to solve a set of four equations with four symbols where there is a conjugate of one in SymPy?

Question: So I have four symbols for a set of four equations in SymPy. I also have the conjugate of one of the symbols. Solution is given as an empty list, while I expect values for the symbols.
'z', 'k', 'l', and 'm', where we also have the conjugate of k and we have the set of equations eq2, eq3, eq4, and eq5.
from sympy import Symbol, solve, I, conjugate

z = Symbol('z')
k = Symbol('k')
l = Symbol('l')
m = Symbol('m')

gamma = 0.01
Om = 8
kappa = 1e4
N = 1e2
w = 10

eq2 = -(w+ gamma)*(z-1) + I*Om*(k - conjugate(k))
eq3 = -((w+gamma)*0.5 + kappa*0.5)*k + I*Om*0.5*(m*z + (z+1)*0.5)
eq4 = -(w+ gamma)*l + (Om*z/ (2*I)) * (k - conjugate(k))
eq5 = -kappa*m + (N*Om/(2*I))*(k - conjugate(k))

solution = solve([eq2, eq3, eq4, eq5], [z, k, l, m], dict= True)

As a result we obtain an empty list,
print(solution)
[]

and this is the first time I work with SymPy, so couldn't not figure what exactly could be the issue here as the documentation was not so helpful. If there is a way to solve this directly with SymPy or if the equations should be adjusted?
Comment: The `m*z` term in `eq3` makes the equations nonlinear. Also `conjugate` is not algebraically linear.
Comment: Also, solve syntax error. We don't know what `w` is (neither an unknown, nor a constant). And we don't know what you mean by `+ + kappa*0.5` or `w+gamma+`
Comment: This system of equations does not have any solution which is why solve returns an empty list.
Comment: Please include the imports, symbol definitions, and constant definitions so we can run this code.
Comment: @chrslg you're right I did not notice that, I fixed it now
Comment: @jared I modified and included the complete code
Comment: You should split the equations and in particular `k` into real and imaginary parts but that requires specifying which of the parameters or unknowns is supposed to be real or not.
Answer: If SymPy is unable to find an explicit solution, it returns an empty list as you observed. Means that the eqn may require numerical methods or further analysis to find approximate solutions.
If you need to find numerical solution, you can use 'nsolve' provided by SymPy.
from sympy import Symbol, lambdify, I, conjugate
from scipy.optimize import fsolve

z = Symbol('z')
k = Symbol('k')
l = Symbol('l')
m = Symbol('m')

gamma = 0.01
Om = 8
kappa = 1e4
N = 1e2
w = 10

eq2 = -(w + gamma)*(z - 1) + I*Om*(k - conjugate(k))
eq3 = -((w + gamma)*0.5 + kappa*0.5)*k + I*Om*0.5*(m*z + (z + 1)*0.5)
eq4 = -(w + gamma)*l + (Om*z / (2*I)) * (k - conjugate(k))
eq5 = -kappa*m + (N*Om/(2*I))*(k - conjugate(k))

eq2_func = lambdify((z, k), eq2)
eq3_func = lambdify((z, k, m), eq3)
eq4_func = lambdify((z, k, l), eq4)
eq5_func = lambdify((k, m), eq5)

def equations(vars):
    z, k, l, m = vars
    eq1 = eq2_func(z, k)
    eq2 = eq3_func(z, k, m)
    eq3 = eq4_func(z, k, l)
    eq4 = eq5_func(k, m)
    return [eq1, eq2, eq3, eq4]

solution = fsolve(equations, [1, 1, 1, 1])
print(solution)

Comment: yes I did check this, the issue is that with nsolve now I get the following error: NameError: name 'Derivative' is not defined
Comment: try the updated code. The equations are converted into numerical functions using lambdify from SymPy. The equations function is defined to encapsulate the system of equations. Then, fsolve from SciPy's optimization module is used to find the numerical solution.
Comment: thank you @Aravind, my concern is that in this case the accuracy of the solution is dependent on the estimate provided for fsolve?
Comment: If there is more than one solution then which you find will depend on your initial guess. It will be accurate but different than another solution. e.g. guessing near 1 for `x**2 -1` will give solution 1 while guessing near -1 will give solution -1.


Substituir valores numericos de um vetor por outro valor em um data frame

Question: eu gostaria de substituir (replace) os valores de um determinado index (vetor) os quais encontram correspondencia em um dataframe (tbl) por outro valor determinado por mim.
Tentei o seguinte
# Criando um index aleatório
index_1 <- sample(mtcars$gear, 5)
index_2 <- sample(mtcars$hp, 5)
index_3 <- sample(mtcars$disp, 5)

# Criando um index aleatório - Todos esses valores devem ser substituído por um determinado número (escolhi 48 também aletóriamente)
index <- c(index_1, index_2, index_3)

# Tentei criar essa função
fun_return <- function(x){
  if(x==index){
    return(48)
  } else
    return(x)
}

# Os valores não são substituídos.
mt_replaced <- fun_return(mtcars)

Alguém tem alguma dica?
Answer: A seguinte função faz o que a pergunta descreve.
fun_replace <- function(x, vetor, novo = 48){
  res <- lapply(x, function(y){
    i <- y %in% vetor
    y[i] <- novo
    y
  })
  res <- do.call(cbind, res)
  if(!is.null(rownames(x))) rownames(res) <- rownames(x)
  as.data.frame(res)
}

mt_replaced <- fun_replace(mtcars, index)

Comment: muito obrigado, Rui! Não imaginei que seria tão complicado assim, não teria chegado a esse resultado! Super valeu, funcionou perfeitamente nos dados originais!
Answer: Com dplyr você pode fazer algo assim:
library(tidyverse)
#> Warning: package 'tibble' was built under R version 3.5.2

index_1 <- sample(mtcars$gear, 5)
index_2 <- sample(mtcars$hp, 5)
index_3 <- sample(mtcars$disp, 5)

index <- c(index_1, index_2, index_3)

mtcars %>%
  mutate_all(~ifelse(.x %in% index, 48, .x))
#>     mpg cyl  disp  hp  drat    wt  qsec vs am gear carb
#> 1  21.0   6 160.0 110  3.90 2.620 16.46  0  1   48   48
#> 2  21.0   6 160.0 110  3.90 2.875 17.02  0  1   48   48
#> 3  22.8  48 108.0  93  3.85 2.320 18.61  1  1   48    1
#> 4  21.4   6 258.0 110  3.08 3.215 19.44  1  0   48    1
#> 5  18.7   8 360.0 175  3.15 3.440 17.02  0  0   48    2
#> 6  18.1   6 225.0  48  2.76 3.460 20.22  1  0   48    1
#> 7  14.3   8 360.0 245  3.21 3.570 15.84  0  0   48   48
#> 8  24.4  48 146.7  62  3.69 3.190 20.00  1  0   48    2
#> 9  22.8  48 140.8  95  3.92 3.150 22.90  1  0   48    2
#> 10 19.2   6 167.6  48  3.92 3.440 18.30  1  0   48   48
#> 11 17.8   6 167.6  48  3.92 3.440 18.90  1  0   48   48
#> 12 16.4   8  48.0  48  3.07 4.070 17.40  0  0   48   48
#> 13 17.3   8  48.0  48  3.07 3.730 17.60  0  0   48   48
#> 14 15.2   8  48.0  48  3.07 3.780 18.00  0  0   48   48
#> 15 10.4   8 472.0 205  2.93 5.250 17.98  0  0   48   48
#> 16 10.4   8 460.0 215 48.00 5.424 17.82  0  0   48   48
#> 17 14.7   8 440.0 230  3.23 5.345 17.42  0  0   48   48
#> 18 32.4  48  78.7  66  4.08 2.200 19.47  1  1   48    1
#> 19 30.4  48  75.7  52  4.93 1.615 18.52  1  1   48    2
#> 20 33.9  48  71.1  48  4.22 1.835 19.90  1  1   48    1
#> 21 21.5  48 120.1  97  3.70 2.465 20.01  1  0   48    1
#> 22 15.5   8  48.0 150  2.76 3.520 16.87  0  0   48    2
#> 23 15.2   8 304.0 150  3.15 3.435 17.30  0  0   48    2
#> 24 13.3   8  48.0 245  3.73 3.840 15.41  0  0   48   48
#> 25 19.2   8  48.0 175  3.08 3.845 17.05  0  0   48    2
#> 26 27.3  48  79.0  66  4.08 1.935 18.90  1  1   48    1
#> 27 26.0  48 120.3  91  4.43 2.140 16.70  0  1    5    2
#> 28 30.4  48  95.1  48  3.77 1.513 16.90  1  1    5    2
#> 29 15.8   8 351.0 264  4.22 3.170 14.50  0  1    5   48
#> 30 19.7   6  48.0 175  3.62 2.770 15.50  0  1    5    6
#> 31 15.0   8 301.0 335  3.54 3.570 14.60  0  1    5    8
#> 32 21.4  48 121.0 109  4.11 2.780 18.60  1  1   48    2

Created on 2019-02-25 by the reprex package (v0.2.1)
Comment: Obrigado, Daniel. Solução simples e eficiente! Tentei umas 100 vezes fazer com o dplyr e não rolou! Abraços!


Singular value decomposition of a compact operator, using matrix notation

Question: Maybe it is a bit naive, but I ask myself the following.
Let $T \in K(X,Y)$ be a compact operator. For example, its singular value decomposition is written this way:
$$Kx=\sum_{n\in \mathbb{N}}{\sigma_n\langle x ,u_n\rangle v_n}$$
But is it forbidden to write it like as if I had matrices of infinite dimensions? Like that:
$$Kx = U\Sigma V^*x$$
Would that break something? Some calculations would come a bit easier and it could avoid with fighting indices.
Comment: If you properly introduce your notation, you're free to use it. But.. what these $U,V$ and $\Sigma$ wants to be? Can you give a specific example?
Answer: I presume you intend $X$, $Y$ to be Hilbert spaces.  I'm going to assume we just have one Hilbert space $X = Y$ for convenience.  This doesn't make much difference.
Also I think you probably meant to write $Kx=\sum_{n\in \mathbb{N}}{\sigma_n\langle x ,u_n\rangle v_n}$.
Infinite matrices:  I'm not sure if you know how one associates infinite matrices with bounded linear operators (given a choice of orthonormal basis).  I'm not able to discuss this at length, aside from saying that it does work; moreover, composition of operators corresponds to matrix multiplication,  Hilbert space adjoint of operators to conjugate transpose of matrices, etc.
If you accept this, I can explain how the SVD arises as an operator factorization, which can be translated into a matrix factorization.  Well, sort of explain. You need some background.
You can start with the polar decomposition of $K$,  $$K = U \sqrt{K^* K},$$ where $U$ is a partial isometetry from the closure of the range of $\sqrt{K^* K}$ to the closure of the range of $K$.  This exists for all bounded operators on Hilbert space, no special assumptions needed.
Since $\sqrt{K^* K}$ is a non-negative self-adjoint compact operator on $X$,  you use the spectral theory of compact operators to obtain an orthnormal basis $u_i \in X$ such that $\sqrt{K^* K} x = \sum_i \sigma_i \langle x, u_i\rangle u_i$, with $\sigma_i$ a decreasing sequence tending to zero.   So put $v_i = U u_i$, and you have the singular value decomposition as you first wrote it.
So far we don't have any matrices. We have an operator factorization of $K$, $K = U \sqrt{K^* K}$, which, together with the spectral decomposition of $\sqrt{K^* K}$ contains exactly the same information as the SVD, written in your first form, with $u_i$, $\sigma_i$ and $v_i$.
The SVD in your matrix form results from writing matrices corresponding to the operator factorization.
The matrix of   $\sqrt{K^* K}$ with respect to the orthonormal basis $(u_i)$ is the diagonal matrix  $\Sigma$ with diagonal entries $\sigma_i$.
If you have some other favorite orthonormal basis of $X$,  say $(e_i)$,   the matrix of $\sqrt{K^* K}$ with respect to $(e_i)$ is going to be $W^* \Sigma W$ with $W$ the unitary change of basis matrix relating the ON bases $(e_i)$ and $(u)_i$.  Abusing notation I'll still write $U$ for the matrix of the partial isometry $U$ with respect to the ON basis  $(e_i)$.  Now the matrix of $K$ with respect $(e_i)$ is going to be
$$
(U W^*) \  \Sigma \ W,
$$
which is the SVD in the matrix form that you wanted.


Javascript/InDesign: Check if Object has certain property

Question: I'm using Javascript for InDesign scripting.
I have an image object and want to know it's bounds (the one that the user sees) -
bounds = (geometricBounds in image.parent)? image.parent.geometricBounds: image.geometricBounds;

returns ReferenceError - geometricBounds is undefined . This error comes when the parent to the image is an Oval object (So, I know geometricBounds property is there for an Oval object).
The problem is in (geometricBounds in image.parent) because when I alerted this statement, I got the same error. I'm surely missing something - because if this is not a property then I should simply get a false.
Any one knows why this is happening?
Answer: The in operator checks for the property name as a string (in your case it is looking for a variable called geometricBounds - which which might contain the property name string - which is obviously not declared anywhere):
bounds = ('geometricBounds' in image.parent)? image.parent.geometricBounds: image.geometricBounds;

Comment: Uhh... I'm sorry, that was embarrassing. I'm new to JS - years of Java and C++, and now i feel dumb while doing JS!


How can we include hourly traffic series data in the rows of train data set for training?

Question: I have a classification problem where I am planning to use hourly traffic data for a day. Is there any way to compress it? instead of creating 24 predictors which account for hourly traffic?
Comment: usual dimensionality reduction methods should go. it also depends on what is your task (what do you want to classify, how do you think it relates to traffic). you may also decide to sum all those columns together and only keep the sums column.
Comment: I have hourly traffic data of which I can use hours value. However, I am hoping to find the correlations between in flow and out flow. For example, a heavy traffic in two hours followed by none might indicate lots of population moving out of the region. I am hoping to find such patterns, can CNNs help to capture that info? As they are used for sensor data
Answer: your problem is a panel data .since you have both time and cross-section.
and yes panel data methods can help you have one predictor which considers time itself.
(it is kind of like considering a 23dummy variables for each time, then applying them in one equation)
you can read about (panel data or longitudinal data)and choose which methods can help you.
good luck
Comment: I don't think this is really the same as usual panel data. each hour data is meaningful in respect to the hour it has been recorded in
Comment: i still think the data is panel. but maybe for this case it should be seprated to clusters according to times for example highly traffic hours or normal traffic or .. something like that. i also suggest to search for what uber use to analyse his data.lots of information about it is available they can inspiring  @carlo
Comment: No, it is not a panel. I have hourly traffic data of which I can use only hours value. However, I am hoping to find the correlations between in flow and out flow. For example, a heavy traffic in two hours followed by none might indicate lots of population moving out of the region. I am hoping to find such patterns.


How does F5 GTM work?

Question: The F5 GTMs automatically reroute users to one of the available datacenters should one fail. How does it do this? For instance the F5 could act as a DNS server and simply serve a different IP to clients but how would it get around caching?
I'm interested in this because I'd like to see whether obtaining this sort of availability is possible with open source tools that could be coded to do similar stuff
Answer: The F5 GTM does its load balanancing ENTIRELY with DNS.  GTM doesn't do any physical routing of traffic.
It has a massive set of health checks it can perform on hosts, on services, or on groups of services that are tied together as "distributed applications" and when any of these fail, it uses DNS to reroute to the right place
In terms of getting around caching, you set your TTLs very low (10-30s) so that you minimize effects of folks hanging onto IPs.
Have a look at this article for a discussion of open source alternatives to F5 BigIP:
https://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/52/aft/1178927/showtab/groupforums/Default.aspx
I've been happy with our F5 GTMs and LTMs so far, but they're indeed expensive.
Comment: Just to note, there seem to be two different strategies here using open source solutions.  One is using GeoIP info ([with bind for instance](http://phix.me/geodns/)) for the user's source address, and the other is relying on BGP information ([Super Sparrow](http://horms.net/projects/supersparrow/) for instance).
Answer: its most likely due to a heartbeat between the devices.  we use radware for our load balancing, but were not doing it across the internet cloud, we have a private connection between our radware devices.
if you have a test enviroment, try inlab's balance product.  its a gpl licensed.
hope this helps!
:)
Comment: Thanks. I run hearbeat2 across mine. I was wondering whether there was extra "magic" concerning the gtm. For instance local balancing in global data centres and detecting failure. It could be that they don't really do that but they seem to.

How does F5 GTM work?


Subdomínio criado por usuário

Question: Estou desenvolvendo um sistema de área de usuários, ao registrar-se, este usuário terá a sua área com seu subdomínio, escolhido por ele.
Dúvida:
Como fazer esse cadastro de subdomínio, sendo que ele não terá acesso ao cPanel.
Eu já vi isso em site gringo. Como posso fazer isto?
Pode ser pelo Apache, ou pelo próprio PHP.
Comment: Use a Api do Cpanel ou pesquise por WildCard -

[Segue a API](https://github.com/CpanelInc/xmlapi-php)
Answer: Você pode tentar olhar o valor de $_SERVER['HTTP_HOST']; essa variável contém a parte logo depois do http://…, até a primeira barra (vulgarmente, o "domínio" do site). Aí você só precisaria fazer um apontamento DNS e configurar o Apache pra fazer qualquercoisa.meudominio.com sempre cair no mesmo site (e portanto no seu script HTTP).

Subdomínio criado por usuário


How to reach 3 VM under machine with 1 public IP

Question: I have one machine (machine A) with 3 VM on Hyper-V (Windows server Datacenter 2019).
Machine A is reached by one public IP.
How can the virtual machines be reached directly from an external network?
Answer: You can not.
YOu need a reverse proxy through it. What you ask is identical to having 3 cars with the same number plate.
A reverse proxy will be able to distribute incoming traffic by some rules.
But it is not "direct", so it is a no as answer.
Comment: "NAT is equivalent to direct nowadays" - no, it is not. It is a step that must be documented as an interim network device with configuration and from experience with large companies I can tell you it is all BUT direct - takes separate paperwork to a seprarate team.
Comment: OP doesn't need a reverse proxy per se, could also use NAT on the host to the VM internal network, for example. But the question was for "directly", so yes, you are correct with "no".
Comment: NAT is equivalent to direct nowadays. This is done on the network ip layer and it is very common to use NAT for some kind of direct access from external networks. Thought in that case you need PAT in addition wich only works with specific ip protocols which are port based and You cannot use same external incoming port for the services on same local port.
Comment: If NAPT were equivalent to direct routing, NAT traversal techniques to work around broken end-to-end connectivity would not be necessary. Not the same.
Answer: IPv6.
Perhaps a significant project if you have not started. However, forget about IP address scarcity when you can allocate /64s.
Edit: regarding concerns v4 only users cannot access, mitigate those in your network design.

v6 WAN and Internet access
mobile data plans on v6 providers
6in4 tunnel
remote access VPN with v4 endpoint

To run a public service, or if client networks are behind on their v6 deployment and will not use transition methods, you would still want v4. Which does not solve your not enough IP address problem.
But if you control the network, you can access over IPv6 anywhere in the world.
Comment: Maybe a choice, but i am thinking the user will not have the ability to set up an ip4t6 or ip6t4 translator
Comment: Regarding v4 only users, see edit.
Comment: Next days i 'll try your solution. I can access to all machines


Best method to analyse survey results with multiple choice of answers

Question: I have designed a survey with multiple choice answers. Each question contains same set of answers

Strongly agree
Agree
Disagree
Strongly disagree
Don't know

There are 25 questions and questions are grouped into 5 areas. I need to analyze each area.
What is the best approach to analyze the survey results which I collected from a number of people? Is it a good method to calculate the mean response from each area as I need area-wise analysis of questions? Can any one suggest a best option?
Comment: Such a response format is typically called a Likert item (or a Likert scale but that's somewhat confusing because the whole point of Rensis Likert's work was to combine several such items to form a single scale). If you use that term to search for more info, you will see that there is a lot of material on the web and several questions on this site addressing various aspects of this type of data.
Comment: Beyond that, you will need to give more details on your data and what you want to find out if you want specific links or advice.
Comment: I just want to test the awareness of people about a topic. The people selected by me is a closed group. I need to analyze the results collected as part of my survey and wish to submit a report baesd on it.
Comment: Obviously people are willing to help, you already got several tips that should have helped you a lot. If you want more, don't offer a bounty but ask a specific question! "Analyze the data" or "Test the awareness" is so broad as to be meaningless. What's the problem with generating some graphs or stating X% of people strongly agree with X or Y? What else do you want to learn? Without significant scale design effort, sophisticated study design (longitudinal or comparing several conditions) or perhaps some other data, such questionnaires don't allow for much more than that.
Answer: I upvoted simply because the downvote was uncalled-for.
To answer the question:

Nobody can tell you how to analyze anything unless they have some idea of why you want to analyze it.  That said:
If you just want to display the data, make a bunch of bar charts.  People can see who put what, and how frequently.  Done.
Do you want to say how two variables are related?  Maybe that agreeing with one question predicts disagreeing with another?  That is regression or one of its flavors.  Do some basic reading.  The simple stuff isn't really that hard, and excel can do a lot of really basic stuff.
Do you want to group people into clusters?  People who think alike?  That is a lot harder.  PCA, kmeans, etc.  Let us know what your goals are and we'll try to point you in the right direction.
Comment: Do you have some measure of the effectiveness of the "business continuity management system"?  Can you call it $y$ and say that $y$ has some distribution of values?  If not, you probably don't want statistics, but rather some other sort of qualitative logical argumentation, maybe bolstered by some bar graphs.
Comment: @Prasanth: ACD wants you to expand on the word "analyze". What do you want to see? Give examples of the kinds of statements you'd like to be able to make. How do the categories enter into what you want to do? Techniques can be simple ("Twice as many people strongly agree with X as with Y") or complicated, and we can't really help you unless you specify what you mean by "analyze". And to be honest, you won't succeed if you don't understand what you mean by "analyse" clearly enough to explain it to us.
Comment: I have different categories of questions as mentioned. I want to analyse each question category separately.
Comment: Then just make a bunch of bar graphs
Comment: @Wayne: I want to measure the degree of understanding on a particular area based on the questions of that area. So I added questions based on each area. The response collected from a group of people should show their knowledge and understanding on that area. Likewise I wish to make an analysis on each area. I don't wish to plot graphs for each question. I want the results based on each area.
Comment: I am getting more and more confused. You are trying to assess each individual's knowledge but you asked questions that can be answered with "strongly agree" or "strongly disagree"? Care to edit your question to give an example? Also, how many people did you ask?
Comment: For example, I have an area with name "Threats and Vulnerabilities". There are 5 questions based on that. I need to measure the knowledge of people on "Threats and Vulnerabilities" in the organization, they are working. There are nearly 40-50 people in the sample size selected.
Comment: I got that now but how can knowledge be assessed through statements to which one can express various levels of agreement? I am wondering what a *question* looks like. Likert-type items usually measure attitude or personality, which can intuitively have degrees, whereas knowledge is typically assessed by asking if a statement is true or false or possibly by picking the right answer from several possibilities.
Comment: Also, are you interested in the organization as whole or in assigning a score to each individual (e.g. for personnel selection purposes)?
Comment: I want to take the organisation as a whole and need to study the effectiveness of business continuity management system in the organisation. So I divided the questions into different areas in business continuity management.
Comment: I don't have a measure of effectiveness.
Answer: Following up on points 3 and 4 by ACD:
I would suggest that you take a look at Latent Class Analysis which handles properly your categorial data, assuming that your grouping of questions represents different latent constructs. In case you are interested in connections (e.g. correlated error terms) or even causal links between your "groups", there are also Latent Class Models which are kind of a mix between Structural Equation Models and Latent Class Analysis.
Comment: This is not entirely unreasonable given the vagueness of the original question but it seems not to address the OP's concerns since his comments suggest that he is not interested in clustering or comparing individual participants at all. Also, the sample size is very small (40-50 people all coming from the organization that should be evaluated). How useful is LCA in this context?
Comment: Indeed, the sample size could lead to problems with identifiability and robustness of the fit measures due to sparse data in the contingency tables.
Comment: I think that you would need something like an aggregation or clustering/comparing if you are actually interested in the outcome of an organiziational process but you have only collected data about the knowledge of individuals.
Comment: If so, what is the best method?
Comment: Possibly, the best starting point to make sense of your data is an analysis using measures of tendency and dispersion (e.g. median and standard deviation) and graphs to describe the distribution of your results as ACD already recommended. Your assumption is that the aggregated answers of the individuals you questioned are representative for the level of (everyday) knowledge in your organization. In case you are not content with mere description and you are looking for certain underlying but hidden types or structures in your data, you should try cluster analysis or latent class analysis.


Web Scraping yelp review rating Python

Question: <code>rating=[]

for i in range(0,10):

    url = "https://www.yelp.com/biz/snow-show-flushing?osq=ice%20cream%20shop&start="+str(10*i)

    ourUrl = urllib.request.urlopen(url)

    soup = BeautifulSoup(ourUrl,'html.parser')
    for r in soup.find_all('span',{'class':"display--inline__373c0__1gaV4 border-color--default__373c0__1yxBb"})[1:]:
        per_rating = r.div.get('aria-label')
        rating.append(per_rating)
</code>
Try to get ratings for each page. Should have only 58 ratings in total, but it includes the rating from the "you might also consider".
How to fix it.
Comment: My guess is that that portion of the page is being populated with JavaScript, which cannot be accessed with urllib's urlopen function. Have you considered trying a package like Selenium to obtain the HTML, then parse it with BeautifulSoup (or equivalent)? This link may be helpful: https://stackoverflow.com/questions/8049520/web-scraping-javascript-page-with-python
Answer: One possible solution would be to retrieve the total number of Reviews from yelp using BeautifulSoup. You can then trim your "rating"-list by the number of reviews.
<code># find the total number of reviews:
regex_count = re.compile('.*css-foyide.*')
Review_count = soup.find_all("p", {"class": regex_count})
Review_count = Review_count[0].text
Review_count = int(Review_count.split()[0]) # total number of reviews
</code>

Web Scraping yelp review rating Python


Need to convert key=value text list to JSON stream in dot net core

Question: I am receiving the Key=Value pairs list as a MemoryStream :
key1=value1
key2=value2
:
keyN=valueN

I'm using AddJsonStream() to bind the configuration in the dotnet core.
How can I convert this type of MemoryStream key/value data to JSONStream like below?
{
   "key1":"value1",
   "key2":"value2",
   :
   "keyN":"valueN",
}

so it can bind as a configuration in dot net. Thanks.
Answer: I've used following way:

// Create a dictionary to load content of configuration
Dictionary<string, string> _werFileContent = new Dictionary<string, string>();

using (StreamReader sr = new StreamReader(result.Result.Content))
{
     string _line;
     while ((_line = sr.ReadLine()) != null)
     {
                string[] keyvalue = _line.Split('=');
                if (keyvalue.Length == 2)
                {
                    _werFileContent.Add(keyvalue[0], keyvalue[1]);
                }
     }
}

// Converting Key=Value extracted data to string
string json = JsonConvert.SerializeObject(_werFileContent);

// Converting to Stream
System.IO.MemoryStream ms = new System.IO.MemoryStream(System.Text.Encoding.UTF8.GetBytes( json));


ASP .net 6 download file by httpclient - problem with stream

Question: I'm creating blazor server app. I use external file storage with RestAPI.
I want to create download button to get file from storage. This may seem easy, but not necessarily.
From file storage I download HttpContent like that
<code>        var request = new HttpRequestMessage(HttpMethod.Get, _url);
        request.Headers.Add("auth-token", token);
        request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/octet-stream"));
        HttpResponseMessage response = await _Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
        response.EnsureSuccessStatusCode();
        var content = response.Content;
</code>
next I act like this tutorial https://learn.microsoft.com/en-us/aspnet/core/blazor/file-downloads?view=aspnetcore-6.0
<code>        var fileStream = content.ReadAsStream();
        using (var streamRef = new DotNetStreamReference(fileStream))
        {
            await JS.InvokeVoidAsync("downloadFileFromStream", "file.txt", streamRef);
        }
</code>
For small files everything work great. But if I try to download large file (100mb), algoritm firstable download file to memory(RAM) of server and later save on local disk of client.
In ideal world I dream that when I click button download, file from external storage will download after delay (with progressbar) like physical file (no stream buffer) form http server e.g. https://www.example.com/file.txt. Of course by my BlazorServer Application with authorization and authentication, and whole neccesery services.
Comment: "For small files everything work great. But if I try to download large file (100mb), algoritm firstable download file to memory and later save on local disk."   Can you clarify the "algoritm firstable download file to memory and later save on local disk" part?  I'm afraid that is utter nonsense when it comes to  my reading ability.
Comment: First this file from storage service is stream to RAM my server, after stream complete file is able to download by user (begin download process). I need sth similar to multipart upload file, of course to download. User(Client) should get file like "directly" from my file storage service but by my blazor server app.
Comment: As for the idea @PanagiotisKanavos, maybe use proxy? But I don't know how you should authorise with your file storage (API) in this case.
Comment: @mwrochna is this a different server from your own web server? In that case create a controller action that retrieves the file as a stream and returns that stream to your UI. You can still use ``. For large files you'll have to store the file to disk first instead of storing it in a 100MB buffer.
Comment: You'd need JS only if you need to talk to the third server from the *browser*. In that case it would be simpler to do everything in Javascript: use `fetch` to retrieve the blob, construct the `a` element in memory with a blob URL and call click on it.
Comment: You don't need all this code then. `_Http` connects to your server so there's no need for extra tokens and headers. A simple `var stream=await _Http.GetStreamAsync(url);` should be enough. You don't need even that though if you only need to retrieve data from a URL without processing. You can create an `` link with the `download` attribute. That's what the Javascript code you use does. Instead of writing any code, create an `` link.
Comment: Storage file server require bearer token and auth data. Those data are common for whole app, otherwise user has been authorised to download this file.
Comment: *moreover (instead otherwise) I add, I don't want to show my client my internal url (security)
Answer: I have solution.

Create Service to service File Storage API
Create controller to avoid cros-origin error
Use microsoft tutorial to create download button https://learn.microsoft.com/en-us/aspnet/core/blazor/file-downloads?view=aspnetcore-6.0


Find the number of elements in $\{0,1\}^n$ with no more than three $1$'s or three $0$'s in a row

Question: I'm trying to find a general formula for the number of elements $s_n$ in $\{0,1\}^n$ with no more than three $1$'s or three $0$'s in a row, where $n\geq1$.
I calculated $s_n$ for small values of $n$ but could not really come up with a formula to prove by induction.
I also approached the problem combinatorially by considering two groups, one of three $0$'s and one of three $1$'s, and arranging them together with other arbitrary elements totalling $n$ elements in all, but since we want to find the number of elements with no more than three $0$'s or $1$'s in a row, and not exactly three, this approach does not work either
Comment: To improve your posting, please show your calculations of $s_n$ for small values of $n$.  Also, if you attempted to write recursion relations or made another attempt, you should add that attempt to your question.
Comment: There is much information missing from your posting.  Did you read the article that I provided the link to, at the end of my last comment?  If not, please read that first.  If so, do you have specific questions on what information that article is suggesting that you provide?
Comment: My first try would be Inclusion-Exclusion.  See [this article](https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle) for an
introduction to Inclusion-Exclusion.
Then, see [this answer](https://math.stackexchange.com/questions/4427605/inclusion-exclusion-principle-what-is-1n1/4427645#4427645) for an explanation of and justification for the Inclusion-Exclusion *formula*.  Just focusing on $4$ or more consecutive ones, I would let $S_k ~: ~k \in \{1,2,\cdots, n-3\}$ represent the subset which contains a violation, starting at position $k$.  ...see next comment
Comment: Re Inclusion-Exclusion, you would have the added complication of dealing with sequences where $4$ or more consecutive $1$'s **and** $4$ or more consecutive $0$'s **both** occur. ...see next comment
Comment: I am ignorant of generating functions.  Also, I may (very plausibly) be overlooking some elegant approach.  Since I regard the *direct* approach as too ugly, the only other try that I can think of is recursion.  Again, temporarily focusing just on the $4$ consecutive $1$'s, you would need to carry separate variables, as a function of $n$, the length of the sequence, where you have a satisfying sequence, that is ending in $r$ $1$'s, where $~r \in \{0,1,2,3\}$.  That is, you would carry the variables $x_{(n,0)}, x_{(n,1)}, x_{(n,2)},$ and $x_{(n,3)}.$
Comment: Now that I consider it, it looks like recursion is not that bad, since you could simultaneously handle the consecutive $0$'s and consecutive $1$'s.  Re the trail end of my last comment, you simply take the $x_{(n,0)}$ variable, and divide it into $3$ categories, replacing $1$ variable with $3$, so that you are carrying $6$ variables, instead of $4$.  The idea is that $x_{(n,0)}$ must be distinguished by whether the satisfying sequence ends in either $1,2,$ or $3$ $0$'s.
Comment: One difficulty with recursion is that converting the result into a closed form expression may be problematic.  I know that a closed form expression must exist, because I know that the problem is solvable by Inclusion-Exclusion.  However, actually conjuring a closed form expression, based on a table of values generated by recursion, might easily be very difficult.
Comment: [OEIS A135491](https://oeis.org/A135491).
Comment: @user I think I understand the gist of what you're saying...could you formulate an answer so that I can accept it?
Comment: No, I am not allowed to.  Personally, I don't feel that it is reasonable to expect the OP (i.e. original poster) to show work on a problem this difficult.  However, the prevailing view at MathSE prohibits my providing a clear answer, until you edit your posting to deal with what other MathSE reviewers regard as defects.  See [this article](https://math.meta.stackexchange.com/questions/33190/how-to-avoid-downvotes-for-beginners-questions/33236#33236)
 for details.
Comment: @user What defects need to be addressed?
Comment: Also, your posting as is, could do with some line breaks, and paragraph breaks.  Line breaks are done via ``.  See [this posted question](https://math.stackexchange.com/questions/3899861/what-does-left-int-0x-ft-dt-right-equal) for a model to follow re posting a question at MathSE.
Comment: @user Haha I just noticed that you wrote that answer
Comment: Did you also happen to notice how MathSE reacted to it?  Also, what about the underlying article that I was merely summarizing, that I had no hand in writing?  Did you happen to notice how MathSE reacted to the underlying article?
Answer: A step-by-step sketch of an approach for a version of the problem (where you're avoiding three consecutive zeros or ones, not four - I misread the problem).  This can be entirely generalized to the case under consideration.   (Details are needed to be filled in):

Let $s_n$ be the number of elements in $\{0,1\}^n$ with no triples of $0$'s or $1$'s.

Let $s_n^0$ be the number of elements in $\{0,1\}^n$ starting with $0$ and no triples of $0$'s or $1$'s.

Similarly, define $s_n^1$ to be the number of elements in $\{0,1\}^n$ starting with $1$ and no triples of $0$'s or $1$'s.

Observe that $s_n=s_n^0+s_n^1$.

By exchanging $0$ and $1$, $s_n^0=s_n^1$.

Observe that there is a recurrence $s_n^0=s_{n-1}^1+s_{n-2}^1$ since a sequence that starts with $0$ has either a $1$ in the second or third position (but no more).

Observe that $s_1^1=1$ and $s_2^1=2$.

Since $s_n^0=s_n^1$, we have that $s_n^1=s_{n-1}^1+s_{n-2}^1$.

Multiplying this through by $2$, we have that $2s_n^1=2s_{n-1}^1+2s_{n-2}^1$ or that $s_n=s_{n-1}+s_{n-2}$.

Moreover, $s_1=s_1^0+s_1^0=2$ and $s_2=s_2^0+s_2^1=4$.

This is a Fibonacci sequence with different starting values, so its characteristic polynomial is $t^2-t-1$, whose roots are $\frac{1\pm\sqrt{5}}{2}$.

The general form of such a second order recurrence relation is
$$
s_n=A\left(\frac{1+\sqrt{5}}{2}\right)^n+B\left(\frac{1-\sqrt{5}}{2}\right)^n.
$$

Plugging in the known values for $s_1$ and $s_2$, it follows that
\begin{align*}
2&=A\left(\frac{1+\sqrt{5}}{2}\right)+B\left(\frac{1-\sqrt{5}}{2}\right)\\
4&=A\left(\frac{3+\sqrt{5}}{2}\right)+B\left(\frac{3-\sqrt{5}}{2}\right).
\end{align*}

Now, you can solve for $A$ and $B$ using linear algebra to get an explicit form for the recurrence.

 Subtracting the first equation from the second yields $2=A+B$.  Further solving gives that $A=1+\frac{1}{\sqrt{5}}$ and $B=1-\frac{1}{\sqrt{5}}$.  You can check that this works for the first few values of $n$ and prove it in general using the recurrence relation.
Comment: While an interesting approach, it appears inconsistent with user955791's solution. It matches for small values of $n$, but starting around $n=6$, the two sequences return different values. Your method return $s_6=42, s_7=68$ while the other approach yields $s_6=48, s_7=88$ (I did not check other values). I am not sure which approach has an error.
Comment: My answer is for avoiding three consecutive $0$'s or $1$'s, not four.  I misread the problem.  I put a disclaimer at the top of the answer.
Comment: No, I added the disclaimer in response you your message!  Thank you!
Comment: Got it, I clearly needed to read more closely.
Answer: This question can be solved in two ways such as using analytical combinatorics techniques and using linear algebra. Here, i will use analytical combinatorics techniques.My method is called Goulden-Jackson Cluster Method.
To be able to understand my answer , you must firstly read the article in given link.
If more than three consecutive ones and zeros are prohibited , it is understood that we cannot have $0000,1111,00000,11111,000000,111111,...$.In other words , we cannot have four consecutive ones or zeros in any string , because all of the banned words contain four consecutive zeros or ones.
Then our bad words are $\{0000,1111\}$ in our string.
Now , it is clear that our alphabet is $V=\{0,1\}$
According to the article in page $7$ , the generating function for the string is $$A(x)=\frac{1}{1-dx-weight(C)}$$ where $d=|V|=2$ and $\text{weight(C)=weight(C[0000])+weight(C[1111])}$
You can find how to calculate $\text{weight(C[0000]) and weight(C[1111])}$ in given article , so i am not getting in it.

$\text{weight(C[0000])}=-x^4-(x+x^2+x^3)\text{weight(C[0000])}$

$\text{weight(C[1111])}=-x^4-(x+x^2+x^3)\text{weight(C[1111])}$

$$\text{weight(C)}=\text{weight(C[0000])}+\text{weight(C[1111])}=\frac{-2x^4}{1+x+x^2+x^3}$$

Then , $$A(x)=\frac{1}{1-dx-weight(C)}=\frac{1}{1-2x+(\frac{2x^4}{1+x+x^2+x^3})}$$
$$A(x)=\frac{1+x+x^2+x^3}{1-x-x^2-x^3}$$
$$=1+2x+4x^2+8x^3+14x^4+26x^5+48x^6+...$$
That expansion means that the number of strings do not contain more than three ones and zeros consiting of $\{0,1\}$ and length $5$ is $26$.Here is the full expansion
To convert this generating function into closed form , we use enumerative combinatorics technique such that

If $$A(x)=\sum_{n=0}^{\infty}a_nx^n=\frac{P(x)}{Q(x)}$$ then $$Q(x)=1+\alpha_1x+\alpha_2x^2+\alpha_3x^3+\alpha_4x^4+...+\alpha_qx^q$$ and $$a_{q+n}+\alpha_1a_{q+n-1}+\alpha_2a_{q+n-2}+\alpha_3a_{q+n-3}+...\alpha_qa_{q}=0,n \geq0$$

As a result :
$$a_n=a_{n-1}+a_{n-2}+a_{n-3}, n \geq3 ,a_1=2,a_2=4,a_3=8$$
Comment: @SlipEternal i have checked it with R language , there is not any mistake in my answer
Comment: While an interesting approach, it appears inconsistent with Michael Burr's solution. It matches for small values of $n$, but starting around $n=6$, the two sequences return different values. His method return $s_6=42, s_7=68$ while your approach yields $s_6=48, s_7=88$ (I did not check other values). I am not sure which approach has an error.
Comment: I misread the problem (and added a disclaimer at the beginning of my answer).  My computation was for at most three 0's or 1's, not four - I misread the problem
Answer: While both user955791's and Michael Burr's solutions are fairly straightforward, here is another.
Consider the Regex for a valid string:
$$1\{0,3\}(01|011|001|0111|0011|0001|00111|00011|000111)^*0\{0,3\}$$
We have sixteen possible combinations of beginnings and endings for our strings with the middle being able to be determined based on solutions to a Diophantine equation.
Let $w_1,\ldots,w_9$ represent the number of $(01),\ldots,(000111)$ respectively in the final string. Then to create any valid string, we can find a multiset where the $w_i$'s are the multiplicity numbers. For beginnings and endings, the total number of digits in the beginnings and endings allows us to group them together.
Let's define some useful notation. For a given choice of $w_i$'s, define $w = \vec{w} = (w_1,\ldots, w_9)$. Then, define the following useful sums and products:
$$D(w) = 2w_1+3(w_2+w_3)+4(w_4+w_5+w_6)+5(w_7+w_8)+6w_9 \\ S(w) = \sum_{i=1}^9 w_i \\ P(w) = \prod_{i=1}^9 (w_i)! \\ f(k) = \begin{cases}1, & k\in \{0,6\} \\ 2, & k\in \{1,5\} \\ 3, & k\in \{2,4\} \\ 4, & k = 3\end{cases}$$
Now, we can write an expression for the total number of valid strings:
$$\sum_{k \in \{0,1,2,3,4,5,6\} \\ \forall i, w_i \in \mathbb{Z}_{\ge 0} \\ D(w) = n-k}f(k)\dfrac{[S(w)]!}{P(w)}$$
Both user955791's and Michael Burr's solutions are probably easier to calculate. But, this should generate the exact same results!
Update:
This is actually easier to calculate than I expected. I wrote a VBA procedure to output to Excel. It calculated the first twenty numbers in a few milliseconds.


C++ lambda expression (anonymous function)

Question:
Possible Duplicate:
What is a lambda expression in C++11?

I found this expression in C++ (one of the most exciting features of C++11):
int i = ([](int j) { return 5 + j; })(6);

Why I get the 11? Please explain this expression.
Answer: [](int j) { return 5 + j; } is a lambda that takes an int as an argument and calls it j.  It adds 5 to this argument and returns it.  The (6) after the expression invokes the lambda immediately, so you're adding 6 and 5 together.
It's roughly equivalent to this code:
int fn(int j) {
    return 5 + j;
}

int i = fn(6);

Except, of course, that it does not create a named function.  A smart compiler will probably inline the lambda and do constant folding, resulting in a simple reduction to int i = 11;.
Comment: Unlike C#, C++ does not capture variables in the outer scope automatically.  The `[]` contains a list of variables in the outer scope that you want to capture into the inner scope.  You can capture things by value or by reference.  (When capturing by reference, things are captured using C++ references; the lifetime of function locals is not extended in C++ like it is in C#.)  In this particular case, the lambda is not capturing anything from the outer scope.
Comment: As a dabbler in C++, and current C# developer, what does the `[]` denote?  Whenever I see brackets, I think array.


Add user to Directory role in azure active directory via Microsoft graph SDK\API

Question: I've created an API project in dotNet to get some data from microsoft.graph using SDK.
I managed to get all the users and all groups and see the role of each user (like Billing administrator / Application administrator, etc.), but I can not find a way to update or change the role to a user or group through the SDK / API.
I guess right now this option is not supported through API / SDK. Am I right?
Answer: You can do a HTTP request to the directory role using role-id or role-templateid for adding the user:
POST /directoryRoles/{role-id}/members/$ref
POST /directoryRoles/roleTemplateId={roleTemplateId}/members/$ref

Example :
POST https://graph.microsoft.com/v1.0/directoryRoles/fe8f10bf-c9c2-47eb-95cb-c26cc85f1830/members/$ref
or
POST https://graph.microsoft.com/v1.0/directoryRoles/roleTemplateId=88d8e3e3-8f55-4a1e-953a-9b9898b8876b/members/$ref
Content-type: application/json

{
  "@odata.id": "https://graph.microsoft.com/v1.0/directoryObjects/UserObjectID"
}

You can use C# to add user to directory role :
GraphServiceClient graphClient = new GraphServiceClient( authProvider );

var directoryObject = new DirectoryObject
{
    Id = "bb165b45-151c-4cf6-9911-cd7188912848"
};

await graphClient.DirectoryRoles["{directoryRole-id}"].Members.References
    .Request()
    .AddAsync(directoryObject);

Reference :
Add user to directory role
Directory Role Id's


Multiset Matching

Question: Suppose we have two multisets of positive integers, $A$ and $B$, where the sum of the elements (counted with multiplicity) of the two multisets is the same. Starting from $A$, we would like to arrive at $B$ by applying one of two rules in turn. On a turn, we may either:

Choose an element, $x$, of $A$ and two smaller integers $y$ and $z$ such that $x = y+z$. Decrease the multiplicity of $x$ by $1$ and increase the multiplicity of $y$ and $z$ by 1 each (increasing the relevant multiplicity by $2$ if $y=z$). There is no requirement for $y$ or $z$ to have been elements of $A$ prior to this. Applying this move incurs a one point penalty.

or

Choose two elements, $y$ and $z$, of $A$, reduce each of their multiplicities by $1$ (by $2$ if $y=z$ but we can only choose two equal elements if the multiplicity was at least $2$ to begin with) and increase the multiplicity of $y+z$ by $1$. This move can be used even if $y+z$ was not an element of $A$. Applying this move incurs no penalty.

The resulting multiset, $A'$, will then be our new "$A$" for the next turn and the sum of its elements (counted with multiplicity) will be the same as it was for $A$. We would like to use these rules to go from $A$ to $B$. For instance, suppose $A$ is $\left\{2, 5, 7\right\}$ and $B$ is $\left\{1, 2, 3, 3, 5\right\}$. We could use move 1 to go from $A$ to $\left\{1, 2, 5, 6\right\}$ and then use it again to go to $\left\{1, 2, 3, 3, 5\right\}$, incurring a total of two points.
My Question
What would be a computationally efficient algorithm for going from $A$ to $B$ while incurring the smallest possible number of points, i.e. using move $1$ as few times as possible? Is it possible to construct a much more efficient algorithm if we are happy to just be close to the minimum number of penalty points in some sense, perhaps even an informal sense of just having a "sensible" algorithm for keeping the number of points incurred small?
My Thoughts so Far
First of all, there's no benefit to applying moves of type $2$ until we have done all of our type $1$ moves. Secondly, if $A$ and $B$ share some elements, there will always be an optimal solution which just preserves them, i.e. at no point reduces their multiplicity in our "$A$" beyond their multiplicity in $B$.
If $A$ has $a$ elements counted with multiplicity and $B$ has $b$ elements counted with multiplicity, then we will need to incur at least $\max{\left(0, b-a\right)}$ penalty points because we will need to "create" at least that many new elements by using a type $1$ move. We will never need to incur more than $b-1$ points because we could always just use move $2$ to merge all of $A$'s elements together and then use move $1$ $b-1$ times to peel off the elements of $B$.
One way of thinking about the number of points incurred that might be helpful is to keep track of how many elements of $B$ each element of $A$ "contributes" to forming. For instance, if there is a $5$ in $A$ and we use rule $1$ to split it into a $2$, which is matched with a $2$ in $B$, and a $3$ which is combined with a $4$ from somewhere else using rule $2$ to form a $7$ that is matched with a different element of $B$, then we can say that the $5$ from $A$ contributed to two elements of $B$, the $2$ and the $7$. Every element of $B$ that a given element of $A$ contributes is another point incurred so we would like elements of $A$ to contribute to as few elements of $B$ as possible on average.
Answer: The problem is NP-hard, by reduction from the partition problem.
Let $A$ be any multiset, whose sum (counting multiplicity) is $s$.  Let $B=\{s/2,s/2\}$ be the multiset containing two copies of $s/2$.  Then there exists a solution to transform $A$ to $B$ with no points (i.e., using only type-2 moves), iff $A$ is a "yes" instance for the partition problem.
You can also reduce from the subset sum problem, if you find that conceptually simpler.

If you need to solve it in practice, you can formulate it as an instance of integer linear programming and solve it with an off-the-shelf ILP solver.  Let $A=\{a_1,\dots,a_m\}$ and $B=\{b_1,\dots,b_n\}$.  Introduce integer variables $t_{i,j}$ and zero-or-one variables $s_{i,j}$, with the following intended meaning:

Via type-1 moves, we split $a_i$ into the sum $a_i=t_{i,1}+\dots+t_{i,m}$ (ignore the 0 terms in this sum).

Then, via type-2 moves, we construct $b_j$ as the sum $t_{1,j} + \dots+t_{n,j}$.

Therefore, $t_{i,j}$ contains the contribution from $a_i$ that flows to $b_j$.

If $s_{i,j}=0$, then $t_{i,j}=0$.

Therefore, $s_{i,1}+\dots+s_{i,n}-1$ counts the number of type-1 moves applied to $a_i$.

Given these ideas, we can enforce them using integer linear inequalities, as follows:

$a_i=t_{i,1}+\dots+t_{i,m}$, for each $i$
$t_{1,j} + \dots+t_{n,j}=b_j$, for each $j$
$t_{i,j} \ge 0$, for each $i,j$
$t_{i,j} \le M \cdot s_{i,j}$, for each $i,j$, where $M=\min(a_i,b_j)$ is a constant that was chosen to be sufficiently large.

Finally, we minimize the sum $\sum_{i,j} s_{i,j} - m$, as that counts the number of type-1 moves needed.
Comment: Ah, so it's presumably not possible to solve it exactly in polynomial time. Do you have any thoughts on how you might construct a polynomial time algorithm that keeps the number of type-1 moves reasonably small?
Comment: @SpringLandMid, I don't, but many ILP solvers have a way to impose a time limit -- you can ask them to give you the best solution they've found within that time bound.  Maybe someone else will have a more clever solution.
Comment: You can take $M=\min(a_i,b_j)$.  That is, $t_{i,j} \le \min(a_i,b_j) s_{i,j}$.
Comment: @RobPratt, good point, thank you.  I've edited my answer accordingly.

Multiset Matching


Can I use z-scores to compare results from different measures?

Question: Can I compare outcome data from different literacy measures with different scales (delivered to different subjects) by transforming the data into z scores?  Ideally I'd like to use regression and ANOVA techniques to determine if there are significant differences between groups that got different types of literacy interventions.  Is this remotely plausible?  If not, is there another legitimate way to compare the scores from different measures?
Comment: It sounds like you have **multiple** outcome measures, do I have that right? Also, I'm guessing that you suspect these outcomes are not independent of each other; eg, you suspect that they are all measures of the same underlying reality (ie, greater or lesser degrees of 'literacy'), is that right?
Comment: Yes, right.  Multiple outcome measures -- although I'm working hard to narrow down the options.  And yes, they all measure early literacy constructs.
Answer: The answer will depend on the following questions:

How many times was each measure given to each subject?
Were the subjects randomly assigned to the different interventions?
Are the measures normed on some other population?

If, as I suspect, the answers are "1", "No" and "No" then I don't see how this can be done.
However, if the answer to 1 is "more than once" then you can compare how each subject changed over time.
If the answer to 3 is "yes" then you can use those norms.
If, in addition to either of these, the answer to 2 is "yes" then you can generalize.
But it would help us if you gave more information; you may find this post from my blog helpful: How to ask a statistics question.
Comment: Can you provide an academic reverence for your answer regarding the possibility to perform repeated measures anova on different scales by using z-scores?
Thanks in advance;
Tomer
Comment: 1. More than once -- this is a multi-year initiative.
Comment: 2. The participants aren't randomly assigned in most cases, but are using Propensity Score Matching.  And 3. Yes, they have been normed.  How would you use the norms in this case?
Comment: If the measures have been normed, you can then use z on the normed data to compare (as long as both measures were normed on similar populations). That is, don't calculate z within each sample, use mean and sd of the normed group to create a z.


Linear relation of prices

Question:
If 100 units have a unit cost of $18 and 500 units have a unit cost of $15.5, what is the cost
  of 2000 units?

I was asked this in an interview and the only thing I could think of was
for a 5 multiple of units cost goes down by $2.5 so for 20 multiple of units the price goes down by $50 which is obviously wrong.
Please help me with the approach.
Comment: Why would $500$ units cost less than $100$ units?
Comment: @lulu: Are you interpreting the question as "100 units cost `$18` *each*"? That's not how I would interpret the question.
Comment: @rowang  Thank you for clarifying.  Yes, that's the only rational way I can see to interpret the question (note:  that does not preclude those rational ways which I have not spotted).  But your phrasing is poor.  I recommend editing the question to say "the per unit price on $100$ units is $\$18$, while the per unit price on $500$ units drops to $\$15.5$ " or something like that.  Also, note that the linearity of the total cost function is an extra assumption.
Comment: It was for a software programmer position. I believe he was trying to test my problem solving abilities.
Comment: Ah, perhaps there was a fixed cost.
Comment: @lulu: Ordering in bulk is often cheaper due, for example, to reduced transport costs per unit.
Comment: @Will R.  Well, yes.  Though the OP says "the units cost"  as opposed to "the per unit price for N units is...".  But there's no way for us to guess the volume discount the seller is willing to offer.  I mean...I suppose the unit price decays exponentially to the marginal cost, or to some fixed spread over that.  But we know neither the marginal cost nor the fixed spread.
Comment: @lulu: I see your point, but I think we both know the intended meaning. Regardless, questioning the question is not a job interview technique that I would recommend. :)
Comment: Ok, technicalities aside:  Assume the total cost of $n$ units is linear.  So $C(n)=mn+b$.  We are given $C(100)=18\times 100=1800$ and $C(500)=15.5\times 500=7750$.  These two data points determine $m,b$ and then you can solve for $C(2000)$ and deduce the per unit price from that.
Comment: @Will R  Ah, so we weren't on the same page.  How else can you interpret the question?  Obviously the absolute cost can not be lower! Nobody would ever buy the $100$ lot!
Comment: @WillR: Sorry if the question is not clear. But it means if you buy 100 units each unit is going to cost you `18$`
Comment: Note:  as you see from the posted solution by @Clarinetist , even the assumption is ambiguous.  I assumed the total cost function was linear.  That user supposed the unit price function was linear.  I think my assumption is more realistic (the other yields negative prices pretty fast), but both are logically possible within some range.
Comment: @lulu: I do realize, of course, that, under my proposed interpretation, the cost per unit on $100$ units would be 18 cents, compared to a mere 3 cents per unit when buying $500$ units, so that nobody would ever buy the $100$ unit bundle; but, to be honest, I wouldn't have been surprised if the question really was intended to be that way. Ultimately the question was not intended to test business smarts, but rather to test mathematical skill, which are (perhaps unfortunately) often treated as separate.
Answer: As clarified in the comments, the costs above are unit costs, rather than total costs.
The quickest way to do this that I would do in an interview is to assume that the unit price were linear. Compute the slope:
$$m = \dfrac{15.5-18}{500-100}\text{.}$$
This is the unit price change per unit increase. So, we could just take
$$18+ \dfrac{15.5-18}{500-100}(1900)$$
where the $1900$ comes from the difference of $2000$ units and $100$ units. If you have a calculator, this comes out to $\$6.125 \approx \$6.13$. If you're looking for the total cost, multiply this by $2000$.
If we assumed instead that the total cost were linear, then we would have:
$$m = \dfrac{15.5(500)-18(100)}{500-100}$$
and similarly,
$$18(100)+\dfrac{15.5(500)-18(100)}{500-100}(1900) = \$30,062.50$$
which is the total cost, and for $2000$ units, this is a unit price of $\$15.03$.
I'm assuming the interviewer wasn't interested in the answer itself, but rather, your method.
Answer: As stated in the comments, there is is considerable ambiguity here.  We are given two values of the per unit cost function, let's call it $U(n)$.  Logically, we could assume that $U(n)$ were linear but, at best, that could only be true in a range (as it is clearly a decreasing function of $n$, linearity in a large range would allow $U(n)<0$ which is not realistic).   But it seems to me that that the only realistic assumption (that we can have enough information to solve for!) is that the total cost function is linear.  That is to say, if we define $C(n)$ to be the total amount to be paid (thus $C(n)=U(n)\times n$) then $$C(n)=m\times n + b$$  for suitable constants $m,b$.
Let's see how this plays out.  We are given that $$C(100)=18\times 100=1800\;\;\;\;\;\&\;\;\;\;\;C(500)=15.5\times 500=7750$$
A quick calculation to solve for $m,b$ then shows that $$C(n)=14.875n+312.5$$
As noted in the solution by @Clarinetist this lets you solve for $C(2000)=30062.5$ and the associated per unit cost of $15.03125$  .
It is interesting to note that, in the limit as $n\to \infty$ we get the marginal cost of a unit to be $$\lim_{n \to \infty}\;\frac {C(n)}n=14.875$$  Which feels fairly plausible in light of the given data.

Linear relation of prices


How can I successfully implement a "Remember This Action" routine?

Question: I am really having a difficult time getting started with this task.
I have 2 forms on same page.
The first form provides a search box for users to enter one of several search filters like Address, Zip Code, Streets, landlots, even the state name.
If the search produces several results, the user can then click on Advanced Search link.
This Advanced Search contains several checkboxes that allow users to check one or more boxes to narrow their search results.
This works great.
The issue now is to give the user the ability to "save" their search results perhaps using cookies so that next time the user loads this page, the search result
s/he saved last time will become his/her default search page until they change it or until they remove the cookies which wipes out this last Action.
This is more like remembering user's last action when s/he searched this page last time.
My thinking so far, first of all, is to store the SQL query in a Session variable in the php code.
Then, when the user clicks on "Remember This Action" button that I am going to create", I could just have my php code save that session variable value in a cookie.
In other words, I would do something like this:
$tsql = "SELECT  Name, FeatType,
         MinX, MinY, MaxX, MaxY
         FROM mytable
ORDER BY ListOrder, Name DESC";

Session("LastAction") = $tsql

And then have the "Remember This Action" button post to an my php page that simply does:
Cookies("LastAction") = Session("LastAction")
Cookies("LastAction").Expires = Date() + 365 ' expires in one year

Finally, I can provide a "Repeat Last Action" button on our pages and all it does is
$tsql = Request.Cookies("LastAction")
$stmt = sqlsrv_query( $conn, $tsql);

One of the many issues that I am having is that I can't even get beyond this query:
$tsql = "SELECT  Name, FeatType,
         MinX, MinY, MaxX, MaxY
         FROM mytable
ORDER BY ListOrder, Name DESC";

Session("LastAction") = $tsql
I keep getting "invalid" error on
  Session("LastAction") = $tsql

Obviously, I am very, very weak on php but working hard to come to speed.
Your assistance is greatly appreciated
I have made some baby steps here.
I am able to get this syntactically correct:
$_SESSION["LastAction"] = $tsql;

I am also able to set the correct
setcookie('LastAction',$_SESSION["LastAction"],time() + (86400 * 365)); // 86400 = 1 year

Now, what is the equivalence of this line in php?
 $tsql = Request.Cookies("LastAction")

Above was my best stab at it but I don't think it is correct.
I am basically trying to put a "Remember Last Action" buttonn on another page.
Then use:
$tsql = Request.Cookies("LastAction")
to grab values from another page into this page.
Answer: What you are doing here is, basically, saving an SQL query to a cookie (user-editable data).
setcookie('LastAction',$_SESSION["LastAction"] /*We're saving the value, not the session. */,time() + (86400 * 365)); // 86400 = 1 year

Regardless, instead of fooling around with resurrecting sessions based on a sessid saved in a cookie, I'd recommend saving the search flags into a cookie as a serialized array, and then just do your run-off-the-mill search using that data when you need to reuse it.
Basically, you can do:
$saved_search = serialize($search_opts);
setcookie('SavedSearch',$_SESSION["LastAction"],time() + (86400 * 365));

And to reverse the process,
$saved_search = $_COOKIE['SavedSearch'];
$search_opts = unserialize($saved_search);

You will need to figure out from your own code where to snag the $search_opts and how to use them later to run a search, as I do not see the code that makes that part tick.
Comment: Thenks a long @mkumpan.

Any ideas how to do what you recommended?
Comment: Not the entire thing, no, you need to figure out where to get the search options from your user's saved search yourself (most likely just save the flags from POST when you build your search-related query), and you will also have to figure out the code that will re-build, execute and render the search query from the re-used options.

I did, however, elaborate on the cookie/serialization method in the original answer.


How to create a column of incrementing string labels based on values in a column?

Question: I have a dataframe with three timeseries data columns. I want to add labels to the dataframe based on the binary values in one of the columns that increments. Below is a demonstration of the output that I desire ('Rate labels' built based on 'Rate pct change'). I made this using excel but want to do it using python.
            Rate pct change      Sharpe Ratio PCA   long_only_ew    Rate_labels
28/06/2019        -1               0.000000             0.024448    neg rates 1
01/07/2019         1               0.000000             0.030487    pos rates 1
02/07/2019        -1               0.000000             0.036835    neg rates 2
03/07/2019        -1               0.000000             0.054662    neg rates 2
05/07/2019        -1               0.000000             0.055340    neg rates 2
08/07/2019         1               0.000000             0.050585    pos rates 2
09/07/2019         1               0.000000             0.059735    pos rates 2
10/07/2019         1               0.000000             0.064335    pos rates 2
11/07/2019         1               0.000000             0.066124    pos rates 2
12/07/2019         1              -0.002202             0.072657    pos rates 2
15/07/2019        -1               0.003897             0.074136    neg rates 3
16/07/2019        -1               0.003278             0.071436    neg rates 3
17/07/2019         1               0.012141             0.072065    pos rates 3
18/07/2019         1               0.007214             0.074099    pos rates 3
19/07/2019         1               0.006617             0.073397    pos rates 3
22/07/2019         1               0.009760             0.078266    pos rates 3
23/07/2019         1               0.003645             0.075539    pos rates 3
24/07/2019         1               0.016116             0.085452    pos rates 3
25/07/2019         1               0.007491             0.075281    pos rates 3
26/07/2019         1               0.016323             0.090989    pos rates 3
29/07/2019         1               0.011050             0.077088    pos rates 3
30/07/2019         1               0.011531             0.073027    pos rates 3

Rate pct change (interest rates) was derived based on whether (rates were > 0 == 1) and (rates were <0 == -1)
How would I go about creating 'Rate_labels' using python?
Comment: What should the logic behind `Rate_labels` be?
Comment: @ozacha If rate pct change is -1, label should be `neg rates 1` up to when rate pct change is 1. Then when this is 1, rate labels should be `pos rates 1` up to when rate pct change changes to -1 which would create rate labels to be `neg rates 2` up to when rate pct change is 1 which creates rate labels `pos rates 2` up to when rate pct change is -1 and then repeat this incrementing up until the end of the dataframe.
Answer: Try using a combination of shift and cumsum:
df['Rate_labels'] = df['Rate pct change'].lt(0).map({True:'neg', False:'pos'}) + ' rates ' + (df['Rate pct change'].shift(1).ne(df['Rate pct change']) & df['Rate pct change'].eq(-1)).cumsum().astype(str)

Output:
>>> df
            Rate pct change  Sharpe Ratio PCA  long_only_ew  Rate_labels
28/06/2019               -1          0.000000      0.024448  neg rates 1
01/07/2019                1          0.000000      0.030487  pos rates 1
02/07/2019               -1          0.000000      0.036835  neg rates 2
03/07/2019               -1          0.000000      0.054662  neg rates 2
05/07/2019               -1          0.000000      0.055340  neg rates 2
08/07/2019                1          0.000000      0.050585  pos rates 2
09/07/2019                1          0.000000      0.059735  pos rates 2
10/07/2019                1          0.000000      0.064335  pos rates 2
11/07/2019                1          0.000000      0.066124  pos rates 2
12/07/2019                1         -0.002202      0.072657  pos rates 2
15/07/2019               -1          0.003897      0.074136  neg rates 3
16/07/2019               -1          0.003278      0.071436  neg rates 3
17/07/2019                1          0.012141      0.072065  pos rates 3
18/07/2019                1          0.007214      0.074099  pos rates 3
19/07/2019                1          0.006617      0.073397  pos rates 3
22/07/2019                1          0.009760      0.078266  pos rates 3
23/07/2019                1          0.003645      0.075539  pos rates 3
24/07/2019                1          0.016116      0.085452  pos rates 3
25/07/2019                1          0.007491      0.075281  pos rates 3
26/07/2019                1          0.016323      0.090989  pos rates 3
29/07/2019                1          0.011050      0.077088  pos rates 3
30/07/2019                1          0.011531      0.073027  pos rates 3

Answer: The one liner of @richardec is awesome, I just love it, very pythonist.
This is how I did it if you need some other point of view :
df.loc[df.Rate < 0, 'Rate_lab'] = 'neg rates'
df.loc[df.Rate >= 0, 'Rate_lab'] = 'pos rates'
df['cmp'] = (df['Rate_lab'].ne(df.Rate_lab.shift()) & df['Rate'].eq(-1)).cumsum()
df['Rate_labels'] = df['Rate_lab'] + ' ' + df['cmp'].astype(str)
df = df.drop(['Rate_lab', 'cmp'], axis=1)

Comment: Thanks for this, this too works as an alternative. The issue seems to be that it negates the previous value of the rate labels. So for instance after the first pos rates 1, the next positive rate label ignores pos rates 2 and goes straight to pos rates 3.  I have attached the output as a list `['pos rates 1',
 'neg rates 2',
 'pos rates 3',
 'pos rates 3',
 'pos rates 3',
 'neg rates 4',
 'pos rates 5',
 'pos rates 5'] ` etc.
Comment: Thanks for your comment, indeed I red it too fast ! I updated the answer to make it works as expected :) !
Comment: Thanks for the response. Have had a look at the amendments, I seem to be getting this result `['pos rates 0',
 'neg rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',
 'neg rates 0',
 'pos rates 0',
 'pos rates 0',
 'neg rates 0',
 'neg rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',
 'neg rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',
 'pos rates 0',`.]  - will also work on my end to make some adjustments


Include third-party library

Question: I have a third party library located in the root/libraries directory, but it is not called on my webpage. I was wondering using subtheme/subtheme.theme or subtheme.libraries.yml. I call it from here? Any information would be great, I can't find documentation on third party libraries for Drupal 8.
Comment: You have to define it in theme.libraries.yml and either attach it with a hook or include it by default in theme.info.yml.
Comment: @Kevin when I add the library to theme.libraries.yml I get an error on my page. The way I call it is as follows, `swiper: (newline, space & 3 line indent) js: /libraries/slick/slick/slick.min.js` . Am I not allowed to call a file outside the scope of the subtheme?
Answer: You can add your library in your file theme.libraries.yml (or module.libraries.yml) like this:
<code># Parsley use for contact_form.
parsley:
  js:
    dist/js/vendors/parsley.min.js: { minified: true }
  dependencies:
    - core/jquery
</code>
After, you can include your library like any other libraries with a hook or a template.

Include third-party library


The YouTube API stopped allowing 480p playback...?

Question: Last week I noticed that my videos stopped playing in 480p on my website.
I use a WordPress plugin that allows me to specify playback at this quality level.
It's been working for years but now the player defaults to 360p.
Does anyone know if the Google boys made a change?
Thanks!
Answer: YouTube removed the non-DASH streams for 480p and 1080p. This means that you'll no longer be able to watch 480p and 1080p videos in YouTube's HTML5 player if you use Firefox or Safari.
Firefox is working on implementing Media Source Extensions.  They're also working with YouTube to prioritize the parts needed to get YouTube's HTML5 player working with adaptive streaming.
Answer: Ok so it looks like when the user is LOGGED IN to their google account, then the youtube player is basing the video playback quality on the user's settings. If you set the player to the new "Auto" option, then the playback returns to normal e.g. 480p.
Also, if you LOG OUT of Google, the player returns to normal.
So it appears this is a newly implemented feature in which, when the user's session is active, then all YouTube players will attempt to play video at the active user's assumed preferred quality level.


Mover elemento centralizado para canto

Question: Tenho um página simples com poucos elementos: logomarca, abaixo tem o slogan, abaixo tem um ul com ícones de rede social e uma seta que disparará algum script.
Eu gostaria de manter centralizado todos esses elementos no centro da tela, vertical e horizontalmente. Exceto a seta, que deve ficar centralizada horizontalmente, mas no rodapé da página.
Acontece que existem muitas resoluções e eu gostaria que esses elementos ficassem sempre centralizados de acordo com cada resolução do usuário que acessar. Até aí tudo bem.
Eu preciso que, quando o usuário clicar na seta pra baixo, mover a logo para o topo e à esquerda da página. Assim como descer a seta para que fique invisível na tela do usuário, mover os ícones de rede social para o rodapé, mas à direita, e que os outros elementos recebam fadeOut.
Porém, para que os elementos possam ser movidos, precisam de "position:absolute/fixed". Como eu poderia fazer com que os elementos fiquem posicionados como eu quero e movê-los, com animate, para onde quero?
Segue um pedaço do código HTML:

                <img id="logo" src="http://<?= $base?>img/logo.png" />

Slogam

Compartilhe

                         <img src="http://<?= $base?>img/google-plus-ico.png" />

                         <img src="http://<?= $base?>img/facebook-ico.png" />

                         <img src="http://<?= $base?>img/twitter-ico.png" />

                <img src="http://<?= $base?>img/seta-baixo-ico.png" />

Comment: O importante é o CSS.
Comment: Mas eu preciso fazer com o jQuery, pois é dinâmico.
Answer: Pelo que eu entendi, o seu problema está em precisar de position:absolute/fixed para fazer a animação e não conseguir centralizar os elementos usando essas propriedades. Se for isso mesmo, eu sugiro uma solução onde é possível centralizar elementos com position: absolute
#elemento{
 position: absolute;
 left: 50%;
 margin-left: -150px; /*Sempre a metade da largura do elemento*/
 width: 300px;
 height: 300px;
}

Comment: É, eu até fiz isso. A questão é justamente manter os elementos posicionados sempre no centro (vertical e horizontal) independente da resolução do usuário.
Comment: Ontem eu tava meio lesado e devo ter feito algo errado quando testei. Valeu!

Mover elemento centralizado para canto


Salvar imagem de um site com o PHP?

Question: Como eu salvo uma imagem de um determinado site em uma pasta do meu site usando o PHP?
E como ficaria para redimensionar essa imagem?
usei dessa forma mas nao deu certo: file_put_contents($diretorio, file_get_contents($url));
também tentei com a função copy e não obtive resultado algum..
Answer: Sua tentativa com file_get_contents só funciona se o php.ini estiver setado desta forma:
allow_url_fopen=true

Mas se estiver habilitado, nem precisa de file_get_contents, já pode simplificar com copy:
copy( $urlOrigem, $arquivoDestino );

Caso não queira ou não possa ativar o allow_url_fopen, uma alternativa é o cURL:
$curl = curl_init( $urlOrigin );
$file = fopen( $fileDestination, 'wb' );
curl_setopt( $curl, CURLOPT_FILE, $file );
curl_setopt( $curl, CURLOPT_HEADER, 0 );
curl_exec( $curl );
curl_close( $curl );
fclose( $file );

Neste segundo caso, a extensão cURL tem que estar habilitada.
Sobre redimensionar imagem com PHP, já temos algumas respostas no site.

Salvar imagem de um site com o PHP?


$A$ be closed in $X$ , $U \subseteq A$ is open in $A$ , $V$ be open in $X$ s.t. $U \subseteq V$ , then is $U \cup (V\setminus A)$ open in $X$?

Question: Let $X$ be a topological space , $A$ be closed in $X$ , $U \subseteq A$ is open in $A$ , $V$ be an open subset of $X$ such that $U \subseteq V$ , then is it true that $U \cup (V\setminus A)$ is open in $X$ ? I can only get that $V\setminus A$ is open in $X$ . Please help . Thanks in advance
Answer: We have that $U\cup V\setminus A=U\cup V\cap A^C$ is open if the complement is closed.
Consider the complement:
$$U^C\cap (V\cap A^C)^C=U^C\cap V^C\cup A$$
then since $U\subset V$ we have that $V^C\subset U^C$ hence the above becomes $V^C\cup A$ which is closed in $X$.
Answer: Since $U$ is open in $A$, $U=W\cap A$ for some open set $W$ in $X$. Let $G=V\cap W$; $G$ is open in $X$, and $G\cap A=V\cap (W\cap A)=V\cap U=U$. Now
$$\begin{align*}
U\cup (V\setminus A)&=(G\cap A)\cup\big(V\cap(X\setminus A)\big)\\
&=(V\cap W\cap A)\cup\big((V\cap(X\setminus A)\big)\\
&=V\cap\big((W\cap A)\cup(X\setminus A)\big)\\
&=V\cap\big(W\cup(X\setminus A)\big)\;,
\end{align*}$$
where the last step follows from the fact that $W\setminus A\subseteq X\setminus A$. Finally, $V,W$, and $X\setminus A$ are all open in $X$, so $U\cup(V\setminus A)$ is as well.
Comment: why is $W\cap(V \cap A)=W \cap A$ ?
Comment: I think it could go like this : $G\cap A=W \cap V \cap A=(W\cap A)\cap V=U \cap V=U$
Comment: @user228169: $A\subseteq V$, so $V\cap A=A$. Your calculation also works.
Comment: How is $A$ a subset of $V$ ?
Comment: @user228169: Sorry, you're right. I'm working on a small mobile device and misremembered which set was which. It'll take me a couple of minutes to fix it.
Comment: No problem :) Thank you
Comment: The rest of your proof looks okay though
Comment: @user228169: You're welcome; sorry for the confusion.


is there any way to calculate confidence intervals (CI) of a ratio?

Question: I need to calculate a ratio of 2 values and each value has its own  95% confidence intervals. The question is: how can I calculate the  95% confidence intervals of the ratio?
Comment: It's best to go back and work with ratios calculated from the original data and even in that case you may find that the log of the ratio is better behaved (assuming that everything is positive). Are you saying that the original data are inaccessible?
Comment: You might consider editing this question, given that you are trying to calculate hazard ratios in Weibull survival models specifically, rather than the ratio of 2 arbitrary values. In your application there might be an issue of whether hazard ratios are appropriate if the shapes of the Weibull distributions differ.
Answer: In general, the variance of the ratio of two random variables can be calculated using the delta method. Specifically, $$\mathrm{var}\left(\frac{X}{Y}\right) \approx \frac{\mathrm{var}[X]}{E[Y]^2} - \frac{2E[X]}{E[Y]^3}\mathrm{cov}[X,Y] + \frac{E[X]^2}{E[Y]^4}\mathrm{var}[Y] $$
You don't give specifics about the two random variables you want to find the ratio of, except that you have the values and the confidence intervals. If you know that the values are the maximum likelihood estimates of the two values and you are willing to take a ride on the assumption train to the mythical land of Asymtopia, then you can treat those values as normally distributed. Specifically, the point estimates can be treated as $E[X]$ and $E[Y]$. You can find $\mathrm{var}[X]$ and $\mathrm{var}[Y]$ by inverting the confidence intervals
$$\text{(e.g. }\ UCL_X - E[X] = E[X] - LCL_X = 1.96\frac{\sigma}{\sqrt{n}},\text{ solve for }\ \sigma^2 = \mathrm{var}[X]).$$
In for a penny in for a pound, so you might as well assume that $\mathrm{cov}[X,Y] = 0$, unless you have good reason to suspect otherwise. After calculating this approximate value of $\mathrm{var}[\frac{X}{Y}]$ it is easy to come up with a confidence interval. So with a small amount of math and very large dose of assumptions you can arrive at a confidence interval for the ratio.
User beware, Asymtopia is a mythical land for a reason. Some very helpful approximating gnomes certainly live there, but there is also a sizeable population of filthy, devious, lying uncertainty trolls that feast on half-baked assumptions and drink the blood of the credulous.
Comment: Dalton,The ratio I want to calculate are risk (hazard) ratios obtained after fitting survival data with the Weibull distribution. I have done so using a commercial statistical package (JMP) which provided me with the hazards and CI but not the hazard ratios.  I tried to "reverse engineer" the CI calculation with my poor statistical background and do something along the lines you suggested . My results are close but off and I don`t fell comfortable making up numbers. Thus I wanted to understand if there was any "proper" way to use the provided CI
Comment: Thanks for this excellent answer! Would you happen to have a reference for the specific formula you use?
Comment: Maybe it is useful to note that this requires $\mu_Y >> \sigma_Y$
Answer: The other answer offers a nice approach, when the two intervals are created using standard normality assumptions. In the more general case, it may not be possible to find the variance of each estimate given only the CI.
More generally, consider two parameters $\theta_1$ and $\theta_2$ with $(1-\alpha)\times 100\%$ confidence intervals $(L_1, U_1)$ and $(L_2, U_2)$ respectively. The set
$$\mathcal C = \{(a,b)|L_1 < a < U_1, \ L_2 < b < U_2\}$$
is a joint confidence region with confidence level $1-\tilde\alpha$. By the Bonferroni inequality, this joint confidence level is at least $1-2\alpha$. If the two confidence intervals are independent, then the joint confidence region is equal to $(1-\alpha)^2$. For example, if the original CI's have a $95\%$ confidence level, then the familywise confidence level is at least $90\%$ (and is exactly $90.25\%$ in the case of independence).
Since the bivariate parameter $(\theta_1, \theta_2)$ will belong to the set $C$ with probability $1-\tilde\alpha$ (as usual, this statement must be interpreted with care), confidence intervals for any function $g(\theta_1, \theta_2)$ can be easily derived. In general, the CI is given by $(\tilde L, \tilde U)$ where
$$\tilde L = \inf_{(\theta_1, \theta_2) \in \mathcal C} g(\theta_1, \theta_2)$$
$$\tilde U = \sup_{(\theta_1, \theta_2) \in \mathcal C} g(\theta_1, \theta_2).$$
For ratios of the form $\theta_1/\theta_2$, we get
$$\tilde L = \frac{L_1}{U_2} \quad\quad\text{and}\quad\quad \tilde U = \frac{U_1}{L_2}.$$

To summarize, if $\theta_1$ and $\theta_2$ have CIs $(L_1, U_1)$ and $(L_2, U_2)$ respectively (with confidence level $1-\alpha$), then $(L_1/U_2, U_1/L_2)$ is a valid confidence interval for $\theta_1/\theta_2$ with a confidence level which is at least $1-2\alpha$.
Note: The exposition here assumes that all values are positive. If not, a confidence interval can still be obtained using the more general version, although one of the confidence bounds may be $\pm \infty$.
Comment: I like this approach, but still, it will give you only a 0.95*0.95 confidence interval and not a 0.95 confidence interval. To get the 0.95 confidence interval you will need some sort of assumptions about the underlying distributions. In addition, this confidence interval is not very powerful. You get a confidence interval that is *at least* 0.95^2, and that is very conservative. The interval might contain a lot more than 0.95^2 and will be possibly wider than neccesary.
Comment: This method will also need to be adjusted when $\theta_1$ can be both negative and positive (while $\theta_2$ is positive), in which case the bounds will be $L_1/L_2$ and $U_1/L_2$. When $\theta_2$ can also be zero, then the bounds go to $\pm \infty$.
Comment: @SextusEmpiricus I agree with your concerns (and thanks for catching the typo). I think if you have additional information about how the original intervals were constructed you should be able to do better. If the CIs are the "only" information you have, this may be the best you can do in general.


Weighted least squares regression on random data, giving large t-statistics more often than "expected"

Question: My question is about the distribution of the t-statistics in Weighted Least Squares regression.
I'm finding that for a fixed Y and random X and W, the "t value" (t-statistic) reported by R (and separately calculated by hand in matlab) has an absolute value greater than 2 close to 10% of the time (as opposed to ~5%, which would be expected for random data). Here is my code:
N = 100
runs = 10000
Y = rnorm(N);

tstats = rbind(lapply(seq(10), function(X){W=abs(rnorm(100,1)); X = rnorm(100,1);
                                  summary(lm(Y~X,weights=W))$coefficient[2,3] }))

This code is just running through a loop where for each row of M (which is a 10,000 x 100 random matrix) --- call the row in a given iteration X ---,  it gets a new set of random (non-negative) "weights", and then regresses Y against X using the weights W, and pulls out the "t value" (which is element (2,3) of the coefficients). Since the W's are random each time, the average correlation between Y and W should be 0 (if that possibly had an affect).
What's worse, even if I skip the weights, the following regression seems to have |t-statistic| greater than 2 about 10% of the time as well:
lm( I(Y*sqrt(W)) ~ I(X*sqrt(W)) )

This regression would have the same inputs, except I'm keeping the intercept (constant vector) and not multiplying it by sqrt(W) as in the weighted regression case. This especially perplexes me.
Alternatively, if you use the code above but replace the weighted lm() there with
lm( Y ~ X )

You will see that the coefficient is significant only 5% of the time.
What is going on here?
Comment: I changed the original code to make it more clear that X is not fixed.
Answer: I think the problem is that you are generating the weights at random, uncorrelated with the y value.  In a real weighted regression the points with lower variance will have higher weights.  Since the true relationship is mean and variance of 0 that means that points furthest from 0 would be consistent with higher variances and therefore lower weights, but you don't given them lower weights, they get random weights which could be high or low giving some more extreme values than expected.
If you do the simulation more realistically by generating a set of weights, then generate Y with variances based on the weights, then analyze (you could use the same set of x's, or randomly generate the x's as well), I would expect the t-values to behave more properly.
Here is a quick example:
tstats <- replicate(1000, { x <- rnorm(N); w <- abs(rnorm(100,1));
    y <- rnorm(100, 0, sqrt(1/w));
    coef(summary(lm(y~x, weights=w)))[2,3]})
mean(abs(tstats)>2)

I saw just under 5% as expected.
Comment: +1 Good catch. There is nothing in the theory of regression that says when you randomize the *weights* that the p-value should average 5%! In fact, because the *data* remain the same during this simulation, we would expect the average p-value (or correspondingly some average t-value) to reflect whatever significance inheres in the data, regardless of the weights. In short, this simulation seems to be just an enormously inefficient way of doing unweighted linear regression.
Comment: Thanks so much for your response. The situation I found myself in was that someone was doing a weighted regression with weights chosen in a way expected (by the theory) to be inversely related to the variance of the data points. However, I found that when I regressed random vectors (X) against the data (Y) using their weights, I got t-statistics with abs value > 2 about 10% of the time. Though they were happily advertising p values of ~0.08. All other data-mining issues aside, it is reasonable to conclude from this brief description, that their choice of weights is affecting the regression?


How can I change a theme for a specific module?

Question: I am new to Drupal, I want to change a theme for a particular module (private message module) current look and feel is not pretty.
I am planning to put a "Garland" theme for Private message module. I have two development environments dev1 & dev2. I have installed private message module in both Environments with different themes Dev1 have "Seven" and Dev2 have "Garland" theme.
Is it possible to set in dev1 with private message module theme as "Garland" and remaining all can use "Seven" theme?
Answer: There's no such concept as "a theme for a particular module": modules generate and manipulate content, working together to build the different parts of a page. Then a single theme renders that content to produce the HTML you see in the browser.
Your best bet would be to identify the paths of pages which you want to use a different theme for, and use something like the Theme Key module to manage the switching for you.
Comment: Hi Clive, Thankyou for Suggestion for Theme Key module. Could you please help me to select the "Property" option to set garland theme in this path http://development1234.dd:8083/?q=messages.


Ajax + Auth CakePHP só retorna false

Question: Estou usando o ajax para fazer login no meu sistema que utiliza CakePHP
Página com Ajax
<form id="login">
    <input type="text" name="username" placeholder="Usuário" required>
    <input type="text" name="password" placeholder="Senha" required>
    <button onclick="enviar()" type="submit" class="button button-block button-outline button-positive">Entrar</button>
</form>

Função do controller do CakePHP
public function login_app() {
    if($this->request->is('post')) {
        if ($this->Auth->login()) {
            echo json_encode(true);
            exit();
        } else {
            echo json_encode(false);
            exit();
        }
    }
}

login_app.ctp
<?php
echo $this->Session->flash('auth');
echo $this->Form->create('User');
echo $this->Form->input('username', array('type' => 'text'));
echo $this->Form->input('password', array('type' => 'text'));
echo $this->Form->end('Entrar');

Answer: Verifique se sua aplicação do CakePhp esta chamando os componentes de de Autenticação, se estiver, verifique se esta setando corretamente o controller e model de Autenticação.

Ajax + Auth CakePHP só retorna false


Finding standard deviation of formula with three variables

Question: Suppose we have the formula
$Y = \frac{2WV^{2}}{\pi D^2}\hspace{1cm}$(1)
where:
w = weight variable,
v = velocity variable,
d = diameter variable
we want to find $SD(Y)$
the solution which was proposed to me was (exactly):
$SD(Y) = \sqrt{(\frac{2v^2}{\pi d^2})^2SD(W)^2 + (\frac{4wv}{\pi d^2})^2SD(V)^2 + (\frac{-4wv^2}{\pi d^3})^2SD(D)^2 }\hspace{1cm}$(2)
and we assume independence of W, D, and V...
The answer provided does not look correct to me...I can see what they did (albeit I'm not sure where the 4s came from), but it doesn't sit well with me, so I decided to do some simulations in R.  I set W, D, and V to be normal distributions with different means and standard deviations and simulated values plugging them into (1) and then estimated the SD of Y.  I plugged the means into the lower case values of the variables in (2) and the standard deviations in the respective areas.  The results were sort of close but not close enough (446 vs 436) (I did enough simulations that the simulation variance was negligible).  I trust the SD from my simulation more than the result from (2).  Any thoughts on if (2) is correct, if not any solutions?
Answer: The origin of that formula is
$ s_f = \sqrt{ \left(\frac{\partial f}{\partial x}\right)^2 s_x^2 + \left(\frac{\partial f}{\partial y} \right)^2 s_y^2 + \left(\frac{\partial f}{\partial z} \right)^2 s_z^2 + \cdots} $
(see https://en.wikipedia.org/wiki/Propagation_of_uncertainty). Those 4's come just from taking the derivative of variables that are squared in your eq. (1).
Comment: You input an estimation of their value and use the uncertainty in that estimation. If the estimation that you have is the average, you can use that together with the error in that average.
Comment: You should make sure about the correlation between the parameters. If it's not zero, you should use a more general formula that can take it into account.
Comment: First of all, if you have 1000 observations {m,d,v}, you can compute 1000 values of Y. From that you can compute EVERYTHING. The average of Y, its SD and even it's PDF. The other method is used when you have just one value for m, d and v (for example you only know their average) and the uncertainty in that value.
Comment: But if you choose to do it the way you said, then you can compute the average  and SD(m). Then, the SD() = SD(m)/sqrt(N), where N is the number of points. Then in the formula you can substitute the averages <...> and the error in the averages SD(<...>) to obtain SD(), where  is approximated by the formula substituting the <...>. But definitely computing 1000 values of Y is preferred, error propagation is just an approximation while the other way is as good as you can get.
Comment: Yes. But it's crucial that if you have several data points you make sure what SD you are using, that of w, v and d or that of their average, with that sqrt(N) factor. If you only have one, then it's the same and it's as you said.
Comment: Thanks, and I assume when solving this you input the expected value of the variables in for the results of the derivatives?
Comment: Ok, just so I'm clear, let's say I have 1000 observations...ex. 1. m = 10, d = 2, v = 100, 2. m = 11, d = 3, v = 102...etc....to find SD(Y) I put in the estimated mean of m, d, and v, into the lower case values in (2) (i.e. the derived parts) and the estimated sd of m, d and v into the SD of (2)...?
Comment: Okay, I think I get it...so ignoring my dumb 1000 sample example...lets say I know the SD of W, V and D.  Then I get a recording, w1, v1, and d1.  I plug in the values of w1, v1, and d1 and the SDs into 2 to get an SD estimate for that new observation...we'll call it y1...
Answer: multivariate Delta method
The gradient is the vector consisting of the terms which are the three fractions under the square root. You are assuming the three random variables are independent, so $\Sigma$ is a diagonal matrix and that is what you will get when you multiply the terms in the formula. It is intended to be used for functions of estimators from a sample of size $n$ and where $n\rightarrow \infty$. It does not necessarily mean it will be a good approximation for the variance of some function of three random variables.
Comment: Thank you, I have been reading about issues with the approximation especially when ratios are present.  I think that's likely why my sim values were off a bit...I cranked up the mean so they were very far away from 0 and now they are converging fairly well.


What's the difference between these examples (* and &)?

Question: I wanted to know what's the difference between these two versions of the main() code in C:
int main() {
   uint32_t a;
   f(&a);
}

and
int main() {
   uint32_t *a;
   f(a);
}

for a function
void f(uint32_t *pointer) {
   // ...
}

Answer: In your first example, you pass a pointer to uninitialized variable a.  f() could store a value there, for instance, and main() would be able to use that value later.
In your second example, you pass an uninitialized pointer a.  f() can't do anything useful with it.
Answer: In the first version you pass a pointer to an un-initialized variable to f().
Don't, unless f()'s task is to initialize the variable.
In the second version you pass an uninitialized pointer to function  f().
Don't.
Answer: They are defining different types of variables, without initialization.
uint32_t a;  defines uint32_t variable on the stack and the function call will pass its address to the f() function.
uint32_t *a; defines a pointer on the stack and pass its value to the function. The pointer is not initialized, thus it could point to any block and any attempt to access that address will result into undefined behavior.
From the perspective of the f() function, it sees pointer values passed to it. In the first call, it can use that address, while in the second, it cannot.
Answer: wee pointer questions!!!
Ok so you know that the function f needs to take a pointer which is exactly as it sounds an address to that actual location in memory. For the first example(f(&a)) you have to pass the address because it lives inside the stack and really isn't shared anywhere outside the life of the function. so when the function returns the memory is destroyed and no longer available. the value of the pointer is not passed just the pointer to the value is passed. which can cause problems because if you change that value then all the things that "point to it" are now changed.
Now for the second one you get the memory from the heap or where ever but that stores the address of the value not the actual value so you can manipulate it and return nothing and the value is still there.