Google Analytics API 401 Invalid Credentials

Question: I'm wracking my brain trying to figure out how to use the Google API PHP Client to access Google Analytics. Specifically, I want to upload cost data from other campaigns. My code does work to get information from GA, I can view traffic data for profiles, but I cannot figure out how to upload.
Here's the code that I'm using for auth:
require_once 'google-api-php-client/src/Google_Client.php';
require_once 'google-api-php-client/src/contrib/Google_AnalyticsService.php';
session_start();

$client = new Google_Client();
$client->setApplicationName("GA-Tester");
$client->setClientId('xxx.apps.googleusercontent.com');
$client->setClientSecret('xxx');
$client->setRedirectUri('http://www.site.com/indext.php');
$client->setDeveloperKey('xxx');

$analyticsService = new Google_AnalyticsService($client);
$dailyUploads = $analyticsService->management_dailyUploads;

if (isset($_GET['logout'])) {
  unset($_SESSION['token']);
}
if (isset($_GET['code'])) {
  $client->authenticate();
  $_SESSION['token'] = $client->getAccessToken();
  $redirect = 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['PHP_SELF'];
  header('Location: ' . filter_var($redirect, FILTER_SANITIZE_URL));
}
if (isset($_SESSION['token'])) {
  $client->setAccessToken($_SESSION['token']);
}
if (!$client->getAccessToken()) {
  header ('Location:http://www.site.com/indext.php');
} else {...

This code does work for requesting data. I can get a list of accounts, profiles, download traffic data for a specific profile, etc. No errors or issues.
When I try to upload my CSV file containing cost data I get a '401 Invalid Credentials' error. Here's the code that sends the file:
$send = $dailyUploads->upload(
  $accountId,
  $webPropertyId,
  $customDataSourceId,
  $start_date,
  1,
 'cost',
  array(
   "reset" => true,
   "data" => $cont,
   "mimeType" => "application/octet-stream",
   "uploadType" => "media"));

I've double checked all of my variables that I'm passing, they're sending the correct information.
So here's my question. If all of my GET requests work without issue, why would my POST request throw an error? I can't tell if this is an error with my code, or with settings in the API console. Can anyone steer me in the right direction?
EDIT:
Here's the error code generated (minus identifiable bits).
Fatal error: Uncaught exception 'Google_ServiceException' with message
'Error calling POST https://www.googleapis.com/upload/analytics/v3/man
agement/accounts/1234567/webproperties/UA-1234567-1/customDataSources/
xXxXxX/dailyUploads/2013-01-17/uploads?appendNumber=1&type=cost&reset=
true&uploadType=media&key=xXxXxX: (401) Invalid Credentials' in /../pub
lic_html/proj/google-api-php-client/src/io/Google_REST.php:66 Stack tra
ce: #0 /../public_html/proj/google-api-php-client/src/io/Google_REST.ph
p(36): Google_REST::decodeHttpResponse(Object(Google_HttpRequest)) #1 /
../public_html/proj/google-api-php-client/src/service/Google_ServiceRes
ource.php(186): Google_REST::execute(Object(Google_HttpRequest)) #2 /..
/public_html/proj/google-api-php-client/src/contrib/Google_AnalyticsSer
vice.php(82): Google_ServiceResource->__call('upload', Array) #3 /../pu
blic_html/proj/dailyUpload_send.php(60): Google_ManagementDailyUploadsS
erviceResource->upload('1234567', 'UA-1234567-1', 'xXxXxXxXxXxXxXx...',
 ' in /../public_html/proj/google-api-php-client/src/io/Google_REST.php
  on line 66

Comment: Did you authorize the auth scope at https://www.googleapis.com/auth/analytics?
Comment: Have you looked at this? https://developers.google.com/analytics/devguides/config/mgmt/v3/mgmtDailyUploadGuide#upload
Comment: Yes, my requests are loosely based on this code. I'm using the PHP library so it doesn't look identical, but my code contains each of these elements.
Comment: I've enabled access for analytics (that's the only service enabled at the moment). I have a client ID for web applications, and a key for browser apps that's set to "any referrer allowed". I had tried entering both my URL and my IPv6 address as referrer, but each of those resulted in a permission denied error so I set it back to any referrer.
Comment: https://developers.google.com/analytics/devguides/reporting/core/v3/coreErrors Here, it says you need a new auth token.
Comment: That's a good tip, I'll spend some time there. I'm currently passing form variables between pages. Maybe tokens can't be passed like this? I'll retry my code with URL parameters to see if that makes a difference.
Comment: I think before anything you should just go and generate a new key, replace your existing one, and see if it works.
Comment: I just tried that (generating a new key). I can still make `GET` requests (accountID, profile, segments, etc), but if I try to `POST` I still get a 401 invalid credentials. I really appreciate your help, btw.
Answer: I just successfully uploaded cost data for the first time. In trading comments with Nicholas Pickering I came across the documentation for Management API Authorization which states that:

You will get a 401 status code if your access_token has expired or if you are using the wrong scope for the API.

The default for scope is read only, which is why all of my GET requests were working without difficulty, but I couldn't complete any POST requests through the API. I can't remember how I found it, but the way to accomplish this within the PHP client library is to append the following line to the client declaration:
$client->setScopes('https://www.googleapis.com/auth/analytics');

As soon as I added that line to my project, I was able to successfully upload cost data. I'm really glad to have cleared this hurdle. I still have a long way to go to complete my project, but at least I know that it's going to work.
Answer: I kept geting this error until I went to Google Console -> APIs & auth ->
APIs and turned on this API usage.


Redirecting in the new React_router-dom v6

Question: I have just started react and using custom hooks and there is an issue in my page redirection
When the button is clicked it is supposed to take me to another page but as soon as i use the Navigate hook from react-router-dom it renders an empty page
This is my home.js
import * as React from 'react';

import { styled } from '@mui/material/styles';
import Box from '@mui/material/Box';
import ButtonBase from '@mui/material/ButtonBase';
import Typography from '@mui/material/Typography';
import Grid from '@mui/material/Grid';

import { useNavigate } from "react-router-dom";

//image button rendering
const images = [
  {
    url: 'https://images.pexels.com/photos/1251861/pexels-photo-1251861.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=650&w=940',
    title: 'Student Login',
    width: '100%',
    path : '/student'
  },
  {
    url: 'https://images.pexels.com/photos/2173508/pexels-photo-2173508.jpeg?auto=compress&cs=tinysrgb&dpr=1&w=500',
    title: 'Staff Login',
    width: '100%',
    path : '/admin'
  },
];

const ImageButton = styled(ButtonBase)(({ theme }) => ({
  position: 'relative',
  height: 200,
  [theme.breakpoints.down('sm')]: {
    width: '100% !important', // Overrides inline-style
    height: 100,
  },
  '&:hover, &.Mui-focusVisible': {
    zIndex: 1,
    '& .MuiImageBackdrop-root': {
      opacity: 0.15,
    },
    '& .MuiImageMarked-root': {
      opacity: 0,
    },
    '& .MuiTypography-root': {
      border: '4px solid currentColor',
    },
  },
}));

const ImageSrc = styled('span')({
  position: 'absolute',
  left: 0,
  right: 0,
  top: 0,
  bottom: 0,
  backgroundSize: 'cover',
  backgroundPosition: 'center 40%',
});

const Image = styled('span')(({ theme }) => ({
  position: 'absolute',
  left: 0,
  right: 0,
  top: 0,
  bottom: 0,
  display: 'flex',
  alignItems: 'center',
  justifyContent: 'center',
  color: theme.palette.common.white,
}));

const ImageBackdrop = styled('span')(({ theme }) => ({
  position: 'absolute',
  left: 0,
  right: 0,
  top: 0,
  bottom: 0,
  backgroundColor: theme.palette.common.black,
  opacity: 0.4,
  transition: theme.transitions.create('opacity'),
}));

const ImageMarked = styled('span')(({ theme }) => ({
  height: 3,
  width: 18,
  backgroundColor: theme.palette.common.white,
  position: 'absolute',
  bottom: -2,
  left: 'calc(50% - 9px)',
  transition: theme.transitions.create('opacity'),
}));

const Home = () => {
  //redirect function
  let navigate = useNavigate();
  const routeChange = () =>{
    navigate('/admin/login');
  }

  return (
    <React.Fragment>

      <Grid
        container
        spacing={0}
        direction="column"
        alignItems="center"
        justifyContent="center"
        style={{ minHeight: '100vh' }}
      >
        <Grid item xs={3}>
          <Box sx={{ display: 'flex', flexWrap: 'wrap', minWidth: 400, width: '100%' }}>
            {images.map((image) => (
              <ImageButton
                focusRipple
                key={image.title}
                style={{
                  width: image.width,
                }}
                onClick={routeChange}

              >
                <ImageSrc style={{ backgroundImage: `url(${image.url})` }} />
                <ImageBackdrop className="MuiImageBackdrop-root" />
                <Image>
                  <Typography
                    component="span"
                    variant="subtitle1"
                    color="inherit"
                    sx={{
                      position: 'relative',
                      p: 4,
                      pt: 2,
                      pb: (theme) => `calc(${theme.spacing(1)} + 6px)`,
                    }}
                  >
                    {image.title}
                    <ImageMarked className="MuiImageMarked-root" />
                  </Typography>
                </Image>
              </ImageButton>
            ))}
          </Box>

        </Grid>

      </Grid>
    </React.Fragment>
  );

};

export default Home;

This is redirect.js
import { BrowserRouter as Router, Route, Switch } from 'react-router-dom';
// import { useContext } from 'react';

import { PrivateStudentRoute } from './PrivateStudentRoutes';
import { PrivateStaffRoute } from './PrivateStaffRoutes';
import { PublicRoute } from './PublicRoutes';

import Action_Page from '../../admin_page/.home/Action_Page';
import Home from '../home/Home';
import LoginStaff from '../home/staff/LoginStaff';
import LoginStudent from '../home/student/LoginStudent';

import React from 'react';

const user_routes = (

    <Switch>

        <PublicRoute path="/home" exact component={Home} />

        <PublicRoute path="/student/login" exact component={LoginStudent} />

        <PublicRoute path="/admin/login" exact component={LoginStaff} />

        <Route path="*" component={Home}/>

    </Switch>

);

const Redirect_page = () => {
    return (
        <Router>
            {user_routes}
        </Router>
    );
};

export default Redirect_page;

I think the error is in how the navigate function is defined in my code
can someone. in redirect.js publicstudentroute and publicstaffroute is just route with proper authentication. page renders normally if i exclude routeChange function
Answer: React-router v6 introduced Routes and elements, which replace Switch and component.
If you want to redirect user to another page, you need to adjust to these new features. Your user_routes should looke like this:
<Routes>

    <PublicRoute path="/home" exact element={<Home />} />

    <PublicRoute path="/student/login" exact element={<LoginStudent />} />

    <PublicRoute path="/admin/login" exact element={<LoginStaff />} />

    <Route path="*" element={<Home />}/>

</Routes>

You can read about upgrading from v5 to v6 here


difference between expectation of a random variable in measure theory and in regular calculus probability books

Question: let X be a random variable with a density function $f_{X}(x)$. The expectation of X is defined as
$E[X] = \int x f_{X}(x) dx$
While in the probability books that uses the measure theory it is defined as
$E[X] = \int X dP$
how are these two definitions related? and if I have another random variable Y is its expectation defined in a similar manner, i.e., :
$E[Y] = \int Y dP$
if yes, how can I know that I am integrating with respect to r.v X or Y ?
any help is appreciated
Comment: It's a question of notational style.  The probability books' notation is intended to be useful even if the distributions do not have densities. But the calculus books' notation is intended to make it easy to recognize integration exercises in story problems.  There almost as many different notations for writing integrals of the sort you ask about as there are mathematics books, so some inconvenience is to be expected.
Answer: Let $(\Omega,\mathcal{A},P)$ be a probabilty space, $(\mathbb{R},\mathcal{B})$ a measurable space and $X\colon \Omega\to\mathbb{R}$ a random variable. The expected value of $X$ is defined as the Lebesgue integral
\begin{align}
\mathbb{E}_P(X):=\int_{\Omega}X\,\mathrm{d}P
\end{align}
and by the change of variables formula it holds
\begin{align}
\int_{\Omega}X\,\mathrm{d}P=\int_{\mathbb{R}}x\,\mathrm{d}P_{X}
\end{align}
where $P_X:=P\circ X^{-1}$ is the distribution (push forward, image measure, $\ldots$) of $X$ with respect to $P$.
If $X$ has a probabilty density function $f=\frac{\mathrm{d}P_X}{\mathrm{d}\lambda}$, we can write
\begin{align}
\int_{\mathbb{R}}x\,\mathrm{d}P_{X}=\int_{\mathbb{R}}x\cdot f(x)\;\mathrm{d}\lambda(x)
\end{align}
Comment: what do you mean by "change of variables formula"?  and what does $\lambda$ represent in this context?
Comment: @Jasmine In this context $\lambda$ represents the lebesgue measure as usual in measure and probability theory. For the other question please take a look at [this wikipedia](https://en.wikipedia.org/wiki/Pushforward_measure) entry and read the sections "definition" and "main property: change of variable formula"
Comment: thank you for your reply. I have another question, in the second formula that you posted, what is the difference between $X$ and $x$, and is the final integration Lebesgue integration ?  and could you please answer the second part of the question, if I have a second R.V, how can I know that I am integrating with respect to y (without using the change of variable formula) ?
Comment: @Jasmine Well, $X$ is the random variable and $x$ is the value of the real function $\mathrm{id}\colon\mathbb{R}\to\mathbb{R},x\mapsto x$. The final integration in my answer above is a lebesgue integration, but often you can calculate it as a usual with fundamental theorem of calculus. For $\mathbb{E}(Y)=\int Y dP$ we have $\int Y dP=\int y dP_Y$
Comment: thanks for the reply, when I need to work with the integration in the original equation, how can I deal with $X$? I mean I keep it as $X$, or I substitute it with its density function? also could you please help me answering [this](https://math.stackexchange.com/questions/2588775/total-expectation-with-lebesgue-integral) question and [this](https://math.stackexchange.com/questions/2590191/marginal-density-using-lebesgue-integral) question


IDE - Divergencia entre IDEs em relacao ao tab [ao Enter]

Question: Por motivos de hadware, sou o único em minha equipe que esta utilizando Sublime Text 3, os demais estão utilizando Eclipse Luna ou Mars, ate ai nada de mais.
Contudo desde a troca tenho tido alguns problemas na parte do commit,
pois meus aquivos começaram ficar com divergências em varias linhas, e quando vou verificar o que é, se trata apenas do espaçamento tab.
Já verifiquei a questão do tab-size, ambos esta com 4, e o encoding ambos estão com UTF-8.
Haveria mais alguma configuração que poderia causar esta divergência?
Penso que se fosse o caso de encode não deveria causar erro no espaçamento do aquivo inteiro em vez do apenas alguns fragmentos?
Edição
Estava verificando novamente os commits foi então que achei a opção, ShowWhiteSpace Characters no eclipse.
E na linha em que estão ocorrendo os erros tem esta peculiaridade :

Como podem ver no final da linha há ¤, alguém sabe disser que caractere é este? e para que serve? ele é setado quando dou enter no eclipse.
Comment: Confirme se o Eclipse está convertendo espaços para tabs ou se é o sublime que está fazendo isso.
Comment: @LuisHenrique discordo, o que tem q ser feito é um alinhamento do padrão, independente de qual ele seja, eclipse por exemplo usa por padrão tabulação, já o sublime usa por padrão espaços, portanto, no lugar de todos os que usam tabulação mudarem pra espaço seria interessante configurar o sublime para usar tabulação, já que ele é o único a usar sublime.
Comment: @gmsantos no Eclipse `Preferences->General->Editors->Text Editors` a opcão `insert space for tabs` esta desmarcada, e em `Preferences->PHP->Code Style->Formatter` estou usando `PHP convention` na qual `tab policity` = `Tab`. No sublime `translate_tabs_to_spaces` = `false`.
Comment: @GuilhermeLautert não utilizo nenhum dos dois, só comentei conceitualmente qual pode ser o problema.
Comment: Prefira usar sempre espaços ao invés de `\t`, evita esse tipo de problema.
Comment: Todo mundo ai usa o mesmo S.O?
Answer: Você pode configurar o seu editor para utilizar o padrão de tabulações, as opções padrão usadas pelo editor acessando o menu Sublime Text > Preferences > Settings - Default. Todas estas opções podem ser personalizadas através do menu Settings - User.
{"translate_tabs_to_spaces": false}: converte espaços em tab.

Comment: a configuracao default ja esta deste modo, mas obrigado e ajuda.
Answer: Essa divergência ocorre nos trechos que você edita do arquivo no
Sublime
?
No eclipse há uma configuração para remover o esse whiteSpace. Você indo em Preference -> [linguagem, por exemplo JAVA] -> Editor -> Save Action, marque o check Perform the selected actions on save, depois Additional actions. Em seguida clique em Configure... e na aba Code Organizing marque a opção Remove trailing whitespace.
Além disso você pode configurar para exibir esses caracteres. Basta ir em Preference -> General -> Editors -> Text Editors e na opção Show whitespace character (deve ser marcada) tem um link configure visibility. Clicando irá abrir uma tela para você marcar os caracteres que deseja visualizar em diferentes posições no código. Para o space (ou qualquer outro que quiser) marque a opção trailing. Reinicie o eclipse e verifique se o problema continua acontecendo.
Comment: Através de sua resposta achei o problema, vendo o símbolo que o eclipse usa para `\r`, que seria justamente o de minha "edição na pergunta", na qual eu conhecia como LF, desta forma vi que o único problema é a divergência entre Linux e Windows, na qual Linux usa apenas CR (`\n`) e o Windows LF CR (`\r\n`).
Answer: Basta vocês usarem: http://editorconfig.org/
simplesmente configura um arquivo .editorconfig na raiz do seu projeto, instale o plugin para a sua respectiva IDE e pronto! todos terão a mesma configuração.
Comment: Agradeço a recomendação, irei sugerir ao resto da equipe.


melody starting point on song parts

Question: Hi I'm writing a song in a Bb Major key
and the chords goes something like
EbM7 , F7 , Gm7
and the first note of the melody starts with G note. (this is the main chorus part)
And if the verse part starts with the Bb above the G note, is it weird ?
since normally chorus is the highest part of the song ?
if there is other cases like mine
let me know !
thanks !
Comment: There are no rules, it's not weird, it's your song, write what you like. Choruses are not 'normally the highest'. If you like what you've written try it out on others. That's a far better test.
Comment: @Tim then do you know any other songs that are like what I said ??
Answer: As G is the 6th of Bb, this will give it a Bb6 chord. It sounds like it may sound good!
Comment: this is the scale B♭, C, D, E♭, F, G, the Bb chord is B♭, D, F and the 6th is G; you are right my bad. I'll edit my answer
Comment: Are you sure G is the 7th of B-flat? It seems to me that G is just a sixth above B-flat!
Answer: I've heard several songs where the chorus is sung in a lower range than the verses, and in pop especially it's not impossible to find songs that do what you've done. The reason why typically it isn't done often is that the chorus is usually high-energy compared to the verse, and it's hard to convey that energy in a lower range.
Examples? Michael Jackson's "Don't Stop 'Til You Get Enough" has a higher range in its verse than chorus (but it's pretty close). "Staying Alive" by the Bee Gees is also very close, but most parts are higher in the chorus. Taylor Swift's "Bad Blood" is a current example, as well as "This is America" by Childish Gambino (this is more common in rap/hip hop, as the transitions from verses to choruses often are decreasing in energy).There are many other examples, I just don't know of any way to easily research that statistic.
Generally, it's not common in genres where the chorus is usually a climax of energy; however, some, like rap/hip hop, often have verses as the important parts of the songs.

melody starting point on song parts


What can you do when you have predictor variables that are based on group averages with different sample sizes?

Question: Consider a classical data analysis problem where you have an outcome $Y_{i}$ and how it is related to a number of predictors $X_{i1}, ..., X_{ip}$. The basic type of application in mind here is that

$Y_{i}$ is some group-level outcome such as the crime rate in city $i$.
The predictors are group level characteristics such as demographic features of city $i$.

The basic goal is to fit a regression model (perhaps with random effects but forget that for now):
$$ E(Y_{i} | {\bf X}_{i} ) = \beta_0 + \beta_1 X_{i1} + ... + \beta_p X_{ip} $$
Does some technical difficulty arise when one (or more) of the predictors are the result of a survey that has different sample sizes for each unit? For example, suppose $X_{i1}$ is a summary score for city $i$ that is the average response from a sample of individuals from city $i$ but the sample sizes these averages were based on are wildly different:
\begin{array}{c|c}
{\rm City} & {\rm Sample \ size} \\
\hline
1 & 20 \\
2 & 100 \\
3 & 300 \\
4 & 5 \\
5 & 3 \\
\vdots & \vdots \\
\end{array}
Since the predictor variables do not all have the same meaning, in some sense, for each city, I'm afraid that conditioning on these variables in a regression model as though they are all "created equal" could cause some misleading inferences.
Is there a name for this type of problem? If so, is there research on how to handle this?
My thought is to treat it as a predictor variable measured with error and do something along these lines but there is heteroskedasticity in the measurement errors, so that would be very complicated. I could be thinking of this the wrong way or may be making this more complicated than it is but any discussion here would be helpful.
Comment: This is called the "heteroscedastic errors-in-variables" problem.  (This phrase is a good target for a Google search.)  Recently (2007), Delaigle and Meister proposed a nonparametric kernel density estimator in a [JASA article](http://www.ms.unimelb.edu.au/~aurored/DelMei_FINAL.pdf).  An abstract about some parametric methods (method of moments and MLE) suggests some additional approaches: http://www.sciencedirect.com/science/article/pii/S1572312709000045. (I am not familiar enough with the research to give you an authoritative answer about how to handle your particular dataset.)
Comment: @whuber +1 for both comments. I think "errors-in-variables" was the missing keyword I was looking for. If no one gives a strong answer below that I can accept then I'll look into the literature and come back to post whatever I end up doing as an answer.
Answer: The paper "A heteroscedastic structural errors-in-variables model with equation error" can be downloaded at the author's page:
http://www.ime.usp.br/~patriota/curriculo-eng.html#Published_papers
basically you must take into account the variability of both variables to avoid inconsistent estimators, non-reliable hypothesis tests and confidence intervals.
Answer: One way to deal with this would be to suppose that every city has a distribution with the same variance $σ^2$ for the individual responses. Then each city's average measurement $X_i$ for the predictor would have variance $σ^2/n_i$, where $n_i$ is the number of individuals in the average for city $i$.  That would be a simple way to deal with the heteroskedasticity. I don't know any special name for this form of the regression problem.
Comment: Macro, as a rule of thumb in homoscedastic errors-in-variables regression, if the errors in the IVs are small compared to errors in the DV, you can safely ignore the former and resort to ordinary regression. That gives you a quick, simple way to triage the problem.
Comment: @whuber, thanks - that's useful. It seems that if that rule of thumb makes sense then it would make sense in the heteroskedastic case to use "if the largest error variance in the IVs is small compared to the error variance in the DV, you can safely ignore the problem" would be a reasonable rule of thumb which is a condition that may actually be satisfied in the data I'm looking at.
Comment: @Michael, in the data I'm looking at the variance of the measurement that is being averaged is not huge. I'd have to check but let's say $\sigma^2 \approx 1$, so the variance of the averages (if its reasonable to say the variance is constant across units - another thing I'd have to check) is $\approx 1/n$, so it ranges between $(.05,1)$ for the sample sizes in my data set. The error variance in $Y_i$ is likely to be one, maybe two orders of magnitude larger than this (again, I'll have to check).
Comment: That seems reasonable although I was hoping to avoid having to model the measurement error at all. If I did go in that direction, what would you use to estimate the effect of a predictor measured with error? I've used one method called SIMEX but this seems uncommon and I'm wondering if there are other options.
Comment: @Macro I am not familiar with specific software for modeling regression with a variance function to estimate.
Comment: @Michael, I was asking what method you would use at a higher level, not as specific as what software you'd use, etc. Whuber has provided some good links on the subject though.
Comment: @Macro I was thinking the same thing.  You could probably even stretch it a bit: I bet that errors in high-leverage points would be the most important and you could get away with relatively large errors in low-leverage ("typical" or "average") points.
Comment: @whuber I am not so sure about these rules of thumb about heteroskedasticity not mattering.  In Macro's case the variances differ so widely that I do think the problem cannot be ignored.


Very low throughput on 10GbE network

Question: I have two Linux machines, each equipped with a Solarflare SFN5122F 10GbE NIC. The two NICs are connected together with an SFP+ Direct Attach cable.
I am using netperf to measure TCP throughput between the two machines. On one box, I run:
netserver

and on the other:
netperf -t TCP_STREAM -H 192.168.x.x -- -m 32768

I get:
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.x.x (192.168.x.x) port 0 AF_INET
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 87380  16384  32768    10.02    1321.34

The measured throughput is 1.3Gb/s. This is 7.5x below the theoretical maximum, and only 30% faster than 1GbE.
What steps can I take to troubleshoot this?
Comment: Did you enable jumbo frames?
Comment: What is the hardware of the boxes involved? You cannot achieve 10GbE line rate with anything less than modern Westmere/Nehalem/Sandy Bridge boxes.
Comment: Do you get low throughput on UDP tests as well?
Comment: @gekkz: Changing MTU to 9000 on both ends made no perceptible difference.
Comment: @ShaneMadden: UDP throughput is about the same.
Comment: Then is the CPU of the sending system the bottleneck?  Random data generation at that speed can be expensive.
Comment: I've had the exact same problem for over a year. I think the problem is with 10GbE. I'm thinking that even 8Gbps Fibre Channel is much better at throughput than 10GbE.
Answer: few things:

did you try adjusting mtu to make use of jumbo-frames?
are you absolutely sure that the link between the two servers does not have any packet losses?
does ethtool show you any errors on the interfaces on both ends?
what does top/atop say during the prolonged test - do you see any of the cores fully occupied by iowait?

you'll [most probably] not achieve full 10gbit on a single tcp session but you can do some additional tuning to get closer to it by tweaking tcp handling and communication with the network card itself - take a look here or here .

Very low throughput on 10GbE network


In a topological space, can an open set be homeomorphic to a non-open set? Why or why not?

Question: So given a topological space $X$ where $\mathcal{T}$ is its topology. Suppose $A$ is an subset of $X$ which is not in $\mathcal{T}$ but it's possible to define a subspace topology on $A$ and we can call it $\mathcal{T}_A$. Similarly, one can define a subspace topology $\mathcal{T}_B$ on an set $B\in\mathcal{T}$. The question is whether there exists a homeomorphism between $(A,\mathcal{T}_A)$ and $(B,\mathcal{T}_B)$
To be specific, I'm considering the case where $B$ is a connected neighborhood in $R^2$ and $A$ is the closure of $B$.
Comment: @ThomasShelby: The homeomorphism may not extend across the entire space.
Comment: What is the definition of a homeomorphism?
Comment: Like the usual definition in terms of continuity.
Comment: But the continuity is terms of the subspace topology.
Answer: Consider the topological space $\{ 0 \} \cup [1, \infty)$ with the subspace topology from the standard topology on $\mathbb{R}$.  In this topological space, $\{ 0 \}$ is open but $\{ 2 \}$ is not.
Comment: Then you could modify the counterexample - for example, consider the subspace $[0, 1] \cup (4, 7)$ of $\mathbb{R}$ where $[0, 1]$ is open but $[5, 6]$ is not.
Comment: What if I add a restriction where the isolated point is not allowed?
Comment: Okay, I see. I'm actually considering a more specific case stated in the question. Do you think that's possible in that case?


Magento 2: dFlip 3D FlipBook - jQuery Plugin

Question: I'm trying to use dFlip jquery plugin on Magento 2 with no success.
I've added the css files to the default_head_blocks.xml
The following to the requirejs-config.js:
<code>"map": {
    "*": {
        'dflip_js': 'js/dflip'
    }
},
</code>
And this to my phtml file:
<code><script>

var dFlipLocation = "https://www.my-domain.pt/dev/testing/pub/static/frontend/vendor/theme/pt_PT/";

require([
    "jquery",
    "dflip_js"
], function ($) {
    var flipBook;

    //dflip assets location is other domain or folder.. once is enough

    var pdf = "https://www.my-domain.pt/file.pdf";
    var options = {height: 500};
    flipBook = $("#flipbookContainer").flipBook(pdf, options);

});
</code>

I know I shouldn't use an absolute url like that but I'm just trying to get this to work.
My console is full of errors..

Anyone here got this plugin to work with magento 2?
thanks!
Answer: The dFlipLocation is incorrect.. It should point to the folder where dflip contents reside. Once correct Three.js will load and it will work.. The dflip.js is getting called properly.. just the supporting libraries needs to be pointed..
If the location is proper you can always pre-load three.js through script tag.. Even that will work..


how to add div tag to field collection

Question: I have a field collection named rmr_bar_li. I want to add a div to each line item of field collection, that is within field collection. How can I do that? Can any one help?
Answer: We need to write a hook_form_alter();
In my case ,
function mymodule_form_alter($form,$form_state,$form_id) {
  form['rmr_bar_li'][und][0]['anyfield']['#suffix'] =

;
}

Here anyfield is field inside the field collection below which you have to place your DIV tag.

how to add div tag to field collection


Configuring Wi-Fi on Android via Intune. Missing Wi-Fi certificate issue

Question: I am having an issue when Android phones are unable to get correct WiFi configuration policy from Intune. Intune is in 'cloud mode' (non-hybrid)
I have the following polices configured in Intune:

Deploy root CA certificate (works OK)
Deploy SCEP certificate (works OK)
Deploy WiFi configuration (this is where the problem is)

Things to note:

Exactly the same configuration for iOS devices works perfectly. So configuration of Intune and WiFi is OK and it seems to be an issue configuring Android device WiFi policy.
I tried using PFX certificate instead of SCEP, hoping it will help in solving issue. It did not
Android is able to receive SCEP certificate from Intune, but it is placed in System store but not in WiFi store. Hence Android is not able to use the certificate for WiFi authentication
Answer: The following blog post provides solution for Intune hybrid-setup.
Unfortunately, it could not be implemented for cloud-based Intune deployment.
So I ended up creating OMA-URI config for Android devices based on the information from the blog (thanks goes to Scott Breen for his blog post).
Important: replace the following values in OMA-URI config

Corporate WiFi: should be the name of your WiFi SSID name
1234567890ABCDEF: should be hex of your WiFi SSID name (hex exact case of SSID name)
00 11 22 33 44 55 66 77 88 99 00 AA BB CC DD EE FF 00 11 22: SHA-1 hash of Root CA certificate

OMA-URI config
Setting name: Corporate WiFi (replace this with your SSID)
Date type: String
OMA-URI (case sensitive): ./Vendor/MSFT/WiFi/Profile/Corporate WiFi/Settings(replace Corporate WiFi with your SSID)
Value: (replace values mentioned previously)
<?xml version="1.0"?>
<WLANProfile xmlns="http://www.microsoft.com/networking/WLAN/profile/v1">
    <name>Corporate WiFi</name>
    <SSIDConfig>
        <SSID>
            <hex>1234567890ABCDEF</hex>
            <name>Corporate WiFi</name>
        </SSID>
    </SSIDConfig>
    <connectionType>ESS</connectionType>
    <connectionMode>auto</connectionMode>
    <MSM>
        <security>
            <authEncryption>
                <authentication>WPA2</authentication>
                <encryption>AES</encryption>
                <useOneX>true</useOneX>
                <FIPSMode xmlns="http://www.microsoft.com/networking/WLAN/profile/v2">false</FIPSMode>
            </authEncryption>
            <PMKCacheMode>disabled</PMKCacheMode>
            <preAuthMode>disabled</preAuthMode>
            <OneX xmlns="http://www.microsoft.com/networking/OneX/v1">
                <cacheUserData>false</cacheUserData>
                <authMode>User</authMode>
                <EAPConfig>
                    <EapHostConfig xmlns="http://www.microsoft.com/provisioning/EapHostConfig">
                        <EapMethod>
                            <Type xmlns="http://www.microsoft.com/provisioning/EapCommon">13</Type>
                            <VendorId xmlns="http://www.microsoft.com/provisioning/EapCommon">0</VendorId>
                            <VendorType xmlns="http://www.microsoft.com/provisioning/EapCommon">0</VendorType>
                            <AuthorId xmlns="http://www.microsoft.com/provisioning/EapCommon">0</AuthorId>
                        </EapMethod>
                        <Config xmlns="http://www.microsoft.com/provisioning/EapHostConfig">
                            <Eap xmlns="http://www.microsoft.com/provisioning/BaseEapConnectionPropertiesV1">
                                <Type>13</Type>
                                <EapType xmlns="http://www.microsoft.com/provisioning/EapTlsConnectionPropertiesV1">
                                    <CredentialsSource>
                                        <CertificateStore>
                                            <SimpleCertSelection>true</SimpleCertSelection>
                                        </CertificateStore>
                                    </CredentialsSource>
                                    <ServerValidation>
                                        <DisableUserPromptForServerValidation>false</DisableUserPromptForServerValidation>
                                        <ServerNames></ServerNames>
                                        <TrustedRootCA>00 11 22 33 44 55 66 77 88 99 00 AA BB CC DD EE FF 00 11 22 </TrustedRootCA>
                                    </ServerValidation>
                                    <DifferentUsername>false</DifferentUsername>
                                    <PerformServerValidation xmlns="http://www.microsoft.com/provisioning/EapTlsConnectionPropertiesV2">false</PerformServerValidation>
                                    <AcceptServerName xmlns="http://www.microsoft.com/provisioning/EapTlsConnectionPropertiesV2">false</AcceptServerName>
                                    <TLSExtensions xmlns="http://www.microsoft.com/provisioning/EapTlsConnectionPropertiesV2">
                                        <FilteringInfo xmlns="http://www.microsoft.com/provisioning/EapTlsConnectionPropertiesV3">
                                            <CAHashList Enabled="true">
                                                <IssuerHash>00 11 22 33 44 55 66 77 88 99 00 AA BB CC DD EE FF 00 11 22 </IssuerHash>
                                            </CAHashList>
                                            <EKUMapping>
                                                <EKUMap>
                                                    <EKUName>Encrypting File System</EKUName>
                                                    <EKUOID>126.96.36.199.4.1.322.214.171.124</EKUOID>
                                                </EKUMap>
                                                <EKUMap>
                                                    <EKUName>Secure Email</EKUName>
                                                    <EKUOID>188.8.131.52.220.127.116.11.4</EKUOID>
                                                </EKUMap>
                                            </EKUMapping>
                                            <ClientAuthEKUList Enabled="true">
                                                <EKUMapInList>
                                                    <EKUName>Encrypting File System</EKUName>
                                                </EKUMapInList>
                                                <EKUMapInList>
                                                    <EKUName>Secure Email</EKUName>
                                                </EKUMapInList>
                                            </ClientAuthEKUList>
                                        </FilteringInfo>
                                    </TLSExtensions>
                                </EapType>
                            </Eap>
                        </Config>
                        </EapHostConfig>
                </EAPConfig>
            </OneX>
        </security>
    </MSM>
</WLANProfile>


Incorrect class displayed by erb

Question: I have the following code:
<% event_class = "eventInfo individual")%>
<div class=<%= raw event_class %>>

In this case, I expect the rendered html to be

but I'm getting

How do I fix this?
Edit: I got the HTML I wanted by writing:
<%= content_tag(:div, nil, class: event_class) %>

I will still leave this question up in case anyone wants to tackle the original problem though.  This looks like a bug in Rails to me.
Answer: It's not a bug, but rather expected behaviour since you did not put the erb tag in quotation marks.
Everything in <% %> tags will be ruby-interpreted server side, so what you end up with in your html is

This would be faulty html, but modern browsers have a lot of tolerance and will translate this into

Just put "" around your erb tag and you are fine
<div class="<%= raw_event_info %>">

EDIT: Your solution is better in every aspect, since you don't mix erb with html. I just wanted to explain what was happening originaly ;)

Incorrect class displayed by erb


Disc write cache on Windows domain controller | always disabled by default?

Question: Is it true that on a Windows 2016 domain controller (bare metall installation), the disc write cache is turned off even if a battery-buffered RAID controller is installed and recognized by the system (by default after every reboot)?
(We don't have experience in Windows bare metall installations and there are contradictory informations on the internet)
Comment: If you set the read/write cache in your controller's BIOS, how should Windows influence this?
Comment: @Lenniey In the properties of the hard disc (windows device manager) you can configure the write cache policy. On our test system (unbuffered cache) the cache can be enabled / disabled there, regardless to the bios settings.
Comment: Ah OK. I don't exactly know what this setting _does_ in combination with a dedicated HDD controller with RAM and a BBU, but I do know that this setting is disabled by default to prevent data loss of the AD database in case of a power outage.
Comment: Yes, Active Directory attempts to disable the disk write cache when the service starts. Any disk that holds a database should never have write caching enabled. That includes Exchange and SQL Server. Disk write caching should also be disabled if there is an array controller cache.
Comment: @GregAskew After reading this my question sounds a bit naive to me. I would humbly accept this as answer :-)
Answer: Yes, Active Directory attempts to disable the disk write cache when the service starts. Any disk that holds a database should never have write caching enabled. That includes Exchange and SQL Server. Disk write caching should also be disabled if there is an array controller cache.
Comment: "Any disk that holds a database should never have write caching enabled" Can you clarify exactly why? Is it to protect against dirty writes that could contain corrupt data that happen from power loss?


lognormal distribution average and variance

Question: I am trying to get acquainted with this type of distribution, and part of it is exploring what random number generation with scipy.stats produces. I have noticed that if I produce a normal distribution and then calculate average and std on the obtained array, the agreement with the imposed parameters is good already for a relatively small number of samples:
import scipy.stats as st
import numpy as np

d3 = st.norm.rvs(1,0.2,size=1000)
print(d3.mean())
print(d3.std())

0.996408828534
0.205429513285

I am not sure I can say the same for a lognormal distribution:
s = 1.2
d1 = st.lognorm.rvs(s, size=3000)
d2 = st.lognorm.rvs(s, size=30000)

d1.std()**2
9.3344792142262989

d2.std()**2
12.281877945365549

whereas from the definition ($\mu = \ln(scale) = 0$)
(np.exp(s**2)-1)*np.exp(s**2)
13.593577362615642

(Note: the parameter s is equivalent to $\sigma$ in scipy, link to very good description)
I wonder why the variance appears to be so sensitive on the number of samples, and in general not very close to its theoretical value. Is it something peculiar to this kind of distribution and the way random numbers are generated, or am I missing something?
EDIT : following a suggestion by @gung, running few tests iteratively confirms 'instability' of the variance, as explained by @hodgenovice:
s = 1.2
for i in range(10):
    d1 = st.lognorm.rvs(s, size=3000)
    d2 = st.lognorm.rvs(s, size=30000)
    lgn_th_mean = np.exp(s**2/2)
    lgn_th_var = (np.exp(s**2)-1)*np.exp(s**2)
    print('d1 stats: mean = {:.3f}, var = {:.3f}, expected mean: {:.3f}, expected var: {:.3f}, mean of log: {:.3f}, var of log: {:.3f}'\
          .format(d1.mean(),d1.std()**2,lgn_th_mean,lgn_th_var,np.log(d1).mean(),np.log(d1).std()**2))
    print('d2 stats: mean = {:.3f}, var = {:.3f}, expected mean: {:.3f}, expected var: {:.3f}, mean of log: {:.3f}, var of log: {:.3f}'\
          .format(d2.mean(),d2.std()**2,lgn_th_mean,lgn_th_var,np.log(d2).mean(),np.log(d2).std()**2))

d1 stats: mean = 2.001, var = 12.595, expected mean: 2.054, expected var: 13.594, mean of log: -0.048, var of log: 1.493
d2 stats: mean = 2.041, var = 11.798, expected mean: 2.054, expected var: 13.594, mean of log: 0.010, var of log: 1.419
d1 stats: mean = 1.912, var = 10.081, expected mean: 2.054, expected var: 13.594, mean of log: -0.037, var of log: 1.379
d2 stats: mean = 2.064, var = 13.583, expected mean: 2.054, expected var: 13.594, mean of log: 0.007, var of log: 1.431
d1 stats: mean = 2.125, var = 12.299, expected mean: 2.054, expected var: 13.594, mean of log: 0.014, var of log: 1.511
d2 stats: mean = 2.047, var = 12.364, expected mean: 2.054, expected var: 13.594, mean of log: 0.001, var of log: 1.443
d1 stats: mean = 2.057, var = 13.110, expected mean: 2.054, expected var: 13.594, mean of log: 0.016, var of log: 1.384
d2 stats: mean = 2.061, var = 20.173, expected mean: 2.054, expected var: 13.594, mean of log: -0.008, var of log: 1.443
d1 stats: mean = 2.129, var = 14.420, expected mean: 2.054, expected var: 13.594, mean of log: 0.038, var of log: 1.429
d2 stats: mean = 2.021, var = 11.572, expected mean: 2.054, expected var: 13.594, mean of log: -0.007, var of log: 1.435
d1 stats: mean = 2.046, var = 15.501, expected mean: 2.054, expected var: 13.594, mean of log: -0.002, var of log: 1.393
d2 stats: mean = 2.050, var = 14.722, expected mean: 2.054, expected var: 13.594, mean of log: -0.006, var of log: 1.444
d1 stats: mean = 2.069, var = 16.008, expected mean: 2.054, expected var: 13.594, mean of log: 0.011, var of log: 1.376
d2 stats: mean = 2.030, var = 19.172, expected mean: 2.054, expected var: 13.594, mean of log: -0.009, var of log: 1.430
d1 stats: mean = 1.966, var = 9.853, expected mean: 2.054, expected var: 13.594, mean of log: -0.009, var of log: 1.412
d2 stats: mean = 2.051, var = 13.964, expected mean: 2.054, expected var: 13.594, mean of log: -0.006, var of log: 1.447
d1 stats: mean = 2.070, var = 15.729, expected mean: 2.054, expected var: 13.594, mean of log: -0.022, var of log: 1.462
d2 stats: mean = 2.073, var = 13.725, expected mean: 2.054, expected var: 13.594, mean of log: 0.005, var of log: 1.447
d1 stats: mean = 2.050, var = 15.448, expected mean: 2.054, expected var: 13.594, mean of log: 0.003, var of log: 1.406
d2 stats: mean = 2.031, var = 12.177, expected mean: 2.054, expected var: 13.594, mean of log: -0.003, var of log: 1.432

Comment: I have included a small set of trials in the question. The effect of the log is to 'calm down' the data, compress them. Sort of the opposite of what the exponential does, which is what @hodgenovice was explaining below.
Comment: How stable would it be if you took the log of the values before calculating the parameters?
Answer: What you've written looks correct to me.
Here's one way of thinking about why the variance is less stable for the lognormal distribution if we estimate it from some generated random samples:
One way of constructing a random variable with a lognormal distribution is by taking e to the power of some random variable with a normal distribution (as you may well know). We could generate 3000 random samples from independent N(0, 1.2) random variables (similar to your lognormal example). If we take the absolute value of these samples (to measure their distance from 0), most of the values are < 1.2, with typically 30-50 values is > 2.5 and 0-3 values > 3.5. The median of our sample is ~ 0.8.
If we repeat the above, and take $e^X$ (take the exponent of each sample) to resemble a lognormal distribution, we find that we have a median value of roughly 1, most values are < 2 with 15-25 values > 12 and maybe one or two > 30.
Therefore, the lognormal distribution is more sensitive to outliers since their distance from your 'typical' point becomes much greater. When we calculate the variance of our samples, the number of these outliers, and exactly how large they are has a greater impact for the lognormal distribution.
Comment: Thanks! I think I get it now. I seem to understand that - for a Normal distribution, the sample $\sigma$ has a variance proportional to $\frac{1}{\sqrt{N}}$, where $N$ is the number of samples..Is there a similar result for lognormal distributions, maybe showing that the dependence is 'slower'?
Comment: @MikeAnblips The $1/\sqrt{N}$ factor will remain, but the variance of the variance may be quite large.


Calculate number of possible operands

Question: I need a function to calculate the number of possible addition equations for a given sum n.
The operands must always be in descending order, cannot be equal, and must be greater than 0.
So, 5+4 is valid but 4+5 5+5 3+3+3 or 9+0 are not.  Only integers are used.
For instance:
n = 2. Result is 0
n = 3. Result is 1. 2+1
n = 5.  Result is 2. 4+1 and 3+2
n = 10. Result is 8.  9+1, 8+2, 7+3, 6+4, 7+2+1, 6+3+1, 5+4+1, 5+3+2
Comment: is it count(10) = 9, you miss the case 4+3+2+1
Comment: n = 2. result should be 1: 1+1
Comment: You may want to specify some more rules. Is 5+0 valid? 0.1+4.9? 6 + (-1)?
Comment: Operands cannot be equal, 1+1 is not valid.
Comment: function calc(){}
Answer: Just divide the number by 2 and floor the result:

function calc(num){
  return Math.floor(num/2);
}

console.log(calc(5)) //4+1, 2+3
console.log(calc(6)) //1+5, 2+4, 3+3
console.log(calc(7)) //1+6, 2+5, 3+4
Comment: This does not work... calc(10) should be 8, not 5
Comment: @JosephTroy It did work prior to OP's edit (and change in requirements).

Calculate number of possible operands


UIPanGestureRecognizer inaccurate on iPhone 5s + iOS 7

Question: I'm using UIPanGestureRecognizer to pan around in my game (cocos2d 2.0), it's been working fine, from iOS 5.x to 6.x. Suddenly in iOS 7 it's getting very choppy. You can pan around for a while, but eventually it becomes inaccurate. Even before my game code does anything with the gesture I can clearly see in my logs that the translation is jumping around 3-5 points. Normally a smooth finger drag only has translation spread out 1 points per change. Swiping near the edges of the screen seems to immediately cause the problem, but just panning causes it as well.
Did anything change that would cause problems with cocos2d? Or the gesture recognizer in general?
Comment: there are apparently some memory leak problems in cocos2d 2.x versions when running on iOS 7, you'll find discussions both here and on the cocos2d forums but atm no solution
Comment: Enough of a memory leak to cause this gesture issue? I'm able to produce the problem very quickly after load. Looks like the memory is more of a slow bleed.
Comment: I haven't followed the thread closely but some say it is about congestion, others experienced it in combination with touch input, others just get a constant increase in memory usage over time though not significant on devices. You may want to check if perhaps your framerate is dropping. Are you maybe adding nodes as a result of panning?
Answer: There seems to be some priority conflict with the CADisplayLink refreshing an openGL view and UIApplication's sendEvent method. I struggled with this for a week!  In my tests I found that a UIPanGestureRecognizer will usually fire with every screen refresh as long as your finger has moved.  When refreshing a GLKView (or any openGL context I'm assuming), every so often the recognizer will skip a few frames. If you hook up a CADisplayLink to refresh a GLKView and pan around you can see it for yourself.
<code>-(void)panRecCallback:(UIPanGestureRecognizer *)rec{
    CGPoint loc = [panRec locationInView:self.view];
    printf("    rec loc %3.3f %3.3f\n",loc.x,loc.y);
}
-(void)display:(CADisplayLink *)displayLink{
    [myGLKView display];
    CGPoint loc = [panRec locationInView:self.view];
    printf("display loc %3.3f %3.3f\n",loc.x,loc.y);
}
</code>
I never did get it totally figured out, but I did find a super hacky way to get it to work consistently smooth.  I found that during the glitch, the recognizer was waiting on a semaphore, which I took to mean that the openGL's drawing was happening before the recognizer's selector was firing, and that it was somehow lagging just enough to block a cycle. So my goal was to delay the recognizer enough that it will fire before the next render. Here's the hack part, I dispatch_async on the main thread the firing of my <code>[glkView display]</code> so that my recognizer would get called, and then I capture the recognizers action in a block and fire it after the cycle has passed.  I hate to say that this was all I could do, but I works!


Como Separar dados em dicionario a partir de um .txt (Python)

Question: Tenho um arquivo de texto (anexo.txt), que tenho a armazenar em um dicionário, com Nome como chave e telefone, valor. o arquivo tem as seguintes informações:
--- nome telefone ---
Ailton-1197765445 Josefa-2178655434

Usei o seguinte código:
ref_files = open("anexo.txt", "r")
for linha in ref_files:
    valos = linha.split()
    print(valos[0],valos[:0],valos[1:])

ref_files.close()

mas ele retorna os valores assim:
Ailton-1197765445 [] ['Josefa-2178655434']
Criei o dicionário mas o retorno eh o mesmo:
dic = { k.split()[0]:k.split()[1:] for k in ref_files.readlines() }

alguém poderia ajudar-me com isso?
Agradecido.
Comment: É importante notar que quando você usa *slicers* em uma lista, tais como está fazendo quando faz `[1:]`, você obtém **uma lista** como resultado.
Comment: São sempre dois nomes por linha ou podem haver mais? A primeira linha do arquivo, `--- nome telefone ---`, também está no arquivo ou só colocou na pergunta para exemplificar?
Comment: @AndersonCarlosWoss são sempre dois nomes por linha, e a primeira linha também está no arquivo.
Answer: def monta_dicionario(caminho=None):
    with open(caminho, "r") as anexo:
        dic = {}
        for linha in  anexo.readlines()[1:]:
            nomes = linha.split(" ")
            for nome_tel in nomes:
                val = nome_tel.split('-')
                dic[val[0]] = val[1]

    return dic

if __name__ == '__main__':
    dic = monta_dicionario('./anexo.txt')
    print(dic)

Comment: Obrigado! ta tudo como queria, obrigado


Bash script. check the website is valid

Question: I need to write a script that tells whether a given website ( first argument ) has been changed. I wrote something like that but it does not work and can not keep find an error
#!/bin/bash

website=$1
time=$2    #frequency of checking the page
x=1
spr=$(diff -s ttt.txt ttt1.txt)
lynx -dump $website | cat > ttt.txt ttt1.txt
until [ 1 -eq 0 ]; do
    sleep $2
    lynx -dump $website | cat > ttt1.txt

        if [ "$(diff -s ttt.txt ttt1.txt)" = "$spr" ]
        then
            lynx -dump $website | cat > ttt.txt
            echo "Changes were made "

        else
            echo "No changes"

        fi
done

Comment: In what way does it not work? Are you getting an error? Is is never reporting changes? Always reporting changes? Something else entirely? Also piping through `cat` like that is pointless you can just redirect from `lynx` directly to the file.
Comment: It's never reporing changes.
Comment: You want `diff -q` or `cmp -s` for your change test for the record. There's no need to compare the output to any other output. What values do those files have when the script starts (i.e. what is the value of `$spr`?)?
Comment: files are empty , the variable spr is to store the value when the files are equal.
Comment: Then is your `if` test backwards? If `$spr` is the output when they are the same then when the new `diff` output is equal to `$spr` doesn't that mean there aren't any changes and when they differ that means changes were made?
Answer: This line is the problem:
lynx -dump $website | cat > ttt.txt ttt1.txt

Because you provide 2 files after the redirection, the shell sees this:
cat ttt1.txt >ttt.txt

So cat is reading from ttt1.txt, not from its stdin

demo:
$ cat > a <<END
hello
END
$ cat > b <<END
world
END
$ echo foo | cat > a b
$ cat a
world
$ cat b
world

"foo" is not stored to either file.

Bash script. check the website is valid


Using undocumented fields in an App Store Receipt

Question: OK I'm working on a app where the user purchases the app from the App Store, gets access to the app for a year, and then after the year has passed they are sent to an In-App Purchase screen where they can purchase a subscription for continued access.
(this is not the subscription model I'd implement if it were up to me, but the client insists that it follow this model)
In order for it to work, though, I need a reliable way to check when the app was purchased, in order to calculate when the first year of access ends. Following the suggestions in one of the answers in...
iOS App Purchase Date
...I've gotten the app receipt and I can find all the data fields that the documentation says there are, but there are also a few others including "original_purchase_date" (not the one in the IAP receipt array; the one for the app receipt itself). This would appear to be what I want. However this field is undocumented.
As far as I can tell, Apple security operates via a form of "Security Through Obscurity/Diversity", so it's been difficult to find further information about these undocumented data fields. My concern is that I might use the data from this field, and then discover that it was only there in sandbox mode and actual receipts don't have it. Worse, what if I reference that field and then Apple releases a new iOS update that completely discards that field from its implementation? Undocumented features are even less reliable than deprecated features, after all.
My question is this: is there a reliable source of information somewhere where I can find out what Apples intends regarding this field? Or better yet, is there another, safer way of achieving the system my client wants?
Answer: You have to implement it as a free app that requires a subscription IAP. Paying for an app cannot be its subscription cost.
Check rule 11.12 here: https://developer.apple.com/app-store/review/guidelines/#purchasing-currencies

Apps offering subscriptions must do so using IAP, Apple will share the
  same 70/30 revenue split with developers for these purchases, as set
  forth in the Program License Agreement

Implementing it the way your client wants will result in rejection.
Comment: Well that makes my life a HELL of a lot easier - early in the design I thought that it would be easier to do it the way you suggest but they nixed it. Now that I have something to point to to tell them the app will get rejected otherwise. Thanks a lot!


Passing Monster variables to Battle controller

Question: I'm building a RPG game in Unity and I have some trouble with my battle system. I have a world scene which is populated by various monsters, if the player collides with the monster it loads the battle Scene. I want to pass the name and ID# of the enemy controller so it can load the correct sprites and prefab (the battle scene is generic). The problem is that my monster object (in this case GrassSlug) is in a different scene (world scene) than my battle controller (battle scene). How can I communicate these values to my battle controller? Do I need to add an extra controller which doesn't get destroyed on load? (I current have a GameManager like that and a SkillManager).
GrassSlug.cs (GameObject in world scene)
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class GrassSlug : Monster {
    private string enemyName = "Grass Slug";
    private int enemyID = 001;
}

Monster.cs (GameObject in world scene)
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.SceneManagement;

public class Monster : Character {
    void OnCollisionEnter2D(Collision2D other)
    {
        if (other.collider.tag == "Player") {
            SceneManager.LoadScene("Battle");
        }
    }
}

BattleController.cs (GameObject in Battle scene)
using Random = System.Random;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.SceneManagement;

public class BattleController : MonoBehaviour {
    private Stat playerHealth;
    private Stat enemyHealth;
    private static Random random = new Random();
    private int playerDamage;
    private int enemyDamage;
    public Button attackButton;
    public Button escapeButton;
    public Transform victoryText;
    public GameObject canvas;
    public GameObject button;
    public Canvas battleUI;

    void Start () {
        InitHealth();
        InitUI();
    }

    //
    // Init player and enemy health
    //
    private void InitHealth()
    {
        playerHealth.Initialize(100, 100);
        enemyHealth.Initialize(100, 100);
    }

    //
    // Set up main battle UI
    //
    private void InitUI()
    {
        Button AttackButton = attackButton.GetComponent<Button>();
        AttackButton.onClick.AddListener(PlayerAttack);

        Button EscapeButton = escapeButton.GetComponent<Button>();
        EscapeButton.onClick.AddListener(ReturnScene);
    }

    //
    // Function for the player attack
    // Calculates player's damage and adds XP to the skills
    //
    private void PlayerAttack ()
    {
        playerDamage = random.Next(20);
        enemyHealth.CurrentValue -= playerDamage;
        GameController.Instance.attackXP += playerDamage;
        GameController.Instance.hitpointsXP += playerDamage;

        if(enemyHealth.CurrentValue <= 0) {
            Victory();
        } else {
            EnemyAttack();
        }
    }

    //
    // Function for enemy attack
    // Calculates enemy's damage and adds XP to the  players' skills
    //
    private void EnemyAttack () {
        enemyDamage = random.Next(10);
        playerHealth.CurrentValue -= enemyDamage;
        GameController.Instance.defenceXP += enemyDamage;
        GameController.Instance.SkillDebug();

        if(playerHealth.CurrentValue <= 0) {
            Debug.Log("You are defeated!");
        }
    }

    //
    // Change UI when player wins the battle
    //
    private void Victory() {
        Instantiate(victoryText, new Vector3(0, 1.75f, 0), Quaternion.identity);

        GameObject newCanvas = Instantiate(canvas) as GameObject;
        GameObject newButton = Instantiate(button) as GameObject;
        newButton.transform.SetParent(newCanvas.transform, false);

        Button NewButton = newButton.GetComponent<Button>();
        NewButton.onClick.AddListener(ReturnScene);

        battleUI.gameObject.SetActive(false);
    }

    //
    // Return to the previous screen
    // Note: Currently only supports going back to the "Hometown" scene
    //
    private void ReturnScene () {
        SceneManager.LoadScene("Hometown");
    }
}

Answer: You can either create a new Don't Destroy On Load object that will hold that information for you, or just keep those data in static field. Static fields are not destroyed while changing scenes.
I guess you could also create a ScriptableObject for keeping this data, but this is not the best solution in this case in my opinion.
Answer: A simple solution is to create a static method and private static fields in your BattleController.
public class BattleController : MonoBehaviour {
    private static string enemyName;
    private static int enemyID;

    public static void SetEnemy(string name, int id)
    {
        enemyName = name;
        enemyID = id;
    }
    //other code
}

Then call the method before loading the scene.
public class Monster : Character {
    void OnCollisionEnter2D(Collision2D other)
    {
        if (other.collider.tag == "Player") {
            BattleController.SetEnemy(enemyName, enemyID)
            SceneManager.LoadScene("Battle");
        }
    }
}

In your example the monster's name and ID are set as private fields in the GrassSlug class, but for this solution you'd need to add them to the Monster class as protected fields, then set them in GrassSlug.
If you need to share more information with the Battle Controller, you might want to just pass the Monster implementation to the Battle Controller instead of just its name and ID.


Get all users associated with a custom post type in WordPress

Question: I have a custom post type institutions that are associate users.
Whenever I add a new user, I can select which institution he/she is associated with via Advanced Custom Fields plugin.
Now that the set up is done, I am trying to display a list of all users associated with the institution I am currently viewing (single-institutions.php)
I cannot use WP_Query() because it won't allow me to get the users and I can't use wp_list_authors() because it won't allow the meta_query.
So I am left with using the old fashioned mysql query.
But I'm not sure how to proceed because I need to query multiple tables in order to get the proper result.
wp_users: ID
wp_usermeta
    user_id
    meta_key: institution
    meta_value: a:1:{i:0;s:4:"2875";} // "2875" is the ID of the institution the user is associated with

How can I achieve this with a direct mysql query?
EDIT: I've switched to WP_User_Query() instead, I had no idea it existed. However, I am unsure what to put in meta_value
$args = array(
  'role'           => 'Author',
  'order'          => 'ASC',
  'orderby'        => 'display_name',
  'who'            => 'authors',
  'exclude'        => [1, 7],
  'count_total'    => true,
  'meta_key'       => 'institution',
  'meta_value'     => $post->ID
);

// The User Query
$user_query = new WP_User_Query( $args );

// The User Loop
if ( ! empty( $user_query->results ) ) {
  foreach ( $user_query->results as $user ) {
    echo $user->display_name . '
';
  }
} else {
  // no users found
}

Right now, nothing gets displayed. If I remove "meta_value" from the query, all users associated with any institution appear on all pages.
How do I narrow it down to match the institution ID ($post->ID) I am currently viewing?
This is the value of meta_value column in wp_usermeta table. It's serialized. Do I need to do anything special to retrieve the ID at the end?
a:1:{i:0;s:4:"2875";}

Answer: You should use WP_User_Query which is similar to WP_Query. More specifically, in your case, look at the Custom Field Parameters and send it the ID of the institution.
Comment: @Halnex Ah, correct, I missed the fact that this was a serialized array. Be aware of a couple of caveats here:

1) Doing a meta query with a `LIKE` operator can be expensive if you have a lot of data and may impact site performance. It's usually fine, but if you experience problems, this may be something to look at.

2) You should include double quotes in your `meta_value`, for example `'meta_value' => '"' . $post->ID . '"'`. This way you'll avoid selecting values like `"19"` when you meant to select `"9"`, etc.
Comment: Thank you, that was really helpful. Had no idea `WP_User_Query()` exists. However, I am unsure what to use for `meta_value` - I've updated my post if you have a look please.
Comment: It worked. I had to use `LIKE` instead of `=` for `meta_compare`. Thank you.


Is there a way to get SSL certificate details using JavaScript?

Question: I'd like to gather certain details of an SSL certificate on a particular web-site. I know this is straightforward using the openssl tool on Linux/MacOSX. However is the same or similar possible in JavaScript?
I understand that the browser handles socket connections and that the SSL handshake occurs prior to any party sending data. However in an XMLHTTPRequest, I'd like to know if its possible to get these details as some sort of response code etc?
Comment: Some people add cert details to response headers. In that setup you could make an xhr request and read the  req.getAllResponseHeaders()
Comment: I don't think this is possible.
Comment: I've been looking for a Javascript API to test the proven credentials of cross-site requests to improve site security.  Sadly the browser is discarding this information, which is then forcing scripts to blindly trust that 3rd party content is trusted now because it was trusted in the past.  That's a sad state of affairs.  :-(
Comment: That's better than nothing, but doesn't provide any actual security.  I'm mostly concerned with pulling data from advertisers. When I author monetizing ad code for a client, I don't want to have to come back and check that the advertiser is still in business every few months to prevent their site from being hijacked because I'm pulling script resources from a site no longer owned by the advertiser, and I don't want to tell the client "This is safe now, but you're on your own now that I'm done.  Good luck!"
Answer: This information simply isn't exposed to javascript, it is so rarely used (well never since it isn't available, but it would be rarely used) that it wasn't deemed important enough to add to the javascript object model I suppose...same for any very rarely used feature left out of anything.
Of course, it could have also been left out for security reasons...I'm not creative enough to come up with one at the moment, but I'm sure there's an exploit to be had there as well.
Comment: @GregS - I can't think of one either...but I've said that 100 times before and someone will come up with a vulnerability I would *never* have consireded, different mindset I suppose...so I was just throwing that option out there.  If you were hosting the javascript in question...wouldn't you be the one with the certificate already? That's what leads me to think there might be more some nefarious use *somehow* for this.  As I said though, definitely not my area of expertise, I'll leave it to you and others who specialize in this area elaborate on what may be possible.
Comment: It would improve the security of my pages if I could use Javascript to test that a 3rd party site I am embedding content from is still the same entity that I trusted when I authored the page.  Currently the browser goes to the effort of proving that information, then discards it, forcing my pages to assume that a trust relationship still exists today because there was one in the past.  I almost -1'd your post for spouting blind FUD.  There's no exploits to be had from knowing who you are talking to. Please rethink your answer.
Comment: @JamesKPolk in fact it is a security problem, as websites can't detect if SSL has been broken and a fake cert being used.
Comment: Thanks Nick, I guess I'll have to think about how to grab SSL details on a client's end in a cross platform way, without JS or installing alternative binaries (such as openssl, curl/wget).
Comment: @aaa90210: It's nice that you have replied 8.5 years later, but certificates are public values and are not hard to get by any of a myriad of means. The fact that the javascript engine of a browser cannot easily do so is not a security problem.
Comment: @JamesKPolk yes but if you can't check the current webpage's cert against the cert you think it *should* be you have a problem.
Comment: @aaa90210: Incorrect. That is what the browser does, not your javascript.
Answer: The certificate isn't part of the DOM, so no, this won't be possible. Sorry!
Comment: Yes, sad but true.  So we need some concerned stakeholders to actually make some noise about this to W3C, Mozilla, Chromium, (and Safari and Edge will follow suit once there's CVE's against their platforms for failing to follow.)
Answer: The current JS language standard does not expose certificate information; beyond that It probably depends on how you're using JavaScript, if you're expecting the end user's Browser to expose certificate information then it's going to be really problematic because you'd need to get at the minimum FF, Chrome, Safari, IE, Edge, ... to expose it.
However, as mentioned on this Information Security post, this is not really a desirable option for these browsers to implement, as it would allow a situation where a website developer could write code to mistakenly trust user side credentials.
It's not so much a visibility security risk that prevents javascript from accessing the browser's current SSL Certificate info, but more of a fourth wall barrier security risk where the JS developer must be aware that the "user-accepted" certificate is not necessarily the one that the website provided. The HTML page really shouldn't be handling the security issues with client side code, but instead it should be able to depend on the security layer to do it's job properly. (I can totally understand wanting to checkup on the security layer, but any managerial work you do at the top layer is just going to be either superficial or a reworking of the entire biosphere)
Because let's assume for a moment that javascript did provide a way to work with the certificate, then when Bob already trusts Mallory because his security is broken there is no way of stopping the following exchange:
Office Worker Bob is on one side of the great firewall of Mega Corp., IT Mallory is in charge of the firewall passing traffic in and out of the company locally, and Web Host Alice's awesome website is out on the WWW.

By Mega Corp. company policy Bob is setup to just accept what Mallory has to say at face value.
Bob who would like to visit Alice's site, but has no direct outside access, tries to establish a secure connection through the firewall by holding up his certificate(Eg:"I hereby declare I am Bob") and asks Alice in a really convoluted way, "what certificate did I send to you?"
Mallory gets Bob's request, but instead passes on her own(Eg:"Uh, Bob says it's ok for me to read his webmail"), and even though Mallory doesn't understand Bob's convoluted question she still repeats it to Alice, "akdvyfenwythnwerhy?".
Alice does some math and figures out that "akdvyfenwythnwerhy?" is asking "what certificate did I send you?" and answers back to Mallory with what she sees("Hi Bob this is Alice you said: Uh, Bob says it's ok for me to read his webmail").
Mallory does some math, has an ah ha moment "akdvyfenwythnwerhy?=what certificate did I send to you?", and answers Bob's question on behalf of Alice("Hi Bob this is Alice(Mallory) you said: I hereby declare I am Bob").
Bob believes life is good and continues on to read his webmail, because by company policy he knows Mallory would never lie to him.
Mallory now able to read both sides of the conversation passes on Bob's request to read his webmail to Alice.
Alice gets Bob's request and says hey wait a minute Bob I need you to run this JS code to prove you know you're talking to Alice.
Mallory gets the code, runs it, and sends the results stating that she knows she's talking to Alice back to Alice.
Alice says, good enough for me here's your webmail.
Mallory reads Bob's webmail before passing it on to Bob, and everyone is blissfully happy.

(Note: I did not address the case where you're running JS server-side, then it would depend on what program you're using to run your JS code.)

Edit 4/4/2018 -- While the above is not wrong it's more from the perspective of embedded and linked JS than it is about the `XMLHTTPRequest` JS object; moreover quite possibly the strongest argument to be made against sharing PKI details with `XMLHTTPRequest` is as follows:
There needs to remain a strong dividing line between the HTTP portion and the S portion of the HTTPS protocol. JavaScript and it's XMLHTTPRequest object reside on the HTTP(app layer) side of that line, while the whole certificate exchange process resides on the S(trans/sec layer) side of that line. In order to keep the security side atomic(hot-swappable) its internal workings cannot be exposed across the line to the application side; because there may come a day when the transport/security layer no longer uses PKI certificates to facilitate its secure communication service, and when that day comes no one would need to rewrite any existing JS code that was relying on details contained within those certificates to deal with the propagation wave caused by the www community slowly adopting their favorite flavor of any new security layer.
That being said, the security side does appear to also be doing legal entity vetting --at least in some cases like EV certificates--, and it is IMO a short coming of RFC7230 section 2.7.2 that it does not redefine the authority of the https-URI to include an optional legalentity that the security layer would use when verifying the url it is communicating with is not only the proper end point but also currently under control of the intended business relation.
authority     = [ userinfo "@" ] host [ "#" legalentity ] [ ":" port ]
legalentity   = *( unreserved / pct-encoded / sub-delims )

Comment: Your answer doesn't address the question.  The question was whether Javascript could access the data in the Certificate which has already been proven by the browser's transportation layer.   Unfortunately the browser discards it.  Nothing in his question implied implementing the transport in Javascript, or bypassing the transport security.   -1
Comment: @Wil, in looking at your comments spread out across the thread I believe you're trying to use JS when you should be looking at a PHP or ASP solution. JS is client side and if your trying to check certificate info on a user's machine that's a little too late in the game.
Comment: I think you're discounting the fact that real applications are being written to run in the browser - applications that should be able to do a simple thing like prove that they are talking to the peer they intended to talk to.  PHP doesn't run in the browser.  ASP doesn't run in the browser.  The future is not server-side applications.  Preventing an application from proving the identity of a peer reduces the security of the transport from SSL-EV to SSL-broken cert.
Comment: @Wil, writing the "proving" function in JS and then handing that function over to a hacker and asking them nicely to be sure and use its code unmolested for validation is a bit optimistic.
Comment: Moreover, preventing the client JS app from believing its self-justified proof of validity on a compromised token stops it from making a very big mistake; the JS app layer gets its data including the app's own code from the security layer below, so when the security token is compromised it is fairly safe to assume likewise for the JS code as well and there is no magical proof of validity which when swapped out with `function check_valid(){return true;}` by an intermediary party is not going to always just rubber-stamp the broken token as shown in the overly simplified real-world example above.
Comment: When I authored the code I had a trust relationship with the organization "Example.Com Ltd".  I'd like to verify the Organization that responded has not changed, ie. response.certificate.peer.o=="Example.Com Ltd".
  Whether my script was loaded in a secure context is a red herring.  The browser itself could be loaded in an insecure context.  Nobody (other than you) suggested that SSL/TLS proofs be conducted in Javascript, and I've already disabused you of that ridiculous notion. The browser itself already verified the DOMAIN and THE PEER. I want to verify the PEER.
Comment: @Wil, It only feels like a red haring because your specific case use is way off topic. The security layer has two responsibilities first to make sure no one is listening-in and/or changing things around, and second to make sure the client is not talking to an imposter.
Comment: In your case `response.certificate.peer.o=="Example.Com Ltd"` would not work because: `response.certificate.peer.o` would be for the current document and should have `MyOwnSite.com LLC`, an intermediary could swap `if(response.certificate.peer.o=="Example.Com Ltd"){DoCode();}` with `if(true){DoCode();}`, likewise an intermediary could just author their own certificate with the o field set to the value the code is looking for ie `"Example.Com Ltd"`...
Comment: I believe what you really want to do in your situation is to use PHP/ASP to forward the 3rd party JS application. Your JS app would use AJAX to request and load the 3rd party code from a PHP/ASP form on your domain. The PHP form then: requests the code from the 3rd party, does the validation, if the validation passes, save a copy on your server and forward it on to the client, but if the validation fails then the PHP form can pass on the last good copy of the 3rd party code to your app running client side.
Comment: This would rely on the security layer to take any intermediary between your domain and the client out of the picture, and by manually storing the 3rd party's credentials on your domain would rely on your PHP to take any intermediary between your domain and the 3rd party's out of the picture as well (which if properly coded would be mathematically provable).
Comment: As for your suggestion of why it wouldn't work, that is just an example of your ignorance of XHR and Javascript programming,  Every XHR is a response object, and I would be testing the response object from the 3rd party peer.  You've now proven that your answer is off-topic because you don't even understand the topic.
Comment: Thankyou for recognising a need to verify the peer.   Now recognise that having to call back to my app's server is a hack and doesn't solve the problem. If the client is loaded on a server with a compromised DNS server and it is being directed to an entirely different host than my app's server, then my app's server results are pointless. There's zero reason to hide the browser-verified certificate details of the request to Javascript via the request object.  Other than the paranoid rantings of a few people who have no clue what they're talking about.
Comment: And while I'm at it the whole "4th wall" argument is FUD.  Currently apps are forced to trust that the user did NOT accept an invalid cert from the 3rd party site.  Letting the script verify peer-identifying parts of the cert, as a 2nd line of defense, does nothing whatsoever to reduce the security of the app or the user.  It does let the app refuse to operate if the user does something stupid, or if there's a MITM or Spoof with a successfully forged cert (there's been tens of thousands of these in the last couple years.)  Your example also demonstrates your misunderstanding of how PKI works.
Comment: 1) This has nothing to do with DNS, but valid https would take care of the redirect problem, 2) I'm sorry you fail to see the hole in your security model, but PKI works because there is a trusted certificate store on the clients' computer. When that store is not breached, then a 2nd layer is superfluous; and when it is breached you cannot pass script code across that breach to patch it. Because that would entail re-implementing a new layer of the certificate store within a compromised environment.
Comment: 3) Superfluous or not, against just about every recommendation I've seen, people like yourself are still doing XSS; so there should probably be a mechanism built into XHR to for the host to 'approve' and/or delegate the 'approval' of the 3rd party resources. Which it looks like there may be some progress on https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
Comment: The problem being CORS only allows the library host to approve the usage of its resources by the franchise host, it is still missing the bit where the franchise approves the code checked out by the client from the library prior to the client running it. Until that gets implemented you're stuck with your host checking out the code approving it and relaying on to the client. Where how you checkout and approve it is up to how you wish implement it server-side, but at least then the JS dev team hasn't exposed the rest of us to the false hope of verifying faulty credentials using compromised code.
Comment: Also, as a side note you're going to want to fix your validator script to walk back up the certificate chain until it gets to the EV certificate before it relies on the legal entity presented by `.certificate.peer.o`, as that's the only one which has gone through the vetting process.
Comment: 1) the client can accept a bad MITM cert when DNS is hijacked.  Firefox and Chrome for example offer this to the client user.  At that point yes all bets are off, but at least we can make it harder for the MITM to succeed by rejecting data that isn't forged well.  Some effort is better than no effort.
Comment: 3) Yes, it's XSS.  Just about all 3rd party ad apps use XSS, which is inherently insecure because we're handing off control of the browser scripting engine to a 3rd party that we trusted IN THE PAST but have no idea whether we should still trust them now.  Hence the need to ensure that they are still the entity that we trusted when the code was written.  Testing server-side has the potential, I agree, but it won't deal with the MITM and it requires extra requests, which elevates the service cost.  Doing it client side would cost literally nothing and be more effective.
Comment: CORS is off-topic. It has nothing to do with this whatsoever.  It's a relaxation of XSS security constraints, not a proof of peer identity.  If CORS let you specify that XYZ.COM was a valid 3rd party script source only when its EV organization is also "same trusted organization", then yes, it would meet the requirements.  But that isn't part of the spec.  CORS does let you use O_AUTH, cookies, etc to validate the 3rd party.  Again that implies more overhead, this time requiring dynamic content from what would otherwise be a static CDN.
Comment: Again I'd like to reiterate that this isn't about distrusting the security of the transport layer, though it isn't entirely trustworthy as previously mentioned.  It's about enforcing an organizational relationship requirement that PKI does enforce, and which the browser presents to the client user via the green bar.  However the script layer then proceeds as the user's agent on their behalf under the guise of the trust relationship demonstrated by that green bar, meanwhile the browser prevents the script layer from behaving in a trustworthy manner - by preventing required trust proofs.
Comment: Your assertion that the script must blindly trust the transport layer based on Cert validity equates to Bob blindly trusting that he has Sharon's phone number because he's spoken to her at that number once in the past. Then Sharon is replaced with Shirley who doesn't have access rights, Bob calls and blindly discloses vitally secure data to Shirley, because he doesn't perform a simple "Hello, is this Sharon?"  handshake after dialling.   But I appreciate that you have edited your text to reflect your growing understanding of these concerns.  Hope now you understand even more.
Comment: that's why I suggested that adding "Sharon" to the URL or in your case phone number would allow the security layer to green bar the user or 'page contains non-secure content' the JS with the option to reject the call when the phone company reassigns the phone number to Shirley. However it still leaves you in a lurch since that wouldn't really let the JS stop the unknown adware from going through if the user blanket ok'd it; at that point `XMLHTTPRequest` would still need to expose a security validation status back to the JS layer. possibly: `{NON_SECURE, VALID_ENDPONT, VALID_EV}`
Comment: although with a targeted set of certificates, adding the security status may cause `XMLHTTPRequest` to leak who's in the end user's circle of trust. because then a malicious site controller could write JS to have `XMLHTTPRequest` ping `https://badguy.com/mixed-content-of-targeted-certificates?cert1_accepted=yes&cert2_accepted=no&cert3_accepted=yes`
Comment: Much ado over nothing post. OP is asking how to get >>server cert<<, nothing to do with trusting the client. If you're going to say "no, just blindly trust the browser and not let clients be able to verify server signature on some data msgs", at least give a suitable reply.
Comment: @user3635998 I'm saying JS has no better choice then to "blindly" trust the browser to do the security layer. Because JS is not a program that the user bought at a store brought home and installed on their PC, it was just a bit of text sent across that very same security layer. You can't trust it to scan itself after it arrives unless you already blindly trusted the browser security to deliver it safely; that's why for the same reason anti-virus software is compiled in a clean room.
Comment: Like i said, much ado over nothing. In the future, 1000000 years in the future may not use cert, blah blah blah herpity derp. There is no excuse not to let the JS have access to the server cert, as seen from the browser. Having a window.getCert() is bad, because the user didn't buy such a function. Just like the user didn't buy the browser. mmmmkay, right. pffft.
Comment: BTW, should the site no longer use a cert, and the JS relies on having a cert, there'll be this magical thing called an error.
Comment: @user3635998 yes [1000000 years](https://www.qiskit.org/) in the future your site still serves up some archaic text file with `window.getCert()` ... and then what? oh yeah the client's pc interprets it as "code" and runs `window.getCert=function(){return function(){return 'valid cert'}}();` because it was delivered "securely" to the browser. But hey, if you have any links on which cutting edge browsers implement getCert() because it's [such a novel idea](https://bugzilla.mozilla.org/show_bug.cgi?id=728650) that nobody has thought of before then please do share.
Comment: Sarcasm is lost on the ____. People like yourself are totally incapable of explaining why letting the JS know the server cert is a bad thing.
Answer: Nope, not possible.
It is possible to detect via javascript whether the current page being viewed is over an SSL connection (document.location.protocol=="https:"), but that's about it.
Comment: and then you can pass that `document.location` value to a REST service to get the certificate information.
Comment: @Saber the "man in the middle" type of attack would still be a concern... I want to check the certificate public key signature, to make sure the browser talks directly to _my_ server.

But yeah, interesting idea... worth exploring...


Probability of obtaining at least 1 face card from 3 draws with a given formula

Question: I'm being given the following formula: () =  − (~)
I'm being given an example to use the formula, which is the following: "Probability of obtaining at least 1 face card (so $12$ out of $52$) by drawing $3$ cards out of a $52$-card deck." And I'm instructed to use the above formula as it is easier to calculate the probability of obtaining 0 face cards ((~)) and then using the above formula instead of taking into account the case "$1$ face card", "$2$ face cards" and "$3$ face cards" because of the "AT LEAST" in the example given.
I don't have any way to confirm if I applied the formula correctly because I don't know how to apply the "drawing 3 cards" part, so please tell me what I did wrong/right:
Probability of drawing $0$ face cards:
$\frac{40}{52}$ because there's $12$ face cards in a deck
By the formula, () =  − (~)
$$P(A) = 1 - \frac{40}{52} $$
$$P(A) = 0.23076923076$$
Conclusion would be: The probability of getting at least $1$ face card by drawing $3$ cards out of a $52$-card deck is $0.23076923076$.
This doesn't seem correct to me.
Answer: You computed $P(\text{~}A)$ incorrectly.
The first card drawn has to be a non-face card with probability $\frac{40}{52}$
Then that card is eliminated so the next card has to be a non-face card with probability $\frac{39}{51}$
Then that card is eliminated so the final card has to be a non-face card with probability $\frac{38}{50}$
All together,
$$P(A)=1-\left(\frac{40}{52}\cdot\frac{39}{51}\cdot\frac{38}{50}\right)\approx0.5529$$


Identity on hyperplane and its complement, then identity on whole space? (Humphreys p.42 Lemma)

Question: This is from Hymphreys 'Introduction to Lie Algebras and Representation Theory', Lemma 9.1 in page 42.

Lemma. Let $\Phi$ be a finite set which spans $E$. Suppose all reflections $\sigma_\alpha$($\alpha \in \Phi$) leave $\Phi$ invariant. If $\sigma \in GL(E)$ leaves $\Phi$ invariant, fixes pointwise a hyperplane $P$ of $E$, and sends some nonzero $\alpha \in \Phi$ to its negative, then $\sigma=\sigma_\alpha$ (and $P=P_\alpha$).

$GL(E)$ is the set of all invertible endomorphism on $E$. In the proof of this Lemma, it says that

Let $\tau=\sigma\sigma_\alpha$. Then $\tau$ acts as the identity on $\mathbb R \alpha$ as well as on the quotient $E/\mathbb R\alpha$.

Next they continue the argument using eigenvalue and minimal polynomial. I understand why it acts as the identity. But I haver a question in the next step.
Since $P$ is a hyperplane and $\mathbb R \alpha$ is its complement, $E=P\oplus\mathbb R\alpha$. Since $\tau$ is the identity on both $\mathbb R \alpha$ and $E/\mathbb R \alpha\cong P$, isn't $\tau$ the identity on whole $E$? Why should we go further using the argument about minimal polynomial?
Any help would be appreciated. Thanks.
Comment: Related: https://math.stackexchange.com/q/4178348/96384.
Comment: Also related now: https://math.stackexchange.com/q/4227594/96384. This lemma and its proof seem to confuse many people!
Comment: @TorstenSchoeneberg Thanks a lot.
Comment: @TorstenSchoeneberg You're so kind. Thank you very much!
Answer: You seem to be thinking that if one has a vector space $V$ with a subspace $W$, and an endomorphism $f$ which is the identity on $W$ and induces the identity on $V/W$, then $f$ must be the identity on all of $V$.
That is not true: Take the vector space $V=\mathbb R^2$ with basis $e_1, e_2$ and consider the linear endomorphism given (in that basis) by $$\pmatrix{1 &1\\0&1}.$$ You will notice it is the identity on $\mathbb R e_1$ (in particular, leaves that subspace invariant), it also induces the identity on $V/\mathbb Re_1$, but it is not the identity on $V$.
In fancier terms, even though one can identify the vector space $V/W$ with any complement of $W$ in $V$, such an identification in general is not compatible with the action of $f$, and so we do not have $V\simeq W \oplus V/W$ as $K[f]$-modules.
In the case at hand, the result is true in hindsight, so of course one cannot come up with a counterexample, but one cannot just invoke a wrong general argument as above. In fact whatever argument works here would equivalently show that there is a hyperplane (complement of $\mathbb R\alpha$) which is invariant under the endomorphism in question.


Markov Chain: Pensioner Problem

Question: A pensioner receives 2000 dollars at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i (i.e. i thousand dollars) with probability Pi, i = 1, 2, 3, 4, $\displaystyle\sum\limits_{i=1}^4P_{i}=1$. If the pensioner has more than 3000 dollars at the end of a month, he gives the amount greater than 3000 to his son.
Q1.If, after receiving his payment at the beginning of a month, the pensioner has a capital of 5000, what is the probability that his capital is ever 1000 or less at any time within the following four months?
Comment: What is the probability that the pensioner will have $\$1000$ at the end of the month?  $\$2000$? etc?  What is the probability that he will have $\$5000$ after receiving his _second_ month's pension?
Answer: Call $X_n$ the number of thousands of dollars the pensioner has at the end of the $n$th month just before he receives his payment. Modify the dynamics by considering that if the pensioner has or has had $0$ dollar or $1000$ dollars at time $n$ or anytime before time $n$, then $X_n=1$.
The process $(X_n)$ is a Markov chain on the state space $S=\{1,2,3\}$ starting from $X_0=3$ and one asks for the probability $p$ that $X_4=1$. This is $Q^4(3,1)$, where $Q$ is the transition matrix of $(X_n)$ indexed by $S\times S$. Thus
$$
Q=\begin{pmatrix}1&0&0\\ 1/2&1/4&1/4\\ 1/4&1/4&1/2\end{pmatrix}.
$$
From here, the quickest route might be to (somewhat tediously) compute  the first column and the third line of $Q^2$, namely,
$$
Q^2=\begin{pmatrix}1&*&*\\ 11/16&*&*\\ 1/2&3/16&5/16\end{pmatrix},
$$
and to deduce the coefficient
$$
Q^4(3,1)=\sum_{i=1}^3Q^2(3,i)\times Q^2(i,1)=\frac12\times1+\frac3{16}\times\frac{11}{16}+\frac5{16}\times\frac12=\ldots
$$
Answer: Here is a partial answer.
At the end of the month, after giving money to his son if possible
under the rules, the pensioner will have $\$3000, \$2000, \$1000$
left with probabilities $P_1 + P_2, P_3$, and $P_4$ respectively.
Thus, after receiving his pension at the start of the second month,
the pensioner will have $\$5000$ again with probability $P_1 + P_2$
and will repeat this cycle all over again (or will have lesser amounts
and go off on a separate path, possibly towards destitution).
Thus, the probability that the pensioner will have $\$1000$ or less
at some time during the next four months is bounded below as follows:
$$P(\$1000~\text{or less during next four months}) >
P_4[1 + (P_1+P_2) + (P_1+P_2)^2 + (P_1+P_2)^3]$$
You need to figure out what happens if the pensioner
begins the second month with only $\$4000$ to find the
value of $P(\$1000~\text{or less during next four months})$.

Markov Chain: Pensioner Problem


Problems sending parameters from Angular to Node

Question: I'm developing an easy website using Angular and nodeJS.
I'm trying to send one parameter "email" from an Angular service when accessing a route defined in node server.
The Angular service function:

The node defined accessing route:

The configuration of the node server:

I tried using bodyParser and using req.body for getting the email parameter sent from the frontend, but it seems to be lost.
It's getting an empty dictionary when req.params or req.body are printed.

Someone knows what can be happening?
Thanks for reading!
Answer: From the looks of it you want to read the path-param from the request url (see route-params under https://expressjs.com/en/guide/routing.html).
On the nodejs side, you need to change your handler to:
router.get('/profile/:email', ...) => {
   const email = req.params.email;
});

On the client-side, you need to correctly add the email path-segment. One way to do this is:
this.http.get<any>(`${this.URL}/profile/` + encodeURIComponent(email));

Instead of path-parameters you can also do this is using query-parameters. You'd need the following changes:
nodejs:
router.get('/profile', ...) => {
   const email = req.query.email;
});

angular:
this.http.get<any>(`${this.URL}/profile`, { params: { email } });

Comment: Thanks, this helped to understand routing parameters. @eol
Comment: Thanks, this worked! It's not any other way for sending it without been the parameter setted in the url? @eol
Comment: Not if you want it as a path-parameter. If you change it to query to can pass a query-object. I've updated my answer :)