Avoiding DNS lookup issues with a cache

Question: I'm running fairly simple PHP websites, on Ubuntu, with AWS EC2 instances, which use *.rds.amazonaws.com domain names to connect to their respective RDS instances.
This introduces 2 issues:

When the DNS cache expires, every 5 seconds, the next request adds a minimum of 12ms to my websites processing time, which isn't ideal (I try to keep within a 100ms budget).
Occasionally this resolution step fails, so rather than showing an error page to my customers, my script will sleep for ~500ms, and try again.

I'm wondering if there is something, like a local caching DNS resolver, that could handle these issues more gracefully?
Perhaps it could cache the IP address for 3 seconds, then automatically try to refresh that cache, knowing that it has 2 seconds left. And if it does continue to fail, still provide the old/stale response, as that's better than no response.
Or, it could offer up the cached response immediately, even if it has expired (it's probably still correct); and if did just provide an expired response, run an update, just incase the RDS instance has just moved.
Or, could I bodge it, with a script that checks the DNS response in a loop (dig +short), and when the IP address changes, update the /etc/hosts file, using a hostname such as database-abc.

As an aside, my sites tend to get about ~4 requests every 5 seconds during the day (so cache updates are fairly frequent), but there is low activity overnight.
Comment: `When the DNS cache expires, every 5 seconds` Why would anyone do that? That should be a DNS worst practice. A short TTL like that is begging for an outage.
Comment: Hi Greg, the 5 second TTL is set by Amazon, for their RDS service. The idea is that, if one of the database servers was to fail, it can switch over to a different server by changing the DNS entry, and that would be updated in less than 5 seconds.
Comment: Not surprised. Doesn't matter, it's still a worst practice.
Answer: As per AWS Route 53 Documentation:

Amazon Route 53 effectively connects user requests to infrastructure running in AWS – such as Amazon EC2 instances, Elastic Load Balancing load balancers, or Amazon S3 buckets – and can also be used to route users to infrastructure outside of AWS. You can use Amazon Route 53 to configure DNS health checks to route traffic to healthy endpoints or to independently monitor the health of your application and its endpoints.

For your specific scenario, here is an article
Comment: I don't think load balancing helps... if you're talking about EC2 instances, it's still a single server that's handing the request, and if it's DNS lookup for the Database server fails, that still effects that request... and if you're talking about the DNS server end, that's more Amazons side of things (part of RDS), and that's on the wrong end of the network connection (so still slow, and prone to errors).
Comment: EC2 is effectively using Route 53, it's just that any network request adds a delay (12ms or more in my case), and a chance of failure (usually happens once every couple of days)... so I'm looking at what I can do on the server to avoid those issues, probably caching in some way.
Comment: You should consider raising your TTL in the zone files. Any reason why they are set so low? Is your DNS changing often?
Comment: This guy - https://www.youtube.com/watch?v=4ZtFk2dtqv0 - Nill, while he is kinda far out there, explains it very well...
Comment: It's set my Amazon, and I can't find any way of changing it... even so, it has to be fairly low incase one of the servers was to fail (it happens quite often due to a bug they have at the moment), and whatever it's set to, eventually a lookup needs to be done, and that will always add a delay, and a chance of failure.
Comment: Please [refer to the documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_BestPractices.html)
Comment: Here is a specific article showing how to configure Route53 for your scenario - [Learn More](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-rds-db.html)
Comment: Yeah, DNS is fun, I've been managing DNS servers for years (BIND, dnsmasq, etc) but I've never had to deal with trying to avoid these issues on the client side... I'm used to TTL's that are much longer, and with systems that are more tolerant of delay/failure.
Comment: I'm sorry, but Route53 isn't the solution, my EC2 instance (the server running PHP) currently needs to resolve a domain name, and the network request to do that is fairly slow, and occasionally fails (no matter what is at the end of that network request)... so I'm looking for something that runs on the local machine to avoid that DNS lookup at the point the IP address is needed (to avoid delays and failures).
Comment: Have you tried any sort of load balancing?

Avoiding DNS lookup issues with a cache


Validação de dados de entrada com uso de funções , try e except

Question: try:
    number = int(input("Por favor, digite o seu número aqui e veja o que acontece: "))
except (ValueError, TypeError):
    print ("Por favor, insira somente números inteiros positivos")

try:
    collatz(number)
except (NameError):
    print ("Não é possível executar a operação sem uma entrada válida.")

Preciso incluir nesse código validação usando try e excep, se caso o usuário digitar string (a,b,c...) ou caractere especial, ele mostre erro e solicite novamente o inteiro.
Comment: A minha pergunta é específica, não tem duplicidade. Com a minha pergunta, obtive a resposta que não obtive antes de faze-la.
Answer: while True:
    try:
        number = int(input("Por favor, digite o seu número aqui e veja o que acontece: "))
    except:
        print ("Por favor, insira somente números inteiros positivos")

Comment: Funcionou, obrigado!
Answer: Código:
while True:
    try:
        number = int(input("Por favor, digite o seu número aqui e veja o que acontece: "))
    except:
        print ("Por favor, insira somente números inteiros positivos")

OBS: se for apenas com números inteiros é int, se for com números com ponto flutuante (decimais) é float.
Comment: Funcionou, obrigado!


Center lock icon indicator for ViewPager

Question: I'm making an app in which I need to have an horizontal scroll view with several Fragments and an indicator.
The indicator of the current item must be centered horizontally, the Fragments can be selected by swiping the Fragments themselves or by scrolling and snapping the indicators or by clicking directly on one indicator (see pictures below).
It seems that the standard way to do that is to use a ViewPager along with a PagerTabStrip, but I cannot find how I can have this center-lock behaviour with the PagerTabStrip.
Any suggestion?
Comment: PagerTabStrip doesn't have all the behaviors you want, you'll have to make something custom. I would look at the PagerSlidingTabStrip library and see if anything there gives you any insight on how to create yours. https://github.com/astuetz/PagerSlidingTabStrip
Comment: Most likely, you will need to create your own indicator, using `PagerTabStrip`, `PagerSlidingTabStrip`, and the indicators in the ViewPagerIndicator library as starting points.
Comment: Yes, it seems that I'll have to implement my own solution. I'll post it here as soon as I'm done.
Answer: I forked Andreas Stuetz's PagerSlidingTabStrip
to implement that PagerSlidingIndicator.


Do all voices deserve rests, or can I tactically omit rests?

Question: The voice 3 (magenta) plays for just two semiquavers at the end of the bar:

should I add a series of rests for that voice?

it's hard to understand which rests belong to which voice when rendered in black-and-white:

we could merge the voices (hard to see that only the E is tied):

Well, probably the pragmatic answer is to just relax the interpretation (this is a piano arrange of a multi-instrument song) and say that the notes don't overlap:

but nevertheless: if we were to try and notate the semiquavers + minim scenario: are all of the aforementioned representations legal? which would be preferred?
Comment: Related: [Unplayed 2nd voice in notation - spaces or rests?](https://music.stackexchange.com/questions/56445/unplayed-2nd-voice-in-notation-spaces-or-rests), [In keyboard style harmony should there be a rest for each voice?](https://music.stackexchange.com/questions/122328/in-keyboard-style-harmony-should-there-be-a-rest-for-each-voice)
Answer: Elaine Gould, in her standard reference to music notation, Behind Bars, indicates that "Parts on one stave share a rest where possible" (p. 312) Thus, the initial half rest should be shared between the magenta and green voices.
In the second half of the measure, there should be a quarter rest and an eighth rest in the magenta voice so that it's clear the green voice is separate. (Again according to Gould, a dotted quarter rest is not permitted. "The longest permitted dotted rest is one value smaller than the beat. In crotchet [X/4] metres, the longest dotted rest is a dotted quaver [eighth note]" [p. 162].)

It also would clean things up to move the "repeated A" voice to the upper staff. If desired, "handedness brackets" can indicate which As to play with which hand. (In MuseScore, these are found on the Master Palette in the Symbols list under Keyboard techniques.)
Comment: @phoog Elaine Gould's authority.
Comment: It's worth noting that Gould (on p. 313) also notes that:
"_In strict contrapuntal writing, each part takes separate stems and rests throughout._"
Comment: "permissible" under what authority?  (Furthermore, every authority I'm aware of forbids dotted rests except to represent integral beats in compound meter, so it's more than preferable to use two rests; it's required.)
Comment: Thanks.  It might be interesting to see what other standard references have to say on the matter (I have the sense that older works would have separate rests, but I don't know when the preference changed).  But I don't have any at hand.


Why does clang emit a 32-bit float ps instruction for the absolute value of a 64-bit double?

Question: Why is clang turning fabs(double) into vandps instead of vandpd (like GCC does)?

Example from Compiler Explorer:
#include <math.h>

double float_abs(double x) {
    return fabs(x);
}

clang 12.0.1 -std=gnu++11 -Wall -O3 -march=znver3
.LCPI0_0:
        .quad   0x7fffffffffffffff              # double NaN
        .quad   0x7fffffffffffffff              # double NaN
float_abs(double):                          # @float_abs(double)
        vandps  xmm0, xmm0, xmmword ptr [rip + .LCPI0_0]
        ret

gcc 11.2 -std=gnu++11 -Wall -O3 -march=znver3
float_abs(double):
        vandpd  xmm0, xmm0, XMMWORD PTR .LC0[rip]
        ret
.LC0:
        .long   -1
        .long   2147483647
        .long   0
        .long   0

(Ironically, GCC uses vandpd but defines the constant with 32-bit .long chunks (interestingly with the upper half zero), while clang uses vandps but defines the constant as two .quad halves.
Comment: Never mind, there is no `vandss` *facepalm*
Comment: My question is, why doesn't either of them use scalar?
Comment: The scalar instruction works only on x87.
Comment: They're exactly equivalent, right?  Bitwise AND is the same operation whether you think of it as operating on two 64-bit elements or on four 32-bit elements.  The instructions are the same length, too.  So probably just an arbitrary choice.
Comment: Unless there is somehow a bypass delay issue with one or the other?  I'm aware of that effect when mixing integer and floating-point vector instructions, i.e. https://stackoverflow.com/questions/26942952/difference-between-the-avx-instructions-vxorpd-and-vpxor?noredirect=1&lq=1, but not sure if it applies when mixing floating-point instructions with different element sizes.  Given that the type is `double`, we assume the value in `xmm0` is the result of a 64-bit load or FP arithmetic instruction, so if there's any difference `vandpd` is presumably better, but there may not be any.
Comment: @NateEldredge: AFAIK, no x86 CPUs have ever had bypass delays between ps and pd instructions for bitwise booleans, only [between actual FP math instructions on Bulldozer-family](https://stackoverflow.com/questions/62111946/what-is-the-point-of-sse2-instructions-such-as-orpd#comment110006402_62112042).  (The likely reason is the CPU keeping some FP normalization metadata associated with the vector for the shorter-latency FMA unit -> FMA unit special forwarding path.)  I thought I'd mentioned that on at least one of the multiple answers I've written about this, but it seems not :/
Comment: @NateEldredge: I did find a mention that no CPUs have bypass delays between float and double in my answer on [What is the point of SSE2 instructions such as orpd?](https://stackoverflow.com/q/62111946) though.  And that probably explains why clang likes to use `ps`: the non-VEX encoding is shorter, and maybe it simplifies the compiler logic to *always* use the `ps` version instead of only doing that for legacy-SSE encodings, not for AVX.  (The Bulldozer reformatting issue is only addps -> addpd or whatever, which is nonsensical, not with booleans between.)
Answer: TL:DR: Probably because it's easier for the optimizer / code-generator to always do this, instead of only with legacy-SSE instructions to save code-size.  There's no performance downside, and they're architecturally equivalent (i.e. no correctness difference.)

Probably clang always "normalizes" architecturally equivalent instructions to their ps version, because those have a shorter machine-code encoding for the legacy-SSE versions.
No existing x86 CPUs have any bypass delay latency for forwarding between ps and pd instructions1, so it's always safe to use [v]andps between [v]mulpd or [v]fmadd...pd instructions.
As What is the point of SSE2 instructions such as orpd? points out, instructions like movupd and andpd are completely useless wastes of space that only exist for decoder consistency: a 66 prefix in front of an SSE1 opcode always does the pd version of it.  It might have been smarter to save some of that coding space for other future extensions, but Intel didn't do that.
Or perhaps the motivation was the future possibility of a CPU that did have separate SIMD-double vs. SIMD-float domains, since it was early days for Intel's FP SIMD in general when SSE2 was being designed on paper.  These days we can say that's unlikely because FMA units take a lot of transistors, and can apparently be built to share some mantissa-multiplier hardware between one 53-bit mantissa per 64-bit element vs. two 23-bit mantissas per 2x 32-bit elements.
Having separate forwarding domains would probably only be useful if you also had separate execution units for float vs. double math, not sharing transistors, unless you had different input and output ports for different types but the same actual internals?  IDK enough about that level of CPU design detail.

There's no advantage to ps for the AVX VEX-encoded versions, but also no disadvantage, so it's probably simpler for LLVM's optimizer / code generator to just always do that instead of ever caring about trying to respect the source intrinsics.  (Clang / LLVM doesn't in general try to do that, e.g. it freely optimizes shuffle intrinsics into different shuffles.  Often this is good, but sometimes it de-optimizes carefully crafted intrinsics when it doesn't know a trick that the author of the intrinsics did.)
e.g. LLVM probably thinks in terms of "FP-domain 128-bit bitwise AND", and knows the instruction for that is andps / vandps.  There's no reason for clang to even know that vandpd exists, because there's no case where it would help to use it.

Footnote 1: Bulldozer hidden metadata and forwarding between math instructions:
AMD Bulldozer-family has a penalty for nonsensical things like mulps -> mulpd, for actual FP math instructions that actually care about the sign/exponent/mantissa components of an FP value (not booleans or shuffles).
It basically never makes sense to treat the concatenation of two IEEE binary32 FP values as a binary64, so this isn't a problem that needs to be worked around.  It's mostly just something that gives us insight into how the CPU internals might be designed.
In the Bulldozer-family section of Agner Fog's microarch guide, he explains that the bypass delay for forwarding between two math instructions that run on the FMA units is 1 cycle lower than if another instruction is in the way.  e.g. addps / orps / addps has worse latency than addps / addps / orps, assuming those three instructions form a dependency chain.
But for a crazy thing like addps / addpd / orps, you get extra latency.  But not for addps / orps / addpd.  (orps vs orpd never makes a difference here.  shufps would also be equivalent.)
The likely explanation is that BD kept extra stuff with vector elements to be reused in that special forwarding case, to maybe avoid some formatting / normalization work when forwarding FMA->FMA.  If it's in the wrong format, that optimistic approach has to recover and do the architecturally required thing, but again, that only happens if you actually treat the result of a float FMA/add/mul as doubles, or vice versa.
addps could forward to a shuffle like unpcklpd without delay, so it's not evidence of 3 separate bypass networks, or any justification for the use (or existence) of andpd / orpd.


How to count number of rows with unique values in each column given following query in SQL

Question: SELECT COLUMN_NAME
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'MYTABLE' AND OWNER = 'MYSCHEMA'

So right now this displays each column present in my table but I want to also display the number of rows in each of these columns.
How would I specifically go about doing this?
Comment: Does this answer your question? [SQL count rows in a table](https://stackoverflow.com/questions/28916917/sql-count-rows-in-a-table)
Comment: It would be as simple as to count the number of rows of the table, unless you need something else you haven't explained well. Please add some sample data and the expected result.
Answer: Number of rows is related to tables, not columns. So you'd
select count(*) from your_table;

to get that value.
Dynamically, a PL/SQL could do it as follows:
SQL> set serveroutput on
SQL> declare
  2    l_cnt number;
  3  begin
  4    for cur_r in (select table_name, column_name
  5                  from user_tab_columns
  6                  where table_name in ('EMP', 'DEPT')
  7                 )
  8    loop
  9      execute immediate 'select count(distinct ' || cur_r.column_name || ')' ||
 10                        ' from ' ||cur_r.table_name into l_cnt;
 11      dbms_output.put_line(cur_r.table_name ||'.'|| cur_r.column_name ||': ' || l_cnt);
 12    end loop;
 13  end;
 14  /
DEPT.DEPTNO: 4
DEPT.DNAME: 4
DEPT.LOC: 4
EMP.EMPNO: 14
EMP.ENAME: 14
EMP.JOB: 5
EMP.MGR: 6
EMP.HIREDATE: 13
EMP.SAL: 12
EMP.COMM: 4
EMP.DEPTNO: 3

PL/SQL procedure successfully completed.

SQL>

Comment: But I want to retrieve the distinct count per column. How would I go about doing that?
Comment: One option is to use dynamic SQL. I modified my answer; have a look, please.
Answer: You'll need to use some kind of dynamic SQL to compose the query shown below:
select
  count(distinct col1),
  count(distinct col2),
  count(distinct col3),
  ...
from my_table

Comment: I have an unknown amount of columns. This process needs to be repeated across multiple different tables with varying sizes
Comment: As I said, you need to *compose* a query using dynamic SQL.


Como fazer para o código olhar um if apenas depois de ter mudado a prop de um objeto?

Question: Eu tenho um formulário que quando clico em enviar deixa vermelho o campo que estiver vazio. Ele faz diferentes validações em cada um dos campos. Apenas quando todos os campos estiverem com borda cinza ele envia os dados via post e desabilita a mensagem de erro.
O problema
Dentro do método handleClick ele faz as alterações de estado caso o campo esteja corretamente preenchido e no fim faz uma verificação final em todos os campos. Se tudo estiver cinza envia os dados, se não mantém o erro. Mas na primeira vez que clico envia mesmo estando errado, e apenas na segunda vez que clico descobre o erro. Isso acontece porque está entrando no meu if antes de fazer as alterações nos campos usando setState. Tentei usar setTimeout mas não funcionou e me disseram não ser uma boa prática. Como proceder para o if ficar async e só ser ativado quando os states estiverem atualizados?
if ((valueColor && nameColor && lastNameColor && emailColor
      && cardColor && cvvColor && cpfColor
      && dateColor && periodicityColor) === 'grey') {
      console.log('dentro'); this.errorMessage.message = 'none'; //this.handleSubmit();
    } else {
      console.log('fora'); this.errorMessage.message = 'block';
    }

Comment: Agir com base no estilo e não no resultado da validação é má pratica, e acaba por se tornar evidente no seu caso. De que forma estão os campos a ser validados? Coloque também o código que valida na pergunta
Comment: Essa foi uma boa dica rapaz, vou analisar.
Answer: EDIÇÃO
Voce tinha pedido async daí eu fiquei confundido.
Nessa edição eu removi o component com metodo assincrono que não tinha a ver com a pergunta.

Agora vamos para o que interessa
//Cabeçalho dos exemplos abaixo
import React from "react";

function verifyEmail(email){
    let phone = document.querySelector(".phone");
    ...whatever
}
function verifyPhone()
{
    let email = document.querySelector(".email");
    ...whatever
}

Sem usar async e sem usar this.state
class Register extends React.Component
{
    componentWillMount() {
        this.fields = {email:false,phone:false};
    }
    handleClick = () => {
        this.fields.email = verifyEmail();
        this.fields.phone = verifyPhone();

        if (this.fields.email === true && this.fields.phone === true) {
            ...Do something
        } else {
            ...Error
        }
    }
    render() {
        return (
            <input onClick={this.handleClick}>
        );
    }
}

Usando callbacks na função this.setState([Object, Callback])
class Register extends React.Component
{
    componentWillMount() {
        this.setState({
            email: false,
            phone: false //false é que não está correto
        });
    }
    handleClick = () => {
        let email = verifyEmail();
        let phone = verifyPhone();

        this.setState({
            email: email, //resultado da função verifyEmail()
            phone: phone //resultado da função verifyPhone()
        }, () => {
            if (this.state.email === true && this.state.phone === true) {
                ...Do something
            } else {
                ...Error
            }
        });

        //NOTE: setState acima tem um callback como segundo parametro,
        //quando os estados atualizarem, esse calback é chamado
    }

    render() {
        return (
            <input onClick={this.handleClick}>
        );
    }
}


Linux Shell Script - Dúvida sobre escopo de variável

Question: Num script shell, quero fazer um array contendo os arquivos de determinada pasta.
Tentei assim:
declare -a array_arquivos
i=0
ls -1 /caminho/minha_pasta | while read file; do
    array_arquivos[$i]="$file"
    ((i++))
done

Porém, não dá certo pois o array array_arquivos criado no início do script não é visível ali dentro do pipe, pois este cria um novo subshell. Com isso o array_arquivos não é populado. Mas, desse jeito aqui deu certo:
declare -a array_arquivos
i=0
while read file; do
    array_arquivos[$i]="$file"
    ((i++))
done <<< $(ls -1 /caminho/minhapasta)

Ou seja, ao invés de usar pipe, mandei o resultado do LS para o meu loop através do operador <<<. Ou seja, problema resolvido.
Porém, ali no primeiro exemplo que não funcionou, fiquei sem entender o seguinte:
Por que a variável array_arquivos não é visível dentro do pipe, mas a variável $i é visível, sendo que as duas variáveis foram criadas juntas no começo do script?
Comment: Só para melhorar para a comunidade amigo. A dúvida tem que ser resumida no título da pergunta.
Answer: Já entendi o que acontece. No primeiro exemplo, as duas variáveis são sim visíveis dentro do pipe. Porém, modificando as variáveis ali dentro, as modificações só valem ali. Ou seja, as mesmas variáveis, vistas fora do pipe, continuam como foram declaradas no inicio do script. Isso não é um problema para a variável $i, pois não a uso mais após terminar o loop. Mas no caso da variável array_arquivos, ela é sim populada, mas isso só terá efeito dentro do loop. Quando tento pegar a variável novamente após terminar o loop, ela continua em branco, como foi declarada no inicio do script. Espero que essa observação possa ser útil pra alguém.


Jersey: adding headers to a server response

Question: I'm using Jersey, I have the following method:
@POST
@Path("hello")
@Produces(MediaType.TEXT_HTML)
public String hello(@FormParam("username") String username)
{
    Gson gson = new Gson();
    CommunicationResponseM result = new CommunicationResponseM();

    String result = "hello";

    return gson.toJson(result);
}

So far all goes well, but now I need to add some headers. How can I do that?
Thanks!
PS:
I start the Jersey server in this way:
    final HttpServer server = HttpServerFactory.create(baseUrl);
    server.start();

Answer: You can return a Response object instead. Have a look at:
https://jersey.java.net/nonav/documentation/latest/user-guide.html#d0e5169
and
http://jersey.java.net/nonav/apidocs/1.17/jersey/javax/ws/rs/core/Response.ResponseBuilder.html
These should get you on the right track...
Comment: Note: your first link has an invalid id (#d4e357)
Answer: If you're looking for a way to get the value of a header parameter from the http request, then you can use the @HeaderParam annotation. It's similar to the @FormParam annotation.
If you're looking to add a header to your response, there are several ways.

You can inject the HttpServletResponse in your resource using the @Context annotation and call the addHeader method on it.
You can return a Response object in your method and use that to add the header. See also ResponseBuilder.

For Jersey 1, there's more info in the Jersey 1.18 user guide. See sections 2.5 and 2.13.
For the Jersey 2 user guide, see chapter 3 and section 3.6.
Comment: This is what I'd want, but links are broken :(
Comment: I've fixed the links and added a link to the Jersey 2 user guide.


Is $\sin^2 x $ the same as $\sin x^2$?

Question: I have seen $\sin^2 x$ in an equation and I don't understand how you can square $\sin$. So does $\sin$ squared $x$ equal $\sin x$ squared? Closed it has been answered
Comment: $(\sin x)^2$ is what it means.
Comment: thanks that has answered it
Answer: Generally $\sin^{2}x$ is used as shorthand for $(\sin x)^{2} = (\sin x) \cdot (\sin x)$, which is different from $\sin x^{2}$.  You can make it less ambiguous by writing the latter as $\sin(x^{2})$.
Edit: You just have to be careful when you see $\sin^{-1}x$, which does not mean $(\sin x)^{-1} = \frac{1}{\sin x}$, but rather the inverse sine, or $\arcsin x$.
Comment: Thanks that has answered my question perfectly!
Comment: It is best never to write $\sin x^2$ at all.  Write either $(\sin x)^2$ or $\sin(x^2)$.  Or, of course, the traditional $\sin^2 x$, as in the OP.
Comment: If you always use the functional notation $\sin(x)$ instead of the justaposition, the problem do not arise.
Answer: Carl Gauss (the most famous person to live on Earth in the 19th century, except for people who did not work in the physical and mathematical sciences) wrote (correctly in my view) that the notation $\sin^2 x$ ought to mean $\sin(\sin x)).$ But in his time it had already become conventional to use that notation to mean $(\sin x)^2,$ and that is what everybody learns in secondary school today and for many decades past.
The notation $\sin x^2$ usually means $\sin(x^2),$ and the latter is how I would normally write it.
Comment: Trig power notation is one of the worst things to happen to basic function notation. So many students fundamentally do not understand inverse functions because of it. I don't know how many students I've tutored and had to explain it several times because of this discrepancy. I really wish we would collectively kick that nonsense notation to the curb and write what you mean. (I'd also opt for writing function notation explicitly everywhere, e.g. dropping $e^x$ notation in favor of $\exp(x)$, not writing $\sin x$ but rather $\sin(x)$, etc. but pipe dreams are exactly that.)

Is $\sin^2 x $ the same as $\sin x^2$?


Como obter a frequência do microfone em Hertz no Java Script?

Question: Estou trabalhando em um componente de uma aplicação, onde tenho que pegar a frequência de um som enviado por uma entrada de áudio (microfone).
Embora eu nunca tenha feito nada disso, dei uma pesquisada, li toda documentação do AudioContext e não identifiquei uma maneira de fazer isso a não ser APIs prontas. Ainda pesquisando eu achei um exemplo que quase de certo.
var display = document.getElementById("display");

navigator.mediaDevices.getUserMedia({audio: true})
.then(function(stream) {
    var ctx = new AudioContext();
    /* Cria um fonte de stream */
    var source = ctx.createMediaStreamSource(stream);
    var analyser = ctx.createAnalyser();
    var processor = ctx.createScriptProcessor(256, 1, 1);
    source.connect(processor);
    source.connect(analyser);
    processor.connect(ctx.destination);

    processor.onaudioprocess = (data) => {
        var dataArray = new Float32Array(analyser.frequencyBinCount);

        setInterval(() => {
            analyser.getFloatFrequencyData(dataArray);
            display.innerHTML = dataArray;
        }, 1000);
    }

})
.catch(err => {
  console.log(err.message);
});

Se a variável dataArray retornasse a frequência em hertz seria o suficiente, mas aparentemente ela reage aos decibéis e não ao tom(nota) pois seus valores não são fieis aos tons das cordas dos instrumentos em hertz.
Meu projeto é fazer um afinador de guitarra/violão/baixo. Então quando a nota for tocada o código precisa dar frequência em hertz para comparar e dizer se as cordas devem ser apertadas ou afrouxadas.
Alguém sabe uma maneira de obter a frequência de uma som em HERTZ ?
Answer: Já falei algumas vezes disso aqui, pode parecer algo bobo querer pegar frequências, mas acontece que não é, tudo vai depender de N fatores, o quão preciso você quer que a captura seja, as frequências capturadas serão de sons monofônicos ou não ? qual seria a frequência máxima que o seu afinador precisa capturar, são muitas variáveis, não existe uma única maneira de se fazer isso, capturar o Pitch/F0/Frequência/Período/Hz pode ser algo físico, ou seja contar quantas vezes um padrão se repete ou algo psicoacústico na qual envolve a percepção tonal do som...
Existem diferentes meios de fazer isso, tem como fazer isso no domínio do tempo (em JS existe o getFloatTimeDomainData para retornar os dados iniciais no domínio do tempo) aplicando técnicas de AutoCorrelação no sinal, existem técnicas derivadas da autocorrelação como AMDF, ASDF, YIN, etc, no domínio da Frequência também existe o equivalente a autocorrelação para encontrar a periodicidade no sinal, existem técnicas mais elaboradas que envolve encontrar o CEPSTRUM, outras técnicas envolvem somar os subharmonicos dos componentes espectrais retornados pelo FFT(Fourier),  pelo seu código acima percebi que está indo pelo domínio da frequência e tentando aplicar a Transformada de Fourier, existe uma maneira bem simplista de encontrar frequências usando uma FFT, não é nada recomendado fazer isso em afinadores mas é uma saída inicial para seus testes e estudos, ao retornar os componentes da FFT você poderá achar a periodicidade do sinal se encontrar qual é o componente de maior amplitude, lembrando que esse método é bem rudimentar e poderá causar diversos falsos positivos .... em suma para se fazer isso você irá precisar dos seguintes dados:
Frequencia = Fs * i / N

Onde:

Fs = taxa de amostragem (Hz)
i = index da maior amplitude/magnitude
N = número de pontos da sua FFT

A taxa de amostragem padrão do Web Audio API é de 44100Hz então se você usar 4096 pontos em sua FFT vc terá uma frequência de ordem de resolução de 10,7666015625hz ou seja, cada componente poderá estar errando aproximadamente 10Hz então não é tão preciso assim, você poderá depois suavizar/melhorar a precisão utilizando interpolação parabólica com a ajuda dos componentes adjacentes ...
Escrevi um código básico conceitual para demonstrar como usar o web audio API para pegar os componentes espectrais, capturar a máxima amplitude do sinal e converter o índice em hertz:
hertz.html
<!DOCTYPE html>

ederwander hertz teste

<button id="start">Start</button>
<button id="stop">Stop</button>

teste básico com senoides puras:
Teste de 440hz com o mic ligado capturando uma senoide de 440Hz (link youtube da senoide):
https://www.youtube.com/watch?v=xGXYFJmvIvk
print do código acima em funcionamento:

Foi bem perto, capturou 441Hz
Mais um teste com senoide, tentando capturar com o Mic uma senoide em 528Hz
https://www.youtube.com/watch?v=qgSb8QdFU7k
print do código acima em funcionamento:

Também passou bem perto ...
Bom isso é o básico do básico de como se capturar freq no domínio da frequência(FFT), o código é bem simplista, se quiser algo mais preciso tenta introduzir interpolação parabólica ou usar a combinação de outros métodos descritos acima ....
PS: Para captura de áudios monofônicos que se aproximem de uma senoide pura esse método pode funcionar, talvez seja um bom início para se capturar ondas de frequências de cordas soltas (corda por corda), esse método conseguiria em suma capturar frequências que ficam acima do limite da audição humana (22050hz) um pouco mais que o limite de um super ouvido humano ... segue tabela das frequências por corda de uma guitarra... Uso um código mais elaborado e mais preciso do que esse exemplo para afinar meus próprios instrumentos em casa, tabela de frequência por corda solta de uma guitarra:

Corda
Nota
Frequência
Notação formal

1 (mais aguda)
Mi
330 Hz
E4

2
Si
247 Hz
B3

3
Sol
196 Hz
G3

4
Ré
146 Hz
D3

5
Lá
110 Hz
A2

6
Mi
82 Hz
E2

Então para cada corda solta tocada você terá um parâmetro para saber se ela está perto ou distante da afinação correta !


what's the most efficient way to find the minimal polynomial of a matrix?

Question: When looking for Jordan canonical form of a matrix lets say $\begin{pmatrix} 2 &0 &0\\0 &5 &0\\1&2&2 \end{pmatrix} $ for simplicity.
It's easy to calculate the characteristic polynomial $c_A(x):=\det(xI-A)=\begin{pmatrix}(x-2)&0&0 \\0 &(x-5)&0\\-1&-2&(x-3)\end{pmatrix}=(x-2)^2(x-5)$.
so we know the possible minimal polynomials are $m_a(x)=(x-2)^d(x-5)$, where $1\leq d \leq2$.
and that the unique minimal polynomial must satisfy $m(A)=0$. This is a pretty straightforward matrix so it's easy enough to check both cases. But for larger matrices checking all the cases by process of elimination can be a long process. Is there anyway to make an educated guess on what the minimal polynomial will be without just checking all the possible cases ?
P.S. The matrix I've given here was only a basic example I used to elucidate what I meant. I'm seeking tips and tricks that could apply to any general matrix.
Comment: As you mention Jordan canonical form, it is easy to compute the minimal polynomial of a matrix which is written in Jordan canonical form.
Comment: @mathcounterexamples.net I'm talking about finding the jordan canonical form of an arbitrary matrix
Comment: Not sure to have a clear idea of what you're asking for. At least you don't need the minimal polynomial to compute Jordan canonical form.
Comment: There is an algorithm stemming from the [Structure theorem for finitely generated modules over a principal ideal domain](https://en.wikipedia.org/wiki/Structure_theorem_for_finitely_generated_modules_over_a_principal_ideal_domain).
Comment: @mathcounterexamples.net the method in my lecture notes uses the minimal polynomial and characteristic polynomial of an arbitrary matrix to find the Jordan canonical form. I'm sure there are other ways  but this is the only one i know about
Comment: Since your original matrix is triangular, you can write down its characteristic polynomial directly without doing any calculations.
Answer: The minimal polynomial is the product of the minimal polynomials of each  restriction to a generalised eigenspace. So the problem comes down to  finding the minimal polynomial of a matrix with a single eigenvalue $\lambda$.
One way to find the structure of the associated Jordan matrix is to consider the following sequence of subspaces:
$$\{0\}\subset\ker(A-\lambda I)\subset\ker(A-\lambda I)^2\subset\dots\subset\ker(A-\lambda I)^r\subset\dotsm$$
This sequence first increases (strictly) and eventually stabilises. We have the following result:
$$\dim\ker(A-\lambda I)^i-\dim\ker(A-\lambda I)^{i-1}= \text{number of Jordan blocks of size}\ge i.$$
In particular,$\dim\,\ker(A-\lambda I)$ is the total number of Jordan  blocks.
Comment: this seems like an interesting method. a side question on a less general set of matrices though, If we have any nxn matrix over a commutative ring ( which I believe means the eigenvalues are elements of a commutative ring ?) then can we always assume that it's minimal polynomial is its characteristic polynomial because of the Cayley-Hamilton theorem ?
Comment: I feel like i may be misinterpreting the Cayley-Hamilton theorem as I've been looking up examples of matrices where this doesn't seem to be the case ?
Comment: Cayley-Hamilton is also valid over a commutative ring $R$, but the notion of minimal polynomial doesn't make sense, since $R[X]$ is not a P.I.D. (polynomial ring being a P.I.D. if and only if the base ring is a field) – not even a domain unless $R$ is.


MySQL performance and optimization with multiple queries on a profile page, can they be reduced?

Question: I am working on social networking site, which has a profile page for each user.
The profile page contains:

info about the user, avatar, hobbies & etc
their videos and photos
their friends
their bookmarks
videos they watched

At the moment i execute 14 different queries each time  profile page is loaded, some of  them listed below:

check if the viewer is allowed to view the profile page
get profile info
get profile videos
get profile photos
get friend list
get bookmark list
get videos they watched

I would like to know best way to improve the performance and optimize my site, or can i reduce the number of queries i run.
Can MySQL handle this much load?
if it helps i am using innodb engine i needed faster updating speed as well realtionships.
thanks :)
Comment: As far as the *amount*, 14 queries is nothing. Don't worry, keep developing. Optimize later.
Answer: If it ever does become a performance issue, here are a couple of easy things that will help:

Cache the queries on the db server
Cache the results of the queries in the web app
Cache the entire profile page, or parts of it which change infrequently.

If you need to get more serious, you can create a new table in the database that stores the results of some of your queries all in one row, and regenerate it periodically. Then have your page select from that (then you have everything you need in one query).
But most importantly: what @Jason said. Chances are if performance does become an issue (because you turn into the next twitter or something), it'll be cheaper to throw hardware at the problem than to rewrite.
Answer: As long as they are short-running queries it's okay. Doing anything to optimize speed might give you, at best, some additional milliseconds. If it still bothers, you could do some caching.
Lookup also memcache.


A trigonometric equation 1

Question: $$4 \sin x \cos^3 x - 4 \sin^3 x \cos x = 1$$
I tried to multiply both members of the equations by $\frac{1}{\cos^4 x}$ and then say $\frac{1}{\cos^4 x} = (\tan^2 x + 1)^2$.
I ended up with the equation $$\tan^4 x + 4 \tan^3 x + 2 \tan^2 x -4 \tan x + 1 = 0$$ but I have to get to this answer: $\left\{\frac{\pi }{8}+\frac{k\pi }{2},\:k\:\in \mathbb{Z}\right\}$
Answer: Using $\cos3A=4\cos^3A-3\cos A,\sin3A=3\sin A-4\sin^3A$ formula,
$$1=\sin x(\cos3x+3\cos x)+\cos x(\sin3x-3\sin x)=\sin(x+3x)$$
Can you take it from here?
Answer: Variant:
$$4\sin x\cos^3x-4\sin^3x\cos x=2(2\sin x\cos x)(\cos^2x-\sin^2x)=2\sin 2x\cos  2x=\sin 4x=1.$$
The solutions are given by
$$4x\equiv \frac\pi2\mod 2\pi\iff x\equiv\frac\pi8\mod\frac\pi2.$$
Comment: How is it different from the rest two answers?
Comment: Sorry, I was writing when other answers were being posted, and I didn't look if a similar answer was posted in the meantime. I'd say it's shorter and expressed in terms of congruences. Not fundamentally different though.
Comment: I actually expect something really unique whenever  u write an answer.
Comment: ?? The solution is not unique. Do you mean you don't like expressing solutions as congruences?
Comment: +1 for congruence representation, have you seen that anywhere or only you write like this?Asking because I am seeing it for the first time.
Comment: It's a very handy notation. I always used it on teaching, but of course it depends on the teaching level.
Answer: The key isn't just solving this one problem, but being able to get to a lot of useful trig identities with a simple method.  de Moivre's theorem is a great method for this.
From Euler's theorem (which most people remember)
$e^{i \theta}=cos(\theta)+i sin(\theta)$
we can easily get de Moivre's theorem (since they are almost the same)
$(cos(\theta)+i sin(\theta))^n=cos(n \theta)+i sin(n \theta)$
If you use Pascal's triangle on the left hand side, you then get some nice trig identities.
$cos(4 \theta)+i sin(4 \theta)=(cos(\theta)+i sin(\theta))^4 $
I like to set up Pascal's triangle elements in rows.
\begin{matrix}
1 & 4 & 6 & 4 & 1 \\
cos^4(\theta) & cos^3(\theta) & cos^2(\theta) & cos(\theta) & 1 \\
1 & sin(\theta) & sin^2(\theta) & sin^3(\theta) & sin^4(\theta) \\
1 & i & -1 & -i & 1
\end{matrix}
Multiplying the columns, we get
$cos^4(\theta)+ 4i cos^3(\theta)sin(\theta)-6 cos^2(\theta)sin^2(\theta)-4 i cos(\theta) sin^3(\theta)+sin^4(\theta)$
Equating the real to the real and the imaginary to the imaginary, we get
$cos(4\theta)=cos^4(\theta)-6 cos^2(\theta) sin^2(\theta)+sin^4(\theta)$
$sin(4\theta)=4 cos^3(\theta)sin(\theta)-4 cos(\theta) sin^3(\theta)$
Using this second identity, your problem comes out pretty quickly.
So generally, when you see numbers in a trig identity that remind you of Pascal's triangle, and powers that line up nice, your first thought should be to look for these sorts of identities.

Just a hint about how math teachers/instructors think.  If a problem like yours appears in the homework, you can bet that a problem involving something like
$cos(3\theta)=cos^3(\theta)-3 cos(\theta) sin^2(\theta)$
$sin(3\theta)=3 cos^2(\theta)sin(\theta)-sin^3(\theta)$
is likely to appear on the test.
And there's no sense in trying to memorize these guys if you can derive them in about a minute.
Answer: Hint:
Observe that
\begin{align*}
4 \sin x \cos^3 x - 4 \sin^3 x \cos x &= 2(2\sin x\cos x)(\cos^2 x-\sin^2 x)\\
&=2\sin (2x)\cos (2x)\\
&=\sin (4x)
\end{align*}
Answer: $$\begin{array}{rcl}
4 \sin x \cos^3 x - 4 \sin^3 x \cos x &=& 1 \\
4 \sin x \cos x (\cos^2 x - \sin^2 x) &=& 1 \\
2 \sin 2x \cos 2x &=& 1 \\
\sin 4x &=& 1 \\
4x &\in& \left\{\dfrac\pi2 + 2k\pi : k \in \Bbb Z\right\} \\
x &\in& \left\{(4k+1)\dfrac \pi8 : k \in \Bbb Z\right\} \\
\end{array}$$

A trigonometric equation 1


Double turns into nan in C

Question: I have a function that uses an int, say 2488, to store temperature values. I have to call a function getTemp() to get the int. The getTemp function returns a double and uses the int to return the correct number. All the getTemp() does is return (double)x / 100.0 where x is 2488 in this case.
The returned double is then 24.88. This value then is sent to another function that adds the double to an array. Function is called DAaddDouble(double m, int x, int y) where m is the value to add, x and y is the coordinates that specify where to add the double.
Problem is, it turns into nan.
double a = getTemp();
//a is correct, i.e. 24.88
DAaddDouble(a, x, y);
/*-----------inside DAaddDouble----------*/
void DAaddDouble(double m, int x, int y)
{
   //at this point, a (or m, same) is 0.nan
   cord = x + y*40; //where to put the double
   snprintf(DARRAY[cord], 5, "%f",m);
   printf(....DARRAY[cord]...);

}

output: -nan
Comment: How you defined DARRAY?
Comment: How you defined DARRAY? Change DAaddDouble function to void DAaddDouble(double m, int x, int y) as a or m is 24.88
Comment: I'm sorry, it was a typo, it is double but the problem still remains.
Comment: Your problem is unlikely to be in the fragment you quoted. Try stepping through the lines in a debugger to see when exactly `m` becomes `NaN`, if ever.
Answer: The signature of function DAaddDouble is:
void DAaddDouble(int m, int x, int y)

Note that m is of type int, and inside the function, you have:
snprintf(DARRAY[cord], 5, "%f",m);

in which %f expects type double, it's undefined behavior.
Probably what you need is to have the parameter m as double(as in your words above the code).
void DAaddDouble(double m, int x, int y)

Comment: I'm sorry, it was a typo, it is double but the problem still remains.
Answer: Mis-match prototype.
Usage of DAaddDouble() is not preceded by its declaration/definition, thus the compiler assumes the function is:
int DAaddDouble(int m, int x, int y);

Precede the usage of DAaddDouble() by the function definition or a function prototype.
void DAaddDouble(double m, int x, int y);

A good compiler will warn of this.  Insure all warning are enabled.
Answer: How do you know m is NAN. If you have infered by the value "printed" to DARRAY[cord], take into account that the format string should be "%lf" and not "%f".
void DAaddDouble(double m, int x, int y)
{
   //at this point, a (or m, same) is 0.nan
   cord = x + y*40; //where to put the double
   snprintf(DARRAY[cord], 5, "%f",m); /* should be "%lf" for printing doubles */
   printf(....DARRAY[cord]...);

}

Comment: _format string should be `"%lf"` and not `"%f"`_ is not correct.  Either will work.  It makes a functional difference with `scanf()`, but not `printf()`  "l (ell) ... has no effect on a following a, A, e, E, f, F, g, or G conversion" C11dr §184.108.40.206 7.  `m` as a `float` or `double` is converted/passed to `snprintf()` as a `double`.
specifier.

Double turns into nan in C


What is this notation, similar to the binomial coefficient?

Question: I've come across this notation:
$$\left\{\begin{eqnarray}n\\m\end{eqnarray}\right\}$$
The only other info I have about this notation is that $\left\{\begin{eqnarray}4\\2\end{eqnarray}\right\}=7$
What's the name of this notation and what is it used for?
Thanks
Comment: I find it slightly disappointing that with the vast knowledge we have here, all three answers do little more than link to wikipedia.
Comment: There is a [Stirling numbers](http://math.stackexchange.com/questions/tagged/stirling-numbers), which I added to your post. If you are interested in them then you might want to browse through the questions there.
Answer: Maybe you're using Stirling numbers of the second kind, where $\displaystyle{n\brace k}$ denotes the number of ways to partition a set of $n$ objects into $k$ non-empty subsets.
You can use {n\brace k} to produce ${n\brace k}$ and \displaystyle{n\brace k} to produce $\displaystyle{n\brace k}$.
An example of a 'real world' application of these numbers can be found in this MSE question which asks how many rooks can be placed on a triangular chessboard so that none of them are attacking each other.
Comment: Any chance of an application? It would make your answer complete...(for example, they can be used to find certain generating sets of certain semigroups).
Comment: Can you elaborate your answer by providing a real life example of its usage? Thanks...
Answer: Stirling numbers of second kind, the number of partitions of a set of $n$ objects into $k$ non-empty subsets.


Spring Bean Injection of Interface object

Question: The code is as follows
public interface ExperimentConfig {
  Boolean func1();
}

public class RuntimeConfig implements IntializingBean{
  private ExperimentConfig experimentConfig;

  public ExperimentConfig getExperimentConfig() {
    return experimentConfig;
  }

  public void setExperimentConfig(ExperimentConfig experimentConfig) {
    this.experimentConfig = experimentConfig;
  }
}

public class Util
{
  public static ExperimentConfig experimentConfig()
  {
     return new Builder.build(1,"a",89);
  }

Spring XML file
<bean id="RuntimeConfig"
   class="bizconfig.RuntimeConfig"scope="singleton">

   <property name="experimentConfig">
      <bean class="bizconfig.Util" factory-method="experimentConfig"/>
    </property>
</bean>

This code dosent seem to work. I think I'm using Spring the wrong way when an Interface is a property. How do I fix this.
Comment: What isn't working?
Answer: <bean id="util" class="Util">
...properties
</bean>

<bean id="RuntimeConfig"
   class="bizconfig.RuntimeConfig"scope="singleton">

   <property name="experimentConfig">
      <bean factory-bean="util" factory-method="experimentConfig"/>
    </property>
</bean>

Comment: I dont understnad by what you mean by "...properties". Util only has static functions
Comment: if you dont have any other properties you dont need to


Best practice to gather data across multiple fragments before db write

Question: Android application using .net/Xamarin. I have several fragments in a view pager. If I were to have some data collected on each fragment in the pager incrimentally, what is the best way to gather all the data as I go and only write to the db once, at the end? I am using a sqlite db. But, lets say for example there are two fragments in the pager. First fragment collects user name, first and last name. Then, a next button takes them to the second fragment for address and phone number. After they are done, they click save and then I can write to the db one I have gathered all of the info.
Sure, I could write to the db after each fragment, but is that really the best practice? I dont think I want to pass the data from fragment to fragment. That might seem like a way to go with the above example but I dont think it scales well, for example if I end up later with 5 fragments collecting data.
I have been thinking about a singleton class. I have read that they should be avoided at all costs but maybe this is an application suited for them? I could create a singleton with all my properties and as I go through the fragments and build data, they are saved in the singleton. At the end, I basically write the singleton data to the db and clear it out, ready for the next time. I dont think I have to worry about thread safety in this use case or at least, I can write it so it is not an issue.
Is that the way to go? Other thoughts or ideas?
Thanks!
Mike
Answer: A singleton is used when you need to manage a resource, such as a database connection. In this case, I would recommend against using a singleton just to store data.
The way you propose to use the singleton isn't much different than passing an object around, the only difference being that with a singleton, you can make a request for the object you are populating.
The way you could do it, would be to have an object representing each fragment. This would allow you to initialize and have fragment dependent logic (such as validation) within a given class.
You would then have a more generic object, one that will hold the information from all the fragments which will be eventually saved to the DB. You can pass this object around from one fragment to the next, and each fragment populates the bits it is interested in.


Opening a gzip file in python Apache Beam

Question: Is it currently possible to read froma a gzip file in python using Apache Beam?
My pipeline is pulling gzip files from gcs with this line of code:
beam.io.Read(beam.io.TextFileSource('gs://bucket/file.gz', compression_type='GZIP'))

But I am getting this error:
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte

We noticed in the python beam source code that compressed files seem to be handled when writing to a sink.
https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/fileio.py#L445
More Detailed Traceback:
Traceback (most recent call last):
  File "beam-playground.py", line 11, in <module>
    p.run()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/pipeline.py", line 159, in run
    return self.runner.run(self)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/direct_runner.py", line 103, in run
    super(DirectPipelineRunner, self).run(pipeline)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 98, in run
    pipeline.visit(RunVisitor(self))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/pipeline.py", line 182, in visit
    self._root_transform().visit(visitor, self, visited)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/pipeline.py", line 419, in visit
    part.visit(visitor, pipeline, visited)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/pipeline.py", line 422, in visit
    visitor.visit_transform(self)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 93, in visit_transform
    self.runner.run_transform(transform_node)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/runner.py", line 168, in run_transform
    return m(transform_node)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/direct_runner.py", line 99, in func_wrapper
    func(self, pvalue, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/direct_runner.py", line 258, in run_Read
    read_values(reader)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/runners/direct_runner.py", line 245, in read_values
    read_result = [GlobalWindows.windowed_value(e) for e in reader]
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/io/fileio.py", line 807, in __iter__
    yield self.source.coder.decode(line)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/apache_beam/coders/coders.py", line 187, in decode
    return value.decode('utf-8')
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte

Answer: UPDATE: TextIO in the Python SDK now supports reading from compressed files.
Today TextIO in the Python SDK does not actually support reading from compressed files.
Comment: I have just looked deeper - I was wrong about the depth of support. The class accepts the compression_type parameter in preparation for actual support.
Comment: Thanks for the quick response! I'll skim through the article. I added the entire traceback to the question.
Comment: The data is compressed in UTF-8. And I verified it by decoding the original text from bytes into unicode. There were no errors. Correct me if I'm wrong, but looking at the beam source code, it seems like TextFileSource doesn't even handle compressed files.
Comment: Thank you for verifying that. Could you give a rough estimate for when this feature will be implemented?
Comment: Following Apache Beam (incubating) JIRA issues track support for adding a new source for reading text files and adding support for reading compressed files on top of that.

* https://issues.apache.org/jira/browse/BEAM-553
* https://issues.apache.org/jira/browse/BEAM-577

We hope to implement these within several months.
Comment: addendum: for some reason, including `compression_type='GZIP'` in my code, caused the error `TypeError: compression_type must be CompressionType object but was `; however including `compression_type='gzip'` solves it. Just if it is of any help, for future readers.
Comment: The reason is here: https://github.com/apache/beam/blob/b35ad6dffb712e1e6347fa00c81245bc562b1f57/sdks/python/apache_beam/io/filesystem.py#L86.

The most robust thing to do would be to use `CompressionTypes.GZIP`
Answer: I ran into a similar problem. I have a custom binary source that I wanted to parse and grab data out of. The issue is the file.io API is based out of CSV or ARVO and no matter what I tried it wouldn't give me the lines without trying to split them on line break. As you can imagine, a binary file doesn't handle this well.
At first I tried a custom source, that ended up being 3 classes long to implement and it was duplicating core Dataflow/Beam code. In the end I coded this WONDERFUL little bit of monkeypatching to get what I needed done (deep level source coding testing here).
import apache_beam as beam
from apache_beam.io.fileio import coders

def _TextFileReader__iter(self):
    # The full data file is had here and can be read like normal
    # You can even limit the character bit here. (I did 9 to grab the file format)
    data = self._file.read()
    # Now you can either yield the whole file as a single data entry
    # and run a ParDo to split it, or you can iterate in here and
    # yield each row. I chose the latter, but I'm showing an example
    # of the former.
    yield data

# This monkeypatch good!
beam.io.fileio.TextFileReader.__iter__ = _TextFileReader__iter

To call this source, and make sure it's BINARY, I did the following:
pipeline | 'start_3' >> beam.io.Read(
    beam.io.TextFileSource( 'gs://MY_BUCKET/sample.bin',
        coder=coders.BytesCoder()
    )
)

Notice the coders.BytesCoders()? Without that it tried to convert the Bytes to something non-binary which wasn't good for my parsing engine.  ;)
Took me a good day chunk of time to figure this out. However, if you use this method you can do almost anything with the file.io class in Dataflow.  ;)
Answer: I ran into the same problem. I was trying to read binary GZ files from GCS, decompress them, and then ship them off to somewhere else for processing. I solved it in two steps.
First, make sure you're using the right Python library; my original library was out of date (I use at least v0.4): pip install --upgrade google-cloud-dataflow.
Second, I constructed my pipeline as follows:
import apache_beam as beam
from apache_beam import (coders, io, transforms)

raw_logs = (p
            | io.Read("ReadLogsFromGCS", beam.io.TextFileSource(
                      "gs://my-bucket/logs-*.gz",
                      coder=coders.BytesCoder()))
            | transforms.Map(lambda x: x)
            | io.Write("WriteToLocalhost", io.textio.WriteToText(
                       "/tmp/flattened-logs",
                       file_name_suffix=".json")))
p.run()

You should have a file called /tmp/flattened-logs.json after running the pipeline.


SOLR ignoring comma and other punctuation while searching

Question: I want to search 100,000 and 100000 only by querying 100000
(or only by querying 100,000).
Is it possible to search like that.
The purpose to perform this is, that user may add a comma at any place or no comma at all but the strings (100,000 or 100000) should be searched anyway...
Answer: Try using solr.WordDelimiterFilterFactory with catenateNumbers="1" in the index and query definition of you field.
You have some docs here:
WordDelimiterFactory
Answer: Don't know if this is correct, but maybe you can write some kind of filter factory where numbers are always indexed as numbers formatted with commas and with out commas. Or you can have both the index & query analyzer store the word w/o comma.
Comment: agreed, seems no other possibility.
Answer: just remove the , from the query if it is in a number
(if you are using java at the client -> parse the number and do toString)


Study convergence of $x_{n+1} = a\left(x_n + {1\over x_n}\right)$, for $x_1 = a$ and $a \in (0, 1)$

Question:
Given a recurrence relation:
  $$
x_{n+1} = a\left(x_n + {1\over x_n}\right) \\
x_1 = a\\
n\in\Bbb N
$$
  Show that:
  $$
\begin{align*}
a \ge 1 &\implies \lim_{n\to\infty} x_n =+\infty \tag1\\
a \in (0, 1) &\implies \lim_{n\to\infty} x_n =\sqrt{\frac{a}{1-a}} \tag2\\
\end{align*}
$$

I think I've been able to prove case $(1)$ but faced difficulties in case $(2)$. Here is what i've done for $a\ge 1$. Suppose a limit exists, then it must equal to one of the fixed points:
$$
\exists \lim_{n\to\infty}x_n = L \implies L = aL + {a\over L}\\
\iff L = \frac{\pm \sqrt{4a(1-a)}}{2(a-1)} = \mp \sqrt{\frac{a}{1-a}}
$$
Since $x_n > 0$ the only possible finite limit is $L = \sqrt{\frac{a}{1-a}}$.
Consider the following expression:
$$
x_{n+1} - x_n = a\left(x_n - {1\over x_n}\right) - x_n =\frac{x_n^2(a-1)+a}{x_n} > 0
$$
Therefore $x_n$ in monotonically increasing:
$$
x_{n+1} \ge x_n
$$
Also for $a\ge1$:
$$
L = \sqrt{\frac{a}{1-a}}\notin \Bbb R
$$
Thus by monotonicity of $x_n$ the only possible options left is:
$$
\lim_{n\to\infty} x_n = +\infty
$$

I've tried to apply a similar reasoning to $(2)$ but unfortunately it lead nowhere. The difficulty is also in the fact that the sequence is not necessarily monotone. Here are two sandboxes i've been using while solving the problem.

How can I show that the sequence has a limit when $a\in(0,1)$?
Answer: Let $f(x) = a (x + 1/x)$, and suppose $0 < a < 1$.  $L = \sqrt{a/(1-a)}$ is the only fixed point of $f$ in the positive reals, and it is an attractor since $|f'(L)| = |2a-1| < 1$.  Moreover, there are no $2$-cycles:
$$ f(f(x))-x =  \frac{(a+ (a+1) x^2)(a + (a-1)x^2)}{x(x^2+1)}$$
The endpoints of the immediate basin of attraction of an attracting fixed point can only be singular points, $\pm \infty$, non-attracting fixed points, and members of a $2$-cycle.  Thus in this case the immediate basin of attraction must be $(0, \infty)$, i.e. the sequence $x_n = f(x_{n-1})$ starting anywhere in $(0,\infty$ will always converge to $L$.
Comment: That you for the answer, unfortunately the concepts you've used are beyond my level of knowledge for now. But I would appreciate if you could point me to the literature required to grasp those concepts.
Answer: You already showed
$$ x_{n+1}\ge x_n. \tag{1}$$
Note that, by the AM-GM inequality, one has
$$ x_{n+1}\ge 2a.$$
From
$$x_{n+1}-x_n=a(x_n-x_{n-1}+{1\over x_n} - {1\over x_{n-1}})=a(1-{1\over x_nx_{n-1}})(x_n-x_{n-1})\le a(1-\frac{1}{4a^2})(x_n-x_{n-1})$$
one has
$$ 1-\frac{1}{4a^2}\ge 0$$
or $a>\frac12$.
Define $r= a(1-\frac{1}{4a^2})$ and then $r\in(0,1)$. So
$$ x_n-x_{n-1}\le r^{n-1}(x_2-x_1)$$
$$ x_n=\sum_{k=2}^n(x_k-x_{k-1})+x_1\le\sum_{k=2}^nr^{k-1}(x_2-x_1)+x_1<\infty.$$
Namely ${x_n}$ is bounded. Now one can use the Bounded Monotonic Convergence Principle to get the limit.
Comment: @roman, this because $x_{n+1}-x_n\ge0$ and $x_{n}-x_{n-1}\ge0$.
Comment: Thank you for the answer. Could you please elaborate on $1-{1\over 4a^2} \ge 0$. Here $a$ is a given initial condition and $a\in(0, 1)$, while you are restricting it to be $a > {1\over 2}$. Is it rigorous to make such assumptions? Would't that contradict the initial conditions on $a$? Also I'm trying to wrap my mind on the last part, you have stated that $x_n < \infty$ why would that mean the boundedness for $x_n$?


Angular calculation not updating in view

Question: i am creating a small application that calculates distance between two points. The console.log outputs the calculation correctly, but not the angular portion in the view. Only after a click or a second selection. I've setup two select fields (with separate ng-models) that are filled with information out of a JSON file. I've searched the internet extensively, but can't find an answer. Hopefully somebody can help me with this... :) This is my code:
<code>var app = angular.module('calculateApp', []);
app.controller('calculateController', function($scope, $http) {
    $http.get('json/stops.json')
        .then(function(item){
        $scope.stations = item.data;
        $scope.station1 = false;
        $scope.station2 = false;
        // Watch and calculate
        $scope.$watch(
            function( $scope ) {
                function distance(lat1, lon1, lat2, lon2) {
                  var R = 6371;
                  var a =
                     0.5 - Math.cos((lat2 - lat1) * Math.PI / 180)/2 +
                     Math.cos(lat1 * Math.PI / 180) * Math.cos(lat2 * Math.PI / 180) *
                     (1 - Math.cos((lon2 - lon1) * Math.PI / 180))/2;
                  return R * 2 * Math.asin(Math.sqrt(a));
                }
                console.log(distance($scope.station1.stop_lat, $scope.station1.stop_lon, $scope.station2.stop_lat, $scope.station2.stop_lon));
                $scope.calculated = distance($scope.station1.stop_lat, $scope.station1.stop_lon, $scope.station2.stop_lat, $scope.station2.stop_lon);
            }
        );
        // end calculate
    });
});
</code>
I use this in my HTML file:
Total distance is: {{calculated}}
Answer: First of all in a watch you dont need to send the scope into the function
you can however send newValue, oldValue  for those.
I think by removing the $scope from the watch function it is solved
edit:
i now see you want to watch the scope but then you dont need a function $scope
look at this
i think it will help you.
And i am not sure if you can watch the whole scope you should select the thing you need to watch.
Comment: Thank you :) I'll look into it. In the meantime i sort of fixed it, by using ng-show (it updates automatically then)
Answer: You could try adding $scope.$apply at the end of your calculation.
Regards
Ian
Answer: In the meantime i sort of fixed it, by using ng-show (it updates automatically then)


Talend: from XML file to String using tXMLMap

Question: I have a job with a tESBConsumer that has this method: "call(string):string".
The input string must be an entire xml file. So, I drag a tFileInputXML linked to a tXMLMap linked to tESBConsumer linked to a tLogRow.
It doesn't work! I don't know if this is correctly way or there is a easier way (like tMap) to do what I want.
thanks all.
Answer: tFileInputRaw is the component which allows you to read the raw file as a string or as bytes array or stream the file. This you can use to read your xml file and return entire file content as string
Comment: Now, I'm using tFileInputRaw linked to a tESBConsusmer. But if the tESBConsumer has an operation like "call(string):string", when the tFIleInputRaw returns a string, why I get this error in compilation "impossible convert from a string to a Document" on tESBConsumer?


Django: how to get value from model in view

Question: I have a view that will conditionally display a breadcrumb trail based on the value of the object's "status" field. This works if I manually set the status value (ex status = "completed"). However, as a newbie, I can't figure out how to retrieve the value?
status = Project.status is not working.
Here is the view:
class CompanyProjectsDetailView(DetailBreadcrumbMixin, UpdateView):
    model = Project
    queryset = Project.objects.get_with_counted_notes_documents_todos()
    template_name = 'company_accounts/project_detail.html'
    context_object_name = 'project'
    form_class = ProjectStatusForm
    status = Project.status

    if status == "completed":
        @cached_property
        def crumbs(self):
            return [
                ("projects", reverse(
                    "company_project:" + CompanyProjects.list_view_name, )
                 ),
                (f"completed projects", reverse(
                    "company_project:" + CompanyProjects.list_view_name, )
                 ),
            ]

Comment: you need the instance of the Class (an object) or a queryset instead of trying to access the class (Project.status)
Comment: Thanks Alvi15. I thought that must be the case. I just haven't been able to figure out  a way to access the instance yet.
Answer: I figured out a way to get this to work. The key is getting the instance in the crumbs method. I imagine there may be a more elegant way to accomplish this.
The working view:
class CompanyProjectsDetailView(DetailBreadcrumbMixin, UpdateView):
    model = Project
    queryset = Project.objects.get_with_counted_notes_documents_todos()
    template_name = 'company_accounts/project_detail.html'
    context_object_name = 'project'
    form_class = ProjectStatusForm
    # pk_url_kwarg = 'project_id'

    @cached_property
    def crumbs(self):
        project = self.get_object()
        if project.status == "completed":
            return [
                ("projects", reverse(
                    "company_project:" + CompanyProjects.list_view_name, )
                 ),
                (f"completed projects", reverse(
                    "company_project:completed_projects" )
                 ),
                (f"{project.title}",
                 reverse(
                     "company_project:" + CompanyProjectsDetailView.detail_view_name,
                     kwargs={'pk': project.id})
                 ),
            ]
        elif project.status == "paused":
            return [
                ("projects", reverse(
                    "company_project:" + CompanyProjects.list_view_name, )
                 ),
                (f"paused projects", reverse(
                    "company_project:paused_projects" )
                 ),
                (f"{project.title}",
                 reverse(
                     "company_project:" + CompanyProjectsDetailView.detail_view_name,
                     kwargs={'pk': project.id})
                 ),
            ]
        else:
            return [
                ("projects", reverse(
                    "company_project:" + CompanyProjects.list_view_name, )
                 ),
                (f"{project.title}",
                 reverse(
                     "company_project:" + CompanyProjectsDetailView.detail_view_name,
                     kwargs={'pk': project.id})
                 ),
            ]


$A : \mathbb{R}^n \to \mathbb{R}^n \implies A(\mathbb{Z}^n)=\Gamma$ on theTorus

Question: In the Analysis on Manifolds via the Laplacian page $51$, it is indicated that if $A : \mathbb{R}^n \to \mathbb{R}^n$ be so that $A(\mathbb{Z}^n)=\Gamma$, then $\text{Vol} (\mathbb{T})=\det A$. Could anyone be able to give me a hint how to solve that?
Comment: What if $\det A$ is negative? Shouldn't it be $|\det A|$?
Comment: I think the matrix $A$ should be a positive definite matrix.
Answer: The volume of a parallelepiped is given by the determinant, and the parallelepiped is a fundamental domain for the quotient torus.
Answer: If $\Omega=\{\omega_1,\dots,\omega_n\}$ generate $\Gamma$, then $\operatorname{vol}(\Bbb T)$ will be the area spanned by $\Omega$, which is $\det\Omega$. As each $a_i=A(e_i)$ is $\Omega(v)$ for some $v\in\Bbb Z^n$, it then follows that $\det A$ is an integer multiple of $\det \Omega$. But it can't be negative if $A$ is positive definite, and it can't be zero because $A$ is surjective onto $\Bbb R^n$.
We can reduce the problem to the case when $\Omega$ is the identity matrix, and $\Gamma=\Bbb Z^n$, by composing $A$ with $\Omega^{-1}$. Then we are proving that if $A(\Bbb Z^n)=\Bbb Z^n$, then $\det A=1$. In this form it is even clearer that $\det A$ is an integer, because $A$ is an integer matrix.
To finish the proof, note that $A^{-1}$ satisfies the same conditions as $A$: it is positive definite, and $A^{-1}(\Bbb Z^n)=A^{-1}(A(\Bbb Z^n))=\Bbb Z^n$. Thus $\det A^{-1}=\frac1{\det A}$ is also an integer, so $\det A=\det A^{-1}=1$.
Comment: Nice explanation, thanks!


Wrapping embedded images with a div

Question: When embedding an image in CKEditor on Drupal 7 any styles or alignment are applied directly to the image itself e.g.

However this doesn't allow for a great deal of CSS control over the appearance for the image because the inline styles override CSS.
How can I do the following 2 things:

Automatically wrap the image in a div and apply classes to this instead of the img element
For alignment use classes (e.g. 'image-left', 'image-right' etc.) instead of inline styles

So the above example would instead be rendered as:

Comment: That's how CKEditor works, nothing Drupal can do to help you there. If you want that sort of functionality you should probably open a feature request on CKEditor's issue queue. It's a pretty configurable library already, though, so you might be able to just get stuck in and produce that sort of functionality with their config. Not really a question for this site, Stack Overflow would probably be your best bet
Comment: I came here from Stack overflow after reading a similar question and a comment saying the questioner should come here and ask instead. I think this is a 'Would like to have' rather than a 'Must have'
Comment: CKEditor isn't made by Drupal though, it's a 3rd-party library. Questions on configuring it that don't relate to Drupal (e.g. a Drupal-specific plugin) aren't on-topic here because they're only related to the JS config inside CKEditor. Unless there's a Drupal-specific discipline involved (which it doesn't look like there is from the question), it's off-topic here I'm afraid
Comment: Look at this post [How to customize the CKEditor on your Drupal 7 Website](http://www.drupalwoo.com/content/how-customize-ckeditor-drupal-7-site) and others similar. It allows you to create custom styles for the styles dopdown, which does not do exactly as per your requirements but is a close approximation. I've used it in the past to create floated divs.
Answer: Instead of adding a wrapper div, you could use !important.
img {
    float: left !important;
    height: 150px !important;
    width: 90px !important;
}

This would override the style added in CKEDITOR. You could do the same for alignment using classes, adding them directly to the image through CKEDITOR.

Wrapping embedded images with a div


How to draw each section of the blue Bezier curves with different color?

Question: This is below I'm drawing Bezier curves, but It consists of many subset of Quadratic curves are joined together to form a one Bezier curve from P0 to P6. Where a curve has been drawn from P0 to M0 and another one from M0 to M1 and so on as shown below. I need to display each curve with different color, Could I get any help please?

I'm expecting something as shown below, not exactly, but same idea

This is the Python code of Bezier:
import numpy as np
import scipy.special
import matplotlib.pyplot as plt

def calc_bezier_path(control_points, n_points=100):
    """
    Compute bezier path (trajectory) given control points.
    :param control_points: (numpy array)
    :param n_points: (int) number of points in the trajectory
    :return: (numpy array)
    """
    traj = []
    for t in np.linspace(0, 1, n_points):
        traj.append(bezier(t, control_points))

    return np.array(traj)

def bernstein_poly(n, i, t):
    """
    Bernstein polynom.
    :param n: (int) polynom degree
    :param i: (int)
    :param t: (float)
    :return: (float)
    """
    return scipy.special.comb(n, i) * t ** i * (1 - t) ** (n - i)

def bezier(t, control_points):
    """
    Return one point on the bezier curve.
    :param t: (float) number in [0, 1]
    :param control_points: (numpy array)
    :return: (numpy array) Coordinates of the point
    """
    n = len(control_points) - 1
    return np.sum([bernstein_poly(n, i, t) * control_points[i] for i in range(n + 1)], axis=0)

def line_bezier(visx, visy, control, mod="nothing"):
    vis = np.column_stack((visx,visy))
    path_x, path_y = np.array([]),np.array([])
    setting = {"nothing":[len(vis)-2, 1, 1], "start":[len(vis)-1, 0, 0], "end":[len(vis)-1, 1, 0], "both":[len(vis), 0, -1]}
    epoch = setting[mod][0]
    start = setting[mod][1]
    end = setting[mod][2]
    if len(vis) > 2:
        current_control = vis[0]
        for x in range(epoch):
            if x != (epoch-1):
                for y in control:
                    if y == control[0]:
                        mid_control = [(vis[x+start,0]+(vis[x+(start+1),0]-vis[x+start,0])*y), (vis[x+start,1]+(vis[x+(start+1),1]-vis[x+start,1])*y)]
                        plt.annotate(f"M{x}", mid_control)
                        bezier_line = calc_bezier_path(np.array([current_control,vis[x+start], mid_control]))
                        path_x = np.append(path_x, bezier_line.T[0])
                        path_y = np.append(path_y, bezier_line.T[1])
                        current_control = mid_control
                    else:
                        mid_control = [(vis[x+start,0]+(vis[x+(start+1),0]-vis[x+start,0])*y), (vis[x+start,1]+(vis[x+(start+1),1]-vis[x+start,1])*y)]
                        plt.annotate(f"M{x}", mid_control)
                        bezier_line = calc_bezier_path(np.array([current_control, mid_control]))
                        path_x = np.append(path_x, bezier_line.T[0])
                        path_y = np.append(path_y, bezier_line.T[1])
                        current_control = mid_control

            else:
                if mod == "end" or mod == "both":
                    bezier_line = calc_bezier_path(np.array([current_control, vis[x+(end+1)]]))
                else:
                    bezier_line = calc_bezier_path(np.array([current_control, vis[x+end], vis[x+(end+1)]]))
                path_x = np.append(path_x, bezier_line.T[0])
                path_y = np.append(path_y, bezier_line.T[1])
    else:
        path_x, path_y = visx, visy
    return path_x, path_y

visx, visy = [1,2,10,15,20,25,21], [0,5,1,4,2,3,3]
control = [0.25,0.57]
path_x, path_y = line_bezier(visx, visy, control,mod="end")
plt.plot(path_x, path_y)
plt.plot(visx, visy, "--o")
for xy in range(len(visx)):
    plt.annotate(f"P{xy}", [visx[xy], visy[xy]])
plt.show()

Answer: Currently, you're joining the bezier curve together (using path = np.append(path, bezier). If you additionally create a list (e.g. line_collection) to hold each part of the bezier, you can plot them separately afterwards:
import numpy as np
import scipy.special
import matplotlib.pyplot as plt

def calc_bezier_path(control_points, n_points=100):
    """
    Compute bezier path (trajectory) given control points.
    :param control_points: (numpy array)
    :param n_points: (int) number of points in the trajectory
    :return: (numpy array)
    """
    traj = []
    for t in np.linspace(0, 1, n_points):
        traj.append(bezier(t, control_points))

    return np.array(traj)

def bernstein_poly(n, i, t):
    """
    Bernstein polynom.
    :param n: (int) polynom degree
    :param i: (int)
    :param t: (float)
    :return: (float)
    """
    return scipy.special.comb(n, i) * t ** i * (1 - t) ** (n - i)

def bezier(t, control_points):
    """
    Return one point on the bezier curve.
    :param t: (float) number in [0, 1]
    :param control_points: (numpy array)
    :return: (numpy array) Coordinates of the point
    """
    n = len(control_points) - 1
    return np.sum([bernstein_poly(n, i, t) * control_points[i] for i in range(n + 1)], axis=0)

def line_bezier(visx, visy, control, mod="nothing"):
    vis = np.column_stack((visx,visy))
    path_x, path_y = np.array([]),np.array([])
    setting = {"nothing":[len(vis)-2, 1, 1], "start":[len(vis)-1, 0, 0], "end":[len(vis)-1, 1, 0], "both":[len(vis), 0, -1]}
    epoch = setting[mod][0]
    start = setting[mod][1]
    end = setting[mod][2]

    ## create a new list to hold the line segments
    line_collection=[]
    if len(vis) > 2:
        current_control = vis[0]
        for x in range(epoch):
            if x != (epoch-1):
                for y in control:
                    if y == control[0]:
                        mid_control = [(vis[x+start,0]+(vis[x+(start+1),0]-vis[x+start,0])*y), (vis[x+start,1]+(vis[x+(start+1),1]-vis[x+start,1])*y)]
                        plt.annotate(f"M{x}", mid_control)
                        bezier_line = calc_bezier_path(np.array([current_control,vis[x+start], mid_control]))
                        path_x = np.append(path_x, bezier_line.T[0])
                        path_y = np.append(path_y, bezier_line.T[1])

                        ## append the segment to the list
                        line_collection.append(bezier_line)
                        current_control = mid_control
                    else:
                        mid_control = [(vis[x+start,0]+(vis[x+(start+1),0]-vis[x+start,0])*y), (vis[x+start,1]+(vis[x+(start+1),1]-vis[x+start,1])*y)]
                        plt.annotate(f"M{x}", mid_control)
                        bezier_line = calc_bezier_path(np.array([current_control, mid_control]))
                        path_x = np.append(path_x, bezier_line.T[0])
                        path_y = np.append(path_y, bezier_line.T[1])

                        ## append the segment to the list
                        line_collection.append(bezier_line)
                        current_control = mid_control

            else:
                if mod == "end" or mod == "both":
                    bezier_line = calc_bezier_path(np.array([current_control, vis[x+(end+1)]]))
                else:
                    bezier_line = calc_bezier_path(np.array([current_control, vis[x+end], vis[x+(end+1)]]))
                path_x = np.append(path_x, bezier_line.T[0])
                path_y = np.append(path_y, bezier_line.T[1])
                ## append the segment to the list:
                line_collection.append(bezier_line)

    else:
        path_x, path_y = visx, visy
    return path_x, path_y, line_collection ## return the line_collection here

visx, visy = [1,2,10,15,20,25,21], [0,5,1,4,2,3,3]
control = [0.25,0.57]
path_x, path_y, line_collection = line_bezier(visx, visy, control,mod="end") ## init the line_collection

## iterate over list elements
for line_seg in line_collection:
    plt.plot(line_seg.T[0],line_seg.T[1],linestyle="solid",linewidth=5,alpha=.5,zorder=10)

# plt.plot(path_x, path_y) ## dont need to plot this now
plt.plot(visx, visy, "--o")
for xy in range(len(visx)):
    plt.annotate(f"P{xy}", [visx[xy], visy[xy]])
plt.show()

yields:


Prompts the user for a word and prints all words in the file containing all characters of the word in python

Question: For example, I have a list of words from in a file.(listed below)
aback

abacus

abandon

abandoned

logo

loincloth

loiter

loll

and some other more,a really big list of words! now the user can enter a word
for example "go", then it will show all words contain the charter 'g' and 'o', "go", "logo", "goo" , and so on.
And I have to make the file into a dictionary type first, I really have no idea, how to do it.
This is something I have done, I was trying to make all the words from the same letter go to together,
for example:
words = {'a': ['airport'], 'b': ['bathroom', 'boss', 'bottle'], 'e':['elephant']}

import operator
file = open("d1.txt","r")
words = {}
for line in file:
        line = line.strip()
        first_char = line[0]
        if first_char not in words:
                words[first_char] = []
                words[first_char].append(line)
sorted_words = sorted(words.items(),key = operator.itemgetter(1))

print(sorted_words)

user_input = str(input("Pleae enter a ward: "))
v1 = words[user_input]
print(v1)

Unfortunately, this is all I have done, can anyone help me out please!
Comment: Would not it be easier with regex?
Comment: I have to do it with dictionary, that is really sad. Also I am not good at it.....
Answer: That looks somewhat strange, but anyway it will be easier to do something like this
 word_to_search = 'gosh' # assume that this is user input
 letters_list = list(word_to_search)
 result = []
 for letter in letters_list:
   for word in file.read().split('\n'):  #here you choose separator by which your words splitted
     if letter in word:
       result.append(word)    #here you'll get a list of all words with matching letters

Note that there will be duplicates, to get rid of them you can just
  result = set(result)  #here you will get list of only unique words

If you want to go with dictionary
import string

alphabet = list(string.ascii_lowercase)
words_list = file.read().split('\n')
words_dict = dict((letter, dict()) for letter in alphabet)
for letter in alphabet:
  for word in words_list:
       if word.startswith(letter):
          words_dict[letter].append(word)

This will give you dict with alphabet letters as keys and lists of words as values
Hope you can figure out how to iterate over lists in your dict.
Hint: you can join values of dict and iterate over them
Comment: yeah, as you said, I got a KeyError: 'b', I tried to put one more loop in it, I am still trying it, thank for your hint btw, at least I have something to start it.:)!


Problema ao fazer um request do tipo POST

Question: Olá, estou tentando fazer um request do tipo POST com Bearer token para um endpoint. Quando realizo essa requisição pelo postamn ele retorna normalmente, mas quando é realizado pelo método que criei usando api RestTemplate do java, ele dá o erro "401 Unauthorized". Segue o código abaixo:
public <T> T sendRequestWithBody(String urlRequest, String jsonValue){
        HttpHeaders header = new HttpHeaders();
        header.set("Autorization", "Bearer  ".concat(getToken()));

        JSONObject jsonObject = new JSONObject();
        jsonObject.put("query", jsonValue);

        RestTemplate restTemplate = new RestTemplate();
        HttpEntity<?> request = new HttpEntity<>(jsonObject.toString(), header);

       ParameterizedTypeReference<T> typeRef = new ParameterizedTypeReference<T>() {
       };

       ResponseEntity<T> response = restTemplate.exchange(urlRequest, HttpMethod.POST, request, typeRef);

        return response.getBody();
    }

Alguém poderia dar uma ajuda? Obs: Fiz um método bem parecido com esse para pegar o token, e ele gera normalmente
Answer: Se o erro é 401 ele tem a ver com seu header.
Monte o cabeçalho da requisição (headers) dessa maneira:
String accessToken = "Bearer " + getToken();
MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
headers.add("Authorization", accessToken);
headers.add("Content-Type", MediaType.APPLICATION_JSON.toString());

Da forma que você está montando o json que está enviando, ele ficará assim:
{'query': 'valor na variavel json value'}
É esse mesmo o objetivo?
Geralmente utilizo o método postForObject ao invés de exchange. Acho mais simples.
Da documentação:
exchange

More generalized (and less opinionated) version of the preceding
methods that provides extra flexibility when needed. It accepts a
RequestEntity (including HTTP method, URL, headers, and body as input)
and returns a ResponseEntity.
These methods allow the use of ParameterizedTypeReference instead of
Class to specify a response type with generics.

postForObject

Creates a new resource by using POST and returns the representation
from the response.

Vou colocar um exemplo completo aqui usando postForObject. Realizei um teste aqui e está funcionando.
    @RequestMapping("/teste")
    @ResponseBody
    public String testeRestClient() {

        String accessToken = "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZWxsZXJJZCI6Niwic2NvcGUiOlsibWFy";
        MultiValueMap<String, String> headers = new LinkedMultiValueMap<>();
        headers.add("Authorization", accessToken);
        headers.add("Content-Type", MediaType.APPLICATION_JSON.toString());

        String urlRequest = "http://localhost:8080/books";

        JSONObject jsonObject = new JSONObject();
        jsonObject.put("name", "titulo 2");
        jsonObject.put("author", "fulano de tal");
        jsonObject.put("price", new BigDecimal(30));

        RestTemplate restTemplate = new RestTemplate();
        HttpEntity<String> entity = new HttpEntity<String>(jsonObject.toString(), headers);
        String result = restTemplate.postForObject(urlRequest, entity, String.class);
        return result;
    }

Comment: Olá. Deu certo dessa forma!


Form Validation. Ensure Placeholder is not submitted value

Question: I have a Drupal registration form with a placeholder set on a first name field. I am trying to get the form to check so that if the placeholder value is the same as the submitted value it will give an error asking them to be sure to enter their first name. Here is my code:
function mytheme_form_alter(&$form, &$form_state, $form_id){
  if ($form_id == 'user_register_form'){
    $form['field_first_name']['und'][0]['value']['#default_value'] = t('First Name');

    $form['field_first_name']['und'][0]['value']['#attributes']['onblur'] = "if (this.value == '') {this.value = 'First Name';}";
    $form['field_first_name']['und'][0]['value']['#attributes']['onfocus'] = "if (this.value == 'First Name') {this.value = '';}";
    // Alternative (HTML5) placeholder attribute instead of using the javascript
    $form['field_first_name']['und'][0]['value']['#attributes']['placeholder'] = t('First Name');

    $form['#validate'][] = 'parenting_validate_register_form';

    $form['actions']['submit']['#value'] = t('JOIN US FREE');
  }
}

function mytheme_validate_register_form(&$form, &$form_state) {
  if ($form_state['values']['field_first_name'] == 'First Name') {
    form_set_error('firstname', t('Please enter your first name.'));
  }
}

So the problem I am having is that with the above, every submission is throwing the error. Even if something other than "First Name" is entered. For some reason if ($form_state['values']['field_first_name'] == 'First Name') is returning true no matter what the "field_first_name" value is.
I've looked into using https://drupal.org/project/clientside_validation, but it just seems like overkill for what I'm trying to do.
Thanks for any help.
Comment: Setting the default value of the field is not the right way to display a placeholder. This is bound to cause problems. The placeholder attribute (which you appear to have set) is the cleanest method. A common alternative is to position the field label using eg. the jQuery Overlabel plugin. There are also several modules that provide placeholder support, eg https://drupal.org/project/form_placeholder
Comment: I'll try that module. It looks good.
Comment: The module https://drupal.org/project/form_placeholder worked great! Thank you!
Answer: In your code these two lines may make issue. remove/comment these lines and try.
$form['field_first_name']['und'][0]['value']['#attributes']['onblur'] = "if (this.value == '') {this.value = 'First Name';}";
$form['field_first_name']['und'][0]['value']['#attributes']['onfocus'] = "if (this.value == 'First Name') {this.value = '';}";

Comment: Indeed, these two lines of code seem to be causing the issue. The problem is that I need to support older browsers as well and the html5 placeholder doesn't always work. @Alfred gave me a good solution in the module form_placeholder that I will try as soon as I can.


Number of connected components of two compact level sets close enough

Question: Let $n > m$ be positive integers.
Let $h:\mathbb{R}^n \to \mathbb{R}^m$ be a smooth map. Let $u \in \mathbb{R}^m$ be a regular value of $h$. Let $\epsilon \in \mathbb{R}^m$. We assume that $h^{-1}(u)$ and $h^{-1}(u+\epsilon)$  are compact.
I wonder if $\epsilon \neq 0$ and sufficiently small then $h^{-1}(u)$ and $h^{-1}(u+\epsilon)$ have the same number of connected components?
Comment: Interesting question! My first guess would be yes: $h^{-1}(u+\varepsilon)$ is a smooth submanifold for $\varepsilon\geq 0$ small enough, and if there is a collapse/join of connected components, the smoothness should be broken at that value. Right now I don't see how to even start formalizing this though.
Answer: If $u$ is in the interior of the set of regular values (eg if $h$ is closed or – I think – if we replace $\mathbb{R}^n$ with a compact Riemannian manifold), yes. Indeed, assume that $B(u,2|\epsilon|)$ contains only regular values for $h$.
Let $\chi:\mathbb{R}^m \rightarrow [0,\infty)$ be a smooth compactly supported function, such that $\chi(B(u,|\epsilon|))=1$, and $\chi$ is zero outside a compact open subset of $B(u,2|\epsilon|)$.
Define, for each $x \in \mathbb{R}^n$ such that $h'(x)$ is onto, $L_x=h'(x)^T \circ (h'(x) \circ h'(x)^T)^{-1}$ a Moore-Penrose inverse, and $x \longmapsto L_x$ is smooth, $h'(x)L_x=\mathrm{id}_{\mathbb{R}^m}$. Consider now the following smooth vector field $X$ on $\mathbb{R}^n$: $X(x)=\chi(h(x))L_x(\epsilon)$.
One can check that under its flow $\phi^t$, if $1 \geq t \geq 0$ and $h(x)=u$, $h(\phi^t(x))=u+t\epsilon$; if $0 \geq t \geq -1$, $h(x)=u+\epsilon$, $h(\phi^t(x))=u+(1+t)\epsilon$.
Thus $\phi^1$ is a diffeomorphism between $h^{-1}(u)$ and $h^{-1}(u+\epsilon)$. QED


If statement using string

Question: Okay, asked a question a while ago without providing all my code, dumb idea.  Here it is.
I'm trying to use an if statement with a string to get a cost.  No matter what I do, my cost is getting returned as 0.  See class CarRental, method getCost.
I was told I could change it up and use a switch, which is fine, but I want to understand why what I'm doing isn't working anyway, for the sake of knowledge.
Update: Tried changing it to a switch, same results.
import java.util.*;

public class UseCarRental {

    public static void main(String[] args) {
        Scanner input = new Scanner(System.in);
        System.out.println("Thank you for choosing ICT car Rentals\n"
                + "Pleae enter your full name:");
        String renterName = input.nextLine();
        System.out.println("Please enter your zip code:");
        int renterZipcode = input.nextInt();
        input.nextLine();
        System.out.println("Please enter the size car you would like:\n"
                + "economy\n"
                + "midsize\n"
                + "fullsize\n"
                + "luxury?");
        String carSize = input.next();
        System.out.println("How many days do you wish to rent this?");
        int rentalDays = input.nextInt();

        if (carSize.equals("luxury")) {
            System.out.println("Will you be wanting a chauffer (y or n");
            String chauffer = input.next();
            LuxuryCarRental rentIt = new LuxuryCarRental(renterName,
            renterZipcode, carSize, rentalDays,chauffer);
            rentIt.display();
        } else {
            CarRental rentIt = new CarRental(renterName, renterZipcode,
            carSize, rentalDays);
            rentIt.display();
        }

    } //  end main method
}  //end class UseCarRental

class CarRental {

    private int days;
    private int zip;
    private double cost;
    private String size;
    private double total;
    private String name;

        CarRental(String renterName, int renterZipcode, String carSize, int rentalDays){
            this.days = rentalDays;
            this.zip = renterZipcode;
            this.name = renterName;
            this.size = carSize;
        }

        double getCost(){
            if(size.equals("economy")){
                cost = 29.99;
            }
            if(size.equals("midsize")){
                cost = 38.99;
            }
            if(size.equals("fullsize")){
                cost = 43.50;
            }
            return cost;
        }

        void display(){
            System.out.println("Thank you for using our service.");
            System.out.println("Your order is as follows:");
            System.out.println("Name: " + name);
            System.out.println("Zip code: " + zip);
            System.out.println("Car size: " + size);
            System.out.println("Cost per day: " + cost);
            System.out.println("Days requested: " + days);
            total = days * cost;
            System.out.println("Total cost: " + total);
            System.out.println("If any of the above information is incorrect, too bad bud, because it isn't.");
        }

}

class LuxuryCarRental extends CarRental {

    private int chauffer = 200;
    private int days;
    private int zip;
    private double cost;
    private String size;
    private double total;
    private String name;

    LuxuryCarRental(String renterName, int renterZipcode, String carSize, int rentalDays, String chauffer){
        super(renterName, renterZipcode, carSize, rentalDays);
        this.days = rentalDays;
        this.zip = renterZipcode;
        this.name = renterName;
        this.size = carSize;
    }

    @Override
    void display(){
            System.out.println("Thank you for using our service.");
            System.out.println("Your order is as follows:");
            System.out.println("Name: " + name);
            System.out.println("Zip code: " + zip);
            System.out.println("Car size: Luxury");
            System.out.println("Cost per day: " + cost);
            System.out.println("Days requested: " + days);
            System.out.println("Chauffer cost: " + chauffer);
            total = days * cost + chauffer;
            System.out.println("Total cost: " + total);
            System.out.println("If any of the above information is incorrect, too bad bud, because it isn't.");
        }
}

Comment: I can also confirm that changing it to a switch didn't fix the issue, I am still getting 0 as a cost when displaying.  There must be some other issue, but I can't see what it is.
Comment: "Please enter the size car you would like" What would be your input there?
 Please read all the comments and answers before asking another question with the same ROOT cause.
Comment: Hint:  Insert calls to `System.out.println` to output values at various places in your code.  Eg, insert one just ahead of the first `if` statement to output "size":  `System.out.println("Size = " + size);`.
Answer: You are not invoking the getCost method in your code.
Answer: Your display method is getting the "cost per day" from an instance variable called cost in the LuxuryCarRental.  None of your code assigns anything to that field, so (naturally!) it is always zero.
Comment: If I run the program, I get 0 even if I choose one of the other three car sizes.
Comment: @E.Watson - My answer explains that.  You get zero *because* the field always contains zero.  It always contains zero *because* you didn't assign anything to it.  Your `getCost()` method is irrelevant *if you don't use it*.

If statement using string