Getting acquainted with preimage orientation

Question: I'm trying to do this problem:
Let $f: S^2 \to \mathbb{R}^3$ given by $f(x,y,z)=z$. For the regular values $-1<t<1$, find the orientations of $f^{-1}(t).$
The hint is to find a positively oriented vector. The preimage orientation as follows:
If $f:X \to Y$ is transverse to $Z$, then, the orientation of $S =f^{-1}(Z)$ is given by the following relations:
$$N_x(S; X) \oplus T_x(S) = T_x(X)$$
and
$$df_x(N(S;X)) \oplus T_{f(x)}(Z) = T_{f(x)}(Y).$$
But I just don't know what to do with this.
For instance, I can see that $T_x^(X)= (a,b,t)^\perp$ for $x=(a,b,t)$ with $a^2+b^2=1-t^2$, but I don't know what the orientation of this manifold is, or how to see it.
Another general question I have is: how do we talk about whether a basis is positive or negative when talking about 2 dimensional subspaces of $\mathbb{R}^3$? We have two 3-dimensional vectors, and I don't see how we can say something about a linear map with positive determinant.
Any help with this and the above problem would be much appreciate.
Comment: You say that $f: S^2 \to \mathbb{R}^3$, but then look for regular values $t \in (-1,1) \subset \mathbb{R}$. Does $f$ actually go to $\mathbb{R}$?
Comment: Also, do you have an explicit expression for $f$? Or is it a general function? Also, what are $a$ and $b$?
Comment: Apologies! I forgot to define $f$.
Answer: Second question first:
An orientation on a linear vector space can be given by a choice of ordered basis to be "positive". The standard orientation on $\mathbb{R}^n$ is given by the standard ordered basis $\{e_1,...,e_n\}$. If you are given a $2$-dimensional linear subspace $H \subset \mathbb{R}^3$, it does not automatically come with an orientation. To talk about "positive" or "negative" bases of it, you first need to choose such a basis. In some cases, an orientation on $H$ may be implicit in however it was given to you (e.g. as the preimage $f^{-1}(c)$ for some linear map $f: \mathbb{R}^3 \to \mathbb{R}$.)
Okay, first question. Fix $t \in (-1,1)$, let $C = f^{-1}(t) \subset S^2$ be the latitude line, and $p = (a,b,t) \in C$. Let's start by taking stock of all the orientations we know.
The image $\mathbb{R}$ carries the standard orientation, with $\{e_1\}$ a positive basis at every point.
$\mathbb{R}^3$ carries the standard orientation.
$S^2 \subset \mathbb{R}^3$ is probably assumed to carry the orientation given by an outward normal vector $n$ (this is standard). The equation
$$N_p(S^2) \oplus T_p(S^2) = T_p(\mathbb{R}^3)$$
tells us that an ordered basis $\{v_1,v_2\}$ for $T_p(S^2)$ is "positive" if the combined ordered basis $\{n, v_1, v_2\}$ is positive for $\mathbb{R}^3$. For example, one positive basis for $T_p(S^2)$ at $p = (a,b,t)$ (away from the poles $t = \pm 1$) is
$$v_1 = (-b, a, 0), \quad v_2 = \left(-at,-bt,a^2 + b^2\right).$$
Now we turn to the curve $C$. The second relation you give says
$$df_p(N(C; S^2)) \oplus T_t(\{t\}) = T_t(\mathbb{R}).$$
Since $T_t(\{t\})$ is trivial ($\{t\}$ is $0$-dimensional), this is just an isomorphism
$$df_p(N(C; S^2)) \cong T_t(\mathbb{R}).$$
Additionally, the restriction of $df_p$ to $N(C;S^2)$ is an isomorphism onto its image, so we actually have an isomorphism
$$df_p: N_p(C; S^2) \to T_t(\mathbb{R}).$$
A "positive" basis for $N_p(C;S^2)$ is one that corresponds, via this isomorphism, to a positive basis for $T_t(\mathbb{R})$. For example, we might take the previously defined
$$v_2 = \left(-at,-bt, a^2 + b^2\right)$$
as a positive basis vector, since $df_p(v) = (a^2 + b^2)e_1$ is a positive basis vector for $T_t(\mathbb{R})$.
Now return to the first relation.
$$N_p(C;S^2) \oplus T_p(C) = T_p(S^2)$$
tells us that a vector $w \in T_p(C)$ forms a positive basis iff $\{v_2, w\}$ forms a positive basis for $T_p(S^2)$. For example, we might take $w = -v_1$, since we know that $\{v_1, v_2\}$ is a positive basis.
That's it! $C = f^{-1}(t)$ is oriented in the $-v_1 \in T_p(C)$ direction at each point. In plain English, $C$ is oriented going clockwise around the positive $z$-axis. Or, thinking of $S^2$ as a globe and $C$ as a latitude line, the preimage orientation runs east to west.


Diagnosing piano hand injury: pain when stretching while holding some notes

Question: I am learning to play the arrangement of Skyfall in the RSL Grade 3 book.  I have a small-ish right hand. As a consequence of trying to reach the chords in the bridge section I have strained my fingers.  The bar (measure) requires me to play E♭ on 1, G on 2, A♭ on 3 for the whole bar and melody of C D and E♭ with my 5 finger.  I could not find  pedalling to help as that spoilt the LH F notes two octaves below (too far away).

The tenderness lies between the 3 and 4 finger knuckles on the side of the 3 finger knuckle.  I searched NHS Direct for treatment and studied Gray's Anatomy for clues.
The injury does not feel severe but I am worried that it will get worse if I do not get treatment. I have not lost any mobility and the hand is not swollen.  I do feel a little heat.
I stopped playing RH parts for a few weeks and tried a Bedford split (strapping 3 and 4 finger together) that I constructed.  It hasn't significantly improved so I returned to practising the other pieces that are not so demanding of my RH.
I cannot decide whether it is a thumb dorsal muscle or tendon / tendon sheath injury.  I think I need diagnosis and treatment from a specialist that understands piano playing injuries, based somewhere in the South East of England.
Any ideas whom I might consider approaching?  I guess I am looking for an osteopath specialising in hand anatomy with a piano I suppose.  I will take your suggestions to my NHS GP and agree the best way forward for a quick recovery.
Comment: Have you read and understood the exam syllabus correctly? Page 7: "- At all grades, if any two handed/open voiced chords notated in the pieces are not achievable, candidates may either reduce the amount of notes in the voicing or play as a spread chord if stylistically appropriate. - At all grades, candidates are free to execute voiced chords notated in the pieces with either hand or combination of hands. - In any Level 3 piece containing chord symbols, voicings may be presented as deemed appropriate."
Comment: I added a picture I sent to my piano teacher a couple of weeks ago. She asked the same question and made the same point.  I have thoughts on how to tackle this, probably shortening or not playing the RH Eb on the 1 finger (thumb).  The errata for Skyfall concerns an earlier bar (measure).  The correction was incorporated into the Revision 1 book, from which the picture was taken.
I found the point about dropping notes you cannot reach too late.  Dropping notes and pedalling messed up the sound too much, so I'll work on it with my teacher when safe to do so.
Comment: I don't know anything about the NHS (I'm American), so forgive me for saying you should go to a generalist doctor who hopefully has the knowledge and authority to refer you to a specialist if you really need one. The fact that you've injured yourself playing piano doesn't change human anatomy nor how muscles and tendons work and heal.
Comment: "The bar (measure) requires me to play Eb on 1, G on 2, Ab on 3 for the whole bar and melody of C D and Eb with my 5 finger" - OUCH!!! Either you should sue the RSL for creating inappropriate arrangement for a relative beginner, or (possibly more likely?) you have misinterpreted something. From your description that looks horrible for anybody to play. Can you attach a picture of the score?
Comment: Also there is a correction to the Skyfall score - see https://www.rslawards.com/shop/grade-books/piano/grade-three#errata.
Answer: If it hurts, stop!  General principle.
The problem is usually too much, too soon.  Try rolling chords you can't reach.  In any case, the Philip Isidor piano technique book has some good exercises you can practice to SLOWLY increase your flexibility and strength.
If you really did hurt yourself, don't keep re-injuring it.  Use some tiger balm and rest.


Counting in an anacrusis

Question: We have a big band number (in 3/4 time), where the chart has the "1st" bar (an anacrusis) written as a one beat rest, for the drums to play a break there. Is it better to count in with 1-2, or 1-2-3-1-2? It's a slow number (Natural Woman), so the former hardly gives enough time for a feel of the pulse, but the latter seems too long.
Comment: Give the band the count-in they need and want. If they need a long count-in to play in time but only want a short count-in, then you should discuss it with *them*.
Comment: What about 3-1-2?
Comment: There are various approaches to this kind of problem, but there is no 'correct' answer, and which solution is best in each particular case is a matter of opinion. Every band is different.
Answer: Who does the count? The drummer would be best-placed for this. All it should need is 'nod, nod, fill'.
I'd be worried with any band that needed two bars to grasp the timing.
Bear in mind, a tight big band with a conductor will get the upbeat only. (Think Whiplash;)
I've used tactics from the lead mic like - left arm out, right arm out, both arms up, to signal a count without looking like I'm conducting. I hate yelling across a stage & I hate drummers clicking sticks. I like it to always look like the band are mind-readers.  I worked in one band where transitions were already rehearsed, so there were no counts, everybody just felt it.
Another, slightly cheesy, trick would be to use the title as the count, as it's announced to the audience. Do it à tempo.
I remember one band, many many years ago. We used to do Cockney Rebel's Make Me Smile (Come up & See Me), which has a repeating chorus at the end that has a one-bar break each time. We would use a snare on the 4 to signal we were going round again "Ooooooh, ooh la la la" etc but each time the break would get longer. Eventually, the band would all wander off the mics, look like they were going to grab a drink of their beer, whatever, then the drummer would hit the snare & everybody had to dive across the stage to reach their mics & also play the 1 in time. This gag could be repeated quite a number of times if the audience were really getting the joke & enjoying the song.
Comment: I'd go so far as to say Whiplash is the only movie to ever get the 'feel' of being in a band right - even if there's still a fair amount of artistic license taken. I usually *hate* movies about music. The Buddy Rich ref above is partly because the kid in the movie wants to be as good as Rich & the conductor [Fletcher] is about as pleasant as Rich ;)
Comment: In a big band, there's the conductor, who does the job. Unsuitable for the drummer, he's at the very back of the band, so no-one would see him clearly. I quite like the stick-click, but it's not feasible when there's a conductor, really!
Comment: Did you see 'Whiplash' - that's how much notice you get from the conductor;) tbh, I've only ever worked in one big band, & that was decades ago. As I recall, he would give us more than Fletcher, but certainly not two bars. A friend of mine was Buddy Rich's last ever pianist & arranger. He used to complain vociferously that Rich expected everybody to simply be a mind reader.
Comment: I suppose we could compare with a 100 piece orchestra, which usually only gets the up-beat! Certainly not a whole bar. No, missed Whiplash.
Comment: @Tim is the conductor speaking the count, gesturing, or both?  Is there only one drummer playing on the anacrusis or multiple people?
Comment: @phoog - often speaking the count. Only one drummer/kit.
Comment: Although your comment about just giving an upbeat is true in theory, after all it works for many other musical situations, in practice it is rarely to never done in actual big band situations. No comment on the movie "Whiplash". ;)
Answer: That's the problem, you have a conductor! (Just kidding, not totally…)
I've played with many big bands and few have conductors. Lead alto is in charge of cutoffs and untimed chords and if something needs conducting for part of a song usually the pianist will handle it. With the bands that do have a conductor they will establish enough of a beat or tempo to let everyone know what it is. Some will silently show the pulse of the song with an index finger for several beats before counting, either verbally or by silently conducting, which I think is the best method. If they do that first then a 1-2… is more than enough.
If there is no conductor usually the drummers are in charge of counting off. It is not necessary to see them, they usually click the sticks at tempo a few times then do a verbal count off. It's pretty foolproof and is the same concept as what I mentioned with the conductor, only sonically. This is also sufficient for a one bar count off in a slow 3/4 as long as everyone knows what to expect, since in jazz it is more common to use a 2 bar count off in 3/4 time.
It is important for the band to know beforehand how a tune will be started in order to avoid a train wreck out of the gate. This should be penciled into players' parts, i.e. "two bar count off" or in the case of say, 4/4 with a 2 beat anacrusis, "6 beat count off" or "1,2,3,4,1,2". When in doubt the conductor or counter should just call out what they are going to do.
Comment: Thanks for your (usual) insightful answer. I'm on keys for this band - at the back of the stage. I can understand that for a 5/6/7 piece, keys can be placed where the rest can see, but not with an 18-25pce band! Also, I have enough on my plate (music stand) for the sometimes 8/9/10 sheets of charts, so I'd hate to have the whole score there to follow and conduct. Drums, along with bass and guitar and keys, are always at the back, so it's fair and square up to the conductor to lead. Last big band had the drummer as conductor, seated up front, so was like you say, a good situation. Not here...+1
Comment: @Tim There's nothing wrong with having a conductor in a large situation. There's also nothing wrong with a conductor silently showing the beat before the agreed upon count off, it just makes the tune start right in the pocket.
Answer: 1,2 should be sufficient.   But mark in the parts what you're GOING to count, and stick to it!

Counting in an anacrusis


Exception handler works for doGet(), but not for doPost()

Question: I have a servlet api where I used to throw my own exceptions from servlets level
When I throw exception from doGet method everything works fine and exception handler catches and processed my exception. the problem ocures when I throw exception from doPost method. in this case unfortunatelly I never see error page
web.xml
<error-page>
    <exception-type>java.lang.Throwable</exception-type >
    <location>/ErrorHandler</location>
</error-page>

exception handler
@WebServlet("/ErrorHandler")
public class ErrorHandler extends HttpServlet {

    private final Logger logger;

    public ErrorHandler() {
        logger = Logger.getLogger(ErrorHandler.class);
    }

    @Override
    public void doGet(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws IOException, ServletException {
        Throwable throwable = (Throwable) httpServletRequest.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
        logger.error("occurred exception: ", throwable);
        httpServletRequest.getRequestDispatcher("/error.jsp").forward(httpServletRequest, httpServletResponse);
    }
}

Servlet
@Override
public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws IOException {
    throw new UserException("error message");
}

Answer: Add to your ErrorHandler
@Override
public void doPost(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws IOException, ServletException {
    Throwable throwable = (Throwable) httpServletRequest.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
    logger.error("occurred exception: ", throwable);
    httpServletRequest.getRequestDispatcher("/error.jsp").forward(httpServletRequest, httpServletResponse);
}

To avoid code duplication consider creating third method
private void processError(HttpServletRequest req, HttpServletResponse resp) throws IOException, ServletException {
    Throwable throwable = (Throwable) httpServletRequest.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
    logger.error("occurred exception: ", throwable);
    httpServletRequest.getRequestDispatcher("/error.jsp").forward(httpServletRequest, httpServletResponse);
}

and invoke it from both doGet() and doPost()
@Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException, ServletException {
    processError(req, resp);
}

@Override
public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException, ServletException {
    processError(req, resp);
}


pandas DataFrame: replace nan values with median of corresponding column

Question: I have some NaN values in my pd dataframe and I wish to replace them with the median value of the column. I am aware that this is very similar to:
pandas DataFrame: replace nan values with average of columns
However when I try the equivalent:
df = df.fillna(df.median())

I get the following error:

Python\Python37\site-packages\pandas\core\generic.py:6287: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._update_inplace(new_data)
Python\Python37\site-packages\pandas\core\frame.py:4244: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  **kwargs

Any advice is appreciated.
Thanks
Comment: If you want to update `data`, do `data.iloc[:,0:-1] = df.fillna(df.median)`. If not, you should include an [MRE](https://stackoverflow.com/help/minimal-reproducible-example).
Comment: `A value is trying to be set on a copy of a slice from a DataFrame`: is your `df` a part of some bigger dataframe?
Comment: do `df = df.copy()` above that line. You can read extensively about it in https://stackoverflow.com/questions/20625582/how-to-deal-with-settingwithcopywarning-in-pandas
Comment: Yes, I extracted it from a larger one. How can I extract it without getting this error?
I used this initially:
df = data.iloc[:, 0:-1]
Comment: agreed with @ALollz, do a copy of your dataframe. Additional resource: https://www.dataquest.io/blog/settingwithcopywarning/
Comment: @ALollz that did not work unfortunately.
Comment: Perfect, that worked. Thanks!
Answer: Try df.fillna(df.median(axis=1, skipna=True))


url() not providing links to correct language alias using i18n in D7

Question: We have a site that features two languages, EN (primary/default) and JA.  On one content page, we link to another node using it's nid: url('node/123').  In EN, this returns the alias and not the node based url.  When in the JA language, this returns a url like /ja/node/123 which then shows text in the EN variant when we expect JA.  There are aliases set for both EN and JA, which both point to the same desired URL like: /products/new-fancy-product and their respective node id urls.  In EN, url returns this alias correctly.
Also, if I change /products/new-fancy-product to /ja/products/new-fancy-product in the address bar, I get the correct JA version of the page as I want.
Why is url not returning me the correct alias instead of the node id path, which then just shows EN content?
Answer: Use l() function to generate a link.

Formats an internal or external URL link as an HTML anchor tag.
This function correctly handles aliased paths and adds an 'active'
  class attribute to links that point to the current page (for theming),
  so all internal links output by modules should be generated by this
  function if possible.
Comment: This just appears to wrap a call to `url` which isn't working.  How will this help?
Comment: @Chris have you actually tried using l() instead of url()?
Comment: Yes, it returns the sam thing as `url` except with link fully built and ready for use in html: `/ja/node/123`.  This still gives me the english version, and doesn't properly resolve the alias which should be `/ja/products/new-fancy-product`


How to append a string in excel formula for a list (DropDown)

Question: I have a drop-down in sheet2 at cell F6 which contains a list from sheet1 with formula =Sheet1!$1:$1 and it gives me a drop with list of values like Header1,Header2,Header3,Header4,...
What I want to achieve is these values should be like [Header1],[Header2],[Header3],..
Is there some way to achieve using excel formulas or VBA (apart from array with for loop).
I'm searching for something like "["&=Sheet1!$1:$1 &"]".
Comment: You can make another column / row which has the data in the format you need then you can refer to that range and make a dropdown list from that.
Comment: @newguy You mean manually ?
Comment: You can use a simple excel formula of string concatenation.
Comment: @newguy can u help me with the formula suppose i have n no of columns in a row.
Comment: If you only want it to look like that then number format `"["@"]"` may be an option :P
Comment: @newguy can you share me a example how can be loop through all cells of  Row 1 in excel.. I tried EntireRow, Rows etc. nothing seems working :(
Comment: @DirkReichel if you can explain a bit :D
Comment: just right-click the cell and format cells (or hit ctrl+1) -> number -> custom -> enter `"["@"]"` and the cell will look like `[Header1],[Header2]...` but the value still is `Header1,Header2...`
Comment: @DirkReichel lolzz, good one but won't gonna help me coz I want these values in dropdown later that means I need a in-memory replica with those `[Header1]...` for dropdown without disturbing the actual one.
Comment: with the build-in drop down you either need a copy or enter it manually... data-validation only accepts ranges or a direct list... (I was hoping that cheating is an option)
Answer: Done it with below code (in case if anyone need a reference for How to read a Row in excel VBA)
    Dim objSheet As Worksheet
    Set objSheet = Sheets(1)
    Dim intLastCellIndexInRow As Integer
    intLastCellIndexInRow = ActiveCell.SpecialCells(xlLastCell).Column
    Dim i As Integer
    Dim strRowValue As String

    For i = 1 To intLastCellIndexInRow
      If objSheet.Cells(1, i) <> "" Then
        strRowValue = strRowValue & ",[ " & objSheet.Cells(1, i) & "]"
      End If
    Next


Application of Bayes Ball in the following example

Question:
As shown in the image, Murphy explains that if 5 is given, then 2 and 6 are conditionally independent (CI), because paths 2-5-6, 2-4-7-6 and 2-1-3-6 are inaccesible. However, couldn't the information travel via the path 2-5-3-6? Since 5 is the child of 2 and 3, if 5 is known, then the information is reflected from 2 to 3 and then transmitted to 6? If there is anything wrong with my reasoning, I would appreciate if someone can point out where I went wrong. Thanks!
Answer: You are correct that Murphy's assertion that $x_2 \perp x_6 \mid x_5 $ is incorrect. As you point out, observing $x_5$ activates the v-structure $2 \rightarrow 5 \leftarrow 3$, so the path $2 - 5 - 3 - 6 $ is active.
Furthermore, since $x_1$ is unobserved, the path $2 - 1 - 3 - 6 $ is also active! Since there are two active paths between $x_2$ and $x_6$, they are not d-separated and thus are not conditionally independent given $x_5$.


$\int_K |u_n-u|^{p-\epsilon} \to 0$ implies that $\int_K|u_n-u|^p \to 0$

Question: Let $K$ be compact set on $\mathbb{R}^n$ with non-empty interior. Assume that $p \geq 2 $ and that it is given a sequence of measurable functions $\{u_n : K \to \mathbb{R}\} \subset L^{p-\epsilon}(K)\cap L^p(K)$, for each $\epsilon > 0$, such that there is $u\in L^p(K)$ such that
$$\lim_{n\to\infty}\int_K|u_n-u|^{p-\epsilon} = 0.$$
My question is: is it true that
$$\lim_{n\to\infty}\int_K|u_n-u|^p = 0?$$
I believe so. These are my thoughts: take $\left\{\dfrac{1}{n} : n > 0\right\}$ and consider the function
$$\Phi : \bigcup_{n\geq 1}L^{p-\frac{1}{n}}(K)\cap L^ p(K)\times \bigcup_{n\geq 1}[p-\frac{1}{n},p]\to \mathbb{R}$$
$$(f,q) \mapsto \|f\|_q.$$
I think that since $\mu(K) < \infty$ then $\Phi$ is continuous, so the result should follows.
Is this correct?
Answer: This is false. Consider $K = [0,1]$ and let $u_n = n 1_{[0,n^{-p})}$. Then
$$\int_K |u_n|^{p-\varepsilon} = n^{-\varepsilon} \to 0$$ for any $\varepsilon > 0$
but $$\int_K |u_n|^p = 1.$$
Comment: Nice answer. Do you know a way to fix the statement in order of it be true?


Transformar DD/MM/YY em timestamp PHP

Question: Preciso que o usuário informe uma data e essa data seja enviada ao banco de dados como timestamp, exemplo
Inserir: 25/12/17 no campo
Quero que chegue ao banco de dados como 1514160000 que significa "Mon, 25 Dec 2017 00:00:00 GMT"
Não tentei nada, porque não sei o que e como cada função funciona no PHP
Agradeço qualquer dica
Answer: http://php.net/manual/pt_BR/function.strtotime.php
<code>$timestamp = strtotime('12/25/17');
</code>
Perceba que eu inverti a posição do dia e mês, é o padrão norte americano invertido de ser. E para fazer isso:
<code>$data = '25/12/17';
$dataParcial = explode('/', $data);
$data =  $dataParcial[1] . '/' . $dataParcial[0] . '/' . $dataParcial[2];
$timestamp = strtotime($data);
</code>
Como @GabrielHeming apontou nos comentários, strtotime vai reconhecer uma data no formato m/d/y na documentação da função você poderá verificar diversos outros formatos possíveis!
Comment: A sua resposta está com o formato correto (m/d/y), mas trará um resultado inesperado para o formato (d/m/y) que o AP possui.
Comment: Obrigado por pontuar @GabrielHeming. Já estava preparando a edição :)
Comment: Perfeito! Obrigado irmão
Answer: <code>$dateTime = DateTime::createFromFormat('d/m/Y','20/10/2017');

$timestamp = $dateTime->getTimestamp();
</code>
Dessa maneira, você pode criar um objeto DateTime, a partir de qualquer formato.
https://secure.php.net/manual/pt_BR/datetime.createfromformat.php
Answer: Você pode utilizar o strtotime(), exemplo:
<code>$timestamp = strtotime(str_replace('/', '-', '08/11/2017'));
</code>
Observando os Formatos de Data e Hora Suportados

Utilizando strptime()
<code>$a = strptime('08/11/2017', '%d/%m/%Y');
$timestamp = mktime(0, 0, 0, $a['tm_mon']+1, $a['tm_mday'], $a['tm_year']+1900);
</code>

Transformar DD/MM/YY em timestamp PHP


Get last viewed index with Scrollify

Question: I'm using scrollify to swap out fixed position components, and adding/removing classes to animate the transition. I have everything working as I'd like as you progress forward through the scroll, but when you scroll up to see a previous section, its not removing the previous class, and the previous section now animates in behind it.
I don't think the prev method will be of use, because it doesn't call the previous section your were at, only the previous section assuming you are always moving forward.
Here is my code below, you can move forward just fine, but trying to go backwards presents a problem as the new current section will load behind the previous one, and the previous one will still be visible.
jsfilddle here
<code>var wrapper = $('.wrapper');
var currentPosition = 0;

$(wrapper).each(function(index) {
    if (currentPosition != index) {
        $(this).css('opacity', 0);
    } else if (currentPosition == index) {
        $(this).css('opacity', 1);
    }
});

$(function() {
    $.scrollify({
        section: ".wrapper",
        scrollSpeed: 700,
        setHeights: false,
        after: function(index, sections) {
            var prevWrapper = $.scrollify.current().prev();
            var currentWrapper = $.scrollify.current();
            var nextWrapper = $.scrollify.current().next();

            $(prevWrapper).removeClass('wrapper-enter').addClass('wrapper-leave');
            $(currentWrapper).removeClass('wrapper-leave').addClass('wrapper-enter');
        },
    });
});
</code>
Answer: The issue is depending on scrolling up or down your "next" or "prev" may not actually be what you think so you are hiding classes incorrectly.
What you could do as scrollify doesn't have an option to detect scroll direction is to create a super quick variable that will detect if you are scrolling up or down. Then simply update your classes then in the after function.
<code>$.scrollify({
        section: ".wrapper",
        scrollSpeed: 700,
    setHeights: false,
        after: function(index, sections) {
    var prevWrapper = $.scrollify.current().prev();
            var currentWrapper = $.scrollify.current();
            var nextWrapper = $.scrollify.current().next();
            let elem = null;

      // Add wrapper-enter to current element.
      $(currentWrapper).removeClass('wrapper-leave').addClass('wrapper-enter');
            if(lastIndex < index) {
          // Scolled down if lastIndex < index
          elem = prevWrapper;
        } else {
          // Scrolled up if last index > index
          elem = nextWrapper;
      }
      $(elem).removeClass('wrapper-enter').addClass('wrapper-leave');
      lastIndex = index;
        },
</code>
In the above I simply created a variable to track the last index.
Here is a working fiddle: https://jsfiddle.net/k1e6x79f/
Comment: That makes a lot of sense. Thanks for this solution.

Get last viewed index with Scrollify


How to programmatically set node field values.

Question: I have a content type.  I have firstname, lastname and email address fields.  I need the default value for each field to be the firstname, lastname and email address of the user profile.  Also, I need to prevent the user from modifying these values. The user should be able to see the values by default, but the user shouldn't be able to edit them.
Comment: Drupal 6 or 7 ?
Answer: You can create a custom module and add use hook_form_alter to change the default_value of your field and change them to #disabled => true. Here is an example. Don't forget to chage modulename to the name of your module and form-id to the id of your form :-).
function modulename_form_alter(&$form, &$form_state, $form_id) {
    global $user;
    if ($form['#id'] == 'form-id') {
        $form['field_name'] = array(
            '#disabled' => TRUE,
            '#default_value' => $user->name
        );
    }
}

Comment: Thanks.  How can I get the "id" of the content type?
Comment: You can use dsm($form); (if the devel module is enabled) to check the id.
Comment: The form ID is already passed as argument to the hook implementation; there is no need to use `$form['#id']`.
Comment: I forgot to say that my version of drupal is 6, this code works for drupal 6?

I don't know to replacement this:  if ($form['#id'] == 'form-id')

if the field of profile is profile_first_name, in default_value should be $user->profile_user_name?

Thanks.
Comment: This code works for Drupal 6.

To know the replacement of form-id you have to run dsm($form['#id']); check the result and replace.

if the field of profile is profile_first_name, in default_value should be $user->profile_user_name?

if you want the name of the loggedin user use $user->name
Comment: Thanks for your answer.  I am beginner with modules.  I have activated the module devel and I put the dsm and I can't see the form[#id].  However, I'll review later and I'll give an answer.  Thanks.
Comment: The code doesn't work.  I tested your code and the field is not disabled and the value =0.  Here I put the code:
function primermodulo_form_alter(&$form, &$form_state, $form_id) {

    global $user;
    if ($form['#id'] == 'story-node-form') {
        $form['field_first_name'] = array(
           '#disabled' => TRUE,
            '#default_value' =>$user->name
        );
    }
}

Thanks.
Comment: What it the result of dsm($form['field_first_name']) ?
Comment: 0 (Array, 10 elements)
      #type (String, 14 characters ) text_textfield
      #default_value (Array, 1 element)
            field_first_name (String, 5 characters ) black
      #title (String, 10 characters ) First Name
      #description (String, 0 characters )
      #required (Boolean) FALSE
      #weight (Integer) 0
      #delta (Integer) 0
      #columns (Array, 1 element)
            0 (String, 5 characters ) value
      #field_name (String, 16 characters ) field_first_name
      #type_name (String, 5 characters ) story
Comment: I tested this code :
function primermodulo_form_alter(&$form, &$form_state, $form_id){
    if($form_id == 'story_node_form'){
        $form['field_first_name'] = array(
            '#type' => 'textfield',
            '#title' => 'Name profile',
            '#default_value' => $user->name,
   '#disabled'=> 'TRUE'
        );
        $form['#submit'][] = 'primermodulo_field_first_name_submit';
    }
}

the field is disabled but the default value doesn't show in the field.  $user->name doesn't work.

Thanks for your help.
Answer: This code works for me:
function test_form_alter(&$form, &$form_state, $form_id){
global $user;
    if($form_id == 'story_node_form' ){
    if ($user->uid) {
profile_load_profile($user);
        $form['field_first_name'] = array(
            '#type' => 'textfield',
            '#title' => 'First name',
           '#default_value' => $user->profile_first_name,
            '#disabled'=> 'TRUE'
        );
        $form['#submit'][] = 'test_field_first_name_submit';
    }
     }
}


Remove a word from a file

Question: I am try to sort through the following file
Fantasy
Supernatural
Fantasy
UrbanFantasy
Fantasy
EpicFantasy
Fantasy
HighFantasy

I want to remove the word fantasy when it appears by itself and put the new list into another file
I tried
def getRidofFantasy():
    file = open("FantasyGenres.txt", "r")
    new_file = open("genres/fantasy", "w")
    for line in file:
        if line != "Fantasy":
            new_file.write(line)
    file.close()
    new_file.close()

This does not work and I am at a lost as to why. The new file is the same as the old one. Can anyone explain what's happening and give an example of the correct solution?
Comment: Try `if line.strip() != "Fantasy":`, there will be newline chars at the end of each line that you want to strip
Answer: Try this
with open('fantasy.txt') as f, open('generes/fantasy', 'w') as nf:
  lines = [line+'\n' for line in f.read().splitlines() if line != "Fantasy"]
  nf.writelines(lines)

In your code when you do for line in f the line variable also include the \n (endline) char, that's why it doesn't work.
Answer: Try this. -

def getRidofFantasy():
    with open("FantasyGenres.txt", "r") as file:
        content = [line.strip('\n') for line in file.readlines()]
        new_list = list(filter(lambda a: a != 'Fantasy', content))

    with open("genres/fantasy.txt", "w") as new_file:
        [new_file.write(f'{line}\n') for line in new_list]

getRidofFantasy()

Answer: Similar to @Atin's answer, you can also do this:
with open('fantasy.txt') as f, open('generes/fantasy', 'w') as nf:
  lines = [line for line in f.readlines() if line.strip() != "Fantasy"]
  nf.writelines(lines)

Answer: That is because a new line is also a character:
Fantasy\n
Supernatural\n
etc.

You have to account for that. One possibility:
def getRidofFantasy():
    with open("FantasyGenres.txt", "r") as f:  # this way Python closes the file buffer for you
        oldfile = f.readlines()
    new_file = open("genres/fantasy", "w")
    for line in oldfile:
        line = line.rstrip('\n')
        if line != "Fantasy":
            new_file.write(line+'\n') # make sure to append the newline character again

    new_file.close()

Answer: Okay so there is one thing you should know. When you read a line like that the variable will look something like this:-
line='Fantasy\n'
So, you need to strip that character. The simple solution without changing any of  your code would be to just change the if statement. Change it to
if not 'Fantasy'== line.strip() and keep your code as it is and the new file that'll be generated will be the one you want.

Remove a word from a file


Can't read from /proc/self/attr/current, but permissions are 0666

Question: I'm working with an externally created script that reads from /proc/self/attr/current to get SELinux configuration info.  It seems that, even though permissions on the file are 0666, on some systems this file can be read with no problem while on others generate an "invalid argument" error.  Specifically, there is 1 server out of 3 in a Kubernetes cluster where this file cannot be read, while on the other 2 it contains the string "unconfined".
I came across this thread which reported the same issue without resolution.
What could be preventing this file from being read?
OS is Debian 8.
Answer: /proc/[pid]/attr/current provides the current security attributes for the process. This isn't necessarily SELinux; it is also used by AppArmor.
In the case of Debian systems, it almost certainly is AppArmor, as SELinux is rarely or never used on Debian.
In particular, the bare string unconfined also indicates it's AppArmor; SELinux would have a much longer "unconfined" string, such as unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023.
When this path can't be read, neither SELinux nor AppArmor is enabled.
Comment: Looks like that was it.  I was looking for SELinux configs and didn't consider AppArmor.  It turns out AppArmor wasn't installed on the one server but was on the other two.


Interior Set of Rationals. Confused!

Question: Can someone explain to me why the interior of rationals is empty? That is $\text{int}(\mathbb{Q}) = \emptyset$?
The definition of an interior point is "A point $q$ is an interior point of $E$ if there exists a ball at $q$ such that the ball is contained in $E$" and the interior set is the collection of all interior points.
So if I were to take $q = \frac{1}{2}$, then clearly $q$ is an interior point of $\mathbb{Q}$, since I can draw a ball of radius $1$ and it would still be contained in $\mathbb{Q}$.
And why can't I just take all the rationals to be the interior?
So why can't I have $\text{int}\mathbb{(Q)} = \mathbb{Q}$?
Comment: notice that "the ball is contained in $E$" means all of the points in the ball is contained in $E$, but any "ball" centered at a rational point contain some irrational point, which is not in $\mathbb{Q}$
Comment: I'm assuming you are using the standard definition, a "ball" on the real line, centered at a point $x$ with radius $r$ is defined (as long as I know)to be the set $\{y: |y-x|<r\}$, certainly in this definition we cannot have "holes"
Comment: *Hint:* what does contained in *mean*?
Comment: @Tao, but why do we even need to check those irrationals?Our ball will have holes, but we can't have that?
Answer: If the whole set is $\mathbb{Q}$, then $\text{int}(\mathbb{Q})=\mathbb{Q}$,
If the whole set is  $\mathbb{R}$ or $\mathbb{R}^n$, then $\text{int}(\mathbb{Q})=\emptyset$,
because, $\forall q\in \mathbb{Q}, and  \,\forall \epsilon>0, B_\epsilon(q)=\{x\in\mathbb{R}:|x-q|<\epsilon\}$ contains irrational numbers, which are not in the $\mathbb{Q}$,  so $q$ is not a interior point of $\mathbb{Q}$.
the statement is proved.
the problem depends on the whole set you are talking about.
Comment: Oh okay, that makes more sense!
Comment: Exactly, this makes perfect sense. If we don't know about (or consider) the rationals, then every point in Q is in its interior. It is only when we consider the rationals, that the definition changes.
Answer: I'm assuming since you're using the Euclidean Metric that you're viewing $\mathbb{Q}$ as a subset of $\mathbb{R}$.  The emptiness of the interior follows from the density of the rationals in the reals.  So in fact, you can't actually take an open set around a rational number and stay within the rationals because real numbers will always get in your way.
Comment: Empty interior doesn't follow from being dense.
Comment: The empty interior follows from $\mathbb{R}-\mathbb{Q}$ being dense in $\mathbb{R}$ and as Kundor remarked not from $\mathbb{Q}$'s density in $\mathbb{R}$.
A thorough answer to this question and afew others can be found here: http://math.stackexchange.com/questions/1306770/example-of-non-open-dense-proper-subset-with-non-empty-interior
Answer: The interior of rational numbers is not empty when the whole set is also rational numbers.
But if the whole set is real numbers, then any rational numbers would have a ball containing real numbers. By definition, this rational number is not an interior point because its ball contains real numbers which are not part of the set of rational numbers.
Answer: Think of it this way: we say $A$ is a subset of $B$ if for all $a \in A$, $a$ is in $B$.
Now, we say that the interior of some set $S$ is the set of all of its interior points. A point is an interior point if there exists a neighborhood $N$ of $x$, for some $x \in S$, such that $N$ is a subset of $S$.
Now, keeping in mind the definition of a subset, there exists no $\epsilon$ such that an irrational number is not contained within the neighborhood $(x - \epsilon, x + \epsilon)$ - hence, since there exists an $x \in N$ such that $x$ is not in $S$, given $I = \mathbb R \setminus \mathbb Q$, then there can be no such $N$ such that $N$ is a subset of $S$ - it fails to meet the definition of a subset.
When you consider what is required for a point to be considered an interior point, you can see that no such interior points exist, and thus $\operatorname{int}(\mathbb Q) = \emptyset$.
Answer:
Definition:
An element x is the interior point of A(subset of X) if there exists
open set U containing x such that U contained in A.
Let x=2, A=Q, X=R(Real Numbers),U=(1,3)  Apply them on definition.
The element 2 is interior point of Q if the open set U=(1,3) and 2 belongs to U such that (1,3)contained in Q.
Look at the condition (bold line).. Do we have (1,3) contained in Q ? No ,since (1,3) contains an irrational number root2(root 2).
but which doesn't belongs to Q.
Hence i can find an open set containing 2 but which not satisfies the condition (Bold one). So int Q = empty.
it's understood that it will not work for any Q's.
Hence int Q= empty.
Answer: It is easy to show that there are irrational numbers between any two rational numbers. Let $q_1 < q_2$ be rational numbers and choose a positive integer $m$ such that $m(q_2-q_1)>2$. Taking some positive integer $m$ such that $m(b-a)>2$, the irrational number $m q_1+\sqrt{2}$ belongs to the interval $(mq_1, mq_2)$ and so the irrational number $q_1 + \frac{\sqrt{2}}{m}$ belongs to $(q_1,q_2)$.
With this in mind, there are irrational numbers in any neighbourhood of a rational number, which implies that $\textrm{int}(\mathbb{Q}) = \emptyset$.

Interior Set of Rationals. Confused!


Testing the Validity of a Test Statistic (incorrectly)

Question: In this post @Wolfgang shares a few lines of code to test in a simulation the ability of a particular test statistic to perform as expected, measured by the percentage of type I errors. He shows by simulation how the $chi \,square$ test works in the case presented in that original post, despite the small number of "successes" in comparison with the large number of observations.
On a post I wrote to sort of piece together what I've been collecting from many other posts on comparison of proportions, I was trying to come up with a similar type of simulation using the $z$-test. I understand that the $\chi^2$ is mathematically equivalent in this situation, but it was one of these exercises in convincing oneself, and understand the $z \,test$ equations. Further, I wasn't sure the tests were going to be identical in every respect.
So to the question... I road-tested the equation of the $z$ test function, and it did not produce the expected results. The same happened with a similar ad hoc formula in R-Bloggers, which I will use in the rest of the question.
My bet is that my code is flawed (in which case, I'd like to fix it on my prior post), but the values it generates seem plausible; the only odd result being the proportion of type I errors. So I wonder if I could get some help, especially if the results are accurate, and the statistical interpretation is incorrect.
This is the actual problem (more details here if needed):
DATA:
           Medication
Symptoms    Drug A Drug B Totals
  Heartburn     64     92    156
  Normal       114     98    212
  Totals       178    190    368

TEST STATISTIC FUNCTION IN R (from R-Bloggers):
For reference: $ Z = \frac{\frac{x_1}{n_1}-\frac{x_2}{n_2}}{\sqrt{p\,(1-p)(1/n_1+1/n_2)}}$ with $ p = \frac{x_1\,+\,x_2}{n_1\,+\,n_2}$
z.prop = function(x1,x2,n1,n2){
  numerator = (x1/n1) - (x2/n2)
  p.common = (x1+x2) / (n1+n2)
  denominator = sqrt(p.common * (1-p.common) * (1/n1 + 1/n2))
  z.prop.ris = numerator / denominator
  return(z.prop.ris)
}

POPULATION SET UP WITH ONE SINGLE (AVERAGE) TRUE PROPORTION:
set.seed(5)

# Number of samples taken to obtain different proportions IF
# we set it up so that the proportions are actually equal:

samples <- 100000

# Number of cases for Drug A and Drug B:

n1 <- 178
n2 <- 190

# Proportion of patients with heartburn:
p1 <- 64/n1
p2 <- 92/n2

# We will make up a population where the proportion of heartburn suffers
# is in between p1 and p2:

(c(p <- mean(c(p1, p2)), p1, p2))
[1] 0.4218805 0.3595506 0.4842105

# And we'll take samples of the size of the groups assigned to Drug A
# and Drug B respectively:

x1 <- rbinom(samples, n1, p)
x2 <- rbinom(samples, n2, p)

# Double - checking that the proportion is kept although the number of
# successes is different:

(c(mean(x1),mean(x2)))
[1] 75.08542 80.15841
(c(prop1 <- mean(x1)/n1, prop2 <- mean(x2)/n2, p))
[1] 0.4218282 0.4218864 0.4218805

SIMULATION:
# Now we run the z-test in what we just did, but we repeat it 100000 times
# and put the results in vector pval:

pval <- 0
for (i in 1:100000) {
    pval[i] <- pnorm(-abs(z.prop(mean(x1[i]), mean(x2[i]), n1, n2)))
}

# What is the fraction of pval's below 0.05 (it should be 0.05 since we
# should only reject 0.05 times the true NULL):

mean(pval <= .05)
[1] 0.09887

To my surprise, the result is 0.09887, and changing the set.seed ends up with larger proportions of falsely rejected null hypothesis.
Comment: @Scortchi So the z test has a normal distribution in this simulation, and I "look up" the area under the curve with `pnorm`, and in doing so I am applying the pdf of a normal $\Phi$ on $z$. But in this case I'm "folding over" both tails, and that's why I'm getting double the estimate that I was expecting?
Comment: Plotting a histogram of the simulated p-values - `hist(pval)` - should give you a clue (you'll kick yourself).
Comment: It should be (more or less, as the test statistic is a discrete random variable) a uniform distribution from nought to one. Did you intend a one-tailed or a two-tailed test of the null?
Comment: If $Z$ has a normal distribution then $\Phi(Z)$ (where $\Phi(\cdot)$ is the normal distribution function) has a uniform distribution between 0 & 1; so does $2\Phi(-|Z|)$. Double the p-value for a two-tailed test.
Comment: Exactly. Suppose you observe $z=1.4$; you're calculating  $\Pr(Z<-1.4)$ when you should be calculating $\Pr(Z<-1.4)+\Pr(Z>1.4)$.
Comment: @Scortchi I did `hist(pval, breaks=seq(0,0.5,0.01), freq = F)`, and it looks like a uniform distribution from 0 to 0.5. What am I missing?
Comment: @Scortchi The key may be in the way I treated the `z` value generated by the function (they called it `z.prop.ris`). I obtained its absolute value, made it negative and applied `pnorm`. But I don't see why I should get more than  $0.05$ frequency of more extreme values even with this manipulation...
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/27237/discussion-between-antoni-parellada-and-scortchi).
Answer: And I want to let you all know that I was absolutely right: my code was flawed! Thanks a million to @Scortchi for walking me through the problem. Since at least two people up-voted the question, here is the answer:
The probability integral function dictates that if $X \sim N(\mu,\sigma^2)$, the random variable $Y = \Phi(X)$ with $\Phi$ corresponding to the error function ($erf$) (who came up with this name?) will be uniformely distributed as $Y  \sim U(a,b)$.
Since we know that the $p-values$ range from $(0,1)$, plotting the histogram should illustrate our expectation (overabundant $p<0.05$ values colored in greenish hue):

So we are getting just half of the span we expected: from $[0,0.5]$, as opposed to $[0,1]$. And the problem is in the code treatment of the $z$ values produced by the function z.prop. Since $z$ is a symmetric distribution the part of the code intended to "look up the probability in the table at the end of the book": (pnorm(-abs(z.prop(mean(x1[i]), mean(x2[i]), n1, n2)))) is effectively sending all the $z$ values above $0$ to the negative part of the bell curve, and we end up with double amount of pval < 0.05.
As @Scortchi says in his comments, if you happened to get a $z=1.4$, "you are calculating $Pr(Z<−1.4)$ when you should be calculating $Pr(Z<−1.4)+Pr(Z>1.4)$.
This can be fixed by simply doubling the value of the pvalues. Or perhaps more neatly by changing the SIMULATION part of the code in the OP to:
pval <- 0
for (i in 1:100000) {
     pval[i] <- pnorm(z.prop(mean(x1[i]), mean(x2[i]), n1, n2))
}
> mean(pval <= .025) + mean(pval >= 0.975)
[1] 0.04923


Finding the pseudo-code of an algorithm that tests if a function is antisymmetric

Question: I am searching for a way to write a pseudocode for a algorithm that can determine if a relation $R$ (on a (finite) set $X$) is antisymmetric.
Now I was thinking that I might try to make a function that tests all the conditions (required by a relation in order to be antisymmetric) for all pairs in the set $X$.
Now the issue here is that I don't seem to be able to make such a function. Can someone make it or guide me towards it? Do you have any examples of similar pseudocode? Your help will be really appreciated. Thanks!
Comment: Do you need it to be efficient? If not just check every pair
Answer: Just do as you say: check all relevant pairs. I.e., pick each $x$ in turn, for all $y$ check if the condition(s) hold for $x, y$. If you find a pair that doesn't check out, bail out early answering "no"; if you run out of pairs to check, answer "yes".
You might streamline this somewhat, depending on the exact properties of the conditions.


How to analyze a dataset with unequal treatment distribution?

Question: I'm looking for the best way to analyze a data set. The experiment was conducted at two different locations, X and Y. The experiment at location X was conducted in one year (2014), whereas the one conducted at location Y was repeated in the next year too (total two years – 2015 & 2016). There were a total of four treatments in this study A, B, C, D. Similar treatments (A, B, C) were compared at location Y. Location X did not have treatment A, but had an additional treatment D.
In summary, only treatments B, C were common at both locations. Similar treatments were compared at location Y, and treatment D was only tested at location X.
What's the best way to analyze this data?

Should data be analyzed separately for each location? For location X, this is better because I can compare treatments B, C and D at location X without introducing any bias, as treatment D was not tested at location Y, so I will have the same number of data points to compare to compare treatments B, C and D. This is also better for location Y because similar treatments were compared and the experiment was repeated in the next year too. I will also be comparing similar number of data points for each treatment if data was analyzed separately at each location.

Since treatments B & C were common at both locations, data should not be analyzed separately at both locations because I will get more data points to compare treatments B & C. The problem is that I will get uneven treatment number distribution in this case because treatment D was only tested at location X, so treatment D has only 23 data points, treatments B & C has 74 (23+25+26) data points, and treatment A has 51 (26 + 25) data points

I'm more inclined towards method 1 but looking for an expert opinion for confirmation. Thanks

Some details about my experiment. I put out plants in the field for a week to treat them with above four treatment, took them back to a controlled environment to count the number of infected leaves and DISCARDED them. I repeated the experiment with fresh plants in the next week.  This is not a time series data. Treatments were applied for a week at both locations, so the duration of each treatment was the same. This is a positive count data. Plot size, treatment duration and sample material were identical. The only problem is uneven treatment distribution between 2014 & 2015-2016
Comment: What is the goal of your analysis?  Other than the location, were the experiments otherwise identical (plot sizes, treatment durations, sample material etc)?
Comment: In what way might location affect the outcome data?
Answer: The different numbers of cases under different treatments isn't a problem on its own. Although there's a long historical preference in ANOVA for having equal numbers of cases in each treatment, a proper regression model will handle different numbers of cases.
The bigger problem is that you can't distinguish certain types of effects. For example, you can't distinguish differences between locations X and Y from differences between the calendar year 2014 versus years 2015-2016.
One way to deal with this is to model all of the data together and then do post-modeling comparisons of specific situations. That uses all of the data to build the model, but restricts subsequent analysis to comparisons that make sense given the limits of your experimental design.
Code each combination of location/treatment/year in a single 9-level categorical predictor: treatments B, C, D at X in 2014; treatments A, B, C at Y in 2015; treatments A, B, C at Y in 2016. You presumably will be using a Poisson or negative binomial model for the count data, and you should also be controlling for week within the year by including that in your model (for example, with a flexible regression spline). You might consider an interaction between your 9-level categorical predictor and your modeling of week within the year.
After you have fit the model, don't pay much attention to the individual coefficient values. Instead, use post-modeling tools like those provided by the R emmeans package to evaluate combinations of coefficients that provide useful information. For example, did the overall 2015 and 2016 estimates of treatment effects at location Y differ? What is the difference between treatments B and C, combining both locations? Between D and B/C at location X? Between A and B/C at location Y?
Comment: Thanks for the detailed answer. I have marked the question as answered, but would appreciate if you could give me some feed back on what I have done. I have run the analyses for each location separately. I checked DHARMa package residuals for the model and there are no issue. I used Poisson family as the data was not over dispersed while controlling for year/week. Does this approach have any issues statistically? I am interested in comparing treatments only, not locations.
Comment: @AhsaanKhan some might question use of `year` as a random effect when you only have (at most) 2 values for it at any one location. Might be better to use `year` as a fixed effect. The separate analysis by location might make your comparison of treatments B and C (done at both locations) a bit less powerful than in a combined analysis, but other comparisons should be pretty much the same as in the combined analysis as they are within single locations. So I don't see any major issues. Lack of over-dispersion and well-behaved residuals are good signs.
Comment: My bad. Year should not have been included as a random effect.  Thanks you very much.
Comment: @AhsaanKhan it's not strictly required to treat predictors with small numbers of levels as fixed effects. That's why I said "some might question..." Sometimes the partial pooling provided by treating such predictors as random effects makes sense in multilevel models. See [this page](https://stats.stackexchange.com/q/477313/28500) and its links for some discussion. The point is to find a way that describes your current data well and is likely to extend to other data samples.


PHP e MySQL - Colocando textboxs em div

Question: Estou tentando uma espécie de engenharia reversa com o uso do PHP, Javascript e consulta MySQL. Estou criando um formulário de edição de eventos cadastrados em uma tabela do banco.
Em um dos campos, chamado Lista de Presença, eis como ele traz no formulário HTML o valor do campo listapresença dentro do div numPart:

                    <?php foreach ($data['evento'] as $evento){ echo $evento['listapresenca']; } ?>

                <input type="hidden" name="part_text" id="part_text" value="" />

No formulário que insere um novo dado, a lista de presença é composta de um ou mais textbox, dependendo de quantos participantes o evento terá. Normalmente o campo fica cadastrado da seguinte forma:
Nome 1
Nome 2
Nome 3
...
Nome 10
O que eu queria era uma forma de transformar o texto da lista de presença (o texto do div numPart) em vários textboxs, cada um indo do nome até o sinal de linha nova
Como posso fazer?
Comment: Quando você diz 'TEXTBOX' estaria se referindo ao ``? ou apenas está querendo criar uma 'caixa' que receberá a lista do participantes?
Comment: echo '';
Comment: @AdrianoLuz é o input textbox sim
Comment: @DaltonMenezes OK, mas como eu posso fazer pra mais de um nome guardado nesse campo?
Comment: Preciso que explique melhor isso. Onde o outro nome ficaria? Qual variável ele está?
Comment: echo ''; Pega o name listapresenca por POST/GET onde esse é um array;
Comment: @DaltonMenezes Por exemplo, se na minha tabela estiver guardado o campo da seguinte forma: "Nome 1Nome 2Nome 3". Nesse caso será necessário transformar em três textbox. Uma contendo Nome 1, outra contendo Nome 2 e a terceira contendo Nome 3. Ou como o Mayron Ceccon falou, se houvesse uma forma de transformar em array,
Comment: Descobri como fazer, vou colocar a solução num novo post desse tópico.
Answer: Descobri a solução. Bastou eu usar o explode:

    <?php foreach ($data['evento'] as $evento){
        $participantes = explode("
",$evento['listapresenca']);
        for($numero=0;$numero<(count($participantes)-1);$numero++){
            echo "
<input name='part".($numero+1)."' id='part".($numero+1)."' value='".$participantes[$numero]."' class='validate[required]'></input>

";
        }

PHP e MySQL - Colocando textboxs em div


How add Ckeditor on EntityType field

Question: I make a custom Entity content module, and for a field i want ckEditor.
Here my code for this field :
<code>$fields['description'] = BaseFieldDefinition::create('string_long')
    ->setLabel(t('Description'))
    ->setRequired(true)
    ->setDisplayOptions('view', array(
        'label' => 'above',
        'type' => 'string_textarea',
        'weight' => -4,

    ))
    ->setDisplayOptions('form', array(
        'type' => 'string_textarea',
        'weight' => -4,
        'settings' => array(
            'placeholder' => 'Description de votre annonce',
        ),
    ))
    ->setDisplayConfigurable('view', TRUE)
    ->setDisplayConfigurable('form', TRUE);
</code>

Type : string_textarea

How can i tell Drupal 8 to get the widget for textarea basic html mode ?
Answer: There are 4 different field types that contain a text

string
string_long
text
text_long

What long means should be obvious, the difference between string and text is that text has a format (like full_html or basic_html) while string is always plain text.
You are currently using string_long, it doesn't make sense to show a wyswiwyg editor for that, since it would always escape all HTML. Use text_long instead.
Comment: Thank you, i was searching for this type text_long.... i don't find any doc on where are all the type.
Comment: Is it possible to also select the default format? Can't really find anything about this.
Answer: Piggybacking on @Berdir's answer, add this to your settings array:
<code>'allowed_formats' => ['ckeditor' => TRUE]</code>

How add Ckeditor on EntityType field


Atribuição de valor variável Javascript

Question: Tenho o código JS abaixo e o retorno do JSON não atribui o valor a variável texto. Fiz um teste com alert(data.frase) e a mensagem veio normalmente.
Alguem saberia dizer o porque a var texto não recebe o valor de data.frase?
var texto=null;
var cod = { cod: codFrase };
$.ajax({
    url: "/Motivos/Getfrase",
    type: 'Post',
    contentType: 'application/json',
    dataType: 'json',
    data: JSON.stringify(cod),
    success: function (data) {
        if (data.success) {
            texto = data.frase;
        }
    }
});

Obrigado
Comment: Você verificou o valor de `data.success`, se é `true`?
Comment: onde voce está verificando se o conteudo de texto mudou ou nao?
Comment: Tem certeza que o resultado que vem do servidor é exatamente uma frase com o nome frase? Debugou pra garantir?
Answer: Você provavelmente tem um problema de assíncronia.
Tudo que ocorre dento do $.ajax sai da fila padrão de execução do seu javascript e roda assíncronamente.
Ou seja, quando você tenta acessar a variável texto o seu código Ajax ainda não foi executado. Já que ele é executado em paralelo.
Você só vai conseguir usar a variável texto dentro do success do Ajax, que é quando o Ajax é finalizado.
Caso não queria ficar com o código preso ali, você pode criar uma função que recebe a variável texto e executá-la dentro do success passando o valor como parâmetro.
var usaTexto = function(texto) {
  // ... aqui você pode fazer o que quiser com a variável texto
}
var cod = { cod: codFrase };
$.ajax({
    url: "/Motivos/Getfrase",
    type: 'Post',
    contentType: 'application/json',
    dataType: 'json',
    data: JSON.stringify(cod),
    success: function (data) {
    if (data.success) {
        usaTexto(data.frase);
    }
}
});

Answer: Não recomendo a utilização deste método e não sei se funcionará em seu projeto C# MVC. Já tive essa necessidade e resolvi da seguinte forma:
var cod = { cod: codFrase };
$.ajax({
   url: "/Motivos/Getfrase",
   type: 'Post',
   contentType: 'application/json',
   dataType: 'json',
   data: JSON.stringify(cod),
   async: 0,
   success: function (data) {
       if (data.success) {
           retorno = "";
           $("#retorno").html(retorno);
       }
   }
});

A opção "async" irá segurar a execução do seu código, retorne a variável no DOM.

Atribuição de valor variável Javascript


Broadleaf commerce cannot integrate with Paypal express module

Question: I have followed Broadleaf's documentation (Here is the link: http://docs.broadleafcommerce.org/2.2/PayPal-Quick-Start.html) to integrate Broadleaf's DemoSite environment with Paypal express module. To implement this for development environment, I created a sandbox account and set its related value to the site.
However, when I tested my DemoSite by adding an item into my shopping cart and clicking on the Express Checkout Button, it caused my DemoSite with the following error:
org.broadleafcommerce.core.payment.service.exception.PaymentException: Unable to execute payment for order -- id: 1
    at org.broadleafcommerce.core.payment.service.CompositePaymentServiceImpl.executePayment(CompositePaymentServiceImpl.java:69)
    at org.broadleafcommerce.core.payment.service.CompositePaymentServiceImpl.executePayment(CompositePaymentServiceImpl.java:74)
    at org.broadleafcommerce.core.payment.service.CompositePaymentServiceImpl.executePaymentForGateway(CompositePaymentServiceImpl.java:90)
    at org.broadleafcommerce.vendor.paypal.service.payment.PayPalCheckoutServiceImpl.initiateExpressCheckout(PayPalCheckoutServiceImpl.java:87)
    at org.broadleafcommerce.vendor.paypal.web.controller.BroadleafPayPalController.paypalCheckout(BroadleafPayPalController.java:84)
    at com.mycompany.controller.paypal.PayPalController.paypalCheckout(PayPalController.java:21)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:100)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:604)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:565)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:643)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
    at org.broadleafcommerce.cms.web.URLHandlerFilter.doFilterInternal(URLHandlerFilter.java:74)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:166)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1302)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.broadleafcommerce.profile.web.core.security.SessionFixationProtectionFilter.doFilter(SessionFixationProtectionFilter.java:96)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.broadleafcommerce.core.web.order.security.CartStateFilter.doFilter(CartStateFilter.java:58)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.broadleafcommerce.profile.web.core.security.CustomerStateFilter.doFilter(CustomerStateFilter.java:54)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.broadleafcommerce.common.security.handler.CsrfFilter.doFilter(CsrfFilter.java:83)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:166)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1302)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
    at org.broadleafcommerce.common.web.BroadleafRequestFilter.doFilterInternal(BroadleafRequestFilter.java:98)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:147)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1302)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:448)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1067)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:377)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1001)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
    at org.eclipse.jetty.server.Server.handle(Server.java:360)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:630)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:77)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:622)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:46)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
    at java.lang.Thread.run(Thread.java:662)
Caused by: org.broadleafcommerce.core.workflow.WorkflowException: org.broadleafcommerce.core.workflow.WorkflowException: org.broadleafcommerce.core.payment.service.exception.PaymentException: The PaymentResponseItem instance did not contain one or more of the following: transactionAmount, transactionTimestamp or transactionSuccess
    at org.broadleafcommerce.core.workflow.DefaultErrorHandler.handleError(DefaultErrorHandler.java:57)
    at org.broadleafcommerce.core.workflow.SequenceProcessor.doActivities(SequenceProcessor.java:84)
    at org.broadleafcommerce.core.payment.service.CompositePaymentServiceImpl.executePayment(CompositePaymentServiceImpl.java:55)
    ... 101 more
Caused by: org.broadleafcommerce.core.workflow.WorkflowException: org.broadleafcommerce.core.payment.service.exception.PaymentException: The PaymentResponseItem instance did not contain one or more of the following: transactionAmount, transactionTimestamp or transactionSuccess
    at org.broadleafcommerce.core.workflow.DefaultErrorHandler.handleError(DefaultErrorHandler.java:57)
    at org.broadleafcommerce.core.workflow.SequenceProcessor.doActivities(SequenceProcessor.java:84)
    at org.broadleafcommerce.core.payment.service.workflow.CompositeActivity.execute(CompositeActivity.java:35)
    at org.broadleafcommerce.core.workflow.SequenceProcessor.doActivities(SequenceProcessor.java:75)
    ... 102 more
Caused by: org.broadleafcommerce.core.payment.service.exception.PaymentException: The PaymentResponseItem instance did not contain one or more of the following: transactionAmount, transactionTimestamp or transactionSuccess
    at org.broadleafcommerce.core.payment.service.workflow.PaymentActivity.execute(PaymentActivity.java:151)
    at org.broadleafcommerce.core.workflow.SequenceProcessor.doActivities(SequenceProcessor.java:75)
    ... 104 more

Therefore, can anyone guide me how to solve the above problem. Thanks in advance.
By the way, my DemoSite is built according to Broadleaf 2.2 and Broadleaf-paypal 2.0.1.
Answer: The Problem is in the method PayPalPaymentModule.reverseAuthorize(). At the line responseItem.setAmountPaid(paymentContext.getPaymentInfo().getAmount()); it uses the deprecated method setAmountPaid(). I think there should be setTransactionAmount(). Maybe you can post that to the broadleaf forum!
I am using Broadleaf 2.3.0-SNAPSHOT and BL paypal 2.2.0-SNAPSHOT
Answer: I code a workaround for this issue!
package de.naschkammer.payment.service.module;

import org.broadleafcommerce.core.payment.domain.PaymentResponseItem;
import org.broadleafcommerce.core.payment.service.PaymentContext;
import org.broadleafcommerce.core.payment.service.exception.PaymentException;
import org.broadleafcommerce.payment.service.module.PayPalPaymentModule;
import org.broadleafcommerce.vendor.paypal.service.payment.type.PayPalTransactionType;

// TODO remove this workaround for paypal, don't forget to adjust /site/src/main/webapp/WEB-INF/applicationContext.xml
public class NaschkammerPayPalPaymentModule extends PayPalPaymentModule {

    @Override
    public PaymentResponseItem authorizeAndDebit(PaymentContext paymentContext) throws PaymentException {
        PaymentResponseItem item = commonAuthorizeOrSale(paymentContext, PayPalTransactionType.AUTHORIZEANDCAPTURE);
        if ( item != null ) {
            item.setTransactionAmount(item.getAmountPaid());
        }
        return item;
    }

}

add following snippet to the file /site/src/main/webapp/WEB-INF/applicationContext.xml

<bean id="blPayPalModule" class="de.naschkammer.payment.service.module.NaschkammerPayPalPaymentModule">
    <property name="payPalPaymentService" ref="blPayPalVendorOrientedPaymentService"/>
</bean>

Answer: It looks like there are other problems that currently make the PayPal module untenable for Broadleaf version 2.3. These are outlined in the GitHub Issues Page.
Another note: we have changed some of our versioning for the PayPal module. If you are targeting Broadleaf version 2.3.0-SNAPSHOT then you should be using PayPal version 2.4.0-SNAPSHOT (which is the 'develop' branch on GitHub.
If you have some fixes for those issues listed there, we would love for you to submit a pull request. Otherwise we will try to get a fix in for those sometime in the next week.


Python REGEX: How to add unique random_value at the end of every regex match

Question: Following is my code. It is working but the problem I am having is that it is adding same random value to every Path value. I want to add unique random value at the end of each and every Path value. Any help would be appreciated.
It can be done by json parse but the requirement of the task is to do it via REGEX.
from io import StringIO
import re
import string
import random

reader = StringIO("""{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "no",
            "Page": 0,
            "Path": "//Document/Sect[2]/Aside/P",
            "Text": "Potsdam, den 9. Juni 2021 ",
            "TextSize": 12.0
        }
    ],

},
{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "de",
            "Page": 0,
            "Path": "//Document/Sect[3]/P[4]",
            "Text": "this is some text ",
            "TextSize": 9.0,
        }
    ],
}""")

def id_generator(size=3, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for _ in range(size))

text = reader.read()
random_value = id_generator()
pattern = r'"Path": "(.*?)"'
replacement = '"Path": "\\1/'+random_value+'"'
text = re.sub(pattern, replacement, text)
#This is working but it is only attaching one same random_value on every Path

print(text)

Comment: It only assigns one random value because you only ever generate one random value. Assigning the returned string from the`id_generator()` function to `random_value` and then using `random_value` in `replacement` doesn't cause the function to be called and a new value to be generated every time `replacement` is used.
Answer: In re.sub you can pass a function which takes a Match object. Then you can expand your match object however you want, including group substitutions.
from io import StringIO
import re
import string
import random

reader = StringIO("""{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "no",
            "Page": 0,
            "Path": "//Document/Sect[2]/Aside/P",
            "Text": "Potsdam, den 9. Juni 2021 ",
            "TextSize": 12.0
        }
    ],

},
{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "de",
            "Page": 0,
            "Path": "//Document/Sect[3]/P[4]",
            "Text": "this is some text ",
            "TextSize": 9.0,
        }
    ],
}""")

def id_generator(size=3, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for _ in range(size))

def new_random_group(M):
    return M.expand('"Path": "\\1/' + id_generator() + '"')

text = reader.read()
pattern = r'"Path": "(.*?)"'

text = re.sub(pattern, new_random_group, text)

print(text)

results
{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "no",
            "Page": 0,
            "Path": "//Document/Sect[2]/Aside/P/R3V",
            "Text": "Potsdam, den 9. Juni 2021 ",
            "TextSize": 12.0
        }
    ],

},
{
    "Bounds": [
        {
            "HasClip": true,
            "Lang": "de",
            "Page": 0,
            "Path": "//Document/Sect[3]/P[4]/HCA",
            "Text": "this is some text ",
            "TextSize": 9.0,
        }
    ],
}


Delete duplicate rows - Oracle

Question: Here is an example of some of my data. I am trying to remove all extra data where the QueryID and RoyID are the same as other entries in the table, however, I would like to keep atleast one record. I'm not sure I'll be able to use the min or max functions on the ID column as some of the records have IDs like DEMO_12345 etc.
ID          QUERY_ID ROYALTY_ID
RTSQR1652   SQ1421  ROY25644005
RTSQR1653   SQ1421  ROY25636406
RTSQR1654   SQ1421  ROY25636557
RTSQR1655   SQ1421  ROY25636558
RTSQR1656   SQ1421  ROY25636559
RTSQR1657   SQ1421  ROY25636560

I was thinking to use ROWID instead of ID. Would the following query work?
 DELETE FROM RT_SOURCE_QUERY_ROYALTIES WHERE ROWID NOT IN (
 SELECT MAX(ROWID) FROM RT_SOURCE_QUERY_ROYALTIES GROUP BY ROYALTY_ID, QUERY_ID);

Comment: possible duplicate of [Removing duplicate rows from table in Oracle](http://stackoverflow.com/questions/529098/removing-duplicate-rows-from-table-in-oracle)
Answer: Just run a test
drop table test1;

create table test1 (a number,b number, c number);

insert into test1 values (1,1,2);
insert into test1 values (1,1,3);
insert into test1 values (1,2,2);
insert into test1 values (2,1,2);
insert into test1 values (2,2,2);
insert into test1 values (1,1,2);

select * from test1;

delete from test1 where rowid not in (select max(rowid) from test1 group by a,b);

select * from test1;

Works as expected, doesn't it ...
table TEST1 dropped.
table TEST1 created.
1 rows inserted.
1 rows inserted.
1 rows inserted.
1 rows inserted.
1 rows inserted.
1 rows inserted.
A B C
- - -
1 1 2
1 1 3
1 2 2
2 1 2
2 2 2
1 1 2

 6 rows selected

2 rows deleted.
A B C
- - -
1 2 2
2 1 2
2 2 2
1 1 2

Comment: rowid (not rownum) has nothing to do with 'order of records' it's an technical representation of where the record is stored. The OP wanted to keep just one record, which one is unspecified, not important.
Comment: How is rowid (order of records) from delete related to rowid from "select max(rowid) from test1 group by a,b" ? select without order by can return records in any order DB likes. There is no grantee. So, seems like you could delete wrong records.
Answer: http://www.dba-oracle.com/t_delete_duplicate_table_rows.htm
Use self-join to delete duplicate rows
The most effective way to detect duplicate rows is to join the table against itself as shown below.
select
   book_unique_id,
   page_seq_nbr,
   image_key
from
   page_image a
where
   rowid >
     (select min(rowid) from page_image b
      where
         b.key1 = a.key1
      and
         b.key2 = a.key2
      and
         b.key3 = a.key3
      );

Please note that you must specify all of the columns that make the row a duplicate in the SQL where clause.

Delete duplicate rows - Oracle


Does the average distance in K-means have to be monotone decreasing?

Question: I'm implementing the k-means algorithm myself. I don't see any obvious mistake in my code and it seems to work well. However, there's something I don't understand.
My algorithm, working on vectors $x_i \in X$, cetroids $c_j \in C$ and labels $l_i \in L$ looks like:

choose centers $C=\{c_1,c_2,...,c_k\}$ from $X_i$ randomly (no repetitions)
label each vector with the index of nearest centroid
$$l_i = \text{arg}\min_{j \in \{1..k\}}(||x_i-c_j||, c_j \in C)$$
find new means of each cluster
$$c_j = \text{mean}\{x \in X_i \;\text{and}\; l_i=j\}\;\text{for}\;j=1..k $$
if the centroids have moved signifficantly in the previous step, goto 2

A plain vanilla k-means, I guess.
However, when I make a plot of the average of point-to-respective-centroid distances after each iteration,
$$\text{mean}\{||x_i-c_j||\;\text{and}\;j=l_i\;\text{for each}\;x_i \in X \}, $$
I can see that the mean distance is not monotone decreasing (although in general it is decreasing and converging nicely):
https://i.stack.imgur.com/ySgWf.png (sorry it won't allow me embed the image)
I remember to be told it should be always decreasing, and my intuition is the same. What's wrong? My algorithm (implementation) or my assumption about the mean of distances?
Comment: Two more comments: 1) Is the 4th step in your algorithm simply to check for convergence? 2) You should be looking at the sum of squared distance not the average distances.
Comment: If the right hand plot in your linked image is the error as a function of iteration then something is wrong. It should be clear that at every step in the K-means algorithm the data points can't get further away from their respective centers. Yours goes down and then back up.
Comment: I think I must have a bug then. I wanted to be sure. Thank you for the explanation.
Comment: I think I found the bug. You were right, the cause was really that I was depicting the mean of distances instead of the sum of squares. Now the bug seems to be gone. However, I don't get it, because both should be monotone decreasing, I guess. Perhaps it's the numerical errors introduced due to square root and division and all the rounding involved?
Comment: I'm not sure. And after thinking about this a little more, I think your originally plot could be possible if you were passing through a local minima.
Comment: @Jan, as jerad pointed out, k-means minimizes sum of squared distences from respective centroids, not sum of distances.
Answer: K-means minimizes the total sum of squared deviations. Don't think of it in terms of distances. Yes, you assign objects by their distance. But that distance (squaredEuclidean) is just the sum of 1d squared deviations. So what you really are optimizing is the sum of squared 1d deviations.
The key part however is the mean, and here it is more obvious that this method optimizes based on single dimensions. The arithmetic mean is the minimum least squares estimation, again.
So you really should plot the sum of squared deviations, not the mean of euclidean distances.
Yes: euclidean distances and squared euclidean distances are monotone - for a single object.
Example in one dimension:
Objects at 0 and 2, mean at 0.5:
average distance: (0.5 + 1.5) / 2 = 1
squared deviations: 0.25 + 2.25 = 2.5
Object at 0 and 2, mean at 1:
average distance: (1 + 1) / 2 = 1
squared deviations: 1 + 1 = 2
So the second example is only better wrt. squared deviations, not with linear deviations. In particular, an optimum for the linear deviations isn't necessarily optimal for the squared deviations.
Above example highlights that one shouldn't just plug in arbitrary distances into k-means. It technically is not distance based. It is based on squared deviations in each single dimension, which happen to sum up to the squared euclidean distance, which is monotone with the regular euclidean distance, so it appears as if we are assigning every object to the nearest cluster center, although what we are actually doing is minimize the sum of squared 1d deviations.
Comment: Thank you for the example. That's exactly what I needed.


Arquivo sendo escrito em diversas listas!

Question: Estou desenvolvendo um programa simples em python, onde eu preciso gerenciar o estoque de uma loja.
Quando tento fazer com que os produtos sejam salvos em uma unica lista, eles acabam sendo salvos em diversas, impossibilitando a leitura no .txt para a pesquisa dos produtos!
Alguém consegue me ajudar? Estou travado nisso à umas 8 horas.
O programa é simples e tem que ser feito em python, para um trabalho de faculdade.
OBRIGADO!!
import sys, json

arquivo = open('LOJA.txt', 'a+')
print("### Loja do Iago e Do Pedro ###")

produto=[]

def menu():
    opcao = int(input('### MENU ###\n1-Cadastro de Produtos\n2-Pesquisar Produtos\n3-Alterar um Produto\n4-Remover um Produto\n5-Listar Produtos em Ordem ALFABÉTICA\n6-Listar Produtos Por CÓDIGO\n'))
    if opcao==1:
        cadastro()
    elif opcao==2:
        nome=input('Digite o nome do produto: ')
        pesquisa(nome)
    elif opcao==3:
        alterar()
    elif opcao==4:
        nome=input('Digite o nome do produto que voce deseja remover: ')
        Remover(nome)
    elif opcao==5:
        alfabetica()
    elif opcao==6:
        codigos()

def cadastro():
    # arquivo = open('LOJA.txt', 'a+')
    qtd=int(input('Quantidade de produtos a serem cadastrados: '))
    for x in range(qtd):
        produto=[]
        nome = input('Nome do produto: ')
        codigo = input('Codigo do produto: ')
        quantidade = input('Quantidade do produto: ')
        produto = json.dumps((nome, codigo, quantidade))
        with open('LOJA.txt', 'a+') as file:
            file.write(str(produto))
    file.close()

def pesquisa(nome):
    # arquivo = open('LOJA.txt', 'r')
    produto = []
    with open('LOJA.txt','r') as file:
        produto = arquivo.readlines() # readlinesssssss
        # produto = eval(file.read())
        for i in range(len(produto)):
            if nome in produto[i][0]:
                print('Nome: %s\nCódigo: %s\nQuantidade: %s' % (produto[i][0], produto[i][1], produto[i][2]))
            else:
                print('Produto não encontrado!')
    file.close()

def alterar():
    arquivo = open('LOJA.txt', 'a+')
    produto = []
    with open('LOJA.txt','a+') as file:
        produto = eval(file.read())
    for i in range(len(produto)):
        if nome in produto[i][0]:
           print('Nome: %s\nCódigo: %s\nQuantidade: %s' % (produto[i][0], produto[i][1], produto[i][2]))
        else:
            print('Produto não encontrado!')
    file.close()

def Remover(nome):
    arquivo = open('LOJA.txt', 'w+')
    produto = []
    with open('LOJA.txt','r') as file:
        produto = eval(file.read())
        # produto = json.load(file)
    for i in range(len(produto)):
        if nome in produto[i][0]:
            del produto[i]
        else:
            print('Produto não encontrado')
    file.close()

def alfabetica():
    arquivo = open('LOJA.txt', 'r')
    with open('LOJA.txt','r') as file:
        produtos=eval(file.read())
        produto_ordenados = sorted(produtos)
        print(produto_ordenados)
    file.close()

def codigos():
    arquivo = open('LOJA.txt', 'r')
    with open('LOJA.txt','r') as file:
        produto=eval(file.read())
        produtos=[produto]
        codigos_ordenados = sorted(produtos)
        print(codigos_ordenados)
    file.close()
menu()

Comment: Dei uma olhada por cima no seu programa.. Pelo q eu percebi vc está com conflitos com a lista `global` Produtos e a lista `local` (criada dentro de uma função) Produtos. Minha sugestão eh mudar o nome das listas, todas tem o mesmo nome, ai vc vi achar onde a lista esta sendo re-escrita onde não deve.
Comment: Sugiro vc trabalhar com listas locais apenas, tira o `produto = [ ]` na linha 4 e trabalha com as listas dentro das funções. caso vc precise usar uma lista local em outro lugar, `return` a lista ou coloca como `global` (com outro nome)
Answer: Dúvida: Precisa realmente ser um .txt? Te recomendo salvar em um .csv, é mais fácil de lidar tanto para o programa quanto para o usuário.
Resolvendo: Na parte de cadastro você precisaria salvar com a vírgula. Agora na parte da pesquisa:
def pesquisa(nome):
    produto = []
    with open('LOJA.txt','r') as file:
        produto = file.read()
        # produto = eval(file.read())
        for i in range(len(produto)):
            if nome in produto[i][0]:
                print('Nome: %s\nCódigo: %s\nQuantidade: %s' % (produto[i][0], produto[i][1], produto[i][2]))
            else:
                print('Produto não encontrado!')
    file.close()

o valor de produto é uma string e não uma lista como você espera, você teria que tratar antes.
-> if nome in produto[i][0]:
(Pdb) i
0
(Pdb) produto[i]
'["Teste", "123", "2"]["Teste 2", "321", "5"]["asdf", "qwer", "2"]'

Qualquer dúvida pode me mandar mensagem que te ajudo.
Answer: O problema é que na hora de cadastrar você está salvando tudo na mesma linha. Para o código funcionar é necessário salvar utilizando a quebra de linha (assumindo que você quer lê o arquivo por linha na pesquisa).
Desta forma em sua função de cadastro seu código deve ser modificado para salvar por linhas.
file.write('{}\n'.format(produto))

Repare que em minha solução eu usei a função format da classe string para salvar com a quebra de linha (\n).
Agora o seu arquivo LOJA.txt será salvo por linhas. Ex:
["a", "1", "0"]
["b", "2", "10"]
["c", "3", "100"]

A sua função de pesquisa deve ser reescrita.
def pesquisa(nome):
    # arquivo = open('LOJA.txt', 'r')
    produto = []
    with open('LOJA.txt','r') as file:
        produto = file.readlines() # readlinesssssss
        # produto = eval(file.read())
        for i in range(len(produto)):
            item = json.loads(i)
            if nome in item[0]:
                return print('Nome: %s\nCódigo: %s\nQuantidade: %s' % (item[0], item[1], item[2]))
        print('Produto não encontrado!')
    file.close()

Repare as mudanças:

Substituí produto = arquivo.readlines() por produto = file.readlines()

Apenas para corrigir uma referencia a uma variável comentada.

Adicionei a linha item = json.loads(i)

Temos que inserir essa linha pois no momento de fazer o cadastro, você executa um json.dumps. Desta forma para recuperar esta informação devemos utilizar o json.loads. Segue a documentação para json do python
Devido a esta mudança modifiquei um pouco como é iterado objetos, mas tentei deixar o mais parecido com o código que disponibilizou. Segue a versão final em destaque:
for i in range(len(produto)):
    item = json.loads(i)
        if nome in item[0]:
            return print('Nome: %s\nCódigo: %s\nQuantidade: %s' % (item[0], item[1], item[2]))
print('Produto não encontrado!')

Nesta implementação eu adicionei um return ao encontrar um elemento. E movi o código que mostra a mensagem de não encontrado para fora do comando iterativo for.


angular velocity around ellipse

Question: If I have velocity at perihelion/apphelion, distance away from sun at perihelion/apphelion, and orbital period. How can I find the angular velocity function for earth and subsequently all the other planets in the solar system?
Answer: Kepler's second law specifies
$$\frac1{2}r^2\frac{d\theta}{dt} = \frac{\pi ab}{P},$$
where $P$ is the period and $a$ and $b$ are the lengths of the semi-major and sem-minor axes, respectively, for the elliptical orbit.  This means that the constant rate -- at which a line segment between the planet and the sun sweeps out area -- equals the area of the ellipse divided by the time required to complete one orbit.
In polar coordinates the velocity is
$$\mathbf{v} = \frac{dr}{dt}\mathbf{e}_r + r\frac{d \theta}{dt}\mathbf{e}_{\theta},$$
and the magnitude of angular velocity is
$$\mathbf{\omega}= \frac{|\mathbf{r}\times\mathbf{v}|}{r^2}= \frac{d \theta}{dt}.$$
Whence,
$$\omega(r) = \frac{d\theta}{dt} = \frac{2\pi ab}{Pr^2}.$$
Using some analytic geometry you can relate $a$ and $b$ to the perihelion ($d_p$) and aphelion ($d_a$) distances. Let the distance between the foci be $2c$.  Note that the center of the sun is located at one focus. We have the following relationships:
$$b^2 = a^2 - c^2,\\a+c = d_a,\\a-c= d_p.$$
Solving for $a$ and $b$ we get
$$a = \frac{d_a + d_p}{2}, \,\,\,\,b = \sqrt{d_ad_p}.$$
Substituting into the expression for angular velocity, we get
$$\omega(r) = \frac{d\theta}{dt} = \frac{\pi }{Pr^2}(d_a + d_p)\sqrt{d_ad_p}.$$
Comment: Thank you! can you please explain the last part once again. Didn't quite understand what you mean.
Comment: You're welcome. In the last part I showed the equations you could use to specify $a$ and $b$ given the perihelion and aphelion distances.  I stopped short of writing the explicit solution.
Comment: I'm sorry but I tried to understand but I still don't. Can you please exaplain? Forever grateful
Comment: Sure.  I expanded the answer.  Kepler's Second Law relates the angular velocity to the period of the orbit and the area of the ellipse (in terms of a and b).  You said you were given the perihelion and aphelion distances. I found the relationship between those distances and the semi-major and semi-minor axis lengths. Then I found the formula for angular velocity in terms of your given data.
Comment: Oh wow that makes so much more sense. Great Thank you so much!!
Comment: Sorry to be such a hassle but I don't understand how b^2=a^2-c^2. it doesn't make a triangle in the diagram i've drawn so how can that relationship be found?
Comment: An ellipse is the locus of points where the the sum of the distances to two fixed points (foci) is a constant. Let the foci be $(c,0)$ and $(-c,0)$.  The point $(a,0)$ is on the ellipse at one end of the major axis and the point $(0,b)$ is on the ellipse at an end of the minor axis. So $\sqrt{b^2 + c^2} + \sqrt{b^2 + (-c)^2} = (a-c) + (a -(-c))$. This implies $2 \sqrt{b^2+c^2} = 2a$ and $\sqrt{b^2+c^2}=a$ and $b^2+c^2 = a^2$.  Hence, $b^2 = a^2 -c^2.$
Comment: Thank you so much! I'm literally forever grateful.
Comment: to calculate the angular velocity, should I take the average value of radius for "r" or how should I approach that? thanks once again.
Comment: Please help. Last query I promise. Do i use the average value of the radius for the r value in the angular velocity? Thank you
Comment: It depends on what you are looking for.  The angular velocity is a function of r -- so you can evaluate it if you know r (distance to sun) at any point of interest.
Comment: I'm looking to equate the angular velocity with time for two planets so that I can find the time of eclipse between them. to do this i will put the faster planet exactly 2π ahead of the slower one and equate for t. would this be right?
Comment: I need to think about that one a bit.
Comment: @Ireallyneedhelpthanks: I'm a bit confused about what you are trying to do.  Perhaps you should post a new question.

angular velocity around ellipse