Como pegar um objeto especifico de um Json/XML?

Question: Preciso pegar o conteúdo dentro de <extract> para utilizar em minha aplicação, mas não estou conseguindo. Já tentei criar uma classe com os mesmos objetos, já tentei utilizar o regex, mas sem sucesso.
Tenho o seguinte XML:
<api batchcomplete="">
    <query>
        <pages>
            <page _idx="21721040" pageid="21721040" ns="0" title="Stack Overflow">
                <extract xml:space="preserve">
                    Stack Overflow is a privately held website, the flagship site of the Stack Exchange Network, created in 2008 by Jeff Atwood and Joel Spolsky. It was created to be a more open alternative to earlier question and answer sites such as Experts-Exchange. The name for the website was chosen by voting in April 2008 by readers of Coding Horror, Atwood's popular programming blog. It features questions and answers on a wide range of topics in computer programming. The website serves as a platform for users to ask and answer questions, and, through membership and active participation, to vote questions and answers up or down and edit questions and answers in a fashion similar to a wiki or Digg. Users of Stack Overflow can earn reputation points and "badges"; for example, a person is awarded 10 reputation points for receiving an "up" vote on an answer given to a question, and can receive badges for their valued contributions, which represents a kind of gamification of the traditional Q&A site or forum. All user-generated content is licensed under a Creative Commons Attribute-ShareAlike license. Closing questions is a main differentiation from Yahoo! Answers and a way to prevent low quality questions. The mechanism was overhauled in 2013; questions edited after being put "on hold" now appear in a review queue. Jeff Atwood stated in 2010 that duplicate questions are not seen as a problem but rather they constitute an advantage if such additional questions drive extra traffic to the site by multiplying relevant keyword hits in search engines. As of April 2014 Stack Overflow has over 4,000,000 registered users, and it exceeded 10,000,000 questions in late August 2015. Based on the type of tags assigned to questions, the top eight most discussed topics on the site are: Java, JavaScript, C#, PHP, Android, jQuery, Python and HTML. Stack Overflow also has a Jobs section to assist developers in finding their next opportunity. For employers, Stack Overflow provides tools to brand their business, advertise their openings on the site, and source candidates from Stack Overflow's database of developers who are open to being contacted.
                </extract>
            </page>
        </pages>
    </query>
</api>

Ou o seguinte Json:
{
   "batchcomplete":"",
   "query":{
      "pages":{
         "21721040":{
            "pageid":21721040,
            "ns":0,
            "title":"Stack Overflow",
            "extract":"Stack Overflow is a privately held website, the flagship site of the Stack Exchange Network, created in 2008 by Jeff Atwood and Joel Spolsky. It was created to be a more open alternative to earlier question and answer sites such as Experts-Exchange. The name for the website was chosen by voting in April 2008 by readers of Coding Horror, Atwood's popular programming blog.\nIt features questions and answers on a wide range of topics in computer programming.\nThe website serves as a platform for users to ask and answer questions, and, through membership and active participation, to vote questions and answers up or down and edit questions and answers in a fashion similar to a wiki or Digg. Users of Stack Overflow can earn reputation points and \"badges\"; for example, a person is awarded 10 reputation points for receiving an \"up\" vote on an answer given to a question, and can receive badges for their valued contributions, which represents a kind of gamification of the traditional Q&A site or forum. All user-generated content is licensed under a Creative Commons Attribute-ShareAlike license.\nClosing questions is a main differentiation from Yahoo! Answers and a way to prevent low quality questions. The mechanism was overhauled in 2013; questions edited after being put \"on hold\" now appear in a review queue. Jeff Atwood stated in 2010 that duplicate questions are not seen as a problem but rather they constitute an advantage if such additional questions drive extra traffic to the site by multiplying relevant keyword hits in search engines.\nAs of April 2014 Stack Overflow has over 4,000,000 registered users, and it exceeded 10,000,000 questions in late August 2015. Based on the type of tags assigned to questions, the top eight most discussed topics on the site are: Java, JavaScript, C#, PHP, Android, jQuery, Python and HTML.\nStack Overflow also has a Jobs section to assist developers in finding their next opportunity. For employers, Stack Overflow provides tools to brand their business, advertise their openings on the site, and source candidates from Stack Overflow's database of developers who are open to being contacted."
         }
      }
   }
}

Eles estão guardados em uma string.
Answer: Existem algumas formas de trabalhar tanto com json e com xml, exemplos:
Para Xml:
[Serializable, XmlRoot("api")]
public class RootObject
{
    [System.Xml.Serialization.XmlAttribute("batchcomplete")]
    public string BatchComplete { get; set; }
    [System.Xml.Serialization.XmlElement("query")]
    public Query Query { get; set; }
}
public class Query
{
    [System.Xml.Serialization.XmlElement("pages")]
    public Pages Pages { get; set; }
}
public class Pages
{
    [System.Xml.Serialization.XmlElement("page")]
    public Page Page { get; set; }
}
public class Page
{
    [System.Xml.Serialization.XmlAttribute("pageid")]
    public int PageId { get; set; }

    [System.Xml.Serialization.XmlAttribute("ns")]
    public int Ns { get; set; }

    [System.Xml.Serialization.XmlAttribute("title")]
    public string Title { get; set; }

    [System.Xml.Serialization.XmlElement("extract")]
    public string Extract { get; set; }
}

Como usar?
using System.Xml.Serialization;

XmlSerializer s = new XmlSerializer(typeof(RootObject), "");
var result = (RootObject)s.Deserialize(new StreamReader("./f.xml"));
var str = result.Query.Pages.Page.Extract;

Para Json:

Usando Json.NET

1) Com classes
public class RootObject
{
    [JsonProperty("batchcomplete")]
    public string BatchComplete { get; set; }
    [JsonProperty("query")]
    public Query Query { get; set; }
}
public class Query
{
    [JsonProperty("pages")]
    public Pages Pages { get; set; }
}
public class Pages
{
    [JsonProperty("21721040", Required = Required.Always)]
    public Data Datas { get; set; }
}
public class Data
{
    [JsonProperty("pageid")]
    public int PageId { get; set; }

    [JsonProperty("ns")]
    public int Ns { get; set; }

    [JsonProperty("title")]
    public string Title { get; set; }

    [JsonProperty("extract")]
    public string Extract { get; set; }
}

Como usar?
string json = System.IO.File.ReadAllText("./j.json");
RootObject result = Newtonsoft.Json.JsonConvert.DeserializeObject<RootObject>(json);

2) Simples:
string json = System.IO.File.ReadAllText("./j.json");
ExpandoObject converter = new ExpandoObject();
dynamic result = Newtonsoft.Json.JsonConvert.DeserializeAnonymousType(json, converter);
result = ((ExpandoObject)result.query.pages).FirstOrDefault().Value;
string str = result.extract;

Usando DataContractJsonSerializer:

1) Com classes
[DataContract()]
public class RootObject
{
    [DataMember(Name = "batchcomplete")]
    public string BatchComplete { get; set; }
    [DataMember(Name = "query")]
    public Query Query { get; set; }
}
[DataContract()]
public class Query
{
    [DataMember(Name = "pages")]
    public Pages Pages { get; set; }
}
[DataContract()]
public class Pages
{
    [DataMember(IsRequired = true, Name = "21721040")]
    public Data Data { get; set; }
}

[DataContract()]
public class Data
{
    [DataMember(Name = "pageid")]
    public int PageId { get; set; }

    [DataMember(Name = "ns")]
    public int Ns { get; set; }

    [DataMember(Name = "title")]
    public string Title { get; set; }

    [DataMember(Name = "extract")]
    public string Extract { get; set; }
}

Como usar?
using System.Runtime.Serialization.Json;

DataContractJsonSerializerSettings settings =
                new DataContractJsonSerializerSettings();
settings.UseSimpleDictionaryFormat = true;

DataContractJsonSerializer jsonSerializer =
    new DataContractJsonSerializer(typeof(RootObject));

RootObject rootObj = null;
string json = System.IO.File.ReadAllText("./j.json");
using (MemoryStream strReader = new MemoryStream(Encoding.UTF8.GetBytes(json)))
{
    rootObj = (RootObject)jsonSerializer.ReadObject(strReader);
}

var str = rootObj.Query.Pages.Data.Extract;

Observação: Essas formas, lhe dão oportunidade de trabalhar com todas as informações contidas tanto no xml quanto no json, se for para dar uma indicação utilize a versão para xml, porque, tem o formato correto de chaves e valores, mas, isso é nesse caso em especifico, se o json estivesse bem formatado seria essa a melhor recomendação.
Referencias

Classe DataContractJsonSerializer
Json.NET
Classe XmlSerializer


Showing the bilinear and commutative map $f:\mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ should be in the quadratic form.

Question: I'm trying to show, any map $f:\mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ that holds, for $x,y,x_1,x_2 \in \mathbb{R}^n, c \in \mathbb{R}$,

$ f(x,y)=f(y,x)$
$ f(x_1+x_2,y) = f(x_1, y) + f(x_2, y)$
$ f(cx, y) = cf(x,y)$

should be expressed in the quadratic form $f(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T A \mathbf{y}$, where $A$ is symmetric.
My attempt
It can be shown that the quadratic form map $f(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T A \mathbf{y}$ satisfies,

$f(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T A \mathbf{y} = (\mathbf{x}^T A \mathbf{y})^T = f(\mathbf{y}, \mathbf{x})$
$f(\mathbf{x_1} + \mathbf{x_2}, \mathbf{y}) = f(\mathbf{x_1}, \mathbf{y}) + f(\mathbf{x_2}, \mathbf{y})$
$f(c \mathbf{x}, \mathbf{y}) = cf( \mathbf{x}, \mathbf{y})$

But I'm stuck in proving the opposite direction.
Answer: Hint: Let $v_i$ be an orthonormal basis. Define
$f(v_i,v_j)=a_{ij}$
Now check that the matrix $A=(a_{ij})$ does the trick by testing it again on the orthonormal basis $v_i$.


Cannot figure out role-based pricing rule for custom Product Display/Variation in Commerce Kickstart

Question: After working thru the Commerce tutorial and understanding roles, I thought it would be straightforward to set up a pricing rule based on user role, but I cannot seem to build the correct "Action" in Rules.
I want to reference a "Dealer Price" field in a Product Variation is the User Role = "dealer"
Error: Data selector commerce-line-item:commerce-product:field-dealer-price:amount for parameter amount is invalid.
Here are the steps I took to get to my current spot:

In a custom Product Variation "Bikes", I created a new field "Dealer Price" that mirrors the "Price" field that is already part of Commerce Kickstart
Dealer price
Field Type: Price
Widget: Price with Currency
2, Created role "Dealer"
Initiated creation of pricing rule
Store Settings >> Pricing Rules
Add Pricing Rule
Add Event: Calculating the sell price of a product
Add Conditions: 1. User has role(s): Parameter: User: [site:current-user], Roles: dealer; 2. Entity has field: Parameter: Entity: [commerce-line-item], Field: commerce_product; 3. Entity has field: Parameter: Entity: [commerce-line-item], Field: field_dealer_price
Then, I get stuck with Add Actions:
Select action to add: Commerce Line Item: Set item to specific amount
Line Item: Data Selector: commerce_line_item (Already inserted)
Amount: [Click on Switch to Data Selector]: commerce-line-item:commerce-product:field-dealer-price:amount
or commerce-line-item:commerce-product:field_dealer_price:amount or commerce-line-item:commerce-product:field-dealer_price:amount

I get an error msg when trying to reference the new price field "Dealer Price".
I feel like I am missing a connection within Commerce that ties the Product Variation to the commerce_line_item, but I haven't been able to identify the missing link. My products show up correctly in the shopping cart, so that hasn't been an issue.
Any help in tying the concepts of "Roles", "Rules", and "Commerce Kickstart" together is appreciated.
Thanks,
Answer:

Entity has field: Parameter: Entity: [commerce-line-item], Field: field_dealer_price

Your price field won't be on commerce-line-item. It will be on commerce-line-item:commerce-product. You need to check for it there.

Amount: [Click on Switch to Data Selector]: commerce-line-item:commerce-product:field-dealer-price:amount or commerce-line-item:commerce-product:field_dealer_price:amount or commerce-line-item:commerce-product:field-dealer_price:amount

So since the field_dealer_price is on the product instance, you'd be looking at this: commerce-line-item:commerce-product:field-dealer-price:amount
An example, without the role condition, would look like this:
{ "pricing_from_another_field_example" : {
    "LABEL" : "Role based pricing example",
    "PLUGIN" : "reaction rule",
    "OWNER" : "rules",
    "REQUIRES" : [ "rules", "commerce_line_item", "commerce_product_reference" ],
    "ON" : { "commerce_product_calculate_sell_price" : [] },
    "IF" : [
      { "entity_has_field" : { "entity" : [ "commerce-line-item" ], "field" : "commerce_product" } },
      { "entity_has_field" : {
          "entity" : [ "commerce-line-item:commerce-product" ],
          "field" : "field_dealer_price"
        }
      }
    ],
    "DO" : [
      { "commerce_line_item_unit_price_amount" : {
          "commerce_line_item" : [ "commerce_line_item" ],
          "amount" : [ "commerce-line-item:commerce-product:field-dealer-price:amount" ],
          "component_name" : "base_price",
          "round_mode" : "1"
        }
      }
    ]
  }
}

Comment: Thank you for taking a look at this. I am missing something that is probably very obvious. In the "Fields search" report, I confirmed where the standard Drupal Commerce "commerce_price" field and my new "field_dealer_price" is used. Right now, they are used only on the new Product Variations that I created: Bikes, Hubs, etc.
Comment: **Correction** I changed my third condition to "Entity has field" Parameter: `Entity[commerce-line-item:commerce-product,Field: field_dealer_price`. I had mistakenly deleted a condition in my zeal to fix things....Now everything works :) Holy cow! Thank you very much.
Comment: No problem! Please be sure to mark your question as answered.


Hotkey in vs code to switch between python interactive window and active editor?

Question: Is there a hotkey ( or a command you can assign one to ) in vs code to switch the cursor between a python interactive window and an active editor widnow?
Comment: @rioV8 that goes to terminal, not the python interactive window.
Comment: have you tried `Ctrl+backtick` (key next to 1)
Answer: CTRL+1 and CTRL+2 will switch between groups. So if you have the interactive window in one group and your code in another, you can uses these shortcuts to switch between them.
Comment: My cursor does not remain active when switching to the interactive pane (using command-2 on a Mac). Does anyone have a working solution to easily jump between the editor and interactive panes that keeps the cursors active? Tx
Comment: My cursor does disappear from interactive window with CTRL+1, and, upon returning to the interactive window with CTRL+2, my keyboard just controls the scrollbar... :_(
Comment: @footfalcon Not ideal but I've been pressing CTRL+2, then SHIFT+hold tab, which cycles through a few windows and eventually resets on the interactive input.
Comment: Thanks, that's the ticket!  If there were something like "alt tab" that just goes back and forth, that's the only thing better, but this is so close, AND, the cursor doesn't disappear ( I was trying something else and the cursor moved off of the input of the interactive window )
Answer: Unfortunately there is no way to switch between those without issues as of now, check these issues: https://github.com/microsoft/vscode/issues/97992
https://github.com/microsoft/vscode-python/pull/12066
Answer: The way the run code to interactive was somehow not specified correctly in
jupyter v.2021.10.11....
I added the following to the user defined keyboard shortcuts
<code>[{
    "key": "ctrl+;",
    "command": "jupyter.execSelectionInteractive",
    "when": "editorTextFocus && !findInputFocussed && !jupyter.ownsSelection && !notebookEditorFocused && !replaceInputFocussed && editorLangId == 'python'"
  }]
</code>
so now "control" + ";"
runs it in interactive without changing other short cuts.


Is there a Microsoft certified SSH server?

Question: Does Microsoft actually validate SSH servers?  My manager will only allow me to use certified products not open source.  I am allow to use my putty client, so I need a ssh server that works with putty.
Comment: What would you use a ssh server for if you had one?  You understand that even if you did install an SSH server that doesn't necessarily mean that you would get a bash shell and a *nix style environment.
Comment: What if the product is both verified and open source?
Comment: How is Putty allowed when it isn't Microsoft Certified? Could you get other products "allowed" in the same way? You should also know that Putty is in fact "Open Source"
Comment: Why not just use Microsoft's certified telnet server?
Comment: What Goyuix said ;) If your manager thinks "certified" products are better for security, it's pointless if your client endpoint is not certified. Therefore, why not allow an "uncertified" server? BTW: I would guess (you might want to verify) that Windows certification quite probably does NOT check all security elements of (SSH) servers, only that they play nice with other software.
Comment: @Steve not to speak for the OP, but telnet is about as insecure as a remote management protocol gets. No one should be using it anywhere nowadays.
Comment: @MarkM but it is certified :-)
Answer: I don't know about their certification status, but PowerShell SSH Server has been featured on Microsoft TechNet.
Surely that counts for something.
Answer: If your manager wants closed source products (for whatever the reason) then VShell Server is a choice.
Comment: But is that certified to run on Windows, that did seem to be the primary requirement.
Comment: Why not drop their sales department an email?
Comment: adamo - Thanks for the suggestion, VShell Server said they are NOT Microsoft certified Any other ideas?
Comment: You are not going to find such a product. It is important to understand what your boss wants. Boss either does not want an ssh server to be installed and for this reason he's sent you to a search in vain, or follows the "nobody ever got fired for buying IBM" doctrine, which means that his job security reflects prompt him to buy "certified" software. So what he in fact wants is some assurance that the product is legit. VShell is such a product, www.ssh.fi is another. Just do not expect a Microsoft stamp on a product not developed by them. So the trick is to find a means to make boss comfortable
Comment: NO, his boss wants to run server software of companies caring enough to actually run through the MS certiicatin process, which in itself is easy enough to pass for any vendor who cares. EVERY sensible approach. Stupid vendors not realizing this.
Comment: Is there a link to this process? Maybe we can push windows ssh vendors to go through it then
Answer: Unfortunately your only choice is FortressSSH . Searching through every SSH server I could find it was the only marked as Windows Certified


Ranking based on binomial data (example: website conversions)

Question: Let's say I have a database of all websites and their corresponding viewers and buyers. I want to rank the websites based on the conversion rate or where we see more conversion.
So example:
website   viewer   buyer
abc.com   1000     10
xyz.com   10       2
def.com   200      1

How do I rank the websites so that I know the top websites are the ones which have higher conversion rates.
Conversion is buyer/viewer
Comment: This is more of a statistics question. I didnt want to mention anything in the beginning and wanted to check the approach people suggest. But apparently there would be some statistical principles which would get used here. And lets just say you are given this question. How would you solve it. Forget the programming part. That anyone can code. What approach to be taken to rank the website if all we have the above data
Comment: Is this a programming question?
Comment: and the `conversion` column is calculated as `buyer/viewer`? Can you provide some more explanation as to whether you are looking for suggestions for more variables to include or how to compare `conversion` values that come from different dataset sizes, e.g. 1000 for abc.com, 10 for xyz.com
Comment: Yes. All we have is that table. So one needs to rank the sites based on the conversion rates defined above. And yes you are right the sample is diff for different sites. so how to rank?
Answer: A "dumb" ranking buy buyer/viewer is generally not what one wants, because then the websites with 1 buyer / 1 viewer are ranked to the top. To account for that, one can use the lower end of the credible interval of the unknown conversion-rate.
Explanation
The observed number of trials (viewers) and successful trials (buyers) can be produced by differing true unknown conversion-rates. The less trials we have, the broader the range of possible true unknown conversion-rates is. The credible (or confidence) interval describes this range up to a desired level of precision.
If we take for example the lower bound of the 95 % interval (which is the (100-95)/2=2.5 percentile), then the (bayesian) interpretation is: Given the data we have, the probability that the true conversion-rate is less than this bound is less 2.5 %.
Regarding the method I prefer the inverse of the cumulative beta distribution (since the beta distribution models the true unknown conversion rate in a bayesian way), but more methods are available. See Binomial proportion confidence interval wikipedia page for an overview and the r-package binom for an implementation of various methods.
Application
The inverse of the beta cumulative distribution function can be calculated by using wolfram alpha.
InverseBetaRegularized[0.025,successfultrials+1,trials-successfultrials+1]

So ...
    website   viewer   buyer  old-rate    new-rate     rank
    abc.com   1000     10     0.01         0.0055       2
    xyz.com   10       2      0.2          0.0602       1
    def.com   200      1      0.005        0.0012       3

Is there more one can do ?
One can add a so called prior. The prior consists of two parts

prior rate: The knowledge we have without looking at a specific shop. In general taking the average across all instances is helpful
prior weight: In your case the number of viewers. If the weight is equal to the viewers of the specific shop, then both the prior rate and the actual rate is treated equally. As soon as a shop generates more viewers, the reliability of its own estimate increases and it shifts more and more away from the prior rate.

Unfortunately, setting the weight is tricky: One approach I found helpful is to estimate the minimum amount of views a site must generate (on average) to generate a desired minimum amount of buyers (on average) given the prior rate.
The resulting ranking is calculated by
$\frac{buyers + priorrate*priorweight}{viewers + priorweight}$
Applied on your example:

The macro-average of conversion-rates ((mean(c(10/1000,2/10,1/200))) is 7.17 % = prior rate
Let's say we need at least 3 buyers, otherwise the site is not interesting. This (3/mean(c(10/1000,2/10,1/200))) leads to ~ 42 Views = prior weight (rounded up)
website   viewer   buyer  old-rate    new-rate     rank
abc.com   1000     10     0.01         0.0125       3
xyz.com   10       2      0.2          0.0963       1
def.com   200      1      0.005        0.0166       2

This approach can be combined with the confidence / credible interval above by first adding a prior and then estimating the lower bound of the confidence interval.
Update 23.06.
Regarding Thomson Sampling
I had to look it up:

https://en.wikipedia.org/wiki/Thompson_sampling
http://jmlr.org/proceedings/papers/v23/agrawal12/agrawal12.pdf

The application makes only sense here if one has to select a particular page to present it to a user while maximizing the expected rewards (conversions). But this is only my view, maybe your frequentist friend had something different in mind.
Keywords: Reinforcement Learning, Mulitarmed Bandit
Regarding the sorting per p-value (by comparison with overall conversion-rate)
Yes, this approach makes sense. But since p is the objective probability that one would observe the current results assuming the Nullhypothesis (both are equal) is true, you may run into a problem when you have a lot of sites and two sites are way better than the average, so both of there p-value is 0. How do you sort these ?
The Bayesian approach overcomes this issue. Another option here is to calculate the pairwise subjective probability p(siteA > siteB| data) instead of using the comparison by mean-conversion-rate. This approach also allows the usage of priors. This link might be helpful:

Bayesian AB testing

Do not get confused by the Beta-Distribution. It is just the distribution of the true unknown conversion-rate given the current data.
Last but not least: The t-test is an unnecessary approximation here, since we are dealing with a binomial distribution. See my answer to this question: Difference between G-test and t-test and which should be used for A/B testing?
Comment: Very well answered. Unfortunately I knew some Bayesian thing is to be used but I am not a very Bayesian guy as of now.

Couple of things:
One friend of mine suggested something called as randomised prob matching using Thomson Sampling. According to him this is a classic example of Thomson Sampling. Is this correct?

Secondly, the way I thought about this is and again since I know as of now only frequentist approach is that consider each site as a binomial dist. Like for abc.com we have 1000 views. so consider it as a sample of 1000 with 1 or 0 as buys.  Contd.. on next comment
Comment: .... and then find conversion rate for each site as given. Then find the overall conversion rate across site. Total buys/total views. Do t-test of each site conversion rate with overall conversion rate and then rank by increasing order of p value. Lower the p value higher the significant diff between conversion rate of a site with the overall conversion rate and hence low p value. Is this approach sound?
Comment: @user Sorry for the late reply, I was away over the course of the weekend. I have updated my answer.
Answer: You can make a score by doing an average. And since the values are not in the same unit, you can use a geometric mean that will make them compatible by raising them to higher dimension. We got 2 components here, 1 is the rate of conversion and second is the amount of buyers. You simply multiply the factors (or divide by the ones where the smaller value is the better). Since you're interested in ranking, then no need to compute the negative power. The multiplication is enough.
score = buyer / viewer x buyer = b^2 / v
Higher score will mean higher ranking. You could even take the log of the scores for easier reading because the log pulls back the too large and too small values to better interval.


if e else entre usurario

Question: Assim, criei um tabela usuario e uma negocio, ambas herdam da tabela user do django.
Perfeito, ambas estão seus usuarios e tabelas.
Porem tenho um menu e uma página perfil.html
e queria que nesse menu aparecesse a foto e nome de quem estivar logando e o mesmo na página perfil
mais como tenho user.usuario e user.negocio
não sei como fazer essa lógica se estiver logado com user.usuario mostrar tal coisa senão mostrar user.negocio
alguém quer perder um tempo e me ajuda?
Answer: Não sei se entendi muito bem sua pergunta mas vamos lá.
Pelo que entendi, você tem as duas classes (Usuario e Negocio) que herdam de User e quer verificar no template se o usuário que está logado pertence à classe Usuario ou Negocio.
Nessa situação, seria algo do tipo:
{% if request.user.is_authenticated %}
    {% if request.user.usuario %}

    {% elif request.user.negocio %}

    {% endif %}
{% endif %}

Aqui, estou supondo que tanto Usuario quanto Negocio possuem um campo foto do tipo ImageField.
Porém essa não é a melhor forma de fazer algo assim no Django. O ideal seria você ter 2 grupos (usando o django.contrib.auth.models.Group) e vincular cada usuário a um dos grupos de acordo com a sua necessidade.
Se você quiser, coloca o projeto no Github e manda o link que te ajudo mais.
Abraço!
Comment: oi amigo excelente lócgica, mais apareceu o seguinte erro `Exception Type: RelatedObjectDoesNotExist at /accounts/login/
Exception Value: User has no usuario.` Uso o accounts do django admin para login
Comment: cara acho que deu certo, a noite te confirmo, mas se deu salvou minha vida
Comment: opa amigo, tentei e não deu certo, ele mostra a empresa, mais dai quando logo no usuario ele não mostra, e caso dentro do if do user.usuario eu peça a foto da user.negocio.foto ele aceita
Comment: Olá Mauricio. Você consegue me enviar o seu código pra eu dar uma olhada? Principalmente do models.py, views.py e o html que está tentando fazer?
Comment: https://github.com/MauMaster/Pet-Aqui    claro, só que lá dentro o menu nao ta com esse if, else elif
Comment: Agradeço qualquer ajuda, faz parte do meu TCC to desesperado
Comment: até publiquei outro post https://pt.stackoverflow.com/questions/369788/desdenti%C3%A7%C3%A3o-entre-dois-tipos-de-usu%C3%A1rios-usando-if/369799#369799
Comment: Mauricio. Estava preparando a resposta quando fechou o tópico. hehe
Comment: já reabri, por que não consegui ainda, agradeço se me ajudar amigo
Comment: Vi a outra pergunta lá. A resposta do Allan está bem completa. Mas, eu criei um novo models.py que acredito que deixaria seu banco de dados mais otimizado. Dá uma olhada e vê se te interessa: https://pastebin.com/0De0Sk0u
Comment: Outra coisa. Como eu disse, não é a forma mais correta de fazer isso mas, para um momento de desespero, funciona bem. hehe Qualquer coisa, estamos aí.
Comment: olha até onde fui, se puder me ajudar https://pt.stackoverflow.com/questions/370822/if-else-entre-dois-tipos-de-usuario

if e else entre usurario


How to plot adjusted Kaplan-Meier Curves?

Question: I am trying to plot adjusted Kaplan-Meier curves.  I know publications like to see something graphical.  But using R, I don't know how to go about adjusting for something like age, gender, income when graphing a survival curve.
Otherwise my curves will always be just crude and unadjusted, which I'm guessing people will not like.  Any ideas?
I have 4 groups to compare.
Comment: The crude KM curve is a kind of prediction at the means. Therefore, you can backtransform a KM curve using a complementary log log, apply the covariate values estimated from the Cox model, and transform back to the survival scale to plot them.
Answer: The only way to provide differential survival with  true KM curves is to generate new curves for different groups. You could then display a curve for all persons of group 3, for example. The number of units in each group will decrease as the number of strata increase. However, this method is empiric and does not truly adjust the sample to some chosen set of values.
I am most familiar with methods for obtaining adjusted curves derived from Cox or parametric survival models. Generally speaking, the role of an adjusted curve is to graphically display the expected mortality (or mortality transformed to survival) of the sample if a single or combination of values is set to some fixed value or set of values, respectively.
For example, one might find from a Cox model the hazard ratio for blood pressure is 1.1. Thus, for each 1 unit increase in blood pressure, the average hazard at a given time point multiplies by 1.1. Now, if we wanted to display the mortality curve for all units under analysis (sample) adjusted to a blood pressure of 1 standard deviation above the mean, we could display an adjusted curve.
Here is a self-contained example using group for your reference. Note that the final, adjusted curve is for the mean of group which, for most applications, would have no real meaning. Also note that transformation from mortality to survival are required for this method.
library("survival")
require("survival")

days <- rpois(100, 3)
status <- rbinom(100,1,0.34)
group <- sample(c(1,2,3,4), 100, replace=TRUE)
df <- data.frame(days, status)

#overall survival
surv <- survfit(Surv(days, status)~1)
summary(surv)
plot(surv)

#survival by group
kmsurv <- survfit(Surv(days,status) ~ strata(group), df)
plot(kmsurv)

#survival adjusted to group effect
cox <- coxph(Surv(days,status)~group, df)
summary(cox)
plot(survfit(cox))

Essential information on the R code can be found here.
Lastly, it is my opinion that adjusted survival analysis is generally statistical sleight of hand as the adjustment process can 1) be used for unrealistic patterns of covariates, 2) fool the reader into believing that non-significant effects can result in some displayed survival/mortality pattern and 3) be confused with empiric curves, leading readers to believe you have more events or information for each subgroup/pattern than you actually possess. I would carefully consider why adjusted curves are desirable over adjusted hazard ratios before spending too much time.
Comment: I also **strongly** suggest plotting the raw KM curves, rather than adjusted Cox PH curves. By plotting the KM curves, a reader can quickly visualize how reasonable the PH assumption is. With the Cox PH adjusted curve, it automatically looks perfect (even if it's horrible).
Comment: +1. I particularly agree with the cautions about how adjusted survival curves can be misleading.
Comment: Multiple things are wrong here. 1.) In your example, you did not adjust the survival curve for anything, because you did not add any covariates to adjust for to the model. 2.) The method you are using is known to be biased as pointed out by @friendlystatsguy. 3.) Statistical adjustment is not "sleight of hand". If there are known measured confounders it is absolutely necessary to get unbiased estimates. 4.) adjusted hazard ratios do not have a causal interpretation as pointed out by multiple authors (see https://doi.org/10.1146/annurev-statistics-040320-114441)
Comment: Raw KM curves are usually misleading.  By not adjusting for anything you are more likely to see apparent non-proportional hazards that can sometimes disappear when accounting for outcome heterogeneity.
Answer: There is a lot of literature on the subject and there is also some R Code available.
The method Todd D described is known as average method and it is conceptually and empirically biased (See for example Nieto (1996). It is generally recommended to use direct adjustment or conditional adjustment instead.
The survminer package contains a function called ggadjustedcurves which implements direct adjustment (given a cox model) in the method marginal. A non-parametric method to adjust survival curves, which is based on a weighted kaplan-meier estimator, can be found in Xie (2000). This method and a corresponding weighted log-rank test are implemented in the R-Package RISCA.
Answer: Unadjusted Kaplan-Meier curves are not a problem in general. The question is: what do you want to show? Usually you would want the plot to be a visualization of the causal average treatment effect of a grouping variable on the survival probability. If your groups were randomized in a large study, you can go usually go ahead and use the standard Kaplan-Meier curves, since randomization should take care of confounding already.
However, if you are analyzing the effect of a non-randomized group (which is always the case in observational studies), it may be very important to adjust for confounders to obtain an unbiased estimate of the treatment-specific survival curves.
The literature on causal inference deals with this problem in great detail. Many methods have been proposed to obtain confounder-adjusted survival curves. Which method you want to use depends on your specific situation. In a recent article me and my colleagues give a detailed overview and comparison of those methods which may provide some guidance (https://doi.org/10.1002/sim.9681).
I also developed the adjustedCurves R package which implements most of the available methods. Here is a very small example on how this package may be used to obtain adjusted survival curves using g-computation:
# install the package if needed
install.packages("adjustedCurves")

# load packages
library(adjustedCurves)
library(survival)

# just used to make the example reproducible
set.seed(42)

# simulate some example data
sim_dat <- sim_confounded_surv(n=50, max_t=1.2)
sim_dat$group <- as.factor(sim_dat$group)

# outcome model
cox_mod <- coxph(Surv(time, event) ~ x1 + x2 + x4 + x5 + group,
                 data=sim_dat, x=TRUE)

# using g-computation with confidence intervals
adjsurv <- adjustedsurv(data=sim_dat,
                        variable="group",
                        ev_time="time",
                        event="event",
                        method="direct",
                        outcome_model=cox_mod,
                        conf_int=TRUE,
                        bootstrap=FALSE)
plot(adjsurv, conf_int=TRUE)

More information can be found in the extensive documentation of the package and the article I cited.
Comment: Glad to call out CI on this.  The causal inference literature has largely failed to correct this error, insisting on marginal estimates that do not necessarily apply to anyone even going so far as to sometimes label sample averaged effects as population average effects.  Elaborated [here](https://fharrell.com/post/ipp).  This causal approach applies only when doing simple random samples or probability samples from a population.  That is far from the case in randomized trials.
Comment: Why is $g$-computation needed for this?
Comment: Because without g-computation you can only obtain *conditional* survival probabilities from the cox-model, namely the survival probability at *t* for specific covariate values (for example someone with group = 1 & x1 = 0 & x2 = 1 & x4 = 1.2 & x5 = 2.3). If you are interested in the survival probability that would have been observed if all people in the dataset had been set to group = 0 or group = 1, this is insufficient.
Comment: The "if all people" estimand is not relevant.  It merely hides differences among people.  And it does not transport outside the sample unless you have a true probability sample from the population.  Covariate-specific estimands transport, not depending on covariate mix in the sample.
Comment: I disagree. The estimand I described is a type of average treatment effect, which is the standard estimand in almost all randomized studies. Even if we accepted your opinion as truth, how would you visualize the effect of the group variable using survival curves if you have many, possible continuos variables in the cox model?
Comment: It is not any kind of standard in RCTs though commonly used.  It is essentially uninterpretable and not what we strive for.  The answer to your question is easy: Pick 5-10 representative covariate combinations and adjust survival curves to those settings.  Example [here](https://hbiostat.org/rmsc/cox.html#fig-cox-cabg-ef).  This example used the mean of all covariates; individual "real" settings are preferred.
Comment: Judging by the very large literature on causal inference, you are pretty much alone with this opinion. Also, "representative covariate combinations" sounds incredibly arbitrary. This is not the right place to discuss this though. Let's just leave it at that.
Answer: There is also recently published paper which can create adjusted survival and cumulative incidence curve for survival and competing risks outcome respectively. https://www.tandfonline.com/doi/full/10.1080/03610918.2023.2245583.
Furthermore, there is also an R package adjSURVCI.
https://cran.r-project.org/web/packages/adjSURVCI/index.html.


Trouble identifying guitar model

Question: I am a beginner to electric guitars, although I have been playing acoustic guitars for a few years. I have recently bought a Squier Stratocaster Pack (SSS), but nowhere on the internet does it mention the model of the guitar, not even in the manual, warranty card... I think it may be the affinity series but I am not sure. Can someone please identify the guitar.
This is the link from where I bought it
Answer: It appears this guitar is specifically made for and sold as part of the Stratocaster Pack with the amp and other accessories. The least expensive Squier Strat is the Bullet for $179 and the Affinity series costs $229. Those prices are just for the instruments so at $219 for the package I can't imagine it's any of the models they sell as a stand-alone guitar. Here's a link to the Squier page:
https://shop.fender.com/en-US/squier-electric-guitars/
These packs are geared for beginners who can buy everything they need to get started in one shot. If the guitar is structurally sound (straight neck, well put together, etc.), plays well (it might need a bit of adjusting) and sounds good it's a very good deal for getting started on electric guitar, good luck!

Trouble identifying guitar model


Are there hook_functions to handle submissions from the webforms module in Drupal 6?

Question: I'm asking this question as a possible way to handle another issue I possted about ("Moving webform uploads to a non-public folderspace"). Is there a hook_function that I can use to handle the submission of a webform in Drupal 6. That is, can I use a hook to relocate a file uploaded through a webform and update the database entry to mark that upload files new place?
Answer: Take a look at these three hooks.

hook_webform_submission_presave() - which is called before saving.
hook_webform_submission_update() -  which is called after the submission is stored in database
hook_webform_submission_actions() - which provides a list of actions that can be executed on a submission.


Do I need humbucking pickups?

Question: I plan to record in a few months and the studio engineer recommends direct input for electric guitar. I'm mainly a drummer and am new to this...
Based on what I've otherwise been advised, I assumed that although the bass would be DI, the guitar would not be. However I listened to songs this guy has produced using DI for guitar and the distortion (rock songs) sounds good. And I assume using this method means being able to choose from a wealth of software-produced distortion effects, rather than having to perfect things physically with just the right amp and/or distortion pedal.
Incidentally, I had been thinking about getting a guitar with humbucking pickups as I understand they're generally reckoned to be better for distortion. But using DI to record, do I need to worry about this any more? (Obviously a live situation is different, but I'll think about that later).
In other words, using my single coil Strat copy guitar, can I achieve the same effect as if I was using an SG/Les Paul-style guitar, given that presumably the sound of the guitar will be determined by the software?
Comment: This really depends on your musical influences and the sound you want.  Who would you like to sound like?  We could name plenty of players who have got great distortion sounds out of a single coil strat-type!  A Strat and a Les Paul will sound different, but they're both capable.
Comment: I've been thinking about this all day and my experience is that different types of distortion sound better with different types of guitar. For instance, huge amounts of germanium fuzz sound nice and thick and rich with a single coil strat, but can come off muddy and annoying with a Les Paul. On the other hand, a Les Paul sounds amazing through an all-tube amp that is set to light to medium overdrive. If I had to have one guitar sound for all uses, it would actually be a single coil telecaster style. Single coil bite, but nice wood tones also.
Comment: FWIW I have a strat with Dimarzio Tone Zone humbucking pickups.(http://www.dimarzio.com/pickups/strat/hum-canceling-strat). I can also coil tap them and get a to get the single coil sound. So having the humbucker option doesn't necessarily mean a new guitar if you're happy to play with the pickups.
Comment: Go to a music shop and play every guitar you like on the same amp
Comment: There's still the option of having a humbucker installed with the option of coil-tapping it.
Answer: The fact that you DI the guitar doesn't change the differnce between distorted single coils or humbuckers, so if you're thinking of getting a SG or Les Paul etc for nicer distortion, that'll still be the case if you DI.
Strats have a nice clear kind of distortion wheras Humbuckers are better for pure grunt. I'm not sure of the science behind it but the original sound is quite different:
Single coil = lighter sounding, full of character, lots of top end in the sound (="detail", if you like)
Humbuckers = generally a bit less top end, more output, and someting-I-don't-know-what that makes them sound the biz though a distorted amp.
So you'd think upping the gain on a single coil and bringing the tone down a bit would make it sound like a humbucker ?
Hmm .. I've modified my strat so that the bottom tone knobule works on the bridge pickup, so I can turn the tone down so it's less twangy. I also turnt he gain up on the amp (well, effects board in my case).  This is specifically for playing things like ACDC. It works a bit, in that the distorted sound is better than if left with tone fully up, but there's definitely something else goiung on with humbuckers which make them sound like they do.
Getting a decent DI sound from a distorted guitar is quite an art. It might be worth finding out from the fella you mention which effect he uses to get that sound, to save trying loads out ?
I have an unjustifiable intuition that it might go lie  this :

Single coil through humbucker effects in search of humbucker sound = fair amount of experimentation potentially resulting in goodly smiles
Les Paul/SG though fairly standard distortion effect = Immediate smiles and potentially some bouncing about like a loon

One othe thing to mention is that if you just plug a distortion stomp box into a recorder of some sort, you'll get a horrible buzzy sound (regardless of guitar) which I've heard best described as "like a wasp in a paper bag". That's because guitar amps do a lot to the sound, and things like that sound ace thogugh an amp but not if plugged right into (effectively) your nice hi-fi monitor speakers. So you need to use an effect designed for DI - or has an 'amp simulator' built in. I put that in inverted commas because some amp simulators are a bit .. kind of.. crap and frustrating. Hence recommendation to find out the make of the one you like the sound of.
Answer: If you want the sound of a humbucker, you need to get one. Emulations using single coils will not do. Sure, you could play around with EQs and compressors and maybe get you in the ballpark, but it will always be noticeable.
If you have to have only one guitar and want to get the widest range of sounds, your best bet would be humbuckers with coil split option (HSH layout).
Split humbuckers do not sound exactly like traditional single coil pickups, but they are pretty close (since they effectively operate as single coils. It's just that the resulting single coil is a bit different than a single coil only pickup would be).
You can play most genres of music with single coils, but for authentic high gain 80's rock, stoner, doom and other metal genres, humbuckers are mandatory.
As for getting a new guitar: Some strat copies come with the bridge pickup slot already routed for humbuckers under the pickguard. Lift up your pickguard and check that. If it's already routed, you could just get a new pickguard and a humbucker and drop it in.
This way you'd get a popular SSH pickup layout which is also very flexible.
IMHO, the bridge single coils in strats don't sound very good anyway, the neck and neck/middle configurations are where it's at.
Answer: The pickups aren't the only part of an electric guitar that affect the sound in noticeable ways. Construction of the body and the neck are high on the list, as are hardware items like the type of bridge, nut material, and so on. Even with just the electronics the specs of the tone and volume controls are a factor.
If you like the previous work of the producer you are working with, then they are the best person to ask any gear questions. They might even have some guitars that they like to use on recordings which produce sounds they know how to work with.
If you just want to buy a new guitar, and who doesn't? go to a store and play all the ones in your price range and get the one that grabs you. Better to find your sound than to chase anyone else's tone.
One more note: imagine a drummer asking if they need a solid brass snare. You'd probably say "it depends", and how they tune it and hit it will always be more important than how it's made. You might even suggest they find a snare that sounds great to them regardless of construction.
Answer: Humbuckers give you more distortion because more is being pushed down the wire.
On my humbuckers and my p90's I would estimate 30% distortion on my amp as its is about the same as 100% distortion with my p90's. But I would never need to put it above 75% for my humbuckers as its just noise.
My p90's have a sharper, more piercing and clear sound. My humbuckers are warmer and more fuzzy / slightly muddy.
If you want a new guitar and can afford one, then get one because they can be very different in character :).
However, if this is a stretch you would probably be better off having your "copy" guitars pickups changed for better single coils.
If this is a stretch, the simple and easy and cheap (and surprisingly effective) way would be:

Larger gauge strings give more beef
As will more hefty picks
You might have some more adjustment left in the height of your pickups

(remember increasing string gauge may need a truss rod and intonation adjustment)
Comment: In many cases, the 'pushing more down the wire' aspect can achieved with a separate gain stage.
Comment: String gauge will come into play for feel, especially if you downtune, not much more tonally. It would be more effective to find the closest pickup height that has the most gain without pulling the string or losing any definition.
Comment: While it is more common for modern humbuckers to have a higher output than modern single-coils, in my experience it's not common enough that I would assert that humbuckers have higher output as a rule. And higher output does not mean "better distortion" (the OP's assumption about humbuckers), more the opposite. And there is definitley a clear tonal difference between different guages - maybe it's really subtle between 9s and 10s but if you put on 11s or 12s with a clear signal path, you'll hear it right away.

Do I need humbucking pickups?


No source available for "Reset_Handler() at 0x8984" while debugging an ARM TrustZone Project in Renesas e2 Studio and embOS Library

Question: Summary
I have a C++ ARM TrustZone project which is running on a R7FA4M2 controller and the embOS-MPU RTOS should be used. When I try do debug this project, the debugger does not find the source files as soon the code jumps from the secure project to the non secure project.
Detail
When the debugger is started, the code is found correctly. I see that there's a breakpoint hit at SystemInit(); at address 000010b0
              void Reset_Handler (void)
              {
                /* Initialize system using BSP. */
-> 000010b0     SystemInit();

The MCU State is also in SSD.Secure, so the program starts in the secure project as it should. I can step through the code until this point where the program jumps to the non-secure reset handler.
               /* Jump to the NS Reset_Handler */
-> 00000f54    p_ns_reset();

After that, the source files are not found correctly and the message No source available for "Reset_Handler() at 0x8984 is shown in the editor window.
Observations
Disassembly View
When I now change to the Disassembly View and step through the code in "Instruction Step Mode", debugging is still possible and somehow the name Reset_Handler is also known by the Disassembly view.
             Reset_Handler:
-> 00008984:   push    {r3, lr}
   00008986:   bl      0x8990 <SystemInit>
   0000898a:   bl      0x83f0
   0000898e:   b.n     0x898e <Reset_Handler+10>

It seems that the code is running correctly and also jumps to the non-secure project (which starts at address 00008000).
Project without embOS-MPU
The non-secure project has an embOS-MPU library included libosT8MLVHLDP_TZ.a and there are some minimal configurations that the project is compiling without errors. There are no Task or other embOS features implemented. When i remove this library from the project, the debugger is working as it should and is also able to find the files in the non-secure project. This means: An empty Bare-Metal ARM TrustZone Project is debuggable.
What I've checked

Compiler Debug level: Default (-g)
Debug format: dwart-4
Debugging with different optimization levels (-Og, -O0, -O2)
The linker command:
-T "C:\\Workspace\\project\\fsp.ld" -Xlinker --gc-sections -L "C:\\Workspace\\project/script" -Wl,-Map,"host_sw_ns.map" --specs=nano.specs --specs=rdimon.specs secure.o "C:\Workspace\project\embos_mpu\Lib\libosT8MLVHLDP_TZ.a"

The .elf Files
There is a .elf file for each project host_sw_s.elf and host_sw_ns.elf. The .elf file of the secure project host_sw_s.elf can be unfolded in the Project Explorer:
host_sw_s Binaries
Unfolding the sw_ns.elf is not possible
host_sw_ns Binaries
When building without the embOS library, the functions are visible.
What I've tested
Adding Symbol File while Debugging
When the Debugger is already running I've tried to load the symbols again. The Debugger Console looks like this
add-symbol-file "C://Workspace//project//Debug//host_sw_ns.elf" 0x8000

add symbol table from file "C://Workspace//Gilgen//project//Debug//host_sw_ns.elf" at
    .text_addr = 0x8000
(y or n) [answered Y; input not from terminal]
Reading symbols from C://Workspace//project//Debug//host_sw_ns.elf...(no debugging symbols found)...done.
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module C:\Workspace\project\Debug\host_sw_ns.elf]

It seems that there is Dwarf 5 defined in the .elf file even if in the setting Dwarf 4 is defined.
Checked gdb Version
I'm using the Renesas e2-studio v4.4.0 with the build in gdb debugger. At the start of the Debugger it is shown that there's an older version of the gdb in use
GNU gdb (GDB) 7.8.2-20221107-Build_1

Dwarf 5 is supported above gdb v10.1 which should cause the Dwarf Error Message.
Assumption
My assumption is that the embOS library was built with Dwarf 5 Debug information and when the linker does his job, there's somewhere Dwarf 5 defined in the .elf file and the old gdb does not understand this symbols.
Unsuccessful trials
Debug with Release embOS library
There's also a release build version of the embOS library libosT8MLVHLR_TZ.a. Maybe the release build library does not contain any debug information and therefore dwarf 5 is not defined.
→ Same error
Running external gdb Debugger
On the System is an gdb v12.1 debugger installed with Cygwin. In the Debugger settings I've changed the debugger to the specific path. The gdb does not start with following error message:
Could not determine GDB version using command: gdb --version.
When a local console is opened in e2 studio the command is working
C:\Renesas\RA\e2studio_v2023-04_fsp_v4.4.0\eclipse>gdb --version
GNU gdb (GDB) (Cagwin 12.1-1) 12.1

I've not troubleshooted this solution further.
Trying to update internal e2-studio gdb Debugger
I've not found a way yet. Is this even possible?
Answer: The Problem was, that the embOS-MPU library was Build with GCC V11.2 which included Dwarf-5 debug information. The integrated gdb Debugger v7.8 in e2-Studio v4.4 does not support Dwarf-5.
I've fixed the Debugging issue by integrate the Source files of the embOS instead of the pre-compiled library File. With this solution, the integrated GCC and gdb Debugger of the e2-Studio can be used with Dwarf-4 debug information.


how to show a summary of a file list grouped by directory

Question: Suppose I have a list of files, and I want to summarize them by directory, to a certain depth.
For example, I want to see which directories a git commit changed, or a git diff affects.
The output of git diff --name-only or git show --name-only lists something like:
foo/1.txt
foo/2.txt
bar/3.txt
bar/4.txt

but a summary by directory would show:
foo/
bar/

Comment: Adding an git commit example and desired output would be nice.
Comment: @StefanBecker done.
Answer: Pipe the file list to perl, then extract the first N pieces of the filename. (This bit expects all the items to have at least this depth!)
Then print the extracted text if it's not already been seen.
For example:
git show --name-only SHA | perl -n -e 'my ($root) = m[^((\w+/){DEPTH})]; print $root . "\n" if !exists $seen{$root}   ; $seen{$root} = 1;'

where DEPTH is replaced with the depth of directories to consider.


Fredholm Alternative as seen in PDEs, part 2

Question: ...continued from part 1.
I have more questions concerning more of Evans' proof in the Fredholm Alternative. As stated in my previous question, I do not have functional analysis background, but I want to still understand the proof. So my five questions will be fundamental and basic. Any answers and clarifications are welcomed.
NB: For anyone who is about to suggest me to read suggested books or point out that I have a lack of background in functional analysis (and even theoretical linear algebra), please don't. Because I already know that. As suggested by a user on my previous question, I plan to read the Haim Brezis textbook over the summer when I don't have classes in the regular school semesters.
With that said, I am printing the theorem below, along with the second excerpt of the textbook's proof and my basic questions to follow.
(PDE Evans, Appendix D, Theorem 5)

THEOREM 5 (Fredholm Alternative). Let $K : H \to H$ be a compact linear operator. Then
(i) $N(I-K)$ is finite dimensional,
(ii) $R(I-K)$ is closed,
(iii) $R(I-K)=N(I-K^*)^\perp$,
(iv) $N(I-K)=\{0\}$ if and only if $R(I-K)=H$,
and
(v) $\dim N(I-K)=\dim N(I-K^*)$.

The second excerpt of the textbook proof:

Assertion (iii) is now a consequence of (ii) and the general fact that $$\overline{R(A)}=N(A^*)^\perp \text{ for each bounded linear operator }A: H \to H.$$

a. Can the overline in $\overline{R(A)}$ be explained? Why do we not need the overline for statement (iii), $R(I-K)=N(I-K^*)^\perp$?

To verify (iv), let us suppose to start with that $N(I-K)=\{0\}$, but $H_1=(I-K)(H) \subsetneq H$. According to (ii), $H_1$ is a closed subspace of $H$. Furthermore $H_2 \equiv (I-K)(H_1) \subsetneq H_1$, since $I-K$ is one-to-one. Similarly if we write $H_k \equiv (I-K)^k(H)$ ($k=1,\ldots$), we see that $H_k$ is a closed subspace of $H$, $H_{k+1} \subsetneq H_k$ ($k=1,\ldots$). Conversely, choose $u_k \in H_k$ with $\|u_k\|=1$, $u_k \in H_{k+1}^\perp$. Then $Ku_k-Ku_l=-(u_k-Ku_k)+(u_l-Ku_l)+(u_k-u_l)$. Now if $k > l$, $H_{k+1} \subsetneq H_k \subseteq H_{l+1} \subsetneq H_l$. Thus $u_k-Ku_k$, $u_l-Ku_l$, $u_k \in H_{l+1}$. Since $u_l \in H_{l+1}^\perp$, $\|u_l\|=1$, we deduce $\|Ku_k-Ku_l\| \ge 1$ ($k,l=1,\ldots$). But this is impossible since $K$ is compact.

b. I know that $I$ and $K$ considered linear transformations because $I$ is the identity operator (is this true?) and $K$ is given to be a compact linear operator. Hence, $I-K$ is a linear transformation, right? Due to a theorem from linear algebra, is it because $I-K$ established to be one-to-one because the nullspace $N(I-K)=\{0\}$ as initially given?
c. Why does the one-to-one property $I-K$ implies that we can write $H_2 \equiv (I-K)(h_1) \subsetneq H_1$, which would allow us to write after $k$ iterations $H_k \equiv (I-K)^k(H)$?
d. How does $u_l \in H_{l+1}^\perp$ and $\|u_l\|=1$ suggest $\|Ku_k - Ku_l\| \ge 1$, which would make $\{Ku_k\}$ not Cauchy and contradict the compactness of $K$? Ultimately, I cannot right now see the connection that allows us to conclude that the rank $R(I-K)=H$, thereby proving the forward direction of (iv)...

Now conversely assume $R(I-K)=H$. Then owing to (iii), we see that $N(I-K^*)=\{0\}$. Since $K^*$ is compact, we may utilize step 5 to conclude $R(I-K^*)=H$. But then $N(I-K)=R(I-K^*)^\perp=\{0\}$. This conclusion and step 5 complete the proof of assertion (iv).

e. Given $R(I-K)=H$, because of (iii) we have $N(I-K^*)^\perp=H$. But how does the transpose of $N$ equivalency with $H$ imply $N(I-K)=\{0\}$?
Follow-up part 3...
Answer: I remember I got stuck on the same questions and hoped that if I read it carefully, it would become cristal clear...
daw already answered a) and b) but I want add one more comment about b): (iv) means that being injective is equivalent to being surjective (for a compact operator), and he says that this is true for finite dimensional maps.

finite dimensional linear maps are actually completely determined by the images of each of the basis vectors (in finite numbers). so we see that this statement is almost like the equivalence between injective and surjective for a maps on finite sets.
one definition of "infinite" (by either cantor or dedekind...) says that such a set is characterized by the existence of a bijection with a proper subset of itself. Hence we can imagine that being injective will not imply bijective in general.

Now the other questions:
c. Let $x\in (H_1)^{\perp}$ which is not trivial by assumption. Its image $(I-K)(x)$ is in $H_1$ but if $(I-K)H_1 = H_1$ then any element $b\in H_1$ can be written $(I-K)(a)$ for  $a\in H_1$. Now we have two pre-images of $b:= (I-K)(x)$,  namely $x$ and $a$ which belong to orthogonal subspaces (i.e. we can be sure they are different). This contradicts injectivity of $(I-K)$
d. the equality is of the form $K u_k +K u_l = (something) - u_l$. The something belongs to $H_{l+1}$ while $u_l \in H_{l+1}^{\perp}$ and has norm 1. Use Pythagoras' theorem.
The general structure of the proof of (iv, $\Rightarrow$) is absurd. Assume the negation of the conclusion, then there is a contradiction to $K$ compact.
e. this is definiteness of the norm: let $x\in N(I-K)$ then for all $y\in H$
$$ \langle x,y\rangle= 0$$
Take $y:= x$...
Comment: Your answer solves my confusion about the proof given in Brezis' book :v
Answer: a) The range of a linear operator is not closed in general, but the annihilator of a subspace is. In the equation $\overline{ R(A)} =N(I-K^*)^\perp$ the space on the right-hand side is always closed. (ii) proves that $R(I-K)$ is closed, hence we do not need to take the closure in the statement of (iii)
b) $I,K,I-K$ are linear operators. However, the statement: $N(I-K)=\{0\}$  implies $R(I-K)=H$ is not true in infinite-dimensional spaces. Hence the lengthy argument.


mariaDB insert multiple rows where 2 cols not exists

Question: I realize similar questions have been asked before, but I can't seem to find a solution that fits this particular scenario.
I would like to insert multiple rows of data into a mariaDB table where the data must be unique (primary key excluded).
Sample table:

enrollmentsID
classID
userID

1
1
2

2
1
3

3
1
4

4
2
2

5
2
7

So if I want to insert a number of rows, I don't want to duplicate what's already present.
The general idea is something like:
INSERT INTO `enrollments` (`enrollmentsID`, `classID`, `userID`)
VALUES (NULL,1,2),(NULL,1,3),(NULL,1,4),(NULL,1,5)
WHERE NOT EXISTS (
    SELECT `enrollments`.`classID`, `enrollments`.`userID`
    FROM `enrollments`)

Here, userID 5 would insert but userID 3 and userID 4 would be ignored.
Unfortunately, the WHERE is causing issues...  Thanks for any help provided.
Comment: Any reason you don't want a unique key on user and class?
Comment: A unique key on 2 columns enforces unique combinations of both columns.  It would not prevent a user being in 2 different classes -- only duplicate rows for a user in the same class.
Comment: @P.Salmon Yes, a user could be in multiple classes.
Comment: @gview, I did not know this!  Thank you to both!  When I tried to "ALTER TABLE `enrollments` ADD UNIQUE(`classID`)" I got error "Duplicate entry '2' for key 'classID_2'"
Comment: @gview, so then I truncated the table and changed classID and userID cols to 'unique' and now I can't add more than 1 user to a class: "Duplicate entry '1' for key 'classID_2':
Answer: As P.Salmon mentioned in the comments, a UNIQUE index on the two columns is likely what you need. The index needs to be on both columns, not a UNIQUE index for each column.
ALTER TABLE enrollments
ADD UNIQUE INDEX (`classID`,`userID`)

From there you can do INSERT INGORE INTO instead of INSERT INTO and that will only insert the unique entries.
Comment: Thanks so much to all.  I was using phpMyAdmin to add unique indexes on each column separately and did not realize this would cause the issue above.  Thanks again!


Why $|| \le\sup\limits_{\|u\|\le 1} || \|z\|^2$?

Question: I'm having trouble understanding a technical argument of a proof similar to Theorem C.51,p 4 from this link, where the statement to prove is: $\|T\|=\sup\limits_{\|x\|\le 1}|<Tx,x>|$ for a bounded linear self-adjoint operator on a hilbert space $T\in L(H)$.
In a sketch it is defined that $M:=\sup\limits_{\|x\|\le 1}|<Tx,x>|$ and it shall be proved that $\|T\| \le M$. The line I don't understand is:
$$4 Re(<Tx,x>)\le |<T(x+y),x+y>|+|<T(x-y),x-y>|\le M\|x+y\|^2+M\|x-y\|^2=M(\|x\|^2+\|y\|^2)$$
My question boils down to the second estimate where it should hold that $|<Tz,z>|\le \sup\limits_{\|u\|\le1 }|<Tu,u>|\|z\|^2 = M\|z\|^2$ for $z\in H$, $\|z\|\le 1$ ( or even for any $z\in H$?).
I tought about using the estimate $|<Tz,z>|\le \|Tz\|\|z\|\le \|T\|\|z\|^2$ but since $\|T\|\le M$ is what should be proved I can't use it.
So I'm stuck here and don't see how to prove the estimate in question?
Comment: Note that, for every $x\ne0$, $\langle Tx,x\rangle=\|x\|^2\cdot\langle Tu,u\rangle$ for some $\|u\|=1$, namely, $u=\frac{x}{\|x\|}$.
Comment: Can you provide more details/references on $=\|x\|^2$ cause i have never seen this statement before.
Comment: Did I state that $\langle Tx,x\rangle=\|x\|^2$? Where?
Comment: seems its not my day today :P - My fault. - I've got it now. Many thanks for your time!
Answer: We need to show:
$$
|\langle Tz,z\rangle| \le \|z\|^2\sup\limits_{\|u\|\le 1} |\langle Tu,u\rangle|
$$
Clearly, it holds for $z=0$. If $z\ne 0$, and $w=\dfrac{z}{\|z\|}$, then $\|w\|=1$, and
$$
|\langle Tz,z\rangle|=|\langle T(\|z\|w),\|z\|w\rangle|=\|z\|^2|\langle Tw,w\rangle|
\le \|z\|^2\sup_{\|w\|=1}|\langle Tw,w\rangle|.
$$


How to restore postgres 12 dump file into postgres 11.5

Question: dump file generated by pg_dump command in postgres 12
command i use in postgres 11.5:
pg_restore -h [host] -p 5432 -U postgres -d [db] -1 backup.dump

error message:
pg_restore: error: could not set default_table_access_method: ERROR:  unrecognized configuration parameter "default_table_access_method"

Is it possible to restore it? please advice.
Comment: Are there datatypes that prevent doing a copy from the old and to the new?
Comment: Were you able to restore using the workaround that i suggested to you Metalik?
Comment: @AmjadShahzad No your solution did not work. We had to install 11.5 version.
Answer: default_table_access_method is not available in pg11. So the workaround is to take a plain dump of source database and remove the following given below entry from your backup.dump file
SET default_table_access_method = heap;

After that use the below command for restoring the dump on target pg1
Comment: Do you know if this could be integrated into pg_restore as a default?
Answer: It seems that only pg_restore throws an error, psql just issues a warning. So you could convert your binary dump to text:
pg_restore dumpfile.Fc -f- | psql -U user -d database

This worked for me on linux.
Comment: It should be `pg_restore -f- dumpfile.Fc  | psql -U user -d database`


Trigger para otimização de tabela

Question: Estou tentando criar uma trigger que roda o comando OPTIMIZE do MySQL.
Em suma, tenho uma tabela A e criei uma tabela B que gravará um datetime onde esse valor será atualizado toda vez que que a tabela A receber o comando OPTIMIZE.
A Trigger iria
Tentei o seguinte código, mas, o MySQL acusa erro e não acho a solução.
DELIMITER $$
CREATE TRIGGER optimizechattable AFTER UPDATE ON lz_chat_archive FOR EACH ROW
BEGIN
    DECLARE @qtd int = 0;
    DECLARE @MAXDATE DATE;
    DECLARE @DATAATUAL DATE;

    SET @qtd = ( SELECT count(c.time) as qtd FROM lz_chats_table_optimize as c );

    SET @MAXDATE = ( SELECT MAX(c.time) FROM lz_chats_table_optimize as c );
    SET @DATAATUAL = (SELECT GETDATE());

    IF(@quantidade <= 0)
        optimize table lz_chat_archive;
        /* break execution */
    END IF;

     /* adaptar para  , se diferença for maior que x horas, executar if*/
    IF (@MAXDATE > @DATAATUAL)

        UPDATE lz_chats_table_optimize as c SET c.time = @DATAATUAL where id = 1
        optimize table lz_chat_archive;

    END IF;
END $$
DELIMITER;

Comment: Qual erro o Mysql acusa?
Comment: Boa tarde  , MySql retorna esse erro: #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'qtd int = 0;
    DECLARE MAXDATE DATE;
    DECLARE DATAATUAL DATE;

   ' at line 3
Answer: Seu problema está na linha  DECLARE @qtd int = 0; não pode setar o valor dessa forma, precisa usar o SET
Desse modo:
DECLARE @qtd int;
DECLARE @MAXDATE DATE;
DECLARE @DATAATUAL DATE;

SET @qtd = 0;

Alem disso, está utilizando @quantidade no lugar de @qtd no seu if
Comment: Boa noite, quanto a variavel foi o corretor que auto completou para quantidade hehe. Retirei os DECLARE  , o erro mudou. 1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'optimize table lz_chat_archive;
        /* break execution */
    END IF;

' at line 10
Comment: [Veja esse link](https://www.techonthenet.com/mysql/loops/if_then.php) de como declarar uma condição
Answer: Bom dia  ,
Resolvi o problema criando uma procedure (o MySql não aceita os comandos optimize  na trigger).E criei uma trigger para chamar a procedure toda vez que atualizar a tabela.
Segue codigo completo:
BEGIN

    DECLARE qtd int(11);
    DECLARE lastatt DATETIME;
    DECLARE diffhour INT(11);

    SET qtd = (SELECT count(id) FROM lz_chats_table_optimize where id = 1 );

    if qtd <= 0 THEN

        SELECT "Inserindo dados na tabela de controle de atualização";
        INSERT INTO lz_chats_table_optimize (`time` ) VALUES ( NOW() );
        SELECT "Otimizando tabela alvo";
        OPTIMIZE TABLE lz_chat_archive;
        OPTIMIZE TABLE lz_chat_posts;
        OPTIMIZE TABLE lz_chat_requests;
        OPTIMIZE TABLE lz_chat_forwards;
        OPTIMIZE TABLE lz_visitor_browser_urls;
        OPTIMIZE TABLE lz_visitor_browsers;
        OPTIMIZE TABLE lz_event_triggers;
        OPTIMIZE TABLE lz_visitors;
        OPTIMIZE TABLE lz_visitor_chats;
    ELSE

        SET lastatt = (SELECT c.time FROM lz_chats_table_optimize as c where id = 1 );
        SET diffhour = ( SELECT HOUR(TIMEDIFF(NOW() , lastatt)) );

        IF diffhour >= 1 THEN

            SELECT "Atualizando dados na tabela de controle de atualização";
            UPDATE lz_chats_table_optimize set `time` = now() where id = 1;
            SELECT "Otimizando tabela alvo";
            OPTIMIZE TABLE lz_chat_archive;
            OPTIMIZE TABLE lz_chat_posts;
            OPTIMIZE TABLE lz_chat_requests;
            OPTIMIZE TABLE lz_chat_forwards;
            OPTIMIZE TABLE lz_visitor_browser_urls;
            OPTIMIZE TABLE lz_visitor_browsers;
            OPTIMIZE TABLE lz_event_triggers;
            OPTIMIZE TABLE lz_visitors;
            OPTIMIZE TABLE lz_visitor_chats;

        ELSE

            SELECT "Não há necessidade de otimizar a tabela";

        END IF;

    END IF;

END

Trigger para otimização de tabela


Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?

Question: Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be seen in practice (even if the variable is unbounded I can take the exponent of the variable and make the unknown values negative).
The main advantage is that the model knows that the variable is missing, which is not the case for say mean imputation. I can see that this could be disastrous in linear models or neural networks but in tree-based models this is handled really smoothly.
I know that there is a great deal of literature on missing data imputation, but when and why would I ever use these methods when missing data for predictive (tree-based) models?
Comment: Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
Comment: The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
Answer: One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a non-missing value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
Comment: That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
Comment: @gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
Comment: To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with *or* you're fairly certain that the missingness distribution is fairly stable.
Answer: I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
Comment: (-1) "It does not really matter whether the frequency of the missing values increase or decrease." It sure does, especially when frequency of missingness may be related to the outcome! See my answer for a clear example.
Comment: Well if missing values are correlated with the outcome (which you can not observe), then supervised learning is not the right approach in the first place. My comment assumed that we are assuming a proper supervised learning environment. Think about it. Trees will be biased in such scenarios. To address "latent" or "hidden" variables you have to use slightly more structural statistical models instead of pure non-parametric models such as trees.
Comment: "To address "latent" or "hidden" variables you have to use slightly more structural statistical models instead of pure non-parametric models such as trees". That is essentially what my answer is saying.
Comment: In addition, having a *feature* related to the outcome and having the *missingness of that feature* related to the outcome are two different things.
Comment: Ok, please be more concise in your answer. I did not read that out of your answer. Please rework it. Thanks
Comment: Personally, I think my answer is fine and fairly straightforward. If you can articulate what you dislike about it, feel free to comment on it.


Como capturar UTC com uma aplicação Android

Question: Boa tarde a todos!
Preciso capturar, através de um aplicativo android, qual o UTC usado no aparelho do cliente para fazer umas validações internas.
Agradeço desde já!
Comment: O que quer dizer com "qual o UTC"?
Comment: É com relação ao fuso horário
Comment: Ve se o http://www.joda.org/joda-time/ te ajuda
Comment: Obrigado +Igor pelo link, vou dar uma olhada aqui! Vlw
Answer: Use a classe TimeZone.
TimeZone timeZone = TimeZone.getDefault();
String name = timeZone.getID();

O método getDefault() retorna o TimeZone que o dispositivo está a usar.
O método getID() retorna o seu ID, por exemplo Portugal ou Brazil/East.
Comment: Obrigado +ramaral vou implementar e realizar alguns testes!
Comment: Você aceitou a resposta do Edson, no entanto fiquei sem saber se o que procurava era o que eu respondi ou o respondido por ele.
Comment: Vou realizar os testes ainda, aceitei as duas por questão de "pendência", pois ainda não sei qual vai dar certo pra mim!
Comment: Você não consegue aceitar as duas ao mesmo tempo. Faça os seus testes e depois volte para escolher. Lembro que além de aceitar pode votar nas respostas.
Comment: Ahh ok então, obrigado pela dica! Não tinha esse conhecimento.
Answer: final Date currentTime = new Date();
final SimpleDateFormat sdf = new SimpleDateFormat("EEE, MMM d, yyyy hh:mm:ss a z");
sdf.setTimeZone(TimeZone.getTimeZone("UTC"));
System.out.println("UTC time: " + sdf.format(currentTime));

Comment: Edson, poderia fazer uma explicação sobre sua resposta ? Ela está entrando para a lista de **Publicações de baixa qualidade**.
Comment: Obrigado +Edson vou implementar e realizar uns testes!!


iid random variables random walk

Question:
In preparation for a course I am doing later in the semester I have been trying to brush up on some probability theory. But I am struggling with the following problem:
Let $\{Y_n,n\geq1\}$ be independent identically distributed random variables with $\mathbb{P}(Y_1=1/2)=\mathbb{P}(Y_1=3/2)=1/2$. Define $X_0=1$ and $X_n=\Pi_{i=1}^n Y_i$ for $n\geq 1$. Show that $\mathbb{P}(\lim_{n\rightarrow \infty} X_n=0)=1$.

What I have done so far:
I believe I recognize a random walk type of problem and I believe $\{X_n,n\geq 1\}$ is a decreasing sequence of events, so I can use the continuity of probability and get:
$$\mathbb{P}(\lim_{n\rightarrow \infty} X_n=0)=\lim_{n\rightarrow \infty} \mathbb{P}(X_n=0).$$
But I am not entirely sure how to proceed. Or am I on the wrong path?
So my main problem is how do I formally show this? Any help would be much appreciated.
Answer: Take the logarithm of $X_n$ to get a sum of iid variables. This represents a 1-dimensional random walk biased toward $-\infty$. You can show that it converges a.e. to $-\infty$, so the exponential of it will converge to zero.
Answer: The sequence $(X_n)$ is a non-negative martingale and so converges almost surely to a random variable $X_\infty$. An infinite product can only converge to a positive limit if the terms converge to 1, and since $Y_n$ can't possibly converge to 1, we conclude that $X_\infty=0$ almost surely.

iid random variables random walk


Which type of statistical test should I use for categorical and continuous IVs and continuous DV?

Question: I am conducting an experimental research. Research model shown as figure.

I hypothesized that X positively predicts Y. All participants were required to answer measurements of X and Y.
Meanwhile, three additional independent variables (conditions / wording of appeals: emotion, difficulty, effort) were manipulated and served as moderators of the X-Y direct relationship. There are eight conditions in total, and participants were randomly assigned to one of the eight conditions:

guilt / easy / low effort
guilt / easy / high effort
guilt / difficult / low effort
guilt / difficult / high effort
shame / easy / low effort
shame / easy / high effort
shame / difficult / low effort
shame / difficult / high effort

Put alternatively, my independent variable is a continuous variable and moderators are categorical variables whereas dependent variable is a continuous variable.
My question is: Which type of statistical test should I use?
Comment: Are you interested in Anova test or regression ?
Comment: I am interested in both ANOVA and regression. By the way, I use SPSS for data analysis.
Comment: scaling your categorical variable like guilt and shame as 0, 1 is obscure. you may have say, Likert scale etc. for one of two terms. I am not aware of  sample data. binary scale should be avoided.
Comment: @SubhashC.Davar Participants are randomly assigned to one of the conditions (i.e., guilt or shame + easy or difficult + low or high). So there are 8 manipulated conditions in total. Binary labelling is just for my easy reference.
Comment: You indicate 6 sets in your Chart. How is 8 number reached ?
Comment: And do these classifications reflect moderator variables ? What is the purpose of randomly assigned ?
Comment: @SubhashC.Davar I have further elaborated the research content which should be clearer now. Yes, those categorical variables (or conditions) serve as moderators of X-Y direct relationship.
Answer: I think using a regression model would be good and then to test if the coefficient estimate of $X$ is 0 or not. The tests can be wald test, likelihood ratio test, score test, etc. You can choose one suitable and easy to use for your model.
Comment: Should I use a regression model even that the moderators (i.e., emotion, difficult, effort in this case) are categorical variables?
Comment: @James you can use a regression model provided you convert the categorical variable to its corresponding dummy variables. If your categorical variable has 3 levels you'll have n-1=2 dummy variables. In your data matrix you'll replace the original categorical column with the dummy variables.
Comment: What's the differences between using a regression model and ANCOVA?
Comment: Regression and ANOVA are two ways if you just want to do some test. Sometimes you need a model to do ANOVA, such as the null model and the specified model. Regression can give you more freedom to describe the relationship between the response variable and the independent variables. Perhaps you can choose one of them based on what results you want.
Comment: Another way to include categorical variables in the model is to convert them into factors, which will be automatically converted into dummy variables by the model function.


Yolo v3 loss function

Question: The original loss function can be seen here and is more or less explained in Yolo Loss function explanation:
\begin{align}
&\lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(x_i-\hat{x}_i)^2 + (y_i-\hat{y}_i)^2 ] \\&+ \lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(\sqrt{w_i}-\sqrt{\hat{w}_i})^2 +(\sqrt{h_i}-\sqrt{\hat{h}_i})^2 ]\\
&+ \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}(C_i - \hat{C}_i)^2 + \lambda_{noobj}\sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{noobj}(C_i - \hat{C}_i)^2 \\
&+ \sum_{i=0}^{S^2} \mathbb{1}_{i}^{obj}\sum_{c \in classes}(p_i(c) - \hat{p}_i(c))^2 \\
\end{align}
However, I still have a few questions relating to the above equation, and how (or if) the loss changed in YOLOv3. For starters, in YOLOv3 the output is $S\times S\times B\times (4+1+C)$ as opposed to $S\times S\times (B\times (4+1)+C)$, meaning that the last term would be $\mathbb{1}^{obj}_{ij}$.
For the rest of the question consider the first 5 terms of the last dimension of YOLOv3 output to be:
\begin{align}
b_x &= \sigma(t_x) + c_x \\
b_y &= \sigma(t_y) + c_y \\
b_w &= p_w\exp(t_w) \\
b_h &= p_h\exp(t_h) \\
Pr(object) &= \sigma(t_o)
\end{align}
See here for a full explanation of these equations, but they simply describe the coordinates of x and y, width and height, and finally the probability of an object being present in the cell.

My biggest question is with regards to what $C_i$ is in the 3rd and 4th term. Suppose that $\hat{C_i}$ is the estimated and $C_i$ is the true value. According to the original paper (v1) "we define confidence as $Pr(object)\times IOU^{truth}_{pred}$". So there's 3 ways that the 3rd loss term can be interpreted:

$C_i=1$ and $\hat{C_i} = \sigma(t_o) \times IOU$, where the IOU depends on the bounding box described by the first 4 equations.
According to this code and other implementations I've seen, it seems that $C_i=IOU$ and $\hat{C_i} = \sigma(t_o)$.
Finally, the most intuitive to me is that  $C_i=1$ and $\hat{C_i} = \sigma(t_o)$. Why bother introducing IOU in here? The first two terms effectively try to maximise IOU anyway?

Finally for YOLOv3 the paper states that:

"During training we use sum of squared error loss. If the ground truth for some coordinate prediction is $\hat{t}_*$ our gradient is the ground truth value (computed from the ground truth box) minus our prediction: $\hat{t}_* − t_*$. This ground truth value can be easily computed by inverting the equations above".

So does this mean I can ignore the first two terms of the loss function above and replace it with squared error?

Would appreciate any input to this including answering just one of the two questions.
Answer: Good questions. For the first question, the score definitions are different between YOLOv1 and YOLOv3. According to the yolov1 paper

These confidence scores reflect how confident the model is that the box contains an object and also how accurate it thinks the box is that it predicts.
If no object exists in that cell, the confidence scores should be zero. Otherwise we want the confidence score to equal the intersection over union (IOU) between the predicted box and the ground truth.

This is the same as your second interpenetration. However in the yolov3 paper

YOLOv3 predicts an objectness score for each bounding box using
logistic regression. This should be 1 if the bounding box prior
overlaps a ground truth object by more than any other bounding box
prior.

This is the same as your third interpenetration. Actually both options are implemented in the code you referred to, I guess they both work in practice.
Second question, yes, just don't forget to apply the inverse functions to the ground truth coordinates first. Source code for reference.
Comment: Based on your answer it looks like we define the objectness for each bounding box, and it requires a j index for the bounding box like, $C_{ij}$, am I correct?
Comment: Could have been explained better: not in terms of first question, second question, etc, but exactly what the loss is. It is daunting for someone who is a beginner and doesn't have much context with respect to Yolo 1 and 2 (deep-diving into which is now irrelevant IMO).

Yolo v3 loss function


C# - Closing form while a thread that creates new controls is still running

Question: I'm trying to add custom controls on a secondary thread, but when I close the window while the thread is still running i get this exception:

Invoke or BeginInvoke cannot be called on a control until the window
  handle has been created.

I don't know if the reason for getting this exception is because of a missused thread or becuse I'm closing the window while the thread is still running.
this is the code that i get the exception for:
panelWall.Invoke(new Action(() =>
            {
                postControl = new FBPostUserControl(m_LoggedInUser.Name, m_LoggedInUser.ImageNormal, post.CreatedTime);
                postControl.PostBody = post.Message;
                postControl.Image = postImage;
                postControl.Dock = DockStyle.Top;
                postControl.BringToFront();
            }));

this is the code of my custom control:
public partial class FBPostUserControl : UserControl
{
    private readonly string m_UserName = string.Empty;
    private readonly Image m_UserProfileImage = null;
    private readonly DateTime? m_DatePosted = null;
    private Image m_Image = null;
    private string m_PostBody = string.Empty;

    public string UserName
    {
        get { return m_UserName; }
    }

    public DateTime? DatePosted
    {
        get { return m_DatePosted; }
    }

    public Image Image
    {
        get { return m_Image; }
        set
        {
            if (value == null)
            {
                pictureBoxImage.Visible = false;
            }
            else
            {
                pictureBoxImage.Visible = true;
                pictureBoxImage.Image = value;
                updateImageSize();
            }
        }
    }

    private void updateImageSize()
    {
        if (pictureBoxImage.Image != null)
        {
            double ratio = pictureBoxImage.Image.Width / pictureBoxImage.Image.Height;
            pictureBoxImage.Height = (int)(pictureBoxImage.Width / ratio);
            pictureBoxImage.SizeMode = PictureBoxSizeMode.Zoom;
        }
    }

    public string PostBody
    {
        get { return m_PostBody; }
        set
        {
            if (string.IsNullOrWhiteSpace(value) == false)
            {
                labelPostBody.Visible = true;
                labelPostBody.Text = value;
            }
            else
            {
                labelPostBody.Visible = false;
            }
        }
    }

    public Image UserProfileImage
    {
        get { return m_UserProfileImage; }
    }

    public FBPostUserControl(string i_Name, Image i_ProfileImage, DateTime? i_PostDate)
    {
        InitializeComponent();
        m_UserName = i_Name;
        m_UserProfileImage = i_ProfileImage;
        m_DatePosted = i_PostDate;

        refreshHeader();
    }

    private void refreshHeader()
    {
        pictureBoxUserImage.Image = m_UserProfileImage;
        labelName.Text = m_UserName;

        if (labelDate != null)
        {
            labelDate.Text = m_DatePosted.ToString();
        }
        else
        {
            labelDate.Visible = false;
        }
    }
}

Comment: Your thread keeps using the control even though it doesn't exist anymore.  Kaboom of course.  It is up to you to ensure that the thread stops before you allow the window to close.  http://stackoverflow.com/a/1732361/17034
Comment: 90% sure it is because of "I'm closing the window while the thread is still running."
Comment: I have gotten that error when attempting to close a window with an active thread. I ended up using a layman's test (if(this.FormClosing)return; but I'm sure there are much more elegant solutions out there.
Answer: 12/1/2020 EDIT START
There are problems awaiting Task.Yield because of Dispatcher Priority as mentioned in https://getandplay.github.io/2019/05/15/transfer-of-execution-rights-Task-Yield-Dispatcher-Yield/
It is safer to await System.Windows.Threading.Dispatcher.Yield()
12/1/2020 EDIT START
First, I don't see that you're launching the operation in a new thread, because the Invoke method just post the action to the dispatcher queue in the UI thread.
So there's no real multithreading in your code, but while the action is performed, the user has had the opportunity to post a CLOSE FORM windows message, and it can be processed before your next Invoke. So to avoid the exceptión, check if the form is closed before your next Invoke.
By the way, I believe there's no real advantage in launching a new thread just to update graphic elements, because, at last, they have to be updated in the UI thread, and you're just spending time and resources in the round trip.
If you have a long graphic operation and you are targeting NET Framework 4.5 or higher, the standard way to do it is await an async method for the long graphics operation, and internally await Task.Yield() at intervals to give the user the opportunity to cancel, close the window, etc.
Basically Task.Yield() posts the method continuation to the UI dispatcher, and when it returns, you can check the form and cancel the long operation if the form is closed:
    async Task LongJustGraphicsOperation()
    {
        while (true)
        {
            //do some work and give a pause
            await Task.Yield();
            if (formIsClosed) break;
        }
    }

Task.Yield() is the Task version of the old VB doevents.
Note. Is somewhat tricky to check if a winform is closed Detect when a form has been closed


Maximize gmap in dialog primefaces

Question: I want to display a dialog which has a gmap as element, it works fine when I set height and width, but what I need is show it as maximized full screen map. The Dialog is shown in fullscreen, but map is empty, only when I back to default size it is shown.
My code is

How can I solve my problem?
Comment: is your map is loaded before the dialog is visible? I had the similar issue with google-map. try loading the map when the dialog opens(become visible).
Comment: I guess it is, but gmap in jsf hasn't a load function until I know.
Answer: You can add following JavaScript on your page

and then add resizePfGmapInsideDialog() into p:commandLink onclick

Actually whenever you call PF('dlg').toggleMaximize(); also call resizePfGmapInsideDialog() right after and p:gmap will automatically resize inside dialog's content area.

Maximize gmap in dialog primefaces


Setting rails environment with Foreman

Question: How do you set the rails environment with Foreman?
For example, if I wanted to run bundle exec rake assets:precompile on my development machine, but I wanted to set the rails environment to production so I could see exactly how this process would run on the production server - I could do this using vanilla rails by doing RAILS_ENV=production bundle exec rake assets:precompile.
But how do I do this with Foreman? For example, I know I can do bundle exec foreman run rake assets:precompile, but that doesn't specify the rails environment as production.
Answer: The solution is to use a .env file in your project's root directory. Inside that file, add this line:
RAILS_ENV=production

Now, anytime you run Foreman locally, it will set the rails environment to production. So, if you run bundle exec foreman run rake assets:precompile, the environment will be production for that task. Don't forget to change it back when you're done! ;-)

Setting rails environment with Foreman