C - Writing structs to a file (.pcap)

Question: I am trying to write a .pcap file, which is something that can be used in Wireshark.
In order to do that, I have a couple of structs with various data types I need to write to a file. (see code)
So, I create the struct instances, fill in the data, use FILE* fp = fopen("test.pcap","w"), and then I'm unsure how to properly write it to the file. I believe I should use memcpy but I'm not sure of the best way to do it. I have mostly resorted to C++ libraries in the past to do this. Any suggestions?
typedef struct pcap_hdr_s {
        uint32_t magic_number;   /* magic number */
        uint16_t version_major;  /* major version number */
        uint16_t version_minor;  /* minor version number */
        int32_t  thiszone;       /* GMT to local correction */
        uint32_t sigfigs;        /* accuracy of timestamps */
        uint32_t snaplen;        /* max length of captured packets, in octets */
        uint32_t network;        /* data link type */
} pcap_hdr_t;

typedef struct pcaprec_hdr_s {
   uint32_t ts_sec;         /* timestamp seconds */
   uint32_t ts_usec;        /* timestamp microseconds */
   uint32_t incl_len;       /* number of octets of packet saved in file */
   uint32_t orig_len;       /* actual length of packet */
} pcaprec_hdr_t;

typedef struct ethernet_hdr_s {
   uint8_t dst[6];    /* destination host address */
   uint8_t src[6];    /* source host address */
   uint16_t type;     /* IP? ARP? RARP? etc */
} ethernet_hdr_t;

typedef struct ip_hdr_s {
   uint8_t  ip_hl:4, /* both fields are 4 bits */
            ip_v:4;
   uint8_t        ip_tos;
   uint16_t       ip_len;
   uint16_t       ip_id;
   uint16_t       ip_off;
   uint8_t        ip_ttl;
   uint8_t        ip_p;
   uint16_t       ip_sum;
   uint32_t ip_src;
   uint32_t ip_dst;
}ip_hdr_t;

typedef struct udp_header
{
  uint16_t src;
  uint16_t dst;
  uint16_t length;
  uint16_t checksum;
} udp_header_t;

Comment: The magic number does *not* let you write out the packet *data* in big-endian or little-endian order; you have to write that out in the order in which it would appear on the wire, which, for Ethernet, IP, TCP, and UDP multi-byte integral fields, is big-endian.  It *does* let you write out the fields in the file and record header in your native byte order, but you might as well just let libpcap/WinPcap do that work.  And, sadly, the magic number is not affected by, for example, FTPing the file in ASCII mode between Windows and UN\*X.
Comment: Be careful with byte order -- I think the captured packets always use network byte order, but you might need to check the byte order of the header.  Maybe not a concern, if the file format uses the `magic_number` field to determine the byte order of the header.
Comment: The magic number lets you write in either big- or little-endian, you can even switch back and forth after every packet if you want. The magic number also alerts you to damage caused by improperly transferring the file.
Answer: Use libpcap or WinPcap - pcap_open_dead() to get a "fake" pcap_t to use with pcap_dump_open() to specify the link-layer header type (for Ethernet, use DLT_EN10MB) and snapshot length (use 65535), pcap_dump_open() to open the file for writing, pcap_dump() to write out a packet, and pcap_dump_close() to close the file.  MUCH easier than directly using fopen(), fwrite(), and fclose() (which are what libpcap/WinPcap use "under the hood").
And, yes, you have to get the byte order in the packets correct.  The byte order depends on the protocol; for the type field in the Ethernet header, and for all multi-byte fields in IP, TCP, and UDP headers, they have to be in big-endian order.  (The magic number in the pcap file is irrelevant to this - it only indicates the byte order of the fields in the file header and the per-packet record header, NOT the byte order of the fields in the packet, as well as, due to the way it's implemented in Linux, the meta-data at the beginning of packets in Linux USB captures.  The packet data is supposed to look exactly as it would "on the wire".)
Comment: can you please share "more code"? I mean write an example?
Answer: Use fwrite(). You need to check this info but I think .pcap files are written in binary mode.
Example:
pcaprec_hdr_t pcaprec_hdr;
// fill pcaprec_hdr with valid info

FILE* pFile = NULL;
pFile = fopen ("myfile.pcap" , "wb"); // open for writing in binary mode

fwrite (&pcaprec_hdr, 1, sizeof(pcaprec_hdr_t) , pFile);

fclose(pFile);

Comment: "You assume they already know. But do they?"  Yes, they do.  They were the first implementation of the file format ever.
Comment: -1 for forgetting to write out the file header and for not just saying "use libpcap/WinPcap, they already know how to write out the file header and the record header".
Comment: *You assume* they already know. But do they? Getting this answer selected as the official demonstrates something else.
Comment: I am having troubles with this, can you edit your answer with "full" example? When I open my pcap file in wireshark I get error saying `The capture file appears to have been cut short in the middle of a packet.` And I can see only "frame" row. Thanks.
Answer: Here's my understanding of what Guy Harris is suggesting. So, as per Kyslik's request, we have:
#include <libpcap/pcap.h>

/* Ethernet/IP/SCTP INIT chunk */
static const unsigned char pkt1[82] = {
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* ........ */
  0x00, 0x00, 0x00, 0x00, 0x08, 0x00, 0x45, 0x00, /* ......E. */
  0x00, 0x44, 0x55, 0xb1, 0x00, 0x00, 0x40, 0x84, /* .DU...@. */
  0x26, 0x83, 0x7f, 0x00, 0x00, 0x01, 0x7f, 0x00, /* &....... */
  0x00, 0x01, 0x00, 0x01, 0x1f, 0x90, 0x00, 0x00, /* ........ */
  0x00, 0x00, 0x68, 0xe5, 0x88, 0x1b, 0x01, 0x00, /* ..h..... */
  0x00, 0x24, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0f, /* .$...... */
  0xa0, 0x00, 0x00, 0x04, 0xff, 0xff, 0x00, 0x00, /* ........ */
  0x16, 0x2e, 0x80, 0x00, 0x00, 0x04, 0xc0, 0x00, /* ........ */
  0x00, 0x04, 0x00, 0x0c, 0x00, 0x06, 0x00, 0x05, /* ........ */
  0x00, 0x00                                      /* .. */
};

int main(int argc, char *argv[]) {

  pcap_t *handle = pcap_open_dead(DLT_EN10MB, 1 << 16);
  pcap_dumper_t *dumper = pcap_dump_open(handle, "/tmp/pktcap/cap.pcap");

  struct pcap_pkthdr pcap_hdr;
  pcap_hdr.caplen = sizeof(pkt1);
  pcap_hdr.len = pcap_hdr.caplen;

  pcap_dump((u_char *)dumper, &pcap_hdr, pkt1);
  pcap_dump_close(dumper);

  return 0;
}

Comment: I had to modify `pcap_pkthdr pcap_hdr;` to `struct pcap_pkthdr pcap_hdr;` to make this compile with `gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609`

C - Writing structs to a file (.pcap)


Quarkus Panache MongoDB: Delete only matching data from Array

Question: Is it possible to delete only filtered elements from an array, using panache with Quarkus?
{
    "id": "1234567",
    "data": [
        {
            "type": "Type1",
            "value": "Value1"
        },
        {
            "type": "Type2",
            "value": "Value2"
        },
    ]
}

I want delete all elements with type equal to Type2. I have tried with: delete("data.type","Type2"), but it delete the entire document from the db
Answer: Hmmm, not sure if it is possible.
Supposing data is a List you can iterate and remove the item Type2 and then update the entire document.


How can I sample from a correlated multivariate Bernoulli distribution with known covariances?

Question: I have $N$ Bernoulli random variables $X_1, ..., X_{N}$ with known parameters $p_1, ..., p_{N}$. They are dependent with known covariances.
How can I sample from the joint distribution of the $X_i$?
I would ideally like a more general approach than this.
Answer: This won't always work, but it may be worth a try. First compute thresholds $t_1,…,t_N$, where $\newcommand{\erfcinv}[1]{\operatorname{erfcinv}\left(#1\right)}
t_i = \sqrt{2} \erfcinv{2p_i}$, so that $\Pr(Z > t_i) = p_i$, where $Z$ is a standard normal random variable.
Then for each pair of variables $(i,j), 1 \leq i < j \leq N$, solve for the tetrachoric correlation $r_{ij}$, where $\int_0^{r_{ij}} f(t_i,t_j,\rho) \mathrm{d}\rho =$ the known covariance of variables $i$ and $j$, and $f(x,y,\rho)$ is the bivariate standard normal pdf with correlation $\rho$.
(Note: the integral, which must be evaluated numerically, is more tractable with the change of variable $\rho = \sin \theta$.)
If the matrix $R$ of tetrachoric correlations is positive definite then you're in business. Let $F$ be any matrix that gives $FF' = R$. Then for each desired binary vector $x$, generate a vector $z$ of $N$ independent standard normals, let $y = Fz$, and for $i = 1,…,N$ let $x_i = 1 if y_i > t_i$, and $x_i = 0$ otherwise.
Comment: Thanks very much @RayKoopman. [This](http://www.kyb.mpg.de/fileadmin/user_upload/files/publications/attachments/macke2009_5157%5b0%5d.pdf)
 paper details a similar approach that I also found useful. It also has [Matlab code](http://www.mathworks.com/matlabcentral/fileexchange/20591-sampling-from-multivariate-correlated-binary-and-poisson-random-variables) that I haven't tried but appears useful. Appreciate your help.
Comment: Please check that I didn't introduce any error in your math notation (especially regarding the `\erfci` operator).
Comment: Thanks for the fixup. `\erfci(2p_i)` is the inverse complementary error function, with argument `2p_i`; the first `i` is part of the function name, signaling "inverse".
Comment: Better to leave it as is then.
Comment: `erfcinv` seems to be the common name for what I originally had as `erfci`, so I added the `nv`.
Comment: Thanks for your answer @RayKoopman! Is there anything that can be done when $R$ is not positive definite? Maybe some smart regularisation or something?


Hiding directory above from being shown in index

Question: I've added an alias to my server. When I go to it, it still shows the "../" that can redirect to the root folder. Can I hide this somehow, specifically in the alias?
<code>server {
            autoindex on;
            root /home/myrootdirectory/;

       location / {

       }
       location /specific {
           alias /home/myrootdirectory/specific/;
       }
}
</code>
Thankyou.
Answer: No, and there's no point to hiding it. One could simply edit the URL themselves. If you don't want files published on your web server, simply do not publish them.


Haskell list comprehension error

Question: I just started to learn Haskell today and is completely overwhelmed by its syntax.
I am trying to apply math calculation to a list of items.
For example, lets say I want to square every item in the list using list comprehension.
My attempt
myfunc (n:lis) = [ k | k <-lis, k == k^k]

result_list = myfunc[1..]

take 10 result_list

My understand of my myfunc code: take a list and loop through elements that is stored in variable k and set k equals to its square.
after i execute the take command, and hit enter, apparently the process is running but does not do anything.
Note that i want to use list comprehension as a way to do it. I can use map do achieve my goal already.
Comment: "is (n:lis) equivalent of Java's " no, they even weren't in the same town.
Comment: No, this is totally wrong. You cannot "set" a variable in Haskell. `k == k^k` means "k is equal to its own kth power" (it's a boolean condition). There aren't many such numbers out there, and your function just happens to skip the first and only one, so there's no output.
Comment: @n.m. is `(n:lis)` equivalent of Java's `(for var in array)`?
Comment: If you want to generate a list of squares, you need something like `[k^2|k<-lis]`.
Comment: @n.m. thank you for the explination
Answer: You misunderstand the list comprehension.
[ k | k <- lis, k == k^k ]

The k == k^k clause is a filter –– it only keeps elements of the list that satisfy this equation.  (== is a comparison operator that returns a bool, which is one hint).  The reason you see no output is that there are no numbers in [1..] that satisfy this equation.  But we get an infinite loop because we keep checking ever higher numbers to see if they satisfy it.
Something to experiment with
[ k | k <- lis, k < 100 ]

As for how to get a list of squares, use a comprehension like this
[ k^2 | k <- lis ]

If you want something more like your original phrasing, you can make let bindings within  a list comprehension:
[ r | k <- lis, let r = k^2 ]

There are other issues with your code, but one baby step at a time!  Good luck!
Comment: @aDev `n` is the first element of the list (which your function throws away). `lis` is the tail of the list (everything but the first element). If you want to use the input list as is, use `myFunc lis = [ ... ]`.
Comment: I dont quite understand the point of (n:lis) in this case.  In the working code here: `myFunc (n:lis) = [ k^k | k <-lis]`, N does not seem to function as anything.
Comment: `[ k | k <- lis, k < 100 ]`, which can be written `filter (<100) lis` has a potential problem; it will never stop checking values from `lis` to see if they happen to satisfy `<100`. If we know they grow monotonically, we can get a terminating list instead using `takeWhile (<100) lis`. In this example, we expect the `take 10` in the original code to make sure we don't see the part where the filter runs indefinitely without completing (which happened immediately for the original comprehension).

Haskell list comprehension error


Error while using ServiceBusProcessor - Azure.Messaging.ServiceBus SDK

Question: I am getting an exception after the StartProcessingAsync() method. The debug pointer goes to the "ProcessorErrorAsync" method. I followed similar steps provided in the link - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Azure.Messaging.ServiceBus/samples/Sample04_Processor.md
Am I missing some steps here?
Exception Details: error.Exception.Message:
Method not found: 'System.Threading.Tasks.Task`1<System.Collections.Generic.IEnumerable`1<Microsoft.Azure.Amqp.AmqpMessage>> Microsoft.Azure.Amqp.ReceivingAmqpLink.ReceiveMessagesAsync(Int32, System.TimeSpan, System.TimeSpan, System.Threading.CancellationToken)'.

private void ListenerBind(string key, ServiceBusProcessorOptions onMessageOptions)
        {
            ServiceBusClient tempClient = this._cacheClient.Get(key);
            ServiceBusProcessor tempProcessor = tempClient.CreateProcessor(this._topicName, this._subscriptionName, onMessageOptions);
            try
            {
                //temp.OnMessageAsync(this.MessageProcessCallBackAsync, onMessageOptions);
                tempProcessor.ProcessMessageAsync += MessageProcessCallBackAsync;
                tempProcessor.ProcessErrorAsync += ProcessErrorAsync;
                tempProcessor.StartProcessingAsync();
            }
            catch (InvalidOperationException ex)
            {
                this._logger.Log($"{ex.Message}", EventLevel.Informational);
            }
            catch (Exception ex)
            {
                this._logger.LogException(ex);
            }
        }
        private Task ProcessErrorAsync(ProcessErrorEventArgs error)
        {
            Exception ex = new Exception(
                $" , Action {error.ErrorSource}, " +
                $" , Endpoint {error.FullyQualifiedNamespace}" +
                $",EntityPath {error.EntityPath} "
                , error.Exception);
            this._logger.LogException(ex);

            this._onErrorCallback(ex, string.Empty);

            return Task.CompletedTask;
        }

Answer: I actually came across this error in my project as well. My VS solution has a Windows Service that listens to the Azure Service Bus and I have a console application which acts as a "Windows Service Runner" for local debugging. I kept getting the exception mentioned in the question while running my Service Runner and it turned out that I was missing the Azure.Messaging.ServiceBus package in my Service Runner console application.


Error Code: 1136. Column count doesn't match value count at row 1

Question: Estou tentando inserir dados em uma tabela, porem não estou conseguindo, esta dando o seguinte erro: "Error Code: 1136. Column count doesn't match value count at row 1.", alguém sabe como resolver?
Criei a seguinte tabela:
create table pessoas (
cpf int (11),
nome varchar(30) not null,
nascimento date,
endereco varchar(30) not null,
cep int(7),
bairro varchar(20),
cidade varchar(30),
uf char (2),
ultima_compra date,
primary key(cpf)
)default charset = utf8;

Ai estou tentei fazer o insert:
insert into pessoas values
('04496332780', 'João da Silva', '25-11-1969', 'Rua Antônio Numes', '88045963', 'Palmeiras','Londrina', 'PR','24-04-2018');

Pensei que pudesse ser algum erro por não ter descrito os campos ai tentei dessa forma:
insert into pessoas (cpf, nome, nascimento, endereco, cep, bairro, cidade, uf) values
('04496332780', 'João da Silva', '25-11-1969', 'Rua Antônio Numes', '88045963', 'Palmeiras','Londrina', 'PR','24-04-2018');

Mas o resultado foi o mesmo.
Answer: Na sua instrução SQL na descrição dos campos há 8 campos "cpf, nome, nascimento, endereco, cep, bairro, cidade, uf)" e no local dos valores há 9 valores "('04496332780', 'João da Silva', '25-11-1969', 'Rua Antônio Numes', '88045963', 'Palmeiras','Londrina', 'PR','24-04-2018');". Está faltando a descrição do campo ultima_compra na descrição dos campos.
A instrução SQL deve ficar da seguinte forma:
insert into pessoas (cpf, nome, nascimento, endereco, cep, bairro, cidade, uf, ultima_compra) values ('04496332780', 'João da Silva', '25-11-1969', 'Rua Antônio Numes', '88045963', 'Palmeiras','Londrina', 'PR','24-04-2018');


Múltiplas regras de Rewrite com .htaccess

Question: Tenho o seguinte .htaccess atualmente:
<IfModule mod_rewrite.c>

RewriteEngine On
RewriteBase /

## Esconder .PHP
    # Redirecionamento externo de /dir/foo.php para /dir/foo
        RewriteCond %{THE_REQUEST} ^[A-Z]{3,}\s([^.]+)\.php [NC]
        RewriteRule ^ %1 [R=302,L]

    # Redirecionamento interno de /dir/foo para /dir/foo.php
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteCond %{REQUEST_FILENAME}.php -f
        RewriteRule ^(.*?)/?$ $1.php [L]

## Redirecionamento para o parametro P do GET na index.php
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ index.php?p=$1 [NC,L,QSA]

# Esconder a palavra "index"
    RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(.*)index($|\ |\?)
    RewriteRule ^ /%1 [R=301,L]
</IfModule>

Agora eu tenho que passar um parametro para uma das páginas que já é acessada  pela regra do index.php?p=$1 que no caso existe e funciona no URL:
www.site.com.br/portfolio?trabalho=14

Agora preciso mesclar as regras para conseguir acessar o url acima a partir de:
www.site.com.br/portfolio/14

É possível? Podem me ajudar?
Grato!
Answer: Voce pode adicionar o código abaixo logo após o bloco # Redirecionamento
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^/]+)/([^/]+)$ index.php?p=$1&id=$2 [L]

Assim voce estará pegando o segundo parâmetro após a barra e o php vai receber na variavél get id ($_GET['id'])


Square of conditional variables

Question: I thought that the following would hold but my classmates doubt me unfortunately:
$Z = X|Y \rightarrow Z^2 = X^2|Y$
The original problem was an expectation of this and I tried to go back to the formal definition that:
$E[X|Y] = E[X|\sigma(Y)] = \sum_i E[X|A_i] 1_{\omega \in A_i}$
where $i \neq j \rightarrow A_i \cap A_j = \emptyset$ and $\cup_{i=1}^\infty A_i = \Omega$
...and that it would not make any sense to try to square the sigma algebra or something the like so that this must be right:
$E[Z^2] = E[X^2|\sigma(Y)] = \sum_i E[X^2|A_i] 1_{\omega \in A_i} = E[X^2|Y]$
Also wrote it like this:
$E[Z^2] = \int_{-\infty}^\infty x^2 f_X(x|y)dx = E[X^2|Y]$
I also though in a more intuitive fashion that the conditioning is simply something like a parameter of a function like letting:
$f(x)|y = xy$
$f(x)^2|y = (xy)^2$
$(f(x)|y)^2 = (xy)^2$
...where $y$ can be a scalar or an other function or whatever.
Am I right or wrong and how do you reason that this is correct or wrong?
Answer: The notation $z=x|y$ is ambiguous, so I am not quite sure if this will answer your question, but I hope it helps.
First, given some function $f[x]$ (e.g. $x^2$), we need to distinguish between its  expectation vs. its probability density. Consistency requires that the expectation should be the same whether we integrate over $x$ or $f[x]$, i.e.
$$\langle f[x]\rangle = \int f[x] \, p[x] \, dx = \int p[f] \, df$$
which means the density of $f[x]$ must satisfy
$$p[f]=p[x]\left|\frac{dx}{df}\right|$$
The point here is that if you use the first integral, $p[x]$ is always the same, needed, i.e. it does not care what the function $f$ is (only that it is deterministic).
Now for $p[x|y]$, you are correct that this can be thought of as a parameterized family of densities for $x$. That is, for a particular value of $y$, the conditional density $p[x|y]$ is just some fixed density function for the variable $x$.
So in the end we have
$$\langle f[x] \mid y \rangle = \int f[x] \, p[x\mid y] \, dx$$
but
$$p[f \mid y]=p[x \mid y]\left|\frac{dx}{df}\right|$$
So in terms of your original question, if you say $z=x|y$, then "$z$" is not really purely a random variable (like $x$ is), but also includes a statement about the "conditions" under which $x$ is being sampled (i.e. $y=c$ for some fixed constant $c$). Given this, the meaning of "$z^2$" seems ambiguous. The above discussion was my attempt to distinguish between the two aspects of $z=x|y$. I hope it was helpful!
Comment: Maybe it was trying to generalise the question too much, we were given that Y|X = x was Bin(n, x) and X was U(0, 1) and then asked to calculate E[Y^2]. I rewrote E[Y^2] = E[E[Y^2|X]] which i thought would be the expected value of the second moment of Bin(n, X) given x. Is this correct?
Comment: @user3685312 in general $\mathbb{E}[f[y]]=\int\mathbb{E}[\, f[y]\mid x\, ]\,p[x]\,dx$. For your particular question in the comment, I would post a new question, show what you did so far, and mark with the [self study tag](http://stats.stackexchange.com/tags/self-study/info).

Square of conditional variables


WPF RenderTargetBitmap of an Items Panel but omit certain elements

Question: I am using RenderTargetBitmap.Render(panel) to take a snap of an items panel.
However I want to omit certain elements (custom items with say OmitWhileRenderingToBitmap property set to false).
Is there some easy way of doing this short of removing the items from the items panel, taking the snap and then adding them back?
Edit: Essentially this is the key bit in my custom listbox class. Remove the filter bits and everything works as expected.
<code>    protected void UpdatePanelBitmapSource()
    {
        if (this._itemsPanel != null) //_itemsPanel here is the canvas within the ListBox. Not rendering the ListBox for a reason.
        {
            Matrix m = PresentationSource.FromVisual(this._itemsPanel).CompositionTarget.TransformToDevice;

            var existingFilter = this.Items.Filter;

            // Add filter
            this.Items.Filter = item =>
            {
                var mgvListBoxItem = this.ItemContainerGenerator.ContainerFromItem(item) as MGVListBoxItem;

                if (mgvListBoxItem != null)
                    return !mgvListBoxItem.IsEditable;
                else
                    return false;
            };

            var panelBitmap = new RenderTargetBitmap(
                (int)Math.Floor(this._itemsPanel.ActualWidth),
                (int)Math.Floor(this._itemsPanel.ActualHeight),
                m.M11 * 96.0,
                m.M22 * 96.0,
                PixelFormats.Default);
            panelBitmap.Render(this._itemsPanel);

            // Remove filter
            this.Items.Filter = existingFilter;

            this.PanelBitmapSource = panelBitmap;
        }
    }
</code>
Answer: Just apply a filter:
<code>// Add filter
yourItemsControl.Items.Filter = o =>
    {
        YourType item = (YourType)o;
        return !item.OmitWhileRenderingToBitmap;
    };

// Render to bitmap
// ...

// Remove filter
yourItemsControl.Items.Filter = null;
</code>
Comment: Am not sure why but this doesn't work. I add a couple of items with the omit flag on for one and off for another. I can see in the debugger that its all fine. But all it renders is the ItemsControl background and no items. Any thoughts?
Comment: Even stranger, the filter does apply to the items control, but the rendered image has no items in it.
Comment: @NVM, I just looked at the code, I can see anything obviously wrong... Did you try to execute step by step to see what's happening? You can put a breakpoint inside the filter lambda expression
Comment: Actually, I'm not sure it makes sense to access the ItemContainer from the filter... the containers are probably not generated yet, since the ItemsControl doesn't know yet if it must display them
Comment: Hmm. Well everything is as expected when stepping through the debugger. I can get to the itemcontainers as well. I can also see the omit flag on in some items and off in others.
Comment: I dont want to use the items directly because the list box is a custom control and I dont want references to VM code in that.
Answer: In the end I did something like this
<code>        protected void OnUpdatePanelBitmapSource()
    {
        if (this._itemsPanel != null)
        {
            Matrix m = PresentationSource.FromVisual(this._itemsPanel).CompositionTarget.TransformToDevice;

            var hiddenItems = new List<MGVListBoxItem>();

            foreach(var item in this.Items)
            {
                var mgvListBoxItem = this.ItemContainerGenerator.ContainerFromItem(item) as MGVListBoxItem;
                if (mgvListBoxItem != null && mgvListBoxItem.IsEditable && mgvListBoxItem.IsVisible)
                {
                    mgvListBoxItem.Visibility = Visibility.Hidden;
                    hiddenItems.Add(mgvListBoxItem);
                }
            }

            var panelBitmap = new RenderTargetBitmap(
                (int)Math.Floor(this._itemsPanel.ActualWidth),
                (int)Math.Floor(this._itemsPanel.ActualHeight),
                m.M11 * 96.0,
                m.M22 * 96.0,
                PixelFormats.Default);

            panelBitmap.Render(this._itemsPanel);

            foreach (var mgvListBoxItem in hiddenItems)
            {
                mgvListBoxItem.Visibility = Visibility.Visible;
            }

            this.PanelBitmapSource = panelBitmap;
        }
    }
</code>


Most efficient way to determine if an HTML element is in the viewport?

Question: I'm currently using prototype, with some code like:
<code>function in_viewport(foo) {
  var offset = foo.viewportOffset().top;
  var viewportHeight = document.viewport.getHeight();
  if (offset > (0 - foo.getHeight()) && offset < viewportHeight) {
    return true;
  } else {
    return false;
  }
}
</code>
Is there a faster/easier/more-efficient/better way of doing this?
Answer: With various JavaScript libraries like Dojo you can get pretty specific and cross browser compatible code that you can use to do this. It still wont look much prettier than what you have there. If you're doing it from scratch it gets much more complex as you have browser quirks, browser version quirks, have to deal with scrolling, etc.


I cannot pass a value on button click in oracle apex 19.2

Question: I have a wizard and i am trying to pass a value from one page to the next onclick a next button on the wizard in oracle apex but when i click next the value is not going through to the other side.
enter image description here
Image above shows the next button and the how i am trying to pass the value
thanks
Answer: What you did is this:

there's page 33
there are (at least) two items on that page

P33_ID
P33_NEW

you are passing P33_NEW item's value into P33_ID item

in other words, you're doing it on the same page, while it seems - according to what you said ("I am trying to pass a value from one page to the next") - that you're expecting something to happen on some other page

If that's so, transfer P33_NEW item value to "some other page", not page 33.
If you still want to do it on the same page, consider using a Set value dynamic action.


Calculating confidence intervals for the variance of the residuals in R

Question: I have three variables:

Number of house sales
Month (in couples)
Region of a city (N-W-E-S)

and I want to calculate confidence intervals for the residual of the errors. So, given the data:
month <- c("1", "1", "1", "1", "2", "2", "2", "2", "3", "3", "3",
     "3", "4", "4", "4", "4", "5", "5", "5", "5", "6", "6", "6",
     "6")
region <-c("1", "2", "3", "4", "1", "2", "3", "4", "1", "2", "3",
      "4", "1", "2", "3", "4", "1", "2", "3", "4", "1", "2", "3",
      "4")
sales <-c(85, 107, 61, 22, 40, 65, 58, 51, 60, 41, 45, 27, 15,
          30, 68, 63, 28, 3, 57, 12, 36, 21, 10, 16)

data <- cbind(sales, month, region)
data <- as.data.frame(data)

salesmod <- lm(sales ~ month + region, data=data)
summary(salesmod)
anova(salesmod)

We can check the degrees of freedom and the sum of the residuals from the summary(salesmod) and anova(salesmod).

Call:
lm(formula = sales ~ month + region, data = data)

Residual standard error: 22.88 on 15 degrees of freedom
Multiple R-squared:  0.4855,    Adjusted R-squared:  0.2111
F-statistic: 1.769 on 8 and 15 DF,  p-value: 0.1623

> anova(salesmod)
Analysis of Variance Table

Response: sales
          Df Sum Sq Mean Sq F value Pr(>F)
month      5 6368.7 1273.74  2.4325 0.0835 .
region     3 1042.8  347.60  0.6638 0.5871
Residuals 15 7854.5  523.63
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

So the variance of the residuals would be $523.63/15=34.90867$, but how do I compute a confidence intervals for this value (of given 95% confidence).
Answer: You seem to have made a small mistake. The variance of residuals is $7854.5/15=523.63$ (you have divided twice).
Now, what you are looking for is distribution of the estimate of the variance of true errors ($\varepsilon$) so that you can construct a confidence interval for it.
First let $\boldsymbol{\varepsilon} \sim N(\mathbf{0},\sigma^2I)$. Now the estimate for $\sigma^2$ is (where $K$ is the number of paramters in the model):
\begin{align}
s^2 \equiv \frac{\mathbf{e'e}}{n-K} = \frac{\boldsymbol{\varepsilon'}M\boldsymbol{\varepsilon}}{n-K}
\end{align}
So for this you can use the following property that (taken directly from this wiki page)

If $Y$ is a vector of $k$ i.i.d. standard normal random variables and
$A$ is a $k\times k$ symmetric,
idempotent matrix with rank $k-n$,
then the quadratic form $Y^{T}AY$
is chi-square distributed with $k-n$
degrees of freedom.

Since $M$ is idempotent and symmetric with rank $n-K$, it is clear that:
$$\Big(\frac{\boldsymbol{\varepsilon'}}{\sigma}\Big)M\Big(\frac{\boldsymbol{\varepsilon}}{\sigma}\Big) = \frac{n-K}{\sigma^2}s^2 \sim \chi^2_{(n-K)}$$
This can be used now to construct the confidence interval:
$$Pr\bigg(q_{\alpha/2}<\frac{n-K}{\sigma^2}s^2 < q_{1-\alpha/2}\bigg)=1-\alpha$$
$$Pr\bigg(\frac{n-K}{q_{1-\alpha/2}}s^2 < \sigma^2 < \frac{n-K}{q_{\alpha/2}}s^2 \bigg) = 1-\alpha$$
where $q_p$ is the $p^{th}$ quantile of $\chi^2_{(n-K)}$
Based on your data, we can use qchisq in R to get these quantilies.
# lower bound:
sum(salesmod$residuals^2)/qchisq(0.975,df=15)
[1] 285.7373

# upper_bound
sum(salesmod$residuals^2)/qchisq(0.025,df=15)
[1] 1254.277

Comment: This is a quite non-robust interval, it would be interesting to see the answer augmented with a more robust alternative!
Comment: @kjetilbhalvorsen: that would be great! I am certainly interested in a better answer as this was just an attempt from me.


Pega valor de um input executar função e retorna em outra input

Question: Estou querendo efetuar uma função bem simples, que é pegar o CPF de uma input, retirar os pontos e traços e retornar o valor em outra input sem os pontos e traços.
A primeira parte consegui fazer, porem não sei como fazer para retornar a função na outra input.

let botao = document.querySelector("#botao");
let retornaCpf = document.querySelector("#retornaCpf")

botao.onclick = function() {
  let cpf_com_ponto_e_traco = document.querySelector("#cpf-com-ponto-e-traco").value;
  let cpf_sem_ponto_e_traco = cpf_com_ponto_e_traco.replace(".", "").replace(".", "").replace("-", "");

  alert(cpf_sem_ponto_e_traco);
}
<input type="text" id="cpf-com-ponto-e-traco">
<input type="text" id="retornaCpf">
<button id="botao">Pegar CPF</button>
Answer: Simples:
retornaCpf.value = cpf_sem_ponto_e_traco;

Ou seja, para pegar o valor você lê o value do primeiro input, e para preencher o outro você define o seu value.
Comment: Muito obrigado, deu certinho.


Rendering 8va and 8vb with Abc notation?

Question: I'm writing up a sheet of music for bass guitar in Abc notation and I'm getting irritated by having to type so many commas to place the notes in the range of the bass. Is there any way to set the octave (have an octave displacement apply to every note I write automatically) so I can write
ABcd

instead of
A,,B,,C,D,

and have it render in the correct octave?
X:1
M:C
K:Cmaj bass
L:1/4
A,,B,,C,D,

Answer: Based on this abc primer, it appears you can specify which note is the middle line and then your notes are relative to that.
X:1
K:C bass middle=d
ABcd
X:1
K:C bass middle=d
ABcd

Answer: Based on the documentation for yaps which is part of abcmidi, there is an alternative way that may work for some software. You can add  octave=+2 or octave=-1 to the key/clef (K:) line or separately on an I: line.
X:1
K:C clef=bass octave=-2
ABcd

X:1
K:C clef=bass octave=-2
ABcd

Doesn't work here, apparently, but may be needed for certain software.


IE6 Won't Display a Javascript Tooltip... how do you even begin to debug this?

Question: I am using a tooltip plugin (developed in house by someone who is on vacation) that works everywhere but in IE6. Naturally, it is being used on a site by people that use only IE6 so I need to find a way to get it displaying.
How would you even start to debug such a thing? I've Googled for the problem to see what other javascript tooltip plugins have done to deal with this but cannt find anything that pertains or solves my problem. So, I would love to debug it myself but I really don't know where to begin!
How would you start trying to crack this nut?
Some code:
In the html we insert the following (with display:none set in the css)...
var noteContainer =
'

        (notes)

Notes

' + noteMessage + '

Click (notes) to ' + actionCap + '

';

Then the plugin sets this to diplay:block on hover over an anchor link.
So... where would you start debugging an IE6 javascript issue? I have IE6 running in a VM so I can work with it... just can't get it to tell me anything about what might be going on.
Comment: Is IE throwing an error?
Comment: No, nothing. Just no tooltip.
Answer: Install the IE developer toolbar, and install Visual Web Developer Express (which has a real debugger)
Comment: Installer says the product isn't compatible with my operating system (XP). So... I have a Windows7 machine I can install on but then, how do I run IE6 with Web Developer Express? The only way I am aware of to get IE6 is to grab it as part of the VPC image, but that also runs XP.
Comment: VWD should work on XP.  At the very least, VWD 2008 certainly will.  You may need to install XP SP3.
Comment: Yep... looks like me not having SP3 is the problem. I'll get that and try again, thx.
Answer: In this case, I would use the IE Developer Toolbar to see if the HTML was properly appended to the document. If it is, then we know that the problem is not with the statement you pasted into your question. If it's not in your document (in which case you're probably receiving a script error), I would try gutting your tooltip's HTML string to
var noteContainer = '

';

Test if that's appended correctly. If so, progressively add more of the tooltip content until you get your error.  At that point you'll have a much better idea of what is causing IE6's parsing error.
In general, 'Old school' debugging in IE6-7 using alert() is still effective in many situations.
Visual Studio and Visual Web Developer Express are great options, but they're a little heavy on the system resource side, and could be annoying to work with in a virtual machine.
Answer: Getting IE to work with JS that works in every other browser is a major pain, so my sympathies for that.
What I would do in a case like this is crack open the source that's actually generating and displaying the tooltip on the page. Throw some alerts in there or change document.title at different points in the code. See if it's even making it to the portions where a tooltip should be appearing.
If it is making it to the "rendering" stages of the code, change your debug setup to tell you what's inside the tooltip element, what the element's size is, etc. As much information about the tooptip container as you can gather. Something in there might stick out, like an element with 0 width/height or completely missing data.
Basically, I'd try to narrow it down to either a "data pulling" or "rendering" problem.
Answer: Is the HTML actually being inserted into the DOM properly? If there's a parsing error the DOM methods in IE will choke (can't insert invalid HTML in IE6). This is step 1. Make sure the markup is actually there (view source is fine).
Then see if your event is firing: put an alert inside the (presumably) click handler. See if your css-changing function is actually firing.
Then see if your css-changing function has a handle on the object. Try something like alert(this.tagName) (or e.tagName, or however else you might have a handle on the element to show/hide).
Let us know how those go.
Comment: Thanks for these steps... helped me solve it. I had already done the first step before I posted here on SO. In trying to find the event that was firing I found that the tooltip was not using the plugin at all (it was loading it, but not using it), rather it had been coded to rely on the CSS pseudo-class :hover. Once I realized that it was simple to add my own JS to handle displaying the tooltip. Thanks for the steps, though, as it got me thinking and helped me solve the problem!


Axes recovery after plotting in MATLAB

Question: After plotting the output of multiple methods from text files, the right and upper sides of the axes are not shown. I need to have them and bold them alike the current axes.
The data plotted comes from files in which data for each method have been stored. Each data file is a 256x2 file that includes values between [0,1]. The first column is the precision and the second column is the recall.
figure
hold on
for m = 1:length(methods)
  prFileName = strcat(readpath,dataset, '_', methods{m}, '_PRCurve.txt');
  R = load(prFileName);
  precision = R(:, 1);
  recall = R(:, 2);
  plot(recall, precision,'color',methods_colors(m,:),'linewidth',2);
end
axis([0 1 0 1]);
hold off
grid on;
legend('methode one','method two')
xlabel('Recall','fontsize',12);
ylabel('Precision','fontsize',12);
set(gcf,'color','w');  %Background color
ax = gca;  % current axes
ax.GridLineStyle='-';
ax.GridAlpha=0.7;
ax.XAxis.LineWidth=4;
ax.YAxis.LineWidth=4;
Grid.LineWidth = 3;
set(gca,'FontName','Arial','FontWeight','bold');

My output:

Original data:
[0.201612 1.000000;0.473895 0.813213;0.509366 0.775938;0.524163 0.756849;0.536319 0.744732;0.544424 0.734892;0.552313 0.727097;0.557654 0.719293;0.561069 0.713474;0.565138 0.708002;0.568720 0.705004;0.572685 0.702411;0.574115 0.698205;0.576529 0.697134;0.579346 0.694858;0.581070 0.693213;0.582026 0.692059;0.584509 0.691119;0.585875 0.690336;0.586489 0.688838;0.588433 0.688227;0.589985 0.687624;0.590905 0.687562;0.591892 0.687045;0.592426 0.686695;0.592972 0.686332;0.593253 0.685255;0.594198 0.684903;0.594672 0.684895;0.595856 0.684581;0.597124 0.684201;0.597278 0.683989;0.597886 0.683861;0.598499 0.683794;0.598724 0.683701;0.599475 0.683571;0.599827 0.683561;0.600251 0.683438;0.600228 0.683277;0.600305 0.683071;0.600476 0.683067;0.600579 0.683004;0.600339 0.682803;0.600213 0.682141;0.600268 0.682016;0.600542 0.681904;0.600381 0.681840;0.600448 0.681840;0.600538 0.681726;0.600627 0.681659;0.600770 0.681635;0.600905 0.681482;0.600917 0.681375;0.600734 0.681307;0.600734 0.681307;0.600642 0.681264;0.600720 0.681253;0.601444 0.681184;0.601480 0.681184;0.601718 0.680955;0.601821 0.680955;0.601821 0.680955;0.601958 0.680955;0.602177 0.680655;0.602539 0.680655;0.602674 0.680606;0.602864 0.680606;0.602911 0.680606;0.603022 0.680606;0.603022 0.680606;0.603273 0.680606;0.603358 0.680579;0.603520 0.680579;0.603818 0.680579;0.604029 0.680565;0.604191 0.680565;0.604191 0.680565;0.604371 0.680494;0.604637 0.680476;0.605386 0.680460;0.606010 0.680460;0.606078 0.680460;0.606290 0.679936;0.606984 0.679548;0.607291 0.679497;0.607659 0.679472;0.607994 0.679306;0.608470 0.679037;0.608442 0.678970;0.608796 0.678807;0.609303 0.678714;0.609547 0.678657;0.609927 0.678467;0.610346 0.678302;0.610899 0.677652;0.611175 0.677652;0.611448 0.677301;0.612511 0.677155;0.612640 0.676402;0.613280 0.676402;0.613623 0.676277;0.614415 0.675417;0.615243 0.674605;0.615911 0.674601;0.617164 0.674601;0.618662 0.674476;0.619177 0.674388;0.619723 0.673993;0.620740 0.673695;0.621668 0.673350;0.622682 0.673323;0.623195 0.673204;0.624675 0.673204;0.625555 0.672728;0.626095 0.672155;0.627775 0.672047;0.628613 0.671499;0.629023 0.670547;0.631015 0.670339;0.631373 0.670017;0.632461 0.668883;0.632942 0.668662;0.634569 0.668323;0.635285 0.668082;0.637139 0.667239;0.637270 0.666458;0.638917 0.666368;0.639765 0.666115;0.640943 0.665808;0.641661 0.665704;0.643633 0.665529;0.646006 0.665529;0.646658 0.665329;0.647392 0.664908;0.648859 0.664090;0.649985 0.663583;0.651193 0.662677;0.653684 0.660258;0.654840 0.660054;0.656786 0.659874;0.657570 0.659235;0.658537 0.659190;0.660890 0.658122;0.662920 0.657568;0.664260 0.656880;0.666298 0.656280;0.666916 0.655978;0.668178 0.655131;0.668919 0.654870;0.670389 0.654441;0.670619 0.653603;0.671624 0.652875;0.673049 0.652183;0.674315 0.651285;0.676130 0.651027;0.677092 0.649799;0.678081 0.649090;0.679300 0.648262;0.680943 0.647865;0.680100 0.646171;0.681178 0.645171;0.683523 0.644520;0.683755 0.643928;0.686075 0.643146;0.686905 0.642111;0.688256 0.641657;0.689233 0.641242;0.690506 0.640521;0.692826 0.638231;0.695227 0.637420;0.696323 0.636512;0.697808 0.635450;0.699692 0.633850;0.700870 0.631655;0.701216 0.629642;0.702549 0.628882;0.704164 0.627596;0.705130 0.626842;0.705018 0.624658;0.705942 0.623999;0.707372 0.622667;0.710428 0.621794;0.711609 0.620519;0.713404 0.619295;0.714931 0.617914;0.718499 0.617104;0.720724 0.615774;0.721662 0.614693;0.723181 0.612491;0.723878 0.611185;0.725945 0.609251;0.730281 0.607118;0.730874 0.603678;0.731970 0.600906;0.731515 0.599001;0.732504 0.595722;0.733421 0.592995;0.737195 0.590840;0.738140 0.588574;0.741949 0.587039;0.742875 0.582400;0.743972 0.580038;0.746218 0.579074;0.746583 0.575588;0.749146 0.573102;0.751960 0.570159;0.754440 0.568951;0.754986 0.565920;0.754947 0.561872;0.756065 0.559606;0.758321 0.557936;0.760394 0.555468;0.762893 0.552402;0.764336 0.550675;0.764178 0.547292;0.764643 0.542969;0.767369 0.538886;0.770228 0.536357;0.770978 0.531705;0.770111 0.525041;0.773011 0.521905;0.772255 0.515662;0.775366 0.512602;0.777668 0.509482;0.778520 0.503849;0.779642 0.498346;0.781965 0.492270;0.784695 0.488000;0.785982 0.479703;0.796597 0.473706;0.798725 0.468719;0.800086 0.461603;0.800464 0.455319;0.798910 0.446695;0.800251 0.439831;0.799404 0.431637;0.799290 0.423563;0.800032 0.417575;0.799825 0.408827;0.801278 0.398664;0.802286 0.389090;0.802335 0.381111;0.803801 0.370771;0.812162 0.360939;0.812584 0.347896;0.816370 0.337255;0.817368 0.324137;0.818456 0.308642;0.814710 0.292338;0.819548 0.278081;0.818203 0.262530;0.822571 0.246382;0.828031 0.223384;0.831524 0.190471;0.819948 0.162071;0.811546 0.118579]

Comment: The upper and right axis do not so much as disappear, as they do "not exist". With making them all bold, do you mean to plot a thick line around your figure as a frame? You could plot a vertical and horizontal thick black line at the desired position
Comment: Please add some sample data as well so that the plot is reproducible.
Comment: Can't you just use `set(gca,'LineWidth',4)` ?
Comment: @Adriaan , When in bar() function, I bold on axis, its own parallel axis also bolds. My plan was to do so with this plot. Is there any efficient way for doing that? If not, how your proposed suggestion can be implemented?
Comment: @kkuilla , The question is updated.
Comment: That's not sample data. Add some actual data we can use to make a plot; `rand(256,2)` would probably already suffice, given the problem is just about the axes.
Comment: @Adriaan ,   rand(256,2) may be good but in few minutes I try to add the original data.
Comment: @obchardon , Unfortunately, that doesn't work.
Comment: It is remarkable that the attached data, stored in a ".txt" file and MATLAB readSsfrom that file.
Answer: box on will turn on the display of the box around the graph area. I believe this is what you are asking for.
You might need to use set(ax,'LineWidth',4) rather than ax.XAxis.LineWidth=4 and ax.YAxis.LineWidth=4, I'm not sure of the difference.
Comment: Thanks for the working solution. "set(ax,'LineWidth',4)"  also bolds the grid.

Axes recovery after plotting in MATLAB


Como inserir parágrafos e depois reconhecê-los com um textarea

Question: Gostaria de saber como eu poderia fazer para inserir e depois reconhecer no meu código momentos específicos de parágrafo na digitação em um textarea.
Exemplo:

Este é meu textarea.
Eu quero que ao resgatar o conteúdo desse textarea e inseri-lo em outra
 o mesmo divida os parágrafos inserindo cada um deles em um
.
Então como ficaria no código:

É claro que a adoção de políticas descentralizadoras ainda não demonstrou convincentemente que vai participar...

A nível organizacional, o novo modelo estrutural aqui preconizado nos obriga à análise da gestão inovadora da qual fazemos parte...

Assim mesmo, a hegemonia do ambiente político estende o alcance e a importância de alternativas às soluções ortodoxas...

Como fazer isso?
Answer: Eu creio que você pode pegar o valor da string da sua textarea e dar um split no "\n" para dividir as linhas em um array. Em seguida, basta percorrer e criar um elemento p, inserindo o valor da linha nele, usando um append para inserir em .wrapper. Como vai usar o append, sempre teria que "limpar" o conteúdo dela.
Veja:

var textarea = document.querySelector('#textarea');
var wrapper = document.querySelector('.wrapper');

textarea.addEventListener('input', function () {

   var parts = this.value.split("\n");

  wrapper.innerHTML = '';

  parts.forEach(function (part) {
     var p = document.createElement('p');

     p.innerText = part;

     wrapper.append(p)
  })

})

<textarea id="textarea"></textarea>

Fiz assim porque você fez questão que fosse com a tag p. Porém, se a necessidade é exibir as quebras de linhas, você pode simplesmente inserir o valor do textarea em .wrapper e estilizar o mesmo com o atributo white-space, com pre-wrap ou wrap.
Assim:

var wrapper = document.querySelector('.wrapper');
var textarea = document.querySelector('.textarea');

textarea.addEventListener('input', function () {
  wrapper.innerHTML = this.value;

})
.wrapper{
 white-space: pre-wrap;
 }

<textarea class="textarea"></textarea>

Essa propriedade faz com que o CSS renderize as quebras de linhas que você adicionar no textarea.
Observação: No exemplo acima, como usei innerHTML, tem que tomar cuidado com XSS Injection. Você pode usar innerText opcionalmente, que aparentemente adiciona um
 ao você colocar a quebra de linha no textarea
Comment: mas então em trechos em branco há um \n escondido?
Comment: @NicolasS. na verdade, a quebra de linha é um caractere, que pode ser representado por `"\n"` (tem que ser com aspas duplas, com simples não funciona)


Drupal: using query string data in Views

Question: i have several moderator roles in my drupal site. the users with this roles can create content of specific content-type called News. let's call the roles the following: role_a, role_b, role_c, ...
now i have a View that shows the last 5 News elements.
my question is how to granulate the News elements in View according to the query string?
i mean on page http://mysite.com/a i want to see only the news that was added by the user with the "a" role. http://mysite.com/b is for the "b"-roled user. etc.
how can i use the query string parameters in the Views filter?
Answer: I think you mean you want to use an Argument, rather than the query string. In any case, I don't think Views can handle rolenames by default (it can handle role IDs just fine), so you'll have to modify your view query in order to achieve what you want.
First, add User: Roles as an argument in your View. Then, in a custom module, implement hook_views_query_alter() and modify the query by replacing the rolename with its role ID.
function MYMODULE_views_query_alter(&$view, &$query) {
  if ($view->name == 'my_view') {
    $rolename = '';
    foreach ($query->where as $where_index => $where) {
      // find the role ID clause
      $clause_index = array_search('users_roles.rid = %d', $where['clauses']);
      if ($clause_index !== FALSE) {
        // found it, so get the rolename
        $rolename = $where['args'][$clause_index];
        break;
      }
    }
    // if the rolename argument was found
    if (!empty($rolename)) {
      // get the role ID
      $user_roles = user_roles();
      $rid = array_search($rolename, $user_roles);
      // if the role exists, then replace the argument
      if ($rid !== FALSE) {
        $query->where[$where_index]['args'][$clause_index] = $rid;
      }
    }
  }
}

So, for example, if your url is http://mysite.com/a, then it will look up the ID of role 'a', then find all nodes by an author with that role. It will also take the actual role ID - for example, if the ID of role 'a' is 10, then http://mysite.com/10 will also return the same result.
If you want it only to look up rolenames, you can modify the hook to fail when it doesn't find the role (just make $rid = 0 and you shouldn't get any results).
Answer: function MYMODULE_views_query_alter(&$view, &$query) {
  if ($view->name == 'my_view') {
    $rolename = '';
    foreach ($query->where as $where_index => $where) {
      // find the role ID clause
      $clause_index = array_search('users_roles.rid = %d', $where['clauses']);
      if ($clause_index !== FALSE) {
        // found it, so get the rolename
        $rolename = $where['args'][$clause_index];
        break;
      }
    }
    // if the rolename argument was found
    if (!empty($rolename)) {`enter code here`
      // get the role ID
      $user_roles = user_roles();
      $rid = array_search($rolename, $user_roles);
      // if the role exists, then replace the argument
      if ($rid !== FALSE) {
        $query->where[$where_index]['args'][$clause_index] = $rid;
      }
    }
  }
}


Do we need guidelines?

Question: I still have problems when seeing new questions (or close votes) to tell if they are on topic or not and what to do with them.
There where some discussions earlier but the results where not really conclusive.

Do we want to allow "layman" questions?
Do we want to allow homework questions?

Should we allow all kinds of homework questions or restrict them to some level (e.g., graduate level questions)?
In practice we only allow them if enough effort is shown, which is imho a good idea.
Comment: And, while I am on it: Why is there a self-study tag?
Comment: And now I think we should remove the self-study tag.
Comment: See also http://meta.economics.stackexchange.com/questions/1291/should-we-really-have-a-homework-self-study-tag
Comment: See also http://meta.economics.stackexchange.com/questions/1270/meta-filter-layman-academic-real-world
Comment: Thanks. That answers the tags questions, I will remove if you don't mind.
Answer: Whilst not a complete solution to these issues, I think a useful thought experiment is to evaluate the potential answers rather than the question. If a hypothetical expert writes a good answer to the question then will that be a useful resource to future visitors with a sincere interest in learning about economics? Often, seemingly simple questions can prompt fairly deep and far-reaching economic analysis and I think such questions should be welcomed. Conversely, questions where the only novelty is some mundane technical detail (e.g. "solve this Cobb-Douglas problem with my particular parametrization") are much less likely to help us build up a useful reference resource.
Comment: I agree, but in many cases the question is so specific that it's hard to keep the answer general. On the other side, educating every OP about how to write a general and abstract question about the problem he's having is quite tiresome.

Do we need guidelines?


MATLAB symbolic variables couldn't be used in nested function

Question: I have a MATLAB function to solve a Inertia Tensor , and I have a nested function in my program . All the variables in it are symbolics but it told me
"Error using assignin: Attempt to add "x" to a static workspace"
and I don't understand why this happens . Here is my test.m code:

function test

syms x y z
f=x
f1=f+1
f2=f1^2
    function r=test2
        r=f2^3;
    end
f3=test2
end

After searching this web-forum I have found some answers . But at the same time I just don't understand it
Andrew Janke explianed it like this : While syms A may look like a static variable declaration, it isn't. It's just a regular function call. It's using Matlab's "command" invocation style to look like syntax, but it's really equivalent to syms('a', 'b', 'c').
on this page : Matlab: "Error using assignin: Attempt to add "c" to a static workspace"
what does static variable mean ?
I also search the HELP doc and it said :In functions and scripts, do not use syms to create symbolic variables with the same names as MATLAB® functions. For these names MATLAB does not create symbolic variables, but keeps the names assigned to the functions.
I only know syms x to create a symbolic variable in the workspace but why does the documentation say MATLAB does not create ?
Comment: If we attempt to dynamically add a variable to the workspace of an anonymous function, a nested function, or a function that contains a nested function, then MATLAB® issues an error of the form
what does dynamically add a varibale mean ?
Answer: 'Static' means fixed, 'workspace' is what Matlab calls the places where all of its variables are stored.  For non-nested functions the workspace starts off as empty when Matlab is at the beginning of the function; as Matlab continues through function's lines of code it continuously add more variables to the workspace.
For functions with a nested function, Matlab first parses the function to see what variable will be created (it specifically looks for x = type lines), then it creates all of these variables (with value as 'unassigned'), and then only does it start to run through the code; but while running through the code, it can never create a new variable.
This is why the code
function TestNestedFunction
syms x;
  function Nested()

  end
end

generates an error, there is no x = to tell it to pre-create the unassigned variable x at the start of the code.  It fails at syms x;, as that line tries to create a new variable x, which fails as it may not.
This is also why the following code runs
function TestNestedFunction
syms x;
x = x;
  function Nested()

  end
end

it sees the x = and then pre-creates x.  (This is why your example of adding [x, y, z] = deal([]); also works).
You can test this with a break point at the beginning of simple non-nested function and a simple nested function.  Just run it step by step.
Answer: I think the pages you found are quite clear.
You need to declare the variables one by one and use:
x = sym('x')

Otherwise syms will try to assign the values into a workspace where this is not allowed.
Comment: sym only takes one input argument
Comment: Thank you all the same for answering ! Actually I want to know the reason why sym would be work but syms not . And more about the meaning of " preallocating the variables ", it seems prove that sym work in this way @DennisJaheruddin
Answer: This code works:
function test

x=sym('x')
y=sym('y')
z=sym('z')
f=x
f1=f+1
f2=f1^2
    function r=test2
        r=f2^3;
    end
f3=test2
end

Comment: Thanks very much ! Yeah , it works , but I want to know the difference between syms and sym , they all creat symbolic variable(s) . In this case sym  would be work but syms wouldn't . Could you please explain more about it ? @Daniel R
Comment: The problem has nothing to to with symbolic variables in special, it's common for all variables. `sym` and `syms` use `assignin('caller'...` to assign the variable to the caller workspace if no return variable is used. Using `x=sym('x')` makes the variable `x` available in nested functions as well.
Comment: Appreciate for your patience. Actually I noticed another way to solve it like this , just put    [x y z]=deal([])    before    syms x y z     in a function that contains a nested function , it will work very well and the program could look like more concise . Just don't konw why


Limit of Integral of "Narrowing Limits" Function

Question: Let $f$ be Lebesgue integrable on $[0,1]$, and define $$\large P_nf=n\sum_{k=1}^n\int_{(k-1)/n}^{k/n}f\,d\mu\cdot\chi_{[\frac{k-1}{n},\frac{k}{n}]}.$$
How do we show that $$\lim_{n\to\infty}\int_0^1 |f-P_nf|\,d\mu=0$$?
Thank you very much.

My attempts:
I would think in the direction of either Dominated Convergence Theorem, Monotone Convergence Theorem, or Fatou's Lemma (these are the only 3 tricks I know).
Firstly, I would like to prove $P_nf\to f$ a.e. I can see it intuitively, but not sure how to prove it. I can see that $P_nf$ is trying to partition $[0,1]$ into $n$ subintervals, then for any $x$, it is in one of the intervals $[\frac{k-1}{n},\frac{k}{n}]$, so that $P_nf(x)=n\int_{(k-1)/n}^{k/n}f(x)\,d\mu\to n\cdot\frac{1}{n}f(x)=f(x)$. (How to show this rigorously?)
I can't seem to use DCT due to the "$n$" in $P_nf$ which grows to infinity, so it is hard to find an explicit dominating function. Similarly, MCT seems hard since it is hard to show that $|P_nf|$ is increasing ($n$ is increasing but the "narrowing limits integral" may be decreasing) . That leaves Fatou's Lemma, and I am stuck.
Thanks for any help once again!
Comment: Prove it first for $f \in C([0,1])$ (it essentially follows from uniform continuity) and then estimate an $L^1$ function by a sequence of continuous functions. It is actually more difficult to prove convergence a.e. I think. $L^1$ convergence is much more natural here.
Comment: You can use [Lebesgue's differentiation theorem](https://en.wikipedia.org/wiki/Lebesgue_differentiation_theorem) to make your argument for pointwise a.e. convergence rigorous.
Comment: I'm not sure Lusin's theorem will help here [though, of course, it might]. I'm simply talking about that fact that $C([0,1])$ is dense in $L^1([0,1])$. Thus for any $f \in L^1([0,1])$, we can find a sequence $f_n \in C^1([0,1])$ so that $\| f - f_n \|_{L^1([0,1])} \to 0.$
Comment: Thanks. I haven't learnt how to estimate an $L^1$ function by continuous functions. Can it be done using Lusin theorem?
Answer: A straight forward computation yields
\begin{align}
\int^1_0|f-P_n f|\ d\mu =&\ \int^1_0\left|\sum^n_{k=1} f(x)\chi_{[\frac{k-1}{n},\frac{k}{n}]}-n\sum^n_{k=1} \int^{k/n}_{(k-1)/n} f\ d\mu\cdot \chi_{[\frac{k-1}{n},\frac{k}{n}]}\right|\ d\mu\\
\leq&\ \sum^n_{k=1}\int^{k/n}_{(k-1)/n}\left|f(x)-n\int^{k/n}_{(k-1)/n} f\ d\mu \right|\ d\mu\\
=&\ n\sum^n_{k=1}\int^{k/n}_{(k-1)/n}\left|\int^{k/n}_{(k-1)/n} f(x)-f(y)\ d\mu(y) \right|\ d\mu(x)\\
\leq&\ n\sum^n_{k=1} \int^{k/n}_{(k-1)/n}\int^{k/n}_{(k-1)/n}|f(x)-f(y)|\ d\mu(x)d\mu(y).
\end{align}
If $f$ is continuous on $[0, 1]$, then it's uniformly continuous on $[0, 1]$, which mean for each $\epsilon>0$ there exists $\delta>0$ such that $|f(x)-f(y)|<\epsilon$ whenever $|x-y|<\delta$.
In our case since $x, y \in [(k-1)/n, k/n]$, then for big enough $N$ we have that $|x-y|<\delta$. Hence it follows
\begin{align}
 n\sum^n_{k=1} \int^{k/n}_{(k-1)/n}\int^{k/n}_{(k-1)/n}|f(x)-f(y)|\ d\mu(x)d\mu(y) < \epsilon n\sum^n_{k=1} \int^{k/n}_{(k-1)/n}\int^{k/n}_{(k-1)/n} d\mu(y)d\mu(x) = \epsilon.
\end{align}
Thus, if we could approximate $f$ by a continuous function then we are essentially done.
Fix $\epsilon>0$. By Lusin's Theorem, there exists $g_\epsilon$ continuous such that
\begin{align}
\int^1_0 |f-g_\epsilon|<\epsilon
\end{align}
and likewise we have
\begin{align}
\int^1_0 |P_n f-P_n g_\epsilon|<\epsilon.
\end{align}
Important Note: the $g_\epsilon$ that you get from Lusin's theorem is actually equal to $f$ expect on a set of small measure.
Comment: Think of $P_n f$ as $\sum a_k \chi_{k}$ and see what you when you integrate.
Comment: You need $\int^1_0 |P_n f-P_n g_\epsilon|<\epsilon$ because $\int |f-P_n f| \leq \int |f-g|+|g-P_ng|+|P_ng-P_n f|$.
Comment: Thanks a lot. When we are using Lusin Theorem, we also need this fact right: If $f$ is integrable, then for any measurable $|A|<\delta$, then $\int_A |f|<\epsilon$?

How would we show that $P_nf$ is integrable? Essentially I can understand except the last line $\int_0^1 |P_nf-P_ng_\epsilon|<\epsilon$.

Thanks a lot.
Comment: Just one more question: Why do we need $\int_0^1 |P_nf-P_ng_\epsilon|<\epsilon$? I don't see it used explicitly anywhere.
Comment: Ok, very nice. Thanks and have a great week ahead!


Como fazer um submit que zera um contador

Question: Eu não entendo de javascript e não to sabendo como pesquisar para fazer um botao que quando apertado zerasse meu contador que é uma variável global
<form id="btn" method="get">
<input type="submit" name="submit" value="zerar"onclick="submit();">
</form>

dai ele chama uma outra função que resetaria a variavel "num"
  aonde eu coloco para chamar a outra função?
Comment: Qual a variável? Poderia fornecer um exemplo de código um pouco mais detalhado?
Comment: oi, então a variavel é um inteiro normal mesmo - >int num = 3;
e ela fica lá no topo do código.
Comment: eu estou trabalhando com arduino num projeto da faculdade, e são 3  blocos as entradas,  o html, e a função de contador + o reset
Answer: Você precisa mesmo que o botão seja um submit? Porque você pode controlar o clique em qualquer elemento para isso. De qualquer forma, segue.

<form id="btn" method="get">
  <input type="submit" name="submit" value="zerar" onclick="ssubmit();">
</form>

Já caso você queira que, em vez de submit, aconteça só a chamada da função, você pode fazer:

<button onclick="ssubmit();">Zerar</button>

Caso eu não tenha respondido sua pergunta, explicite melhor o que você precisa.
Comment: Oi amigo, eu dei um outro jeito, trabalhar com arduino é muito confuso e tenho outros trabalhos pra fazer u.u
fiz uma função que lê a url e quando isso acontecer entra no meu if e zera, tem uns bugs, mas ta funcionando xD
Obrigado mesmo assim


How to restore subdirectory of a duplicity backup?

Question: I have a duplicity backup script running under Ubuntu 18.04 which has backed up the contents of my /etc directory like this:
duplicity --archive-dir=/home/bkp/.cache /etc rsync://backup.host::/bkp/etc

I now want to restore the /etc/postfix directory from that. But when I try to restore it with:
duplicity restore rsync://backup.host::/bkp/etc/postfix ./postfix.restored

it says:
rsync: change_dir "/etc/postfix" (in bkp) failed: No such file or directory (2)

If I do duplicity list-current-files on the /etc backup I can see it's there though. I can't see anything in the man page about this though.
Answer: duplicity backups are no file system, so you cannot navigate them this intuitively. from the duplicity man page

--file-to-restore path
This option may be given in restore mode, causing only path to be restored instead of the entire contents of the backup archive. path should be given relative to the root of the directory backed up.

so in your case the command should look something like
duplicity restore --file-to-restore 'postfix' rsync://backup.host::/bkp/etc ./postfix.restored

..good luck ede/duply.net


See https sites in local with Debian / Apache2

Question: I'm working on Debian Testing. I see local http sites at localhost/~myhome/... To this end I made the classic change to /etc/apache2/mods-available/php8.2.conf
To re-enable php in user directories comment the following lines
...

However I have an https site in which .htaccess I added
RewriteEngine on
RewriteCond %{HTTP_HOST} ^sito.net$ [OR]
RewriteCond %{HTTP_HOST} ^sito.it$ [OR]
RewriteCond %{HTTP_HOST} ^www.sito.it$
RewriteRule ^/?$ "http\:\/\/www\.site\.net" [R=301,L]`

and still from local I cannot see it, receiving:
Internal Server Error
The server encountered an internal error or misconfiguration
and was unable to complete your request.`
Please contact the server administrator at webmaster@localhost
to inform them of the time this error occurred, and the actions
you performed just before this error.
More information about this error may be available
in the server error log.
Apache/2.4.58 (Debian) Server at localhost Port 80`

I followed some (old) instructions found in this thread: How do I allow HTTPS for Apache on localhost?
I did the following steps:
# openssl req -x509 -newkey rsa:2048 -keyout mykey.key -out mycert.pem -days 365 -nodes
#cp mycert.pem mykey.key /etc/ssl/private/
#a2enmod ssl
#a2ensite default-ssl.conf
#systemctl restart apache2

but receiving:
root@debian:~# systemctl restart apache2
Job for apache2.service failed because the control process exited with error code.
See "systemctl status apache2.service" and "journalctl -xeu      apache2.service" for details.

And after:
root@debian:~# systemctl status apache2.service
× apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Mon 2023-12-04 21:35:15 CET; 8s ago
Docs: https://httpd.apache.org/docs/2.4/
Process: 4216 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
    CPU: 93ms

dic 04 21:35:15 debian systemd[1]: Starting apache2.service - The Apache HTTP Server...
dic 04 21:35:15 debian apachectl[4218]: AH00526: Syntax error on line 227 of /etc/apache2/apache2.conf:
dic 04 21:35:15 debian apachectl[4218]: Cannot define multiple Listeners on the same IP:port
dic 04 21:35:15 debian systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
dic 04 21:35:15 debian systemd[1]: apache2.service: Failed with result 'exit-code'.
dic 04 21:35:15 debian systemd[1]: Failed to start apache2.service - The Apache HTTP Server.

The line referring to that error:
# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf
Listen 80
NameVirtualHost *:80
<VirtualHost *:443>
DocumentRoot "/home/samiel/public_html"  #your wamp www root dir
ServerName debian
SSLEngine on
SSLCertificateFile "/etc/ssl/certs/mycert.pem"
SSLCertificateKeyFile "/etc/ssl/certs/mykey.key"

I am unable to make any further attempts...
Comment: I would first "dump" your whole running config with `apachectl -S`
Answer: First install
apt install libapache2-mod-php8.2
Dump your whole running config with apachectl -S
And when you want to make an application available to be served by apache, it's best practice to create a virtual host in /etc/apache2/sites-available
An example of a virtual host looks like this
    <VirtualHost *:80>
    ServerName yourdomain.com
    ServerAlias www.yourdomain.com

    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    RewriteEngine on
    RewriteCond %{SERVER_NAME} =yourdomain.com [OR]
    RewriteCond %{SERVER_NAME} =www.yourdomain.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

<VirtualHost *:443>
    ServerName yourdomain.com
    ServerAlias www.yourdomain.com

    DocumentRoot /var/www/html

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/yourdomain.crt
    SSLCertificateKeyFile /etc/ssl/private/yourdomain.key
    SSLCertificateChainFile /etc/ssl/certs/yourdomain.ca-bundle

    <Directory /var/www/html>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    <FilesMatch \.php$>
        SetHandler "proxy:unix:/var/run/php/php8.2-fpm.sock|fcgi://localhost/"
    </FilesMatch>
</VirtualHost>

Then save it in /etc/apache2/sites-available/ as yoursite.conf

Then enable the vhost  sudo a2ensite yourdomain.conf
Test your configuration  sudo apachectl configtest
If everything is ok restart apache  sudo systemctl restart apache2
Comment: Thank you for your detailed answer. I created a .crt and a .key certificate by command: openssl req -x509 -newkey rsa:2048 -keyout mykey.key -out mycert.pem -days 365 -nodes. How can I create a .ca-bundle too?
Comment: For example `openssl req -x509 \
            -sha256 -days 356 \
            -nodes \
            -newkey rsa:2048 \
            -subj "/CN=demo.mlopshub.com/C=US/L=San Fransisco" \
            -keyout rootCA.key -out rootCA.crt` but you need create new self signed certs, because they need to be signed with the ca certificate https://devopscube.com/create-self-signed-certificates-openssl/


Playing mandolin just strumming

Question: I don't understand how to play chords on mandolin, the most of the tabs avaiable are with mandolin chords but I know it is possible to play a song just strumming, and I find that way so much easier... Can someone explain this to me? How can I know how the correct notes for a certain song, just strumming?
here is an example of what i'm trying to explain
Answer: What you're featuring here are two mandolins playing. The one shown is playing a single note tune, while the other is playing chords - strumming.
So, your question is confusing as it stands. Chords are generally strummed - and really one wouldn't call playing the two strings at a time, producing a melody, (as in the video) strumming.
Could you please, with this info. available, explain more clearly exactly what you need - chords or melody. Both of which are available on mandolin music. By the way, strumming the chords alone will not make the melody stand out - it's mainly as an accompaniment.

Playing mandolin just strumming


What function/method should I use for file_directory_path()?

Question: In Drupal 6, I am using the following code.
$id = fileowner(file_directory_path());

What is the Drupal 8 equivalent of that code?
Answer: You can use the following code.
$real_path = \Drupal::service('file_system')->realpath(file_build_uri(''));

On my test site, it returns '/Library/Server/Web/Data/Sites/avpnet/sites/default/files'.
You can also pass a filename to file_build_uri().
$real_path = \Drupal::service('file_system')->realpath(file_build_uri('file.jpeg'));

With this code, I get '/Library/Server/Web/Data/Sites/avpnet/sites/default/files/file.jpeg'.
Answer: You can use  \Drupal\Core\File\FileSystem::realpath() to get the realpath of the file or  \Drupal\Core\File\FileSystem::dirname(). As file_directory_path() is deprecated from Drupal 7. For more details please visit at drupal_realpath and drupal_dirname


How to create index for the following kind of data in mongodb

Question: <code>db.test.insert({a:1, b:[1])
db.test.insert({a:1, b: 1})
db.test.insert({a:[1], b: 1})
db.test.insert({a:[1], b: [1]})
</code>
what should be a right way to index the fields so that i can query over a and b?
Answer: <code>{ _id: 1, a: [1, 2], b: 1, category: "A array" }
{ _id: 2, a: 1, b: [1, 2], category: "B array" }
</code>

A compound multikey index { a: 1, b: 1 } is permissible since for each
document, only one field indexed by the compound multikey index is an
array; i.e. no document contains array values for both a and b fields.
However, after creating the compound multikey index, if you attempt to
insert a document where both a and b fields are arrays, MongoDB will
fail the insert.

---- Documentation
But one of your doc's a & b both contain array.
Comment: So for this kind of document scopes , any index is not possible ?
Comment: Documentation says so.


How to get install and uninstall stats from Google Developer Play store?

Question: Does google developer API provides install and uninstall information? I found one that provide reviews but I couldn't find anything related to install/uninstall. Is there any option to get install/uninstall information using API?
Answer: I found solution using C#.
<code>var storageService = new StorageService(new BaseClientService.Initializer
            {
                ApplicationName = "App Name",
                ApiKey = "APIKey goes here ",
            });

            var storage = StorageClient.Create();
            string localPath = "C:\\test\test.csv";
            using (var outputFile = File.OpenWrite(localPath))
            {
                storage.DownloadObject("gs://pubsite_prod_rev_23628868/stats/installs/", "installs", outputFile);
            }
</code>

ApiKey --> Create a google project and generate API Key
gs://pubsite_prod_rev_23628868/stats/installs/ --> You can get this path in your Google Developer Dashboard under Reports


Function that opens a web page and returns a dictionary of all the links and their text

Question: Write a function that opens a web page and returns a dictionary of all thelinks and their text on that page.  A link is defined by an HTML tag that looks like
< ahref="http://my.computer.com/some/file.html">link  text
The link is everything in quotes after thehref=, and the text is everything between the > and the . For the example above, the entry in a dictionary would look like:
"{"http:// my.computer.com/some/file.html" : " link text ", ...}"
Here's my code so far I've been stuck on for a few hours. How do I accomplish this problem?
import urllib.request

def Urls(webpage):
    url = webpage
    page = urllib.request.urlopen(url)
    url_list = {}
    for line in page:
        if '<a href=' in line:

Answer: While the answers suggesting to tackle this problem with regular expressions might work, they'll fail (unless you take measures) when e.g. the link is split over several lines. E.g. this is perfectly valid HTML:
link

There are some other edge cases to consider. In general, HTML cannot be parsed with regular expressions and there's some excellent prose written about that. By the way, the construction "a href" in line is a less powerful form of regular expression, that just searches in a line and has the same drawbacks.
Instead, you should be looking into libraries that parse HTML as a properly formatted XML-document. In Python, the go-to library would be beautifulsoup. With it, you can quickly get all links in a webpage, e.g. like this:
import urllib
from bs4 import BeautifulSoup
url = "http://www.imdb.com/"
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page, 'html.parser')
soup.find_all('a') # returns all links as a list
links = [a['href'] for a in soup.find_all('a', href=True)] # some anchors might have no href attribute, by specifying href=True, you'll get only those that do

The beautifulsoup documentation is very well-documented, with tons of examples. Well worth a read.
Answer: def Urls(webpage):
    url = webpage
    page = urllib.request.urlopen(url)
    url_list = {}
    for line in page:
        if '<a href=' in line:
            try:
                url = line.split('')[0]
                txt = line.split('')[-1].split('')[0]
                url_list[url] = txt
             except:
                 pass
     return url_list

Answer: r=re.compile("<\s*a\s*href=\"(.*?)\">(.*?)<\s*/a\s*>")
list = r.findall(line)
for tuple in list:
    url_list[tuple[0]] = tuple[1]

Comment: Please don't recommend regular expressions to parse HTML. There are stable libraries for this task.
Answer: Using requests and SoupStrainer for simplicity/efficiency:
import requests
from bs4 import BeautifulSoup, SoupStrainer

def get_urls(webpage):
    res = requests.get(webpage)
    links = [l for l in BeautifulSoup(res.text, parseOnlyThese=SoupStrainer('a'))
             if l.has_attr('href')]
    return links


How do I add taxonomy to my custom entity?

Question: I am working with a custom entity and I would like to be able to add taxonomy to the entity.  The drupal 8 entity documentation is lacking.
I have tried reading Node.php and term.php entitiy files but I don't understand how any sort of taxonomy is linked to any other entity though this.
Do I need to change something in my entity definition to allow taxonomy to be added? Or is there a way to access the taxonomy entity?
Edit:  I am working on the same pattern to add this to the install file of my module
<code>'category' => array(
                'type' => 'varchar',
                'not null' => TRUE,
                'default' => '',
                'description' => 'The section front  to appear on',
            ),
</code>
this is the exception that I get trying to install the module
<code>exception 'PDOException' with message 'SQLSTATE[42000]: Syntax error [error]
or access violation: 1064 You have an error in your SQL syntax; check
the manual that corresponds to your MySQL server version for the
right syntax to use near 'NOT NULL DEFAULT '' COMMENT 'The section
front  to appear on',
</code>
Comment: Taxonomy terms are entities too, so an entity reference field would probably be the best choice
Comment: For the edit: `varchar` columns require a `length`
Answer: As Clive said, you use an entity reference field for that, just like articles have a tags reference field to terms.
If you want to define it in code, it's really the same as the node.uid field, which is an entity reference to a user. Or the user.roles field, which is a multivalue reference to user roles. Entities reference others using an entity reference field, it's always the same pattern.
Comment: i am working on the same pattern to add this to the install file of my module
Answer: For custom entities, I would suggest to work with @ContentEntityType and baseFieldDefinitions() method, instead of creating the table on .install file.
You can see more details on why take this path in When to use BaseFieldDefinition instead of hook_schema an on this implementation.
A taxonomy field is now on D8 only an entity reference field, and you can have an example on how to use it at the field "voted_entity_id" at https://github.com/pedrorocha-net/votingapi/blob/8.x-1.x/src/Entity/Vote.php
Comment: Awesome!  This works,  now I just need to take it one step further and limit the vocabulary it can be.  Is that possible?
Comment: I think it is. You probably can add these settings when calling BaseFieldDefinition::create.
Take a look at https://api.drupal.org/api/drupal/core!modules!node!src!Entity!Node.php/function/Node%3A%3AbaseFieldDefinitions/8 to have as reference.


html Form within a block

Question: I have a form within custom block in drupal 8 and I know that this is not the right method to write a form within  a block but someone wrote it and I am trying to fix it without changing the method.
In the return of the  public function build() , he rendered the html of a form within the markup.
 return array(
      '#markup' => '<form>
//we have here many inputs
//then at the end:
 <input  type="hidden" name="form_id" value="country_language_form">

'.t("Submit").'

</form>',

I was wondering how does this code work (the form is actually submitted) when the user is anonymou s and it doesn't work (click on submit doesn't do anything) when the user is logged in as admin.
Comment: Honestly you'd be better off writing it with form api and calling that from the block.
Answer: My first guess would be an incorrect targeting of the submit button when the user is logged in (be aware that Drupal adds a lot of elements to the DOM when being logged in).
I'd highly recommend you to use the right method and build your form with the form API, though.
Your block build code would look like this:
  public function build() {
    $form = \Drupal::formBuilder()->getForm('Drupal\your_module_name\Form\YourForm');
    $build['content'] = $form;
    return $build;
  }

And all your form-related code would be in the YourForm.php file.
Comment: Thank Youuuu !!

html Form within a block


What is POSIX's equivalent of Win32's Mutex?

Question: POSIX's mutex is equivalent to Win32's CRITICAL_SECTION -- its scope is limited to a single process. Win32's mutex (Actually called a "mutant" in NT land) serves as a cross process locking mechanism. What is pthreads' equivalent for cross-process locks?
Comment: @Billy: you can set it not to on multi-processor systems: http://msdn.microsoft.com/en-us/library/ms683476(v=vs.85).aspx
Comment: `CRITICAL_SECTION` is closer to `pthread_spinlock_t`, mutex would be `WaitForSingleObject`/`ReleaseMutex` vs. `pthread_mutex_t`.
Comment: @Steve-o: What is the difference between those two? To my (albeit limited) understanding, the only difference between a critical section and a mutex is that one is within a single process, while the other is cross-process. What then is the diffrence between the posix spinlock and the posix mutex?
Comment: @Billy: This answer might help, spinlocks busy-wait, mutexes yield   http://stackoverflow.com/questions/195853/spinlock-versus-semaphore
Comment: @Steve-o: Critical sections yield.
Comment: @Steve-o: You can use the optimization of not always yielding, but you're in general not going to use up an entire time slice spinning for a critical section.
Comment: Also Mutex-with-spinlock on Windows is equivalent to PTHREAD_MUTEX_ADAPTIVE_NP option on mutex in Linux (only).
Answer: It's a pthread_mutex_t with a pshared attribute set to PTHREAD_PROCESS_SHARED . However, you're responsible to place such a mutex in shared memory, that all processes can access - so it's not as simple as the win32 api.
Perhaps closer to win32 is a posix or sysv semaphore. Traditionally, synchronization across processes has also been done using file locks e.g. flock or lockf (this is in no way as slow as it might sound)
Comment: Doesn't scope only apply to threads? You can only pass `pthread_attr_t` to `pthread_create`. You need to use `pthread_mutexattr_t` with `pthread_mutex_init`.
Comment: You're right, it's the pshared of pthread_mutexattr you'll need
Comment: Do you have a link on the slowness (or lack thereof) of locks? I searched for more detail on that but haven't found anything interesting. We have a big project with big shared files and I suspect we have some performance problems because of the locks, but I have no idea how to find out.
Comment: Using filesystem-based locks will take a good 5000-20000 cycles per lock or unlock operation; syscall overhead is incurred regardless of whether there is contention. With mutexes, the only overhead is your cpu's memory synchronization penalty unless there is contention. Using `lockf` may be acceptable for rarely-used locks, but for anything performance-oriented, forget it.
Comment: Thank you R. it's even worse than that because the files are mapped in memory and modified by memory operation. I dare not imagine what the kernel has to do detect a modification in the locked part of the file.
Answer: You should use IPC for cross-process operations: pipes, semaphores, message queues, or shared memory.
I think in your case named semaphores would be fine. For more information:
man 7 sem_overview

Comment: I do want to use shared memory. But there still needs to be a way to perform locks on that shared memory.
Comment: sem_init() can be used to perform locks on shared memory.
Comment: Forget this useless semaphore stuff. Create a shared memory object and then create a pshared mutex in it.


Examples where the evaluation of the posterior distribution $p(Z|X)$ of the latent variables $Z$ is a central task?

Question: In PRML Chapter 10 Approximate Inference, the fist sentence of the chapter says

A central task in the application of probabilistic models is the evaluation of the posterior distribution $p(Z|X)$ of the latent variables $Z$ given the observed (visible) data variables $X$, and the evaluation of expectations computed with respect to this distribution.

The book then uses EM as an example, for we need to compute the expectation of $p(Z|X)$ in the E step and the expectation of the log likelihood with respect to $p(Z|X)$ in the M step, which is clear.
What else are some common problems that require the evaluation of the posterior $p(Z|X)$ (of latent variables)?

Update
After a year I've realized at the time this question was posted, my experience with machine learning was mostly limited to solving MLE/MAP by gradient descent that only require computing a likelihood and its gradient. Followings are some models I've worked with later on that require the evaluation of the posterior probability.
EM
EM itself is a optimization algorithm for MLE/MAP, nevertheless in every iteration the E-step computes the posterior $p(Z|X)$, which could be solved efficiently in the HMM/GMM setting.
recursive filtering
In the prediction step we compute the marginal
$$p(z_t|x_{1:t-1})=\int p(z_t|z_{t-1})p(z_{t-1}|x_{1:t-1})dz_{t-1}$$
in the correction step we compute the posterior
$$p(z_t|x_t, x_{1:t-1})\propto p(x_t|z_t)p(z_t|x_{1:t-1})$$
The Kalman filtering algorithm assumes an underlying linear Gaussian system, so both the marginal and the posterior have close-form solutions.
The Extended Kalman filtering algorithm is designed for nonlinear Gaussian systems. In this setting the posterior no longer have close-form solutions, so approximate inference is used to approximate a Gaussian distribution for $p(x_t|z_t)$, so that the posterior becomes Gaussian again.
While the particle filtering algorithm uses sampling methods to approximate a broader range of probability distributions (and nonlinear dynamics).
Answer: Any method involving a latent variable can be represented in the terms you mentioned and the EM algorithm is often used as a part of the estimation or prediction procedure for $Z$, such as

Mixture modeling, such as latent class analysis
Item response theory
Factor analysis

Note that not always the use of the posterior distribution of $Z$ is obvious, since not all estimation approaches use Bayesian statistics. However, in each of the three examples distinct representations can be given to $X$ and $Z$, i.e.

Latent class analysis: $Z$ is a discrete latent variable representing class membership, $X$ is a vector of the observed discrete indicators of latent classes.
Item response theory: $Z$ is a continuous variable representing a "trait" or other true cause of $X$, which is a vector of observed discrete indicators of the latent trait.
Factor Analysis: $Z$ is a continuous variable representing a trait or other true cause of $X$ which is a vector of continuous indicators of the latent factor.

I all of the above examples, $Z$ may also be a vector, but there are many practical applications where unidimensionality of $Z$ is assumed.


A whole view disappears

Question: I have deployed user generated views to a production site using Views (6.x-2.12) bulk export feature; that is, views are saved in the code as mymodule.module and mymodule.views_default.inc, the latter of which contains 12 views in more than 11,000 lines.
Sometimes, one or more views suddenly disappear from the site. Flushing all caches usually fixes it, but it is hard to duplicate the issue and it bothers me that it may happen again in the worst time.
Possible causes that I think of are:

Does the .inc file have too many lines (> 11K)?
I have saved the .inc file with Windows new lines, while the server is Linux.

Could one of these be the cause?
Anyone experienced a similar issue?
Comment: How did you fix this. It doesn't seem that simply moving your views into separate modules would work?
Answer: Solution for the "to many lines" problem:
function YOURMODULE_views_default_views() {
    $files = file_scan_directory((drupal_get_path('module','YOURMODULE').'/views','.view'));
    foreach($files as $absolute => $file) {
        require $absolute;
        if(isset($view)) {
            $views[$view->name] = $view;
        }
    }
    return $views;
}

This way every view-code can be stored in it's own file inside a views subfolder inside YOURMODULE as some_name.view.

A whole view disappears


Wordpress posts display in Zigzag format using bootstrap

Question: I'm trying to display posts in the following attached format. But, Posts are displaying one after one..
Expected output :

And Here is my code
<?php
  $type = 'post';
  $args=array('category__in'=> array(4), 'post_type' => $type, 'posts_per_page'=>3, 'order'=>'DESC');
  $my_query = new WP_Query($args);
  $i=1;
  while ($my_query->have_posts()) : $my_query->the_post(); ?>

      <div class="col-xs-12 col-sm-6" <?php if($i%2==1){?> style="float:left;" <?php }?> style="text-align: center;">

        <?php the_post_thumbnail(); ?>

<?php the_title();?>

<?php the_content(); ?>

<?php
$i++;
endwhile;
wp_reset_query();
?>

Current Output screenshot
Comment: There should only be one style attribute at each tag
Comment: share screenshot of current output.
Comment: @VasimVanzara, Attached my current output screenshot.
Answer: For re-ordering columns in bootstrap you can use pull/push concept (till bootstrap 4). Or order- concept(from bootstrap 4). As you are using bootstrap 4. try below code:
<?php
  $type = 'post';
  $args=array('category__in'=> array(4), 'post_type' => $type, 'posts_per_page'=>3, 'order'=>'DESC');
  $my_query = new WP_Query($args);
  $i=1;
  while ($my_query->have_posts()) : $my_query->the_post(); ?>

      <div class="col-xs-12 col-sm-6 <?php if($i%2==1){ echo 'order-sm-2';} ?>">

        <?php the_post_thumbnail(); ?>

<?php the_title();?>

<?php the_content(); ?>

<?php
$i++;
endwhile;
wp_reset_query();
?>

Comment: Glad to help @kumar. :)
Answer: Get a Zigzag layout you have execute code alternativly.
<?php
  $type = 'post';
  $args=array('category__in'=> array(4), 'post_type' => $type, 'posts_per_page'=>3, 'order'=>'DESC');
  $my_query = new WP_Query($args);
  $i=1;
  while ($my_query->have_posts()) : $my_query->the_post(); ?>

        <?php if($i%2==1){?>

        <?php the_post_thumbnail(); ?>

<?php the_title();?>

<?php the_content(); ?>

      <?php } else {?>

<?php the_title();?>

<?php the_content(); ?>

            <?php the_post_thumbnail(); ?>

      <?php } ?>

<?php
$i++;
endwhile;
wp_reset_query();
?>

Comment: Same out out  :(
Comment: Is variable $i=1 or $i=0 before while loop..?
Comment: @kumar try Jasbir's answer. It will help you
Comment: Jasbir's code worked awesome, But it shows different in mobile view.
The format should come one after one like Image, Title and Content
Comment: Design purspactive same here. :)
Answer: <?php
  $type = 'post';
  $args=array('category__in'=> array(4), 'post_type' => $type, 'posts_per_page'=>3, 'order'=>'DESC');
  $my_query = new WP_Query($args);
  $i=1;
  while ($my_query->have_posts()) : $my_query->the_post();
  if($i%2==0) {
  ?>

<?php the_title();?>

<?php the_content(); ?>

        <?php the_post_thumbnail(); ?>

  <?php } else { ?>

        <?php the_post_thumbnail(); ?>

<?php the_title();?>

<?php the_content(); ?>

  <?php }
$i++;
endwhile;
wp_reset_query();
?>

Comment: @kumar if this solution works for you please mark this answer as a solution
Comment: Your code works for me in desktop. But, when we go in mobile view, the format should come (Image, Title, Content) And (Image Title Content) And (Image Title Content). Your code displaying in mobile (Image Title Content) And (Title Content Image) and (Image title Content)
Comment: http://forealapp.com/new/  I have used your code.. Please check this in mobile, Hope you can understand then...
Comment: @kumar try my answer below, Hope it will help you
Answer: With pure CSS, you could just apply float: left and float: right alternatingly to the figureelement, using row:nth-child(odd) and row:nth-child(even) for the parent row to adress them accordingly.
Since you'll be using the .row class also at other positions on the page, you should apply a dedicated class to those rows which serve as containers for the posts. So if .post-container would be the additional class for these rows, the CSS would be
.post-container:nth-child(odd) .pro-innerpage-img {
  float: left;
}
.post-container:nth-child(even) .pro-innerpage-img {
  float: right;
}

Note: In this case the figure element should be in the same container as the content.
Answer: Use CSS to style the contents:
.row:nth-child(odd) div figure { float: left;}
.row:nth-child(odd) div div { float: right;}
.row:nth-child(even) div figure { float: right;}
.row:nth-child(even) div div { float: left;}

Answer: Try This,
<?php
  $type = 'post';
  $args=array('category__in'=> array(4), 'post_type' => $type, 'posts_per_page'=>3, 'order'=>'DESC');
  $my_query = new WP_Query($args); $i=1; ?>

        <?php while ($my_query->have_posts()) : $my_query->the_post(); ?>
            <?php if($i%2==1){ $class = 'odd'; } else { $class = 'even'; } ?>
                <div class="col-sm-6 img <?php echo $class; ?>" style="text-align: center;">

                        <?php the_post_thumbnail(); ?>

                <div class="col-sm-6 desc <?php echo $class; ?>" >

<?php the_title();?>

<?php the_content(); ?>

        <?php $i++; endwhile; wp_reset_query(); ?>

in CSS,
.desc.odd {
    float: left;
}
.img.odd {
    float: right;
}


Generate and combine functions from collections

Question: I want to compose a sequence of functions that when given a string passes it through all the created functions and produces a modified string.
e.g.
string[] arr = {"po", "ro", "mo", "do"};

var modify = "pomodoroX";
foreach (var token in arr)
{
    modify = modify.Replace(token, "");
}
Console.WriteLine(modify); // Output: X

This solves the problem, but I am interested in the Functional solution:
Console.WriteLine(
    arr.Select<string, Func<string, string>>(val => (s1 => s1.Replace(val, string.Empty)))
       .Aggregate((fn1, fn2) => fn1 += fn2)
       .Invoke("pomodoroX")
);
   // Output: pomoroX -> Only last element applied because:
   // the functions are not getting combined.

So basically, take the array "arr" and for each string create a function that removes that string.
The current solution is flawed and applies only the last function and I can't seem to convert this to delegates so as to combine them with += operator.
Or are there better functional solutions?
Answer: Well, your Select gives you the collection of delegates which take in a string, and produce the modified string, so you're halfway there. All you need is to chain these together via Aggregate - and the way you do it is as follows:
string[] arr = { "po", "ro", "mo", "do" };

string result = arr
    // Produce our collection of delegates which take in the string,
    // apply the appropriate modification and return the result.
    .Select<string, Func<string, string>>(val => s1 => s1.Replace(val, string.Empty))
    // Chain the delegates together so that the first one is invoked
    // on the input, and each subsequent one - on the result of
    // the invocation of the previous delegate in the chain.
    // fn1 and fn2 are both Func<string, string>.
    .Aggregate((fn1, fn2) => s => fn2(fn1(s)))
    .Invoke("pomodoroX");

Console.WriteLine(result); // Prints "X".

Comment: Thanks a lot friend! Soo function composition goes just as in math fn(fn2(param)).
Answer: I don't really know what counts as "functional". I assume you don't want to use any flow control structures.
This is simpler, don't you think?
string[] arr = {"po", "ro", "mo", "do"};
arr.Aggregate("pomodoroX", (x, y) => x.Replace(y, ""))

Comment: Cool, just learning c# and it is so powerful. (Am still thinking Java way).
With functional I mean to not have a variable that I am constantly modifying externally. This way we are passing it through a sequence of functions that each makes its modification and passes it to the other as a parameter.