Is this a possible multivariate generalization of pointwise Mutual Information? If so, how can it be formalized? I understand that pointwise Mutual Information (pMI) is the probability of a sequence of dependent events conditioned on the probability of its elements treated as independent events.

The definition of mutual information given in Church &amp; Hanks (1990) and based on Fano (1961) is the following:

"[...] if two points (words), x and y, have probabilities P(x) and P(y), then their mutual information, $I(x,y)$, is defined to be:

$$I(x,y) = log_2\frac{P(x,y)}{P(x)P(y)}$$

Informally, mutual information compares the probability of observing $x$ and $y$ together (the joint probability) with the probabilities of observing $x$ and $y$ independently (chance). If there is a genuine association between $x$ and $y$, then the joint probability $P(x,y)$ will be much larger than chance $P(x)P(y)$, and consequently $I(x,y) \gg 0$. If there is no interesting relationship between $x$ and $y$, then $P(x,y) \approx P(x)P(y)$, and thus, $I(x,y) \approx 0$. If $x$ and $y$ are in complementary distribution, then $P(x,y)$ will be much less than $P(x)P(y)$, forcing $I(x,y) \ll 0$.''

Pointwise Mutual Information typically applies to sequences of two events, but it has sometimes been extended to longer sequences. For example, the pointwise mutual information of a sequence ABCD would be:

$$I(ABCD) = log_2\frac{P(ABCD)}{P(A)P(B)P(C)P(D)}$$

I would like to condition of the probability of a sequence not only on its atomic elements but also on all its possible subsequences. In the context of the above example, this would be:

$$I(ABCD) = log_2\frac{P(ABCD)}{P(A)P(B)P(C)P(D)P(AB)P(BC)P(CD)P(ABC)P(BCD)}$$

Does this also fall under mutual information? If not, does this way of conditioning the probability of a set of dependent events have ab explicit mathematical treatment in probability theory and/or information theory? Is there a way to express this way of conditioning events formally in a general fashion applicable to sets of events of arbitrary length? I tried formalizing it this way (see image below) but it seems very clunky and, more importantly, I don't know where to encode the information that there will be N-1 rows in the denominator:

mutual-info-extended

References:

Church, K. W. and P. Hanks (1990). Word association norms, mutual information and lexicography. Computational Linguistics 16(1), 22-29.

Fano, R. (1961). Transmission of Information: A Statistical Theory of Communications. Cambridge, MA: MIT Press.
The second equation in your post looks sufficient, assuming order is not important. The original definition of mutual information doesn't include order, it's just "these two things happen together more often (or not) than you'd expect them to occur based on their individual probabilities, assuming independence".  Your 2nd equation says the same thing: "these 4 things happen together more (or less) frequently than they would if they were all independent". That's exactly what I'm looking for in my application (and now that I see your 2nd equation, it's obvious); while I can't immediately say it's wrong, I'm not sure why you'd want to look at the subsets. And note that the way you've written it, they are subsets, not subsequences; if they were sequences then you would have separated P(AB) and P(BA).

Since both are plausible, but the first is simpler, I'd invoke Occam's Razor; however, another way to figure out which one you want is just to try both on some data where you think you know the "correct" answer, and see which ones gives the most sense. Certainly that's what the author did of the only published paper I can find on the subject, which is called "Two multivariate generalizations of pointwise mutual information". He just tried them both and looked at which one worked better. Link below, though if you use Google Scholar on the title above you can find several versions.

https://dl.acm.org/citation.cfm?id=2043124
The second equation in your post looks sufficient, assuming order is not important. The original definition of mutual information doesn't include order, it's just "these two things happen together more often (or not) than you'd expect them to occur based on their individual probabilities, assuming independence".  Your 2nd equation says the same thing: "these 4 things happen together more (or less) frequently than they would if they were all independent". That's exactly what I'm looking for in my application (and now that I see your 2nd equation, it's obvious); while I can't immediately say it's wrong, I'm not sure why you'd want to look at the subsets. And note that the way you've written it, they are subsets, not subsequences; if they were sequences then you would have separated P(AB) and P(BA).

Since both are plausible, but the first is simpler, I'd invoke Occam's Razor; however, another way to figure out which one you want is just to try both on some data where you think you know the "correct" answer, and see which ones gives the most sense. Certainly that's what the author did of the only published paper I can find on the subject, which is called "Two multivariate generalizations of pointwise mutual information". He just tried them both and looked at which one worked better. Link below, though if you use Google Scholar on the title above you can find several versions.

https://dl.acm.org/citation.cfm?id=2043124