In what language-level in Chomsky's hierarchy would the chord-progressions of classical music be categorized?

Question: Formal languages
In language-theory and grammar-theory, a language is the output generated by a grammar.
Let's say "the grammar" is the rules and "the language" is the sentences and words that may be produced by those rules.
For example, we can say that this text "The some the although are is about over on under thin then over we the when." is not part of the English language although punctuation is correct and the words belong to English. That expression is not a sentence in the English language because cannot be generated with the grammar rules (ie: Sentence => subject + predicate, Subject => xxxx, Predicate => [complements] + verb + [complements]) etc.
Music theory
In terms of chord-progression, I know that "what sounds good" will depend on the "culture" you are, the style of music, what epoch/century is your analysis corpus is located and so on. It's not the same Flamenco, than Jazz, than Chinese popular music. Completely agree. Narrow this question down to western classical music.
So... there are chord sequences that would "violate classical rules". Hey, I already know that one can use whatever chord sequence one wants. But there are chord-sequences that are considered "grammatically correct" and others "not correct".
For this question, I don't care if "the rules existed centuries ago" or the rules are discovered now that we analyzed music from centuries ago. For this question, let's assume the rules just exist in classical music theory books.
Question
I would want to know if the "generative rules" of a "formal grammar" able to tell if "the next chord in a sequence" is "part of the language or not" (ie: Belongs to the set able to be generated by the grammar rules) belong to language level 0, 1, 2 or 3 in the Chomsky's categorization and WHY those rules are in that level.
For more concrete references:
References on chord-progression compiled rules
There's this old VST plugin "Harmony Improvisator" that was able to "generate" chord suggestions in function of the current chord. I don't know if the suggestion was also conditioned by the previous chords before the current one.

Official page here: http://www.synleor.com/improvisator.html
Video using version 1 of the plugin here:

Video using version 2 of the plugin here:

My interest in this question is only limited to a set of rules that might have been the one that the authors of "Harmony Improvisator" could have used.
They say in their page:

Composing with the rules of classical harmonic theory

and

learn functions and rules by heart in order to create classically correct, colorful music

Related questions in this site
There are other questions on this SE site that reference Noam Chomsky's grammar theory:
This one asking about generative music: What are the newest innovations in generative theories of music?
This other one comparing "learning music" with learning your "mother language": Why is music is taught by reading sheet music?
Chomsky's work
Chomky devoted his life to the theory of any grammar for any language invented or to invent. This encompasses the human-languages like English, French or Catalan, the computer languages like C++ or Java, or many other languages that are made of "sequences of things".

Formal grammars in general: https://en.wikipedia.org/wiki/Formal_grammar
Chomkys's work in particular: https://en.wikipedia.org/wiki/Chomsky_hierarchy

So Question
How can I tell what Chomsky's hierarchical level is the "classical rules of chord progressions" in?
Comment: The basic premise that music is a language , therefore it must obey certain rules is flawed. Take any three diatonic notes or chords, and they can go into 6 different orders, and still be acceptable. With words, you *do can't that*!
Comment: Since this is a question about the expressive capability of a formal language, I think you're going to have to specify your algorithm precisely before it can be answered. (NB when talking about "levels" here, you're referencing Chomsky's work in mathematical logic, not natural languages). It could well be that the VST you mention is using, e.g., a Markov model and isn't using a "generative grammar" in any useful sense.
Comment: @MichaelCurtis - for every one of those examples, there's dozens of examples that *do* work. *The red ball kicked I*. And your 3rd example cheated! *I was the kicked red ball*.
Comment: Cheated? _The ball I kicked: red._ I suppose you need to punctuate with a colon to use only the words of the original line. I would also say that re-orderings aren't all equivalent, they can have different implications. The point is there is some degree of flexible reordering in both language and music.
Comment: @Tim, _I kicked the red ball. The red ball I kicked... The ball I kicked was red._ You can re-arrange word order. And certain chord progression are not reversible. `iv6 Fr+6 V` cannot be randomly re-ordered.
Comment: Chord progressions are part of the semantics, not part of the syntax or lexical structure. They don't fit into the Chomsky hierarchy at all.
Comment: Does Chomsky allow the existence of a language that lacks any of { subject,  verb, object} ?   I fail to see how you'd map those into a chord progression.
Comment: Perhaps this assumes Chomsky's view of language is correct and wants to fit music into it.  It may not be, there are competing theories of language.
Comment: I must say I do find this to be a fascinating question but it may get voted out due to its nature.  Have you also posted on the Philosophy stack exchange?  Again, to ask "How can I tell what Chomsky's hierarchical level is the "classical rules of chord progressions" in?" assumes that Chomsky is correct (and me may not be).
Comment: Language. Flawed?  I am going to the store to buy two apples, too. Which 2 is witch ( er... which...).
Comment: I've stared at this question now for 20 minutes, and I really think it's rather unanswerable as written. It seems to confuse Chomskian generative grammar concepts (seemingly as related to natural languages, as some music theorists have tried to do) with Chomskian grammar hierarchy theory (which is mostly relevant to computational logic). And then it seems to restrict itself to asking what sort of algorithms some specific plugin may have used, which no answer here has ANY insight into, as we don't know the internals of this application. If it didn't have a bounty active, I'd vote to close.
Answer: First, there are not the rules of classical music. Different people have put forth different rules. I would also rather think of these rules as strong suggestions instead of invariable laws. Therefore, if you would use formal language theory to create a classical music parser, you will most likely fail to apply that parser to all but a tiny fraction of pieces.
Now, let's pick an example of rules, for example Percy Goetschius's Exercises in melody writing. Here, chords are grouped in different classes. For example, a chord of the Tonic class may progress into any other class whereas a chord of the Dominant class most likely progresses into the Tonic class. And so on. What you should see here is that every chord (i.e., the literals of our formal language) has some associated state (the class). This class is clearly finite and it is easy to construct a finite automaton that accepts these rules (if one would really want to do that, which I find rather questionable). This shows that this formal language of chord progression rules is regular (or Type 3).
Musical form aside (since it is not mentioned in the question), I cannot think of a reason why a formal language of chord progressions should not be regular. It is after all not that difficult to tell which chord progessions sound good. And, more important, you do not need knowledge of a possibly infinite number of decisions for chord progressions that you have done in the past. So you can encode all the information needed to decide which chord to use next into a finite number of states. If we want to conform to some musical form, however, things will be more complicated.
Last, I agree with the comment by helveticat that for use cases like the Harmony Improvisator, Markov Chains seem much more applicable. After all, the rules are often phrased with words like "tendency". And I would take the advertisement of the Harmony Improvisator with a grain of salt. :)
Answer: Not being familiar with Chomsky's work I cannot directly answer this but I would point out that in the case of Language the example sentence you wrote does not invoke any coherent ideas in the mind of the reader or listener.  I do not know if Chomsky's rules predict this and never lead to an ambiguity but it would seem that this mapping of words to ideas in the mind is a requirement for a language to be useful.  This is how I communicate my thoughts to others so that they can replicate what I am thinking based on the signals I provided them.  Their replica may not be a faithful representation of the original thought.
Some musicians do refer to music as "a language".  This serves many useful purposes one of which is to express to new students that it is more important to "play" and come to an understanding of why some ideas work rather than others than to learn a set of rules then try to play.  However, this is an analogy and not necessarily confirmation that music is in fact a legitimate language.  Even in western classical music these "rules" are really guidelines for best practices that have evolved over centuries (perhaps the rules of grammar are the same, a set of best practices that have evolved over millions of years).
A non standard chord order is not the same thing as a disallowed word order.  No chord order is capable of invoking a coherent thought like "Feral dogs are approaching and will attack us in less than 4 minutes".  The only thought I can think of is "Hey, that chord progression sounds like most others I've been told follow the "rules" of music", or "That is an interesting choice of chord order".  But in both cases a string of words in my language come to mind to describe whatever state of mind I find myself upon experiencing the chords.  The "chords" themselves do nothing to convey this or that idea.
Also, I would not necessarily say that chords are the words and their order dictated by grammar.  Chords, even in classical music are secondary to melodic theme.  That has more of a linguistic character if anything.  One can also have 2 part harmony, parallel or counterpoint, homophonic or polyphonic.  Why single out the chords as being the musical equivalent of words?
I realize that you have framed the question "in terms of chord-progression" and "western classical music" but this very constraint might be a huge red herring.
Answer: I like the analysis in the other answers here, but considering the concepts of key and modulation in a classical context I think chord progressions must be at most Type 2 (Context-Free Grammar) or lower. A Context-Free Grammar is more powerful than a Regular Grammar which can only encode sequences and repetitions since it can encode local context or bracketing (like the previous Key during a modulation section).
Classical works up into the Romantic period predominantly have a prescribed key or tonal center which determines a set of harmonies. Deviating from this set is either a small ornament (passing chord which would be grouped into a phrase with its surrounding chords) or an extended section which is a modulation to a new key. In the classical style, a modulation normally returns to the previous key so there is a sort of local context that is carried through the section.
I'm assuming that a modulation to a new key is always paired with a modulation back to original key later on, forming a phrase structure. Since these structures can also be nested (ie. A modulates to E which modulates to B before modulating back to E and then finally back to A), representing this structure in the grammar is beyond the power of a Regular Language.
Using the chord categories from Goetschius, here's a stab at such a grammar:

<tonic> := I | vi
<dominant> := V | V7 | V9 | vii° | vii°7 | iii
<subdominant> := ii | ii7 | IV | ii9 | IV7
<from-tonic> := <tonic>
              | <tonic> ( <from-tonic> | <from-dominant> | <from-subdominant> )
<from-dominant> := <dominant> <from-tonic>
<from-subdominant> := <subdominant> ( <from-dominant> | <from-tonic> )

So far, this is a Regular Grammar. I'm thinking about how to tackle the rest... (to be continued).
Comment: But detecting balanced parentheses is not regular only because there may be an infinite number of those parentheses, so you cannot construct a finite automaton that detects an arbitrary number of these nested levels (you would need an infinite number of states). This is, however, the core of my objection: There is no infinite number of possible modulations so sooner or later you _must_ return to the original key.
Comment: I think you should state in your answer that you assume that each modulation requires an inverse modulation (like blocks in a structured programming language). I assumed that you regard modulations more like a journey (you needn't travel back exactly the same way). Regarding the recursion: Let's assume we have a LISP-like SExpr grammar that we want to limit to a depth of 3. Then we can introduce nonterminals sexpr0, sexpr1, sexpr2, sexpr3 for each depth. Thus, we can eliminate the recursion if we have a bounded depth (my assumption). But I agree that dealing with motifs etc. changes everything
Comment: I'm not quite sure if I understand you correctly, but since there is only a finite number of possible modulations (and an even smaller number that "makes sense") shouldn't it be possible to just stick the automata for the different new keys together with transitions back to the original key? That would still be a finite automaton. Maybe I'm missing something, but I don't see anything distinctly context-free here.
Comment: I'm considering *modulation-away* and *modulation-back* as a bracketing structure, like [balanced parentheses](https://en.wikipedia.org/wiki/Context-free_grammar#Well-formed_parentheses). In a finite automata you can't carry along the memory of "original key" more than a finite number of steps. You could certainly construct finite automata for any finite list of examples, but it doesn't have the power to describe the balanced structure in general. Hence the common wisdom that regular expressions are the wrong tool for parsing html or any text with brackets or parens that must be balanced.
Comment: I think my point is more about the *nesting* than the number of possibilities per se. Suppose you have a piece that starts in A then modulates to E and then within that section modulates to B before going back to E and then back to A. The recursive nature of the phenomenon IMO requires a grammar with the power to describe that structure. ...But, I'm pretty persuaded by @MarkLutton's answer. Describing a theme or motif (and recognizing its recurrence) seems to require more memory than even a CFG has.
Comment: (re-reading this later with a fresher perspective) I think I see what you're getting at. You may be right. If we can fix the maximum depth to something like 4 or 5 then it can be flattened out into a regular grammar. And we probably can. 5 seems like a reasonable boundary to me after some speculation. I agree anything beyond 3 and the listener is probably going to forget the original key (3+ levels above) and just go with it. So a little cushion for extreme cases leads me to 5 as a reasonable hard bound for anything you're likely to analyze this way.
Answer: Well, when you get into fugues and symphonies and operas you are definitely beyond a context-free grammar. In a computer language like Java, the meaning of a + b depends on the declarations of a and b; i.e. integer addition or floating-point addition? Making sense of a Beethoven symphony or Wagner opera requires noticing the declaration of the motif, like the first four notes of Beethoven's Fifth or Siegfried's Horn Call, and noticing how it pops up later in changed form. This is getting into the semantics of music, but for me, syntax and semantics are tied together in music just as in literature.
Answer: To start, there is a pretty significant difference between classical harmonic theory and Classical harmonic theory. It sounds like you're asking the question with respect to Classical harmonic theory (the harmonic theory characteristic of European Classical music) but they capitalize "classical" in the screenshot you posted. You could take this any number of ways but I'm willing to guess that they mean classical music theory in the way that most people would simple describe as "music theory". Swing rhythms and the blues scale are both fair game in classical music theory but not so much in Classical music theory. "Classical" is one of the slipperiest words available for regular use.
I'm gonna push back, though, against the instinct to cry "music is a mystical force incompatible with the mundane machinework of language theory" and try to actually see if it might fit.  Like any other reasonable pedestrian, I'd never heard of the Chomsky hierarchy but wikipedia tells me that you need terminal, non-terminal, and start symbols. It seems like a pretty good place to start to figure out what our non-terminal elements would be in this model.  If you're just talking about the chord progressions and not actually the music, then we can ignore:

the fact that chords are made up of notes
amplitude (and thus sound, dynamics, etc)
time (and thus tempo, rythmn, etc)

It might sound like I'm sidestepping your question here but I think you're only asking about the chord progressions, which are a drastic layer of abstraction, and not the music itself. If I remember my Music History class properly, the composers  of classical music were guided by rules relating to harmony, which has already been abstracted away if our atomic unit here is the chord, so the language in question would not be the same as the ones the composers were "speaking".
It would, however, be recursively enumerable, as any chord progression is comprised solely of chords by definition and none of them, to my knowledge, are infinitely long.
I have to stop here because I'm beginning to realize the time commitment, but everybody feel free to tear this to shreds or build on it, as you see fit.
Comment: This is a good start. Can you add a little more?
Answer: My first guess would be that music can only be described by a Type 0 language. However, I couldn't think up a quick example. Also, it depends on what is included in the question; is only a chord progression needed or rhythm (which we can impose on chord sequences by requiring or allowing harmonic rhythm in addition to the nominal rhythm of the music.) And to reference Goetschius, with respect to melody, almost any irregularity (something rejected by the grammar) can be made regular by repetition. I would think that most irregular rhythms can be regularized by repetition. I think that (as irregularities may occur in an irregular type of repetition), description of melody or rhythm is at least unboundedly context-sensitive; it's not clear that chord progressions cannot be treated similarly. Irregular "atonal" or serial composition may not count as chord progression need not be part of their description.
Several attempts have been made to make the connection over the last few years. Most of these papers seem to go with Type 1 at least (context bounded is too restrictive).
https://royalsocietypublishing.org/doi/10.1098/rstb.2014.0097
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367686/
http://webprojects.eecs.qmul.ac.uk/marcusp/papers/RohrmeierPearce2018.pdf
https://puredata.info/groups/pd-graz/label/book/bang_10_juri_80-89.pdf
https://mtosmt.org/issues/mto.15.21.2/gjerdingen_bourne_structure.html
One example of problems in such a description is exhibited by the Folia progression: i-V-i-VII-III-VII-i-V-i where the first V-i isn't really a cadence (or need not be) but if the composition ends as here, the last V-i is. The cadential quality of of the VII-III can be modified (I tend to end the phrase on the first VII so the VII-III doesn't occur in cadential position.
Similarly, examples of a cycle of fifths in minor may have a minor v chord until the last couple of measures then switch to V-i. There's an arrangement of Greensleeves as a Romanesca in Bukofzer's book on Baroque Music which goes i-VII-i-v-i-VII-i-V-i-III-VII-i-v-III-VII-i-V-i. Others use an opening i-VII-i-V. The point is that the usual i-VII-i-V isn't necessarily a cadence but one needs a possibly unbounded context to figure this out.
Also contributing to the analysis problem (and why I like Type 0) is that certain features may be omitted. The "trio" section of marches (or polkas) is traditionally in the subdominant but the music often ends there. Is this an omitted I in I-IV-I overall structure or a I-IV or a V-I deceptively written as I-IV?
Answer: I think none.
That's because there's no way to encode rhythm in any of them, and it's absolutely crucial. You would need to do a lot of bookkeeping on the side (count beats) which would not "embed" nicely into your grammar.
It's similar to how you cannot parse C++ source without actually keeping track of semantics (available constructors vs functions).
Answer: Try using the Mingus module in Python (use <code>pip install mingus</code> to get it).
[Mingus is a package for Python used by programmers, musicians, composers and researchers to make and investigate music.]
You 'll get all you need from it (I believe). Thus, you needn't be worried about programming a language for music.
Comment: **Welcome to Music.SE, @Sam!** Thanks for contributing. This question isn't about programming; it's about possible similarities between harmony and linguistics. Hopefully everyone will be kind, but be prepared for quite a few down-votes...
Comment: Thanks for the warning... I haven't learnt of such a relation, but when I read through the question I felt it was mostly related to programming than anything else. No worries ! I have always been scoring downvotes for most of my answers (except on Quora) ☺ ..
Answer: Short proof that context-free is not enough:

suppose you have a song that simply repeats a verse V. That makes your language <code>V^n</code>, which is regular.
Let's say that the verse consists of three lines, so that makes it <code>(abc)^n</code> which is context-free.
But now in the final verse you repeat the last line 3 times (before the final chord, but let's ignore that), a very common device. Now you have <code>(abc)^n abccc</code> and that's no longer context-free because it does not satisfy a pumping lemma.


MySQL too many connections

Question: On my server I have 7 databases.  Our server has 512 MB of RAM which I'm getting upgraded this evening to 2GB and has a 2.4 single processor.
I've gotten an error about the connection limit exceeded.  With increasing my RAM, is it ok to increase the number of connections?  Currently it's set to 200 but a single page may connect to 3-4 databases considering JOINs and things.
We've setup so many databases for mere organization.  We have a total of about 250-300 tables in all of the databases.
Any advice would be appreciated :)
Update
Turns out, I have a lot more connections than I thought....
With 2GB RAM and a 2.4 single core, is 50 user connections accurate or should I increase that?
<code>[mysqld]
safe-show-database

skip-locking
safe-show-database

interactive_timeout=30
wait_timeout=30
connect_timeout=10
max_user_connections=50
max_connections=1000

key_buffer=64M
join_buffer=2M
record_buffer=2M
sort_buffer=4M
max_allowed_packet=24M

query_cache_limit = 2M
query_cache_size = 64M
query_cache_type = 1

tmp_table_size=128M
table_cache=2048
thread_cache=256
thread_concurrency=1
</code>
Comment: Number of tables or "schemas" which in mysql-language is a "database", has nothing to do with connections. One connection can send several queries, join as many tables as it like as well as change database/schema. You should consider one connection per simultaneous user, if you don't do multiple connectiosn per client (and in that case, look over your app/website to see if you can't optimize it)
Answer: You should be able to substantially increase your connections.  However, if you do not properly tune to the per thread memory usage, you risk MySQL running out of memory.  MySQL running out of memory can be a disastrous situation and is best to avoid.
An example of what happens when you do not properly tune:
MySQL keeps restarting due to running out of memory
Some earlier tuning recommendations I provided:
How do I set my.cnf in Mysql so that there are no limits to connections?
The link to InnoDB Memory Usage includes a formula to calculate InnoDB memory usage, which you can plug into a spreadsheet to calculate the memory usage on a per thread basis.
Here is some more information regarding MySQL Server Memory Usage.
Ultimately, your goal should be to calculate your maximum per thread memory usage based on your configuration.  Once that is calculated, you can set your maximum connections to the proper value.
Comment: You don't by chance have a spreadsheet you already use for this or something that would be helpful do you?  Even a web page to do this would be awesome
Comment: I do but it is arbitrary and something I hack out depending on what I'm doing at the time, which includes being more focused on InnoDB.  I considered giving it out but it's not really suitable for someone else to use.
Answer: On 1gb machines, we routinely set max_connections to 750 without issues
Comment: It's irresponsible to do that without the proper discretion.
Answer: If you use PHP, then edit the php configuration file, usually at /etc/php.ini and change persistence to OFF.
[MySQL]
; Allow or prevent persistent links.
mysql.allow_persistent = Off
That's all. Restart Apache.

MySQL too many connections


Adding data attribute to slim with no value

Question: I have the html  from https://foundation.zurb.com/sites/docs/v/5.5.3/components/topbar.html. But I'm not sure how this converts to slim.
nav class="top-bar" role="navigation" data-topbar
This causes problems because data-topbar has no value.
nav class="top-bar" role="navigation" data-topbar=''
This is valid slim but it adds the ='' which I don't want. Is there any way to add this attribute with no value in slim?
Comment: I'm un-sure why you need data attribute if you don't need a value.
Answer: If you write like this:
nav class="top-bar" role="navigation" data-topbar

You will get this:
data-topbar

If you write like this:
nav(class="top-bar" role="navigation" data-topbar)

When you inspect code you will see

But when you copy the dom or view source code you will get this one (like every boolean attributes)


how to get this functionality? views: Format options in ctool modal modify field data

Question: In views module, when we edit view page and add fields to it, there is an option Format:unformatted-list and when we click on unformatted-list, a popup appears displaying radio buttons of grid, table,jump-menu, etc. Choosing any of the radio button, the field adjust themselves in either grid, table, whatsoever the option we choose.
I need the exact functionality of choosing radio buttons, modify my fields display output.
Thanks in advance.
Answer: You may use Display Suite module

Display Suite allows you to take full control over how your content is displayed using a drag and drop interface. Arrange your nodes, views, comments, user data etc. the way you want without having to work your way through dozens of template files. A predefined list of layouts (D7 only) is available for even more drag and drop fun!
Comment: RE "I can't use any external module": You are already using an external module, since the Views module is not part of Drupal core until Drupal 8 comes out.
Comment: I am doing everything custom. I cant't use any external module. So far I used chaos modue api to modify form results in popup. But formating fields in various styles is different. Any approach which you can suggest?
Comment: I m replicating views module. I am not using Views, just trying to implement views like functionality in my custom module.


For discrete optimization problems, shouldn't there be a similarity metric over the label space?

Question: Let's say we want to map images of animals to categorical language label of what the animal is.
Let's say we have multiple sub-classes within the class of "dogs" e.g. poodles, terriers, bulldogs. Let's say we have a class "cat" e.g. Persian, Siamese, Maine Coone. The labels are always specific sub-classes.
The language label is a 1-hot vector from 1 ... N where N is the max number of sub-classes.
There are two ways we can setup this 1-hot vector:
(a) [poodles, terriers, bulldogs, Persian, Siamese, Maine Coone]
or
(b) [poodles, Persian, terriors, bulldogs, Siamese, Maine Coone]
Shouldn't (a) give better generalization than (b)? But most classification algorithms these days treat these two types the same. Basically, there is often a notion of "similarity" in the label space that I don't think is exploited.
Answer: Your remark is perfectly valid, but there are a couple of problems with your proposed solution.
There are in fact models where the classes are not assumed to be completely independent. For example in Zero-Shot Learning classes from training and evaluation might be different, but there is assumption that for each one of them a feature vector is provided. See Zero-Shot Learning - The Good, the Bad and the Ugly for details. An example method that tackles this problem is Embarrassingly simple zero-shot learning.
The problem with using similarity is that you actually need to specify it, because there are many different types of similarity - (note that ZSL does that implicitly, because you have cosine/Euclidean distance etc for vectors).
For example is a whale more similar to a salmon or a horse? (you care about being a member of common taxon or visual similarity, or where do they live?).
Another problem is that if you actually have a similarity in mind, how do you encode it? There will be problems with scalability, as for $n$ classes you will need to specify $n^2$ distances. Also adding a new class will require specifying similarities to all other classes, and not just adding a few new examples.


textarea html content updated to iframe (real time) using jQuery

Question: Is there a way I can transfer html contents entered into a textarea across to an iframe using jQuery?
I would like this to happen in real time.  i.e. when the user types into the textarea it is automatically updated to the iframe without the need to refresh the page?
I have looked for an answer to this but there seems to be no clear direction so a snippet of code and explanation would be great to help me understand?
Comment: why you need iframe?
Comment: The textarea and iframe are on the same domain?
Comment: I am looking to display html code entered into a text area so that it shows in the iframe as normal text e.g.  textarea shows hello world but iframe will only show hello world.  I have been told iframe is the best method for this
Answer: This seems to be a good walk-through of how to do what you want.
Here is a copy paste of the content there. As a reference as to no rely solely on a link as an answer.
Accessing an iframe from your page:
<code>var ifrm = window.frameElement; // reference to iframe element container
var doc = ifrm.ownerDocument; // reference to container's document
var form = doc.forms[0]; // reference to first form in container document
</code>
jQuery Version:
<code>$("#myiframe").contents().find("#myForm");
</code>
Accessing a parent from your iframe:
<code>// reference to first form in parent document
var form = parent.document.forms[0];
// or, using form id instead
var form = parent.document.getElementById('myForm');
</code>
jQuery Version:
<code>$('#myForm', window.parent.document).html();
</code>
There is a caveat to this though XSS: Which basically means you can only do these things if the site domain and the iframe domain are the same.
There are workarounds for this though: If you use your web server to get the content of another domain and server it as your own then the browser will not complain about XSS.
To directly answer your question the jQuery below this should work from the parent page using the jQuery contents function (see last demo on page):
<code>$(function () {
    $("#myTextArea").on("change keyup paste", function () {
        $("#myiframe").contents().find("html").html($("#myHtmlDiv").val());
    });
});
</code>
I have updated the above jQuery example so it should be working now but if not here is a tested JavaScript approach seemed to do the trick:
<code>$(function () {
    $("#myTextArea").on("change keyup paste", function () {
        var doc = document.getElementById("myiframe").contentWindow.document;
        doc.open();
        doc.write($(this).val());
        doc.close();
    });
});
</code>
Comment: @Vickar Updated based on your recommendations :) Can be hard to test this stuff without a working environment.
Comment: your JS code is absolutely working,
but there were 2 changes to your jQuery code, I've posted an answer with the modifications.
Answer: Just 2 changes to @fjoe's solution, and the jQuery code worked a 100%.
<code>$(function () {
    $("#myTextArea").on("change keyup paste", function () {
        $("#myiframe").contents().find("html").html($(#myTextArea).val());
    });
});
</code>
change the parameters of

find("#myHtmlDiv") to find("html")
and, html(this).val()) to html($("#myHtmlDiv").val())


Use Implicit Function Theorem

Question: I have these two equations $u + e^u + z +e^z = x$ and $u + u^5 + z^3 = y$. I'm wanting to try to show that there exist solutions $z = f(x,y)$ and $u = g(x,y)$ for $(x,y)$ in a neighborhood of $(2,0)$ and that $f$ and $g$ are smooth.I'm trying to figure this out and, in turn, figure out the bottom part as well. What i've tried so far is taking $$F_1(u,x,y,z)=u+e^u +z+e^z-x$$ $$F_2(u,x,y,z)=u+u^5+z^3-y$$
I've tried taking the determinate $$det=
        \begin{matrix}
        \frac {\partial F_1}{\partial u} & \frac {\partial F_1}{\partial z}\\
        \frac {\partial F_2}{\partial u} & \frac {\partial F_2}{\partial z} \\
        \end{matrix}
$$
however, I believe i'm approaching this the wrong way because I get$$
        \begin{matrix}
        1+ ue^u & 1+ze^z \\
        1+5u^4& 3z \\
        \end{matrix}
$$
I don't know where to plug in any other information and it seems I've run into a dead end.
I need to know this in order to do the example problems like:
Find $f(2,0)$ , $\frac {\partial f}{\partial x} (2,0)$ , $\frac {\partial f}{\partial y} (2,0)$ , $\frac {\partial^2 f}{\partial x^2} (2,0)$ , $\frac {\partial^2 f}{\partial y^2} (2,0)$ $\frac {\partial^2 f}{\partial x^2y} (2,0)$ Am I crazy or have i just screwed up big time somewhere?!?
Comment: Does the question ask you to use the implicit function theorem to show the existence of $f$ and $g$, or are you allowed to assume that $f$ and $g$ exist and then answer the questions at the bottom?
Comment: The reason for my above comment: either the question is not well-posed, or you have withheld some information, or there is some error in your first line.  The issue is that if $x=2$ and $y=0$, you cannot solve your first equations uniquely for $u$ and $z$, and so there could be multiple answers for $f(2,0)$ and the derivatives of $f$, depending on which $u$ and $z$ you choose.  Does this make sense?
Comment: @michaelharrison it does make sense. For the question itself, it doesn't explicitly say to use the Implicit Function Theorem. I just assumed it was the route to take. I also assumed that f and g existed o try to get the below results
Comment: Okay, I see.  Let's say instead you have the equation for the parabola $y=x^2$ and I ask you to find a function $x=f(y)$ which is defined in a neighborhood of $y=1$.  The problem is that there are two points on the parabola which have $y=1$: specifically $(-1,1)$ and $(1,1)$.  In a neighborhood of *each* of these points, you can find a function $f$.  If you choose the point $(1,1)$, $f(y)=\sqrt{y}$.  If you choose $(-1,1)$, $f(y)=-\sqrt{y}$.  But you cannot find a single function $f$ which works for both points.  So asking "what is $f(1)$", doesn't make sense; it depends which you chose.
Comment: Now in your question: in place of my $y$ you have $x$ and $y$, and in place of my $x$ you have $u$ and $z$, but still there is the same issue: there is not a unique point $(u,z)$ corresponding to $(x,y) = (2,0)$.
Answer: As mentioned in the comments, your question needs a little more info, because there are multiple points $(u,z)$ for which $x=2$, $y=0$.  However, notice that $(u,z) = (0,0)$ works, and I think that this is a natural point to choose.  So let's assume that the point $u=0$, $z=0$, $x=2$, $y=0$ was given in the problem instead of just $x$ and $y$.
Now if you weren't asked to verify that the implicit function theorem holds, your problem is much easier.  Let's rewrite your first equations, assuming that $u = u(x,y)$, $z=z(x,y)$ are functions of $x$ and $y$:
\begin{align*}
u(x,y) + e^{u(x,y)} + z(x,y) + e^{z(x,y)} & = x \\
u(x,y) + u(x,y)^5 + z(x,y)^3 & = y.
\end{align*}
(You don't always need to write all the dependencies on $(x,y)$, but it's nice to make sure you know exactly which variables are functions of which other variables.)
Now we can implicitly differentiate both of these with respect to $x$:
\begin{align*}
\frac{\partial u}{\partial x}(x,y) + e^{u(x,y)}\frac{\partial u}{\partial x}(x,y) +
\frac{\partial z}{\partial x}(x,y) + e^{z(x,y)}\frac{\partial z}{\partial x}(x,y) & = 1 \\
\frac{\partial u}{\partial x}(x,y) + 5u(x,y)^4\frac{\partial u}{\partial x}(x,y) + 3z(x,y)^2\frac{\partial z}{\partial x}(x,y) & = 0.
\end{align*}
Now plugging in $x=2$, $y=0$, $u(2,0)=0$, $z(2,0)=0$ gives:
\begin{align*}
2\frac{\partial u}{\partial x}(2,0) + 2\frac{\partial z}{\partial x}(2,0) & = 1 \\
\frac{\partial u}{\partial x}(2,0) & = 0,
\end{align*}
so $\frac{\partial z}{\partial x}(2,0) = \frac{\partial f}{\partial x}(2,0) = \frac12$.
You can do the higher derivatives similarly, though it looks like it will be quite messy.

By the way, since you ask about implicit function theorem, and you were close to showing that it holds, let's finish that off.  You compute the matrix of partial derivatives as you did above (but I've fixed some of yours which were incorrect):
$$\left( \begin{array}{cc} 1 + e^u & 1 + e^z \\ 1+5u^4 & 3z^2 \end{array} \right).$$
The determinant of this matrix is $3z^2(1+e^u) - (1+e^z)(1+5u^4)$.  Now we can plug in $u=0$, $z=0$, and notice that the derivative is nonzero.  By the implicit function theorem, this guarantees the existence of the functions $z=f(x,y)$, $u=g(x,y)$ which we assumed above.

Use Implicit Function Theorem


Removal of SQLite database file using Java doesn't work

Question: These code blocks work but...
If I use this blocks one by one my application doesn't remove database file. (result is equeal to false)
sqlite manage part:
    // sqlite manage

        Class.forName("org.sqlite.JDBC").newInstance();
            conn = DriverManager.getConnection("jdbc:sqlite:/"+ myDBpath);
            stmt = conn.createStatement();
    //some calculations
    stmt.close();

removal part:
    //remove this file
    boolean result = new File(myDBpath).delete();

But if I use only remove code without database operations it works! Why? How can I avoid it?
Comment: You also need to close the connection:  `conn.close();`
Comment: @Andie2302, you are right. already found by the answer below. thanks.
Answer: Because when you are using the database connection then your sql file is in use and hence you cannot delete it.
And when you don't use the database operation then the sql file is not in use and hence it can be deleted easily. So delete the file you need to make sure that there is no open connection with your database and once all the connections are closed you can delete the file.
change
stmt.close();

into
stmt.close();
conn.close();

Comment: How can I remove this connection?
Comment: @Vyacheslav:- As you are doing it. You have to close the connection and then delete the file.


ghost-cursor Error : elem.remoteObject is not a function in puppeteer

Question: whenever I run the script in the Local system the cursor properly worked and when I run inside docker that time I fetch an Error so anybody guid me what is wrong or whether this issue is package side or not.?
when I run in Locally that time I headless: false and Run inside docker that time headless: true but I tried inside docker headless: false but it can't work
here is code
const puppeteer = require('puppeteer');
const {path,createCursor} = require("ghost-cursor");
const url = process.argv[2];

async function run() {
    browser = await puppeteer.launch({
        args: ['--disable-gpu', '--no-first-run', '--no-sandbox', '--no-zygote'],
        "dumpio": true,
        "devtools": false,
        "ignoreHTTPSErrors": true,
    });

    const page = await browser.newPage();
    const cursor = createCursor(page);
    await cursor.move(selector)
    await page.goto(url);
    await page.screenshot({ path: 'screenshot.png' });
    browser.close();
}
run();

Here is Error Message

    TypeError: elem.remoteObject is not a function
    at Object.<anonymous> (/usr/app/node_modules/ghost-cursor/lib/spoof.js:458:61)
    at step (/usr/app/node_modules/ghost-cursor/lib/spoof.js:44:23)
    at Object.next (/usr/app/node_modules/ghost-cursor/lib/spoof.js:25:53)
    at fulfilled (/usr/app/node_modules/ghost-cursor/lib/spoof.js:16:58)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)

Answer: selector is undefined
You need to grab an element from the UI to move to like so
import { createCursor } from "ghost-cursor"
import puppeteer from "puppeteer"

const run = async (url) => {
  const selector = "#sign-up button"
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage()
  const cursor = createCursor(page)
  await page.goto(url)
  await page.waitForSelector(selector)
  await cursor.click(selector)
}

Answer: This must be happening due to version discrepancies with the way remoteObject can be accessed. The temporary solution I arrived at was to modify the source code.
Go to /node_modules/ghost-cursor/lib/spoof.js and change the references from .remoteObject().objectId to ._remoteObject.objectId.
Obviously modifying code from your deps is a bad idea, but I couldn't find the right solution anywhere.


Include original data with current data from temporal tables in a view?

Question: I am creating a view based on a system-versioned table that records account information. The primary key is an automatically-incremented "Account No". My period columns are "Valid From" and "Valid To". I also assign a new "Version No" each time the data changes.
Within this view, I would like to use the "Valid From" value from the most recent version of each account to represent the account's "Time Modified." Then, I would like to use the "Valid From" value from the oldest version of each account (i.e., where "Version No" = 1) to represent the account's "Time Created."
How should I do this?
Answer: While it is possible to use FOR SYSTEM_TIME ALL and aggregation, it's probably easiest (and fastest) to just join the history table directly.
SELECT
  a.*,
  ISNULL(ah.ValidFrom, a.ValidFrom) AS TimeCreated
FROM Account a
LEFT JOIN (
    SELECT ah.*,
      rn = ROW_NUMBER() OVER (PARTITION BY ah.AccountNo ORDER BY ah.ValidFrom)
    FROM Account_History ah
) ah ON ah.AccountNo = a.AccountNo AND ah.rn = 1;

You can also do this as an APPLY or scalar subquery. This may or may not be faster depending on the cardinalities involved. You should try both options.
SELECT
  a.*,
  ISNULL(ah.ValidFrom, a.ValidFrom) AS TimeCreated
FROM Account a
OUTER APPLY (
    SELECT TOP (1) ah.*
    FROM Account_History ah
    WHERE ah.AccountNo = a.AccountNo
    ORDER BY ah.ValidFrom
) ah;

Comment: Thank you! I ran both queries in a batch with the execution plan turned on, and it appears that the query cost of the left join method was only 4% of the batch. I also separated the queries and included client statistics for several trials. It appears the total execution time of the left join method also averages to a faster time. I'm fairly new to testing performance though. Are there other things I should be checking?
Comment: The "query cost" is only an estimate, I wouldn't rely on it. Look at actual elapsed time, as well as CPU time and IO usage, using the Show Server Statistics option


Can an offbeat note be loudest when playing Bach?

Question: My primitive understanding of Bach's music tells me that an offbeat note cannot be the most accented or the most important note in a phrase. But in the last gesture of the ubiquitous C major prelude, that's exactly what seems to happen.

What are the possible ways of thinking about this? Am I simply way off and the dynamics of a note has nothing to do with its count? Or is this phrase meant to be an unexpected exception meant to be a surprising change in rhythm? Or is everyone just playing it wrong?
Comment: Bachs music is eternal and it is not my understanding that it is written just for his own time. You are free to make experiments and accents on any note you want. Have you ever tried to emphasize the 3rd and 6th note in the r.h..?
Answer: Well, Bach has many exciting rhythm patterns in his music. The accents certainly don't have to be on the first beat, but you will normally still feel the first beat as the most heavy part of the bar. What I mean is that the stress on the first beat is often felt even it is not played as the loudest note.
Often the stress on syncopated notes works because you kind of feel the normal stress on the beats and then play the syncopation against that so to speak.
Anyway in this piece the highest note, the f, will stand out because it is the highest note. You don't need to accentuate it. You could say that the accentuation is build into the music. The way it is written creates excitement and makes the ending of the piece really great.
By the way, the piece is written for harpsichord where you can not play an accent on the note, but as I said it stands out anyway because of the way it is written. When you play it on a modern piano the note will also stand out for the same reason.
Since it is possible to play an accent and/or make a crescendo and a diminuende on a modern piano that might very well be done. But you don't need to do that in order to feel the f somewhat accentuated as I indicated above.
Comment: "The accents don't have to be on the first beat". You seem to suggest off-beat, dynamic accents would be consistent with Bach's style. Can you give an example? I couldn't come up with any.
Comment: @Aaron Bach's double violin concerto 1st movement second bar, where the second beat is syncopated. Later on in the same movement when the solo starts, the theme there has some 10th jumps, four eight notes, where the second and the fourth eight notes are a 10th up from the former note, they stand out even if you play them softer than the lower notes. Also syncopation happens several places in this piece in all 3 movements. Then there is his Italian Concerto for piano where there are many syncopes especially in the first and second movement.
Comment: Good examples. Keep in mind thought, those aren't "dynamic accents" and wouldn't be played that way. They are agogic and tonic accents, which sound emphasized (even when played softer, as you say), but are not accents in the way the OP is asking. I suggest changing the wording in your answer to avoid confusion.
Comment: I meant that those upper notes, off beat notes, don't need to be accentuated. But you can certainly accentuate the syncopation that appears many places in the music. Bach didn't write dynamics and articulations in this piece (the double concerto), but that doesn't mean you shouldn't play with life and energy and you can definitely play with articulation. Here is a version played on periode instruments: https://www.youtube.com/watch?app=desktop&v=bVzN8O54rew
Answer: None of the recordings of this Bach prelude I recall accent the notes with arrows louder than their respective notes one 16th note before. Every time I imagine a rendition of this piece, only the on-beat notes have the mild accents. I'm in the camp that those recordings I've never listened to are playing it wrong.
Answer: Baroque music is inherently metric and has phrasing accents that are retained when shifting a theme by non-beat timings.  A number of courtly dance forms are structured by their inherent meter and the omnipresence of the dance steps makes it possible to integrate voices and phrases against the beat.  Indeed, dances like the Courante inherently have "a limp".
It's similar to poetry where good poetry by and large follows an inherent meter but a good reciter will not bend the language flow and accents and pauses to the meter and rhymes in the rare cases where they don't bow to the rigidity of a scheme.
Now all that being said and done, the points you indicate are pickup phrases, anacruses.  They are the most important point of their phrase: the preceding on-beat note ends a phrase on the beat and is followed by the tiniest of a caesura (you'd change bow direction, string, position, take a breath or whatever else there) and the on-beat note after the anacrusis is the culmination of the phrase.
The notes you point out are important as the starting note of the phrase but you would not rest on them even when playing them the most distinctive of the off-beat note group: the rest note with a caesura after it (if at all) is the preceding note.
Answer: It would be strange -- or at least a rare exception -- to make an offbeat note the loudest within a phrase in Bach. For certain, it would be stylistically anachronistic in the C Major Prelude to make the indicated notes the loudest.
The key to the question is that "that's what seems to happen" (emphasis mine). Rather than making the note loudest in volume, which is called dynamic accent, it appears to be loudest because it is the highest.
At equal volumes, we tend to perceive high notes as being emphasized relative to the surrounding lower ones. This is called a tonic accent. Some performers in this piece also linger slightly on those top-most pitches. Longer notes also tend to feel emphasized, which is called an agogic accent.
Answer: Bach has written a flowing phrase in which the highest note doesn't come on a strong beat. I suppose we could describe it as a slight syncopation.
Bach is sometimes accused of writing 'sewing machine music'.  But his  strings of semiquavers - maybe interminable at first glance - are rarely predictably four-square, he rejoices in this sort of asymmetrical phrasing which I've re-written in an exaggerated fashion below.


How to create sql partitions for multiple columns?

Question: I try to create partitions for a table. I want to create partitions for 2 different columns. Is it possible?
So this works:
  ALTER TABLE MY)TABLE MODIFY
    PARTITION BY RANGE(SETTLEMENT_DATE) INTERVAL(NUMTODSINTERVAL(1,'day'))
( partition MY_PARTITION values less than (to_date('2019-06-01', 'yyyy-mm-dd')));

This is what I would like to do, but it does not work:
  ALTER TABLE MY_TABLE MODIFY
    PARTITION BY RANGE(FILE_SUBMISSION_DATE_TIME) INTERVAL(NUMTODSINTERVAL(1,'day'))
( partition MY_PARTITION values less than (to_date('2016-06-01', 'yyyy-mm-dd'))),
    PARTITION BY RANGE(FILE_ACK_DATE_TIME) INTERVAL(NUMTODSINTERVAL(1,'day'))
( partition MY_PARTITION_2 values less than (to_date('2016-06-01', 'yyyy-mm-dd')));

Comment: Are you sure you need partitions at all? Indexes are what you use to "help find the data by date"
Comment: What you partition is the table, not columns. You might be able to do what you want using subpartitions (search for "oracle composite partitioning").
Comment: @Mat composite partitioning is not really what I need. I need to create partitions that would help me find the data by date. I have 2 columns containing date data and they do not really have a relationship that would allow me to use composite partitions.
Comment: @Mat The table will contain a lot of data. I was using indexes but was told to try partitions if possible. I did it for some tables, but as you see in this example some selects will be done by FILE_SUBMISSION_DATE_TIME and some by FILE_ACK_DATE_TIME and I was wondering if partitions could be applied here.
Comment: You don't have to have a relationship between columns to do composite partitioning. In fact, what you are looking for is what composite partitioning does. Thing is you cannot subpartition just one partition.
Comment: @gsalem Lets suppose I do partitions on FILE_SUBMISSION_DATE_TIME and subpartitions based on the FILE_ACK_DATE_TIME. How will this affect a select that is done only on FILE_ACK_DATE_TIME? Will there be any improvements in performance and would it make sense. I guess not, but please correct me if I am wrong.
Comment: I guess what I want to do is not possible. It would be like attributing 1 row to 2 partitions...
Answer: You can partition on two columns in two ways:

Multi-column partitioning
Composite partitioning (one partition column, one subpartition column)

Multi-column partitioning
This uses the first column to choose which partition to place a row in. The second column is only evaluated when there is a tie. This can lead to rows being in a partition you didn't expect:
create table t (
  c1 int, c2 int, c3 int
) partition by range ( c1, c2 ) (
  partition p0 values less than ( 1, 1 ),
  partition p1 values less than ( 2, 2 ),
  partition p2 values less than ( 3, 3 )
);

insert into t values ( 0, 0,  1 );
insert into t values ( 0, 99, 2 );
insert into t values ( 1, 1,  3 );

select * from t partition ( p0 );

C1    C2     C3
    0     0     1
    0    99     2

Filtering on the second column in the list will minimal partition pruning:
alter session set statistics_level = all;
set serveroutput off

select * from t
where  c2 = 0;

select *
from   table(dbms_xplan.display_cursor(null, null, 'ROWSTATS LAST +PARTITION'));

----------------------------------------------------------------------------------------
| Id  | Operation                    | Name | Starts | E-Rows | Pstart| Pstop | A-Rows |
----------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |      |      1 |        |       |       |      1 |
|   1 |  PARTITION RANGE MULTI-COLUMN|      |      1 |      1 |KEY(MC)|KEY(MC)|      1 |
|*  2 |   TABLE ACCESS FULL          | T    |      3 |      1 |KEY(MC)|KEY(MC)|      1 |
----------------------------------------------------------------------------------------

(note the Starts = 3 for TABLE ACCESS FULL; this means it read all the partitions)
Composite partitioning
This is "two-level" partitioning. The database first splits the rows on the top-level partition. Then further subdivides on the second level.
A key difference between this and multi-column is you can have different partition methods for each. e.g. range-hash, hash-list, etc.
create table t (
  c1 int, c2 int, c3 int
) partition by range ( c1 )
  subpartition by range ( c2 )
  subpartition template (
    subpartition s0 values less than ( 1 ),
    subpartition s100 values less than ( 101 )
  ) (
  partition p0 values less than ( 1 ),
  partition p1 values less than ( 2 ),
  partition p2 values less than ( 3 )
);

insert into t values ( 0, 0,  1 );
insert into t values ( 0, 99, 2 );
insert into t values ( 1, 1,  3 );

select * from t partition ( p0 );

C1     C2     C3
    0     0     1
    0    99     2

select * from t subpartition ( p0_s0 );

C1     C2     C3
    0     0     1

Again, filtering on the subpartition column does minimal partition pruning. The database will first scan all top-level partitions. Then only prune the subpartitions within these:
select * from t
where  c2 = 0;

select *
from   table(dbms_xplan.display_cursor(null, null, 'ROWSTATS LAST +PARTITION'));

-----------------------------------------------------------------------------------
| Id  | Operation               | Name | Starts | E-Rows | Pstart| Pstop | A-Rows |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT        |      |      1 |        |       |       |      1 |
|   1 |  PARTITION RANGE ALL    |      |      1 |      1 |     1 |     3 |      1 |
|   2 |   PARTITION RANGE SINGLE|      |      3 |      1 |     1 |     1 |      1 |
|*  3 |    TABLE ACCESS FULL    | T    |      3 |      1 |       |       |      1 |
-----------------------------------------------------------------------------------

Notice the PARTITION RANGE SINGLE operation - this means the database only accessed one of the subpartitions.
Depending on your partitioning scheme and value distribution, this could lead to a decent reduction in rows read. You'll need to try out various partition methods and boundaries to see if this helps you.
Comment: Unfortunately, it does not work in the case of interval partitions. I found the answer here:
https://docs.oracle.com/cd/E24693_01/server.11203/e16541/part_admin001.htm
Comment: Multi-column doesn't, composite does. That link has examples of interval-range, -list, and -hash tables!
Comment: Dinu,
You can create interval-range, but not interval/interval. It also depends on the version you're using, which you did not specify (is it 11, as your doc pointer is for that version)?
Answer: What I was trying to do is impossible. It is possible to make range partitions using multicolumn partitioning keys, but not if you create interval partitions. More info here.


NoReverseMatch at Django

Question: and I do not know why and what will be the problem, could they help me? Thanks beforehand, greetings!
Reverse for 'entregado' with arguments '()' and keyword arguments '{'cod_experto': 'ASE-0048', 'id_pedido': 1770}' not found. 1 pattern(s)      tried: ['solicitar/confirmar/(?P<id_pedido>\\d+)/(?P<cod_experto>\\d+)/$']

Error during template rendering admindata.html, error at line 74:
Entregar

url global:
urlpatterns = [
# Examples:
url(r'^solicitar/', include(urls, namespace="usuario")),
]

url APP:
urlpatterns = [

url(r'^confirmar/(?P<id_pedido>\d+)/(?P<cod_experto>\d+)/$', login_required(Update_stock), name='entregado'),
]

and views.py:
def Update_stock(request, id_pedido, cod_experto):
if request.method == 'GET':
   pedido = Pedido.objects.get(id=id_pedido)
   articulo = Articulo.objects.get(id=cod_experto)
   articulo.stock -= pedido.cantidad
   stock.save()
return render(request, 'admindata.html', {'pedido':pedido, 'articulo':articulo})

Comment: You'll need to post the `import` statements for the global `urls`. Can't see anything wrong in what you've already posted.
Comment: The global url is not the problem since they work perfectly with others I have, except that I throw that mistake
Answer: Your cod_experto value is "ASE-0048", which does not match the regex \d+ - that only matches integers.
If you want to be able to accept values like that, you need to change your regex:
r'^confirmar/(?P<id_pedido>\d+)/(?P<cod_experto>[\w-]+)/$',

Comment: Thank you, sir. You're a genius.

NoReverseMatch at Django


Strategies for protecting third party PHP web app config files

Question: I'm interested in comments and strategies for protecting PHP config files, particularly those with Db credentials.
The stock advice, set the file extension to *.php, is fine as far as it goes, but it does not protect against directory perusal attacks. The particular scenario I'm considering:

We have a server that hosts a variety of third party apps.
We cannot thoroughly vet all apps
If an attacker compromises an app, they can potentially use that app to read files from other directories. Db config files are an obvious target.

I have a solution, one that seems to have merit, but I'm not entirely comfortable with it. I'd love to hear comments on it, as well as

find what php_include is set to
place the config file in that directory
make sure the require/include statements that load the config do not specify hard path (no leading ./ etc.)

GOALS

hides the config file completely from an errant sister web app (and more importantly from the webserver itself).
obscures the actual location of the config file. It cannot easily be determined from reading the code. This is opposed to specifying a path outside of the web tree.
minimal intervention, both on a server level and on an app level.
Avoids hard coding the path.

This does not address config file name conflicts, although more then a few apps it's probably worth changing the include config file name.
Unfortunately a lot of our web apps appear to require hard coded paths.
What approaches have others come up with?
Answer: Standard advice applies:

Place config file above the web root
Use .htacess or similar to deny access to these folders
Use file permissions to protect folders

Even better advice:

Ensure your DB server can only be accessed by your webserver by both network/firewall ACLs and by it's own host based authentication
Don't use passwords, use certificates/Kerberos/PKI (a brief google has this suggestion for implementing this in MySQL)
If you don't trust an app, run it on it's own webserver that doesn't have access to other apps files
Use a tripwire for intrusion detection

Hiding config files as you suggest seems futile, once an attacker has control of your system in this way, it's game over anyway.  You are just making your own job harder of maintaining your webs apps harder with little security gain.
Comment: Thank for the PKI suggestion. I was unaware of this possibility.
Comment: I appreciate the response. That said, I do think there is some point to "hiding the config file". Essentially I'm cutting one class of attack (compromised sister web app that can be used to read FS).

What I'm attempting to do is pretty much bullet point 1, "Place config file above the web root". The point of loading it in the default include directory is to avoid changing the code of the third party web apps. Unfortunately most of the web apps go through some pains to keep you from moving the config file.


Reference category in multinominal logistic regression

Question: I was wondering how you get beta values for the final model if you don't
have a reference condition?
I am running a multinominal logistic regression using a dependent variable
that has 6 levels - there is no condition that was a control or justifies
separation from the others, so I don't want a reference category.
In the output the only table that will tell me what my beta values are is
the one in which you state a reference category. Is there another way that
I can find out the beta values so that I can report the regression
equation?
I hope that makes sense! I would really appreciate your help on this.
Comment: You _must_ set any one category to be reference. It is mathematical, not substantive, prerequisite.
Comment: Expanding on @ttnphns comment - think of it this way: The beta values are comparisons. To make a comparison, you have to compare TO something. E.g. you can't ask "Are men taller?" you have to ask (e.g.) "Are men taller than women?"

You can, of course, get the mean height for men and women, but that's before the regression.
Comment: @ttnphns Not quite (see below).
Comment: @cassie-hazell Did the answer help?
Answer: Don't mistake the statistical identification requirement that a reference category fulfils with the analysis of an experiment that has a control condition.
Consider first this identification requirement: Because the probabilities of $K=6$ categories have to add up to 1 for every observation, there must be a linear constraint added somewhere. For the regression coefficients $\beta_1 \ldots \beta_K$, we can state this constraint as $\sum^K_i a_i \beta_i = 0$ for some fixed $a$, not all zero.  One rather easy way to fulfil this is to set $a_K=1$ and the rest to 0.  That effectively sets $\beta_K$ to 0, and makes $K$ the reference category.  But you can probably see other ways to fulfil the constraint.  For example, we could force the $\beta$s to sum to zero, ANOVA-style.  Although these identification options alter the numerical values of the $\beta$s and some choices make estimation easier than others, they don't affect the predicted probabilities that the model produces, so in a fairly strong sense they don't matter and aren't part of the model's assumptions.
Because of this, even when you have a control condition to compare other conditions to and you have decided to pick a reference category then it's at best a reporting convenience, not some deep mathematical requirement, that you make the reference category the one that represents the control condition.  You can model experiments just as well if you don't line these things up.
It also implies that you can duck the entire question of how your parameters were coded by reporting predictive probabilities under various combinations of explanatory variables.  Then it no longer matters whether there is a natural baseline condition because you don't have to spend your time interpreting parameters at all (although of course you still want to report them).  A good paper on this approach is Fox and Anderson.


Getting the logged in username and email in a view

Question: Im trying to get the logged in users username and email adress to display in a view is that possible?
Answer: Yes. Just create a view with base table of (user). Then add your fields of username and email. Then add a contextual fileter of uid and set a default value for logged in user.
Comment: But if I have a view that is content based(as base table) and I need the username to construct a custom URL. I thought this would be very easy but it gets more complicating all the time.
Comment: Sorry I am confused. Are you saying you have another view and you also want to attach the username a d email of the current user to it?  If so then you are going to have to create two different views and attach one to the other. You are going to have to provide more info about your setup instead of leaving us to guess based on the limited info you provided. Based in your question I took it as you just needed a single view that shows the current users info.
Comment: Sorry if I was unclear with the question. I have a content type that is a "organic group" group content.
Comment: Sorry if I was unclear with the question. I have a content type that is a "organic group" group content. called "post"

That content type is attached to an organic group. called team.

Than i want to create a view that lists all posts that belong to a organic group. Until now everything is fine.

Now i need the logged in users username and create a custom url like http://hipphopp.com/q="the logged in users name". But I cant get the logged in users username.... That field is not available in the field.
Comment: You won't find that field, because you are using (content) as your base table. These will need to be separate views.
Comment: Thank you for your answer, do you have any suggestions how to manage to make an custom url in drupal where you combine an content from a node with the users information. I think ill start to investigate if its possible to do it in template.php as a hook function. Thank you again Nigel!
Comment: Use a contextual filter in your view then set it to use the uid from the URL then add a menu item in the view such as user/%1/your-url


How to access query result with the selection as object name

Question: I have made a query on a data and it returns a result:
[ RowDataPacket { 'COUNT(t.id)': 2 } ]
How can I access it like the count number?
I have tried result.COUNT(t.id) but it isn't worked.
Answer: The way the object is named, you need to access it like so:
result[0]["COUNT(t.id)"]

I would rather name the columns explicitly by using aliases in the query:
select count(t.id) as count_of_id
from ...

Comment: The first method works well. Thank you. However, it outputs undefined if I rename the name as the second method. (I calling as result[0].count_of_id).
Comment: It should work. Log the result and check, may be there is a typo or upper/lower case issue.


installing PyQt5 in Windows 7

Question: I wrote a program in Python 3.4 and I want to make a GUI for it. I"ve found that PyQt5 - is the tool for it.
1) I've downloaded and installed a binary package of PyQt5 (http://sourceforge.net/projects/pyqt/files/PyQt5/PyQt-5.4.1/PyQt5-5.4.1-gpl-Py3.4-Qt5.4.1-x32.exe).
2) I tried to run this example code in Python:
import sys
from PyQt5.QtWidgets import QApplication, QWidget

if __name__ == '__main__':
    app = QApplication(sys.argv)
    w = QWidget()
    w.resize(250, 150)
    w.move(300, 300)
    w.setWindowTitle('Simple')
    w.show()

    sys.exit(app.exec_())

Python returned en error:
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\PyQt5.py", line 2, in <module>
    from PyQt5.QtWidgets import QApplication, QWidget
  File "D:\PyQt5.py", line 2, in <module>
    from PyQt5.QtWidgets import QApplication, QWidget
ImportError: No module named 'PyQt5.QtWidgets'; 'PyQt5' is not a package

3) I found an advise to install QT. So, I downloaded and installed QT (http://download.qt.io/official_releases/online_installers/qt-opensource-windows-x86-online.exe).
4) Then I've uninstalled and installed again a binary package of PyQt5.
No results.
Python doesn't return any errors if I run:
import PyQt5

But if I try to run:
from PyQt5.QtWidgets import QApplication, QWidget

It returns the same error as in the beginning.
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\PyQt5.py", line 2, in <module>
    from PyQt5.QtWidgets import QApplication, QWidget
  File "D:\PyQt5.py", line 2, in <module>
    from PyQt5.QtWidgets import QApplication, QWidget
ImportError: No module named 'PyQt5.QtWidgets'; 'PyQt5' is not a package

What am I doing wrong?
Comment: does your code include `import sys`?
Comment: Yes, it does include "import sys". But it does not change the situation (still gives the same error), so I didn't mention it.
Comment: Pynchia, I edited code in question. thanks, but this doesn't help. the problem is in importing QtWidgets or some other part of PyQt or Qt. And not in the last line of code.
Comment: ok, then it import PyQt5, but it complains it is not a package. Locate it and check it is indeed a package (i.e. its directory contains the file `__init__.py` )
Comment: if I understood you correctly - yes. ther is a directory "c:\Python34\Lib\site-packages\PyQt5\" and ther is a file __init__.py in it. (with underlines)
Comment: but this file (__init__.py) has no code inside: # Copyright (c) 2015 Riverbank Computing Limited
#
# This file is part of PyQt5.
#
# This file may be used under the terms of the GNU General Public License
# version 3.0 as published by the Free Software Foundation and appearing in
# the file LICENSE included in the packaging of this file.  Please review the
# following information to ensure the GNU General Public License
......
Comment: yes, it does not need to contain anything. But it makes PyQt5 a package. It's a rather odd behaviour then. If I were you I'd start the installation over from scratch.
Comment: Do not call your program file the same as a package name. You're trying to import your own file `D:\PyQt5.py`
Comment: YOU ARE RIGHT!!!!!! )))))))) Thanx!!!! I changed the name of the file and it worked! (how to mark your comment as an answer?)
Comment: well, now you can mark the answer -)
Answer: As Python documentation states, the directory containing the input script precedes all the other search paths unless sys.path list is tampered with.
And your directory D:\ already contains module PyQt5 which is being imported. Trying to import submodule QtWidgets from your module that is not a package then leads to the following error:

File "D:\PyQt5.py", line 2, in <module>
  from PyQt5.QtWidgets import QApplication, QWidget
ImportError: No module named 'PyQt5.QtWidgets'; 'PyQt5' is not a package

So, do not call your program file the same as a package name.
Answer: For the googlers out there, this is what solved my problem:
You need to make sure you import sys and have the proper casing for the import command.
import sys
from PyQt5 import QtGui

That is the correct casing of the PyQt5 string. That took me a while to figure out.

installing PyQt5 in Windows 7


8 - How do I load a Term entity from an EntityReferenceItem?

Question: I have a content type with a taxonomy term. I can load the first EntityReferenceItem using
$node->country->get(0)
then I can tid doing ->target_id (and then manually load the Term object via Term::load.
But is there a function of EntityReferenceItem that I can call to get it to load the corresponding term? I'm thinking something like:
  $node->country->get(0)->getReferenced()

I don't see anything at the Drupal API reference
Answer: $node->country->entity.

See http://wizzlern.nl/drupal/drupal-8-entity-cheat-sheet, where it lists the properties of common field types.
If you want to see the properties of a field type, one way to do that is by looking at the propertyDefinitions() method of the field type class, in this case EntityReferenceItem.
Comment: What is there multiple values allowed? ->country[0]->entity ?
Comment: not specifying a delta means [0],works always, also on multi-value fields. Delta works with [1] or ->get(1).
Answer: Update code for 8.6, following example of term reference field
$tid = $para->get('field_category')->getString();
$tname = $para->get('field_category')
      ->get(0)
      ->get('entity')
      ->getString();


Text formatting in the "Help Text" section

Question: I'm wondering where the formatting is set for the "Help Text" section. I was thinking that maybe it was plain text, but it does not appear to be so - I want to use the module Collapse Text to collapse the text and make the help text available to the user only if they want to view it, while filling out their registration form.
Is there a way to do this, or will I have to write a third party module?
This is the module I want to use with the help text.
https://drupal.org/project/collapse_text
Comment: That module will not do what you are looking for, it is only for text that you enter into text boxes when editing content.  You could use a couple lines of jQuery to expand/collapse the help box with .slideToggle() or something similar
Comment: Well how is text formatting set for the help fields?

I will probably just write jQuery for it though yes.
Answer: The Advanced Help module will let you create help texts that can be tied to a help icon on any page, form or form item. Clicking the icon will pop-up a Ajax window with the text.


IOUtils.toString() exception java.net.SocketTimeoutException: null

Question: On a web application, i use async servlet.
When i use IOUtils.toString() method, convert http inputstream to String.
But get java.net.SocketTimeoutException: null on production server,
and unable to reproduce on my local machine.
Below is the stack trace:
java.net.SocketTimeoutException: null
    at org.apache.tomcat.util.net.NioBlockingSelector.read(NioBlockingSelector.java:202) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:250) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:231) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInputBuffer.java:133) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.coyote.http11.InternalNioInputBuffer$SocketInputBuffer.doRead(InternalNioInputBuffer.java:177) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:110) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.coyote.http11.AbstractInputBuffer.doRead(AbstractInputBuffer.java:362) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.coyote.Request.doRead(Request.java:476) ~[tomcat-coyote.jar:8.0.42]
    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:350) ~[catalina.jar:8.0.42]
    at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:395) ~[tomcat-util.jar:8.0.42]
    at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:375) ~[catalina.jar:8.0.42]
    at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:190) ~[catalina.jar:8.0.42]
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) ~[na:1.8.0_65]
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) ~[na:1.8.0_65]
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) ~[na:1.8.0_65]
    at java.io.InputStreamReader.read(InputStreamReader.java:184) ~[na:1.8.0_65]
    at java.io.Reader.read(Reader.java:140) ~[na:1.8.0_65]
    at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2001) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1980) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.IOUtils.copy(IOUtils.java:1957) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.IOUtils.copy(IOUtils.java:1907) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.IOUtils.toString(IOUtils.java:778) ~[commons-io-2.4.jar:2.4]
    at org.apache.commons.io.IOUtils.toString(IOUtils.java:803) ~[commons-io-2.4.jar:2.4]

Answer: Socket timeout implies either the connect timeout or the read timeout has been reached. This error is not caused by IOUtils.toString().
The connect timeout will be thrown immediately when trying to connect to a remote endpoint. if the socket is open at the remote end but no data is returned when trying to read data from the socket you will receive a read timeout.
Both the read and connect timeouts can be set although I suggest using telnet to validate the port is open at the remote end first outside of your application.
Comment: Thank you for your reply. I read data from ios or android client http request, how i using telnet to validate the port?Now i only set timeout on tomcat server.xml connectionTimeout field


How to create a custom create route in rails

Question: I currently have a create method in an invoices controller. However i need another one, which is called 'create_inappointment_invoice'. I was wondering what the syntax was in the routes file to create a route in relation to this. I basically want the same route as this resources :invoices, controller: "admins/invoices", only: [:create] but changing the name of the controller action. Thanks
Answer: post 'admins/all_invoices', to: 'admin#invoices'
where admin is your controller and invoices in your action in it.
Comment: thanks for your answer, however i dont quite understand your description.
Comment: If the above route `admins/all_invoices` matches then the control would go to your controller `admin` and action `invoices`
For Eg:

   `class Admincontroller < ApplicationController
         def invoices
             # write your code  here
         end
      end`
Answer: Something like the below snippet should do the job:
resources :invoices, controller: "admins/invoices", only: [:create] do
    collection do
        post :create_inappointment_invoice
    end
end

OR
resources :invoices, controller: "admins/invoices", only: [:create] do
    post :create_inappointment_invoice, on: :collection
end

This will hit the create_inappointment_invoice method in your admin/invoices controller
Comment: Thanks for your answer. However, this is taking me to the original create method still.
Comment: It shouldn't be! What's the url you are hitting?
Comment: it's currently hitting the `render new` part of my other create action condition, which would suggest that it is still hitting the original create method.
Answer: This seemed to work:
    `post "/invoices", to: "admins/invoices#create_inappointment_invoice", as: :create_inappointment_invoice`


Automating subscription to iCalendar (RFC5545) feed

Question: I maintain an RFC5545 calendar for my school and I'd like to simplify the subscription process. I'm wondering if it is possible to create a URL or something similar that would facilitate calendar discovery and subscription. Ultimately what I'd like is a "click here to subscribe" link. Ideally for Outlook, but macOS Calendar (iCal), and Google Calendar would also be nice to support.
Is this possible? So far my Google foo is failing me and I'm only finding instructions for manually adding a calendar subscription, nothing for how to automate the process for my users.
Answer: I stumbled upon the a calendar example with a URI using the webcal protocol. There is a bit more information in this post: What is a webcal protocol?. The URI would be constructed like this:
webcal://calendar.example.com/calendar.ics
A user who clicks on the link is prompted to open their calendaring app and add subscribe to the calendar. It works for iOS and macOS Calendar and Outlook.


Extract action/task from text using nltk

Question: Hi i am using nltk first time  and i want  to Extract action/task  from text using nltk
Hi prakash, how are you ?. We need to complete the speech to action by 8 June  then you will have to finish the UI by 15 july

Above here the speech to action and UI  is the action.
I have started the token creation, don't know what to do next, Please guide.
from nltk import sent_tokenize
sample_text ="""Hi prakash, how are you ?. We need to complete the speech to action demo by 8 June  then you will have to finish the Ui by 15 july"""
sentences = sent_tokenize(sample_text)
print(sentences)    import nltk
from nltk.tag import pos_tag
from nltk.tokenize import word_tokenize

sample_text = """Hi prakash, how are you ?. We need to complete the speech to action by today
then you will have to finish the UI by 15 july after that you may go finish the mobile view"""
sample_text = "need to complete the speech to action by today"
tokens = word_tokenize(sample_text.lower())
# the lower is very much required, as June and june have diffrent code NN, NNP

pos_tags = pos_tag(tokens)

result = []
for i in range(len(tokens)):
    if (pos_tags[i][1] == 'VB') and (pos_tags[i][0] in ['complete','finish']):
        # Here we are looking for text like (finish, complete, done)
        owner = ''
        for back_tag in  pos_tags[:i][::-1]:
            #traverse in back direction to know the owner who will (finish, complete, done)
            if back_tag[1]=='PRP':
                owner = back_tag[0]
                break

        message = ''
        date = ''
        for messae_index , token in enumerate(pos_tags[i:],i):
            #traverse forward to know what has to be done
            if token[1]=='IN':
                for date_index, date_lookup in  enumerate(pos_tags[messae_index:],messae_index):
                    if date_lookup[1]=='NN':
                        date = pos_tags[date_index-1][0] + ' ' + pos_tags[date_index][0]
                    if date_lookup[1]=='PRP':
                        # This is trick to stop further propegation
                        # Don't ask me why i am doing this, if you are still reading then read the nest line
                        # Save futher interation as the next sentance is i/we/you
                        break
                break
            else:
                message = message + ' ' + token[0]
        result += [dict(owner=owner, message=message, date=date)]
print(result)

Please guide how to extract the actions(action demo, UI) from the paragraph.
Answer: If you're using NLTK, you can get the POS tags of your tokens and come up with a regex or pattern using those tags. For example, an action will be a verb. (For better tagging, you may require Spacy. There is another library called Pattern for these purposes)
But I'm not sure if this is going to help you a lot for a scaled application.
N.B: There are well-trained Named Entity Recognizers available, you may try them.
Comment: Hi Sleeba , Thanks for your help, i have updated the code. Can review this as the complexity is too high.
Comment: Try POS regex using `Pattern` library. Again, I'm not confident that you can build a scalable solution using this approach.
Answer: Here is my thoughts:
If i try to identify parts of speech for your sentence using nltk.tag.pos_tag , i get below:
 import nltk
 from nltk.tag import pos_tag
 from nltk.tokenize import word_tokenize

 s = 'Hi prakash, how are you ?. We need to complete the speech to action by 8 June  then you will have to finish the UI by 15 july'
 tokens = word_tokenize(s)
 print(pos_tag(tokens))

Output:
[('Hi', 'NNP'), ('prakash', 'NN'), (',', ','), ('how', 'WRB'), ('are', 'VBP'), ('you', 'PRP'), ('?', '.'), ('.', '.'), ('We', 'PRP'), ('need', 'VBP'), ('to', 'TO'), ('complete', 'VB'), ('the', 'DT'), ('speech', 'NN'), ('to', 'TO'), ('action', 'NN'), ('by', 'IN'), ('8', 'CD'), ('June', 'NNP'), ('then', 'RB'), ('you', 'PRP'), ('will', 'MD'), ('have', 'VB'), ('to', 'TO'), ('finish', 'VB'), ('the', 'DT'), ('UI', 'NNP'), ('by', 'IN'), ('15', 'CD'), ('july', 'NN')]

If you observe, every action word i.e. "speech to action" or "UI" occur after a preceding verb tag i.e 'complete' and 'finish' respectively.
I would suggest to try this problem with below steps:
1) Find verb in a sentence.(something like below)
for i in range(len(tokens)):
    if pos_tag(tokens)[][1] == 'VB':

2) If found, then fetch the next words based on their pos tags. (may be retrieve all next words until you find 'IN' tag)
This may work for your dataset.
Comment: Hi aditya, Thanks for your help, i have updated the code. Can review this as the complexity is too high.


Overlay values on BoxChart Matlab plots (when using 'groupbycolor')

Question: I would like to overlay all the values (not just the outliers) on top of a boxchart such that they are displayed over the corresponding categories.
MWE shows the box and whisker plots:
tbl = table;
h=100;
tbl.cat = categorical(randi([1,4],h,1 ));
tbl.y = rand(h,1);
tbl.color = randi([5,8],h,1 );
% figure
boxchart(tbl.cat,tbl.y,'groupbycolor',tbl.color)
legend(Interpreter="none",AutoUpdate='off',Location="northeastoutside")
         %% create division lines between categories, useful so I thought nice to share
        for iDivide = 0:height(gca().XAxis.Categories)
             line([iDivide+0.5, iDivide+0.5],[min(ylim),max(ylim)],'Color','black','LineStyle',':')
        end

I found a related example (https://www.mathworks.com/matlabcentral/answers/398012-adding-a-scatter-of-points-to-a-boxplot#comment_2180020) showing the use of swarmchart to overlay the values. However, the results do not align with the additional use of 'groupbycolor' categories.
    hold on
    swarmchart(tbl.cat,tbl.y,[],tbl.color)

Is there a way to overlay the values from each of the categories and subcategories from 'groupbycolor' over a boxchart?
Is there a way to modify the xaxis location for the values displayed using swarmchart so that the categories align with those from boxchart?

Many thanks in advance,
Jesse
Comment: You might file the `daboxplot` FileExchange entry interesting: https://uk.mathworks.com/matlabcentral/fileexchange/74851-daboxplot (or any of the other boxplot tools on FileExchange). Or you could just `hold on` and `plot( x, y, 'ok' )` where `x` and `y` are the subset of points for each box, similar to what you've done for the dividing lines
Comment: I am having trouble getting `daboxplot` to work. I have one column of data, and two columns which are the x axis categories and 'groupbycolor' categories. For daboxplot, there are separate data columns for each of the xaxis categories and another 'groups' column. All the columns have the same length, which is not the case for me... Is there a way around this? Using `plot( x, y, 'ok' )` looping over each of the categories would be good (after adding some xjitter), however I dont know how to get the xcoordinates for each of the groups
Answer: Picture the space between the divider lines you've added

There's 4 coloured boxes
Each bar occupies an area on the x-axis which is space, box, space
The spacing from left to right in each block is therefore sbssbssbssbs, or sbs for each of the 4 bars.
Each category, or block of these coloured boxes, is centred around an integer, so the sbs.. sequence starts from an integer minus 0.5.

Therefore we know the x position for the centre of box icol within category icat is
x = icat - 0.5 + ...   % start from left of "block" for this cateogry
    s + (b/2) + ...    % at least one space plus half the first box
    (icol-1)*(s+b+s);  % shift by one "sbs" width for each colour

The boxwidth by default is 0.5 (within its own space, not of the whole axis), but you can set that in boxplot to be explicit. Then we have something like this:
b = 0.5; % box width
boxchart(tbl.cat,tbl.y,'groupbycolor',tbl.color,'BoxWidth',b)

ucol = unique(tbl.color); % unique colours    (sorted)
ucat = unique(tbl.cat);   % unique categories (sorted)

s = (1-b)/(2*numel(ucol)); % spacing left & right of each box
b = b / numel(ucol);       % actual box width within each category

% Overlay points on each box, per unique combo of category and colour
hold on;
for icat = 1:numel(ucat)
    for icol = 1:numel(ucol)
        % Index of rows for this bar's data
        idx = (tbl.cat == ucat(icat)) & (tbl.color == ucol(icol));
        % Work out the x-axis position
        x = icat - 0.5 + s + (b/2) + (icol-1)*(s+b+s);
        % Plot the individual points
        plot( x*ones(nnz(idx),1), tbl.y(idx), 'xk', 'handlevis', 0 );
    end
end

You can style the plot however you want, I've added the 'HandleVisibility', 'off' input here to hide the additional markers from the legend.
Example output:


Is there a smart way to override form action links in sfDoctrineGuard plugin?

Question: I've set up my Symfony 1.4 project to route /admin in the URL to my backend app. This works fine until I want to edit a user or interact with the generated forms in any way. This is because the generated links are not aware of my /admin prefix.
Is there a smart way to prefix these links without copying the generated templates to my app folder structure?
UPDATE
Contents of generator.yml for the sfGuardUser module:
generator:
  class: sfDoctrineGenerator
  param:
    model_class:           sfGuardUser
    theme:                 admin
    non_verbose_templates: true
    with_show:             false
    singular:              ~
    plural:                ~
    route_prefix:          sf_guard_user
    with_doctrine_route:   true

    config:
      fields:
        password_again: { label: "Password (again)" }

      list:
        title:   User list
        display: [=username, created_at, updated_at, last_login]

      form:
        class: sfGuardUserAdminForm
        display:
          "User":                   [first_name, last_name, email_address, username, password, password_again]
          "Permissions and groups": [is_active, is_super_admin, groups_list, permissions_list]

      edit:
        title: Editing User "%%username%%"

      new:
        title: New User

I have added my own sf_guard_user in my routing.yml which works for new links I create but the generator doesn't pick this up.
Answer: Take a look at a sample configuration file for the admin generator (generator.yml): http://www.symfony-project.org/jobeet/1_4/Doctrine/en/12#chapter_12_final_configuration
Most specifically, look at the route_prefix: parameter.
Try setting the route_prefix to
route_prefix: admin

Comment: Thanks but this doesn't work. It tries to look for the admin_collection route.


Uso da função set e for na mesma estrutura

Question: Eu estou estudando data science através da linguagem Python e me deparei com o seguinte código:
world_trends_set = set([trend['name'] for trend in world_trends[0]['trends']])
us_trends_set = set([trend['name'] for trend in us_trends[0]['trends']])

Este exemplo eu retirei do livro "Mining-the-Social-Web-2nd-Edition". Eu não consigo entender o porquê do trend['name'] antes do loop. Eu imagino que o código está criando um conjunto chamado name com os itens da primeira linha da coluna trends.
Alguém pode me explicar a vantagem do uso da sintaxe dessa forma e me corrigir se eu estiver errado?
Answer: Considerando a linha:
world_trends_set = set([trend['name'] for trend in world_trends[0]['trends']])

O código equivalente seria:
temp = []

for trend in world_trends[0]['trends']:
    temp.append(trend['name'])

world_trends_set = set(temp)

O resultado produzido por ambos os códigos será exatamente o mesmo, então, sim, o que o código está fazendo é criar um conjunto a partir de uma lista. Esta lista, por sua vez, é composta pela coluna name do valor trend. Isto é, world_trends é uma lista em que sua posição 0 também é uma lista de dicionários que possuem a coluna name.
Considerando que você esteja primeiro criando uma lista, ou seja, armazenando todos os valores em memória, e depois convertendo para um conjunto, não nenhum ganho de performance ou custo de memória. A diferença está mais na escrita do código: a primeira forma é menor e até mais legível para humanos que a segunda. Ou seja, a primeira forma é mais fácil de entender.
Aliás, você pode até remover os colchetes em torno da expressão for da seguinte forma:
world_trends_set = set(trend['name'] for trend in world_trends[0]['trends'])

Dentro dos parenteses teríamos uma generator expression sendo convertida para um conjunto. Também não apresentaria melhores de performance, acredito eu, pois converter o generator para um conjunto também armazenaria todos os elementos em memória, mas seria uma operação a menos para o Python processar.


Retrieve translated taxonomy term in current language programatically

Question: How do I retrieve a specified taxonomy term translated (if translation exists for this term) from the current language context programmatically with D8?
Comment: [Here's how to do it with Drupal 7.](https://drupal.stackexchange.com/q/79037/27135)
Answer: Use the following code:
$curr_langcode = \Drupal::languageManager()->getCurrentLanguage(\Drupal\Core\Language\LanguageInterface::TYPE_CONTENT)->getId();

// retrieve term
$taxonomy_term = \Drupal\taxonomy\Entity\Term::load($tid);

// retrieve the translated taxonomy term in specified language ($curr_langcode) with fallback to default language if translation not exists
$taxonomy_term_trans = \Drupal::service('entity.repository')->getTranslationFromContext($taxonomy_term, $curr_langcode);

// get the value of the field "myfield"
$myfield_translated = $taxonomy_term_trans->myfield->value;

Answer: You should (must) use service instead at the first line for language_manager. Also I would shorten the code by using use tags.
Somewhere in the beginning of file:
use Drupal\taxonomy\Entity\Term;
use Drupal\Core\Language\LanguageInterface;

and later in the code in some function
$curr_langcode = \Drupal::service('language_manager')->getCurrentLanguage(LanguageInterface::TYPE_CONTENT)->getId();
// Retrieve term.
$taxonomy_term = Term::load($tid);
// Retrieve the translated taxonomy term in specified language
// ($curr_langcode) with fallback to default language if translation not
// exists.
$taxonomy_term_trans = \Drupal::service('entity.repository')->getTranslationFromContext($taxonomy_term, $curr_langcode);
// Get the value of the field "myfield".
$myfield_translated = $taxonomy_term_trans->myfield->value;

Answer: The snippets above will return untranslated terms also. You must check if a term is translated with the hasTranslation function:
$vocabulary = 'MY_VOCABULARY_NAME';
$language =  \Drupal::languageManager()->getCurrentLanguage()->getId();
$query = \Drupal::entityQuery('taxonomy_term');
$query->condition('vid', $vocabulary);
$query->sort('weight');
$tids = $query->execute();
$terms = \Drupal\taxonomy\Entity\Term::loadMultiple($tids);
$termList = array();

foreach($terms as $term) {
    if($term->hasTranslation($language)){
        $translated_term = \Drupal::service('entity.repository')->getTranslationFromContext($term, $language);
        $tid = $term->id();
        $termList[$tid] = $translated_term->getName();
    }
}

// To print a list of translated terms.
foreach($termList as $tid => $name) {
 print $name;
}

To link the tags to their term page: See: Get taxonomy terms
Answer: An approach in preprocess node (handles multiple term references):
function HOOK_preprocess_node(&$variables) {

  $node = $variables['node'];
  $langcode = \Drupal::languageManager()->getCurrentLanguage()->getId();
  $pathAliasManager = \Drupal::service('path_alias.manager');

  if ($node->hasField("field_name") && !$node->get("field_name")->isEmpty()) {

    $terms = $node->get("field_name")->referencedEntities();

    foreach ($terms as $term) {

      $name = $term->name->value;
      if ($term->hasTranslation($langcode)) {
        $name = $term->getTranslation($langcode)->name->value;
      }

      $variables['termValues'][] = [
        'pathAlias' => $pathAliasManager->getAliasByPath('/taxonomy/term/' . $term->id()),
        'name' => $name,
      ];
    }
  }
}

Then in Twig:
{% for items in termValues %}

    {{ items.name }}

{% endfor %}