Case insensitive search without additional rules?

Question: I'm trying to find a collation in MySQL (my version is 5.0) where strings that differ in case are considered the same but there're no other rules like:
á = a
and so on.
I tried to find the proper collation here: http://www.collation-charts.org/mysql60/by-charset.html but it seems that the collation I'm looking for doesn't exist.
I can't use in SQL query: SELECT ... WHERE lower(column1) = lower(column2) because indices on columns column1 and column2 are not used then and my query is terrible slow.
Thanks for any suggestion!
Answer: I was given an advice: simply have table like this: id, word, word_in_lowercase.. it's true that data are redundant but otherwise it fulfils all my needs.
Automatic update of word_in_lowercase may be done via trigger or some additional programming.
Comment: according to http://stackoverflow.com/questions/234591/upper-vs-lower-case/234751#234751, the simple 'lowercase' comparison can be incorrect for some languages
Answer: Which type of collation set in the tables that in question? I'm currently using a lot of tables with utf8_hungarian_ci because of this one is case-insensitive.
Comment: I need it for Czech language but it's not important because the only thing I need is that A = a and so on but á is not equal A and so on.
Answer: http://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html indicates that nonbinary strings are case insensitive by default. Have you tested to see that it is not working properly without using lower()?
Comment: if the string is nonbinary I need to set a collation and the question is what collation it should be if I don't want the rules like á = a and so on.
Comment: I see your point. I was not aware that this occurred in the default character set.
Answer: Why don't you use the full text search functions of MySQL for your search query?
For tasks like yours I am using the MATCH AGAINST function.
Read the Specifications at mysql.com to make it clear - Link
One example:
SELECT * FROM customer WHERE status = 1 AND MATCH (person, city, company, zipcode, tags) AGAINST ('".$searchstring."' IN BOOLEAN MODE)

And this will be executed case insensitive.
Comment: You're welcome. The only thing i have not tried is if mysql "ignores" charactrs with e.g. an acent like "a" and "á".
Would be nice if you'll share your experiences.

ps. dont forget to mark your accepted answer ;-)


Fitting a curve to the edge of a distribution

Question: I need to be able to find outliers in my data. I thought it best to test for this using the Kolmogorov-Smirnov Test.
I have over 800,000 points so I wanted a way to filter the data first to only test those that are on the edge of my sample. By eye I can fit an exponential decay (red line) but I was wondering if there was a statistical way to determine the parameters for my exponential.
Comment: Might help if you say *why* you need to find outliers. By eye, I wouldn't say there are any particularly severe outliers, considering the size of the data set. The only obvious anomaly is the little line of values with y=0 for x=1,2,3,4.. with a gap just above it.
Comment: You will find it illuminating to plot these *bivariate* data on log-log axes.  (To include the zero values you will need to add a small constant to all values first.)  Among other things, you will find that the exponential decay is not quite right; the relationship may be better approximated with a power law.
Comment: xboxrob, it is unclear what you are asking for, and consequently the replies you are getting may or may not be applicable; at any rate, future readers will not be able to gauge whether they are appropriate solutions.  The K-S test does not seem applicable--you display bivariate data and it's for univariate data only--and finding outliers does not seem related to your red curve.  What exactly do you want to accomplish?
Comment: On the contrary, I noted these are **bivariate** data, as your description confirms.  It is really strange that the variance of the measurements would *increase* as the measurements themselves decrease.  This suggests that both $x$ and $y$ are actually differences relative to a large value (perhaps around $100$ or so).  It would be more productive and informative to work with the original $(x,y)$ data.
Comment: +1 whuber. @xboxrob: Aren't you, by fitting a curve, already making assumptions about what might be an outlier? I'm not sure.
Comment: Yes, this is a weird one:  if your points are random, their boundary is not a density function... There is a large literature on non-parametric support estimation, see e.g. [Devroye 1980](www.jstor.org/stable/2100656) or [Tsybakov 1997](http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aos/1069362732).
Comment: Hi all,

Firstly thank you for your responses. The y-axis is a measurement of difference between two measurements. The x-axis is the minimum value of the two measurements. As you have noted this is univariate data and hence when x is small there is much larger differences by random chance.

The idea therefore is that points that are far away from the edge of the distribution (roughly the red line) are actually important values as the difference between the two measurements should be more believable
Answer: Perhabs nonlinear Quantile Regression?
Answer: My suggestion is to look for "extreme value" theory and distributions.
If you are using R do the following:
install.packages("texmex", dep=T)
then type following in the
help(package=texmex)
A


How can we find the elementary divisors?

Question:
We consider the polynomial ring $R=\mathbb{R}[t]$ and the $R$-module $M=\mathbb{R}^3$, where $a\cdot x$ ($a\in R,x\in M$) is defined as usual if $a\in \mathbb{R}$, and $a\cdot x=(x_1, 0, 0)$ if $a=t, x=(x_1, x_2, x_3)$.

From the structure theorem for finitely generated $R$-module there is a list $p_1, p_2, \dots , p_n$ irreducible of $R$ and a list $k_1, k_2, \dots , k_n$ positive integers such that $$M\cong R/\langle p_1^{k_1}\rangle \oplus \dots \oplus R/\langle p_n^{k_n}\rangle$$
How can we find these lists?
Answer: Hint: The polynomials $p_1,\dots,p_n$ must each annihilate a nontrivial submodule of $M$, so to identify these polynomials, try seeing if you can find elements of $\mathbb R[t]$ that annihilate some non-zero element of $M$.
Comment: The elements of $M$ are just triples of real numbers. So, for example if we tried checking the action of the irreducible polynomial $p=2+t$ on $m=(1,2,3)$, we would get $p \cdot m = (2+t) \cdot (1,2,3) = 2 \cdot (1,2,3) + t\cdot(1,2,3) = (2,4,6)+(1,0,0) = (3,4,6)$. If the result had come out to $(0,0,0)$, then we would have identified $p$ one of the $p_i$ and $m$ as an element of one of the summands in the decomposition of the structure theorem. So try different values for $p$ and $m$ and see if you can find one that works.
Comment: It looks like you dropped the coefficient $a_i$ of $t^i$ for $i=1,\dots,n$.
Comment: Yes, that looks good.
Comment: More or less right. Now can you find specific choices of the $m_i$ and $p_i$ that work and are non-trivial, i.e., as in your cases (2) and (3)?
Comment: Which form have the elements of $M$ ?
Comment: When we have that $p=\sum_{i=0}^n a_it^i, m=(m_1, m_2, m_3)$ then $$p\cdot m=a_0\cdot (m_1, m_2, m_3)+\sum_{i=1}^n t^i (m_1, m_2, m_3) \\ =(a_0m_1, a_0m_2, a_0m_3)+n(m_1, 0, 0)=(a_0m_1+nm_1, a_0m_2, a_0m_3)\\ =((a_0+n)m_1, a_0m_2, a_0m_3)$$

We get $p\cdot m=(0,0,0)$, in the following cases:

 $$1) \ \ a_0=0 \land n=0$$
 $$2) \ \ a_0=0 \land m_1=0$$
 $$3) \ \ m_2=0 \land m_3=0 \land n=-a_0 (\text{ when } a_0<0 )$$
 $$4) \ \ m_2=0 \land m_3=0 \land m_1=0$$

or not?
Comment: Oh yes... It should be as follows: $$p\cdot m=a_0\cdot (m_1, m_2, m_3)+\sum_{i=1}^n a_it^i (m_1, m_2, m_3) \\ =(a_0m_1, a_0m_2, a_0m_3)+\sum_{i=1}^n (a_i m_1, 0, 0)=(a_0m_1+m_1\sum_{i=1}^na_i, a_0m_2, a_0m_3)\\ =((a_0+\sum_{i=1}^na_i)m_1, a_0m_2, a_0m_3)$$ or not?
Comment: Is this correct so far?
Comment: Therefore, we have the following: $$ 1) \ \ a_i=0 , i=0, \dots , n$$
$$2) \ \ a_0=0 \land  a_i\neq 0 (i=1, \dots , n) \land m_1=0$$
$$3) \ \ m_2=0 \land m_3=0 \land a_0+\sum_{i=1}^na_i=0  $$
$$4) \ \ m_2=0 \land m_3=0 \land m_1=0$$ or not?
Comment: For $p=t+2t^2$ and $m=(0,1,2)$, we get $p\cdot m=(0,0,0)$, right?
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/40442/discussion-between-brent-kerby-and-mary-star).


Running SQL Server 2008 with an Equallogic PS-series

Question:
Possible Duplicate:
Anyone have any experience with SQL over iSCSI on Equallogic boxes?

We've currently got a dedicated Microsoft SQL Server 2008 R2 server that's quickly running out of disk space. The server is hammered 24/7 pretty much and unfortunately requires a lot of IO to keep running smoothly.
We're considering purchasing a Dell Equallogic PS6010 and get some 10GbE equipment to handle it. I've seen all the Dell promo videos showing snapshots and SQL-Aware functionality and their reps keep sending me whitepapers and statistics, but does anyone have any real-life experience with such a scenario? (I'm guessing this is quite common in larger datacentres)
If so, how did you find the deployment and throughput/IOPS? 3323 IOPS keeps getting thrown around as its maximum performance, but has anyone found anything close to that in the real world?
Comment: I just noticed my own question is almost an a dupe (just more wordy) of one from a while ago, so feel free to vote to close as dupe!
Answer: @Farseeker,
First I'd like to note that these views and comments are of my own accord and not driven in anyway by Dell. I work for Dell in the EqualLogic engineering team and I specialize in SQL Server integration. The question you pose around IOPS is pretty open ended because a 1Gb infrastructure can handle 3323 IOPS as well as a 10Gb infrastructure. The question you need to think about is what does your IO look like? Is it prmarily read or write, large or small IO etc. This will determine if you need to consider 10Gb or not.
If you simply want to know if an EqualLogic environment can handle 3323 IOPS then the answer is yes but in order to really size your environment for the right type and # of arrays you would need to know the following: total IOPS (which you do), IO type (sequential/random), read/write mix, and size of average IO. This will give a better idea of your workload and how to successfully build a solution to meet it's needs. With this information your Dell sales rep should be able to run the numbers through their sizing tools and give you more clear answer.
Are you aware of Dell TechCenter? There's a lot of resources there and it's a community type site where you can reach out to other professionals as well as Dell technologists through the forum. The url is: http://www.delltechcenter.com/
Comment: Thanks Darren. I've been speaking with Dell a bit more over the last 24 hours, and after running some perfmon counters they sent us on our SQL servers I've submitted them to their storage guys and they're going to analyse them. We only have the budget for one Equallogic to begin with. Thanks for your advice though, I'll check out that techcentre link.
Answer: @Farseeker, you might find this Dell Whitepaper to be helpful in your research:
Deploying Microsoft SQL Server 2008 DSS Systems using Dell EqualLogic 10GbE iSCSI Storage


Cross-Platform API to monitor file system in C?

Question: I'm looking for a cross platform library to detect when files in a directory are added or modified.
I know there are OS specific way to do this (inotify for Linux, FindFirstChangeNotification for windows, etc...).
But is there a platform independent library that works specifically in C? (Like the QFileSystemWatcher in C++)
Answer: It's now 2021 so maybe septag/dmon is your cup of tea.
From the description:

Single header C99 portable library for monitoring directory changes

... or maybe the more beefy fswatch ?
Answer: inotify is Linux specific, if you want some UNIX portable features you are pobably looking for something like libfam. it is name of library. Full package name is fileschanged.
fileschanged is a GNU/Linux command-line utility that reports when files have been altered.
Comment: Cool! I didn't know that library. But what I'm searching specifically is, something that will work for Windows and for Linux
Comment: The APIs are totally different for Windows, Linux, Mac OS X, and any other Unix you can name, it seems. I don't know of any cross-platform library that handles this in a consistent way.
Answer: Sounds like a good use case for golang.
Simply change $GOOS and/or $GOARCH and run go build.
Boom - trivial cross-platform development.
Comment: The question asks about C, not GO.
Comment: And if using C is an arbitrarily self-imposed constraint, Go is a reasonable consideration.


How to compare two arrarys and create new array with multiple conditional statement?

Question: I have a two arrays x and y with same dimensions.  I want to do multiple comparisons (or operator) values in these arrays and generate a new array with same dimensions. The new array should have the values assigned by me. Here is the little demonstration of what I am trying to do:-
In [1]: import numpy

In [2]: import numpy as np

In [3]: x = np.array([5, 2, 3, 1, 4, 5])

In [4]: y = np.array([2, 3, 3, 8, 8, 6])

In [5]: result_array = [y > 3] or [x < 5]

In [6]: print(result_array)
[array([False, False, False,  True,  True,  True], dtype=bool)]

I am able to compare multiple statement and result in new array. However, I would like to replace the True with value 10. So when I try this line, it gives me an error:-
result_array = 10 if [y > 3] or [x < 5]:
  File "<ipython-input-21-780bf095bc56>", line 1
    result_array = 10 if [y > 3] or [x < 5]:
                                           ^
SyntaxError: invalid syntax

What I am expecting is:-
[array([False, False, False,  10,  10,  10], dtype=bool)]

Any help is appreciated
Comment: Your output is not possible, since 10 is not a `bool`. `bool` may only be `True` or `False`; or, equivalently, 1 or 0.
Comment: Is it not possible to replace the True with some values in the array? The out put need not contain False, may be 0.
Comment: Your output is not possible, since 10 is not a `bool`. `bool` may only be `True` or `False`; or, equivalently, 1 or 0.
Comment: Your `result_array` possibly is not what you want: the expression `[y > 3] or [x < 5]` evaluates `[y > 3] ` that is a list containing a boolean array, next because the first element is `True` (a non-empty list is `True` Boolean-ly speaking) the value of the expression is a list containing the boolean array `y>3` — in other words, you are not doing an element-wise comparison of two boolean arrays.
Comment: You seems to be correct. Can you give some lead?
Answer: You need to convert your result to integer in order to see the 1:
x = np.array([5, 2, 3, 1, 4, 5])
y = np.array([2, 3, 3, 8, 8, 6])
result_array = np.logical_or(y > 3, x < 5)
res = result_array.astype(int)
res[result_array] = 10
print(res)

Output:
[ 0 10 10 10 10 10]

Comment: Just over looked your answer. Again gone through it and accepting as an answer.
Answer: You can get close to the result you mentioned to be expecting using this:
import numpy as np

x = np.array([5, 2, 3, 1, 4, 5])
y = np.array([2, 3, 3, 8, 8, 6])

result_array = np.where(y > 3, 10, False)
print(result_array)

Result:
[ 0  0  0 10 10 10]

Note that there are 0s instead of Falses, because it contains only numbers


Does restoring a SQL database from backup rebuild its indexes?

Question: Does restoring a SQL database from backup rebuild its tables and indexes from scratch? Or does it keep it in the same internal physical order it was in at the time of backup?
We're using SQL 2000 with Quest Lightspeed compressed backup, if that makes any difference.
Answer: The answer is no, for whatever backup software is being used.
A backup is a physical operation, not a logical operation. It reads all extents containing allocated pages (i.e. even though only a single page from an 8-page extent is allocated, it will backup the entire 64K extent), and it does it in physical order.
A restore is a physical operation, not a logical operation. It lays down the extents in their rightful places in the data files.
Rebuilding an index (or anything like it) is a logical operation, which must be logged. Backup and restore manipulate the data files directly, without going through the buffer pool, which is one reason why this cannot be done. Another reason this cannot be done is that backup and restore have no understanding of what is contained in the data being backed up.
The main reason this cannot be done, however, is that moving pages around during a restore operation would break the b-tree pointers. If page A points to page B, but page A is moved by the restore process, how is page B updated to point to page A? If it's updated right away, then it may be overwritten by the rest of the restore process. If it's deferred-updated, what if the restore process restored some transaction log that removed page A, or page B? It simply cannot be done.
Bottom line - backup and restore are physical operations that never change the data.
Hope this helps!
PS Although it doesn't directly address this question, check out the article I wrote for the July TechNet Magazine which explains how the various backups work internally: Understanding SQL Server Backups. The September magazine will have the next in the series on understanding restores.
Comment: I love the fact that you explained that indexing is a logical operation.
Comment: @John What are you saying is nonsense? Compression isn't mentioned anywhere here - just rebuilding the indexes, which isn't in any way the same as compression (which doesn't change the pages or their location in the database during a restore, whereas a rebuild does). Recreating indexes from scratch during a restore would be incredibly slow compared to just restoring them as-is from the backup. I think you're misunderstanding something here.
Comment: @John The difference is that SQL backup compression operates only on the 64k physical extents that Paul describes, it doesn't know or care about the internal *meaning* of the bits. During restore the original 64k extent is recreated and written directly to disk as a physical operation. Also, Paul, who you replied to, WAS one of the MS SQL engine team developers you are decrying as "lazy". I think I'll defer to his expertise on these matters.
Comment: Ah, that makes sense. Backup grabs full 64k physical extents, not 8k data pages.
Comment: This is nonsense. Backup operations frequently compress the data they back up, and that's the kind of "change" we're talking about: After all, the indexes can be re-created from the tables, they are redundant data (provided that the schema is backed up, of course). The real reason is laziness on the part of the database developers.
Comment: Dropping and rebuilding indexes is a type of compression, and compression could change anything, depending on the type of compression. Lossy compression in image processing is still called compression. Any mechanism that represents given information in smaller space is a form of compression, and dropping indexes is a trivial example of that. The slow rebuild time may indeed be a real argument though, at least against the implementation being a worthwhile endeavor.
Answer: A native SQL backup is just a page-by-page dump of the backup files, so the answer there is "no".  A Quest lightspeed backup likely uses some sort of compression compression algorithm, but it still won't "rebuild" the data files or indexes, which would take a horrendously large amount of time on a big database.
Comment: Yeah, but it has to write every page to disk anyway. Why not write them in logical sequence, instead of in a fragmented order? (Maybe this is more of a *backup* question instead of a *restore* question: does the backup write the pages in physical order, or in logical order?)
Comment: assuming there was a product that wrote the data out in index order, which order would you like the table saved in? Lets say I have a table with 3 columns product_id, productname, and price.  Which is the correct column to sort on to save the pages in the indexed order?  BTW there is nothing stoping you from indexing on the entire table (a clustered index) or each of the rows (composite index).
Comment: @Jim B: That's easy. Tables would be saved in clustered index order. Non-clustered indexes would be stored in key order. Heaps would be kept in original (non-sorted) order. (Aaron and Paul have mentioned valid reasons that backup/restore doesn't do this. Not being able to figure out the "preferred order" isn't one of these reasons. Or else doing a full index rebuild would have the same problem.)
Comment: The data is backed up in the exact same physical page order that it is saved in within the database files.  When the data is restored it is restored in the same page order that it was backed up in.  SQL doesn't move data around for various reasons.  Including that there could be issues with transactions restoring logs and page chaining issues, not to mention the massive amounts of extra time needed to shuffle the data around on multi-TB databases.
Answer: Backup is done regularly and very often (I hope). So designers made sure backup is as quick as possible. What is the quickest I/O? Sequential. You read blocks from disks in exact physical order, you have the best performance.
Why on Earth should database perform cumbersome random I/O operation every single night, trashing the disks' heads all over the place? The difference would be around two orders of magnitude. There is no possible gain in this.
Comment: I agree with your overall point, but depending on the underlying storage configuration, random I/O may not be orders of magnitude worse than sequential I/O (a SAN drive that is spread over dozens of spindles, for example). In fact, if the data file is fragmented on the hard drive, then even "sequential" I/O isn't really sequential at all. But Paul's points override this anyway (the problem with updating pointers, and that defragmentation should be a logged operation)
Answer: Hmmm. BradC, do you have worked with Firebird/Interbase before - where the main backup/restore utility/API is more alike the "Copy Database..." of the SSMS/EM ? If so, know that MS SQL Server is NOT like it.
A SQLServer Backup is more a database dump which is restored "AS-IS" - so it's more like a confortable online shortcut for an "detach-copy-reattach on other place" operation. The restored database is almost an exact copy of the original database file (almost because you can change the placement of database files of an restored database)...


Why does Bach sometimes end with flat 7ths?

Question: I've been studying the Bach inventions, and I see a common patterns in the first 3 inventions: They tend to include the flat 7ths in the very end. Examples:
C major:

D major:

E♭ major:

From a music theoretical standpoint, it's confusing, but in my ears this harmony "sounds like the piece is ending". Does anyone here have an analysis on what's going on?
Edit: I understand that the music is modulating towards the subdominant but that seems counter-intuitive to do towards the very end of a piece. Bach left nothing to chance, and I'm looking for a way to motivate this choice from a theoretical point of view.
Comment: @Tim - No, I believe the OP is referring to the flattened 7th degree in each of the excerpts. The pieces are indeed in C, D, and E Flat major - the OP is referring to the excerpts' B flat, C natural, and D natural, respectively.
Comment: Surely it's just the b7 making the tonic move towards the subdominant briefly?
Comment: @Dekkadeci - you did mean Bb, C and Db respectively?
Comment: @Tim - Sorry, I did mean D flat.
Comment: The C, D and Eb are the last chords. Are you saying the pieces are in D E and F?
Comment: @Tim In that case you mean that it's just incidental that the flat 7 appears near the end of the pieces?
Comment: I would agree with Tim.
Answer: Well spotted! This is very common. Bach often uses a brief modulation to the subdominant key near the end of his fugues, preludes and inventions (presumably other pieces, too). Sometimes this is so brief, that we feel like we are just travelling through this key, without really modulating to it. Sometimes this is over a final tonic pedal, which is really "bringing the harmony home", although not in your examples.
The reason he does this, is related to how much of Western common practice music is harmonically structured. Although we may modulate to a wide range of related keys, by far the most commonly used structural modulations are basically an arch: Tonic -> Dominant -> Tonic (again). If we imagine this as a "swing" one place to the sharp side around the circle of fifths and then back again, the move to the subdominant helps to "balance" this, by swinging "a bit too far" to the flat side, as we return to the tonic, with the final settling on the tonic feeling even more like a resolution as a result.
Another way to think about the same structural harmonic movement, is considering the brightness of related keys. As we move to sharper keys around the circle of fifths they sound comparatively brighter; as we move to flatter keys they sounds comparatively darker. [Thanks to Tim for commenting on this, hence the edit for clarity...] We start at the tonic, our home key, move to a brighter key, the dominant, and then back to our home key. To emphasise this movement back from a brighter key to our home key, we go "a bit darker" than our home key before finally settling there.
We could come up with numerous analogies for this kind of movement: tuning a sharp guitar string down below the note we want, before coming back up to pitch; taking a clipping recorded volume down below the volume we want before gradually raising it to the optimum position etc.
Comment: Tim, I agree that no keys are inherently brighter or darker than any others, what I'm talking about here is the *relationships* between keys. The dominant *is* brighter than the tonic key; the subdominant *is* darker. It's the same relationship you get between modes one step away around the circle of fifths: the Lydian is brighter than the Ionian; the Mixolydian is darker - this is a modal way of thinking about the same Tonic/Dominant/Subdominant relationship (or thinking about the relationship between these keys over an unchanging root note...)
Comment: @Tim Part of that is that you're hearing modern instruments with equal temperament, where the difference in pitch from one note to the next is basically constant.  It is not mathematically possible for an instrument to be perfectly tuned in all keys, so we split the difference to get as close as possible.  Instruments in Bach's day would be in meantone temperament, where the further you get from C, the more out of tune it sounds.  There's debate over exactly what Bach intended by "Well-Tempered", but it is most likely not the same as modern Equal Temperament.
Comment: I'm quite sceptical on this 'sharper keys = brighter, flatter keys = darker' idea. Go a step further. In key F# (say). and moving to C#. Brighter? What about moving to Db instead? *really* darker? Never subscribed to any of this! Devil's advocate! Convince me!
Comment: All major keys are the same set of semitones apart, there should be no real change in sounds between the keys, there is just a lot of practical considerations in regards to keys choice.
Answer: In addition to what's been mentioned above, the motion to the subdominant also has a plagal sound. What's more final than a big "aaaa-men" plagal cadence at the end of a hymn? Sometimes Bach's flat-7s come after the authentic cadence. Sometimes before.
The use of the flat-7 was much more widespread in Renaissance polyphony and early baroque music. It was the natural function of some major modes to lower the 7th during motion downward in the scale. Baroque composers from the early 17th century would continue to pare down use of the flat-7 as music became less melodically- and more harmonically-driven. By Bach's time, you see he saves it for a special moment near the final cadence. (This is not to answer your question of WHY, but just to add some history.)
Comment: Thank you for your answer, I really appreciate it. Before the bars referenced in these screenshots there are indeed some hints of an authentic candence. So authentic candence following a plagal cadence, and then an authentic candence again to mark a super-final ending.
Answer: It's the same cliche as the 'Saints Go Marching In' ending.  Not sure if Bach ever went the whole hog with a iv as well as the I7 though!  And he generally did it over a tonic pedal, the modern(ish) version walks the bass line down.   Same idea though.  Decoration.  No functional analysis required.
C, C7, F, Fm, C, G7. C.
Comment: Well, everything must develop a BIT over a couple of hundred years!  They're certainly similar in just being decoration, no functional analysis required.
Comment: Interesting idea, but in this case F minor plays a really important role in this cliche ending and Bach omits it. It seems a little bit far fetched
Comment: I don't think the comparison is that clear. With "Saints Go Marching In", a big part of what makes that progression work is the chromatic decent. On the other hand, you can view the progression Bach uses in functional terms as I IV V I, with V/IV inserted before IV to strengthen the forward motion of the cadence. In other words the b7 makes the mode mixing sound better in the former case by providing a stepwise line, while it *resolves* functionally in the latter case.
Comment: As I've argued elsewhere, the **ⅳ** is best understood as a simplification of **ⅶ⁰**, and Bach definitely used that a lot in this context. Example, Cello suite 3 Prelude ends with C - C₇ - F/C - B⁰/C - {Gsus⁴-G-ish suspension, but sans G} - C. The B⁰/C could be interpreted as Fm₆j⁷/C.
Answer: It's not really ending with a flat seventh. I briefly tonicizes the subdominant, but ends most emphatically with a perfect cadence.
Moving to the dominant (adding sharps) has an effect of intensifying.
Moving to the subdominant (adding flats) has an effect of relaxing.
Of course the dominant and subdominant are the two tonal pillars on either side of the tonic so including both provides a kind of symmetry and tonal compliment to the main tonic. The question of order then is a matter of form and general sense of dramatic arch.
From a structural point of view the intensifying move to the dominant would logically come first to build toward a climax and the move to the subdominant would follow after the climax as a denouement before the final ending.
It's interesting to note the difference or ordering depending on the scale (scale in the sense of duration.) On the phrase level you see the subdominant center followed by the dominant in the QUISCENZA harmonic schema (exs. Bach's B flat invention and various Mozart sonatas) but on the compositional level (the whole composition) it's the opposite order.
Answer: Actually, I would not even call it a modulation (e.g.,  into a subdominant in the first example--the one I consider here) because for a progressiin to be called a modulation it needs to follow the cadential formula T-S-(D 6/4)- D-T in the new tonality.  What we see though is  a progression that stays in the original tonality, except for the non-harmonic tone B flat.  I think that the explanation for the use of this tone lies in modal scales, which Bach has used all the time.  In the mixolydian C scale, the seventh degree is B flat, and this is the only tone that makes it different from the C major scale, and so on its approaching to the final cadence, Bach uses the progressions in the mixolydian scale (T I 7 -- S IV--SII--DIII), and then employs T-S-T in C major 6 chords before concluding with the conventional T-S-DT6/4-D7-T.
This is, in my opinion "what" Bach is doing.  As to the "why," I think the only answer is: It spunds beautiful.  In the beginning of the 20th century many composers turned to using modal harmonies in addition to the tonal ones.
Answer: I think this is closely related to a widespread device known as "backdoor" cadence: Bb7 resolving to C rather than the obiquitous G7 resolving to C.
This could be preceded by a F chord, perhaps. Which would be a subdominant.
Or maybe it would be a previous tonal centre from which you return. In which case, the Bb7 works much like the Fm in When the Saints Go Marching In as described in a post above.
Answer: The C major example looks like it could be a modulation to the sub dominant (F Major), that would explain the Bb flat. The D major example, again looks like a modulation to the sub dominant G major, which is the only key that has a F# and a C natural. The last example looks like a quick modulation to Ab major for half a bar, that explains the D flat.
He ends with the key the piece was written in, it just modulates often and sometimes just for a measure. Modulations or the movement between keys is how you give movement to music. It is also how you make music interesting. Classical music rarely stays static in regards to keys.
Comment: Thank you for the answer Neil. However, that's only half an answer since it doesn't explain how that makes a more powerful ending. Bob's answer is so far a more complete hypothesis
Answer: This is nothing else than an extended perfect athentic cadence (the b7 appears as the Ib7 is used as V7/IV)
I - IV - I46 - V7 - I
Answer: It's not a modulation.   It's only a secondary dominant for IV.  An addition for an authentic cadence
Answer: I honestly can't believe the length and Amount of philosophical persiflage of these answers.  They're just plain common, garden-variety secondary dominants to add interest to the harmony, which may be one reason most people who play such music play Bach and not Georg Böhm or Johann Adam Rein[e]ken.  Also, the remark about dark flat keys and bright sharp keys, while having some truth, is deeply dependent, more so than personal emotional perceptions, on what tuning is being employed.  Bach alone, that is, as opposed to the tunings used in places where Bach did not work, used three different tuningS in his lifetime.  They were called the Chor-Ton, the Kammer-ton, and the Tief Kammer-Ton. These were not necessarily his choice. They varied as much as a major third, one from the other, not to mention how the split-key tunings were handled. So!   Easy on the deep-significance philosophisizig and a little more of old-fashioned musicology better answers the question. By the way, this is the first time I have heard of widespread bVII in renaissance music. There 's not one renaissance theorist who speak of chords, certainly not in a function way. Johann s Tinctoris would have seen what we call a simple c-major triad as three chords, one built on c, one on e, and one on g. As some of you clearly would know, it was Jean Philippe Rameau in 1722 who first explained that these three sounds were actually one chord, or, put another way, he was the first to formulate the invertibility of chords.
Peace
L
Comment: *By the way, this is the first time I have heard of widespread bVII in renaissance music.* Not sure what you mean by this. Bach isn't a renaissance composer. *They varied as much as a major third, one from the other, not to mention how the split-key tunings were handled.* This is also unclear to me. Are you talking about different ways of tuning the same keyboard instrument, like just tuning versus equal temperament, etc.? Clearly those can't differ by more than a fraction of a semitone.
Comment: Could you clarify your answer a little bit, with respect to how to speak about chords. You're talking about a secondary dominant, but you also say we can't talk about chords in a functional way. Isn't this a contradiction? How can you even have any kind of cadence without functional harmony?
Answer: In the C major and Eb major examples, this accidental also sets up an effective ii-V-I progression, to provide a stronger cadence at the end of the piece. The I7 chord would resolve authentically to IV, but these examples can be analyzed as that chord instead resolving deceptively to ii, and then moving around the circle of fifths back to I. By stringing together cadences like this, the piece has a stonier ending.


Reverberation in a recording to play with big speakers in a hall

Question: I am writing a piece in Sibelius and export the audio and play it in some room for people. I suppose the geometry of the room produces some reverberation of the sound coming from the speakers, probably large speakers.
What should I try to achieve in terms of reverberation in the sound I hear in the headphones such that the reverberation in the room with large speakers is nice but not excessive?
I imagine there is not a precise answer but this. Maybe the only way is to try it out. But what I am asking is a general vague guideline. Should I make the reverberation in the headphones slightly less than what I find ideally pleasant so that in the room speakers it sounds better (taking into account this equation in which larger volume of the room gives more reverberation)? Or does the, less than zero, coefficient in the equation tells me that the difference doesn't matter too much?
Sibelius 7 has several options according to the feel of reveb of the room one wants to obtain: Dry, small room, ..., Ambient Hall, ... , Cathedram. Should I put the option that sounds more pleasant in my heandphone or should I put less reverb so that playing it in the larger room with larger speakers is ideal?
Answer: Generally, yes - you would play it through headphones and set it up so it sounds as good as you can get, then back the reverb right off for playing it in a large room, if that room has a fair amount of echo anyway.
If the room isn't too large, is full of people or has sound deadening surfaces (wall hangings, carpets, etc) then don't worry about it too much, but if you are playing it in a church, for example, you won't want to have any reverb in the mix as you'll get plenty.
In this situation I might even save two versions, and give them both a test in sound check (remembering that empty room reverb will be more than a full room.)


javascript stopped working after adding another element to html

Question: I'm new in web development, so just cannot get a logic. Please help me.
I have this index.html file.

function unhide() {
  var x = document.getElementById("hiden").innerHTML;
  if (x.style.display === "block") {
    x.style.display = "none";
  } else {
    x.style.display = "block";
  }
}

function changemenu(x) {
  x.classList.toggle("change");
  unhide();
}
body {
  min-width: 700px;
}

.TopLine {
  position: absolute;
  z-index: -1;
  top: 0;
  width: 100%;
  height: 50px;
  background-color: black;
}

#withak {
  position: relative;
  top: 15px;
  color: white;
  font-size: 25px;
  font-family: Blackadder ITC;
  padding-left: 50px;
  z-index: 1;
  text-decoration: none;
}

.container {
  top: 15px;
  left: 95%;
  display: inline-block;
  cursor: pointer;
  position: absolute;
  z-index: 1;
  /*border:1px solid red;*/
}

.bar1,
.bar2,
.bar3 {
  width: 25px;
  height: 3px;
  background-color: white;
  margin: 5px 0;
  transition: 0.4s;
  /*border:2px solid red;*/
}

.change .bar1 {
  -webkit-transform: rotate(-45deg) translate(-5px, 6px);
  transform: rotate(-45deg) translate(-5px, 6px);
}

.change .bar2 {
  opacity: 0;
}

.change .bar3 {
  -webkit-transform: rotate(45deg) translate(-5px, -6px);
  transform: rotate(45deg) translate(-5px, -6px);
}

.topnav {
  position: absolute;
  top: 15px;
  left: 70%;
  z-index: 1;
  display: none;
}

.topnav a {
  color: white;
  font-size: 20px;
  padding-left: 15px;
  font-family: Blackadder ITC;
  text-decoration: none;
}

.topnav a:active {
  color: pink;
}

.content {
  position: relative;
  top: 50px;
  left: 20%;
  width: 60%;
  border: 2px solid red;
}

  WitHak

    About Me
    My Work
    Blog

About Me

Hello, happy to see you here, dear guest!

After adding div with class "content" JavaScript stopped working.
 Why? What I did wrong? So if cut out this:

About Me

Hello, happy to see you here, dear guest!

Without this extra div everything was working ideally. I want to add many other elements and don't lose beauty of scripting.
What script is doing. Three menu lines after clicking on them transforms to X and unhides top menu. Clicking on X will hide menu again.
Comment: You get `innerHTML` instead of the dom node `var x = document.getElementById("hiden").innerHTML;`. `innerHTML` has no `style` property.
Comment: can you include a working code snippet? I think in here you are missing the css file, also can you explain what was working and what are you attempting to achieve
Comment: your current code is still not working so first paste here your working code.
Comment: What exactly isn't working? Try closing the `` tag and see if it fixes your problem; like so: ``. Notice the `/>`.
Comment: @MariusBughiu `` is a [void element](http://w3c.github.io/html-reference/syntax.html#void-element). Doesn't need closing in HTML.
Comment: How to edit my question to add more details?
Comment: morilynx add css also
Comment: There are edit link option next line to your question tags javascript/html
Comment: Can you provide code example how to fix my problem?
Comment: yes sure I shall
Comment: @Morilynx please review an answer I have posted
Answer: There are 2 mistakes in your codes

in css you do not set float:left to .content class.

in unhide() function you hide html text content with var x = document.getElementById("hiden").innerHTML to store in x variable and that is a reason it shows an error of undefined x.display.style.

You should write var x = document.getElementById("hiden");

Please review working snippet.

// Code goes here

function unhide() {
  var x = document.getElementById("hiden");
  if (x.style.display === "block") {
    x.style.display = "none";
  } else {
    x.style.display = "block";
  }
}

function changemenu(x) {
  x.classList.toggle("change");
  unhide();
}
/* Styles go here */

body {
  min-width: 700px;
}

.TopLine {
  position: absolute;
  z-index: -1;
  top: 0;
  width: 100%;
  height: 50px;
  background-color: black;
}

#withak {
  position: relative;
  top: 15px;
  color: white;
  font-size: 25px;
  font-family: Blackadder ITC;
  padding-left: 50px;
  z-index: 1;
  text-decoration: none;
}

.container {
  top: 15px;
  left: 95%;
  display: inline-block;
  cursor: pointer;
  position: absolute;
  z-index: 1;
  /*border:1px solid red;*/
}

.bar1,
.bar2,
.bar3 {
  width: 25px;
  height: 3px;
  background-color: white;
  margin: 5px 0;
  transition: 0.4s;
  /*border:2px solid red;*/
}

.change .bar1 {
  -webkit-transform: rotate(-45deg) translate(-5px, 6px);
  transform: rotate(-45deg) translate(-5px, 6px);
}

.change .bar2 {
  opacity: 0;
}

.change .bar3 {
  -webkit-transform: rotate(45deg) translate(-5px, -6px);
  transform: rotate(45deg) translate(-5px, -6px);
}

.topnav {
  position: absolute;
  top: 15px;
  left: 70%;
  z-index: 1;
  display: none;
}

.topnav a {
  color: white;
  font-size: 20px;
  padding-left: 15px;
  font-family: Blackadder ITC;
  text-decoration: none;
}

.topnav a:active {
  color: pink;
}

.content {
  position: relative;
  top: 50px;
  left: 20%;
  width: 60%;
  border: 2px solid red;
  float: left;
}

  WitHak

    About Me
    My Work
    Blog

About Me

Hello, happy to see you here, dear guest!

Comment: Just how float stopped script from working, I don't understand. Magic? And when to use innerHTML and when not, what it is doing?
Comment: By the way it helped, so big thanks to you. Will try to get logic.
Comment: innerHTML is a function which may retrieve HTML content so whenever you required storing element reference just use var xyz = document.getElementById("idenOfElement"); like that way and later on you may use xyz.innerHTML like method you should use.
Comment: still let me what is the issue to access an answer I shall try to help you.
Comment: How did you noticed that I miss float:left, because not only innerHTML was the reason of my problem. Where is the logic? How do you know that I need float: left?
Comment: because if I put simple HTML with no content without CSS it's working so I replace it with your content without CSS, it 's working so definitely there is an issue with CSS. That is the way I find it.


Differentiation Under the Integral Sign w Variable Substitution

Question: Let $f\in C^1(B_\rho(\xi))$, $\xi\in\mathbb{R}^n$ and $\rho>0$. I wish to show
$$ \frac{d}{d\rho} \int_{B_\rho(\xi)} f(x)\ dx = \frac{1}{\rho} \int_{B_\rho(\xi)} nf(x) + x\cdot Df(x)\ dx $$
I'm using the following fact: let $f\in C^1(\mathbb{R}^n)$, $\psi\in C^1(\mathbb{R}^{n+1})$ where $\psi(y,\rho) = f(y\rho)$ for $y\in\mathbb{R}^n$ and $\rho>0$. Then
$$ \frac{\partial}{\partial\rho} \psi(y,\rho) = \frac{1}{\rho} \sum_{i=1}^n y_i \frac{\partial \psi(y,\rho)}{\partial y_i} $$
I've begun with the variable substitution $x=y\rho$ which implies $f(x) = f(y\rho) = \psi(y,\rho)$. Then
$$ \frac{\partial}{\partial\rho} f(x) = \frac{\partial}{\partial \rho} \psi(y,\rho) = \frac{1}{\rho} \sum_{i=1}^n y_i \frac{\partial \psi(y,\rho)}{\partial y_i} $$
Here's the part I'm stuck on. Is the following correct?
$$ \frac{1}{\rho} \sum_{i=1}^n y_i \frac{\partial \psi(y,\rho)}{\partial y_i} = \frac{1}{\rho} \bigg[x \cdot Df(x)\bigg] $$
I think I'm forgetting an important step with the $B_\rho(\xi)$ in the bounds of integration and/or the $dx$ with the variable substitution.
Answer: \begin{align}
\frac d{d\rho}\int_{B_\rho(\xi)}f(x) \ dx
&\overset{1.}{=} \frac{d}{d\rho}\int_{B_1(0)} f(\xi + \rho y) \ \rho^{n}dy
\\&\overset{2.}{=} \int_{B_1(0) }y\cdot \nabla f(\xi+\rho y)\rho^n + f(\xi+\rho y) \  n\rho^{n-1} \ dy
\\&\overset{3.}{=} \frac{1}{\rho} \int_{B_\rho(\xi)} nf(x) + (x-\xi)\cdot Df(x)\ dx\end{align}
where:

Change of variables $x-\xi= \rho y $, means that $y$ is in $B_1(0)$, and $dx = \rho^n dy$.
Follows from product rule, $$ \frac d{d\rho}(fg) = (\frac d{d\rho} f )g + f \frac{d}{d\rho} g  $$and chain rule,
$$ \frac d{d\rho} f(\vec X(\rho)) = (\nabla f)(\vec X(\rho)) \cdot\left(\frac d{d\rho} \vec X (\rho)\right).$$
Undoing the change of variables $x-\xi= \rho y$. Note $\rho^{n-1} dy = \frac1\rho dx$.

$Df = \nabla f$ is the gradient of $f$. Regularity is used in step 2 to allow differentiation under the integral sign.
There is an additional term which is a multiple of
$$ \int_{B_\rho (\xi)} \xi \cdot \nabla f = \int_{B_\rho(\xi)} \nabla_x \cdot (\xi f(x)) \ dx = \int_{\partial B_\rho (\xi)} f \xi\cdot n $$
Comment: @Surb it should be. I guess there's a clever way of using Gauss theorem to show this but I don't see it
Comment: I think the OP forgot $(x-\xi)$ (or maybe $\xi=0$). Otherwise I don't think that his formula would be correct.
Comment: @Surb thank you very much for for comments.
Comment: Great answer. I will erase mine after. I'm curious about something : is it equal to $\int_{\partial B_\rho(\xi)}f(x)dx$ ? (see my answer)
Comment: @Surb, I have been careless and I don't think the answer before was right. Do you think the claimed formula is still correct, and if yes, how would you remove this extra term I found? I feel like this makes more sense, since $x-\xi$ is normal to $\partial B_r (\xi)$


Concatenate files with prefix

Question: Assume a folder with files named like this:
FOO.1
FOO.2
...
BAR-1.1
BAR-1.2
...
BAR-2.1
BAR-2.2
...

And I would like to concatenate them such that it results in 3 files:
FOO (consisting of FOO.1 + FOO.2 + FOO.N)
BAR-1 (consisting of BAR-1.1 + BAR-1.2 + BAR-1.N)
BAR-2 (consisting of BAR-2.1 + BAR-2.2 + BAR-2.N)

How would this be done in bash/shell script? Assume all the files are in one folder (no need to go into subfolders)
Requires not knowing the filename prefixes in advance
Answer: for file in *.*
do
  prefix="${file%.*}"
  echo "Adding $file to $prefix ..."
  cat "$file" >> "$prefix"
done

Answer: for i in  $(ls | sed  's/\(.*\)\..$/\1/' | sort -u)
do
  cat $i* > $i
done

Comment: Requires not knowing the filename prefixes in advance - edited above to reflect that

Concatenate files with prefix


Using paired t test if data is not normally distributed but the data set is large

Question: I need to test some before and after data, to see if there is a significant shift (n = 114) but the data differences are not normally distributed (or, at least, a histogram would seem to indicate that). Can I use a paired t-test?
Comment: 114 is not *that* large.
Answer: Have you thought about a non-parametric test such as the Mann-Wilcoxon? You could test the before and after distribution to see if the average value has shifted significantly in one direction or the other.
Edit
@Scortchi is correct in that in your specific case, the Wilcoxon signed rank test would be a more appropriate non-parametric test.
Comment: Yes, I have considered using the Wilcoxon Signed Rank Test, but without a data analysis package like SPSS that is very tedious. I am hoping to use t tests as they are easy to complete using an Excel formula
Comment: If you have `R` it is easy. The [coin](http://cran.r-project.org/web/packages/coin/index.html) package for R has a lot of non-parametric tests with both exact and asymptotic power calculations. I've done MWW tests in Excel; it isn't that difficult using RANK. If you have ties, you'll have to use some IF statements.
Comment: @Avraham: Note they're *paired* observations: you'd want the [Wilcoxon signed-rank test](https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) to test the null hypothesis that the mean difference within *each pair* is zero. And the OP might want to consider relying on the central limit theorem to justify the $t$-test approximately - it's quite a large sample size.
Comment: @Scortchi Yup; I should have been more precise in the specific non-parametric test. Personally when I use them, it is in the context of demonstrating changes in claim severity/frequency or aggregate loss distributions to indicate if prior experience for a given insurer may no longer be so credible vis-a-vis reinsurance pricing. As such, I'm not dealing with paired observations. Also, while the sample size may make the $t$ test still OK, the OP says the data does not look normal, and if e.g. there is heavy skew, the t wouldn't work so well, but non-parametric tests would still be appropriate.


Como formatar um número double para N decimais sem arredondar?

Question: Eu tenho um valor do tipo double e gostaria de formatá-lo para que fique com 3 decimais apenas, sem arredondamento. Para isso, eu estou tentando utilizar o String.format da seguinte maneira:
float value = Float.parseFloat(String.format("%.3f", myDoubleValue));

O problema é que quando eu executo esse código, o seguinte erro é gerado:
Exception in thread "main" java.lang.NumberFormatException: For input string: "-23,319"
        at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
        at sun.misc.FloatingDecimal.parseFloat(Unknown Source)
        at java.lang.Float.parseFloat(Unknown Source)

O que eu estou fazendo de errado? Existe uma maneira "ideal" para realizar essa formatação? Sei que existem outras formas de se formatar os decimais — como o DecimalFormat — porém elas arredondam o número, e não é isso o que eu desejo.
Detalhe: para esse projeto, eu preciso do máximo de otimização possível. Então eu gostaria de uma solução "leve" para formatar o número.
Comment: Você quer gerar uma string ou um `float` com as 3 primeiras casas decimais? Apesar de parecer, não é a mesma coisa, porque "formatar" geralmente quer dizer "transformar em string", podendo mudar o separador decimal e dos milhares, etc. Inclusive, esse foi o erro: `String.format` usa o locale default da JVM, que no seu caso é um que usa a vírgula (já expliquei isso [na sua outra pergunta](/q/510792/112052)), e `parseFloat` só reconhece o ponto.
Comment: De qualquer forma, `Decimalformat` tem a [opção de mudar a forma de arredondamento](https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/text/DecimalFormat.html#setRoundingMode(java.math.RoundingMode)), teste aí (acho que é o `DOWN`, mas veja aí). Mas se quiser um `float`, acho mais fácil fazer a conta mesmo: https://pt.stackoverflow.com/a/459884/112052 (essa é em JavaScript, mas a ideia é a mesma)
Comment: De qualquer forma 2: `String.format` arredonda sim :-)
Answer:
elas arredondam o número, e não é isso o que eu desejo

Bom, String.format também arredonda:
// quarta casa decimal é >= 5, arredonda para cima
System.out.println(String.format("%.3f", 1.2225)); // 1,223

// quarta casa decimal é < 5, arredonda para baixo
System.out.println(String.format("%.3f", 1.2224)); // 1,222

Creio que quando você diz "não arredondar", quer dizer que o resultado deveria ser 1,222 para ambos os casos, certo?
Vale lembrar que o separador decimal poderá ser a vírgula ou o ponto, dependendo do locale default da JVM, conforme já explicado em outra pergunta. Mas quando você usa String.format, o arredondamento sempre é feito, inclusive isso é dito na documentação:

the value will be rounded using the round half up algorithm

E o tal algoritmo de arredondamento citado é descrito aqui.

De qualquer forma, o que exatamente você quer fazer?
Se você quer obter um float somente com as 3 primeiras casas decimais, aí só precisa usar a boa e velha matemática:
// manter apenas as N primeiras casas decimais
static float manterCasas(float valor, int qtdCasas) {
    float fator = (float) Math.pow(10, qtdCasas);
    return (float) (Math.floor(valor * fator) / fator);
}

public static void main(String[] args) {
    // sempre mantém as 3 primeiras casas, sem arredondar
    System.out.println(manterCasas(1.2225f, 3)); // 1.222
    System.out.println(manterCasas(1.2224f, 3)); // 1.222
}

A ideia para manter as N primeiras casas é multiplicar o número por 10N, arredondar para baixo e em seguida dividir por 10N (conforme já explicado aqui e aqui).

Mas se você quer uma String com as 3 primeiras casas decimais, controlando inclusive o separador decimal, aí pode usar DecimalFormat mesmo:
public static void main(String[] args) {
    System.out.println(formatar(1.2225f, 3)); // 1.222
    System.out.println(formatar(1.2224f, 3)); // 1.222
}

static String formatar(float valor, int qtdCasas) {
    NumberFormat fmt = DecimalFormat.getNumberInstance();
    // limitar quantidade de casas decimais
    fmt.setMaximumFractionDigits(qtdCasas);
    // arredondar para baixo tem o mesmo efeito que "não arredondar a última casa decimal"
    fmt.setRoundingMode(RoundingMode.DOWN);
    return fmt.format(valor);
}

Lembrando que o separador decimal pode ser o ponto ou a vírgula, de acordo com o locale default da JVM. Se quiser um específico, pode passar o locale para o NumberFormat. Ex:
// usar locale pt-BR (português do Brasil) para o separador decimal ser a vírgula
NumberFormat fmt = DecimalFormat.getNumberInstance(new Locale("pt", "BR"));

Lembrando que por padrão o separador de milhares também é colocado (1000 é formatado como 1.000). Se quiser desabilitar isso, faça fmt.setGroupingUsed(false);.
Outro detalhe é que se o número tiver menos que 3 casas decimais, não serão mostradas todas. Se quiser que sempre seja mostrado 3, basta adicionar fmt.setMinimumFractionDigits(qtdCasas);.

Por fim, vale lembrar de não confundir um número com a sua representação. Um número como 1,5 é apenas um valor... numérico que não possui formato em si (é só um conceito, uma ideia: ele representa um determinado valor, só isso). Mas este mesmo número pode ser representado de várias formas: como 1.5, 1,5, um e meio, 000001.500, etc.
Ou seja, um valor numérico guardado em um float (ou int, ou double, etc) não tem formato, mas ele pode ser convertido para uma string que representa este valor de diferentes formas. Quando eu imprimo o número, ele é convertido para uma string em determinado formato (com println, por exemplo, eu não controlo o formato, mas com format e DecimalFormat sim). Então é importante saber o que você precisa: mudar o valor numérico (gerar outro float somente com as 3 primeiras casas decimais) ou apenas gerar uma string que representa o número com as 3 primeiras casas.
Answer: Tente usar o NumberFormat
   Double variavel_qualquer = 5.5641;
   NumberFormat formatter = new DecimalFormat("#0.00");
   System.out.println(formatter.format(4.0));
   System.out.println(formatter.format(variavel_qualquer));

Resposta:
4.00
5.56


entity_load_multiple returns empty array although an eitity lives in the database?

Question: I am trying to debug something related to an issue I am facing. The bottom line is that entity load is not returning the peroper values for some reason. For example:
$cids = array(119570363);
$comments = comment_load_multiple($cids);
dpm($comments);

return an empty array although the comment with id=119570363 exists in the database. I traced entity load function the the comment controller which inherits the load method from DrupalDefaultEntityController.While looking at the method I failed to notice anything that would cause this. Perhaps I am missing something. Are there implicit conditions on the load? Also, I tried with both $reset = FALSE and $reset = TRUE .
Comment: comment_load will only load a comment if the node and the user linked to the comment both exist. Are you sure they do ? What happens if you run this query on your database: SELECT cid FROM comment INNER JOIN node ON node.nid = comment.nid INNER JOIN users ON users.uid = comment.uid WHERE cid = 119570363;
Comment: The same code works for me - there is an issue elsewhere. I notice the comment id is quite high - do you really have that many comments on your site ?
Comment: yes 90 000 000 + comments
Comment: Thank You, this answer my question `Are there implicit conditions on the load?`. I actually wrote a script that get the ids and check those that don't load using entity load. Some are missing users and others are yet to be discovered. Step debuggin all the way. Thanks Again, submit the answer and I shall accept your answer. Can you please tell me how you did find out about the user? because  I traced all the methods and did not see anything about it nor anything in the documentation.
Answer: comment_load_multiple will only load a comment if the node and the user linked to the comment both exist.
You can see this in the code for CommentController::buildQuery where it does an INNER JOIN on both the node and user tables. Note that if you trace through the execution of comment_load_multiple you have to pass in the RESET argument (or clear the caches before hand) otherwise you will get the cached result, and will never make it into CommentController::buildQuery.
Now the reason CommentController::buildQuery does an INNER JOIN rather than a LEFT JOIN (and the reason I didn't add this as an answer straight away) is that you are not supposed to be in a situation where you have a comment that does not have a node or a user.
You can see this because:

The code for comment_save clearly expects the associated node and user to be set ;
The comment module implements hook_node_delete, and always ensures that comments are deleted when a node is deleted ;
The comment  module implements hook_user_delete, and always ensures that comments are deleted when a user is deleted.

So the problem you are seeing suggests your database is somewhat inconsistent. This could happen if you deleted nodes or users without using the proper API, though it could happen for other reasons (eg. sometimes when dumping and re-saving MySql database you might loose the entry for the anonymous user - it has id 0, and must be in the database users table).
Another note: If this is the only issue affecting the loading then you can get the comments that won't load by doing an SQL query:
SELECT cid
FROM comment
LEFT JOIN node ON node.nid = comment.nid
LEFT JOIN users ON users.uid = comment.uid
WHERE (node.nid IS NULL)
      OR (users.uid IS NULL)

Comment: Thank you. It is migrated database which explains why this happens.


How to eMail a file uploaded to a Webform to a sysadmin using the Rules module?

Question: I have a webform where a user can upload a file to the webform. I would like to be able to email the file via Rules to a sysadmin. I know I can send it from Webform directly, but I am also needing to attached a FillPDF file that can only be sent via Rules.
Answer: Yes it is possible to use the Rules module for sending an eMail with an attached file. To do so, make sure you have the Mail System and Mime Mail modules installed/enabled.
The Rules Event you want to use is probably when the Webform is submitted.
Then create a rule with a Rules Action to Send an HTML email. Specify the eMail body content, and then the email's attachment(s). If you have multiple attachments, make sure to specify 1 file (only) per line. The tricky part you probably run into is to correctly specify the filename of the attached files. The correct way to do so is like so:
<code>[node:field-file-attachments:file:path]
</code>
That is also what is suggested in Comment # 25 of support issue (not bug ...) #1431572.
In your case, make sure that this path refers to the webform that is submitted and contains the attached file. Using the "Webform Rules" module, you should be able to have Rules react on an event like "Webform submitted". And if you then add a Rules Condition like "entity has field" (with the field corresponding to your attached file), you should be able to "do something" with that filename of your attachement. E.g. trigger another Rules Component that uses as a parm ... the path of your attached file. With that, your Rules Component should then take care of sending the actual eMail, with the correct value of the filename for the file to be attached.
Use Rules debugging
This question is pretty tough to answer with the limited info you provided (and without access to "your" site). Therefor I suggest you also consider 2 variations of Rules debugging, which are:

Within the rules settings, experiment a bit with the debugging facilities that come with the rules module. At least have a look at the "Rules Evaluation Log" (to check that the Rules logic being executed, actually matches what you expect that logic to be).
(Temporary) enable the Devel module, so that you can (temorary) add a Rules Action like "Display a value" (in your case of the entire node/webform that's used to submit that attached file).

For way more details about these 2 variations, refer to Rules debugging, as explained in the answer to "How can I display the value of a variable inside a condition?".
Comment: Can you explain a bit more the [node:field-file-attachments:file:path]? Am I to place that as is or do I need to change the values to reflect my specific need.
Comment: Thank you Pierre but that is where I run into a problem. Since the user is the one uploading the file I do not know the name of the file. The goal is to send the file that the user updates when completing the form (ie. the contents of the file field) via Rules to an admin.
Comment: Hm, I don't give up yet ... I might be able to enhance my answer if you first better explain how you implemented your "*... upload a file to the webform ...*". Is that via some field you added to your webform? If so what's the filetype. And also should they add (upload) such file before they submitted the form? And is it optional (as per our "can"). If it's something else, please explain how you implemented it then instead.
Comment: I am using the Webform multiple file module. I did find a patch that let's me rename the files. What I have done is had the rename to the submission serial number and then "attachment" (ie.1234_attachment.jpg). Now am trying to figure out how to mail that attachment using Rules.
Answer: The video tutorial Send email with attachments via Rules explains how to use Rules and modules Mail System & Mime Mail for sending emails with attachments
I tried it before and it works.
Comment: Thanks. Saw that video in my quest but my issue lies where I do not know what the file name will be. The goal is to email the file that the user uploads when completing the form (ie. the contents of the File field).
Comment: Whilst this may theoretically answer the question, [it would be preferable](http://meta.stackoverflow.com/q/8259) to include the essential parts of the answer here, and provide the link for reference.


ordem em que WebApp.connectHandlers.use é avaliado

Question: Numa aplicação de teste, tenho vários registros de dns apontando para ela, e tenho uma pasta onde guardo códigos específicos para cada origem.
O que pretendo é evitar a possibilidade de ser requisitada algum conteúdo de um site por outro, mas o problema é que todas as consultas para os elementos internos do Meteor são avaliados primeiro.
Ex:
/imports/@/bulica.local/index.js
/imports/@/caneco.local/index.js
/imports/@/192.168.0.1/index.js

e em /server/index.js logo nas primeiras instruções:
WebApp.connectHandlers.use('/imports/@/', (req, res, next) => {
  //  aqui vai minha lógica que compara
  //  req._parsedUrl.pathname com req.headers['x-forwarded-server']
});

A questão é que é possível um dominio acessar codigo de outro pela url, porque o Meteor avalia sempre o que existe em WebApp.clientPrograms['web.browser'] antes de tudo, entregando o arquivo. Essa minha lógica só é processada se a url não casar com nada que o Meteor já tenha "compilado"
Como resolver isso? Eu não tenho a menor idéia de como fazer isso no Apache, que também me ocorreu
Answer: Encontrei.
No Meteor existe uma outra instância do connect da que não está na documentação que é processada antes do código criado pelo desenvolvedor. É a WebAppInternals ficando assim:
WebAppInternals.connectHandlers.use ...
de resto é igual


wildcard DNS not found only from some desktops

Question: We built an easy website with a form and a thanks page: http://healthcity.giveitatry.nl/ . This works like a glove on our computers. Using a CMS the admin of the site can create several of these forms, for example sportsclubx.giveitatry.nl or sportsunlimited.giveitatry.nl (these last 2 are examples. not yet made).
To get the subdomains to work we have 2 DNS A-records:
 A  *.giveitatry.nl 126.96.36.199
 A  giveitatry.nl 126.96.36.199

And on the server we have the following Virtual host:
    <VirtualHost *:80>
            DocumentRoot "/var/www/giveitatry"
            ServerName giveitatry.nl
            ServerAlias *.giveitatry.nl
            <Directory "/var/www/giveitatry">
                allow from all
                Options +Indexes
                AllowOverride All
            </Directory>
    </VirtualHost>

On all our computers but one in our office we can go to the url using several browsers and even using ping etc... But there is one MAC that gets SERVER NOT FOUND. We have flushed every DNS cache we could think of on his computer but that didnt's change anything. Several computers belonging to our customer have the same error or DNS not found or ERR_NAME_NOT_RESOLVED. It appears as though they never get to our server's IP.
Ill add my entire Virtual host config if maybe someone needs it:
    <VirtualHost *:80>
    DocumentRoot /var/www/giveitatry
    </VirtualHost>

    <VirtualHost *:80>
    DocumentRoot "/var/www/xxxxxx"
    ServerName xxxxxx.nl
    ServerAlias www.xxxxx.nl
    <Directory "/var/www/xxxxx">
    allow from all
    Options +Indexes
AllowOverride All
    </Directory>
    </VirtualHost>

     <VirtualHost *:80>
        DocumentRoot "/var/www/"
        ServerName xxxxxx.nl
        ServerAlias www.xxxxx.nl
        <Directory "/var/www/">
        allow from all
        Options +Indexes
    AllowOverride All
        </Directory>
        </VirtualHost>

        <VirtualHost *:80>
                DocumentRoot "/var/www/giveitatry"
                ServerName giveitatry.nl
                ServerAlias *.giveitatry.nl
                <Directory "/var/www/giveitatry">
                allow from all
                Options +Indexes
            AllowOverride All
                </Directory>
                </VirtualHost>

<VirtualHost *:12322>
    SSLEngine on
    ServerAdmin webmaster@localhost
    DocumentRoot /usr/share/adminer/adminer
    Alias /adminer/static /usr/share/adminer/adminer/static
    Alias /externals /usr/share/adminer/externals
    Alias /editor   /usr/share/adminer/editor
</VirtualHost>

<Directory /usr/share/adminer/adminer>
    Options FollowSymLinks
    DirectoryIndex index.php

    <IfModule mod_php5.c>
        AddType application/x-httpd-php .php

        php_flag magic_quotes_gpc Off
        php_flag track_vars On
        php_flag register_globals Off
        php_value include_path .
    </IfModule>
</Directory>

<Directory /usr/share/adminer/editor>
    Options FollowSymLinks
    DirectoryIndex index.php

    <IfModule mod_php5.c>
        AddType application/x-httpd-php .php

        php_flag magic_quotes_gpc Off
        php_flag track_vars On
        php_flag register_globals Off
        php_value include_path .
    </IfModule>
</Directory>

Comment: I'm able to reproduce this by looking up using google DNS (`dig foo.giveitatry.nl @22.214.171.124` fails), both on OSX and Linux. Looking up www.giveitatry.nl works. http://dnsviz.net/d/foo.giveitatry.nl/dnssec/ has some errors you might want to look into, both the PMTU errors and the NSEC3 errors look relevant.
Comment: Look like there is problem on client's side. My MacBook works ok with your site. Could you try to execute `nslookup healthcity.giveitatry.nl` on problem Mac?
Comment: Hi Alexander, I am the one with the mac ;) for every nslookup on domain giveitatry.nl, I get the right ip address. For all subdomains, the response is: host healthcity.giveitatry.nl is not found: NXDOMAIN
Comment: @TollefFogHeen Thanks, this looks interesting. I'm not familiar with what this means yet, but I'll find out! keep you posted!
Answer: DNS wildcards and DNSSEC do not always play nice together, and you are in this case for your domain. Have a look at http://0skar.cz/dns/en/ to test things, specifically for the computers not able to reach your websites. Also make sure to check which recursive nameserver they are using.


Find the value of equation?

Question: Suppose
\begin{align*}
  a+b+c &= 20\\
  \frac 1a + \frac 1b +  \frac 1c &= 30
\end{align*}
Then find the value of
$$
\frac ab + \frac ba + \frac ac + \frac ca + \frac bc + \frac cb
$$
How i can apply A.M. $\ge$ G.M $\ge$ H.M.  in this?
How i can achieve this?
Comment: Shouldn't it be $\frac ab + \frac ba + \frac ac + \frac ca + \frac bc + \frac cb$?
Comment: Why do you think the AM-GM-HM inequality will help?
Comment: Even if it does say that, are you sure it's not a typo?  Seems odd to have 2 separate $\frac bc$'s
Comment: @Mike ya.i think that was a error.There was a typing mistake in book.
Answer: I'm guessing that first $\frac bc$ should be $\frac ba$.  I don't know what A.M., G.M., or H.M. are.  But as for how to come up with a solution, I can see good things will happen if you multiply both equations together.
$(a+b+c)(\frac1a+\frac1b+\frac1c)=a(\frac1a+\frac1b+\frac1c)+b(\frac1a+\frac1b+\frac1c)+c(\frac1a+\frac1b+\frac1c)$
I'm thinking you should be able to take it from there.
Comment: No problem.  It looks like someone actually beat me to posting the answer and erased it though.
Answer: $$\frac ab + \frac ba + \frac ac + \frac ca + \frac bc + \frac cb $$ can be simplified as
$$ \frac{a^2c+b^2c+a^2b+bc^2+ab^2+ac^2}{abc} $$
which is
$$ \frac{\left[ac(a+c)+bc(b+c)+ab(a+b)\right]}{abc}$$
and that is
$$ \frac{\left[ac(20-b)+bc(20-a)+ab(20-c)\right]}{abc}$$
$$ = \frac{\left[20(ab+bc+ca)-3abc\right]}{abc}  $$
But
$$\frac{1}{a}+\frac{1}{b}+\frac{1}{c}=30$$ implies
$$ ab+bc+ca=30abc$$
So you can do further simplification to get $597$
Answer: Exploit the innate symmetry! This is solvable purely mechanically, i.e. no lucky guesses are needed. The rational functions are all symmetric functions of $\rm\:a,b,c.\:$ By the Fundamental Theorem of Symmetric Polynomials, every symmetric polynomial is a polynomial in the elementary symmetric polynomials $\rm\:e_i.\:$ There is a simple effective algorithm due to Gauss to compute this representation. Putting all over a common denominator, we are given
$$\rm e_1\ =\ a + b + c$$
$$\rm \frac{e_2}{e_3}\ =\ \frac{ab + bc + ca}{abc}\ =\ c^{-1} + a^{-1} + b^{-1}$$
and we seek
$$\rm \frac{f(a,b,c)}{e_3}\ =\ \frac{a^2 (b+c) + a\: (b^2+c^2) + b^2 c + b c^2 }{abc}$$
Now use Gauss's algorithm to rewrite $\rm\:f\:$ in terms of the elementary symmetric polynomials $\rm\:e_i.\:$ This works as follows: if  $\rm\ a^i\ b^j\ c^k\ $ is the highest term w.r.t. lex order $\rm\ a > b > c\ $ then kill the highest term of $\rm\:f\:$ by subtracting $\rm\ e_1^{i-j}\ e_2^{j-k}\ e_3^k\:.\:$ Here, since $\rm\ a^2 b^1 c^0\: $ is the highest term in $\rm\:f,\:$ we subtract from $\rm\:f\:$ the term $\rm e_1^{2-1}\ e_2^{1-0}\ e_3^0\ = (a+b+c) (ab+bc+ca),\:$ leaving $\rm\: -3abc = -3\:e_3.\:$ Thus we infer$\rm\:f = e_1 e_2 - 3\:e_3,\:$ so $\rm\:f/e_3 = e_1 (e_2/e_3) - 3\: =\: 20\cdot 30 - 3\: =\: 597.$
While this problem is so simple that one may be lucky enough to stumble upon this representation, one will rarely be so lucky with problems of even slightly greater complexity.

Find the value of equation?


How to find a $\delta>0$ from information in a graph so that $0<|x-2|<\delta$ implies $|\frac1x-0.5|<0.2$?

Question: Use the graph below to find a $\delta \gt 0$ such that
$$
0 \lt \lvert x -2 \rvert \lt \delta \Rightarrow \lvert \frac {1}{x}-0.5\rvert \lt 0.2 $$

Provided answer is $\delta = \frac {4}{7}$

What I have done:
Given $\varepsilon \gt 0, \varepsilon = 0.2$, for $0 \lt \lvert x -2 \rvert \lt \delta \Rightarrow \lvert \frac {1}{x}-0.5\rvert \lt \varepsilon $
$\lvert x-2 \rvert \\= \lvert (x)(1- \frac{2}{x}) \rvert \\= \lvert (-2x)(\frac{1}{x}-0.5) \rvert$
Since $\lvert x-2 \rvert \lt \delta$ and $\lvert \frac {1}{x}-0.5 \rvert \lt 0.2$
$0.2\lvert -2x \rvert \lt \delta$
$\lvert 2x \rvert \lt 5\delta$
$-5\delta \lt 2x \lt 5\delta$
Comment: The question says "Use the graph", and you should really do.
Comment: @mfl Whoops. Edited the title.
Comment: In your question, should it be *assumed* that $\epsilon=.2$?
Comment: @Jack My bad. Instead of $\varepsilon$, it should be 0.2
Comment: I have edited the title of your post since your question is not about a proof of $\lim_{x\to 2}\frac{1}{x}=\frac{1}{2}$ *per se*.
Answer: $$|\frac {1}{x}-0,5|=|\frac {x-2}{\color {green}{2}x}|$$
as we are near $2$, we can assume that
$|x-2| <\color {red}{1}$ or $1<x <3$.
thus
$$|\frac {1}{x}-0,5|<\frac {|x-2|}{\color {green}{2}} $$
hence, to get $$|\frac{1}{x}-0,5|<\epsilon$$
we just need to have
$|x-2|<\color {red}{1}$ and $|x-2|<\color {green}{2}\epsilon $.
So we will take
$$\delta=\min (\color {red}{1},\color {green}{2}\epsilon) $$
Answer: The questions says "use the graph ...". We are looking for a value $\delta$ such that if $2-\delta<x<2+\delta$ then the graph of the function is between $0.3$ and $0.7.$ It is clear that if $$\dfrac{1}{0.7}<x<\dfrac{1}{0.3}$$ then the graph satisfies the conditions. Now $\delta$ is given by the minimum distance from $2$ to $\frac{10}{7}$ and $\frac{10}{3}.$ Since
$$\left|2-\dfrac{10}{7}\right|=\dfrac 47<\dfrac 43=\left|2-\dfrac{10}{3}\right|$$
we get that $\delta =\dfrac 47.$
Comment: Sorry but I think there may be a mistake. Isn't $\frac {10}{7}-2 = -\frac {4}{7}$
Comment: @KevinKoh Yes. Sure. Thank you for pointing it. I will fix it in a moment.
Comment: Could you further elaborate on this?

_Now $\delta$ is given by the minimum distance from $2$ to $\frac {10}{7}$ and $\frac {10}{3}$.
Since

$\lvert 2− \frac{10}{7} \rvert=\frac{4}{7} \lt \frac {4}{3} = \lvert 2− \frac{10}{3} \rvert$

We get that  $\delta=\frac{4}{7}$._
Comment: If you look at the graph there are two important points: $10/7$ where the function takes the value $0.7$ and $10/3$ where the function takes the value $0.3.$ Since you want to have the graph between those values by a condition on $|x-2|$ you have to choose the most restrictive condition. That is, the minimum distance to $2.$
Answer: We will prove the stronger statement

For every $\epsilon > 0$, there exists $\delta > 0$ such that, if
  $|x-2| < \delta$, then $\left| \dfrac 1x - \dfrac 12 \right| < \epsilon$

Let $\delta = \min\{ 1, \epsilon\}$.
Then $0 < \delta < 1$. It follows that $2-1 < 2 - \delta$ and $2 + \delta < 2+1$.
\begin{align}
   |x-2| < \delta
   &\implies 2-\delta < x < 2 + \delta \\
   &\implies 1 < x < 3 \\
   &\implies \frac 13 < \frac 1x < 1 \\
      &\implies \frac 16 < \frac{1}{2x} < \frac 12
\end{align}
It follow that
\begin{align}
   \left| \dfrac 1x - \dfrac 12 \right|
   &= \left| \dfrac 12 - \dfrac 1x \right| \\
   &= \dfrac{1}{2x}|x - 2| \\
   &< \dfrac 12|x - 2| \\
   &< \dfrac 12 \delta \\
   &< \dfrac 12 \epsilon \\
   &< \epsilon
\end{align}
Answer: Such exercise suggests students to think about the geometric meaning of the $\epsilon$-$\delta$ proof instead of blind algebraic manipulations.

Find a $\delta \gt 0$ such that $0 \lt \lvert x -2 \rvert \lt \delta \Rightarrow \lvert \frac {1}{x}-0.5\rvert \lt 0.2 $

Note that $0<|x-2|<\delta$ means
$$
x\in(2-\delta,2+\delta)\quad\hbox{and }\ \ \ x\neq 2,
$$
which means that $x$ falls in the interval $(2-\delta,2+\delta)$ (except the center of the interval: $2$).
On the other hand,
$\lvert \frac {1}{x}-0.5\rvert \lt 0.2$ means
$$
\frac1x\in(0.5-0.2,0.5+0.2)=(0.3,0.7).
$$
(Can you see that how this coincides with the interval on the $y$-axis of the graph?)
Now look at the graph, which suggests (make sure that you understand this!) that if $x\in(\frac{1}{0.7},\frac{1}{0.3})$, then
$$
\frac{1}{x}\in(0.3,0.7)\tag{*}
$$
But wait, we are looking for $(2-\delta,2+\delta)$ so that if $x\in (2-\delta,2+\delta)$ then (*) is true. ($x\neq 2$ is really irrelevant.) Now observe that if we can make
$$
(2-\delta,2+\delta)\subset (\frac{1}{0.7},\frac{1}{0.3})
$$
then we are done. Can you see how to go on?


Residual connection in Xception

Question: I am trying to understand residual connection in Xception.
If I am getting right, there's nothing really happen in residual connection (right figure) because it is just addition.
But, I could not get idea what is going to happen in the residual connection in Xception (left image) while it backpropagates. I see there is a 1x1 convolution in residual connection. How do we train the weights for the 1x1 conv?
Please understand if I made mistakes in asking a question in this community. I am a noob.
Thank you for any helps.
left image from :https://arxiv.org/pdf/1610.02357.pdf
right image from :https://arxiv.org/pdf/1512.03385.pdf
Answer: It's not really clear what you're asking. Are you confused about what a 1x1 convolution is? If yes, it's literally a 1x1 convolution, with depth equal to 64 in this case, so each such 1x1 has 64+1 weights. Since it has stride 2, it's applied on every other pixel. This output is then added to the maxpool result.
Comment: I am sorry for my unclear question. Training is updating weights of convolution. For the right figure, the residual connection adds 256-d featuremap (input of the block) to the featuremap before relu. while it backpropagates, the residual connection is simply cancelled by differentiation because it is addition. For the left figure, it is also addition and also be cancelled. Which means no error information will be backpropagated to the 1x1 convolution. it means to me that we initialize all the weights in the beginning of training, and there is no weight updates in 1x1 conv while training.
Comment: The Xception block output is $C(x)+M(x)$, $C(x)$ is the 1x1 convolution output, and $M(x)$ is the three sets of layers ending in a maxpool on the right. $C(x)$ absolutely has weights, which will contribute to backpropagation when differentiated.
Comment: Oh my... I was such a foolish idiot. Your simple equation makes every things clear. I stupidly thought C(x) as C like a constant. Sorry for wasting your time, and thank you very much. How foolish I am...

Residual connection in Xception


Jquery getJSON php array to javascript

Question: I am testing the getJSON with php array, but it is not working, can anyone check for me ? I feel the problem is from the php code because when i test the html code with getJSON url https://graph.facebook.com/zombies, it is working.
In my array.php
<?php

 header("Content-type: text/javascript");

 $arr = array(

            "name" => "Tim",
            "age" => "28"     );

echo json_encode($arr);

?>

In my test.html :

Comment: Try the content type application/json
Comment: ok i have solve my problem,

in php : echo $_GET["callback"] . "(" . json_encode($arr) . ")";

in html : $.getJSON('array.php?callback=?', function(data)

so the cross server can be done
Answer: Change
header("Content-type: text/javascript");
To
header('Content-Type: application/json');

For JSONP
header('Content-Type: application/javascript');

Answer:
header("Content-type: text/javascript");

You're telling the browser you're sending it JavaScript, when what you're sending it is JSON. The content type for JSON is application/json.
If you fix that, it should work, provided you're not running afoul of the Same Origin Policy. If you're making a cross-domain request, your options are:

Don't use ajax and JSON, use JSON-P.
Use CORS, but it requires that the server allow your document's origin, and that the browser supports it (most modern ones do, older ones don't).
Use YQL as a cross-domain proxy.
Comment: oh ... i found my problem, because my script is inside Ubuntu server and i running it from win 7, thx for it !
Comment: i have a question, if the array transfer happen in local and cross domain, what should i do ?
Comment: @jc_tan: The cross-domain options work for the same domain as well.
Answer: Try to set the content type to
application/json

So jQuery will interpte as json data

Jquery getJSON php array to javascript


I need help with solving a math problem that involves clocks

Question: Here's the question.
"It is now between 10:00 and 11:00. Six minutes from now, the minute hand of a watch will be exactly opposite the place where the hour hand was three minutes ago. What is the exact time now?"
It's from Art of Problem Solving Volume 1, Chapter 4 Proportions. I haven't solved clock problems in the past and I'm not sure how to do this. The solution didn't help and was extremely complicated, so I was hoping someone could provide a solution that I might be able to understand. Any tips for future problems that might be similar to this would also be greatly appreciated!
Comment: Try to replace the clock's hands by their respective angles, then write down the equation the problem gives you.
Comment: Is this actually from a math contest?
Comment: What are the angular velocities of the hour and minute hands, respectively?
Answer: I'd set $t=0$ to correspond to 10:00. Let's write formulas for the position of each hand $t$ minutes after that, where position is measured as the angle (in degrees) clockwise from the "12".
Minute hand: $m(t)=\frac{360\text{ deg}}{60\text{ min}}t=6t$.
Hour hand: $h(t)=\frac{360\text{ deg}}{12\cdot60\text{ min}}t+300=0.5t+300$.
Now let $T$ be the time now. Your information says:
$$
\begin{align}
m(T+6)&=h(T-3)-180&&\text{(Note $h(T)$ is between 300 and 330.)}\\
6(T+6)&=0.5(T-3)+300-180
\end{align}
$$
You can solve this linear equation to get $T=15$. So it's 10:15.
Answer: Consider 10:00. The angle formed at 10:00 is 60 degrees. So, for every minute past ten, the minute hand increases the angle by 6 degrees (360 deg/60min), but the hour hand decreases it by 1/2 degree since it moves up. This leaves us with a net gain of 5.5 deg. We can this write the angle at 10: min to be 60+5.5(min).
Six minutes from now, the minute hand moves 66=36 degrees.3 minutes ago, the hour hand moves 1/23=1.5 degrees. Since the minute and hour hand are opposite, they forms a straight line, and our time 10:min must form an angle of 180-36-3/2= 142.5 deg. So, 60+5.5x = 142.5 and x= 15


Converting process.hrtime() to milliseconds

Question: I have existing code that does the following use of hrtime
before = process.hrtime()

// stuff happens...

after = process.hrtime(before)

latency = (after[0] * 1000000000 + after[1]) / 1000000;

What unit is latency above?
Answer: Turns out the above is in fact milliseconds. Since setTimeout takes the delay value in milliseconds, it's easy to validate my hrtime juggling:
const times = []
const delays = [1, 10, 100, 1000, 1001, 1100];

function sleep(ms) {
    return new Promise((res) => {
        setTimeout(res, ms);
    });
}

async function main() {
    for (let ms of delays) {
        const ping = process.hrtime();
        await sleep(ms);
        const pong = process.hrtime(ping);
        const latency = (pong[0] * 1000000000 + pong[1]) / 1000000;
        times.push(latency);
    }

    console.log(times);
    // >>> [2.031091, 11.623596, 101.971041, 1000.554953, 1001.192077, 1104.29338]
}

main();


Remover espaço e quebra de linhas em string

Question: Estou fazendo um WebAPI que gera um XML, esse XML é lido diversas vezes ao dia, então na primeira execução ele serializa todo o meu XML e salva em disco, e durante 24h ele lê do disco ao invés de serializar todo o objeto novamente.
Faço isso pois tem vários acessos, os XML são grandes alguns com até 300mb, e as informações podem ser cacheadas por 24h
O Problema é que o campo descrição, creio que ele poderia ser 'compactado' ou melhor poderia tentar fazer um minify no xml antes de gravar ele em disco. estou tentando remover os espaços em brancos e quebras de linhas só desse campo por enquanto assim já reduzo alguns bons megas.
Utilizo Webapi em C#, Redis, MSSQL
Hoje estou enviando ela assim:
    <description><![CDATA[SOBRADO

Área Terreno: 8 x 28
Área Construída: 170m²

Pavimento Superior:
2 dormitórios sendo 1 dormitorio com armario embutido planejado e um maste
banheiro
jardim de inverno
sacada

Pavimento Térreo:
2 salas
Copa
Cozinha
Corredor lateral
jardim na frente
quintal

Edícula:
1 dormitórios
banheiro
lavanderia
deposito

4 vagas

IPTU R$ 1.200,00 anual]]></description>

Gostaria de enviar assim:
<description><![CDATA[SOBRADO Área Terreno: 8 x 28    Área Construída: 170m²...

Utilizo 2 funções para tentar limpar o código, mas ainda não está como gostaria.
description = Biblioteca.RemoveTroublesomeCharacters(Biblioteca.CorrigeDescricao(imovel.Descricao)),

internal static string CorrigeDescricao(string descricao)
{
    var tab = '\u0009';
    descricao = descricao.Replace("  ", " ");
    descricao = descricao.Replace("=\r\n", "");
    descricao = descricao.Replace(";\r\n", "");
    descricao = descricao.Replace("\t", " ");
    descricao = descricao.Replace(tab.ToString(), "");
    return RemoveHtml(descricao);
}

E
 internal static string RemoveTroublesomeCharacters(string inString)
        {
            if (inString == null) return null;

            var newString = new StringBuilder();
            char ch;

            for (int i = 0; i < inString.Length; i++)
            {
                ch = inString[i];
                // remove any characters outside the valid UTF-8 range as well as all control characters
                // except tabs and new lines
                //if ((ch < 0x00FD && ch > 0x001F) || ch == '\t' || ch == '\n' || ch == '\r')
                //if using .NET version prior to 4, use above logic
                if (XmlConvert.IsXmlChar(ch)) //this method is new in .NET 4
                {
                    newString.Append(ch);
                }
            }
            return newString.ToString();
        }

Comment: E qual é seu problema?
Comment: quero remover espaços em brancos, quebras de linha tentar minificar o arquivo, alguns tem 300mb se eu conseguir remover alguns caracteres já pode significar uns 20mb no final
Comment: @Dorathoto, que tipo de aplicação vai consumir o seu serviço? todas são .NET?
Comment: não, a maioria creio que seja em java..mas não sei as tecnologias deles.
Answer: Você pode resolver a questão dos excessos de quebra de linha e espaço com REGEX :
Removendo excessos
pattern : (\s){2,}
replace : $1

Vai capturar carateres de espaçamento que se repitam mais de duas vezes e substituir por um so. Note que ele substitui pelo primeiro encontrado.

Exemplo
'teste de quebra    '
'de linha     '

Aplicando ficaria assim:
'teste de quebra de linha '

pois ela uniu o '     \n' e substituiu por ' ', pois ' ' foi o primerio encontrado
Removendo quebras de linha
pattern : (\n){2,}
replace : $1

São similares, mas não igual pois esta considera apenas quebra de linha, talvez seja necessario alterar para (\n\r?){2,} pois Windows algumas IDEs de windows ainda acrescentam o retorno de carro.

Exemplo
'quebra de linha     '

'em duas     '

Aplicando fica assim:
'quebra de linha'
'em duas'

Comment: bem interessante...
Comment: como faria isso em C# ? descricao = Regex.Replace(descricao, @"(\n){ 2,}");
Comment: @Dorathoto `Regex rgx = new Regex(pattern); String result = rgx.Replace(descricao, replacement);`
Answer: NUNCA trate XML como texto. Existem termos para quem faz isso que, embora sejam termos técnicos e inclusive utilizados em livros, fariam com que eu fosse banido daqui se os utilizasse ;)
Ao invés disso, encapsule tudo o que você quer ter no XML como um objeto que seja serializável. Em seguida, utilize as classes de XML do Framework para gerar o XML quando for gravar, ou ler a partir de um arquivo. Isso não só manterá o XML compacto, como garantirá boa formatação e lhe economizará horas de desenvolvimento.
Comece por esta classe: XmlWriter.
Comment: não trato como texto, faço toda uma serialização do meu xml.
Comment: nem tinha reparado que ele estava a montar o `XML` na mão, pensei que ele queria apenas remover os caracteres da `Descricao`, em todo caso ele não deveria implementar nada neste sentido, afinal a `WebAPI` vai retornar um `JSON` ou `XML` depedendo do header `Accept` enviado pelo cliente.
Comment: @Dorathoto, o que estamos tentando dizer, é que o metodo `RemoveTroublesomeCharacters` parece ser desnecessário, o JSON.NET deveria ser capaz de lidar com isto sem a necessidade de qual quer intervenção. Então você precisaria apenas definir um modelo, popular o mesmo, e retornar ele, a WebAPI já iria serializar o modelo como XML caso o cliente informe que deseja um XML pelo Header Accept.
Comment: RemoveTroublesomeCharacters  uso pra garantir apenas encoding utf-8 Acho que não fui claro na pergunta, faço o modo correto, serializo perfeitamente como deve ser com o webapi, é que alem disso eu acabo também gravando o xml fisicamente e esses arquivos tem se tornado grandes
Comment: fiz uma edição na pergunta, não estava clara o suficiente
Answer: Dorathoto, acredito que melhor que remover os espaços da string, seja compactar todo o Response.
A maneira mais facil de se faze-lo sem configura-lo direto no IIS, é instalando o seguinte NuGet: Microsoft ASP.NET Web API Compression
PM> Install-Package Microsoft.AspNet.WebApi.Extensions.Compression.Server

Então execute a seguinte configuração no StartUp da sua WebAPI.:
GlobalConfiguration.Configuration.MessageHandlers.Insert(0,
    new ServerCompressionHandler(
        new GZipCompressor(),
        new DeflateCompressor()));

Resposta Original (EN)
Comment: Parece interessante, no meu primeiro teste, percebi que parece impactar bastante na perfomance o que é algo bem delicado no meu caso, tive até que aplicar o Redis para tentar acelerar o processo que já não era leve.
Comment: @Dorathoto, como disse esta é uma forma de aplicar a compactação direto no codigo, uma outra forma seria ativar ela direto no IIS, procure por `WebAPI GZip on IIS`, tente faze-lo e compatilhe o resultado conosco.
Answer: Estava com um problema parecido porém eu precisava que a descrição ficasse com no máximo uma quebra de linha e no máximo um espaço entre palavras, incluindo tabs.
Segue abaixo o uso do pattern que pode resolver seu problema:

Usando o pattern com regex replace


Distribution of the a random variable defined on the index of a set of independent random variables

Question: Scenario 1: Let $X_i,i\in I$ where $I$ is the index set. For simplicity let $I$ be the finite set $I=\{1,2,\ldots,n\}$. Let each $X_i$ be independent and distributed as
$$
X_i\sim N(\delta,1),
$$
and let $\mathbf{X}=(X_1,X_2,\ldots,X_n)$.
Define the  index estimator
$$
\gamma(\mathbf{X})=\text{argmax}_{i\in I}{\mathbf{(X)}}
$$
The distribution of $\gamma(\mathbf{X})$ is uniform. This is easy to see because each $X_i$ is equally likely to be the maximum since they are all independent and identically distributed.
Scenario 2: In this scenario, $X_i$ are still independent, however, not identically distributed. Each $X_i$ has the distribution
$$
X_i\sim N(\delta_i,1),
$$
and $\gamma(\mathbf X)$  has the same definition, i.e.,
$$
\gamma(\mathbf{X})=\text{argmax}_{i\in I}{\mathbf{(X)}}.
$$
My question is what is the distribution of $\gamma(\mathbf{X})$ in this scenario?
I started with
\begin{eqnarray}
\Pr(\gamma(\mathbf X)=i)&=&\Pr(X_i=\max\{X_1,X_2,\ldots,X_n\})\\
                        &=& \Pr(X_i\geq\max\{X_k:k\neq i,k=1,2,\ldots,n\})\\
\end{eqnarray}
And define $Y=\max\{X_k:k\neq i,k=1,2,\ldots,n\}$ which implies
$$
\Pr(Y\leq y)=\prod_{k \neq i}\Pr(X_k\leq y)=\prod_{k \neq i}\Phi(y)
$$
Then my problem simplifies to
$$
\Pr(\gamma(\mathbf X)=i)=\Pr(X_i-Y\geq 0).
$$
It is the distribution of $X_i-Y$ that makes life complicated for me. However, if you can proceed in another way it might be easier.
Answer: For $n = 2$, $p(\gamma(x) = 1) = p(x_{1} - x_{2} > 0)$. Let $z_{1,2} = x_{1} - x_{2}$. Obviously, $z_{1,2} \sim N(\delta_{1} - \delta_{2}, 2)$. Hence, $p(\gamma(x) = 1) = 1 - \Phi((\delta_{1} - \delta_{2}) / \sqrt{2})$.
For $n > 2$, we use multivariate normal distribution. Take $n = 3$ for example. $p(\gamma(x) = 3) = p(x_{3} - x_{2} > 0, x_{3} - x_{1} > 0)$. Let $z_{3,2} = x_{3} - x_{2}$ and $z_{3,1} = x_{3} - x_{1}$. Of course, the vector $(z_{3,2}, z_{3,1})$ follows bivariate normal:
\begin{equation*}
 (z_{3,2}, z_{3,1})' \sim
 N
 \left(
  (\delta_{3} - \delta_{2}, \delta_{3} - \delta_{1})',
  \begin{pmatrix}
   2 & 1 \\
   1 & 2
  \end{pmatrix}
 \right)
\end{equation*}
Then the probability of $p(\gamma(x) = 3) = p(z_{3,2} > 0, z_{3,1} > 0)$ can be calculated from the CDF of the above bivariate normal distribution.
The general case of $n > 2$ is similar. You will define a $n-1$-dimensional multivariate normal, whose covariance matrix has diagonal elements of $2$ and off-diagonal elements of $1$.
Comment: Something counter-intuitive is going, however, consider $n=2$ and if  $\delta_1>\delta_{2}$,  I will expect  $p(\gamma(x)=1)>p(\gamma(x)=2)$, somehow the reverse is true. What are we missing?
Comment: Ok I see the error you omitted a negative sign.


Prepare Payments page not showing PayDate after adding a filter

Question: I added a new filter to Prepare payment(AP503000). The filter works as expected.
After testing I realized that the Pay Date(APInvoice.PayDate) doesn't show data in the grid
How can I make the paydate data show?

I see that a lot of the data comes from APAdjust and the data missing comes from APInvoice.
CODE:
<code>namespace PX.Objects.AP
{
    // Acuminator disable once PX1016 ExtensionDoesNotDeclareIsActiveMethod extension should be constantly active
    public class APPayBills_Extension : PXGraphExtension<APPayBills>
    {
        #region Event Handlers

        [PXFilterable]
        public PXFilteredProcessingJoin<
            APAdjust, PayBillsFilter,
            InnerJoin<APInvoice,
                On<APInvoice.docType, Equal<APAdjust.adjdDocType>,
                And<APInvoice.refNbr, Equal<APAdjust.adjdRefNbr>>>,
            LeftJoin<APTran,
                On<APInvoice.paymentsByLinesAllowed, Equal<True>,
                And<APTran.tranType, Equal<APAdjust.adjdDocType>,
                And<APTran.refNbr, Equal<APAdjust.adjdRefNbr>,
                And<APTran.lineNbr, Equal<APAdjust.adjdLineNbr>>>>>>>>
            APDocumentList;

        protected virtual IEnumerable apdocumentlist()
        {
            PayBillsFilter row;

            row = Base.Filter.Select();

            PayBillsFilterExt payBillsFilterExt = row.GetExtension<PayBillsFilterExt>();

            foreach (APAdjust item in Base.APDocumentList.Select())
            {

                if (!string.IsNullOrEmpty(payBillsFilterExt.UsrEPEmployee))
                {
                    //To test remove all code and only leave the else ( yield return item;)
                    BAccount bAccount = PXSelect<
                        BAccount,
                        Where<BAccount.bAccountID, Equal<Required<APAdjust.vendorID>>>>
                        .Select(Base, item.VendorID);
                    BAccountExt bAccountExt = bAccount.GetExtension<BAccountExt>();

                    if (bAccountExt.UsrEmployeeID == payBillsFilterExt.UsrEPEmployee)
                    {
                        yield return item;
                    }
                }
                else
                {
                     yield return item;
                }
            }
        }

        #endregion
    }
}
</code>
Answer: You're yielding an APADjust record when your view is a join between the two tables APAdjust & APInvoice.  Try something like this:
<code> protected virtual IEnumerable apdocumentlist()
 {
    var sel = new PXSelectJoin<APAdjust... (view definition)
    foreach( var rec in sel.Select())
    {
      Check your field;
      yield rec;
    }
  }
</code>