Why do many classical guitarists move the fingers for vibrato like a violin?

Question: I have seen many videos where it looks like the guitarist is doing some vibrato and they make a rolling motion like in a violin, cello...
How can this work in a fretted instrument? As an amateur in both instruments, I don't know how can that work on a guitar.
I made a sketch of my understanding:

As far I understand, the length of the string (blue), pressed over the fret (green), should be independent of the point where the finger (pink) is pressing inside the pressing area (yellow).
UPDATE: I find it hard to pick an answer as I think most of them are really good. So, if I understood correctly:
The change in pitch is done as follows: The increase in the position of a fixed point in the string (which means higher tension and therefore higher pitch) represented by the red d):

This would also mean that the vibrato goes in the opposite direction than the violin or other non fretted instrument right? Because usually you get the note  in the right pitch and then start moving your arm away and therefore your fingertip (lowering the pitch) to then go back.
While according to this graph you would start in the position more "far away" and then close the gap. Right?
PD: The picture is exaggerated to show the two positions, usually you would reach both ends without lifting the finger.
Comment: What's really entertaining is watching pianists do vibrato on a key! They swear it has an effect. "Hear it? Hear it?" "I dunno, dude... only with my eyes open."
Comment: @DmitriUrbanowicz The other factor is: what you see and how you feel *do* matter. If the listener *sees* vibrato, they might perceive it. If the performer does something imperceptible that helps them feel more expressive, it impacts perceptible things like timing and articulation. Stagecraft isn't *just* pageantry.
Comment: @AndyBonner - and in real life with 'aftertouch'!
Comment: @AndyBonner: Re vibrato on piano: do not forget the clavichord, where the string runs over the hammer directly attached to the far end of the key you strike and hold depressed to sound and sustain the note. Greater pressure increases tension and ¿marginally? length, allowing one to alter the pitch while it sounds.
Comment: Please don't answer in the question. Especially if the answer is so incomplete that it's almost wrong.
Comment: @PJTraill the part of a clavichord that touches the string is called a "tangent," not a "hammer."
Comment: Good answer from @Ramillies, below, as far as explaining why it works. Have you tried it to see that it does work? Lot's of guitar players do this, not just classical players. It's especially effective on electric guitar and can help lengthen sustain.
Comment: Watching violinists it seems their vibrato is done from the wrist, maybe with some arm motion. My classical guitar teacher has instructed me to perform vibrato from the elbow, which is important for performing vibrato on chords. So I'm pretty sure it's a different technique.
Comment: @AndyBonner There actually have been some experimental designs of electric pianos with a sensor in each key to allow you to do some vibrato or pitch-bending effects.  Not sure this could be easily implemented on a purely mechanical piano.  Though the [Fluid Piano](https://www.thefluidpiano.com/) would allow it - albeit requiring the use of the other hand (or a 2nd person).
Comment: @AndyBonner I once had a dream about exactly that (vibrato on a grand piano). Didn't know people actually try it. I guess it could work if your hands are heavy enough to wobble the entire instrument.
Comment: @PJTraill Hehe, that's probably the purest embodiment of the physics thought-experiment ("Wait, is it tension or length?") that everybody's getting into here. The "bend" of the blues guitar, decontextualized into a pure and simple mechanism. (But wait... does the tangent actually slide *along* the string's length at all? ;-D )
Comment: Since the keys are at right angles to the strings, I think there should be no slipping, as the ratio of lengths remains constant between the two sides of the triangle formed by the two parts of the string while sounding (the original position being the third side). I _think_ that is true even if the string moves sideways (very marginally, but I think that does not matter). P.S. Links: [Clavichord](https://en.wikipedia.org/wiki/Clavichord), [Bebung (the vibrato)](https://en.wikipedia.org/wiki/Bebung).
Comment: Pay attention to whether their finger is between frets or on the fret when they vibrate like this. Are you saying you see them doing it when they press between frets? Because it definitely works when pressing on the fret (which isn't surprising).
Comment: *you get the note in the right pitch and then start moving your arm away (lowering the pitch)* - just for completeness, on the 'Cello, you push your arm down and then let it bounce back - so you're increasing the pitch first (generally).  Difference due to orientation of the instrument.
Comment: The answers need some demonstrations of the pitch actually changing when people do this.
Answer: The length of the string is truly not changing, but there are other things that affect the frequency of the string vibrations. One is the density of the material (obviously this is not what vibrato changes), another is the cross-section of the string (ditto). The last one is the tension in the string. And that's what's being used here.
Essentially, you periodically pull at the string more and less, which periodically adds a bit of tension, and that results in the change of the pitch. (The tuning pegs on the guitar and other strings work in this way as well: by turning them, you add/reduce tension and that makes the pitch change.)
This is also definitely playing its part in vibrato on unfretted instruments; however I don't play any, so I don't really know which part of the vibrato on these instruments is due to what.
Comment: @Aaron - it is, but it's microscopic.
Comment: @Tim - you are applying lateral pressure to it, it's most definitely getting stretched so the entire string is actually longer. This is just basic material elasticity. Sure, it's not as stretchy as a rubber band, but on a microscopic level tension must stretch objects.
Comment: Same question as I left for @Tim: how is the tension changing if the length isn't?
Comment: It's worth distinguishing between the length of the *string* (which gets longer, because you're applying a force on it with your finger - also increasing the tension), and the *vibrating part* of the string (which remains the same length - from fretwire to saddle, as mentioned). Because the string is longer its tension is higher; because the part that is vibrating is the same length (but tenser), the note it sounds is higher in pitch.
Comment: @Davor What I thought. Thanks. Now I can sleep! :-)
Comment: @Aaron "How is tension changing if length isn't" - Well, what do you think tuning machines do? Note that the length in question is the *sounding length*. In the case of a tuning machine, the saddle to the nut. In the case of a fretted note, the saddle to the fret. Doing vibrato on the guitar does not change the length between the saddle and fret.
Comment: @Tim, the string doesn't change length, the tension changes, as you "pull" and "push" the string sideways. You don't actually roll the finger with this kind of vibrato, instead the whole hand moves side-to-side.
Comment: +1  Also, classical and Spanish guitars are typically strung with nylon, (or steel-wound nylon for the low strings) which is much easier to stretch with a rolling motion like this.  A bend would produce too strong a vibrato, and nylon strings are much more prone to permanent stretching (and take a long time to settle into pitch when new) so a gentler vibrato (vs a bend) is also less likely to throw the instrument out of tune - particularly with newer strings.
Comment: @ToddWilcox "Sounding length" was the idea I was missing. Thanks.
Comment: @Davor - how, where, is the string changing length?
Comment: @BobBroadley - if we're talking about the 'blues' type of vib., then the tension changes, yes, by virtue of the fact that the string has been stretched - out of line. So it must be longer, as that straight line is now two sides of a triangle whose base measurement was (is) the straight original string, from fretwire to saddle. Classical vib. is subtly different. It ain't easy to explain... Ironic that longer strings produce lower pitches, but that's compensated for by the extra tension.
Comment: @Davor It doesn't matter if the string is longer as long as the distance from the saddle to fret doesn't change
Comment: @ToddWilcox - you're talking about something that isn't the actual length of the wire and telling everyone else that they are wrong. Sigh.
Comment: @Davor - with the classical type vib. the sting does get longer, but the extra length is *behind* the fretwire, not between fretwire and saddle. Obviously with blues type bending the *sounding length* of the string does increase, as it's pulled out of line.
Answer: The movements are similar, the effects are similar, but the causes are different.
Vibrato on a violin string is due to the length of the string changing. The player's finger rolls up and down the length of the string, which changes its pitch.
On classical guitar (electric and acoustic players also use it) the action of rolling the finger, within the confines of the fret it's in, causes the tension in that string to change. Roll the fingertip towards the guitar head, and the string's tension increases, roll towards the bridge, it decreases. Albeit slightly. There's also a slight change of tone. You're correct - the length of the string doesn't change - it's still from fretwire to saddle.
Compare that with the lateral movement used by electric guitar players, which stretches the string even more - a semitone or tone variance is quite common. Because the string is also being dragged across the fretwire, it increases the sustain, not really a phenomenon produced by the 'classical' vibrato.
Comment: @fyrepenguin - saddle - fixed, fretwire - fixed. So, no, the sounding string length cannot change. The *tension* in the string is what changes, thus changing the string's pitch, and often, its timbre.
Comment: @Tim I think I see. So when a string is stretched, even though the string itself is getting longer, its length relative to two fixed points remains the same, but with increased tension? Put another way: the string gets longer (or shorter), but the *speaking length* remains constant. Yes?
Comment: @Aaron - I guess so. The string itself is stretched, therefore it's (slightly) longer, but that's over its entire length, not the vibrating part. So, it's the *increased* tension that changes the pitch.
Comment: the angle cannot change without the length changing. That would seem to be basic geometry.
Comment: @Aaron - it's the angle *behind* the fretwire that changes. I went to the geometry lesson!
Comment: @Aaron - to simplify things - fret a note on one string. When someone turns the appropriate machine head, the pitch changes. The length of that string hasn't altered.
Comment: How does the tension change if the string length (between fret and bridge) stays the same?
Comment: @Aaron - due to the different angle the string has over the fretwire. Pressing directly behind the fretwire, the angle is steep. Moving the fingertip back makes it more shallow, allowing a litlle more pressure to be put on it. The string length cannot change. The pitch difference is quite small. Changing the tension has nothing to do with its length.
Comment: It's akin to pulling a rope around a corner, vs. pulling on it directly.  You can use the bend to get more tension in the rope between the corner and the load than you would by pulling directly.
Comment: @fyrepenguin Maybe there's something I don't understand about the definition of "tension", but this just isn't adding up for me. The speaking length of the guitar string is from fret to bridge, right? The bridge is fixed, so unless the contact point between fret and string is changing, how can the string tension change between fret and bridge?
Comment: @Aaron - a student of mine used to press harder and release some when fretting a long note. This produced vibrato, simply by changing the tension in that fretted string. The length certainly didn't alter, but the pitch did.
Comment: @Aaron for one, the string can either stretch or slide.  If you drew a mark exactly where it contacts the fret, I would be reasonably sure that you would be able to change the relative positions of the mark & fret by varying the pressure and whatnot.
Comment: @Tim the bridge-to-fret length may not have changed, but in that case, the overall string length almost certainly did (however slightly).  That distinction could be a point of confusion.
Comment: There are two ways to affect the tension without altering the length (i.e. no side-to-side movement): one, as described so far, is to press the string more or less deeply into the space between the two frets; the other is to use friction to push or pull along the length of the string. Either way, apart from vibrato, one can also use this to eliminate a beat (interference) between two notes an almost perfect justly intoned interval apart.
Answer: On both fretted and non-fretted instruments, the vibrato works as a combination of change of resonating length, change of overall length, and change of tension. Many musicians simplify things when they describe how it works, and highlight only one or maybe two of these aspects. The reality is a bit more complex :)
The fret-to-string contact is a low mechanical impedance, so it reflects most of the incoming wave energy and establishes a null in the standing wave. The fret contact changes the length of the resonating part of the string. This is the key overlooked point: the overall length of the string and the length of the resonating part of it are independent.
Now, a string is a spring. If you press sideways on any linear spring constrained at the ends - be it a piece of wire, or a helical spring from a pen, you make the spring longer. Same happens to the string: it gets longer if you press on it. It happens whether you press it down towards the board, or laterally.
You can get fret action without changing the length nor tension of the string. Use a triangular rod, sized to fit exactly between the fingerboard/fretboard and the string, to provide the low mechanical impedance without deflecting the string, and thus without changing its length. You would press the string directly on top of that pseudo-fret to lower the mechanical impedance and improve the quality (q-factor) of the resonating part of the string.
But, obviously, on a fretted instrument, you not only bring the string down onto the fret - that already stretches it, but you then bring it further down towards the fretboard, making it even longer. This increases the length of the string, necessarily increasing its tension (unless the tuning pegs are loose).
You can see how the string stretches on the diagram below. Imagine that the string was painted in two sections, converging right above the fret when the string is free.

As the string is pressed down, the left (purple) part of it stretches past the fret (compare A, B and C). This is in spite of the length of the resonating string being largely the same in all three cases.
Now, when there's a nearby fret, the compliance of the string to lateral deflection (towards the fretboard and/or sideways) isn't constant like it would be in an unfretted instrument. The closer to the fret, the less compliant the spring is. And what does it mean? Suppose you deflect two identical springs the same amount: the more compliant one will experience less tension (tensile stress) than the more rigid (less compliant) one.
Thus, a string right behind the fret is a rather nonlinear spring, and the closer to the fret you press it down onto the fretboard, the more tension you introduce into the string. Note that the length of the resonating part of the string has not changed, but the overall length of the string then increases, and the tension in the string increases as well. The higher the tensile stress, the stiffer the string, and the higher the frequency of oscillation, and thus moving the finger towards the fret, while keeping the string down to the fretboards sharpens the note.
Of course the compliance of your finger also plays a role: the less compliant it is, the more it emphasizes the change in string's compliance towards the fret. Heavily callused fingers produce higher vibrato pitch range for the same range of finger motion.
And of course, we've only mentioned deflections toward the fret/fingerboard. You can also, additionally, deflect the string laterally, to further increase the string tension. And the vibrato can be a combination of rolling motion and sideways motion of the finger.
On the diagram below, 1 is the free string, 2 is the string lightly depressed straight down, and 3 is the string bent sideways.
Comment: In diagram B, it will be dependent on which side is longer, as to where the colour split goes. As shown, I'd guess the purple side is longer. But that won't always be the case with a fretted string.
Comment: @Tim True! Then, as the string is pushed all the way down, the split will reverse and go over the fret and towards the inter-fret area :)
Answer: Already good answers, but I think I can add some info.
[I can think of at least two ways to do vibrato without the fingers: by subtly bending the neck and by using the erroneously named "tremolo" bar. But these are not relevant here.]
You can do two kinds of finger vibrato on guitar: moving the string up and down (like a bend); moving the string side to side. We're talking about the second method here.
With this kind of vibrato, the string doesn't change length, instead you are changing the tension in the string. As you "pull" the string to the left the tension increases and the note goes sharp, as you "push" the string to the right the tension decreases and the note goes flat. This is similar to how vibrato is done on bowed string instruments (e.g. violin), but it is not the same. On violin you roll the finger from side to side, which does make the string length change, and hence changes the pitch this way. With this style of vibrato on guitar, the finger remains perpendicular to the neck, and the whole hand moves side to side; the finger doesn't roll. (Incidentally, I often refer to this kind of vibrato to pupils, as "violin-style" vibrato - maybe I shouldn't!)
Lastly, it is worth mentioning why the "bend-style" and "sideways-style" vibrato are both used on guitar. Basically, they sound different and work better on different kinds of guitar.
Firstly, the sound: bend-style vibrato always makes a note slightly sharp. If you think about it, whether you bend a string upwards or downwards (towards floor or ceiling) the pitch always goes up. This makes this kind of vibrato less reliable tuning wise, and so maybe more suitable for electric guitar styles, e.g. Blues, Rock etc. Basically, it is less subtle. Whereas, side-to-side vibrato periodically goes above and below the main pitch. So on average, the note stays at the same pitch. This is more natural, more subtle, with more reliable tuning, and I would say more suited to classical music. It is more like the vibrato a singer would use.
And these usages match up well with the kinds of instruments broadly associated with these styles too. Bend-style vibrato doesn't work very well on nylon strings; hence why side-to-side vibrato works better on classical guitar. Either work well on electric or steel-string acoustic, so the choice is more about sound and style. I would guess bend-style vibrato is more commonly used on electric and acoustic, but this is not a scientific observation!
Comment: As a violinist [aware that the discussion is about guitar technique, but still:], the day I learned vibrato my teacher told me "Pretend there's a wad of chewing gum on this table. You get your finger stuck in it, and then try to remove the gum by rubbing side to side." This gets the right points: There is lateral motion, but the actual flesh of the finger doesn't slide along the string.
Comment: Still not with you! My finger doesn't slide anywhere, but the tip rolls left-right in the fret. I'll try to describe:put a fingertip on aa hard surface, hand relaxed. Keep the pressure on enough, as if fretting a string.Then move your whole arm side to side. The fingertip squidges in a sideways movement. That's what I think the question is about.
Comment: It might be we're describing the same thing in different ways! Ah well, that's the beauty of SE, it seems you've described this better, so no doubt you're answer will stay above mine. I might rethink my answer later if I have time, or just admit defeat!!
Comment: Never give up! It's not an easy concept to describe - and you may well do it in a way that those who can't understand the other answers go 'Oh, it's that simple..!' Garn...
Comment: This is exactly what I'm getting at, Andy. Interesting that a violinist was taught that too… I'm glad that this post is exploring this technique through multiple answers.
Answer: Here's the part of string that does change the length — it's within the fret that's pressed. It's not the part that resonates, but by making the string stretched, more tension is on all of the string length.

That being said, it's quite a subtle vibrato and to get anywhere near an expressive vibrato with a technique like this, one would have to apply the finger not between frets but behind the nut.
Btw the usual vibrato technique on the guitar also works by stretching the string. By bending it upwards, the resonant part even gets slightly longer, but the increased tension is what makes the pitch higher.
Comment: @JirkaHanika - the movement you describe, moving the string across the fretwire, is not the usual one for classical guitar playing. It's usually a roll between the fretwire you're on and the fretwire at the lower end of the fret. Fret at fret 5, and roll towards fret 4.
Comment: The more expressive vibrato is typically done by a left hand finger movement in parallel with the fret, as opposed to applying pressure behind the nut.  You might run out of hands if you place one of them behind the nut.
Comment: @Tim, I'm not sure this is correct. I don't roll my finger when doing vibrato on classical guitar. And as far as I can tell, if you do it doesn't produce much noticeable vibrato.
Comment: @Tim See my answer!
Comment: @DKNguyen the thing is that classical guitarists actually pretend to do the vibrato between the fretbars: https://www.youtube.com/watch?v=oh-IUxWzfQk
I agree, there's near 0 audible difference, that's why I take a stab at it and suggest trying to do it behind the fretbar :) But that's the technique this question is about.
Comment: The diagrams are nice, but I'm not sure whether accurate.  I think that the idea about moving the finger closer to the active fret is to press the string much shallower (not deep enough to touch the fingerboard), and that's what leads to lower, as opposed to higher, tension and thus pitch when the finger tip is closer to the fret.
Comment: @BobBroadley - if you don't roll, (in the same direction as the neck) then what do you do?
Comment: This isn't accurate. The movement you're describing is how the sideways vibrato motion works. Rolling along the length of the fingerboard only works if you're pressing on the fret.
Comment: @DKNguyen can you elaborate? I don't really understand what you put as the right and what's the wrong description.
Comment: Oh wait a second, I interpreted your pictures has an ovehread view of the frets when I thin it's a lateral view and though you were talking about sideways vibrato when the OP is asking about longitudinal. Regardless rolling in the direction on the fretboard while in between frets has rather minimal results if any. In my experience it makes zero audible difference You argue that the tension of the string is changed at the fret but friction at the fret tends to hold the string in place. It doesn't easily slide over the fret. [cont...]
Comment: Now, if your finger is *on* the fret then the vibrato is very pronounced and the mechanics are identical to a fretless instrument. There is no reason I can see why your finger can't be on the fret  if you're vibrating this way (and would naturally preclude multi-string vibrato unless you are on the higher end of the fingerboard since human fingers aren't generally long enough to reach the exact position of the frets on multiple strings, which is the reason the frets exist in the first place).
Answer: Two types of guitar strings:

Nylon / other polymer / natural materials:

The tuning tension is low (compared to metal)
Bending the string doesnt achieve much of pitch change, because the string elasticity module is low as well. The tension simply doesn't change much.
The string is thick, making it easier to pull it by friction towards the head or the saddle, changing the tension of the sounding part of the string.

That's how vibrato is made. You pull the string longitudinally. You get vibrato towards both sharp and flat and it is not very deep (generally, less than half tone).

Steel-based strings

The string is heavier, so the tuning tension is higher for the same length and tone
The elasticity module is high, small changes in lenght make audible changes in tension and pitch. The tone is sensitive to bending.
The strings are quite thin, so no significant longitudinal friction.

The possible vibrato approaches are bending (the tone always going sharp) or a dedicated vibrato machinery that is present on electric guitars.
One can even bend (non-destructively) or shake the whole (electric!) guitar and get more than a tone flat or sharp. Don't try this with an acoustic guitar, it will break.
Comment: Digital tuners (those that are powered by a coin-size battery) are way too slow to catch vibrato. They measure the frequency averaged over 1/2 or 1 second. You need a full-blown spectral analyzer (or a trained ear) to catch this.
Comment: It's virtually impossible to *pull* a string longitudinally - especially the plain strings. A finger would merely slide along.
Comment: @Tim how do you explain that the classical vibrato technique can bend the pitch slightly below the normal fretted pitch?
Comment: @ojs - I don't, as it doesn't.
Comment: @Tim how do you know? You're exactly in agreement with reality, but I'm interested in how did you end up being so sure.
Comment: @ojs - using a digital tuner. Try it.
Comment: @Tim I can't try your digital tuner, maybe it's broken or doesn't react fast enough, but Audacity spectrum plot shows about 20 cents down from non-vibrato pitch.
Answer: To answer the question of why this is done, the fretboard on a classical guitar is flat and it's easy for a nylon string to fly out from under your finger if you try to do bending vibrato like on an electric guitar.
Comment: Never found that to actually be the case. It's far more to do with the nylon strings and their tension (or lack of it) rather than flat fingerboards.
Answer: The other answers are great but I have to chip in because an obvious piece is missing in all of them.
You can see guitarists vibrato in all styles of music. Here's the kicker: a guitar vibrato isn't usually completely horizontal -. It's usually a  bit diagonal / or circular o in motion. The difference is a vertical bend in the string.
Vertical bends of the strings will raise the pitch. I'm sure you've seen the stereotypical blues guitarists doing big bends with their eyes closed and a face of nearly constipated concentration.
P.S.
You made me pick up a guitar and see how I do it. I mostly make a tiny oval.
Answer: All of these answers seem correct. Sonny Edwards might be able to confirm, but for me, every classical guitar I've ever played the strings seem to roll or twist when you try to use vibrato, which feels horrible and makes me feel like I've lost some control. Maybe that also has something to do with it.
Answer: If you are pressing on the fret itself (not between frets) rolling in the direction of the fingerboard definitely works. It's identical to what you would do on a fretless guitar (or violin, or other instrument for that matter).
Rolling sideways works when you press in between frets because even though the fret is still pinching the string, the extra length from diagonal ealignment changes the tension.


mdadm RAID5 random read errors. Dying disk?

Question: First the long story:
I have a RAID5 with mdadm on Debian 9. The Raid has 5 Disks, each 4TB of size. 4 of them are HGST Deskstar NAS, and one that came later is a Toshiba N300 NAS.
In the past days I noticed some read errors from that Raid. For example I had a 10GB rar archive in multiple parts. When I try to extract I get CRC errors on some of the parts. If I try it a second time, I get theses errors on other parts. That also happens with Torrents and a re-chack after download.
After a reboot my BIOS noticed me that the S.M.A.R.T status of a HGST drive on SATA Port 3 is bad. smartctl had sayed to me that there are DMA CRC errors, but claims that the Drive is OK.
Another reboot later, I can't see the crc errors in the smart anymore. But now I get this output
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-4-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
Failed Attributes:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   001   001   005    Pre-fail  Always   FAILING_NOW 1989

As the HGST aren't aviable for normale prices anymore, I bought another Toshiba N300 to replace the HGST. Both are labeled as 4TB. I tryed to make a Partition of the exact same size but it didn't worked. The partition programm claimed that my number is too big (I tried it with bytes and sectors). So I just made the Partition as big as posible. But now it looks like it is the same size, I'm a bit confused.
sdc is the old, and sdh is the new one
Disk /dev/sdc: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 4CAD956D-E627-42D4-B6BB-53F48DF8AABC

Device     Start        End    Sectors  Size Type
/dev/sdc1   2048 7814028976 7814026929  3,7T Linux RAID

Disk /dev/sdh: 3,7 TiB, 4000787030016 bytes, 7814037168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3A173902-47DE-4C96-8360-BE5DBED1EAD3

Device     Start        End    Sectors  Size Type
/dev/sdh1   2048 7814037134 7814035087  3,7T Linux filesystem

Currently I have added the new one as a spare disk. The RAID is still working with the old Drive. I still have some read errors, especially on big files.
This is how my RAID Currently looks:
/dev/md/0:
        Version : 1.2
  Creation Time : Sun Dec 17 22:03:20 2017
     Raid Level : raid5
     Array Size : 15627528192 (14903.57 GiB 16002.59 GB)
  Used Dev Size : 3906882048 (3725.89 GiB 4000.65 GB)
   Raid Devices : 5
  Total Devices : 6
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Sat Jan  5 09:48:49 2019
          State : clean
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 512K

           Name : SERVER:0  (local to host SERVER)
           UUID : 16ee60d0:f055dedf:7bd40adc:f3415deb
         Events : 25839

    Number   Major   Minor   RaidDevice State
       0       8       49        0      active sync   /dev/sdd1
       1       8       33        1      active sync   /dev/sdc1
       3       8        1        2      active sync   /dev/sda1
       4       8       17        3      active sync   /dev/sdb1
       5       8       80        4      active sync   /dev/sdf

       6       8      113        -      spare   /dev/sdh1

And the disk structure is this
NAME                       MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                          8:0    0   3,7T  0 disk
└─sda1                       8:1    0   3,7T  0 part
  └─md0                      9:0    0  14,6T  0 raid5
    └─storageRaid          253:4    0  14,6T  0 crypt
      └─vg_raid-raidVolume 253:5    0  14,6T  0 lvm   /media/raidVolume
sdb                          8:16   0   3,7T  0 disk
└─sdb1                       8:17   0   3,7T  0 part
  └─md0                      9:0    0  14,6T  0 raid5
    └─storageRaid          253:4    0  14,6T  0 crypt
      └─vg_raid-raidVolume 253:5    0  14,6T  0 lvm   /media/raidVolume
sdc                          8:32   0   3,7T  0 disk
└─sdc1                       8:33   0   3,7T  0 part
  └─md0                      9:0    0  14,6T  0 raid5
    └─storageRaid          253:4    0  14,6T  0 crypt
      └─vg_raid-raidVolume 253:5    0  14,6T  0 lvm   /media/raidVolume
sdd                          8:48   0   3,7T  0 disk
└─sdd1                       8:49   0   3,7T  0 part
  └─md0                      9:0    0  14,6T  0 raid5
    └─storageRaid          253:4    0  14,6T  0 crypt
      └─vg_raid-raidVolume 253:5    0  14,6T  0 lvm   /media/raidVolume
sdf                          8:80   1   3,7T  0 disk
└─md0                        9:0    0  14,6T  0 raid5
  └─storageRaid            253:4    0  14,6T  0 crypt
    └─vg_raid-raidVolume   253:5    0  14,6T  0 lvm   /media/raidVolume
sdh                          8:112  1   3,7T  0 disk
└─sdh1                       8:113  1   3,7T  0 part
  └─md0                      9:0    0  14,6T  0 raid5
    └─storageRaid          253:4    0  14,6T  0 crypt
      └─vg_raid-raidVolume 253:5    0  14,6T  0 lvm   /media/raidVolume

I'm a bit confused that the spare disk (sdh) is already in the crypt volume.
Questions:
Under what criteria will mdadm say that a disk has Failed?
Can the random read errors come from one broken Disk?
Dosn't detect the raid it when a disk sends the wrong data?
Is it dangerouse to mark a disk manually as failed when the spare Disk has not the exact same size?
Comment: except of being "too broad" it's also an off-topic ("consumer workstations or networking (which belong on our sister site, Super User)") — https://serverfault.com/help/on-topic
Comment: I had presented the same problem, I've checked disks with smartctl -a long and disk it's ok some problems but corrected (less than 7), but I can't build the array (raid 6).
I can force with this command but if the disk fails, would be a big problem.
Answer: MD raid is far too conservative with kicking out disks, in my opinion. I always watch for ATA exceptions in the syslog/dmesg (I set rsyslog to notify me on those).
I must say I am surprised that you get errors on the application level. RAID5 should use the parity information to detect errors (edit, apparently, it doesn't; only during verification). Having said that, whether the disk is the cause or not, it's bad. Nearly 2000 reallocated sectors is really bad.
Partitions can be bigger, otherwise you can't add them as spare either, but to be sure everything is fine, you can clone partition tables using fdisk, sfdisk and gdisk. You have GPT, so let's use its backup feature. If you do gdisk /dev/sdX, you can use b to back the partition table up to disk. Then, on the new disk, gdisk /dev/sdY, you can use r for recovery options, then l to load the backup. Then you should have an identical partition and all mdadm --manage --add commands should work. (you will need to take out the new disk from the array before changing the partition table)
I actually tend to keep those backup partition tables around on servers. It makes for fast disk replacements.
And, a final piece of advice: don't use RAID5. RAID5 with such huge disks is flaky. You should be able to add a disk and dynamically migrate to RAID6. Not sure how from the top of my head, but you can Google that.
Comment: "RAID5 should use the parity information to detect errors. " — what do you mean "should"? Linux software RAID doesn't work that way
Comment: @kevinq, if will work if you mark the old device as failed. You will just have to remember that in the future, new disks only need to have partitions has big as the smallest drive. As for the UUID, I think you're right. If you have a new enough `sfdisk` (with gpt support), you can see the UUIDs with `sfdisk -d /dev/sdX`. I used to clone MBR partition tables with `sfdisk -d /dev/sda | sfdisk /dev/sdb`, but with UUIDs, I need to rethink this.
Comment: As far as I'm aware, *nobody's* RAID uses the parity information to verify the read. The closest I'm aware of is ZRAID and BTRFS RAID, which use checksums to verify the read, and then the parity information to recover if the checksum fails.
Comment: I already added the new disk (sdh1) with mdadm --manage --add and it is there as as spare disk now. Can i Just mark the old device as failed? Or should I remove the spare again, and vreate a Partition with a gdisk backup? I read somewhere that I need to set a new uuid if I install the parition backup.
Comment: @poige can you elaborate? You're saying (Linux software) RAID5 doesn't use the parity to verify what it just read? If not, I stand corrected, and what the OP sees is not unexpected, though the drive should have given an error instead of return wrong data.
Comment: I'm saying what I'm saying, the q-n is why are you telling before prior checking(?)
Answer: it's pretty common to have cron task initiating parity mismatch checks. i'm pretty sure debian 9 does it by default when mdadm package installs and hence your system's logs would have reports in regards.
Besides if your system's RAM fails it might be the primary reason
Comment: To test the system RAM, the OP can run memtest86.
Comment: whatever, that's another story


Writing a score in C (as it will sound)

Question: I want to learn how to write a score for transposing instruments as it will sound (in C). I have a vague idea of how to do this, like for example, you need to know the transposing intervals of the instruments, etc. But it would be great if I could get an example of how is this done.
If this question or anything similar to this topic has been previously answered, it would be helpful if someone could attach a link to the same below. Thanks.
Comment: One major advantage of  writing a score in C is that you **don't** have to worry about the transpositions.
Comment: Which app are you using to prepare the score?
Comment: @RosieF I am not using an app for this. I want to learn this for music theory purposes, which needs it to be manually done.
Comment: @PiedPiper Like what did you mean when you said not to worry about the transpositions?
Comment: No, when writing a score in C you DON'T have to know the transposing intervals of the instruments.  That's the whole point of a score in C.   You do have to know the ranges of each instrument (expressed in concert pitch) though.  Paradoxically, this can be easier when looking at their parts transposed, as the player will.
Answer: B♭ Trumpet, for example, is called that because when the player sees a C note and plays it, a B♭ note comes out. So to compensate for this, a writer would have to write a note one whole tone above the note he wanted played. Thus, to get a B♭ trumpeter to play an actual C at concert pitch, he would write a D note.All B♭ instruments must have the dots written similarly.
All other transposing instruments will need writing for in simialr compensating ways.
E♭ sax players see a C written, and blow what comes out as a E♭.
Take a look at some orchestral scores - you'll see that on a page, several of the lines of music are written in different keys, to compensate.
Comment: Write in written pitch. Not sounding pitch.
Comment: So writing a score in C would mean to write the music for transposing instruments at written pitch, or the pitch the instrument would sound at?
Answer: Bb instruments are transposing a major second down (and + or -1 octaves or 2)
Eb instruments are transposing a minor 3rd up or a major 6th down)
F Horns are transposing a 4th up (respectively a 5th down)
To write  the voices for transposing instruments  you have transpose the notation of their voices by the same amount - of the intervals of their transposition - in the opposite  direction (up or down).
Bb (transposing 1 major second down)
Transposing instruments -> notation
Bb (major 2nd down) -> major 2nd up
Eb (minor 3rd up)   -> major 3rd down
examples: Bb trumpet plays C -> sound = Bb
but writing D -> trumpet reads D -> plays D -> sound = C
Eb Horn plays C -> Eb
however notation A -> horn reads -> sound = C
The key and signatures are also the same amount of  the interval in the opposite direction of C:
Bb-instruments have 2 # more than concert picht (respectively 2 naturals less)
Eb-instruments have 3 # more than concert pitch and 1 sharp more than Bb instruments.
etc.
Mind that some musicians with transposing instruments  read and play  concert pitch in G or F clef by transposing themselves. You have to know what your musicians are reading, e.g. tenor and bass instruments as Euphonium or Trombones.
Comment: I'm in awe of trombone players, who can read just about anything.
Comment: so you can be in awe of me too! ;) the point of this is: thinking in movable Do ... but this would be limited in 12 tone music. :)
Comment: I think in terms of moveable do when reading for guitar, but that's mainly because the relative notes retain their shapes very well on that instrument. Can't see how it helps that much on any others.
Answer: When you write a score in C you do not care about transposing except in special circumstances. As a small note you should probably call the score non-transposing.
You write all instruments as of they were non-transposing, a written C sounds C. The transposing part is left for creating the parts.
What you need to consider is firstly sounding ranges of the instruments. Don't ask a piccolo to play a tuba part. You might need to think about un-playable tones on some instruments. As example on a normal tenor trombone there is a gap between low E and pedal Bb unless the instrument has a trigger. That Liw Bb flat was used in as example In the Mood by Glenn Miller (who played the bone).
Additionally when you write for less advanced players you are better off avoiding some keys. Lots of sharps in the transposed score tends to not be liked by beginner trumpet players. There is a reason why clarinet players has both A and Bb instruments.
A non-transposing score in my experience makes life easier for beginning composers and conductors. The seasoned pros have No problems reading transposing scores. My two cents though.
Many scoring programs allows you to easily switch between transposing and non, say Dorico, Sibelius, Finale.
Answer: The main purpose of "Writing a score in C" is to facilitate the conductor/leader of the ensemble to "appreciate" the melodic/harmonic flow of the piece "at a glance" - It is that much easier for the conductor/leader if they are all in ONE AND THE SAME KEY. The "music mind" works by judging intervals in given/same key and NOT different keys in each stave

Writing a score in C (as it will sound)


prompt keeps coming up

Question: i have calculation form i want to put empty validation check on each textbox. but the problem is alert box does not close after checking first box.i want to check 10 fields before calculation button click.
function checkTextField(field) {
   if (field.value == 0) {
       window.prompt(field.name + "cannot be empty");
       field.focus();
       return false;
   }
   return true;
}

<input type="text" class="textfield" value="" id="txt1" name = "Rate " onkeypress="return isNumber(event)" onblur = "checkTextField(this)" autofocus/>
<input class="form-control" id="txt2" name="1" type="number" required onblur = "checkTextField(this)" >

Comment: Additionally, you've now found out why `alert()` can be evil since they block the UI. A better approach would be to have a message appear in a DOM element next to the field. This won't block the UI. ... And, don't use in line HTML event handling attributes (`onclick`, `onmouseover`, etc.). Use dedicated JavaScript with `.addEventListener()`.
Comment: Blur -> Event Handler -> Prompt -> Focus -> Blur ... and the cycle continues. Commenting `field.focus()` will work.
Comment: @Rohan Kumar This isn't the kind of code that should be in a snippet.
Comment: @ScottMarcus i think i would b better if u help me out rather than commenting what is wrong and right
Comment: @SarahSalar I was typing my answer during that time. Please be patient as we are all here to help.
Answer: You've got a vicious cycle of events that lead back to each other, so every time you clear your propmt, a new one is generated. Whenever  the user leaves a field and you detect that it is invalid, you set the focus back to it, causing whatever field the user just moved to to lose its focus, causing another validation error.
Additionally, you've now found out why alert() and prompt() can be evil since they block the UI.
A better approach would be to have a message appear in a DOM element next to the field. This won't block the UI.
... And, don't use in line HTML event handling attributes (onclick, onmouseover, etc.). Use dedicated JavaScript with .addEventListener(). Here's why.

// Place this code in a

prompt keeps coming up


Automorphism of Tree

Question: Let $\sigma$ and $\theta$ be two automorphisms of tree $X$. I want to show that min$_{v\in V(X)}d(v,\sigma(v))=$min$_{v\in V(X)}d(\theta^{-1}\sigma\theta(v),v)$.
I know every automorphism of tree is either translation or inversion or rotation. So if $\sigma$ is rotation, then we done. But i do not have any idea for cases translation or inversion.
Answer: This follows from $d(\theta^{-1}\sigma\theta(v),v) = d(\sigma(\theta(v)),\theta(v))$.
Comment: Yes. To elaborate, since $\theta$ is an automorphism, it preserves distances, so that $d(x,y)=d(\theta(x), \theta(y))$. In particular, $d(\theta^{-1} \sigma \theta(v),v) = d(\sigma(\theta(v)),\theta(v))$. The minimum of this quantity over $v \in V(X)$ is the same as the minimum of $d(\sigma(u),u)$ over $u \in V(X)$ because $\theta$ is bijective on $V$.

Automorphism of Tree


SystemVerilog array of interfaces with unique parameters

Question: I have a module (lets call it "mem") that has array of interfaces as a port, parameter NB_ITFS that declares the number of interfaces and a data WIDTH parameter.
module mem
    #(NB_ITFS=2)
    (itf itfa [NB_ITFS-1:0]);
    logic [31:0] data [NB_ITFS-1:0];
    always_comb
    begin
        data[0] = {4{itfa[0].data}};
        data[1] = {2{itfa[1].data}};
        data[2] = itfa[2].data;
    end
endmodule

Interface looks like this:
interface itf #(WIDTH = 8)();
    logic [WIDTH-1:0] data;
endinterface

How can I create an array of interfaces where every interface has unique parameters?
How can I pass them to "mem" module?
Example:
module MyModule #(NB_ITFS=2) ();
    for (genvar i = 0; i<NB_ITFS; i++) begin : ITFA_GEN
        case (i)
            0: itf #(8) itf_if();
            1: itf #(16) itf_if();
            default: itf #(32) itf_if();
        endcase
    end

    mem #(.NB_ITFS(NB_ITFS)) my_mem (.itfa(itf_if));
endmodule

I've tried to connect everything like in example, but it doesn't compile or synthesize.
Answer: As a suggestion, use an array of parameters:
module MyModule #(NB_ITFS=2) ();
  localparam int iparam[] = '{8,16};

  for (genvar i = 0; i<NB_ITFS; i++) begin : ITFA_GEN
    itf #(iparam[i]) itf_if();
  end
endmodule

You can declare it in the module or pass as a parameter to the module.
Otherwise, you can continue using your case statement. However, it is more difficult to maintain.
Comment: Thanks for suggestion, but I still can't connect generated array to a 'mem' module
Answer: I'll try to create another answer here. Unfortunately, there is no way in verilog to do what you want. Every array element in interface array must have the same parameterization. It is a feature of arrays. The code you provided does not create any interface array whatsoever. Instead it creates multiple separate instances of interfaces.
So, there are only two ways to handle your situation:

list all interfaces one by one on the port list: module mem(itf a, itf b, itf c) and deal with this non-flexible situation
design your module mem in such a way that it can handle every interface separately. Here is an modified example of yours:

module mem
  #(WIDTH = 32)
  (itf itfa, output logic [31:0] data);
    always_comb
    begin
      if (WIDTH == 32)
        data = itfa.data;
      else
        data = {32/WIDTH{itfa.data}};
    end
endmodule

interface itf #(WIDTH = 8)();
    logic [WIDTH-1:0] data;
endinterface

module MyModule #(NB_ITFS=2) ();
  localparam int iparam[] = '{8,16,32};
  logic [31:0] data [NB_ITFS+1];

  for (genvar i = 0; i<=NB_ITFS; i++) begin : ITFA_GEN
    itf #(.WIDTH(iparam[i])) itf_if();
    mem #(.WIDTH(iparam[i])) mem(itf_if, data[i]);
  end
endmodule

Answer: // Code your testbench here
// or browse Examples
interface itf #(WIDTH = 8)();
    logic [WIDTH-1:0] data;
endinterface

module mem
    #(NB_ITFS=2)
    (itf itfa [NB_ITFS-1:0]);
    logic [31:0] data [NB_ITFS-1:0];
    always_comb
    begin
        data[0] = {4{itfa[0].data}};
        data[1] = {2{itfa[1].data}};
        data[2] = itfa[2].data;
    end
endmodule

module MyModule #(NB_ITFS=3) ();
    itf itf_p [NB_ITFS-1:0]();
    defparam itf_p[0].WIDTH=8;
    defparam itf_p[1].WIDTH=16;
    defparam itf_p[2].WIDTH=32;

   mem #(.NB_ITFS(NB_ITFS)) my_mem (.itfa(itf_p));
endmodule

Above code fix your issue.
I tried on all simulator on edaplayground. it is only not supported for VCS.
Comment: as far as I know, defparam was a candidate for removal from SV lrm. So, no wonder that some vendors do not do new development in this area or it has a very low priority.  But this case would be a good  example in favor of leaving it alone.


Expected value of steps needed to exceed the sum

Question:
Suppose you have numbers $\{x_{1}=40,\ldots,x_{21} = 60\}$. Next, on each step you are given one of these numbers randomly (the probability of getting the particular number is the same for all of them). What is the expected value of steps before sum of numbers exceeds $10\;000$?

I've solved this problem by just considering $\mathbb{E}(s) = 1 + \dfrac{1}{21}\left[\mathbb{E}(s+40)+\ldots+\mathbb{E}(s+60)\right]$, starting with $s=10\;000$ and going down, so that the pattern becomes obvious. But I wonder if there is a more beautiful approach? Because this one is still a bit tedious.
Comment: There is an elegant way to write it in terms of the probability that, after $N$ steps, the total is at least $M$. Not sure that has an elegant form though.
Answer: There is a beautiful way to get an approximate answer.
Let $N$ be the number of steps it takes to first exceed $10,000$. On the one hand, the expected total after $N$ steps is
$$
\mathbb E[N]\cdot 50,
$$
since each step contributes $50$ on average. On the other hand, after $N$ steps, you know the total will be between $10,001$ and $10,060$. Therefore,
$$
10,001\le \mathbb E[N]\cdot 50\le 10,060,
$$
which implies
$$
200.02\le \mathbb E[N]\le 201.2
$$
So $E[N]$ is just a little more than $200$.


Does "... induces same topology" mean there is a homeomorphism between them?

Question: The following text is from Munkers' Topology:

Well, we can define two sets of real numbers, $I_1=(0,1)$ and $I_2=(0, \infty)$ which are homeomorphic as we can find a homeomorphism between them in standard topology. We can define $\bar{d}$ on space $I_1=(0,1)$ and $d$ on space $I_2=(0, \infty)$. But then they won't be homeomorphic anymore since there won't be a homeomorphism between those two space in metric topology since there is no bijective function between them. So how they are same topology, after all? I think the last paragraph of the proof in the text is not rigorous argument.
Comment: well yes, but considering an open subset for $d$, say $A$, then $A$ is the union of sets $\overline{A}_i$ which are open subsets in the metric basis for $\overline{d}$.
Comment: These spaces are still homeomorphic after changing the metric. Note that a homeomorphism does not need to preserve the specific metric of a space, but only the topology of these spaces.
Comment: @TomUltramelonman, book says they induce same topology meaning that any basis of $d$ must be a basis of $\bar{d}$ which is not correct since in $d$ we can also have a basis with 'radius' greater than $1$.
Comment: What do you mean, "there is no bijective function between them"? There is a simple bijection between the points of every pair of nonempty open intervals in $\mathbb R$. It doesn't matter whether or not either is finite.
Answer: "Induces the same topology" means that the topologies are identical collections of subsets of $X$, in other words a subset of $X$ is open with respect to the first topology if and only if it is open with respect to the second topology. This is equivalent to saying that the identity map on $X$ is a homeomorphism from one topology to the other.
Note: Your example of $I_1$ and $I_2$ are two different metric spaces based on two different sets. That's an entirely different context from the text you pasted into your question, namely of two different metrics on the same set $X$.
Comment: @Liebe yes, that is what Munkres shows in this proof.
Comment: Can two typologies be same but have different bases?
Answer: The two spaces you mention are indeed homeomorphic.  In particular, we can take the following homeomorphism between them:
$$
f:(0,1) \to (0,\infty)\\
f(x) = x/(1 - x)
$$


Looping findnext inside another loop & find

Question: I have used a loop to find the closest name to a supplier from Sheet 1 out of Sheet 2.
Dim LastRow As Long
LastRow = Sheets("BBB").Range("A" & Rows.Count).End(xlUp).Row

Dim i As Integer
For i = 2 To LastRow
Dim ra As Range
Dim a, k As Integer
a = Len(Sheets("BBB").Range("A" & i))

Do
Set ra = Sheets("AAA").Cells.Find(What:=Left(Range("A" & i), a), LookIn:=xlValues, LookAt _
        :=xlPart, SearchOrder:=xlByRows, SearchDirection:=xlNext, MatchCase:= _
        False, SearchFormat:=False)

a = a - 1

Loop Until Not ra Is Nothing Or a = 3

If ra Is Nothing Then
Sheets("BBB").Range("C" & i).Value = a
Else
Sheets("BBB").Range("B" & i).Value = ra.Value

It works great but now I am thinking that It is possible that some occurences are twice in the sheet "AAA"
Example:
Supplier in Sheet BBB: "SICK"
If Sheet AAA has 2 suppliers: "SICK" and "NOSICKHERE LTD"
My code will only find one of the two supplier but will not return both.
How can I use findnext to find all occurences?
Anyone see a better solution?
I tried to use the following at the bottom of my code before the "next i", but I fail to use the findnext
Dim firstCellAddress As String
firstCellAddress = ra.Address

k = 1
Do

    Set ra = Sheets("AAA").Cells.FindNext()
    Sheets("BBB").Cells(i, 2 + k).Value = ra.Value

    k = k + 1

Loop While firstCellAddress <> ra.Address

Please tell me if my question is too hard to understand
Comment: Possible duplicate of [Find and FindNext for Excel VBA](https://stackoverflow.com/questions/30380490/find-and-findnext-for-excel-vba)
Answer: This generates the required output.
Option Explicit

Public Sub GetMatches()

    Dim wb As Workbook, wsSource As Worksheet, wsSearch As Worksheet, masterDict As Object, arr() As Variant, i As Long
    Set wb = ThisWorkbook
    Set wsSource = wb.Worksheets("BBB")
    Set wsSearch = wb.Worksheets("AAA")
    Set masterDict = CreateObject("Scripting.Dictionary")

    With wsSource
        arr = Intersect(.Columns(1), .UsedRange)
        For i = 1 To UBound(arr, 1)
            If Not masterDict.exists(arr(i, 1)) Then masterDict.Add arr(i, 1), GetAllMatches(arr(i, 1), wsSearch)
        Next i
    End With

    Dim key As Variant
    For Each key In masterDict.keys
        Debug.Print masterDict(key)
    Next key
End Sub

Public Function GetAllMatches(ByVal findString As String, ByVal searchRng As Worksheet) As String

    Dim foundCell As Range
    Dim concatenatedString As String
    concatenatedString = vbNullString
    With Intersect(searchRng.Columns(1), searchRng.UsedRange)

        Set foundCell = .Find(findString)
        concatenatedString = foundCell

        Dim currMatch As Long
        currMatch = 0

        For currMatch = 1 To WorksheetFunction.CountIf(.Cells, "*" & findString & "*") - 1

            Set foundCell = .Find(What:=findString, After:=foundCell, _
                                  LookIn:=xlValues, LookAt:=xlPart, SearchOrder:=xlByRows, _
                                  SearchDirection:=xlNext, MatchCase:=False)

            If Not foundCell Is Nothing Then
                concatenatedString = concatenatedString & "," & foundCell
            Else
                concatenatedString = foundCell
            End If
        Next currMatch
    End With
    GetAllMatches = concatenatedString
End Function

TestData:

AAA:
| Absinthe    |
| Antibiotics |
| Random      |
| Antisocial  |
| Antipodean  |
| Motorcycle  |
| Random      |
| Random      |
| Motorbike   |
| Random      |
| Motown      |

BBB:
| Ab   |
| Moto |

Output:
Answer: The code below will loop through all values in sheet B and output it's findings.
I've re-used QHarr's values for my example
Option Explicit
Public Sub findValue()
    Dim firstAddress As String
    Dim c As Range, rng As Range, v As Range
    Dim tmp As Variant
    Dim j As Long

    With ThisWorkbook
        With .Sheets("AAA")
            Set rng = .Range(.Cells(1, 1), .Cells(.Cells(.Rows.Count, 1).End(xlUp).Row, 1))
        End With

        With .Sheets("BBB")
            For Each v In .Range(.Cells(1, 1), .Cells(.Cells(.Rows.Count, 1).End(xlUp).Row, 1))
                ReDim tmp(1 To rng.Rows.Count)
                j = LBound(tmp)

                Set c = rng.Find(what:=v, LookIn:=xlValues, lookat:=xlPart, MatchCase:=False)
                If Not c Is Nothing Then
                    firstAddress = c.Address
                    Do
                        tmp(j) = c.Value2
                        j = j + 1
                        Set c = rng.FindNext(c)
                    Loop While c.Address <> firstAddress And Not c Is Nothing
                    If j > 0 Then
                        ReDim Preserve tmp(LBound(tmp) To j - 1)
                        Debug.Print v & ": " & Join(tmp, ",")
                        v.Offset(0, 1).Value2 = Join(tmp, ",")
                    End If
                End If
            Next v
        End With
    End With
End Sub

Sheet("AAA")

Sheet("BBB") before running code

Sheet("BBB") After code run

Immediate Window after code run


Help Needed for parsing FTP files list in c#

Question: I am using this code for getting list of all the files in directory
here webRequestUrl = something.com/directory/
<code>FtpWebRequest fwrr = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://" + webRequestUrl));

fwrr.Credentials = new NetworkCredential(username, password);
fwrr.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
StreamReader srr = new StreamReader(fwrr.GetResponse().GetResponseStream());
string str = srr.ReadLine();
ArrayList strList = new ArrayList();
while (str != null)
{
  strList.Add(str);
  str = srr.ReadLine();
}
</code>
but I am not getting the list of files, but getting some HTML document type lines.
This ftp server is windows based while it is working fine in unix server.
Please help.
Thanks.
Comment: With ListDirectoryDetails method you will get a UNIX style list of files that you will need to parse. Can you show what you are getting when you say "some HTML document type lines"
Comment: Are you building your own library for ftp? Why don't you just use an existing ftp library?
Comment: This is what I am getting in array list

FTP Directory: ftp://firstname.lastname@example.com/directory/

and more... although there are file name of inner files in this directory.
Answer: It works for me when the FTP on a internal machine and I do a ftp://192.168.0.155 - If I try that in IE I get the same HTML result like yours.
I doubt if its happening because of the url. Can you try replacing the url with the IP address (just a wild guess). Even if you are getting HTML, you can strip the unnecessary part and parse the files.
I even tried with a ftp://sub.a.com/somefolder and it worked for me. It seems the browser wraps the HTML around the FTP response because I get different HTML when I opened the FTP site in IE and Chrome.


Problema com Uppercase e Lowercase no Angularjs

Question: Pessoal segui o exemplo de um script no stackoverflow.com como segue o link abaixo:
[https://stackoverflow.com/questions/19040732/filter-php-list-with-angularjs][1]
Acontece que estou tendo problemas com as palavras que são capitalizadas por exemplo, uma palavra que está "Protocolo" e eu digito "protocolo", o filtro não encontra ela.
app.js
var app = angular.module('plunker', []);

app.controller('MainCtrl', function($scope) {
    $scope.model = {
        filter: ''
    };
});

app.directive('myHtmlFilter', [function() {
    return {
        restrict: 'A',
        scope: {
          filter: '=myHtmlFilter',
          element: '@'
        },
        link: function(scope, elem, attrs) {
          scope.$watch('filter', function(newval, oldval) {
            elem
              .find('ul>li')
                  .hide()
                  .find(scope.element)
                  .filter(':contains("'+scope.filter+'")')
               .parent()
                  .show();
          })
        }
    }
}]);

Index.html
<input type="text" ng-model="model.filter" />

Protocolo

The Message...

My second Post

The Message...

My third Post

The Message...

Comment: [Nessa outra pergunta](http://pt.stackoverflow.com/questions/48755/como-filtrar-um-texto-em-uma-div-com-angular-js) você postou uma solução. Seria a mesma resposta para essa pergunta?
Comment: Não, nesta caso o problema consiste em realizar uma pesquisa onde quando digita-se uma palavra e a mesma corresponda aos valores encontrados independente se é maiúscula ou minúscula.
Answer: O problema está na linha:
.filter(':contains("'+scope.filter+'")')

O seletor :contains do jQuery faz distinção entre letras maiúsculas e minúsculas (case sensitive). A função .filter() aceita como parâmetro um callback que você pode usar para avaliar se o elemento atende aos seus critérios e se deve ou não ser adicionado à lista dos itens filtrados.
Um exemplo de callback que faz uma comparação case insensitive:
.filter(function(){ return $(this).text().toLowerCase().indexOf(scope.filter) > -1 })

Na sua diretiva ficaria assim:
app.directive('myHtmlFilter', [function() {
    return {
        restrict: 'A',
        scope: {
          filter: '=myHtmlFilter',
          element: '@'
        },
        link: function(scope, elem, attrs) {
          scope.$watch('filter', function(newval, oldval) {
            elem
              .find('ul>li')
                  .hide()
                  .find(scope.element)
                  .filter(function(){ return $(this).text().toLowerCase().indexOf(scope.filter) > -1 })
               .parent()
                  .show();
          })
        }
    }
}]);


Amazon Linux instance running beanstalkd

Question: What's the best way to run beanstalkd on an instance of Amazon Linux in EC2?  Beanstalkd doesn't exist in the yum repository and the upstart script included with beanstalkd doesn't seem to work out of the box.  I'm wondering if someone has already solved this problem, or if I need to go and muck around with a custom EC2 user-data script + beanstalkd upstart script.
Comment: `yum search --enablerepo=epel beanstalkd` should turn it up on Amazon Linux.
Comment: Thanks, this is helpful!  Unfortunately, the version is 1.4.6 and there was a critical bug fixed in one of the more recent versions that I ran into.  Any other secret repositories that might have a more up to date version?
Answer: I ended up writing a user data script that runs when an ec2 instance is created.  The script pulls down the latest source for beanstalkd, compiles, and installs the binary.  The script uses a modified upstart script to control the lifecycle of the beanstalkd process.
Here's the user data script:
<code>#!/bin/bash -ex
set -e -x

# Log output
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# Update yum first
yum groupinstall "Development Tools" -y
yum update -y

# Download /etc/default/beanstalkd
curl -J -O https://gist.github.com/phaitour/6482469/download
tar -zxvf *.tar.gz
/bin/cp -f gist*/beanstalkd /etc/default/
ln -s /etc/default/beanstalkd /etc/beanstalkd.conf
rm -rf gist*

# Download /etc/init/beanstalkd.conf
curl -J -O https://gist.github.com/phaitour/6482467/download
tar -zxvf *.tar.gz
/bin/cp -f gist*/beanstalkd.conf /etc/init/
rm -rf gist*

# Create beanstalkd user
/usr/sbin/useradd -c "beanstalk user" -s /bin/false -r -m -d /var/lib/beanstalkd/ beanstalkd

# Compile latest version of beanstalkd
wget https://github.com/kr/beanstalkd/archive/v1.9.tar.gz
tar -zxvf v1.9.tar.gz
cd beanstalkd-1.9
make
mv beanstalkd /usr/bin/beanstalkd-1.9
cd /usr/bin
rm -rf beanstalkd
ln -s beanstalkd-1.9 beanstalkd

# Start beanstalks with upstart
initctl reload-configuration
initctl start beanstalkd
</code>
The two scripts it's downloading from gist is the upstart beanstalkd script and a configuration file which sets the options used when running the program.
Here's the upstart beanstalkd.conf script:
<code>description "simple, fast work queue"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 5 2

expect fork

script
    . /etc/default/beanstalkd
    exec su -c "exec /usr/bin/beanstalkd $BEANSTALKD_OPTIONS"
end script
</code>
Here's the configuration file copied into /etc/default/beanstalkd:
<code># Set some constants to be used to construct BEANSTALKD_OPTIONS

BEANSTALKD_ADDR=0.0.0.0
BEANSTALKD_PORT=11300
BEANSTALKD_USER=beanstalkd
BEANSTALKD_BINLOG_DIR=/var/lib/beanstalkd
BEANSTALKD_BINLOG_FSYNC_PERIOD=0

# Create the actual BEANSTALKD_OPTIONS string
# Copied from https://gist.github.com/shiki/515422

BEANSTALKD_OPTIONS="-l ${BEANSTALKD_ADDR} -p ${BEANSTALKD_PORT} -u ${BEANSTALKD_USER}"
if [ "${BEANSTALKD_MAX_JOB_SIZE}" != ""  ]; then
    BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -z ${BEANSTALKD_MAX_JOB_SIZE}"
fi

if [ "${BEANSTALKD_BINLOG_DIR}" != "" ]; then
    if [ ! -d "${BEANSTALKD_BINLOG_DIR}" ]; then
        echo "Creating binlog directory (${BEANSTALKD_BINLOG_DIR})"
        mkdir -p ${BEANSTALKD_BINLOG_DIR} && chown ${BEANSTALKD_USER}:${BEANSTALKD_USER} ${BEANSTALKD_BINLOG_DIR}
    fi
    BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -b ${BEANSTALKD_BINLOG_DIR}"
    if [ "${BEANSTALKD_BINLOG_FSYNC_PERIOD}" != "" ]; then
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -f ${BEANSTALKD_BINLOG_FSYNC_PERIOD}"
    else
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -F"
    fi
    if [ "${BEANSTALKD_BINLOG_SIZE}" != "" ]; then
        BEANSTALKD_OPTIONS="${BEANSTALKD_OPTIONS} -s ${BEANSTALKD_BINLOG_SIZE}"
    fi
fi
</code>


Trouble with color datagridview

Question: I have DataGridView on my form and I'm trying to color cell.
Here's the code:
<code>public partial class Form1 : Form
    {
        DataTable dtToGrid = new DataTable();
        BindingSource bs = new BindingSource();

        public Form1()
        {
            InitializeComponent();

            dtToGrid.Columns.Add("Group");

            DataRow dr;

            dr = dtToGrid.NewRow();
            dr["Group"] = "must become lightGreen";

            dtToGrid.Rows.Add(dr);

            bs.DataSource = dtToGrid;
            dataGridView1.DataSource = bs;

            ColorData();

            dataGridView1.Refresh();
        }

        public void ColorData()
        {
            dataGridView1[0, 0].Style.BackColor = Color.LightGreen;
        }

        private void button1_Click(object sender, EventArgs e)
        {
            ColorData();
        }
    }
</code>
As you can see, function ColorData invoke twice, but when it's invoked from constructor - nothing happens and cell is still white. Invoke from button1_Click, which is event-function, colors cell correctly.
What's the matter?
Comment: what class is the object dataGridView1?
Comment: System.Windows.Forms.DataGridView
Comment: [Similar question](http://stackoverflow.com/questions/1365617/how-to-force-refresh-the-datagridviews-content) Check the answer on the link.
dataGridView1.EndEdit();   will probably do the trick since you're changing stuff you're editing.
Comment: Try swap ColorData() to before the DataSource assigning.
Answer: I've found the answer myself.
First - the form should load and only after that you can change cell's color.
I change color in event-function Load for the main form and everything works.

Trouble with color datagridview


Finding volume by rotation of a triangle

Question: The following problem is from the book, Calculus and Analytical Geometer by Thomas and Finney.
Problem:
Find the volume generated by revolving the following region: The triangle with vertices $(1,1)$, $(1,2)$ and $(2,2)$ about the
y-axis.
Answer:
Since we are going around the y-axis, our integral will integrate with respect to $y$ not $x$. Observe that the points
$(1,1)$ and $(2,2)$ are on the line $y = x$. Let $v$ be the volume we seek.
\begin{align*}
v &= \pi \int_1^2 2^2 - y^2 \,\, dy = \pi \left( 4y - \frac{y^3}{3} \right) \Big|_1^2 \\
v &= \pi \left( 8 - \frac{8}{3} - \left( 4 - \frac{1}{3} \right) \right)
 =  \pi \left( 8 - \frac{8}{3} - 4 + \frac{1}{3} \right)  \\
v &= \pi ( 4 - \frac{7}{3 } ) \\
v &= \frac{5\pi}{3} \\
\end{align*}
The book's answer is $\frac{4\pi}{3}$. What did I do wrong?
Comment: The way you set up that integral is wrong.
Answer: When the triangle is rotated, a "washer" of outer radius $y$ and inner radius 1 is formed. Thus, the volume should be given by $\pi \int^{2}_{1} y^2 - 1 dy$, which should evaluate to $4 \pi/3$.
Answer: $$V=\pi\int\limits_1^2(y^2-1)dy=\pi\left(\frac{y^3}{3}-y\right)\big{|}_1^2=\pi\left(\frac{8}{3}-2-\left(\frac{1}{3}-1\right)\right)=\pi\left(\frac{2}{3}+\frac{2}{3}\right)=\frac{4\pi}{3}.$$
I think, the mistake is that you rotated around $x$-axis.
Comment: @Bob No. Our triangle is placed between $x=y$ and $x=1$, where $1\leq y\leq 2$ and we obtain:$\pi\int\limits_1^2(y^2-1^2)dy$. Draw it! Remember, our triangle is rotated around $y$-axis.
Comment: I am thinking that the general form of the answer would be $\int_a^b \pi ( R^2(x) - r^2(x)) \,\, dx$ where $R(x)$ is the outer radius and $r(x)$ is the inner radius. This formula assumes rotation about the $x$-axis. I am also thinking that $R(x) = 2$ in this case. Do I have that right?


ICMP maximum delay between echo request and reply sustained by a router NAT

Question: I was trying to slow down network on my server (arch linux vm) with netem
sudo tc qdisc add dev eth0 root netem delay 1600ms

With 1600ms delay client doesn't receive the echo reply packet though the packet is generated from the serverside. However It can withstand till 1500ms delay. If delay is less than that the echo reply is received in the client side.
But In the RFC does it specify any fixed max latency between echo request and echo reply ?
I have used ping as well as scapy to generate ICMP echo request packets and tcpdump to check the incoming echo reply packets. So it is not the problem of ping program.
So I think somewhere in the intermediate router the ECHO reply packets are being dropped due to the delay. So what is the maximum ammount of delay between echo request and reply that common routers sustain ?
Comment: You don't want this to be too big as it'll increases the chances of success for flood attacks.
Answer: There are recommendations in RFC 5508 "NAT Behavioral Requirements for ICMP", as to how long a router should retain state for ICMP packets, but there's no guarantee that anyone follows them.
Answer: Context: ICMP is not a stateful protocol.
In a scenario where you are sitting behind a router that NATs your traffic and you are pinging a device at the other side of the router, the router needs to keep track of your ICMP echo requests in order to let the respective ICMP echo replies in. How to do this if ICMP is not stateful? Just write an entry in the NAT table and keep it for a while.
The vendor of your router decided that 1500ms was a good compromise between functionality (ICMP works) and security:
1) The router can free up memory fast instead of keeping potentially millions of entries for one-sided ICMPs.
2) The router blocks unwanted ICMP echo replies potentially used for probing, fingerprinting, etc.
As side notes, the default ping timeout for Windows is 4 seconds and for many Linux distros is 10 seconds (in reference to the default "ping" tools in both OSes).


tcp connection in symbian c++ using Carbide

Question: I am new in symbian c++.
I have a assignment to write a program to "create a tcp/ip connection" in Symbian c++ using Carbide.
How can I do this?
Answer: Take a look at RSocketServ and RSocket. It is essentially analogous to BSD style sockets. You will need to also familiarize yourself with Symbian's asychronous APIs and Active Objects


No differences among aggregate standard deviations in MySQL?

Question: In a MySQL database I have a view (view_one) that contains a number of vegetation indices for several agricultural fields. The fields are further devided into plots, transects and points.
Here's the basic table design:
measure_id | field_id | plot_id | transect | point | vegetation_index_1 | .... |
I create a second view for the average values of the vegetation indices per plot and their respective standard deviation.
create view plot_avgs (measure_id, field_id, plot_id, avg_VI1, stddev_VI1, ...
as select measure_id, field_id, plot_id, avg(VI1), stddev_samp(VI1), ...
from view_one group by measure_id, field_id, plot_id;
What I want is the standard deviation of the plot, thus of the sample (stddev_samp()).
What MySQL gives me, however, is the stddev of the population, regardless the use of stddev(plot_avg), std(plot_avg), stddev_pop(plot_avg) or stddev_samp(plot_avg).
I'm suspecting that it has something to do with the group by aggregation I'm using.
Who can help me out? Thanks.
Answer: Ok I solved things myself:
if I use the very same as select ... from ... statement that I had intended to create the view with and either just run the query or use it to create a table, I do get the desired standard deviation of the sample (plot). As described in the question, this does not happen when creating the view.
So what I will do is use a permanent table instead of the view and have it updated by a trigger any time I'm inserting new data into the raw data table.


Does CXF 3.1.X uses Apache HttpComponents - HttpClient?

Question: Does CXF 3.1.x really uses Apache HttpComponents - HttpClient for http transport? Maven dependencies does not list "Apache HttpComponents" as dependency nor do I see any HttpClient class in the CXF jars? However CXF JIRA CXF-6704 discusses about it?
I need access to HttpClient in CXF for NTLM support per connection basis which CXF by default relies on Java 6+ Authenticator which is set per JVM.
Answer: CXF has a decoupled runtime layer. Mostly they start with cxf-rt-*.
In the case of transports, cxf-rt-transports-*.
Take a look at this. I believe that this is the HTTP Components transport runtime.
EDIT: I think this is for asynchronous transport - (looked at the pom after I posted this).
EDIT2: I did a little more research on this and found this documentation. It says that the async client can be used for synchronous calls by setting use.async.http.conduit bus property to true. This also specifically refers to NTLM authentication.
Comment: ah! I see * -hc (httpclient). Any idea how to access "HttpClient" instance CXF would use behind the scenes?
Comment: Does CXF uses HttpClient for async web services only?
Comment: I'm checking that now - I'm not 100% sure of the transport mech at the moment.
Comment: I've unfortunately led you into a very complex area of the CXF runtime. The HTTP Client functionality is provided as a Conduit. This is kind of an interceptor around a particular soap port. That said, you may have to slog through some source code: https://github.com/apache/cxf/blob/3.1.x-fixes/rt/transports/http-hc/src/main/java/org/apache/cxf/transport/http/asyncclient/AsyncHTTPConduit.java. As for getting the actual HttpClient, that is stored as an attribute on the Conduit implementation which you /may/ be able to reach from an interceptor.
Comment: I have already read CXF async and NTLM docs, unfortunately my requirements are not for async services
Comment: As I tried to state, this can be used for synchronous requests even though the underlying handler is asynchronous


Show $\widehat{\mathbb{Z}}$ is isomorphic to $S^1$

Question: I am trying to solve a problem from the book "Fourier series and integrals" by McKean. :)

Problem:
Prove that the group $\widehat{\mathbb{Z}}$ (the dual to $\mathbb{Z}$) is isomorphic to $S^1$.

I have been given that $e_n\mapsto n$ gives an isomorphism between $(S^1)^{\wedge}$ and $\mathbb{Z}$. I tried to kinda mimic the idea and I ended up constructing the function:
$$\begin{cases}f:S^1\to\mathbb{Z}^{\wedge}\\
\alpha\mapsto \chi_{\alpha}.
\end{cases}$$
In the book, they defined $S^1$ as "The circle $[0,1)$ under addition modulo $1$". When i read the sentence, I interpret it as the operation is addition on the unit circle, that is, if $\alpha,\beta\in S^1$ then $\alpha+\beta\in S^1$. But I don't think this is the correct interpretation of it, since $\alpha=\frac{1}{2}+\frac{\sqrt{3}}{2}i$ and $\beta=1$ are both on the unit circle, but $\alpha+\beta=\frac{3}{2}+\frac{\sqrt{3}}{2}$ and $|\alpha+\beta|=\sqrt{3}$ which shows us that $\alpha+\beta\not\in S^1$. Since I know multiplication of complex numbers on the unit circle gives us a group, I ended up thinking that's what they mean. Even though I don't know how I could interpret the above sentence as multiplication.

What we now have to do is to prove $f$ is a homomorphism, bijective, continuous and that its inverse is continuous.
Homomorphism:
For the homomorphism part, we have $f(\alpha\beta)=\chi_{\alpha\beta}(z)=(\alpha\beta)^z=\alpha^z\beta^z=f(\alpha)f(\beta)$.
$\text{ }$
Bijection:
I have two suggestions for the injectivity. Either we consider the equation $f(\alpha)=f(\beta)$ and try to conclude $\alpha=\beta$ or we try to prove kernel is trivial.
For the first suggestion, we have that $f(\alpha)=f(\beta)\iff\alpha^z=\beta^z\iff \alpha^z-\beta^z=0$ and I know a factor to $\alpha^z-\beta^z$ will be $\alpha-\beta$. Perhaps we can use it to conclude $\alpha=\beta$?
For my second suggestion, we have that $f(1)=\chi_1$ and $\chi_1(z)=1^z=1$, this shows us $1\in S^1$ gets mapped to the identity in $\mathbb{Z}^{\wedge}$. Perhaps it is trivial that this is the only element which gets mapped to $1\in\mathbb{Z}^{\wedge}$ and this is the better approach?
I have actually no suggestions or idea how to prove the map is surjective.
$\text{ }$
Continuity:
We have to check $f(\alpha)=\alpha^{z}$ is continuous for all $z$. But since $f(\alpha)=\alpha^z$ is a function from $S^1$ to $S^1$. Since it is a restriction of the map $\mathbb{C}\to\mathbb{C}$ defined by $\alpha\mapsto\alpha^z$, which is continuous, the map $f(\alpha)=\alpha^z$ is also continuous.
$\text{ }$
Continuity - Inverse function:
Unfortunately, I do not even know where to begin here. How Should I construct an inverse function? Can I "just" define it as $\chi_{\alpha}\mapsto\alpha?$ How do I prove it is continuous?

My questions are:

Is my interpretation of the group structure correct? If not, can you guide me in the right direction to help me understand this problem and be able to solve it?
Is my solution for the homomorphism part and the continuity correct?
What do you think about the proof of the injection? Do you think you
can help me conclude it is, indeed, injective?
Can you help me with the surjection and the inverse map?

I appreciate any help, thanks!
Answer: "Addition modulo $1$" is not the same as addition of real numbers. Also, where you say $\frac 1 2 + \frac{\sqrt3} 2$ is on the unit circle, I suspect you meant $\frac 1 2 + i \frac{\sqrt3} 2.$ Not notice that $\frac 1 2 + i\frac{\sqrt3} 2 = \cos\frac \pi 3 + i\sin\frac\pi 3,$ and $\frac \pi 3 = \frac{\text{whole circle}} 6.$ The number within $[0,1)$ to which $\frac 1 2 + i\frac{\sqrt 3} 2$ corresponds is therefore $\frac 1 6 = \frac{\pi/3}{\text{whole circle}}. \vphantom{\dfrac{\displaystyle\sum}1}$ Suppose you have $\alpha,\beta\in[0,1).$ Then
\begin{align}
& \Big(\cos(\alpha\times\text{whole circle)} + i\sin(\alpha\times \text{whole circle}) \Big) \\
\times {} & \Big( \cos(\beta\times\text{whole circle)} + i\sin(\beta\times \text{whole circle}) \Big) \\[10pt]
= {} & \cos((\alpha+\beta)\times\text{whole circle)} + i \sin((\alpha+\beta)\times\text{whole circle}),
\end{align}
where $\alpha+\beta$ is the sum modulo $1$ of $\alpha$ and $\beta.$ For example, if $\alpha= 0.8$ and $\beta = 0.9,$ then the ordinary sum of $\alpha$ and $\beta$ is $1.7,$ so the sum modulo $1$ of $\alpha$ and $\beta$ is $0.7.$
Comment: Thank you for catching that, I meant $\frac{1}{2}+\frac{\sqrt{3}}{2}i$. :) Also, thank you for the clarification, what is meant by modulo. I have to think about what you wrote for a little while. But now, when you say it, to think about modulo in the way you described it in this answer should really have been the thing which first appeared in my mind. It's like when you work in, for instance, $\mathbb{Z}_3$.There we have $2+2=4\equiv 1$ (mod $3$). So perhaps I should have brought my inspiration from there! :)


Understanding weight distribution in neural network

Question: I am training a deep neural network with several convolutional layers and a fully connected layer at the bottom and I am generating histograms of the weight distributions to try and understand how the network is training.
When looking at the graphs, I found something puzzling: most of the weights are near zero and only a small portion of the weights are getting very large. Why is this happening? Is this good and expected, or is this undesirable? Although I have only posted two example layers, this is happening throughout my network.

Additional information:

Data is very sparse and nearly binary (mostly 1's, very few 0's)
Input is normalized to be in the range 0-1
Not using L1/L2 yet since weights are mostly small
Activations are all leaky Relu (a=0.3)
I am performing batch normalization after each preactivation
Comment: What type of data are you using? Is it normalized to a range of [0,1] or did you make your data have a mean of 0 with a std of 1? Are you using any type of weight regularization (L1/L2)? What activation functions are you using?
Comment: It would be interesting to see whether if you pruned (set to 0) all small weights, the network would keep the same accuracy.
Answer: This is expected. Weights in a CNN form feature detectors, so that a certain pattern in an image is connected to strong weights, but the rest of the image pixels should not cause any activations in the next layer neurons.
Only a small fraction of neurons in a layer is activated every time an image is shown, and a small fraction of weights is needed to be large to activate (or suppress) any particular neuron. Moreover, the number of patterns a network needs to detect is fairly small, especially in the early layers. Therefore, overall connectivity for the network is usually very sparse.
The same reasoning applies to regulatization methods, such as L2/L1 - forcing the weights to be small makes the network more robust to noise in the data, and forces the network to learn only the features present in many images.
Answer: This is a really good question and there can be multiple explanations to this phenomenon. I will try to give out a non-exhaustive list for the same.

Sparse data ? You mean sparse/binary feature set ? hmmm...this can happen. See in that case your network does not require much parameters to encode this function. Every neural network is a function that encodes the feature set to the set of labels (one hot or whatever !).

Now lets talk about regularization because that has soooo much to do about the weight distribution of a network.

You are not using dropout. Try using it.  It is a great regularization technique which can actually help you to use all of your neurons. Without dropout on Sparse data, this is an expected event.

Try using L2, it might have some effect to your weight distribution.

You are using batchnorm and still getting this ? Hmm...I dont have an answer right now but if this post generates enough attention I will do a study on dropout vs batchnorm...!!


C# - Convert Jpg to Png with index transparency

Question: I need to efficiently convert a JPG image to a PNG image with index transparency.
Do you know if this is possible (or have code) to do this?
Notes:
Index-transparency is used to make only one colour transparent (typically white) while alpha-transparency is used to make all colours in the image transparent.
Answer - Continued
Thanks for help. Here is the code I just tested.
Bitmap b = new Bitmap(Image.FromFile("c:\\input2.jpg"));
b.MakeTransparent(Color.White);
b.Save("c:\\output2.png", ImageFormat.Png);

It converts fine, but there are various hues of white and if you are considering making white borders around your image transparent, you will need to make all these hues of white transparent too.
cheers.
Comment: That's not going to work well, since JPEG compression will destroy some of the color.
Comment: [This question][1] should contain all information you need for conversion.

  [1]: http://stackoverflow.com/questions/388677/can-you-make-an-alpha-transparent-png-with-c
Answer: try this... for me it works
Bitmap b = Image.FromFile(/*Image*/);
b.MakeTransparent(Color.White);
g.DrawImage(b, new Point(0, 0));


What do these three lines through the measure mean?

Question: The below image is from the Clarinet II part of Samuel Barber's Violin concerto. I don't have the faintest clue as to what these three lines could mean in this context. The notation is preceded by a volti subito, but I'm not sure what impact that has on this. Does anyone know what this could mean?
Comment: @Aaron Boosey and Hawkes (and other publishers, no doubt) published quite a bit of mid-20th-century music in photo-reproduced manuscript.  This was after photoreproduction became technologically feasible and before computer engraving existed or before it could produce satisfactory results -- between the 1940s and the 1980s, I would guess.
Comment: That helps, but what I really meant is that it appears to be hand written, at least in part. What is the source of the image itself?
Comment: Here is a copy of the score: https://www.dropbox.com/s/qoggq4wip9lkp8t/Barber_2nd-Clarinet.pdf?dl=0
Answer: The three lines are the equivalent of scribbling out the bottom staff line. It's just attempting to make clear that there is no more music to be played on that page.
In the interest of overkill, here's an expanded image of the portion of the score involved.
Comment: I'm going to add boxed text with an arrow marking "bottom of the page" to the next piece I write...
Comment: …and the **VS** at the end of the previous stave means "turn the page quickly"- copyist is trying to be really clear.
Comment: VS stands for Volto Subito which indeed means turn immediately.
Comment: @BrianTHOMAS  And I would like to thank the copyist for making the page turn occur during an extended rest!  :-)
Comment: @BrianTHOMAS You can contact my agent about the copyright fee.