Zabbix - issue with LLD: lots of became supported / became not supported

Question: I'm using zabbix 2.4.7 and Low Level Discovery who parses the output of varnishadm  backend.list command and send to zabbix, ex:
  [root@host4 ~]# varnishadm  backend.list | column -t | grep apetit -C 2
  be_bucket(127.0.0.1,,2676)                       1     probe  Healthy    (no    probe)
  be_apetitar(127.0.0.1,,2677)                     3     probe  Healthy  (no    probe)
  be_adeliveria(127.0.0.1,,2675)                   1     probe  Healthy  (no    probe)

Everything is working good (the itens are plotting, the graph is working and also the trigger), except for one detail:
On the host, I can see lots of "active check "varnish.backends[be_xxxx,conn]" is not supported".
And in the server logs, I can see lots of, for example:
7603:20160304:163453.953 item "host4:varnish.backends[be_apetitar,conn]" became supported
7603:20160304:163544.366 item "host4:varnish.backends[be_apetitar,conn]" became not supported: Unsupported item key.
7598:20160304:163654.251 item "host4:varnish.backends[be_apetitar,conn]" became supported
7603:20160304:163738.887 item "host4:varnish.backends[be_apetitar,conn]" became not supported: Unsupported item key.
7600:20160304:163854.747 item "host4:varnish.backends[be_apetitar,conn]" became supported
7610:20160304:163942.025 item "host4:varnish.backends[be_apetitar,conn]" became not supported: Unsupported item key.

-- edit this is the debug level:
- Server log NOK:
 20567:20160308:131759.808 item "barueri1.datac.foobar.com:varnish.backends[maisapps_server,conn]" became not supported: Unsupported item key.
 20567:20160308:131759.808 item "barueri1.datac.foobar.com:varnish.backends[maisapps_server,status]" became not supported: Unsupported item key.

- Agent log NOK:
[tiago.cruz@barueri1 ~]$ cat zabbix_agentd.log | grep 131 | grep maisapps_server -C 2 --color
 18150:20160308:120233.131 listener #3 [waiting for connection]
 18151:20160308:120258.815 JSON before sending [{"request":"agent data","data":[{"host":"barueri1.datac.foobar.com","key":"check.s3.logs","value":"OK - Files were sent to Amazon S3","clock":1457438573,"ns":791666712},{"host":"barueri1.datac.foobar.com","key":"check.timezone","value":"OK: Server is configured with UTC (environment: production)","clock":1457438573,"ns":802349922},{"host":"barueri1.datac.foobar.com","key":"proc.num[crond]","value":"3","clock":1457438577,"ns":812777346}],"clock":1457438578,"ns":813168976}]
 18151:20160308:120747.131 In add_check() key:'varnish.backends[maisapps_server,conn]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:120747.131 End of add_check()
 18151:20160308:120747.131 In add_check() key:'varnish.backends[maisapps_server,status]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:120747.131 End of add_check()
 18151:20160308:120747.131 In add_check() key:'varnish.backends[nginx,conn]' refresh:120 lastlogsize:0 mtime:0
--
 18151:20160308:131347.649 sending [{"request":"active checks","host":"barueri1.datac.foobar.com","host_metadata":"Linux vpc-spo 184f9616-a7d1-11e4-858b-435988766bcf"}]
 18151:20160308:131347.649 before read
 18151:20160308:131347.657 got [{"response":"success","data":[{"key":"check.drift[0.500000000]","key_orig":"check.drift[{$DRIFT_TIME}]","delay":30,"lastlogsize":0,"mtime":0},{"key":"check.raid","delay":3600,"lastlogsize":0,"mtime":0},{"key":"check.readonly","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.s3.logs","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.timezone","delay":300,"lastlogsize":0,"mtime":0},{"key":"proc.num[crond]","delay":60,"lastlogsize":0,"mtime":0},{"key":"proc.num[rsyslogd]","delay":60,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish_collect","delay":30,"lastlogsize":0,"mtime":0}]}]
 18151:20160308:131347.657 In parse_list_of_checks()
 18151:20160308:131347.657 In disable_all_metrics()
--
 18151:20160308:131347.658 In add_check() key:'proc.num[rsyslogd]' refresh:60 lastlogsize:0 mtime:0
 18151:20160308:131347.658 End of add_check()
 18151:20160308:131347.658 In add_check() key:'varnish.backends[maisapps_server,conn]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:131347.658 End of add_check()
 18151:20160308:131347.658 In add_check() key:'varnish.backends[maisapps_server,status]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:131347.658 End of add_check()
 18151:20160308:131347.658 In add_check() key:'varnish.backends[nginx,conn]' refresh:120 lastlogsize:0 mtime:0
--
 18151:20160308:131357.671 buffer: new element 0
 18151:20160308:131357.671 End of process_value():SUCCEED
 18151:20160308:131357.671 active check "varnish.backends[maisapps_server,conn]" is not supported
 18151:20160308:131357.671 In process_value() key:'barueri1.datac.foobar.com:varnish.backends[maisapps_server,conn]' value:'Unsupported item key.'
 18151:20160308:131357.671 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:1/100
 18151:20160308:131357.674 JSON before sending [{"request":"agent data","data":[{"host":"barueri1.datac.foobar.com","key":"proc.num[crond]","value":"1","clock":1457442837,"ns":671355807}],"clock":1457442837,"ns":671543386}]
--
 18151:20160308:131357.680 buffer: new element 0
 18151:20160308:131357.680 End of process_value():SUCCEED
 18151:20160308:131357.680 active check "varnish.backends[maisapps_server,status]" is not supported
 18151:20160308:131357.680 In process_value() key:'barueri1.datac.foobar.com:varnish.backends[maisapps_server,status]' value:'Unsupported item key.'
 18151:20160308:131357.680 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:1/100
 18151:20160308:131357.680 Will not send now. Now 1457442837 lastsent 1457442837 < 5
--
  9126:20160308:131402.661 zbx_popen(): executing script
 18151:20160308:131402.691 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:5/100
 18151:20160308:131402.694 JSON before sending [{"request":"agent data","data":[{"host":"barueri1.datac.foobar.com","key":"varnish.backends[maisapps_server,conn]","value":"Unsupported item key.","state":1,"clock":1457442837,"ns":680054682},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[maisapps_server,status]","value":"Unsupported item key.","state":1,"clock":1457442837,"ns":680347852},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[nginx,conn]","value":"Unsupported item key.","state":1,"clock":1457442837,"ns":680652456},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[nginx,status]","value":"Unsupported item key.","state":1,"clock":1457442837,"ns":680941634},{"host":"barueri1.datac.foobar.com","key":"proc.num[rsyslogd]","value":"1","clock":1457442838,"ns":690664930}],"clock":1457442842,"ns":692053472}]
 18151:20160308:131402.700 JSON back [{"response":"success","info":"processed: 5; failed: 0; total: 5; seconds spent: 0.000085"}]
 18151:20160308:131402.700 In check_response() response:'{"response":"success","info":"processed: 5; failed: 0; total: 5; seconds spent: 0.000085"}'
--
 18151:20160308:131747.525 sending [{"request":"active checks","host":"barueri1.datac.foobar.com","host_metadata":"Linux vpc-spo 184f9616-a7d1-11e4-858b-435988766bcf"}]
 18151:20160308:131747.525 before read
 18151:20160308:131747.533 got [{"response":"success","data":[{"key":"check.drift[0.500000000]","key_orig":"check.drift[{$DRIFT_TIME}]","delay":30,"lastlogsize":0,"mtime":0},{"key":"check.raid","delay":3600,"lastlogsize":0,"mtime":0},{"key":"check.readonly","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.s3.logs","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.timezone","delay":300,"lastlogsize":0,"mtime":0},{"key":"proc.num[crond]","delay":60,"lastlogsize":0,"mtime":0},{"key":"proc.num[rsyslogd]","delay":60,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish_collect","delay":30,"lastlogsize":0,"mtime":0}]}]
 18151:20160308:131747.533 In parse_list_of_checks()
 18151:20160308:131747.533 In disable_all_metrics()
--
 18151:20160308:131747.534 In add_check() key:'proc.num[rsyslogd]' refresh:60 lastlogsize:0 mtime:0
 18151:20160308:131747.534 End of add_check()
 18151:20160308:131747.534 In add_check() key:'varnish.backends[maisapps_server,conn]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:131747.534 End of add_check()
 18151:20160308:131747.534 In add_check() key:'varnish.backends[maisapps_server,status]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:131747.534 End of add_check()
 18151:20160308:131747.534 In add_check() key:'varnish.backends[nginx,conn]' refresh:120 lastlogsize:0 mtime:0
--
 18151:20160308:131754.620 buffer: new element 1
 18151:20160308:131754.620 End of process_value():SUCCEED
 18151:20160308:131754.620 active check "varnish.backends[maisapps_server,conn]" is not supported
 18151:20160308:131754.620 In process_value() key:'barueri1.datac.foobar.com:varnish.backends[maisapps_server,conn]' value:'Unsupported item key.'
 18151:20160308:131754.620 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:2/100
 18151:20160308:131754.620 Will not send now. Now 1457443074 lastsent 1457443074 < 5
--
 18151:20160308:131754.620 buffer: new element 2
 18151:20160308:131754.620 End of process_value():SUCCEED
 18151:20160308:131754.620 active check "varnish.backends[maisapps_server,status]" is not supported
 18151:20160308:131754.620 In process_value() key:'barueri1.datac.foobar.com:varnish.backends[maisapps_server,status]' value:'Unsupported item key.'
 18151:20160308:131754.620 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:3/100
 18151:20160308:131754.620 Will not send now. Now 1457443074 lastsent 1457443074 < 5
--
 18148:20160308:131759.421 listener #1 [waiting for connection]
 18151:20160308:131759.641 In send_buffer() host:'wonderland1.datac.foobar.com' port:10051 values:8/100
 18151:20160308:131759.644 JSON before sending [{"request":"agent data","data":[{"host":"barueri1.datac.foobar.com","key":"check.s3.logs","value":"OK - Files were sent to Amazon S3","clock":1457443074,"ns":609757471},{"host":"barueri1.datac.foobar.com","key":"check.timezone","value":"OK: Server is configured with UTC (environment: production)","clock":1457443074,"ns":620447983},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[maisapps_server,conn]","value":"Unsupported item key.","state":1,"clock":1457443074,"ns":620767313},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[maisapps_server,status]","value":"Unsupported item key.","state":1,"clock":1457443074,"ns":621065121},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[nginx,conn]","value":"Unsupported item key.","state":1,"clock":1457443074,"ns":621354240},{"host":"barueri1.datac.foobar.com","key":"varnish.backends[nginx,status]","value":"Unsupported item key.","state":1,"clock":1457443074,"ns":621642390},{"host":"barueri1.datac.foobar.com","key":"proc.num[crond]","value":"2","clock":1457443077,"ns":631686558},{"host":"barueri1.datac.foobar.com","key":"proc.num[rsyslogd]","value":"1","clock":1457443078,"ns":640911821}],"clock":1457443079,"ns":641313858}]
 18151:20160308:131759.649 JSON back [{"response":"success","info":"processed: 8; failed: 0; total: 8; seconds spent: 0.000108"}]
 18151:20160308:131759.649 In check_response() response:'{"response":"success","info":"processed: 8; failed: 0; total: 8; seconds spent: 0.000108"}'

- Server log OK:
 20557:20160308:132211.253 item "barueri1.datac.foobar.com:varnish.backends[maisapps_server,conn]" became supported
 20557:20160308:132211.254 item "barueri1.datac.foobar.com:varnish.backends[maisapps_server,status]" became supported

- Agent log OK:
[tiago.cruz@barueri1 ~]$ cat zabbix_agentd.log | grep 132 | grep maisapps_server -C 2 --color
 18151:20160308:132347.012 sending [{"request":"active checks","host":"barueri1.datac.foobar.com","host_metadata":"Linux vpc-spo 184f9616-a7d1-11e4-858b-435988766bcf"}]
 18151:20160308:132347.012 before read
 18151:20160308:132347.021 got [{"response":"success","data":[{"key":"check.drift[0.500000000]","key_orig":"check.drift[{$DRIFT_TIME}]","delay":30,"lastlogsize":0,"mtime":0},{"key":"check.raid","delay":3600,"lastlogsize":0,"mtime":0},{"key":"check.readonly","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.s3.logs","delay":300,"lastlogsize":0,"mtime":0},{"key":"check.timezone","delay":300,"lastlogsize":0,"mtime":0},{"key":"proc.num[crond]","delay":60,"lastlogsize":0,"mtime":0},{"key":"proc.num[rsyslogd]","delay":60,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[maisapps_server,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,conn]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish.backends[nginx,status]","delay":120,"lastlogsize":0,"mtime":0},{"key":"varnish_collect","delay":30,"lastlogsize":0,"mtime":0}]}]
 18151:20160308:132347.021 In parse_list_of_checks()
 18151:20160308:132347.021 In disable_all_metrics()
--
 18151:20160308:132347.021 In add_check() key:'proc.num[rsyslogd]' refresh:60 lastlogsize:0 mtime:0
 18151:20160308:132347.021 End of add_check()
 18151:20160308:132347.021 In add_check() key:'varnish.backends[maisapps_server,conn]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:132347.021 End of add_check()
 18151:20160308:132347.021 In add_check() key:'varnish.backends[maisapps_server,status]' refresh:120 lastlogsize:0 mtime:0
 18151:20160308:132347.021 End of add_check()
 18151:20160308:132347.022 In add_check() key:'varnish.backends[nginx,conn]' refresh:120 lastlogsize:0 mtime:0

Thanks!
Comment: Would it be possible to post a longer part of DebugLevel=4 agent log that shows the case when the item is supported and how it then transitions to the unsupported state? Could it be that you have two agents running on the host, where one supports the item and the other does not?
Comment: @asaveljevs, I added the debug level as you asked. No, there is no two agents running, I'm using Chef to make sure and this issue also happen in others hosts using this template. Thanks!
Comment: Unfortunately, the "Agent log OK" part seems to be too short - it does not show that the check was processed by the agent correctly and what preceded it. Would it be possible to show a longer, continuous DebugLevel=4 agent log that shows the full transition between supported and not supported (ideally, both ways)? How are these items implemented - through user parameters or loadable modules?
Answer: Solved!

'Item prototypes' changed from 'Agent (active)' to 'Zabbix Trapper'
'Item' varnish_backends_collect changed to 'Zabbix agent (active)'

End configuration:

Discovery = varnish_backends = Zabbix agent = 600 seconds
Item prototypes = varnish.backends[{#BACKEND},conn] = Zabbix trapper
Item = varnish_backends_collect = Zabbix agent (active) = 120 seconds

The last flapping log is from yesterday:
 20560:20160308:154433.256 item "miami97.com:varnish.backends[bogota01,status]" became supported
 20560:20160308:154433.256 item "miami97.com:varnish.backends[bogota02,status]" became supported
 20567:20160308:154435.467 item "miami97.com:varnish_backends_collect" became supported

Answer: I had a similar issue, but it was all SNMP and NetApps and other devices. No Zabbix agent or trapper involved at all.
Using a SNMPv1 test against a SNMPv2 device (like a NetApp or NetBot) will cause the test to flap in exactly the same way. Updating the flapping test items in Zabbix to SNMPv2 will solve the problem.


Get .txt file instead of .jpg - using Webclient and DownloadFile();

Question: Get .txt file instead of .jpg - using Webclient and DownloadFile();
I'm trying to download the .jpg from this URL:
http://1.bp.blogspot.com/_pK6J3MTn5co/S6kuH3aqbeI/AAAAAAAACUY/06axvmjU91k/s1600-h/avengers02_B&W_UL.jpg
Using this code:
private void TEST_button1_Click(object sender, EventArgs e)
{
    WebClient MyDownloader = new WebClient();
    MyDownloader.DownloadFile(@"http://1.bp.blogspot.com/_pK6J3MTn5co/S6kuH3aqbeI/AAAAAAAACUY/06axvmjU91k/s1600-h/avengers02_B&W_UL.jpg", @"c:\test.jpg");
}

However, when I run this, I end up with a file called test.jpg, which contains html mark up... :

<title>avengers02_B&W_UL.jpg (image)</title>

How can I download the actual .jpg?
Any help is greatly appreciated - thank you!
Answer: If server returns HTML to your request at particular Url you can't do much to force it to return something else at that Url.
What you can do is parse response with HtmlAgilityPack and find url to actual image and get it in another request.
Comment: I'm probably missing something, but if you paste, or click on that URL it shows the correct image...  What am I not understanding?
Comment: @ChrisL - check view source in your browser or use Fiddler to see what server actually returns. You'll see HTML with IMG inside - nothing you can do about server's behavior (unless you know for sure that there is a way to set headers/query parameters for this particular type of requests on this particular server to return image directly).
Answer: There is a way to do it. First you download the HTML content to a string and extract the correct image URL. Then use the correct URL to download the file.
 WebClient client = new WebClient();
 var path = @"http://1.bp.blogspot.com/_pK6J3MTn5co/S6kuH3aqbeI/AAAAAAAACUY/06axvmjU91k/s1600-h/avengers02_B&W_UL.jpg";

 var content = client.DownloadString(path);
 System.Text.RegularExpressions.Regex regex = new Regex(@"(?<=]*?src=(?<q>['""]))(?<url>.+?)(?=\k<q>)");
 var match = regex.Match(content);
 if (match.Success)
 {
     client.DownloadFile(match.Value, @"e:\test1.jpg");
 }

Comment: I'd prefer to use HtmlAgilityPack over regex to parse html, although this *specific* case is trivial enough for it not to matter.
Comment: Thanks for the awesome code -- this is definitely what I can use, and I get it.  Sorry for being dense, but hugely appreciate the patience and help!
Answer: Clicking that link causes 2 downloads, first a page of HTML (mislabelled with suffix .jpg), and next an image in the HTML.
So perhaps you need to fetch the url of the img tag in the HTML fetched by the previous request?
http://1.bp.blogspot.com/_pK6J3MTn5co/S6kuH3aqbeI/AAAAAAAACUY/06axvmjU91k/s1600/avengers02_B%26W_UL.jpg
I'm guessing that removing -h from the original URL might point to the actual file that you're after.
Here's hoping that you have permission to scrape these files...
Comment: This completely makes, thanks for making me see how to get this done!


Load a Module versus JavaScript embedded in Theme

Question: I am new to theming Drupal and I am trying to create a responsive website based on Bootstrap with an image slider at the top.
I know that there are modules like Views Slideshow and FlexSlider.  However, I don't want to install a module when I don't have to.  I would like to just create a content type specific to the slider with just the fields that I need and have the slider that is "embedded" in the theme simply load the content depending on whether it was published or not.
Can this be done?  Are there benefits/drawbacks to what I am attempting to do here?
I am not fluent in the Drupal theming lingo yet so I don't know if I am asking the question the right way.
Comment: You could create a content type, a view to display published nodes, and alter the rendering of the view to use Bootstrap's carousel.
Answer: You can do pretty much anything you want with custom code, including this.
The question is should you use custom code or an existing module that provides the functionality you're after.
The answer depends on your specific situation.
For example if you have limited time/budget it may be better to go with something that's already available, like views slideshow, and keep as much of your budget as possible for other tasks.
However there might be some reason that option isn't ideal, like maybe the modules have lots of functionality you don't need and you want to code something more specific to your requirements.
Another possible issue might be that the existing module doesn't do something exactly the way you need it to, so then you have to decide whether you write a patch to modify that module or if you want to write custom code.
This question deals with pros and cons of using custom code instead of existing modules.


Is $|\mathbb{R}^{\omega_1}| > \aleph_1$?

Question: Intuatively I'd guess no, since $|\omega_1^{\omega_1}| = \aleph_1$.
But then there's cardinal exponentiation $\aleph_1^{\aleph_1}\geq2^{\aleph_1}=\aleph_2$.
A formal explanation would be appreciated.
Comment: $2^{\aleph_1}$ is not necessarily $\aleph_2$; it is consistent with $\mathsf{ZFC}$ that $2^{\aleph_1}$ be any cardinal of cofinality greater than $\omega_1$. $\left|\Bbb R^{\omega_1}\right|=\left|\left(2^{\aleph_0}\right)^{\aleph_1}\right|=2^{\aleph_0\cdot\aleph_1}=2^{\aleph_1}>\aleph_1$.
Comment: When you write $\omgea_1^{\omega_1}$, that makes me think about *ordinal* exponentiation. But when you write $\Bbb R^{\omega_1}$ that makes me think about *all* functions between the two sets, which would correspond to *cardinal* exponentiation. If you mean something else, you should specify what.
Answer: Write $\Bbb R=2^{\aleph_0}$, so $$|\Bbb R|^{\aleph_1}=(2^{\aleph_0})^{\aleph_1}=2^{\aleph_0\cdot\aleph_1}=2^{\aleph_1}>\aleph_1,$$
where I used that for infinite cardinals $\kappa,\lambda$ we have $\kappa\cdot\lambda=\max\{\kappa,\lambda\}$.
Also note that $2^{\aleph_1}=\aleph_2$ is independent of $\mathsf{ZFC}$, but nonetheless we have $2^{\aleph_1}>\aleph_1$ by Cantor's theorem.
Comment: You didn't really need $\kappa\cdot\lambda=\max\{\kappa,\lambda\}$ since you could have simply written $2^{\aleph_0\cdot\aleph_1}\ge2^{\aleph_1}$ or just $(2^{\aleph_0})^{\aleph_1}\ge2^{\aleph_1}$.
Comment: It felt cleaner to write equalities as far as possible, but that's a good point
Comment: I see. I would have avoided using $\kappa\lambda=\max(\kappa,\lambda)$ because that's a more difficult result that the OP may not have learned yet, given the level of the question.


How to work on offline mode in SUP(sybase unwired platform) for ios application

Question: I am pure iOS developer I dont know any thing about Sybase Unwired Platform, I have some task related to SUP offline mode, I dont have any idea what I do for offline mode, I already research on http://infocenter.sybase.com but I dont get any help from this link.
Please help me if you know any useful link or any sample source code or anything related to the same share with me.
Thanks in Advance.
Answer: After 2.1#3, the method offlineLogin got deprecated. I am not sure how to get it done now.
Answer: SUP APIs works very intelligently. You can login to SCC in offline mode as per infocenter:
OfflineLogin authenticates against the last successfully authenticated credentials. There is no communication with Unwired Server in this method.
You have to call below method instead of onlineLogin: or login:
- (BOOL)offlineLogin:(SUPString)user password:(SUPString)pass;

While subscribing the package in offline mode, it will return you the previous sync persisted data.
Answer: Any SUP App has to be ONLINE on there first run.  This uses the following method
onlineLogin method of the DB file. For all other run you can use offlineLogin method of the DB file.
Answer: You can achieve the offline capabilities like creating local database(sqllite) in device by using phonegap. I have tested it in my hybrid app and it is working fine.


What are the consequences of not deallocating memory from a sql cursor on sql server 2000?

Question: I would truly appreciate anyone who could point me toward some resources or share some personal experience on this.
I noticed that my sql 2k box cpu was pegged at near 100%, it spiked at almost exactly the same time that an etl process which used a sql cursor but did not deallocate memory failed.  I stopped and started the service, and it fixed the problem.
This etl process failed earlier in the morning and when the server spiked the developer was rerunning the process manually.
So in conclusion: my question is this: it seems very obvious that the etl process may have caused the server to spike, however, i would like to know if/how/why this procedure which did not deallocate memory could cause this problem.  The query looked similar to this:
<code>USE AdventureWorks
GO
DECLARE @ProductID INT
DECLARE @getProductID CURSOR
SET @getProductID = CURSOR FOR
SELECT ProductID
FROM Production.Product
OPEN @getProductID
FETCH NEXT
FROM @getProductID INTO @ProductID
WHILE @@FETCH_STATUS = 0
BEGIN
Insert into sometable values(@ProductID)
FETCH NEXT
FROM @getProductID INTO @ProductID
END
CLOSE @getProductID
GO
</code>
Thank you very much.
Comment: 95% of the time, if you see a cursor being used, the proc is wrong and needs to be rewritten -- SQL is a set-based language, so use the natural sets that are the results of queries.  In this case, this should be a single INSERT using the results of the SELECT -- INSERT INTO sometable SELEcT ProductID from Production.Product.
Comment: Right i couldn't agree more, i didnt even know the dev had used a cursor
Comment: Why do you think the missing DEALLOCATE statement is the cause of the problem? I'd suggest having a loop for a set-based operation is the bigger of the two problems, but as a best practice I would always use both CLOSE and DEALLOCATE for cursors.
Answer: Yikes.  You have a nested cursor, which could create a very large number of resources allocated.  I won't say that's definitely your problem, but it is definitely a problem.


How can I obtain a list of currently running applications?

Question: I want to programmatically obtain a list of running (desktop) applications, and then I want to display this list to the user. It should be something similar to the application list displayed in the Windows Task Manager.
How can I create this in C#? Specifically, I need a way to obtain that list of currently running applications.
Answer: You can use the Process.GetProcesses method to provide information about all of the processes that are currently running on your computer.
However, this shows all running processes, including ones that are not necessarily shown on the taskbar. So what you'll need to do is filter out those processes that have an empty MainWindowTitle.The above-linked documentation explains why this works:

A process has a main window associated
  with it only if the process has a
  graphical interface. If the associated
  process does not have a main window
  (so that MainWindowHandle is zero),
  MainWindowTitle is an empty string
  ("").

So, you could use something like the following code, which will print out (to a console window) a list of all currently running applications that are visible on your taskbar:
Process[] processes = Process.GetProcesses();
foreach (var proc in processes)
{
   if (!string.IsNullOrEmpty(proc.MainWindowTitle))
        Console.WriteLine(proc.MainWindowTitle);
}

Comment: Processes do not have monitor affinity, @Alok. Some processors don't display any user interface whatsoever, and thus don't appear on a monitor. Other processes may display multiple windows, and those windows may appear on several different monitors. You need to choose a window, and then see which monitor that window is on. In Win32, you'd use the `MonitorFromWindow` function, passing in a window handle (`HWND`) and getting a monitor handle (`HMONITOR`). You can P/Invoke that in .NET, or you can use `System.Windows.Forms.Screen.FromPoint` directly from WinForms.
Comment: I apologize for this 2-years-later comment, but I'm looking for a similar solution, and wanted to mention that the above solution does not work (AFAIK) in cases like WORD or ACRO-READER that may have more than one document opened on the desktop.  In that case, it appears that only the topmost window is listed as an application using the above method.  If I'm wrong I apologize, but assuming I'm right, I'd like to know how to list all windows, even if they are from the same underlying process (like acrord or word).
Comment: @Ed. Yeah, this enumerates processes, but those applications are only running a single process. It's inconsequential that the process manages multiple top-level windows. You can enumerate top-level windows using the [`EnumWindows` function](http://msdn.microsoft.com/en-us/library/windows/desktop/ms633497.aspx). Call this function and specify a pointer to a callback function. The callback function will be called once for each top-level window on the screen.
Comment: if we have 2 display monitors, how to know which monitor this process is running?


Is there a specific name for the use of IV chord over the V in the bass, e.g. F/G in the key of C

Question: You see this chord a fair amount, usually leading to the tonic as a sort of souped-up plagal cadence (plagal in the treble, perfect in the bass).
Is there a specific name for this, other than "4 over 5"?
Alternatively, a name for the chord itself, a major triad over the note a tone above it.  I suppose that at a stretch it could be considered a special case of a 13 chord, but only in the way that a 6/9 also could be (aka not really at all).
Comment: Possible duplicate of [Is there a term for chords like F/G and C/D?](https://music.stackexchange.com/questions/42507/is-there-a-term-for-chords-like-f-g-and-c-d)
Comment: @MattL. thanks, I hadn't seen that.  Is "none of the answers there really give me a satisfactory answer to the question" good enough grounds to keep this one open?  probably not, in which case, probably best to mark this as a duplicate
Comment: You could try to explain (in your question) why none of those answers are satisfactory. After all, in the key of C that chord is basically (and functions as) a G7/sus4 chord (with an added 9th).
Comment: @Some_Guy I don't think the question is a duplicate. You're asking for a more specific theoretical term, not a broad label, which is what that question is doing.
Answer: Some popular-music scholars call this the "rock dominant"; I know I've encountered this term in Mark Spicer's (Ac)cumulative Form in Pop-Rock Music:

...the distinctive voicing of the so-called 'rock dominant', a polychord which can best be thought of as a close-position IV triad in the right hand sounding above ^5 in the bass, hence conflating subdominant and dominant functions.

In a footnote at the end of that sentence (see page 38) he says:

Because the rock dominant is somewhat difficult to voice on the guitar, it seems to be more characteristic within the harmonic language of keyboard-based pop and rock songs. Two obvious examples are 10cc's 'I'm not in love' (1975) and Paul McCartney's Wings song 'With a little luck' (1978), each of which opens with a rock dominant (played on electric piano) as its signature hook. The chord likely has varied origins, both in gospel piano playing (indeed, the chord is as prevalent in soul music as it is in rock) and in American popular song from the first half of the twentieth century, where often at cadences one finds the melody moving directly to 1, without a preceding 2 or other note that would support a pure V, as in the final cadence of Cole Porter's 'Night and day' (1932). A true rock dominant, with
a IV triad in the right hand over 5 in the bass, can be found at the final cadence of Rodgers and Hart's 1938 song 'Who are you?' (not to be confused with the aforementioned Who song of the same name).
Comment: Interesting quote. However, the term "rock dominant" sounds like it comes from someone who comes from a tradition that is very remote to rock. In what most people call rock music, that form of a dominant hardly ever occurs. You are much more likely to find it in soul/funk/pop music (as also acknowledged by Spicer). Also, Mr. Spicer doesn't seem to know much about playing the guitar when he writes "somewhat difficult to voice on the guitar". This chord is very easily played on the guitar, in several voicings, and most (pop, etc.) guitar players know how to play it.
Comment: Which is why I questioned Mark Spicer's "a polychord which can best be thought of as a close-position IV triad in the right hand..."
Comment: great contribution, thanks.  I've always though of it as a really gospelly chord personally, but it's nice to see a written reference to it. (and he mentions its gospel roots too)
Comment: Doesn't a 'polychord' require TWO triads, not just a 'slash bass' note?
Comment: @LaurencePayne: Yes, a polychord is indeed two chords above each other, not even necessarily triads. What is meant here is usually called "slash chords", i.e.,  chords over a bass note other than the root. They could be either simple inversions like C/E, or so-called hybrids like G/C.
Answer: This chord is extremely common in modern musical theater, to the point that I sometimes call it "the musical theater chord". I've also had a lot of arguments about how to properly analyze it.
I've run into a lot of people who absolutely insist on calling it G9sus4. With G in the bass it's natural to call it some kind of G chord, but I think "9sus4" does a poor job of explaining the other notes. In particular, it doesn't explain the lack of a D. This D isn't omitted because of voicing issues (the way it is in a #11 chord, for example) or because we've run out of fingers, it's explicitly missing when it could easily be present, and that's a tough thing to account for in the analysis. Furthermore, accounting for the C as a suspended 4th has several problems:

The lack of a D weakens the sound of a suspension
It's often doubled (on piano, the right hand often has C-F-A-C), which is terrible voice leading for a suspension.
It never resolves to a B, and instead the voices on C stay on C as the chord resolves.

I think this analysis has too many holes to hold water. Calling it an 11th chord at least avoids these problems with the C, but still doesn't account for the missing D or explain the very particular voicing in the treble.
Calling it an F chord happily explains all of the treble notes, but that G in the bass is hard to explain. It would be Fadd9, with that 9 in the bass and explicitly not present in the treble, which is a flimsy analysis.
Whenever there's a chord that's ambiguous to analyze, I use a tool that I call the "simplification test". If you hypothesize that a chord is a G[insert crazy extensions], then a plain G chord (or maybe the appropriate 7th chord) should function the same in its place and merely have less color. At least in the musical theater context, the chord that most strongly passes this test is F. I agree completely with your idea of it being a plagal cadence, with a little bit of authentic motion in the bass to strengthen it (and also add a bit of tension to be resolved).
Chord symbols in sheet music have two main purposes:

To explain the theory
To provide instructions to a musician and get them to play the notes you want

The theory debate may rage on, but I hope everyone can agree that as a practical instruction to players, F/G is the label that will cause people to play the intended notes. A guitarist will see this and play an F chord, a bassist will play a G, and a pianist will play F-A-C in the right and G octaves in the left, and that's exactly what we want. G11 or G9sus are both going to cause most people to play Gs and/or Ds in the treble.
Comment: thanks for the answer.  I guess when talking in a key agnostic way, I think I'm going to have to stick with "4 over 5". I suppose a problem is that it only works for when it literally is the 4/5 chord; 5 over 6 and 1 over 2 are also possible diatonically (if less common), and of course the mixolydian sounding b7/1, and you can use it as a secondary dominant too.  I agree that calling it 9sus4 isn't really clear or theoretically that justifiable but that then leaves me with no real word for it (other than F/G, Bb/C etc., which is fine in a piece but doesn't work as a general term).  Oh well!
Comment: "over 9" sounds weird since a 9, by definition should be at the top of the chord (that's why it's not a 2).  I suppose we could call it an "11 9 7"" chord, by analogy with figured bass (how people often call a third inversion a 6-4 chord), but that's a mouthful too.
Comment: I'd say the 'musical theatre chord' was a tonic with added 9.   It seems obligatory to end a ballad with it.  And it infects the 'keyboard ostinato' style of accompaniment, beloved of composers without the resources or inclination to orchestrate more interestingly.
Answer: No, I don't know of any generally-understood and unambiguous name for this chord, other than IV/V or F/G.
You'll see it labelled 'G11'.   But you'll also see 'G11' used for Dm7/G and even for G7(sus4).  And for any subset of GBDFAC that includes the root, 3rd and 11th. Lots of potential for confusion there.
And you'll see it called 'G9sus4'.  But, as @MattPutnam says this ignores the absence of the 5th, D.  And although this makes no functional difference, it's a very different colour if you add the D.  Perhaps those who dislike the 'slash bass' would be happier to label it a 'dominant pedal'.  Bach did that a lot!
Comment: I've always found that "11" thing you sometimes see in lead sheets to be a really crap summary of the chord too, and it can result in the ugliest of voicings from a reading rhythm section that doesn't know the song well (or how to use their ears haha)
Comment: I'd (just maybe) use 'G11' as shorthand for F/G in a keyboard part.   I wouldn't trust a guitarist with it!
Comment: amen to that
Comment: I'm going to mark this as the answer, because it's clear and concise, and covers the key points.  However, if someone comes along with a better answer, I will not hesitate to revoke this green tick I have bestowed upon thee ;)
Comment: You mean 'don't feel too good about it'?   :-)
Comment: If it's any consolation, I think the problem is our system of chord nomenclature as a whole, rather than with your answer in particular!
Comment: I hope I'm wrong though, and someone will swing in with some name for it that neither of us are aware of.  If "drop 2" wasn't taken by something else, I'd vote for it to be used for this!
Comment: Oh, I expect someone will make one up.  But what's the point? No-one else will understand it.
Answer: I'm sure I've seen it written as '11'. The intro. chords to Midnight at the Oasis use several. E chord over F# bass is called F#11.
It becomes a sort of sus chord, with a b7. 7+4=11.
It's often just written as a slash chord, like your F/G.


Given $0\leq f(x) \leq xh(x)$ and the existence of $\frac{\mathrm d}{\mathrm dx}\big(xh(x)\big)(0)$, can we conclude that $f'(0)$ exists?

Question: I have some troubles with the following exercise. I think that it's trivial but I can't see the solution.
Let $f,h:\mathbb{R}\to\mathbb{R}$ be continuous functions over $\mathbb{R}$. Given that for all $x\in\mathbb{R}$ we have $$0\leq f(x)\leq xh(x)$$ and that the derivative of $xh(x)$ exists at $0$ (in fact, $\big(xh(x)\big)'(0)=h(0)$). Can we conclude that $f'(0)$ exists?
Firts, because $0\leq f(0)\leq 0h(0)=0$, we have that $f(0)=0$. Next
$$f'(0)=\lim\limits_{x\to 0}\dfrac{f(x)-f(0)}{x-0}=\lim\limits_{x\to 0}\dfrac{f(x)-0}{x}=\lim\limits_{x\to 0}\dfrac{f(x)}{x}.$$
By the inequality of the hypothesis, we have $0\leq \dfrac{f(x)}{x}\leq h(x)$. Thus
$$0\leq \lim\limits_{x\to 0}\dfrac{f(x)}{x}\leq \lim\limits_{x\to 0} h(x)=h(0).$$
But then what can I do? If $h(0)=0$ the result follows. But if $h(0)>0$ I don't know how to proceed.
Comment: Also, if $x$ is negative, we don't have $0\leq \frac{f(x)}{x}\leq h(x)$. Rather $0\geq\frac{f(x)}x\geq h(x)$.
Answer: You are on the right track. It suffices to show that $h(0)=0$.
Note that $0\leq xh(x)$ for all $x\in\mathbb{R}$ implies that $h(x)\geq 0$ for $x\ge0$ and $h(x)\leq 0$ for $x\le0$. Hence, since $h$ is continuous, it follows that $h(0)=0$.


Python add up 2 unevenly sized 2d arrays at given index as center

Question: Given 2 arrays, for example:
main_arr = np.array([
    [0, 0, 0, 0, 0],
    [0, 1, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 1, 0],
    [0, 0, 0, 0, 0]
])

small_arr = np.array([
    [2, 2, 2],
    [2, 0, 2],
    [2, 2, 2]
])

result_arr = np.array([
    [2, 2, 2, 0, 0],
    [2, 1, 2, 0, 0],
    [2, 2, 4, 2, 2],
    [0, 0, 2, 1, 2],
    [0, 0, 2, 2, 2]
])

How to efficiently add up small_arr into the main_arr where center for merging is some known index (in this example number ones).Is there is any handy tool for this operation?
In my specific case, I have hundreds of thousands of indexes and need to perform this same operation. So ideally the solution should be looping process that runs through the set of indexes.
Comment: explain _Why_ the result looks what it looks like. Explain your "given index"
Comment: explain what to do if the "1" sits in a way so that not all overlayed numbers of _small_arr_  are inside the bigger one - f.e.:  `[1,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]`
Comment: what you want to do is a kind of 2d "kernel" operation on the values that are 1 - you can look / research that  f.e. https://medium.com/analytics-vidhya/2d-convolution-using-python-numpy-43442ff5f381 - it is f.e. used in image computation for edgefinding, smoothing, etc
Comment: The point of the question is, to simply add a small array into the big one at specific indexes. In the example I used 1s as a position, where the index would be extracted. The index then is used as a place where the small array is added. So where 1 is in the main_arr, the small_arr is added. This creating a combined grid. 1 doesnt change because values 0 is added onto it.
Comment: I think a simple 2D convolution should do what you want (see my update)
Comment: @XDOL  What happens if a 1 is on the first row of your bigger array - the smaller 3x3 if centered on top of the 1 would "hang over" by 1 row into nothingness ... an **edge-case** so to say ....
Comment: @PatrickArtner Yes an edge case is a problem indeed. However in my case I can fortunately ignore that, because in my case there can't be any values at the edges
Answer: 2D convolution
simple example 0/1 only
If you want to add the values where you have the 1s, use a 2D convolution:
from scipy.signal import convolve2d
out = convolve2d(main_arr, small_arr, mode='same')+main_arr

output:
array([[2, 2, 2, 0, 0],
       [2, 1, 2, 0, 0],
       [2, 2, 4, 2, 2],
       [0, 0, 2, 1, 2],
       [0, 0, 2, 2, 2]])

more complex example
main_arr  = np.array([
    [1, 0, 0, 9, 0],
    [0, 0, 0, 0, 0],
    [0, 9, 0, 0, 0],
    [0, 0, 0, 1, 0],
    [-1, 0, 0, 0, 0]
])

small_arr = np.array([
    [2, 2, 2],
    [2, 0, 2],
    [2, 2, 2]
])

from scipy.signal import convolve2d

convolve2d((main_arr==1).astype(bool), small_arr, mode='same')+main_arr

output:
array([[ 1,  2,  0,  9,  0],
       [ 2,  2,  0,  0,  0],
       [ 0,  9,  2,  2,  2],
       [ 0,  0,  2,  1,  2],
       [-1,  0,  2,  2,  2]])

older answer
Assuming you want to add from the positions 0,0 and 2,2, you could use:
main_arr  = np.array([
    [0, 0, 0, 0, 0],
    [0, 1, 0, 0, 0],
    [0, 0, 0, 0, 0],
    [0, 0, 0, 1, 0],
    [0, 0, 0, 0, 0]
])

small_arr = np.array([
    [2, 2, 2],
    [2, 0, 2],
    [2, 2, 2]
])

x,y = small_arr.shape
main_arr[0:x, 0:y] += small_arr    # actually main_arr[0:0+x, 0:0+y]
main_arr[2:2+x, 2:2+y] += small_arr

output:
array([[2, 2, 2, 0, 0],
       [2, 1, 2, 0, 0],
       [2, 2, 4, 2, 2],
       [0, 0, 2, 1, 2],
       [0, 0, 2, 2, 2]])

Comment: @NiziL yes, otherwise you can use `(main_arr==1).astype(bool)` to perform the convolution ;)
Comment: @mozway Yes thank you, that solution is simple and good :). This and example from Nizil are indeed the right choises. Thank you very much :)
Comment: On a side note for future reader, this solution will only work if `main_arr` is populated with 1 and 0, 1 being the place you want to add the kernel.
Comment: The 1s and 0s are only for demo purposes. The solution in reality needs to work with tuple of indexes. But your solution could be used if I converted all values on each index with 1
Comment: @XDOLK009 see the more complex example, would this work for you?
Answer: I'll assume small_arr is always square, with an odd dimension (so we have a center).
indexes = [(1,1), (3,3)]
offset = small_arr.shape[0] // 2
for x, y in indexes:
  main_arr[x-offset:x+offset+1, y-offset:y+offset+1] += small_arr

As pointed out in the comments, this still not solve case where your index is "on the border" of main_arr.
Comment: Border lines can be neglected in my case, but yes it's a valid point. That could be a problem in some other cases
Answer: Since you already know the indicies, you just have to go over every position that is a neighbor of that index. So you have to find the coordinates using the indices first, which will be a list of tuples. Assuming the small array always is a 3x3 matrix, you can do the following:
coordinates = [(x_1,y_1), ... , (x_n,y_n)]

for(co in coordinates) {

  x = co[0]
  y = co[1]
  result_arr[x][y + 1] = result_arr[x][y + 1] + small_arr[x][y + 1]
  result_arr[x][y - 1] = result_arr[x][y - 1] + small_arr[x][y - 1]
  result_arr[x + 1][y] = result_arr[x + 1][y] + small_arr[x + 1][y]
  result_arr[x + 1][y + 1] = result_arr[x + 1][y + 1] + small_arr[x + 1][y + 1]
  result_arr[x + 1][y - 1] = result_arr[x + 1][y - 1] + small_arr[x + 1][y - 1]
  result_arr[x - 1][y] = result_arr[x - 1][y] + small_arr[x - 1][y]
  result_arr[x - 1][y + 1] = result_arr[x - 1][y + 1] + small_arr[x - 1][y + 1]
  result_arr[x - 1][y - 1] = result_arr[x - 1][y - 1] + small_arr[x - 1][y - 1]

Comment: My "small_arr" isn't always 3x3, but is always predefined


Retrieve internet proxy server address via PowerShell

Question: I have to retrieve the proxy server address and port via PowerShell, so I can use the Invoke-Webrequest -uri http://some-site.com -Proxy command. The expected output should looks like http://proxy-server.com:port.
I there a PowerShell function to retrieve the proxy server address and port, so we can use it in a script ?
Comment: Bear in mind that sometimes the definition of a proxy can be in a wpad.dat file.
Answer: Here is the PowerShell function to achieve my goal :
function Get-InternetProxy
 {
    <#
            .SYNOPSIS
                Determine the internet proxy address
            .DESCRIPTION
                This function allows you to determine the the internet proxy address used by your computer
            .EXAMPLE
                Get-InternetProxy
            .Notes
                Author : Antoine DELRUE
                WebSite: http://obilan.be
    #>

    $proxies = (Get-ItemProperty -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings').proxyServer

    if ($proxies)
    {
        if ($proxies -ilike "*=*")
        {
            $proxies -replace "=","://" -split(';') | Select-Object -First 1
        }

        else
        {
            "http://" + $proxies
        }
    }
}

Hope this helps !
Comment: should there be a check for "ProxyEnable"?
Answer: [System.Net.WebProxy]::GetDefaultProxy() | select address
[System.Net.WebProxy] is an object and one of its static methods is GetDefaultProxy(). The select shows for us from all the columns, what interests us, and it is the address.
Comment: Thanks. But it looks a little bit cryptic, maybe a little bit of explanation could make it more useful for the readers.
Comment: `[System.Net.WebProxy]` it's an object and one of its static methods is `GetDefaultProxy()`, then from all the columns, what interest us is the `address`. So that line just solves the problem. Hope it's crystal clear now ;)
Comment: To me it was clear even without your comment. To the readers of the future, not surely. The site has a complex review process, and such single-line snippets mostly don't pass it. My advice tried to save your post (note, I've found it by reviewing the "low quality posts" queue).
Comment: Furthermore, such comments which are essentially extensions of an answer, should be edited into the answer and not into a comment below. I inserted your comment into your answer in the here required style, and voted it up because so it already deserves it. :-)
Comment: This does not always work. My workplace uses Z-scaler as a proxy and GetDefaultProxy() returns $null, although the actual proxy address can be seen in Settings > Network and Internet > Proxy.
Answer: Same host:
Get-ItemProperty -Path "Registry::HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings"

source: https://www.mowasay.com/2016/08/windows-check-proxy-settings-from-powershell/
Remote host(s):
Invoke-Command -ComputerName Member01, Server01 {Get-ItemProperty -Path 'HKLM:\SOFTWARE\VMware, Inc.\VMware Tools\' -Name InstallPath | select InstallPath}
source: http://vcloud-lab.com/entries/powershell/powershell-get-registry-value-data
Answer: You first get the IE settings. Then make use of proxy settings if they are active.
if ($IESettings.ProxyEnable -eq 1)
$IESettings = Get-ItemProperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings'
$Proxy = "http://$(($IESettings.ProxyServer.Split(';') | ? {$_ -match 'ttp='}) -replace '.*=')"
Invoke-WebRequest 'http://some-site.com' -ProxyUseDefaultCredentials -Proxy $Proxy

Comment: Actually, your script returns only HTTP proxy address in any case, even if you are trying to reach HTTPS page. So it does about the same as my solution. Also, your function can be missleading in case where differet proxy servers used for different types and there is no proxy for HTTP at all, since your function returns '-First 1'. I slighty changed my previous post to cover the case where there is several servers for different types.
Comment: Thanks for your participation, but this does not work if you configured different proxies addresses (HTTP, HTTPS, FTP). The function I posted as answer will work for both scenario : one proxy for all protocols, or different proxies.
Comment: This is definitely a better answer for simple use cases. Thanks :)
Comment: This worked for me where all other answers failed.


What does it mean if $\det(A)$ equals $1$?

Question: What does it mean if $\det(A)$ equals $1$? Does it mean that the identity matrix can be obtained from $A$ by only adding multiples of rows onto others?
Comment: @k.stm actually "adding multiples" should include subtracting rows as well. You know, adding negative multiples. that can be converted to identity
Comment: @Sabyasachi You still won't be able to fix the very first entry to be $1$ by adding or subtracting multiples of the other row on it.
Comment: @k.stm actually you can. let me demonstrate. i am editing it into my answer. hold on.
Comment: OP, clarify an argument. Are column manipulations allowed or not?
Comment: Err yeah allowed I guess. Will it mean something different, then? I thought row/col manipulations have the same effect on the determinant. No?
Comment: Sorry for misleading you, Nic. The others corrected me, as you probably have seen now.
Answer: Matrices with determinant $1$ preserve volume.  If all points inside a shape are transformed by the matrix to form a new shape, the proportional change in area (or volume) is the determinant of the matrix.  For example if the determinant of a matrix $A$ is $5$, then a unit cube will transform into a shape with volume $5 \times 1=5$. The identity matrix would transform a unit cube into a shape with volume $1 \times 1 = 1$.
Added
After i read the appreciated comment of @danielV, i added this image from Determinant and linear transformation to give more explanations.

It represents the transform of a unit square by a determinant 4 matrix.  Notice that the area of the new square is $1 \times 4 = 4$.
Comment: Reference for your comment: http://math.stackexchange.com/questions/71981/how-can-i-prove-that-a-matrix-is-area-preserving  .  This is so important, I wish it was said more clearly.  If all the points of a shape are translated through a matrix with determinant 1, then the resulting shape has the same area/volume.
Comment: @DanielV, i added some new to explain the answer.
Answer: Usually when we talk about the determinant without any other information attached, the only really relevant information is whether or not it is zero. However, if we are interested in geometry, there is some significance to matrices with determinant $1$. Namely, an important subset of them form the so-called special orthogonal group, which is just a fancy way to say they are like generalizations of rotations.
[note: As mentioned in the comments, I was a bit too eager. There are (many) matrices of determinant $1$ which are not special orthogonal. I do not know of any signficant properties that all matrices of determinant $1$ possess, except that they are volume-preserving, as noted by Semsem.]
For example, matrices of the form
 $$\begin{pmatrix}\sin\theta & \cos\theta \\ -\cos\theta & \sin\theta \end{pmatrix}$$
have determinant 1, and you can prove these correspond precisely to a rotation by $\theta$ (can't remember if it's clockwise or counterclockwise when $\sin$ is in the upper-left).
What properties of rotations do they possess? To answer this question, you need some familiarity with a matrix not just as a tool for putting together a system of equations -- which is how I was first taught them, unfortunately. You can think of them as really being functions that take in vectors and spit out other vectors, and these corrospond to geometric transformations of the vector space (so, the plane, or three-dimensional space, or beyond). I'm not sure what level you're coming at this from; I just mean to say that there are several ways of looking at matrices and one of them is a bit more abstract (so not taught in some classes) but is good for understanding why determinant-one matrices are like rotations.
First, they are orientation preserving which just means that a right-handed system never turns into a left-handed system. Or, no picture is mapped under the transformation to its mirror image (unless it was already symmetric). They are also distance-preserving in that points which were $d$ distance apart before being acted upon remain $d$ distance apart afterwards.

For what it's worth, Sabyasachi's construction is valid even if only row manipulations are allowed. However, see the comments for some subtleties about row-reduction; some minor definitions can change your ability to do or not do this.
$$A = \begin{pmatrix}-1 & 0 \\ 0 & -1\end{pmatrix}$$
$$R_2 \to R_2 - R_1$$
$$\begin{pmatrix}-1 & 0 \\ 1 & -1\end{pmatrix}$$
$$R_1 \to R_1 - R_2$$
$$\begin{pmatrix}0 & -1 \\ 1 & -1\end{pmatrix}$$
$$R_2 \to R_2 - R_1$$
$$\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix}$$
$$R_1 \to R_1 + R_2$$
$$\begin{pmatrix}1 & -1 \\ 1 & 0\end{pmatrix}$$
$$R_2 \to R_1 + R_2$$
$$\begin{pmatrix}1 & -1 \\ 2 & -1\end{pmatrix}$$
$$R_2 \to R_2 - 2R_1$$
$$\begin{pmatrix}1 & 0 \\ 0 & 1\end{pmatrix}= I_2$$
Comment: Not every $2\times2$-matrix with determinant $1$ is orthogonal.
Comment: Hm, that's a pretty serious problem :(
Answer: NOTE: My original answer was about a different question. (I have considered only adding integer multiples of some row.) I have completely changed it, hopefully the new answer is correct.

Other answers have already gave explanations about geometric meaning of $\det(A)=1$, so let me address the questions about row operations.
Let us start by checking what can be obtain from some given regular matrix $A$ using only one type of row operations. (Adding multiple of some row to another row.)
We can exchange rows, with changing signs of one of them
If we want to exchange rows $\vec\alpha_1$ and $\vec\alpha_2$ we can proceed like this:
$$\begin{pmatrix}\vec\alpha_1\\\vec\alpha_2\end{pmatrix}\sim
\begin{pmatrix}\vec\alpha_1\\\vec\alpha_1+\vec\alpha_2\end{pmatrix}\sim
\begin{pmatrix}-\vec\alpha_2\\\vec\alpha_1+\vec\alpha_2\end{pmatrix}\sim
\begin{pmatrix}-\vec\alpha_2\\\vec\alpha_1\end{pmatrix}.$$
We can get from $A$ a diagonal matrix.
We proceed similarly as with Gaussian elimination, only we are not allowed to multiply rows, so we do not get $1$'s on diagonal, instead we obtain some non-zero elements.
We inductively repeat these two steps:

If in the $k$-th step the element $b_{kk}$ is non-zero we use it to get zeroes in all other rows in the $k$-th column.
If $b_{kk}=0$, at least one of elements $b_{k,k+1},\dots,b_{k,n}$ must be non-zero. (Otherwise we would have $\det(A)=0$.) We exchange rows in a such way that we get non-zero number on this position. Then we can proceed as before.

From the diagonal matrix, we can get identity matrix $I$.
We inductively repeat steps like these:
$$\begin{pmatrix}d_1&0\\0&d_2\end{pmatrix}\sim
\begin{pmatrix}d_1&0\\1&d_2\end{pmatrix}\sim
\begin{pmatrix}0&-d_1d_2\\1&d_2\end{pmatrix}\sim
\begin{pmatrix}0&-d_1d_2\\1&0\end{pmatrix}\sim
\begin{pmatrix}1&-d_1d_2\\1&0\end{pmatrix}\sim
\begin{pmatrix}1&0\\0&d_1d_2\end{pmatrix}$$
By repeating such step several times we can get to
$$\begin{pmatrix}
1 & 0 & \dots & 0 & 0 \\
0 & 1 & \dots & 0 & 0 \\
\ldots & \ldots & \ddots & \ldots & \ldots \\
0 & 0 & \dots & 1 & 0 \\
0 & 0 & \dots & 0 & d_1d_2\dots d_n
\end{pmatrix}=
\begin{pmatrix}
1 & 0 & \dots & 0 & 0 \\
0 & 1 & \dots & 0 & 0 \\
\ldots & \ldots & \ddots & \ldots & \ldots \\
0 & 0 & \dots & 1 & 0 \\
0 & 0 & \dots & 0 & 1
\end{pmatrix}=I_n
$$
Comment: Seems right. I thought so much, but hesitated since Wikipedia [claims that the group generated by transvections need not be $SL(n)$](https://en.wikipedia.org/wiki/Special_linear_group#Relations_to_other_subgroups_of_GL.28n.2CA.29). I guess it's another case of WP being wrong. (I've posted a question on its talk page about this, but I guess one may correct even if it gets not answer.)
Comment: @MarcvanLeeuwen I see in the article you linked: *The special linear group over a field or a Euclidean domain is generated by transvections.* And later: *If working over a ring where SL is generated by transvections (such as a field or Euclidean domain), one can give a presentation of SL using transvections with some relations.* What I did was for matrices over a field. (However, I have not encountered SL(n) in different context than a field before.)
Comment: I see, you refer to this sentence: *These are both subgroups of SL ... but in general do not coincide with it.*
Answer: Saying $\det(A) = 1$,means A is the determinant is
$$
\begin{matrix}
a_{11}&a_{12}&\cdots &a_{1n}\\
a_{21}&a_{22}&\cdots &a_{2n}\\
\vdots&\vdots&\ddots &\vdots\\
a_{n1}&a_{n2}&\cdots &a_{nn}\\
\end{matrix}
$$ and the determinant evalutes to one. Since $\det(A) = \det(I)$, $A = I_n$ where $I_n$ is the identity matrix of n rows.
Therefore, by row manipulation should in principle be able to yield the identity matrix, but it is hard to say how complicated the manipulations have to be.

In response to k.stm's question about one particular matrix in a comment below OP's question.
$$A = \begin{matrix}-1 & 0 \\ 0 & -1\end{matrix}$$
$$C_1 \to C_1 - C_2$$
$$A = \begin{pmatrix}-1 & 0 \\ 1 & -1\end{pmatrix}$$
$$R_1 \to R_1 - R_2$$
$$A = \begin{pmatrix}0 & -1 \\ 1 & -1\end{pmatrix}$$
$$C_1 \to C_1 - C_2$$
$$A = \begin{pmatrix}1 & -1 \\ 2 & -1\end{pmatrix}$$
$$R_2 \to R_2 - 2R_1$$ and $$R_1 \to R_1 + R_2$$
$$A = \begin{pmatrix}1 & 0 \\ 0 & 1\end{pmatrix}$$
and voila!
Comment: No, it is very much not arbitrary. Columns and rows play extremely different roles in the linear algebraic manipulation of matrices. The columns span the image of the transformation (when viewed as a left-action), whereas the rows span the image of the adjoint, which may in general may exist in an entirely different vector space. But to the specific example: if you're allowing differences, I don't see why you wouldn't just multiply both rows by -1. In fact, I'm not sure that any manipulation that works wouldn't end up being equivalent to that.
Comment: @tomasz: Oh, boy. I had a major fallacy there – I was somehow assuming that all row manipulations would yield triangular matrices. Thank you.
Comment: Sorry, but this makes no sense: $A = I_n$? What about the example of $\begin{bmatrix}-1 & 0 \\ 0 & -1\end{bmatrix}$ I have given above? You know $\det (a_{i,j}) = \sum_{i,j} \prod_{σ ∈ S_n} a_{i,σ(j)}$, right?
Comment: @k.stm I mean it can made into Identity. At least I think so. I am trying to manipulate.
Comment: @k.stm there you go. your counter example is invalid. I created the identity matrix simply by addition of multiples(as long as you allow negative multiples of course, but that seems like a reasonable requirement)
Comment: having a huge obsession with Rubik's cubes certainly helps. :D I got habituated to changing corners by shifting rows and columns successively.
Comment: Nic was asking about row manipulations only.
Comment: @k.stm oh come on. Columns are *kind* of rows. Now that is just arbitrary.
Comment: @EricStucky The normal way to do it is by multiplying by -1 of course, but the OP asks if it *can* be done. As I showed, it can. And I am not an expert here, but I am pretty sure when matrices are being used for mapping transformations, there determinants are not calculated. Determinants are entirely different things.
Comment: Entirely different is a bit of an exaggeration. To wit, determinants are more naturally defined on transformations than matrices because they are an invariant under conjugation ($A=PBP^{-1}$ implies $|A|=|B|$). But I think the bigger concern is that if I were reading your answer as first-year Lin Al student, I would think that you are implying that any matrix that can be row-reduced to the identity has determinant 1. Of course that's not what you're saying, but it wasn't tremendously clear to me on first reading.
Comment: @k.stm: For only row transformations, do $R_2-=R_1$, $R_1+=2R_2$, $R_2-=R_1$ and finally $R_1+=2R_1$.
Comment: @EricStucky: But that's true. It's actually elementary and follows easily from the determinant being alternative and linear in each row.
Comment: @tomasz: I thought any invertible matrix could be row-reduced to the identity? Maybe my memory isn't as good as I thought it was :/ (Also, beat me to the punch on the row transformations…)
Comment: @EricStucky: Only if you allow multiplying a single row by a nonzero scalar. Just adding a multiple of one row to another row you won't change determinant. I guess it depends on what you mean by row reduction. ;)
Comment: @k.stm: You're welcome. :)
Comment: Thanks for all the answers!


Best Way to "Learn" Jazz Piano

Question: Let me preface the question with a bit of background because I'm not even sure if I classify as "beginner" or "intermediate" or anything like that.
I took your run of the mill piano lessons for 8 years. Started at 3 Blind Mice and ended with attempting to learn the full Rhapsody in Blue before junior year of high school happened and I dropped lessons because of time.
Now it's four years later, and while I've been playing all the time, learning songs here and there, and writing a few short songs of my own. I really got into Jazz last year, mostly listen to Herbie Hancock and Ramsey Lewis, and I figured learning a few of their sons wouldn't be too hard. This was my entrance into learning how much different Jazz is than regular piano. I "learned" Canteloupe Island and The In Crowd, but soon I realized all I knew how to do was play the main chords and the basic structure of the song. I wanted to be able to do the fancy solos that were in the recordings, and soon I was on a google hunt trying to learn improv.
I learned the blues scale, and I immediately thought I was the best soloist around. It sounded good and I figured it could be applied to every song all the time. And that's about where I am now. I can play several Hancock songs, I know one or two standards, but I want more and seem to have hit a place where everything seems to be hard to understand and most videos assume you have knowledge of all the things like every scale and every chord.
I'm a huge fan of fast Jazz, like the big band type stuff. I also like the piano bar style of it, and was very impressed when I watched a jazz pianist going to town one night at a club.
So here's the question: where do I go next? Learning songs doesn't seem to really help me. I know you can't exactly "learn Jazz" but I'm doing something wrong here. I'll sit down, play the songs I know, try and make up a few solos over them, play with some chords, get nowhere, and then burn out because I'm no longer having fun. It's not that healthy to my enjoyment of playing.
I'd love to know the best route to take for someone who doesn't have two hours a day to sit down and practice scales and chords over and over again.
Answer: You mentioned "videos assume you have knowledge of ... every scale and every chord" - well, the best way to attempt to tackle a lot of Jazz is to understand the theory very well.
Jazz improvisation may look spur of the moment, but it is all based on a strong understanding of what notes, scales, modes and chords will work at any point. Without that you will remain stuck in the same rut.
Learning the theory doesn't however, have to mean practicing scales over and over again. It means understanding the components of the various scales and how they can be used.
We have a lot of questions here on specific parts of theory, on modes and scales, on improvisation, and loads tagged with jazz. Additionally, there are countless videos online displaying all of these.
Comment: Actually - the one common thread you will see across Music.SE is **Find yourself a good teacher** - this works when you are a beginner, but also whenever you are stuck or need a new direction.
Comment: Very helpful answer. I think the problem for me is not finding the information and videos but being able to actually use it in practice. Sure, you can learn every scale of every key, and you can learn a few dozen jazz variations of chords, but again how do you apply that knowledge? My biggest issue here is probably the lack of a teacher or path. I don't have a defined "curriculum" to stick to, and no real way to apply all the chords and scales that I learn. That's more of what I was looking for in this question. How can I learn a jazz song the right way, ie without sheet music.
Comment: @Patrick, the best way to gain that knowledge of how to apply the theory is to transcribe jazz piano solos from your favorite recordings. If you're not sure where to start, begin with Wynton Kelly. After transcribing a few of his solos, you'll find that you can start accurately predicting where his lick will go just after hearing the first few notes. When you reach this point, you know you're absorbing not just the scales/rules, but the *ideas* that he uses to apply the theory to actual musical settings.
Comment: Step out of your comfort zone and survey across the field and even outside of…I'd look at jazz that springs from gospel, rock, old New Orleans and Chicago styles and musicians. If you do too much at once, it'll just be confusing, but if you let it build, you should both increase your mental library and discover more affinities — eventually leading to a great style of your own. It just doesn't all happen at once, and some frustration is a required part of the process.
Answer: First of all, yes you can learn Jazz.
Where do I go next?
Ok, so you have a good repertoire of songs that you have learned over the course of playing the piano, you know what scales are, you know what chords are, and you know how to read sheet music. Great!
Now, if you want to get anywhere advanced with playing Jazz (or any genre in general) you need to learn music theory. There's no way around it. At the start it's difficult to learn on your own, but you'll get better at it so long as you stick with it. Remember, you want to get better at playing the piano, right?
So, where do you start? I highly recommend you start with intervals. What are they? Why are they important? What do they sound like? What do they look like on the staff?
After that learn how to construct and read big chords. It is extremely important to know what you're playing and be able to communicate it if you're working with other musicians. Here are some examples: F7b9#5, AbM7#11, C+M7add13, D°7, GmM7 etc. This might seem a bit complex but if you want to understand it the only thing hindering you is yourself.
At this point you may want to dive into the world of scales and modes. If you're getting bored of everything you're playing and want something to spice up your improvisation, this is a great place to start. I suggest you start with the Greek scales and move on from there.
Comprehending only these three areas of music theory will dramatically increase your skills as a Jazz pianist.
There are so many Jazz topics that it would take me forever to list them all here. I don't believe there really is a best route to take in learning music - though some might disagree. So learn whatever interests you and eventually you will run into a new topic. It's all interrelated.
If you get stuck on a certain concept, move on! Learning should be fun and not stressful.
Also remember that music is more of an art than it is a science. If your ear is telling you to do something else from what your mind is telling you, do it! The rules are just a "standard" guideline.
Here are some links that helped me in learning Jazz and music theory.
http://www.musictheory.net/lessons
https://www.youtube.com/playlist?list=PLB585CE43B02669C3
http://tobyrush.com/theorypages/index.html
http://www.hooktheory.com/
https://en.wikipedia.org/wiki/Mode_(music)

Good luck!
Answer: I think the simple answer is to find a good jazz piano player to teach you. That may be hard to find depending on your location but, although jazz requires knowledge of the theory that others have commented on, it's also very much an oral tradition that's difficult to pick up without a teacher to put you on the right footing.  There are books like The Jazz Piano Book that contain lots of fantastic info on theory but the I doubt they're a replacement for a good teacher.
Comment: There are also some good jazz piano teachers on YouTube, like Aimee Nolte for bebop and Bill Hilton for cocktail and blues jazz. Again, it's no substitute for having a teacher in the room, but you can learn a lot about voicing and improvisation from them.
Answer: Find a recorded solo that you like.  Listen to it repeatedly, until you can sing it.  Then work it out on the piano.  Repeat.
The purpose of this excercise is: 1. Teach your ears and your fingers some jazz vocabulary; 2. Learn to translate what your hear into notes that you can play.  That ear training will help you pick up more ideas from what you hear, and also make it easier to play the ideas that you hear in your head.
Learn some common jazz chord voicings well enough that you can sight read a lead sheet.
Find some people to play with.  When they're not available you can play along with recordings or software (like band-in-a-box or iReal Pro).
Also, find a teacher.
Disclaimer: I'm a fourth-rate beginning jazz pianist myself (and have been for years).
Answer: If you are capable of/enjoy self-study (I'm not very good at it), check out Jamey Aebersold's jazz "method" series that begins with How to Play Jazz and Improvise.
There are a LOT of books there; most revolve around learning songs in a particular genre or from a particular artist, but several are more theory/exercise-oriented.
Answer: A very wise jazz musician once told me "play what you feel" when I asked about how he knew how to improvise. I then asked something along the lines "how am I meant to show my feelings through playing?" and I was given the answer "listen, then listen more". I was no more than 9, and it made no sense, but I now think (at almost 40) that this is the key to it all.
Much later I asked a great friend and fabulous guitarist (Kit Morgan) some questions about what he was playing and was told frankly "I have no idea; I just went with it - was it awful?" and then he confessed he'd have no idea how to play the same thing again (but rest assured he'd have played something just as great).
The point to all of this (there probably isn't one) but I've learnt that a good technical understanding of the instrument mechanics is very important such that one doesn't think about the instrument (similar to operating a proper camera). Secondly, a good understanding of music theory and the idioms of the style being played, as well as sympathy and empathy to the creators of the style (ever heard a purist hard bebop player in a dixie band? usually it doesn't work for that reason).
How does one get to this level? By all accounts listen, listen, and listen again, then practice, practice and more practice. Then record yourself and listen, listen, and listen again to your playing no matter how bad it sounds (if you think it sounds great you should take advice from a psychologist - no credible musician has ever felt that way).
Can I do any of those things well? No, but every time I play I try and play what I'm feeling before anything else as I feel that to do anything else is dishonest.
Answer: I heard Andre Previn having a discussion with Oscar Peterson and they spoke of playing along with the recordings of Art Tatum. Andre studied Tea for Two  for 3 years learning it measure by measure through listening and repetition.  Licks and their fingering  can be practised and repeated, inverted and transposed.
Greatness can be attained: practice, practice, practice.  Mozart never had to study a piece with the discipline and rigidity of his teacher...but Mozart was a Genius.  Those who possess great talent can bypass rudiments, but if KNOWLEDGE is what you want to attain, go to the Greats, the Masters, and sit at their feet, so to speak; refine your ear; break down the masterpieces, bar by bar, lick by lick an hour a day  or an hour a week. Self-determine how proficient you'll become.
Can you replicate what you hear? That is the "dialogue" of your instrument(piano).  As you increase in proficiency, you'll have more to "voice", There are still only 88 keys on the piano...so it's your IMAGINATION that holds your Greatness, your unique melodic-speak. Find what YOU like and "wash, rinse, repeat", so to speak.  ENJOY.

Best Way to "Learn" Jazz Piano


Jwt filter accessed even on permitted urls

Question: I'm working on a spring boot application where I defined my filter to be executed to manage getting and validating the token. So in my Web security class configuration, I managed to do this:
@Override
    protected void configure(HttpSecurity httpSecurity) throws Exception {
        httpSecurity
                // we don't need CSRF because our token is invulnerable
                .csrf().disable()

                // don't create session
                .sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS).and()

                .authorizeRequests()
                // .antMatchers(HttpMethod.OPTIONS, "/**").permitAll()

                // allow anonymous resource requests
                .antMatchers(HttpMethod.GET, "/", "/*.html", "/favicon.ico", "/**/*.html", "/**/*.css", "/**/*.js").permitAll()
                .antMatchers("/auth/**").permitAll()
                .antMatchers("/places/public").permitAll()

                .anyRequest().authenticated();

        // Custom JWT based security filter
        httpSecurity.addFilterBefore(authenticationTokenFilterBean(), UsernamePasswordAuthenticationFilter.class);

        // disable page caching
        httpSecurity.headers().cacheControl();
    }

The authenticationTokenFilterBean is a method annotated with @Bean that returns an instance of my filter:
public class JwtFilter extends OncePerRequestFilter{

    @Autowired
    private UserDetailsService userDetailsService;

    @Autowired
    private JwtService jwtService;

    @Value("${jwt.header}")
    private String authorizationHeader;

    @Override
    protected void doFilterInternal(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, FilterChain filterChain) throws ServletException, IOException {

        String token = httpServletRequest.getHeader(authorizationHeader);

        String username = jwtService.getUsernameFromToken(token);

        if(username!=null && SecurityContextHolder.getContext().getAuthentication() ==null){
            UserDetails userDetails = userDetailsService.loadUserByUsername(username);

            if(jwtService.isTokenValid(token, userDetails)){
                UsernamePasswordAuthenticationToken authenticationToken = new UsernamePasswordAuthenticationToken(userDetails, null, userDetails.getAuthorities());
                authenticationToken.setDetails(new WebAuthenticationDetailsSource().buildDetails(httpServletRequest));
                SecurityContextHolder.getContext().setAuthentication(authenticationToken);
            }
        }

        filterChain.doFilter(httpServletRequest, httpServletResponse);
    }

}

Since I'm extending OncePerRequestFilter, this filter is invoked only one time per request which is not the case with GenericFilter that need to be executed one time by the servlet and another time with Spring security.
The problem that I have is that the ant matcher described in my configuration class are also intercepted by the filter even if I permit them with the permitAll() method, I tried to override configure(WebSecurity web) method from WebSecurityConfigurerAdapter and ignore them but still didn't work.
How can I configure Spring security to skip my filter for these requests? I already checked this question Spring Security JWT Filter applies on all requests but it has no solution.
Thank you
Comment: You can always build another/second configuration without your filter for some URLs.
Answer: Still handled inside the filter, but cleaner approach.
You can override shouldNotFilter(HttpServletRequest request) method of OncePerRequestFilter and based on URL patterns return true to skip filtering.
private static List<String> skipFilterUrls = Arrays.asList("/", "/*.html", "/favicon.ico", "/**/*.html", "/**/*.css", "/**/*.js");

@Override
protected boolean shouldNotFilter(HttpServletRequest request) throws ServletException {

  return skipFilterUrls.stream().anyMatch(url -> new AntPathRequestMatcher(url).matches(request));
}

Comment: A very effective way to bypass the filter without changing the security fabric. Thanks !
Answer: Change your WebSecurityConfiguration class, it'll look like:
@Override
protected void configure(HttpSecurity httpSecurity) throws Exception {
    httpSecurity
            // we don't need CSRF because our token is invulnerable
            .csrf().disable()

            // don't create session
            .sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS).and()

            .authorizeRequests()

            .anyRequest().authenticated();

    // Custom JWT based security filter
    httpSecurity.addFilterBefore(authenticationTokenFilterBean(), UsernamePasswordAuthenticationFilter.class);

    // disable page caching
    httpSecurity.headers().cacheControl();
}

 @Override
public void configure(WebSecurity web) throws Exception {
    // Ignore spring security in these paths
    web.ignoring().antMatchers("/", "/*.html", "/favicon.ico", "/**/*.html", "/**/*.css", "/**/*.js","/auth/**","/places/public");
}

You have to remove the permitAll matchers from HttpSecurity configure method and put it in WebSecurity configure method.
Comment: could you help to explain why it make difference?
Answer: I avoid this problem in this way, you can take for reference:
@Override
protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res, FilterChain filterChain) throws ServletException, IOException {
    if("/login".equals(req.getServletPath()) && "POST".equalsIgnoreCase(req.getMethod())){
        // ......
    }
    filterChain.doFilter(req, res);
}

Comment: This is handled inside the filter which is what I want to avoid. Thank you anyway


The simulated $((1-\alpha )\times 100)^{th}$ percentiles of the null distribution of a specific statistic

Question: I've written this algorithm in R program:

as following
n=20
mu=100
sigma=10
N=10000
alpha=0.05
D=array(0, dim=N)
for(j in 1:N){
    x=rnorm(n,mu,sigma)
    x=sort(x)
    z=dnorm(x, mean(x), sd(x))  # R can do it at once
    num = sum( abs(z - seq(1,n)/n ) )
    den = sum( pmax(z, seq(1,n)/n) )
    D[j]=num/den
}
sort(D)[N*alpha]

The output is $0.4690684$. But it must be about $0.1232$. Is there something wrong in my code?
Thanks in advance.
Comment: I'm sorry to say, this is not a coding site.   As it happens, though, you have two readily-identifiable problems.   First, you are calculating the $\alpha$ percentile in your last line, not the $(1-\alpha)$ percentile.   Second, you are using `dnorm` instead of `pnorm` to calculate the $z_i$.
Comment: This is a new question, so it's better that you ask it as one.
Comment: @jbowman You are right. Thank you very much for your comments.
Comment: Dear @panda , thank you very much for your answer. Could you please tell me how I can compute the power of the test below in R?
Comment: @jbowman I am very sorry to bother you again. I was wondering if you could tell me how I can compute the power of the proposed test in R?
Comment: @jbowman I asked it as a new question here https://codereview.stackexchange.com/q/266095/247184. Thank you very much for your help.
Answer: @jbowman is right.
delta <- vector()
for (i in 1:10000){
x <- rnorm(20, 100, 10)
x <- sort(x)
z <- pnorm(x, mean(x), sd(x))
delta[i] <- sum(abs(z-seq(1,20)/20))/sum(pmax(z, seq(1,20)/20))
}
plot(density(delta))
quantile(delta, 1-0.05)

Comment: Thank you so much for your answer. It works properly.
Comment: I am very sorry to bother you again. I was wondering if you could tell me how I can compute the power of the proposed test in R Here https://stats.stackexchange.com/q/540480/242174 .


What is the relationship between the mean squared error and the residual sum of squares function?

Question: Looking at the Wikipedia definitions of:

Mean Squared Error (MSE)
Residual Sum of Squares (RSS)

It looks to me that
$$\text{MSE} = \frac{1}{N} \text{RSS} = \frac{1}{N} \sum (f_i -y_i)^2$$
where $N$ is he number of samples and $f_i$ is our estimation of $y_i$.
However, none of the Wikipedia articles mention this relationship. Why? Am I missing something?
Comment: @Bill: Well, it is exactly the kind of relationship that typically leads to articles being linked on Wikipedia. Your point regarding the degree of freedoms also shows that is not quite as obvious and definitely something worth mentioning.
Comment: I know this seems unhelpful and kind of hostile, but they don't mention it because it is obvious.  Also, you want to be a little careful, here.  Usually, when you encounter a MSE in actual empirical work it is not $RSS$ divided by $N$ but $RSS$ divided by $N-K$ where $K$ is the number (including the intercept) of right-hand-side variables in some regression model.
Comment: @Bill: Agree, however obviousness is very subjective. The statistics/machine learning grey area is littered with notation hell and therefore it is good to be explicit.
Answer: Actually it's mentioned in the Regression section of Mean squared error in Wikipedia:

In regression analysis, the term mean squared error is sometimes used
  to refer to the unbiased estimate of error variance: the residual sum
  of squares divided by the number of degrees of freedom.

You can also find some informations here: Errors and residuals in statistics
It says the expression mean squared error may have different meanings in different cases, which is tricky sometimes.
Answer: But be aware that Sum of Squared Errors (SSE) and Residue Sum of Squares (RSS) sometimes are used interchangeably, thus confusing the readers. For instance, check this URL out.
Strictly speaking from statistic point of views, Errors and Residues are completely different concepts. Errors mainly refer to difference between actual observed sample values and your predicted values, and used mostly in the statistic metrics like Root Means Squared Errors (RMSE) and Mean Absolute Errors (MAE). In contrast, residues refer exclusively to the differences between dependent variables and estimations from linear regression.
Answer: I don´t think this is correct here if we consider MSE to be the sqaure of RMSE. For instance, you have a series of sampled data on predictions and observations, now you try to do a linear regresion: Observation (O)= a + b X Prediction (P). In this case, the MSE is the sum of squared difference between O and P and divided by sample size N.
But if you want to measure how linear regression performs, you need to calculate Mean Squared Residue (MSR). In the same case, it would be firstly calculating Residual Sum of Squares (RSS) that corresponds to sum of squared differences between actual observation values and predicted observations derived from the linear regression.Then, it is followed for RSS divided by N-2 to get MSR.
Simply put, in the example, MSE can not be estimated using RSS/N since RSS component is no longer the same for the component used to calculate MSE.
Comment: I don't understand this answer.
Comment: Look, based on the mentioned example of sampled prediction and observed data values, the linear regression is established: Observation (O)= a + b X Prediction (P) (a, b are intercept and slope respectively).
In this case, MSE = Σ(O-P)^2/n, where Σ(O-P)^2 is the Sum of Squared Erros (SSE) and n is the sample size. However, Mean Squared Residues (MSR) = Σ(O-O´)^2/n-2, where Σ(O-O´)^2 equals to Residue Sum of Squares (RSS) and O`=a+b X P. MSR and RSS are mainly used for testing overall significance of linear regression.

Also note, SSE = Systematic Erros (SE) + RSS, where SE= Σ(P-O´)^2


An equivalent condition for Lebesgue measurability

Question: I'm trying to solve following problem:

Let $A \subseteq \mathbb{R}$. Suppose that for any $\epsilon > 0$, there are Lebesgue measurable sets $B, C$ so that $B \subseteq A \subseteq C$ and $m(C \cap B^c) < \epsilon$. Show that $A$ is Lebesgue measurable.

Here $m$ is the Lebesgue measure on $\mathbb{R}$. A set $X \subseteq \mathbb{R}$ is Lebesgue measurable if $$m^*(E) = m^*(E \cap X) + m^*(E \cap X^c)$$ for every $E \subseteq \mathbb{R}$, where $m^*$ is the outer measure given by $$m^*(E) =  \inf\{\space \sum_{n=1}^{\infty} \text{length}(I_n) : (I_n)_{n \in \mathbb{N}} \space \text{a cover of}\space E \space\text{by open intervals}\}.$$
I can show that $m^*(E) \leq  m^*(E \cap A) + m^*(E \cap A^c)$, by sub-additivity. Can anyone help me with the other direction?
Answer: Use the completeness of the Lebesgue measure.
Let $\epsilon = \frac{1}{n}$, and create a sequence of measureable sets $B_n \subset A \subset C_n$ such that $m(C_n\setminus B_n) < \frac{1}{n}$. The sets $B = \cup_n B_n$ and $C = \cap_n C_n$ are measurable, and $B\subset A \subset C$, with $m(C\setminus B) = 0$.
We have $A = (A\cap B) \cup (A \setminus B) = B \cup (A \setminus B)$. However, $(A \setminus B) \subset (C \setminus B)$, and any subset of a set of measure zero is Lebesgue measurable, hence $(A \setminus B)$ is measurable and has measure zero. Hence $A =  B \cup (A \setminus B)$ is measurable and $mA = mB$.
Note: To show that any subset of a set of measure zero is measurable, suppose $Z \subset N$, where $mN = 0$. By inclusion, we have $m^* Z = 0$. Now let $E$ be an arbitrary set, then since $N$ is measurable, we have $m^* E = m^* (N \cap E) + m^* (E\setminus N) = m^* (E\setminus N)$.
Since $Z \subset N$, we have $(E \setminus N) \subset (E \setminus Z) \subset E$ and so we have $m^* (E \setminus Z) = m^* E$. Since $m^* (Z \cap E) \le m^* Z = 0$, we have $m^* E = m^* (Z \cap E) + m^* (E\setminus Z)$, hence $Z$ is measurable. It follows that $mZ = m^* Z = 0$.
Comment: Thanks! To show that m(C\B) = 0, I used that C\B = intersection of all the (C_n \ B_n), so for any N, 0 =< m(C\B) =< m(C_N \ B_N) -> 0 as N goes to infinity. Is this OK?
Comment: Yes, you have $m(C\setminus B) <\frac{1}{n}$ for all $n$, hence it is zero.
Comment: Brilliant. Thank you again.
Comment: You are very welcome!


Rational Number Form

Question: I was reading Rational Points on Elliptic Curves by Silverman and Tate and they state:
Every non-zero rational number may be uniquely written in the form $\frac{m}{n}p^v$, where $m,n$ are integers prime to $p,n>0$ and the fraction $m/n$ is in lowest terms. Then
$$
\text{ord}\left(\frac{m}{n} p^v \right)=v
$$
However, it offers no example of this and I can't see how such a representation is unique. For example, if the integer was $13$, then $m=n=1$ and $p=13,v=1$. However, if the integer was $10$, we have
$$
\frac{2}{1}5^1
$$
as a possible representation and we also have
$$
\frac{5}{1}2^1
$$
The problem is even worse with non-integer rationals
$$
\frac{15}{7}=\frac{3}{7}5^1=\frac{5}{7}3^1=\frac{15}{1}7^{-1}
$$
as there is also no statement on $v$. Is it intended that $v$ be negative? Even if that is the case, then we still have
$$
\frac{1}{10}=\frac{1}{2}5^{-1}=\frac{1}{5}2^{-1}
$$
and the representation is not unique. I checked the errata and it makes no mention of error here. So what have I misinterpreted?
Comment: Here the prime $p$ is *fixed*, not variable.
Comment: @BillDubuque Of course! It would be something horribly obvious! I was fixated on the fact that I was assuming they intended $v>0$ when they began to talk about 'order'. Thank yo so much for clearing this up! I stared at this for some time wondering what they could mean. You should copy any of your comments as an answer! You certainly deserve credit for clearing this up so quickly and efficiently!
Comment: @BillDubuque But then how would that work? If I took $p=5$ then wanted to represent $13$, I would have $\frac{m}{n}\cdot 5^v$ and there's no combination that works that keeps $m,n$ integers prime to $p$. At least not that I'm seeing.
Comment: @BillDubuque I'm sorry but I still don't see that fixes the example I gave in my previous comment, $n \in \mathbb{Z}$ or $n \in \mathbb{Z}_+$ still doesn't allow a combination with $p=5$. Again, unless I'm missing something.
Comment: Sure there is: $\, 13\,=\,\frac{13}1\cdot 5^{0}\ \ $
Comment: It's page 49, and they say "so we let $p$ be some prime..." meaning some *fixed* prime. Is everything clear now?
Comment: Don't sweat it - that's a common type of oversight that probably everybody makes at one time or another.
Answer: Here the prime $\,p\,$ is fixed, not variable. Indeed they write on p. 49 "so we let $p$ be some prime...".
If $\,p\nmid a,b\,$ then said unique rep of $\ t =\dfrac{a}b\ $ is $\ \dfrac{a}b\cdot p^{\large \color{#c00}0},\,$ so $\ v_p(t) = \color{#c00}0.$

Rational Number Form


TSQL- How to count number of instances

Question: I'm using SQL Server 2014 and I need to know which set of students have an action code used only once, and which students have an action code used more than once.

I'm trying to see if a student has more than one of 5 action codes.
e.g.
SELECT StudentID
FROM #T
WHERE ActionCode IN (51201,51206,51207,51208,51209)

How would I code this please?
Thank you for your help.
Comment: you try anything? like `Count()`?
Comment: Yes, I tried Count, but I don't know how to use it the way I need to.  I have 5 Action codes I'm trying to see if a student has more than one of.  I simplified the answer above hoping I could modify it to do what I need.  I have the 5 response codes in a WHERE IN (...)
Comment: SELECT StudentID
  FROM #T
  WHERE ActionCode IN (51201,51206,51207,51208,51209)
Comment: Thanks @DhruvJoshi, I was working through SQL Fiddle to better understand what you did.  It's a good answer and helps, thank you.
Answer: SELECT
   StudentID,
   ActionCode,
   ActionCodeCount = Count(*)
FROM
   dbo.StudentActionCodes
WHERE
   ActionCode IN (51201, 51206, 51207, 51208, 51209)
GROUP BY
   StudentID,
   ActionCode
ORDER BY
   StudentID DESC,
   ActionCode
;

See this working in a live demo at SqlFiddle
Given this set of values (including some unwanted ActionCodes):
StudentID   ActionCode
----------- -----------
987654      51201
987654      51206
987654      51207
987654      51208
987654      51209
987654      51210
987653      51201
987653      51208
987653      51208
987653      51211
987652      51201
987652      51206
987652      51206
987652      51207
987652      51208
987652      51209
987652      51212
987652      51213
987651      51201
987651      51206
987651      51209
987651      51209
987651      51214
987651      51215
987650      51201
987650      51201
987650      51201
987650      51201
987650      51208
987650      51208
987650      51216

Here are the results:
StudentID   ActionCode  ActionCodeCount
----------- ----------- ---------------
987654      51201       1
987654      51206       1
987654      51207       1
987654      51208       1
987654      51209       1
987653      51201       1
987653      51208       2
987652      51201       1
987652      51206       2
987652      51207       1
987652      51208       1
987652      51209       1
987651      51201       1
987651      51206       1
987651      51209       2
987650      51201       4
987650      51208       2

You might also enjoy this query:
SELECT
   *
FROM
   dbo.StudentActionCodes S
   PIVOT (
      Count(S.ActionCode) FOR S.ActionCode IN ([51201], [51206], [51207], [51208], [51209])
   ) P
;

Which gives this result:
StudentID   51201       51206       51207       51208       51209
----------- ----------- ----------- ----------- ----------- -----------
987650      4           0           0           2           0
987651      1           1           0           0           2
987652      1           2           1           1           1
987653      1           0           0           2           0
987654      1           1           1           1           1

Comment: Thanks @ErikE, this worked!  I'm assuming the Count(*) is looking to the WHERE clause.  If I had two aggregations and I used the Count(*), how would it know which to count?
Comment: What do you mean by "two aggregations"? The `Count(*)` simply counts the number of rows that were "collapsed" as defined by the `GROUP BY` columns that had the same set of values.
Comment: This makes sense.  Thank you.
Answer: Please try the below query:
SQL fiddle link: http://sqlfiddle.com/#!6/d1dc6/6
It gives sum of same type of actions per student as well as sum of all allowed actions per student
SELECT DISTINCT
    StudentID,
    Actioncode,
    COUNT(1) OVER (PARTITION BY StudentID,Actioncode  ) as [Count_of_this_action_instance],
    COUNT(1) OVER (PARTITION BY StudentID  ) as [Count_of_all_action_instance] FROM StudentActionCodes
WHERE ActionCode IN (51201,51206,51207,51208,51209)

Comment: Using my SQL Fiddle, or even copying the code from my SQL Fiddle, is in my mind rude. Please do your own work.
Comment: @ErikE Thank you for the fiddle. I upvoted your answer and then copied your fiddle create data. I'll be sure to not repeat in future.
Comment: @ErikE I also want to let you know - that as a fellow SO user I admire you very much. Thank you for helping everyone. I really hope imitation is best form of flattery.
Comment: I would of course defer to the community's thoughts on copying fiddles, but  I can't find anything on meta about it. Thank you for the compliment!
Answer: count and group by
SELECT
   StudentID,
   Count(*) as 'count'
FROM
   dbo.StudentActionCodes
GROUP BY
   StudentID;

TSQL- How to count number of instances