Netconsole not working

Question: I've gotten netconsole setup on a server that's been having a variety of kernel panics lately and I'm trying to log kernel messages to another server. I've tested netconsole after setting it up on the host server by start a netcat session and typing in some random bits of data. These arrive on the capturing server without any issue and I'm able to view the text in the log file I have setup.
However, nothing that is being logged in dmesg (i.e. kernel messages) is being forwarded to the capturing server. My printk is set to 6 1 4 7 (I've gone up to 8 1 4 7 as well) and nothing. I've tried enabling/disabling modules with modprobe and get no output to the capturing server.
Any advice on this would be appreciated as I'm not certain of what else to check at this point since everything is apparently setup correctly.
Comment: I've just had the similar problem. The solution was to specify the netconsole parameter in a correct way. This is how it worked for me `firstname.lastname@example.com/,@192.168.173.150/`. It's with the default ports, src-dev and without tgt-mac.
Answer: You say you've tested netconsole by setting it up on the host server. Do you mean you set up a listening syslogd there, and used netcat on the client to send messages to it? If yes, then it doesn't seem like you actually tested netconsole.
Have you verified the netconsole setup, with all mac addresses being correct? The documentation is nice and detailed. Once you've loaded the module with the appropriate attributes, you can test it by writing to /dev/kmsg as root:
<code># echo my kernel message > /dev/kmsg
</code>
Alternatively, trigger a memdump or crash with sysrq. Netconsole should pick this up and send it to your target. Tcpdump is very useful to verify what sort of packets are sent out while you test. Something like this would get you started (notice -e, which will include ethernet addresses):
<code># tcpdump -i eth0 -n -e port 514
</code>
You mention that you want to capture panics. The nature of these panics could be such that they completely kill the system before netconsole (or kexec/kdump) is able to do anything at all (as was the case for the recent leap second issues), or you could indeed get successful log entries.
On a side note, an alternative to netconsole is to use the kexec+kdump kernel facilities. Upon "manageable" panics, the kernel will kexec a kdump-enabled kernel which will load a minimal initrd, and then write the kernel dump to disk. It can then later be analyzed by tools such as crash.

Netconsole not working


What is not possible to do with Spring XML defined context?

Question: The options are: a)Autowire beans by their type
                 b)Inject values from property file
                 c)Use annotation in defined bean
                 d)Override beans defined in a parent application context
                 e)None of the above
Possible multiple options:
Comment: apparently: googling this or reading a text-book on the subject, is the answer.
Answer: We can autowire beans by their type. But we cant inject the values from the properties file. And rest options seems correct
So b option may be. Am I right ?


Custom node Template for script tags

Question: I wanted to add script code to implement "LiveChat" functionality on just three pages of our website. So, I wanted to add one script code given by our LiveChat agency into three webpages of our website.
Script Code

So, I have created three separate html node templates based on their node ID.
Theses three custom node templates are under
/sites/all/themes/mytheme/templates along with (node.tpl.php and html.tpl.php)

(contact page) html--node--28.tpl.php

Url of this page: www.example.com/contact

(admissions page) html--node--1212.tpl.php

Url of this page: www.example.com/admissions

(market page) html--node--378.tpl.php

Url of this page: www.example.com/market
Code used in all custom node templates (html--node--28.tpl.php, html--node--1212.tpl.php, html--node--378.tpl.php) is the same:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"
  "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="<?php print $language->language; ?>" version="XHTML+RDFa 1.0" dir="<?php print $language->dir; ?>"<?php print $rdf_namespaces; ?>>
<head profile="<?php print $grddl_profile; ?>">
  <?php print $head; ?>
  <title><?php print $head_title; ?></title>
  <?php print $styles; ?>

<body<?php print $attributes;?>>
<?php print $scripts; ?>
//Adding script for LiveChat

    <?php print t('Skip to main content'); ?>

  <?php print $page_top; ?>
  <?php print $page; ?>
  <?php print $page_bottom; ?>

But, I am not able to see my script () when I debugged it in view pagesource.
Thanks!
Comment: Just to double check cus you didn't mention doing it, did you clear you cache after adding these template files?
Comment: Yes, I have cleared cache
Comment: If you make a change to your html.tpl.php file in that containing folder, do your other pages get changed? And do these pages change?
Comment: Yes, if I add this
Answer: Instead of adding three template files for three nodes with the same content in it in my opinion will be a better solution to add the JS file conditionally using drupal_add_js
For eg:

 $allowed_nid = array(28, 1212, 38);
 if(in_array(arg(1), $allowed_nid)){
  drupal_add_js("http://www.examp.com/scripts/invitation.ashx?company=4444", "external");
 }

You can add the snippet in a hook_init function in any of your custom modules. (Best is add in a custom utility module which hold all general function to be used throughout.)
Else you can add it in your theme's template.php in the template_preprocess_page function.
Comment: Thanks Malabya .. Where should I add this function? Should I add it in template.php file?
Comment: @Vidushi Have updated the answer with it. :)
Comment: Awesome.. Thanks. Instead of just three nodes, Can we make it available for under entire 'Admissions' sections? I mean all the subpages of 'Admissions' section?
Comment: What do you mean by subpages? As per my understanding from your question its better to have 3 separate pages.
Comment: Yes, that is correct. But, now my client wants it on all the admissions pages as well. If 2.0 Admissions, 2.1 - About admissions, 2.1.2 - Contact us, 2.1.3,2.1.4, 2.3,2.4... So, all the subpages of Admissions along with three different pages.
Comment: OK. So they will be separate pages then just add the nid of the pages in the `$allowed_nid` array
Comment: There are more than 50 pages under admissions, so is there any other way?
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/24765/discussion-between-malabya-tewari-and-vidushi).
Answer: You can do this to your html.tpl.php
<?php $node = menu_get_object();
      if ($node && $node->nid == NID):?>

<?php endif;?>

Custom node Template for script tags


Error compiling Rglpk on Windows

Question: I need to use the R package Rglpk in a project and need to modify the package - in particular, I need to add a time limit control parameter option to the interface (it is a option in GLPK v4.5.2 itself, but Rglpk 0.5-2 does not support this parameter).  This will allow R to call GLPK and limit the search time to be user specified (i.e., search time <= t).
I have modified Rglpk code and have successfully compiled the package on Mac OS X v10.9.2 and verified that it worked as expected.  My project needs to run on Windows and I have not been able to compile the modified Rglpk package, or for that matter the unmodified package (Rglpk 0.5-2), on Windows.  I have tried uploading the original package to Win Builder (http://win-builder.r-project.org/), but I have received the same errors.
The following is a summary of what I have done on Windows 7 (my machine):
(1) Download the Rglpk 0.5-2 package source from http://cran.r-project.org/web/packages/Rglpk/index.html
(2) Unzip the file to a directory.  All the code is in a directory called Rglpk.
(3) Start RStudio v0.98.490.  I have devtools v1.4.1 installed, Rtools v3.1 installed, and R v3.0.2
(4) Open a new project and select the Rglpk directory mentioned in (2).
(5) Under the 'Build' menu, select 'Build and Reload'.  The result of this is that the package compiles and is loaded:
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (Rglpk)

(6) Under the 'Build' menu, select 'Check Package'.  The result is that the installation fails.  The log file is shown below.
I have tried to compile another package that has C code and have not been able to compile it either.  The package I tried to compile is devtools v1.5  The error that came back in that case is   'LoadLibrary failure:  %1 is not a valid Win32 application.'
Does anyone have any experience compiling R packages on Windows that include C code?  I have been able to compile packages that involve only R code, but it seems C code is troublesome.
Any help would be appreciated.
Log File
* using R version 3.0.2 (2013-09-25)
* using platform: x86_64-w64-mingw32 (64-bit)
* using session charset: ISO8859-1
* checking for file 'Rglpk/DESCRIPTION' ... OK
* this is package 'Rglpk' version '0.5-2'
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... NOTE
Found the following apparent object files/libraries:
  src/GLPK/alloc.o src/GLPK/amd/amd_1.o src/GLPK/amd/amd_2.o
  src/GLPK/amd/amd_aat.o src/GLPK/amd/amd_control.o
  src/GLPK/amd/amd_defaults.o src/GLPK/amd/amd_dump.o
  src/GLPK/amd/amd_info.o src/GLPK/amd/amd_order.o
  src/GLPK/amd/amd_post_tree.o src/GLPK/amd/amd_postorder.o
  src/GLPK/amd/amd_preprocess.o src/GLPK/amd/amd_valid.o
  src/GLPK/bignum.o src/GLPK/cfg.o src/GLPK/cfg1.o
  src/GLPK/colamd/colamd.o src/GLPK/dmp.o src/GLPK/env.o
  src/GLPK/error.o src/GLPK/ffalg.o src/GLPK/fhv.o src/GLPK/fhvint.o
  src/GLPK/glpapi01.o src/GLPK/glpapi02.o src/GLPK/glpapi03.o
  src/GLPK/glpapi04.o src/GLPK/glpapi05.o src/GLPK/glpapi06.o
  src/GLPK/glpapi07.o src/GLPK/glpapi08.o src/GLPK/glpapi09.o
  src/GLPK/glpapi10.o src/GLPK/glpapi11.o src/GLPK/glpapi12.o
  ..... (more lines are here, I have removed them for brevity)
  src/GLPK/triang.o src/GLPK/wclique.o src/GLPK/wclique1.o
  src/GLPK/zlib/adler32.o src/GLPK/zlib/compress.o
  src/GLPK/zlib/crc32.o src/GLPK/zlib/deflate.o src/GLPK/zlib/gzclose.o
  src/GLPK/zlib/gzlib.o src/GLPK/zlib/gzread.o src/GLPK/zlib/gzwrite.o
  src/GLPK/zlib/inffast.o src/GLPK/zlib/inflate.o
  src/GLPK/zlib/inftrees.o src/GLPK/zlib/trees.o
  src/GLPK/zlib/uncompr.o src/GLPK/zlib/zio.o src/GLPK/zlib/zutil.o
Object files/libraries should not be included in a source package.
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking whether package 'Rglpk' can be installed ... ERROR
Installation failed.

The contents of the 00install.out file are:

* installing *source* package 'Rglpk' ...
** libs

*** arch - i386
gcc -m32 -I"C:/PROGRA~1/R/R-30~1.2/include" -DNDEBUG -IGLPK -IGLPK/amd -IGLPK/colamd -IGLPK/minisat -IGLPK/proxy -IGLPK/zlib    -I"d:/RCompile/CRANpkg/extralibs64/local/include"     -O3 -Wall  -std=gnu99 -mtune=core2 -c Rglpk_initialize.c -o Rglpk_initialize.o
gcc -m32 -I"C:/PROGRA~1/R/R-30~1.2/include" -DNDEBUG -IGLPK -IGLPK/amd -IGLPK/colamd -IGLPK/minisat -IGLPK/proxy -IGLPK/zlib    -I"d:/RCompile/CRANpkg/extralibs64/local/include"     -O3 -Wall  -std=gnu99 -mtune=core2 -c Rglpk_read_file.c -o Rglpk_read_file.o
Rglpk_read_file.c: In function 'Rglpk_read_file':
Rglpk_read_file.c:84:6: warning: 'status' may be used uninitialized in this function [-Wuninitialized]
Rglpk_read_file.c: In function 'Rglpk_retrieve_MP_from_file':
Rglpk_read_file.c:192:6: warning: 'status' may be used uninitialized in this function [-Wuninitialized]
gcc -m32 -I"C:/PROGRA~1/R/R-30~1.2/include" -DNDEBUG -IGLPK -IGLPK/amd -IGLPK/colamd -IGLPK/minisat -IGLPK/proxy -IGLPK/zlib    -I"d:/RCompile/CRANpkg/extralibs64/local/include"     -O3 -Wall  -std=gnu99 -mtune=core2 -c Rglpk_solve.c -o Rglpk_solve.o
Rglpk_solve.c: In function 'R_glp_solve':
Rglpk_solve.c:25:10: warning: variable 'kl' set but not used [-Wunused-but-set-variable]
gcc -m32 -shared -s -static-libgcc -o Rglpk.dll tmp.def Rglpk_initialize.o Rglpk_read_file.o Rglpk_solve.o GLPK/libglpk.a -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/PROGRA~1/R/R-30~1.2/bin/i386 -lR
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: i386:x86-64 architecture of input file `GLPK/libglpk.a(glpmpl04.o)' is incompatible with i386 output
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: i386:x86-64 architecture of input file `GLPK/libglpk.a(error.o)' is incompatible with i386 output
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: i386:x86-64 architecture of input file `GLPK/libglpk.a(glpmpl03.o)' is incompatible with i386 output
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: i386:x86-64 architecture of input file `GLPK/libglpk.a(alloc.o)' is incompatible with i386 output
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: i386:x86-64 architecture of input file `GLPK/libglpk.a(glpenv07.o)' is incompatible with i386 output
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe:
.....
Error: 64-bit reloc in dll
Error: 64-bit reloc in dll
Error: 64-bit reloc in dll
.....
Rglpk_initialize.o:Rglpk_initialize.c:(.text+0x33): undefined reference to `glp_term_hook'
Rglpk_initialize.o:Rglpk_initialize.c:(.text+0x44): undefined reference to `glp_version'
Rglpk_read_file.o:Rglpk_read_file.c:(.text+0x10): undefined reference to `glp_delete_prob'
Rglpk_read_file.o:Rglpk_read_file.c:(.text+0x4f): undefined reference to `glp_term_out'
Rglpk_read_file.o:Rglpk_read_file.c:(.text+0x60): undefined reference to `glp_delete_prob'
Rglpk_read_file.o:Rglpk_read_file.c:(.text+0x65): undefined reference to `glp_create_prob'
.....
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x4326): undefined reference to `strcpy'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x4d39): undefined reference to `strcpy'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x4dc5): undefined reference to `strcpy'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x4e05): undefined reference to `floor'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x4e16): undefined reference to `floor'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x502e): undefined reference to `floor'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x50e2): undefined reference to `sprintf'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x5111): undefined reference to `sprintf'
.....
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x990): undefined reference to `exp'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0x9e0): undefined reference to `log'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0xa30): undefined reference to `log10'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0xaee): undefined reference to `sin'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0xb4e): undefined reference to `cos'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0xb65): undefined reference to `atan'
GLPK/libglpk.a(glpmpl03.o):glpmpl03.c:(.text+0xb79): undefined reference to `atan2'
.....
GLPK/libglpk.a(deflate.o):deflate.c:(.text+0x2ec0): undefined reference to `memcpy'
GLPK/libglpk.a(deflate.o):deflate.c:(.text+0x2ed5): undefined reference to `memcpy'
GLPK/libglpk.a(deflate.o):deflate.c:(.text+0x2ee7): undefined reference to `memcpy'
GLPK/libglpk.a(zutil.o):zutil.c:(.text+0x47): undefined reference to `malloc'
GLPK/libglpk.a(zutil.o):zutil.c:(.text+0x54): undefined reference to `free'
collect2: ld returned 1 exit status
ERROR: compilation failed for package 'Rglpk'

Comment: wait !!!   I see mismatch of the Rtool pack and the version of R ..... how can that succeed? I'm also seeing hints that there may be a 64-bit/32-bit mismatch. I won't be the source of a final answer `cuz I'm a Mac user.
Comment: The Rtools v3.1 supports R v3.0.x to v3.1.x (see http://cran.r-project.org/bin/windows/Rtools/).  I did see the 64-bit/32-bit mismatch as well.  What I found odd is that the Win Builder service also returns the same errors - I'd be very interested to know how the package's maintainer compiled the binary file for Windows.  I have emailed them the problem, but have heard back yet.
Answer: It looks like a couple things are going wrong:

Object files are getting pre-compiled, but not cleaned out.
Because these object files are being pre-compiled, and it looks like they were compiled for 64-bit architecture, when you try to use these in a 32-bit build, the installation process barfs.

The fix here is going to be deleting all of those *.o files (they need to be compiled as part of the R CMD build step, I imagine), and the resulting tarball can then be R CMD checked.
You should consider installing a more up-to-date version of RStudio and devtools, and ensure you use devtools to handle the build process -- it might be able to help you out here.

Error compiling Rglpk on Windows


How to load different javascript libraries depending on browser?

Question: I have a library that does not work in IE unfortunately, because IE does not support promises and other html5 features. I want to keep that library for Chrome & Firefox, but use a different one for IE. How can I separate the two like following:

Obviously, conditionally formatting only works on >IE9 browsers, but I wanted something similar to this.
Answer: you can use browser detection functions on your page load event:
function BrowserDetection() {
    //Check if browser is IE
    if (navigator.userAgent.search("MSIE") & gt; = 0) {
        // insert conditional IE code here
    }
    //Check if browser is Chrome
    else if (navigator.userAgent.search("Chrome") & gt; = 0) {
        // insert conditional Chrome code here
    }
    //Check if browser is Firefox
    else if (navigator.userAgent.search("Firefox") & gt; = 0) {
        // insert conditional Firefox Code here
    }
    //Check if browser is Safari
    else if (navigator.userAgent.search("Safari") & gt; = 0 & amp; & amp; navigator.userAgent.search("Chrome") & lt; 0) {
        // insert conditional Safari code here
    }
    //Check if browser is Opera
    else if (navigator.userAgent.search("Opera") & gt; = 0) {
        // insert conditional Opera code here
    }
}

Source: https://www.learningjquery.com/2017/05/how-to-use-javascript-to-detect-browser
Answer: you can dynamic create script
like:

Comment: Thanks, I used something like in the end. Found another solution using jQuery's $.getScript but it seemed to load slower than simply creating script elements.


NSMutable String "Out of scope"

Question: I have two NSMutableString objects defined in my viewController's (a subclass of UITableViewController) .h file:
NSMutableString *firstName;
NSMutableString *lastName;

They are properties:
@property (nonatomic, retain) NSMutableString *firstName;
@property (nonatomic, retain) NSMutableString *lastName;

I synthesis them in the .m file.
In my viewDidLoad method - I set them to be blank strings:
firstName = [NSMutableString stringWithString:@""];
lastName = [NSMutableString stringWithString:@""];

firstName and lastName can be altered by the user. In my cellForRowAtIndexPath method, I'm trying to display the contents of these strings:
cell.detailTextLabel.text = [NSString stringWithFormat:@"%@ %@", firstName, lastName];

but this is causing the app to crash as soon as the view controller is displayed. Using the debugger, it seems that both firstName and lastName are "out of scope" or that they don't exist. I'm new to Xcode but the debugger seems to halt at objc_msgSend.
What am I doing wrong?
Comment: So where are the:

NSMutableString *firstName;
NSMutableString *lastName;

Defined?, at the file level, or the Class level, or in the Method?
Just FYI you were right to add the objective-c tag BTW, as this is definitely "classic" Objective-C.
Answer: The problem is that you need to do:
self.firstName = ...
self.lastName = ...

That will call the auto-generated setter methods, which will retain the values. By just assigning directly, you are bypassing the setters, and as soon as the current autorelease pool is drained the two variables will have dangling pointers.
Answer: make habit of putting self.variableName... and instead of stringWithsString try using initWithString...hope it will solve the problem...BTW you have explained the question very well...
Answer: Your cell is loading and accessing firstName and lastName before they are initialized. viewDidLoad is too late, try awakeFromNib or viewWillLoad or whatever it is on iPhone.
Comment: That would make the values nil, which wouldn't give the expected strings but wouldn't crash either.
Comment: 'Out of scope' is not nil, it means that the compiler hasn't gotten around to assigning any value, nil or not, to the variable in question yet.
Answer: You can't use:
firstName = [NSMutableString stringWithString:@""];

Since it will deallocate as soon as viewDidLoad execution finishes.
Try:
firstName = [[NSMutableString alloc] initWithString:@""];

Or:
[self setFirstName:[NSMutableString stringWithString:@""]];

And don't forget to release firstName and lastName on dealloc message.
Answer: what cell style did you used?
if (cell == nil) {
  cell = [[[UITableViewCell alloc] initWithStyle:UITableViewCellStyleSubtitle
                                 reuseIdentifier:BasicCellIdentifier] autorelease];
}

perhaps this will help you

NSMutable String "Out of scope"


VSTS - Widget only works for some people anod not others. What might be the issue?

Question: ORIGINAL POST:
I've built a widget that works for some people in the company, but not other people and I was wondering if anyone might help me.
For the people that it works for, it doesn't matter what computer or browser they are on, it works. While for thosethat it does not work for, it just doesn't work no matter what computer or browser they use.
The Widget is very simple. It builds a query using the query API before plotting the result on a graph.
The main problem seems to be that the query is returning zero result for some people, for some unknown reason.
It doesn't seem to have anything to do with licenses or security permission as some people on stackholder licenses can see the widget, while others on basic license with project admin permission cannot.
Any suggestions would be much appreciated.
Code:
First, I have a method that builds the query depending on the type of query I want.
var queryString = buildWiql(queryType, dateString, iterationPath);

The implementation of buildWiql() is:
var buildWiql = function (queryType, dateString, iterationPath) {

    var queryString;

    if (queryType=== "someType") {
        queryString = {
            "query": "Select [Microsoft.VSTS.Scheduling.RemainingWork] From WorkItems Where [System.WorkItemType] = 'Task' AND [System.State] <> 'Done' AND [System.State] <> 'Closed' AND [System.IterationPath] = '" + iterationPath + "' order by [System.CreatedDate] desc ASOF " + dateString
        };
    }
    else if (queryType === "someOtherType") {
        ....
    }
    else {
        ....
    }

    return queryString;
};

Then inside the main function:
return TFS_Wit_WebApi.getClient().queryByWiql(queryString)
    .then(function (query) {
        console.log(query);
        ....
    }

When the console logs the query result. For those whom it doesn't work for, the query returns an array with 0 elements.
UPDATE:
For accounts that the widget doesn't work for. Opening the console in the browser gives this error:
Failed to load resource: the server responded with a status of 400 (Bad Request)
*******ERROR*******:
Error: Expecting date-time. The error is caused by <<'20/8/2016'>>.(....)

Comment: Are the results the same for all iterations? What's the result if these users query work items in a iteration through query?
Comment: what's the result if these users retrieve data through REST API directly?
Comment: Add error function handler (second parameter of then function) .then(function (query) {
                        // Do something with the query

                        return WidgetHelpers.WidgetStatusHelper.Success();
                    }, function (error) {
                        return WidgetHelpers.WidgetStatusHelper.Failure(error.message);
                    });
Comment: @starain This applies for all iterationPath/Team areas. For eg. My account can see the widget working in all team areas. While some others can't see it in any team areas.
Comment: @starain I shall try adding a error function handler. Can you elaborate on how those users can retrieve data through REST API directly?
Comment: POST https://{instance}/DefaultCollection/[{project}/]_apis/wit/wiql?api-version={version}  https://www.visualstudio.com/en-us/docs/integrate/api/wit/wiql
Comment: @starain The problem seem to be related to date-time formatting. I have updated the post with the error message
Comment: What's the result if you try to use 8/20/2016 date time?
Answer: The problem with the widget was that apparently, everyone can have a different date-time format preference set in their profile. eg. '25/08/2016' vs '25 Aug 2016'.
However, my widget assumed that the format was always 'dd/MM/yyyy', which is the format of the dateString it passes into the 'ASOF' part of the query.
For users who's got a different date-time format setting. The result would be zero, because VSTS wouldn't understand what '25/08/16' is
The solution is to convert the string into this format: 2016-08-20T20:49:34.193Z
And now, regardless of the user's date-time format setting, the query will return correct results. The code I used to do the conversion is:
var dateToString = function (dateIn) {

    var date = new Date(dateIn.getTime() + 86400000);
    var day = date.getDate();
    var month = date.getMonth() + 1;
    var year = date.getFullYear();

    //Format date as '2016-08-20T20:49:34.193Z'"
    var dateString = year + '-' + month + '-' + day + 'T00:00:00.000Z';

    return dateString;

};


Do not send registration mail to fake users, who haven't specified their gender in registration form

Question: Since several years I run a Drupal 7 web site - as a frontend to a multi-player card game.
I have the following SPAM protection at my web site: the new users should specify their gender at the registration (it should be "Male" or "Female", but not the default "Robot") and then they have to play the card game:

Every night I run a cronjob, which deletes users who have wrong gender or who have registered 10 days ago and still haven't played:
delete from drupal_users
where uid <> 0 and
to_timestamp(created) < (now() - interval '10 day') and
uid not in (select distinct id from pref_money); /* haven't played the game */

This does work well, BUT recently the amount of registering fake users has increased to hundreds per hour and my site keeps sending lots of registering mails (which bounce then to my poor mailbox) every minute.
So my question is: how NOT to send the registration mail to users who have their gender set to "Robot"?
Here is what I have found sofar:
1) The gender (an additional field at my reg. form) can be found the following way:
        $result = db_query('select field_gender_value from {field_data_field_gender} where entity_id=:id', array(':id' => array($viewer_id)));
        $gender = $result->fetchField();
        if ('Robot' == $gender) {
           ....
        }

2) And I probably should provide a hook_mail_alter in my custom module:
function MY_CUSTOM_MODULE_mail_alter(&$message) {
  # XXX how to find gender here?
  if ('Robot' == $gender) {
      # XXX how to abort mailing for registration only?
      $message['send'] = FALSE;
      return;
  }
}

But in 2) I don't know 2 things - how to find the gender value there and how to cancel the registration mails only?
Any suggestions please?
Please do not suggest the Mollom module or any standard Drupal anti-spam measures, because my question is very specific.
Comment: mail_alter only has access to the message. Is the gender information part of it?
Comment: No and that is my problem - how to find the gender (an additional field in registration form) from the `hook_mail_alter`?
Comment: @AlexanderFarber Does `$message` contain a reference to the related user account?
Comment: I don't know, that is why I am asking the Drupal experts here :-)
Comment: Ah ok - it would be _much_ easier (and quicker) for you to check that yourself than wait for someone here to set up a test and do it. If you don't already have it, install the Devel module, and add `dpm($message);` to the mail alter hook. Then invoke the email you're trying to catch into sending, and a message will appear on screen containing all the contents of `$message`
Comment: Do you mean I should add the following to my custom module? (I do have one to generate custom blocks): `function MY_CUSTOM_MODULE_mail_alter(&$message) { dpm($message); }`
Comment: Even if that were to be the case, scrapping the text for this information and then starting to dig in the DB would be terribly brittle. I don't think this is the way to go.
Comment: Yes something just like that. @Letharion I meant is there a _real_ reference to the account, not something in text (a uid in a parameter perhaps). That could be used to reliably load the user and get field info without being brittle. But I don't know if the params are passed on with the message after it's gone through `hook_mail()`
Comment: @Clive, I see. I started down the path of parsing out "Robot" form the text, before I realized how bad that would be, and then I interpreted your comment along the lines of that.
Comment: Assuming that if someone refused to specify gender equals being fake is pretty much pointless. Bots probably will specify it, they are written to specify everything and usually have some decent defaults and decent field recognition built in. On the other hand many real people thinks that specifying gender is a bad idea as they want to be judged by their words, not by their perineum. They might choose "robot" in good faith expecting it to be an option for people that don't care to be treated as either male or female.
Comment: Not at my website where I have it this way since over 2 years
Answer: I suggest you implement hook_form_id_alter to either add a new validation function to the registration form, or replace the submit callback.
In a validation function, you have access to the form values, and you can do whatever checks you want, and fail the submission. This will reject the form all together.
If you replace the submit callback instead, you could override only the sending of the e-mail. but otherwise let the form pass through, which seems closer to what is being asked for.
Comment: Do you suggest to modify `modules/user/user.module`? Which method there? I'd like to change `user_validate_mail` - but how to find the gender field value there?
Comment: This is the best approach, but I would add an additional validation handler instead of replacing the submit function.
Comment: Additional validation handler does sound better, I'll update with that.
Comment: Idea is nice, but rejecting the form for bot will tell him he failed. So it's good idea to not highlight the field that failed. With CAPTCHA it's OK as it is supposedly unsolvable anyway, but here it would just tell bot to select another option and try again. Of course only smarter bots will ever try, but their number is growing.
Comment: Yeah, I thought of that as well, which is why I ended up keeping the submit option in there as well. Overriding the submit one can still fake a successful POST.
Comment: The CAPTCHA Drupal module isn't stopping bots anymore - neither the Image, nor the Math. My Drupal 7.24 site had been under attack in recent 2 months by a bot, which uses firstname.lastname@example.com addresses and CAPTCHA (on password and user reg. forms) didn't stop it - until I installed a custom font + switched to russian letters.
Comment: How do you "fail a submission" in a `hook_form_id_alter` method?
Comment: You don't. If you read my answer above, I state however that you can do so in a validation function.


pdf of a member of a sequence of dependent random variables

Question: I would very much appreciate a hint for the following problem
Let $\left(X_n\right)_{n=1}^\infty$ be a sequence of random variables s.t.:
$$X_1 \sim U_{[0,1]}$$
and for all $n>1$:
$$X_n \sim U_{[0,X_{n-1}]}.$$
Give a general expression for $f_{X_n}$ the pdf of $X_n$.
Thanks
Comment: Have you solved it for small $n$?
Comment: This is very closely related to [product distribution of two uniform distribution, what about 3 or more](https://math.stackexchange.com/questions/659254/product-distribution-of-two-uniform-distribution-what-about-3-or-more).
Answer: Hint: Letting $V_1,V_2,\dots$ be a sequence of iid random variables distributed uniformly on $[0,1]$, show that $X_n$ has the same distribution as $V_1\cdot V_2\cdot\ldots \cdot V_n$. Next, find the distribution of $\log X_n$, which is a sum of the iid variables $\log V_i$ (what distribution does $\log V_i$ have?). The $\log$ trick is useful since pdfs of sums are easier to find than pdfs of products. Finally, use a transformation to get the pdf of $X_n$ from that of $\log X_n$.
Comment: Also, a hint for the pdf of $\log V_1+\dots+\log V_n$: compute it for $n=1,2,3\dots$ until you see a pattern, then prove it by induction.
Answer: I do not guarantee that this hint will lead to results.
If $F_{n}$ denotes the CDF and $f_{n}$ the PDF of $X_{n}$ then
for $x\in\left[0,1\right]$ we find:
$\begin{aligned}F_{n+1}\left(x\right) & =\int_{0}^{x}P\left(X_{n+1}\leq x\mid X_{n}=y\right)f_{n}\left(y\right)dy+\int_{x}^{1}P\left(X_{n+1}\leq x\mid X_{n}=y\right)f_{n}\left(y\right)dy\\
 & =\int_{0}^{x}f_{n}\left(y\right)dy+\int_{x}^{1}\frac{x}{y}f_{n}\left(y\right)dy\\
 & =F_{n}\left(x\right)+x\int_{x}^{1}\frac{f_{n}\left(y\right)}{y}dy
\end{aligned}
$
Differentiating both sides we find:
$$f_{n+1}\left(x\right)=f_{n}\left(x\right)+\int_{x}^{1}\frac{f_{n}\left(y\right)}{y}dy-x\frac{f_{n}\left(x\right)}{x}=\int_{x}^{1}\frac{f_{n}\left(y\right)}{y}dy$$
Further we can start with $f_1(x)=1_{[0,1]}(x)$.
Comment: This was the sort of direction I was taking, but I could not find a justification for the first equality which seems intuitive (looks like a variation of the law of total probability) but wasn't proven in my class.
Comment: I think it leads to $f_{n+1}\left(x\right)=\frac{1}{n!}\left(-\ln x\right)^{n}$. Sorry if it is useless for you.
Answer: If $[0\le x\le1]$ is the pdf for $x$, then the cdf for $x$ is $x\,[0\le x\le1]$. Thus, the cdf for $y=\log(x)$ is $e^y\,[y\le0]$, and therefore the pdf for $y$ is $e^y\,[y\le0]$. The pdf for the sum of $n$ values of $y$ is the $n$-fold convolution of the pdf $e^y\,[y\le0]$ with itself. The Fourier Transform of this $n$-fold convolution is the $n^\text{th}$ power of the Fourier Transform of the pdf $e^y\,[y\le0]$, which is
$$
\int_{-\infty}^0 e^{-2\pi iyt}e^y\,\mathrm{d}y=\frac1{1-2\pi it}\tag1
$$
Thus, the pdf for the sum of $n$ values of $y$ is
$$
\begin{align}
\sigma_n(y)
&=\int_{-\infty}^\infty\frac{e^{2\pi iyt}}{(1-2\pi it)^n}\,\mathrm{d}t\tag{2a}\\
&=\frac{e^y}{2\pi i}\int_{1-i\infty}^{1+i\infty}\frac{e^{-yz}}{z^n}\,\mathrm{d}z\tag{2b}\\
&=e^y\frac{(-y)^{n-1}}{(n-1)!}\,[y\le0]\tag{2c}
\end{align}
$$
Explanation:
$\text{(2a)}$: take the inverse Fourier Transform
$\text{(2b)}$: substitute $t=\frac{1-z}{2\pi i}$
$\text{(2c)}$: if $y\gt0$, close the contour on the right half-plane, missing the singularity at $z=0$
$\phantom{\text{(2c):}}$ if $y\le0$, close the contour on the left half-plane, enclosing the singularity at $z=0$
The cdf for the sum of $n$ values of $y$ is the integral of $(2)$
$$
\Sigma_n(y)=e^y\sum_{k=0}^{n-1}\frac{(-y)^k}{k!}\,[y\le0]\tag3
$$
The cdf for the product of $n$ values of $x=e^y$ is therefore
$$
\Pi_n(x)=x\sum_{k=0}^{n-1}\frac{(-\log(x))^k}{k!}\,[0\le x\le1]\tag4
$$
The pdf for the product of $n$ values of $x$ is the derivative of $(4)$
$$
\bbox[5px,border:2px solid #C0A000]{\pi_n(x)=\frac{(-\log(x))^{n-1}}{(n-1)!}\,[0\le x\le1]}\tag5
$$
The pdf of $X_n$ is given by $(5)$.


Really confused about the relationship between set theory, functions, ZFC, Peano axioms, etc.

Question: I don't understand how everything relates. It seems that ZFC is a "first order theory" with axioms described in the language of first order logic, and it can recreate all the same axioms of Peano arithmetic (but not the other way around), so I suppose this makes PA a first order theory as well.
But then I am hearing that Peano's axioms are technically a second order theory? But then there's the first order theory that isn't as strong? Then I am unsure where natural numbers are defined exactly, and if this technically requires us to have set theory first in order to talk about membership? And what about functions? Don't these require set theory as well? Does this mean functions require ZFC? And if not, then what exactly are the "sets" we're using here?
I'm just totally lost as to what's defined where in terms of what and what's required to do this or that, it's all so hazy and vague and unclear and after reading countless answers on this website where everyone recommends the same unclear links that only partially answer the question, I'm losing a bit of hope.
Can anyone just plop the stuff down in a super easy to understand relationship hierarchy that clearly delineates what builds on what?
Comment: This is a great great good question (+2).
Comment: Tao's exposition is informal; but basically it is FOL (see **Remark 2.1.10**, page 19)
Comment: Look at Eric Wofsey's answer to a [similar question](https://math.stackexchange.com/questions/2612925/how-can-the-peano-postulates-be-categorical-if-they-have-nonstandard-models) I asked.  Well actually, my question was different, but I think his answer will satisfy you.
Comment: We have *numbers* and we have *theories* studying them. Peano's axioms is the "best choice" : it can be perfectly formalized with second order logic. Unfortunately, SOL has some "difficulties" and thus we can use first-order theory of arithmetic: it has limitations, but it is very simple to manage it. In $\mathsf {ZFC}$ (and far less : the theory of Hereditary Finite Sets is enough) we can define a mathematical structure that is a perfect proxy for the natural numbers.
Comment: @MauroALLEGRANZA Is the version of Peano's Axioms in Tao's Analysis Vol 1 first order or second order?
Answer: Peano's name is attached to two different theories about the natural numbers, which unfortunately don't always have clearly different names. The following convention is fairly common, though:

The "Peano axioms" is a second-order theory, which just describes the successor function and a general induction axiom. With some amount of set theory as a background feature of the logic, we can then define addition and multiplication without needing specific axioms for them.
"Peano Arithmetic" is a first-order theory, developed long after Peano's time as a "best-effort" first-order approximation of the second-order Peano axioms. It has specific axioms for the successor function and addition and multiplication, and an induction axiom schema that only works for properties that can be expressed in its first order language of successor+addition+multiplication.

Peano Arithmetic is what is usually meant by just the abbreviation PA. (Note capital A and no "the" for PA).
Because the induction axiom in Peano Arithmetic is not as strong as the full second-order induction axiom, the theory is weaker -- it has models that are not isomorphic to the usual $\mathbb N$. (It is hard-to-impossible to describe one of these non-standard models; we just have an existence proof for them. It depends crucially on the fact that first-order logic is complete: every consistent theory has a model. This is not true about the standard semantics for second-order logic, which is why the second-order axioms are stronger).
Despite being weaker, first-order PA has a lot more theoretical interest, because first-order logic is a lot better behaved than second-order.

For "don't functions require ZFC?", see When does the set enter set theory? or perhaps What is the dependency hierarchy in foundational mathematics?.
Comment: @user709833: To me the question looked like you were asking why different sources describe Peano's system as alternately first-order and second-order, because you were not aware those sources speak about two different systems. If you did know that, I'm afraid I don't understand what the question is.
Comment: @user709833: No. When we do ordinary mathematics we're not assuming **any** formal foundational system "powering something underneath". Mathematics works just fine without them, as it did for thousands of years before any of the formal foundational systems were even thought of. The formal systems are _models_ of ordinary mathematical thought, not what mathematical thought "really is". [Read this!](https://math.stackexchange.com/a/2379859/14366)
Comment: Well I mean for example, raising a real number to a real power or something similar, doesn't proving such results require us to agree on what a real number is and how it works, how it's defined, etc?
Comment: I don't think this really answers the question, a lot of this is just restating what I already mentioned/am aware of, in the OP.
Comment: Perhaps [this will be helpful too](https://math.stackexchange.com/a/2379859/14366).
Comment: My question is basically the same as the second link you posted and I have read that answer (and most of yours on the subject), I just have a hard time understanding them. When we define natural numbers, what are we using? PA? Peano's axioms? ZFC? I understand that ZFC axiomatically defines how sets work, but then when I see Peano's axioms or PA they both seem to rely on sets already? And why does PA "overshadow" Peano axioms despite being weaker? I also see FOL using functions, which are defined in terms of sets... it's all so confusing and seemingly circular.
Comment: @user709833: Ultimately the natural numbers are _defined as_ "the counting numbers you learned about in primary school". We have nothing _better_ than that in mathematics, though we certainly have things that sound more obfuscatively learned. PA and/or the Peano axioms are best thought of as "some claims we're pretty sure are true about the natural numbers we already know and love, and which seem to be a good basis for using our nice systems for formal proofs to reason about those natural numbers". That is a worthy task for them, but they do not work well as an authoritative definition.
Comment: So when we are using real numbers, doing algebra, calculus, etc are we usually assuming Peano's axioms or PA "powering" everything underneath?
Comment: Sure but if I do something in calculus and someone wants a proof that it all works I will need to resort to those frameworks to show that it's all true, and depending on how far down I go (e.g. real numbers, etc) I may need to resort to some kind of foundational system would I not?
Comment: @user709833: No you won't. **Most proofs in mathematics are not carried out in formal systems**. The only time anyone even _contemplates_ writing down proofs in formal systems is when they're convincing themselves that the formal systems actually can work as _models of ordinary mathematical reasoning_. But the model is not the thing itself, and formal proofs in any foundational system is **NOT** "what ordinary mathematics really is".
Comment: @user709833: I think most real analysis texts work from a set of (informally stated) axioms of the real numbers, which tells how the real numbers _behave_, but not "what they really are". Some do present a construction of the reals based on the naturals and some set theory, but as a rule they never even attempt to define "what the natural numbers really are", nor even to present axioms for them. You're supposed to _know_ how the counting numbers work, from primary school!
Comment: Maybe I am being imprecise with my language but I don't mean some metaphysical claim like "what they are," more like "whatever they are, here's some descriptors of how they behave," yes. So for a real number I am referring to the formalization of how they work and the rules for exponentiating them etc.
Comment: @user709833: You're wrong when you expect that there's a foundational formalization in play at all when we're doing ordinary mathematics. [Read this already, dammit!](https://math.stackexchange.com/a/2379859/14366)
Comment: I have read that answer many times, even before this week! Maybe an easier example to ask: Let's say you have written some proof for something. I look at your work and say, "Hey right here you use this thing called the 'distributive law' $a(b + c) = ab + ac$. That's kind of peculiar, how do you know for sure you can do that?"
Comment: @user709833: Then I tell you that my work assumes we're working in a context where the distributive law holds. If you're in a different situation where the distributive law _doesn't_ hold, then my work may not apply to it -- caveat emptor.


Comparing two multinomial distributions

Question: Background: Imagine a pizza cut into 8 slices.
[
On each straight edge of the slice, I insert a magnet with opposite polarities facing outwards. If I separate these components, prevent flipping and shake them, they should form a full pizza.
Now, if I put an additional slice (same dimensions, 1/8th of a full pizza), a full pizza will not always form. It could form clusters of: 4 & 5, 3 & 6, 2 & 7 and 1 & 8.
A model (given by Hosokawa et al. (1994)) gives me the probability of each cluster forming. To validate the model, I do some physical experiments. I conduct 20 trials for each experimental condition.
My results look like this:
Cluster  Theoretical  Physical
3,6:    6.01961132827  4
1,8:    2.77455224377  5
4,5:    6.62198848501  5
2,7:    4.58384794294  6

The aforementioned data is a multinomial distribution (akin to a distribution obtained when rolling a dice). When there are 9 slices, each trial can end in one of 4 states.
In addition to the 9 slice experiment, I have data for a 40 slice (and a couple others) experiment as well. (Please let me know if you would like me to include it here)
Problem: To test the goodness of fit, I would run a Pearson's Chi-squared test. However, since the means of both distributions are "close", I cannot reject the null hypothesis. However, I cannot accept the null hypothesis either.
Question: Is there a better way show how "close" the model gets to the physical experiments? The multinomial equivalent of "standard deviation", or perhaps a confidence interval? Regression with confidence intervals?
Update: A colleague of mine suggested the following approach for regression analysis in R:
d=read.csv("data.csv")
length(unique(d$abs_state))
nrow(d)
d$n_componentsf=as.factor(d$n_components)

ncomps=9
dsubs=d[d$n_components==ncomps,]
# using exact multinomial test in EMT (better)
library(EMT)
# using Chi square test statistics
EMT::multinomial.test(dsubs$freq_obs,
                      dsubs$freq_pred/sum(dsubs$freq_pred),
                      useChisq=TRUE,MonteCarlo=FALSE)
# using p value calculated via Monte Carlo approach
EMT::multinomial.test(dsubs$freq_obs,
                      dsubs$freq_pred/sum(dsubs$freq_pred),
                      ntrial = 100000,
                      MonteCarlo=TRUE)

# other exact multinomial test implementation
library(RVAideMemoire)
RVAideMemoire::multinomial.test(dsubs$freq_obs,
                                p=dsubs$freq_pred/sum(dsubs$freq_pred))

# if you're interested in 95% confidence limits
# on your multinomial proportions you can
# calculate these like this
library(DescTools)
MultinomCI(dsubs$freq_obs, conf.level=0.95)

library(ggplot2)
library(lattice)

# if you would like to analyse all data together
# you could also analyse the observed frequency
# counts as having approx Poisson expectation,
# using a Poisson GLM
# (or a binomial GLM if you put cbind(freq_obs,freq_notobs) as your dependent var and use family=binomial)
library(afex)
library(MASS)
library(car)
library(effects)
set_sum_contrasts() # use effect coding, as data is unbalanced

fit=glm(freq_obs ~ freq_pred,
        family=quasipoisson, data=d)
summary(fit)
Anova(fit, test="LR", type="III")

plot(allEffects(fit),type="response", rug=F)
plot1=plot(Effect(c("freq_pred"),
            mod=fit),type="response",rug=F)
plot2=xyplot(freq_obs ~ freq_pred,
             data=d,
             type=c("p"),
             par.settings=
               simpleTheme(col=c("blue","red"),pch=16,cex=1.1))
library(latticeExtra)
plot1+plot2 # nice correspondence between observed and expected frequencies

# sticking model predictions in data frame and
# plotting this in ggplot2 instead
df=data.frame(effect(term="freq_pred",
                  mod=fit,
                  xlevels=list(freq_pred=seq(0,15,1)),type="response"))
df

library(ggplot2)
library(ggthemes)
ggplot(data=df) +
  geom_ribbon(data = df,
              aes(x=freq_pred, ymin=lower, ymax=upper), linetype=2, alpha=0.1) +
  geom_line(data = df, aes(x=freq_pred, y=fit)) +
  geom_point(data = d, aes(x=freq_pred, y=freq_obs, color=n_componentsf), cex=3) +
  ylab("Predicted frequency") +
  xlab("Observed frequency") +
  theme_few(base_size=16)

Data.csv has:
n_components,abs_state,freq_pred,freq_obs,freq_notobs
9,"3,6",6.019611328,4,16
9,"1,8",2.774552244,5,15
9,"4,5",6.621988485,5,15
9,"2,7",4.583847943,6,14
15,"3,6,6",1.81009773,0,20
15,"4,5,6",3.927622683,7,13
15,"7,8",13.49657189,13,7
15,"5,5,5",0.765707695,0,20
40,"4,5,5,5,6,7,8",0.803987454,0,20
40,"5,6,6,7,8,8",3.376961833,3,17
40,"2,7,7,8,8,8",0.230595232,0,20
40,"5,7,7,7,7,7",0.175319358,0,20
40,"5,6,7,7,7,8",2.709196442,1,19
40,"5,5,5,5,6,7,7",0.170797287,0,20
40,"4,5,5,6,6,7,7",0.847746645,1,19
40,"8,8,8,8,8",0.230119576,0,20
40,"4,6,7,7,8,8",1.795116571,0,20
40,"6,6,6,6,8,8",0.483846181,0,20
40,"5,5,5,5,6,6,8",0.185675465,1,19
40,"4,7,7,7,7,8",0.4072505,0,20
40,"5,5,5,6,6,6,7",0.274814936,1,19
40,"5,5,6,8,8,8",0.708881447,1,19
40,"4,6,6,8,8,8",0.479526825,1,19
40,"4,5,5,5,5,8,8",0.071967085,0,20
40,"5,5,7,7,8,8",1.233981848,1,19
40,"4,5,6,6,6,6,7",0.34756786,1,19
40,"6,6,6,7,7,8",2.208449237,2,18
40,"4,5,5,6,6,6,8",0.494611498,1,19
40,"5,5,5,5,5,7,8",0.061650486,0,20
40,"4,5,5,5,5,5,5,6",0.010322162,0,20
40,"3,6,6,6,6,6,7",0.071274376,0,20
40,"4,6,6,6,6,6,6",0.015244456,0,20
40,"6,6,7,7,7,7",0.656508868,1,19
40,"4,5,5,5,7,7,7",0.155256863,2,18
40,"5,5,6,6,6,6,6",0.04917738,0,20
40,"3,6,7,8,8,8",0.634760319,0,20
40,"3,7,7,7,8,8",0.430171526,1,19
40,"5,5,5,5,5,5,5,5",0.00022644,0,20

The code creates 3 impressive [to me at least :) ] plots. However, I am skeptical about their correctness. From what I understand, the regression curve represents the correspondence between the predictions and observations. This curve is probably the "best fit" given the correspondences. It seems, however, it concerns me that bulk of the corresponding points are made at the start. Is this concern valid?
With references to the regression curve, how is the confidence interval computed? I found the Bauer (1972) paper, but wasn't able to understand it satisfactorily. Help understanding how the confidence interval is computed will be appreciated.
Other (possibly rudimentary) questions:
 - What is the effect of not observing a state on the regression plot? For instance, the 1st and 4th states of the 15 slice experiment are not observed physically?
 - In the third plot (which you obtain after executing the R code), is there a particular reason why the axes might be flipped?
Finally, what conclusions can I draw from the plots? Most of the correspondence points on the regression plot are outside the confidence interval. Is this a foolproof indication of the model's (in)ability to make predictions?
Lastly and obviously, the data is insufficient. In order to find the required amount of data, one would have to conduct sufficient number of trials until the confidence interval is reduced to a desirable width (I've come to realize that choosing this width is not trivial). I've tried random replication of predicted frequencies (poor man's bootstrapping?), but once again, I run into the problem of not understanding how the confidence interval is computed from the Pearson's chi-squared statistics.
Comment: @Chill2Macht thanks for your super quick response! I did consider the covariance matrix, but a couple of concerns: (1) I'm not sure what in the covariance matrix I would have to use, and (2) "standard deviation" is (var)^0.5. So, what would that be in a covariance matrix?

My grip on stats is not good enough to figure this out by myself.
Comment: This comment might not be entirely helpful, but since the multinomial distribution is multivariate, the analog of the variance would be the (co)variance matrix https://en.wikipedia.org/wiki/Covariance_matrix I am not familiar enough with multivariate statistics to give recommendations for specific tests to use with the covariance matrix.
Comment: Well a matrix is a covariance matrix of some distribution if and only if it is positive semi-definite, and every positive semi-definite matrix has a "square root" http://linear.ups.edu/jsmath/0210/fcla-jsmath-2.10li108.html -- as for how to apply this fact in a statistical context, I have no idea. It would be the blind leading the blind. I wasn't sure if you had thought of the covariance matrix, and sometimes knowing terminology can make accidentally stumbling on the right result on google a lot easier. But other than that I don't really know, I'm sorry.
Comment: @Chill2Macht : Thanks a lot for the input. If you know someone who might be able to shed light on my problem, please do share my questions with that person.
Comment: A table of observed vs expected values is a good approach. Consider measuring the entropy for a single, albeit mildly abstract, measure.
Comment: Could the images from the code be added to this question? And also the full names of the references? Also the process for cluster formation could be explained in the question directly.
Comment: @Martin: Ipparthi D, Winslow A, Sitti M, Dorigo M, Mastrangeli M. Yield prediction in parallel homogeneous assembly. Soft matter. 2017;13(41):7595-608.
Comment: @Todd: Thanks a lot for your suggestion. As for measuring the entropy of a single measure, I'm not a 100% sure I understand what you mean. However, at the moment, I have a full plate which will prevent me from looking into the problem in-depth. Collaborations are welcome though :-) .
Comment: Are you interested in a measure of uncertainty, inference or a descriptive measure?
Comment: @Todd: As I'm looking for how "close" the model gets to the physical experiments, a measure of uncertainty would be useful.

In our paper, we present the respective yields and the maximum difference (95% confidence) between the predicted and the experimental absolute yields that we obtain through bootstrapping.
Answer: I believe the best test for your data is the multinomial test, which you already estimated.
For a "measure" of the difference between the expected and observed distributions, consider the Kullback-Leibler Divergence. For your first experiment:
require(entropy)
cv <- d[1:4,]
set.seed(42)
probs <- cv$freq_pred/20
cv$freq_exp <- table(sample(c(1:4), 10000, prob=probs, replace=T))
KL.empirical(cv$freq_exp, cv$freq_obs)

The KL Divergence is 0.072. If 0, the 2nd distribution is similar to the 1st distribution. If 1, the 2nd distribution is less related to the 1st distribution. More here for a finer interpretation based in information theory.

Comparing two multinomial distributions


How do I customise the header and footer using a page template for a particular content type?

Question: I have a content type.
I want to customise the node pages for this content type, I've used node--[content-type].tpl.php but I need to use a page template to customise the header and footer and page--[content-type].tpl.php doesn't work.
I've tried page--node.tpl.php, but this changes the home page as well, then i created page--node--[content-type].tpl.php which is not working.
Please help me.
Thanks
Comment: Should i use page--node.tpl.php but this will go for all content type.
Answer: You can have page templates based on a content type, you just have to add some code so Drupal will recognise the template file: This link shows how to do this.
It depends on your Drupal version. You would think this would be default but for some reason, probably performance related, it is not.
For D7 add this to the template.php file in your theme directory.
<?php
function themeName_preprocess_page(&$vars) {
  if (isset($vars['node']->type)) {
    // If the content type's machine name is "my_machine_name" the file
    // name will be "page--my-machine-name.tpl.php".
    $vars['theme_hook_suggestions'][] = 'page__' . $vars['node']->type;
  }
}
?>

End now you can customize your footer header etc.
Comment: And [this link](https://drupal.org/comment/8639299#comment-8639299) shows that the code you copy-pasted from is buggy.
Answer: This is explained on this page at Drupal.org.
Below is a more detailed description about how you do this in Drupal 7:

Open the template.php  file in your theme for editing.
Look for a function called THEME_preprocess_page (replace THEME with the theme's name).
If this function already exists, you will need to add your code to it inside an if-statement that goes at the end of this function, just before the closing bracket.
Otherwise, if it doesn't exist, you'll need to create a new function:

This is how the function should look like:
function THEME_preprocess_page(&$vars) {
  // If the function already exists, just leave the code here intact, and add
  // the three lines below to the end.
  if (isset($vars['node']->type)) {
    $vars['theme_hook_suggestions'][] = 'page__' . $vars['node']->type;
  }
}

NOTE: do not string replace any underscores inside the content type's machine name if you're using Drupal 7 (I think you had to do that in Drupal 6).
Now, copy the theme's page.tpl.php to a file named page--contenttype.tpl.php and place it inside the themes templates/folder.  For contenttype, use the machine name of the content type.  You can inspect the machine name if you navigate to Administration » Structure  » Content types.
Now, edit page--contenttype.tpl.php so that the header and footer is changed the way you want.  Do not change the parts you don't want to appear changed.
All nodes with content of that that type will use the new template file.
Provided you're using Drupal 7, observe the following conventions when naming the template override file.

Use two hyphens to separate the elements in the filename.
If you're overriding a content type, and your content type is two or more words, replace the underscore character (_) in the machine name with a single hyphen (-) in the file name.

For instance, the name of a template override for content type with a machine name my_machine_name should be named (in D7): page--my-machine-name.tpl.php.
Comment: This working but the changes also reflecting in content-type page.
Comment: yes, i did exactly the same which you have explained me.let me elaborate all the steps i have done. 1. i declared the function in the page.vars.php because that function is already declared in the file and i have just added this code, and then created a file named as page--[content-type].tpl.php(content type name is a one word and same as machine name),but the changes i have made in this file reflecting on drupal/content-type/nodes pages(that's what i need)as well the page created by views named as drupal/content-type which i don't want this.
Comment: the same i was thinking, because it worked fine in my dummy site, wel thanks for everything let me try changing the views path and let see, Thanks buddy..


How to get reverse DNS to work for multiple IPs and domains on Windows Server EC2 instance?

Question: I have a EC2 instance setup running hMailServer.  I have a primary IP on the system that I am using for example1.com email.  I would like to also add a second domain example2.com using a different IP address on the same server.
I have added a secondary IP address to the instance, setup elastic IPs for both private IPs and setup MX records and A records on Route 53 to point to the proper IP addresses.  I have also requested the pointers for reverse DNS from Amazon for each domain and IP.
I can send and receive email on both domains.
When I go to MXToolbox to check the SMTP settings for example2.com, it indicates that there is a reverse DNS mismatch and it shows example1.com as the name it is returning for the IP.
I assume this has something to do either with the way that Amazon maps the IPs or how they setup the reverse DNS entries.
How do I set it up so that I can get a reverse DNS match on both IPs / Domains?
Answer: You need to configure the reverse DNS (PTR) records for your elastic IPs by filling out this form.  Make sure your A records are correct or they will not create the PTR record.


Comparar Salário dentre as Arrays()

Question: Tenho a seguinte Query:
SELECT * FROM folhapagamento_storages WHERE idFolhaParametro = '1' AND valor_min >= '1083.4' AND valor_max <= '1083.4'

Só que o retorno sempre será 0, o que estaria fazendo de errado? Qual seria a lógica correta?
Neste caso, precisaria retornar apenas a faixa 1, como result, e vem zerado...
Answer: SELECT * FROM folhapagamento_storages WHERE idFolhaParametro = 1 AND valor_min >= 1083.4 AND valor_max <= 1083.4

<=
Tem que colocar <= (menor igual) na consulta do valor_max. E número decimal não precisa de aspas.
Comment: Dá mesma forma, o resultado vem zerado, nesta sua condição
Comment: Olhando melhor agora seus registros realmente não vai retornar nada essa sua consulta. Nenhum valor mínimo ou máximo adere a sua condição. Agora, se você colocar um `OR` no lugar do `AND` entre os valores vai retornar registros.
Answer: Você está consultando utilizando: valor_max = '1083.4'. O correto seria <= '1083.4'.
Query completa:
SELECT * FROM folhapagamento_storages WHERE idFolhaParametro = '1' AND valor_min >= '1083.4' AND valor_max <= '1083.4'

Comparar Salário dentre as Arrays()


Prime number proof

Question: Given that $n>2$. Prove that if $2^n-1$ is prime then $2^n+1$ is composite or vice versa.
I looked on wikipedia on Fermat number and Mersenne prime, but I still don't know how they work.
Answer: Hint $\ a\mid (a\!-\!1)^n\!\pm 1\ $ since $ $ mod $\ a\!:\ (a\!-\!1)^n \equiv (-1)^n\equiv \pm 1$
e.g. $\,\ 10\mid 9^n\pm1\, $ is a well-known case (power's of $9$ end with digit $1$ or $9)$
Your case can be viewed as the analogy in radix $\,3.$
Answer: Clearly you can't prove that $2^n-1$ composite implies $2^n+1$ is prime.  Take $n=6$, for example.  Going the other way, think about divisibility by $3$.
Comment: I'm not sure what the OP meant by "or vice versa".  I suspect they probably meant "and if $2^n+1$ is prime then $2^n-1$ is composite", in other words just the contrapositive of the first statement.
Comment: @David:  I took it as "if $2^n-1$ is prime then $2^n+1$ is composite", but I can see your reading as well.  I think mine would be the more standard English reading of vice versa.
Comment: Maybe I didn't understand the question right. It asks "Prove that if one of the number 2^n−1 and 2^n+1 is prime, where n>2, then the other is composite."
Comment: Ok, take a look at the pattern of remainders when you divide $2^n-1$ and $2^n+1$ by $3$.  Just make a table for small $n$.  See if you can find a pattern, then prove it.
Comment: Interesting. Thanks for the help. I'll take a look at the modular route.
Answer: At least one of $2^n-1$ and $2^n+1$ must be divisible by $3$, and neither of them is $3$. Thus, at least one of them is composite.
Comment: As simple and short as that. +1

Prime number proof


Intuition on t-SNE visualization technique

Question: I generated a t-SNE visualization of a dataset (about 10 standardized (mean = 0, sd = 1) numerical features) and I arrived at a following 2-dimensional plot. I don't have a good intuition why the points are aligning in sort of arc-shaped groups, see for example right bottom part of the enclosed picture (or its leftmost part).
I found a similar effect at the original paper http://www.cs.toronto.edu/~hinton/absps/tsne.pdf - see Figure 4.a. However, I didn't find a good explanation of this kind of a phenomenon.
Any intuitions on why this happens?
Comment: Obligatory cool approximate t-SNE animation: [link](https://www.kaggle.com/c/bnp-paribas-cardif-claims-management/forums/t/19940/t-sne-neatness/114259)
Comment: Firebug's link is broken.

Maybe check this tutorial out: http://distill.pub/2016/misread-tsne/
Comment: Kind of a random question, which could change with any given dataset.
Answer: The a similar phenomenon can be seen if you search for images of "spring graph layout", which show you many examples of such arcs, such as this one from wikipedia. Near the top on the right edge of the image, we see one such arc. Admittedly this isn't the best example. The top right corner of this image from this paper shows the effect a bit
Most of these graph visualizations are generated by simulating a spring force between each pair of connected nodes, and allowing the nodes to move according to this force.
In t-SNE, a similar interpretation of the algorithm is possible -- points in 2D space have a spring, whose resting length depends on the distance of the points in the original high-dimensional space. So points that are closer in 2D space than they are in high-dimensional space are pushed farther apart, and points that are farther in 2D space than in high-dimensional space are pulled together.
So it's likely that the arcs form because they are trying to maintain a constant distance to another group of points in the data.
Unlike in the spring graph layout above, every pair of points in t-SNE has a spring/force attached to it, so it's a valid question to ask why the arcs don't clump together into blobs, as shown in the graph visualization from wikipedia, where some groups of nodes on the edges have formed roundish clusters rather than arcs.
I suspect the reason for this is that each point in t-SNE has a variance attached to it. Points in a sparser region of the high-dimensional space have a higher variance compared to points in a lower dimensional space. The force on the springs of high-variance points is reduced, so if points in an arc were located in a sparse region of the original space, there would only be a weak force trying to pull them into a cluster, which might not overcome other opposing forces.
Furthermore, the method that the authors used to reduce crowding was to use a heavy-tailed distribution in the 2D space, which means points aren't penalized too heavily for being farther away from each other than they should. This also reduces the forces which would try to pull an arc into a cluster.


Calculating time difference of each occurance of a IP

Question: I have a Apache log file in the following format:
184.108.40.206 - [13/May/2013:23:55:04 +0530] "GET /SVRClientWeb/ActionController HTTP/1.1" 302 2 "https://www.example.com/sample" "Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B329" GET /SVRClientWeb/ActionController - HTTP/1.1 www.example.com

The log file is imported to a pandas data frame.
df = df.rename(columns={'%>s': 'Status', '%b':'Bytes Returned',
                        '%h':'IP', '%l':'Username', '%r': 'Request', '%t': 'Time', '%u': 'Userid', '%{Referer}i': 'Referer', '%{User-Agent}i': 'Agent'})

I want to get a certain ip and find the time difference of each hit from that IP. (E.g e.g 22.214.171.124 first occurs at 06.05.02 and again there is a hit at 06.10.03)
I have a kind of idea using following code, which I can not exactly find the way to do it. Help me with this problem.
selected_ip = df['IP'][df['IP'] == '22.214.171.124']
df.index = pd.to_datetime(df.pop('Time'))
df['tvalue'] = df.index
df['delta'] = (df['tvalue']-df['tvalue'].shift()).fillna(0)

   Status   Bytes   IP           Username   Request                               Time          Userid       Referer        Agent
0   200     974     184.108.40.206  -   GET /favicon.ico HTTP/1.1   06/Jun/2013 06:03:08 -0600  -   -   Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKi...
1   200     739     22.214.171.124  -   GET /js/themes/dark/next.gif HTTP/1.1   06/Jun/2013 06:03:09 -0600  -   http://www.gadgets.lk/full-detail-of-used-herc...   Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKi...
2   200     699     126.96.36.199  -   GET /js/themes/dark/prev.gif HTTP/1.1   06/Jun/2013 06:03:09 -0600  -   http://www.gadgets.lk/full-detail-of-used-herc...   Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKi...
3   200     770     188.8.131.52   -   GET /images/nav-hover.jpg HTTP/1.1  06/Jun/2013 06:03:19 -0600  -   http://www.gadgets.lk/used-brand-new-security-...   Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.3...
4   200     366     188.8.131.52   -   HEAD / HTTP/1.1     06/Jun/2013 06:03:29 -0600  -   -   Mozilla/5.0+(compatible; UptimeRobot/2.0; http...
5   200     36709   126.96.36.199  -   GET /js/jquery.validate.js HTTP/1.0     06/Jun/2013 06:03:40 -0600  -   -   Mozilla/4.0 (compatible; MSIE 6.0; Windows NT ...

Expected Output :  Given IP : 126.96.36.199
Times of Hit     Differece between occurances
06.05.02
06.10.00         00.04.58
07.30.00         00.30.00

Answer: Does your Time columns contain datetime info? If so,
df[df.IP == '220.127.116.11']['Time'].diff()

Edit
To retain the original time info:
df = df[df.IP == '126.96.36.199'][['Time']]
df['diff'] = df['time'].diff()

Comment: Thanks waitingkuo! I have deleted the time column with this code? So how to proceed? df.index = pd.to_datetime(df.pop('Time'))
Comment: Can you print your `df` for me?
Comment: what's your expected output?
Comment: I have put the expected output in the question, please explain me how to get this?
Comment: df['delta'] = (df['tvalue']-df['tvalue'].shift()).fillna(0) This shows the time difference for each hit, how to get it for one particular IP?
Answer: Following Code Worked!
    df['tvalue'] = df.index
    df['delta'] = (df['tvalue']-df['tvalue'].shift()).fillna(0)
    df[(df.IP == '220.127.116.11')][['delta']]


Derivative of a random variable w.r.t. a deterministic variable

Question: I'm reading about time series and I thought of this procedure: can you differentiate a function containing a random variable.
For example:
$f(t) = a t + b + \epsilon$
where $\epsilon \sim N(0,1)$. Then:
$df/dt = \lim\limits_{\delta t \to 0} {(f(t + \delta t) - f(t))/ \delta t} = (a \delta t + \epsilon_2 - \epsilon_1)/\delta t = a + (\epsilon_2 - \epsilon_1)/\delta t$
But:
$\epsilon_2 - \epsilon_1 = \xi$
where $\xi \sim N(0,2)$.
But this means that we have a random variable over an infinitesimally small value. so $\xi/\delta t$ will be infinite except for the cases when $\xi$ happens to be 0. Am I doing something wrong?
Comment: Different values ("realizations") of the random variable $\epsilon$ gives different functions. But when you derive  with respect to $t$, you are deriving one of those functions (for some $\epsilon$). Hence, it does not make sense to assume that $f(t+\delta t)$ and $f(t)$ have different $\epsilon_1$ and $\epsilon_2$
Comment: I think what is meant is that $f(t) = at + b + \epsilon(t)$ where  $\epsilon(t)$ is a stochastic process.  In that case, depending on the nature of $\epsilon(t)$ this may or may not be differentiable.
Comment: Robert, no I didn't mean that $\epsilon$ is a function of t.
Answer: A random variable is a function from sample space to the real line. Hence $f(t)$ really stands for $f(t,\omega) = a t + b + \epsilon(\omega)$. This function can be differentiated with respect to $t$, for fixed $\omega$, of course. The resulting derivative, being a function of $\omega$, is a random variable. In this case:
$$
   \frac{\partial f}{\partial t}(t, \omega) = a
$$
Since it does not depend on $\omega$, the derivative is deterministic, in this example.
Comment: Why exactly can't you also differentiate w.r.t. $\omega$?
Comment: Do you know of an example?
Comment: Because it may be non-differentiale, or even non continuous
Comment: From Sasha's explanation, $\omega$ is not an ordinary variable, it is a function that lives in function space, such as an infinite-dimensional Hibert space.


Exp. distributed r.v. as function on sample space

Question: Let $(\Omega,\mathcal{F},P)$ denote the probability space upon which the exponentially distributed random variable $X (\omega)$ with $\omega\in\Omega$ is defined. Furthermore, let $E$ denote the space into which $X$ maps. Is it true that $\Omega = \left(0,\infty\right)$? And $E=\Omega$? And $X (\omega)=\omega$?
Answer: We cannot say what $\Omega$ is. But we can say $E=(0,\infty)$ (though it can be any bigger set with $X$ taking values outside this interval with probability $0$). We also can not write down $X(\omega)$.
Having said that we can construct an exponential random variable on $\Omega =(0,\infty)$ with an appropriate sigma algebra and a probability measure so that $X((\omega)=\omega$ defines  a random variable with the given exponential distribution.


Tracking browsing pattern of website user

Question: I own website (typical registration and various links.Made in CAKEPHP, MYSQL).
I want to track every user browsing pattern without hurting speed of website.
I tried following DB.
<code>--------------------------------------------------------------------------
USER_ID      |     source_link       | destination_link   |  time_spent(second)
--------------------------------------------------------------------------
1              wwww.../link1       ww..///link2             300
2              link3               \link4                   160
.......
</code>
Then i store row in DB whenever user click to any URL.
Now am i in right direction.
If this is practical then how can i implement this.
If not then please suggest me better way to do that.
Comment: have you tried google analytics
Comment: That information should already be in your site's access_log. There is no reason to slap on a database layer which will only hurt performance. Use a log analyzer to build the click trail AFTER the fact.
Answer: Depends on how large your traffic is.
If you want to follow a pattern then the best way would be to use Google Analytics funnels.
If you want to process all traffic by yourself I would just process the access_log files using Hive or some dedicated tool.
I would not save the traffic directly in a database (it's not cheap to scale for high traffic). Maybe you can save the traffic in a csv file on disk and insert it in mysql using a cron and the LOAD DATA syntax which is much faster.
Comment: I already using that. but now i want to see the browsing pattern for specific login user not guest user.
Comment: Here is more important section of my website like which portion of my website is good for user.I also want to create a analysis front end on my own for this.


Which technique of integration should use to solve the question?

Question: Quote from the paper I read:
Given $F=(1-\lambda)f$ + $\lambda zf'$,
we find that
$f(z)= \tfrac{1}{\lambda} z^{1- \tfrac{1}{\lambda}}\int_0^zF(t)t^{\tfrac{1}{\lambda}-2}dt.$
My questions is Which technique of integration should use to get answer for $f(z)$?
Thanks for help.
Answer: $\textbf{Hint:}$
$$
F(z) = (1-\lambda)f + \lambda z f'
$$
re-write as
$$
f' + \frac{(1-\lambda)}{\lambda z}f = \frac{1}{\lambda z}F(z)\tag{*}
$$
using integration factor leads to your solution.
$\textbf{update}$
Eq.(*)  becomes
$$
\mathrm{I.F} = \mathrm{e}^{\int \frac{(1-\lambda)}{\lambda z}dz} = \mathrm{e}^{\frac{(1-\lambda)}{\lambda}\ln z} = \mathrm{e}^{\ln z^{\frac{1-\lambda}{\lambda}}} = z^{\frac{1-\lambda}{\lambda}}
$$
thus
$$
f(z)z^{\frac{1-\lambda}{\lambda}} = \int^{z}_{0} F(t)\frac{1}{\lambda t}t^{\frac{1-\lambda}{\lambda}}dt
$$
here I use a dummy variable t for the integration.
Comment: Which integration factor should used? Can show some steps?...Thanks
Comment: Sorry, i thought you said you only wanted the method? I will update  then
Comment: ok, really thanks a lot...
Comment: For the final equation (right hand side), $lambda$ should be in denominator, correct?
Comment: Oops yep sorry. Typo.
Comment: is ok, nevermind...Once again, thanks for the help...
Comment: Another one I find from the book...
F(z)=(1−λ+μ)f(z) + (λ−μ)zf′(z) + λμz^2f′′(z)
where λ≠0 and μ≠0. Also, z is a complex number.

I suspect it is Cauchy-Euler equation. How to find f? Can show some steps? When μ =0, it will reduce to F given in the question I asked yesterday.

Thanks for help.
Comment: @measure2012 put it in the form of another question and link this answer for some context. For sure let me know the question link, but I'm sure others will get there first ;)
Comment: this is the link http://math.stackexchange.com/questions/894639/how-to-find-it?noredirect=1#comment1847387_894639


How to remove subdir from base URL after installing site with Composer on a shared host?

Question: I've managed to install drupal 8 using composer and following script:
composer create-project drupal-composer/drupal-project:8.x-dev my_site_name_dir --stability dev --no-interaction
I decided to use composer since it seems to be the new way and you get all the dependencies with it.
But the structure is different then installing drupal by downloading it to the server. And that's my problem.
I'm using shared hosting and before composer everything landed in the folder public_html. But with composer there's a folder called web inside public_html that holds drupal itself (or at least it seems that way to me). So instead of having a url like www.mydomain.com I end up with www.mydomain.com/web to reach the site.
My question is how do I do to hide /web from the url? Or do I do something when installing via composer?
I've done a temporary solution that hides /web just from the frontpage. Had to create a new .htaccess in public_html which included following:
RewriteRule ^$ web/$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ /$1 [L]
But as soon as you click on a different page the /web is there again.
Any hints or of course answers are most welcome.
Thanks //Tobias
Comment: You could run the create in the web root itself, and then update the apache conf to point to /web/ instead of /public_html/ as the docroot. But I am guessing that with this level of hosting, you won't have that access.
Comment: Thanks @Kevin for your thougts and hints. But as you say I'm not sure I have that access to apache but I'm looking more thoroughly in cpanel to try to find the right way there. What do you think of changing the thought of using redirect instead of rewrite? Is that another way to do this?
Once again, thanks for your comment :)
Answer: Me and my collegue managed to find a working solution.
In our public_html folder we created a .htacess file with the following lines.
RewriteRule ^$ web/$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ web/$1 [L]

We edited the .htaccess file in ~user/public_html/web.
# Modify the RewriteBase if you are using Drupal in a subdirectory or in a
# VirtualDocumentRoot and the rewrite rules are not working properly.
# For example if your site is at http://example.com/drupal uncomment and
# modify the following line:
RewriteBase /web

In the ~/user/public_html/web/sites/default/settings.php file, we added this code at the bottom.
if (isset($GLOBALS['request']) and
'/web/index.php' === $GLOBALS['request']->server->get('SCRIPT_NAME')) {
$GLOBALS['request']->server->set('SCRIPT_NAME', '/index.php');
}

We also changed trusted_host_patterns setting as well.
# will allow the site to run off of all variants of example.com and
# example.org, with all the sub-domains included.

$settings['trusted_host_patterns'] = array(
  '^DOMAIN.com$',
  '^.+.DOMAIN.com$',
  '^DOMAIN.com/web$',
  '^.+.DOMAIN.com/web$',
);

As I said, this is most likely not the correct way to do it, but it worked for us.
Answer: We're currently in a similar boat. We have dedicated hosting, and we're looking to start development of Drupal 8 sites with composers as a way to manage sites.
We took a different approach, we created a directory and adding therein the core, modules, and theme files. Then made a symlink between those files and the public_html folder. It seems to work fine, but, like you, I am not sure what the best practices.
FYI: Here's how we created the symlink:

mv public_html examplesite
ln -s public_html examplesite/webfiles

That seemed to make it work for us.
Edit: I just ran across an article immediately after writing this. It says moving the files out of docroot adds additional security to your site. So, what we did, with the symlink, might be considered the better option.
https://www.drupal.org/node/2767907
Comment: Thanks! Exactly what I've been looking for. I agree with your thought that as the article says it's adds additional security, your way seems the better option.

Will dig into this and try to understand what it really does and how I should do to not screw things up :)

Again, Thanks @UnqtiousDude :D
Answer: Suppose you're in mysite.com/public and you want the index.php or drupal root should come directly under mysite.com/public then do this:
Then execute:
create-project drupal-composer/drupal-project:8.x-dev my_site_name_dir --stability dev --no-interaction --no-install

now go to my_site_name_dir and move everything including * and .* to under mysite.com
Then open   composer.json under mysite.com and then replace "web" occurrences to "public" in installer-paths.
Now run composer install within mysite.com. It will work.
Comment: Thanks @user5858!!
A great way since I didn't manage to use symlink. Will try this ASAP!
Thanks again! :D
Answer: Everywhere I check the answer to this question of the 'web/' folder is always to reconfigure server and write rewrites, but I have a very simple way I solve this issue.
During initiating a new Drupal project, I would make what is called a 'modified install'. I would stop the project initiation half way to tell Composer what my web-root is. By default Composer assumes my web-root to be 'web/' but making a modified install allows me to tell Composer where it is. If on shared hosting it would be 'public_html' and could be called anything in some other situation. So when I pause the installation process, I would tell Composer, 'hey don't go ahead to install Drupal in 'web', install it 'here' instead' and 'here' could even be a nested folder several steps away from pwd. When I tell Composer this, I then resume the installation process and finally Drupal would be installed wherever I told Composer.
I wrote the steps to do this in a blog post, How to install Drupal 9 with Composer on Cpanel. This post is about installing on a cPanel account but the process demonstrates making a modified install, and same principle can be used to install locally or in any folder structure.


Panels content wrapper

Question: I realised that this was a horribly verbose question, so decided to edit it and try again!
I am trying to find a way of wrapping a set of three (or more) Panels content areas with a div - is that possible?
Comment: I question the usefulness of this. Why would you not add a region instead?
Answer: You can definitely use the Panels Extra Styles module to wrap multiple panes in as many tags as you need.
On the first pane of your new group click the option cog and, under the Style section (which should read No style), choose 'Change'. Change the selected style to Wrapper:Raw and enter your opening tags in the text area 'Content prefix'. Leave the suffix area blank (as you don't want to close any tags yet)
Save the changes and then do the same with the last pane you want to group. This time close all the tags opened in the first pane by entering them in the 'Content suffix' text area.
Save and update your panel page and et voila! Your panes are grouped with your own HTML!
Comment: I just tried this and it works great, thanks for the tip!
Answer: Try Panels Extra Styles module. You can wrap panes or panes in HTML
Comment: the panels extra styles doesnt work to wrap panes.
Comment: I havent tried it yet, but it looks to be exactly what I needed, thanks!
Answer: There is a way without additional modules using custom Panels content.

Add a custom content pane where you want to start the wrapper, add the opening tag as body, make sure to not use WYSIWYG Editors which may add additional markup and don't add a title.
Set style to: No markup at all
Add another custom content pane analogous to the first one where you want to end the wrapper with the closing tag.

Thath's it, after saving the panel content you will see the grouping produced by the wrapper, as shown in the attached screenshot.

Panels content wrapper


jakarta.servlet.ServletException: At least one resource class should be specified at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet

Question: Trying to upgrade to corretto-Java17, Tomcat 10.1.16/10.1.17 . I have moved all javax package to jakarta package, upgraded  spring to version6 and cxf to version-4.When I deployed my application in tomcat, I see below error as:
jakarta.servlet.ServletException: At least one resource class should be specified
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.getServiceClasses(CXFNonSpringJaxrsServlet.java:304)
at org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet.init(CXFNonSpringJaxrsServlet.java:153)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:944)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:901)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:805)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4601)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4894)*
web.xml
<code> <?xml version="1.0" encoding="UTF-8"?>
 <web-app  xmlns="https://jakarta.ee/xml/ns/jakartaee"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

 xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
 https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd"
                   version="5.0">

     <context-param>
             <param-name>contextConfigLocation</param-name>
             <param-value>classpath*:papi-
    plugin.xml,classpath*:papi-
    server.xml,classpath*:applicationContext.xml</param-value>
         </context-param>
     <listener>
  <listener-class>
   org.springframework.web.context.ContextLoaderListener
  </listener- class>
         </listener>

         <servlet>
             <servlet-name>CXFServlet</servlet-name>
             <servlet-class>
 org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet
 </servlet- class>
             <init-param>
                 <param-name>
 jakarta.ws.rs.core.Application</param-name>
                 <param-value>
 com.resource.ProvisionCxfApp</param-value>
             </init-param>
             <load-on-startup>1</load-on-startup>
         </servlet>

         <servlet-mapping>
             <servlet-name>CXFServlet</servlet-name>
             <url-pattern>/prv/v1/*</url-pattern>
         </servlet-mapping>

     <servlet>
             <servlet-name>abc</servlet-name>
             <servlet-class>
 org.springframework.web.servlet.DispatcherServlet</servlet-
 class>
             <load-on-startup>1</load-on-startup>
         </servlet>

         <servlet-mapping>
             <servlet-name>abc</servlet-name>
             <url-pattern>/servlet/*</url-pattern>
         </servlet-mapping>

 </web-app>
</code>
My ProvisionCxfApp:
<code>     import jakarta.servlet.ServletException;
     import jakarta.ws.rs.core.Application;
     public class ProvisionCxfApp extends Application {

      private final Set<Object> singletons = new HashSet<Object>();
        /**
         * @return
         * @see jakarta.ws.rs.core.Application#getClasses()
         */
        @Override
        public Set<Class<?>> getClasses() {
            Set<Class<?>> set = new HashSet<Class<?>>();
            set.add(AResource.class);
            set.add(AcsResource.class);
            return set;
        }

        public ProvisionCxfApp() throws ServletException{
            singletons.add(AResource.class);
            singletons.add(AcsResource.class);
      }}
</code>
<code>Also have classes like :
</code>
<code>  @Path("/ac/{id}")
  public class AResource extends BResource {}

  @Path("/acs")
  public class AcsResource extends BResource {}
</code>
Answer: changed this to : jakarta.ws.rs.Application. Since CXFNonSpringJaxrsServlet - jar using "jakarta.ws.rs.Application" as its param. So issue got resolved !!!


Displaying json data in angularjs

Question: I would like to display data returned from service call into view:
Service Code :
.service('HomeExchangeList', function ($rootScope, $http, $log) {
    this.getHomeExchange = function() {
        var rates = $http({
            method: 'GET',
            url: 'http://localhost:8080/feeds/homerates_android.php'
        }).success(function (data) {
            $log.log(data);
            return data;

        });

        return homeRates;
    };
})

JSON Data returned by service
            {
               "record":[
                  {
                     "Name":"GBP\/USD",
                     "Ticker":"GBP\/USD",
                     "Price":"0.5828",
                     "Open":"0.5835",
                     "High":"0.5848",
                     "Low":"0.5828",
                     "PercentagePriceChange":"0.1371",
                     "Movement":"0.0800",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"EUR\/USD",
                     "Ticker":"EUR\/USD",
                     "Price":"0.7330",
                     "Open":"0.7344",
                     "High":"0.7351",
                     "Low":"0.7327",
                     "PercentagePriceChange":"0.2585",
                     "Movement":"0.1900",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"GHS\/USD",
                     "Ticker":"GHS\/USD",
                     "Price":"3.3350",
                     "Open":"3.2650",
                     "High":"3.3500",
                     "Low":"3.2650",
                     "PercentagePriceChange":"0.8915",
                     "Movement":"3.0000",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"KES\/USD",
                     "Ticker":"KES\/USD",
                     "Price":"87.7000",
                     "Open":"86.2970",
                     "High":"87.6500",
                     "Low":"86.1800",
                     "PercentagePriceChange":"0.0661",
                     "Movement":"5.8000",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"MUR\/USD",
                     "Ticker":"MUR\/USD",
                     "Price":"30.2925",
                     "Open":"29.1460",
                     "High":"29.4300",
                     "Low":"29.0500",
                     "PercentagePriceChange":"-0.0909",
                     "Movement":"-2.7500",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"MWK\/USD",
                     "Ticker":"MWK\/USD",
                     "Price":"393.5000",
                     "Open":"393.3900",
                     "High":"393.3900",
                     "Low":"385.0000",
                     "PercentagePriceChange":"-0.2548",
                     "Movement":"-100.0000",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"NGN\/USD",
                     "Ticker":"NGN\/USD",
                     "Price":"162.3000",
                     "Open":"160.0600",
                     "High":"162.4000",
                     "Low":"160.0600",
                     "PercentagePriceChange":"0.2459",
                     "Movement":"40.0000",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"ZAR\/USD",
                     "Ticker":"ZAR\/USD",
                     "Price":"10.6659",
                     "Open":"10.6751",
                     "High":"10.7162",
                     "Low":"10.6523",
                     "PercentagePriceChange":"0.9840",
                     "Movement":"10.6000",
                     "DateStamp":"2014\/07\/09",
                     "TimeStamp":"22:15:00"
                  },
                  {
                     "Name":"ZMK\/USD",
                     "Ticker":"ZMK\/USD",
                     "Price":"47.7014",
                     "Open":"47.3850",
                     "High":"47.7000",
                     "Low":"46.8900",
                     "PercentagePriceChange":"0.0067",
                     "Movement":"0.3165",
                     "DateStamp":"2013\/07\/27",
                     "TimeStamp":"01:55:00"
                  }
               ]
            }

Controller code
function HomeCtrl($scope, Page, $location, HomeExchangeList) {

    $scope.rates = HomeExchangeList.getHomeExchange();
    $scope.$on('HomeExchangeList', function (event, data) {
        $scope.exchangeRates = data;
    });
}

View

{{rate.Name}}{{rate.Price}}

I would like to display the data returned by in the service in the view but it doesn't seem to be working. Please help
Answer: try to assign data.record to $scope.exchangeRates instead of data... as data doesnt hold the array of records... it holds record which then holds the array
Answer: First of all, your service function always returns undefined:
var rates = ...,
return homeRates;

It should be
return rates;

Second, once that is fixed, the service doesn't return data. It returns a promise, and you can't iterate on a promise. What you need in the controller is:
HomeExchangeList.getHomeExchange().then(function(data) {
    $scope.rates = data.record;
}

The call to $scope.$on doesn't make any sense. $scope.$on is used to listen for events. Not to get data from a promise.
And finally, your view must iterate over these retes, and not over exchangeRates:
ng-repeat="rate in rates">

Answer: First, $http invocations all return a promise, not the result of your request. Your service should just return the result of the $http call, and your controller needs to attach a .success handler to receive the data and set it on the scope of your controller.
.service('HomeExchangeList', function ($rootScope, $http, $log) {
    this.getHomeExchange = function() {
        var rates = $http({
            method: 'GET',
            url: 'http://localhost:8080/feeds/homerates_android.php'
        }).success(function (data) {
            $log.log(data);
            // removed your return data; it doesn't do anything, and this success is only added to log the result. if you don't need the log other than for debugging, get rid of this success handler too.
        });

        return rates;
    };
})

function HomeCtrl($scope, Page, $location, HomeExchangeList) {
    HomeExchangeList.getHomeExchange().success(function(data) {
        $scope.exchangeRates = data;
    });
}

Second, the root of your JSON is not an array, so you can't enumerate through just exchangeRates alone. Perhaps you meant exchangeRates.record.
Comment: Thank.. this is working to a certain extent but the resulting exchangeRates.record used in ng-repeat="rate in exchageRates.record" is only displaying one record. Am i mission something??
Comment: @munya.nyika See the demo I put together here http://plnkr.co/edit/oJPLZUpzYG3NEwsf1G9M?p=preview and compare my service and controller declarations in scripts.js to yours, as well as the HTML in my index to yours. It works as I described once you change your repeater to use `exchangeRates.record` (alternately, you could just assign `$scope.exchangeRates = demo.record;` in your controller).

Displaying json data in angularjs