Export version history event log of SharePoint List

Question: I'm looking for an event log or full version history which holds all changes ever made to the items of a SharePoint List.
Version history is enabled. When I right-mouse click on an item and select Version History, it actually shows me all the data I need:

Timestamp of modification
(Column, value) pair of the new modified column and it's new value
Modified by who

Now I "only" need to get this data structured in a table or a file (xlsx, xml, json, csv). And for all the List's items.
I can see that the data is there. But I haven't found a way to export it. So far I've tried customizing an .iqy file, using PowerShell and using Power Automate, without success.
I hope there is some way for me to get the full history. I need it to calculate average run times.
Answer: For SharePoint Online:
<code>#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"

Function Export-VersionHistory()
{
  param
    (
        [Parameter(Mandatory=$true)] [string] $SiteURL,
        [Parameter(Mandatory=$true)] [string] $ListName,
        [Parameter(Mandatory=$true)] [string] $CSVFile
    )
    Try {

        #Delete the Output report file if exists
        if (Test-Path $CSVFile) { Remove-Item $CSVFile }

        #Get Credentials to connect
        $Cred= Get-Credential
        $Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)

        #Setup the context
        $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
        $Ctx.Credentials = $Credentials

        #Get the List
        $List = $Ctx.Web.Lists.GetByTitle($ListName)
        $Ctx.Load($List)
        $Ctx.ExecuteQuery()

        #Get all items
        $Query = [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery()
        $ListItems = $List.GetItems($Query)
        $Ctx.Load($ListItems)
        $Ctx.ExecuteQuery()

        #Array to hold result
        $VersionHistoryData = @()

        #Iterate throgh each item
        Foreach ($Item in $ListItems)
        {
            write-host "Processing Item:" $item.id -f Yellow

            #Get all versions of the list item
            $Versions = $Item.versions
            $ctx.Load($Versions)
            $Ctx.ExecuteQuery()

            If($Versions.count -gt 0)
            {
                #Iterate each version
                Foreach($Version in $Versions)
                {
                    #Get the Creator object of the version
                    $CreatedBy =  $Version.createdby
                    $Ctx.Load($CreatedBy)
                    $Ctx.ExecuteQuery()

                    #Send Data to object array
                    $VersionHistoryData += New-Object PSObject -Property @{
                    'Item ID' = $Item.ID
                    'Title' =  $Version.FieldValues["Title"]
                    'Version Label' = $Version.VersionLabel
                    'Version ID' = ($Version.VersionId/512)
                    'Created On' = (Get-Date ($Version.Created) -Format "yyyy-MM-dd/HH:mm:ss")
                    'Created By' = $CreatedBy.Email
                    }
                }
            }
        }

        #Export the data to CSV
        $VersionHistoryData | Export-Csv $CSVFile -Append -NoTypeInformation

        write-host -f Green "Version History Exported Successfully to:" $CSVFile
     }
    Catch {
        write-host -f Red "Error Exporting version History to CSV!" $_.Exception.Message
    }
}

#Set parameter values
$SiteURL="https://tenant.sharepoint.com"
$ListName="list name"
$CSVFile="C:\VersionHistory.csv"

#Call the function to generate version History Report
Export-VersionHistory -SiteURL $SiteURL -ListName $ListName -CSVFile $CSVFile
</code>
For SharePoint Server:
<code># ******* Variables Section ******************
#Define these variables
$WebURL="site collection URL"
$ListName ="list name"
$ReportFile = "C:\VersionHistory.csv"
# *********************************************

#delete the file if exists
If (Test-Path $ReportFile)
 {
 Remove-Item $ReportFile
 }

#Get the Web and List
$Web = Get-SPWeb $WebURL
$List = $web.Lists.TryGetList($ListName)

 #Check if list exists
 if($List -ne $null)
 {
  #Get all list items
  $ItemsColl = $List.Items

  #Write Report Header
  Add-Content -Path $ReportFile -Value "Item ID, Version Lable, Created by, Created at, Title"

  #Loop through each item
  foreach ($item in $ItemsColl)
  {
      #Iterate each version
      ForEach($version in $item.Versions)
      {
         #Get the version content
         $VersionData = "$($item.id), $($version.VersionLabel), $($version.CreatedBy.User.DisplayName), $($version.Created), $($version['Title'])"
         #Write to report
         Add-Content -Path $ReportFile -Value $VersionData
       }
    }
 }
Write-Host "Version history has been exported successfully!"
</code>


SwiftUI - DragGesture and .onDelete conflict

Question: I have a conflict between gestures:
<code>import SwiftUI

struct ContentView : View {

@State var showSearchBar = true

var body : some View {
            NavigationView {
                 Form {
                      ForEach(//code) { values in
                      NavigationLink(//code)
                      }
                      .onDelete(perform: //delete action)
                      }
                    .simultaneousGesture(
                        DragGesture()
                            .onChanged({ gesture in
                                if gesture.location.y > gesture.startLocation.y {
                                    showSearchBar = false
                                } else {
                                    showSearchBar = true
                                }
                            })
                    )}}}
</code>
However, both gestures don't work. If I remove the .simultaneousGesture, the .onDelete works, but if I leave both, only the .simultaneousGesture works. How can I make both work?
Thanks to everyone who will help me!
Answer: Suggest you raise this with Apple developer support.


zurb foundation error messages displayed by default?

Question: I am working with foundation.abide.js in a rails 4 application, the basic code setup included below.  No matter what I try or what I google for I can not seem to determine why the default form always shows the red error messages.  Any ideas on what I am missing?

Abide

Abide is an HTML5 form validation library that supports the native API by using patterns and required attributes.

    <form class="custom" data-abide>
      <fieldset>
        <legend>Fieldset</legend>

            Password required
            <input type="password" id="password" placeholder="LittleW0men." name="password" required>
            Passwords must be at least 8 characters with 1 capital letter, 1 number, and one special character.

            Phone
            <input type="tel" id="phone" placeholder="1 234-567-8910">
            Please enter a properly formatted telephone number.

            Email
            <input type="email" id="email" placeholder="firstname.lastname@example.com">
            Valid email required.

            URL required
            <input type="url" id="url" placeholder="http://zurb.com" required>
            Valid URL required.

              Hardest to find in grocery required
              <select id="customDropdown1" class="medium" required>
                <option value="">Select grocery item</option>
                <option value="first">Green Chilies</option>
                <option value="second">Raisins</option>
                <option value="third">Panko bread crumbs</option>
                <option value="fourth">Assistance</option>
              </select>
              Broke.

            <input name="radio1" type="radio" id="radio1" required> Radio Button 1
            <input name="radio1" type="radio" id="radio1" required> Radio Button 2
            <input name="radio1" type="radio" id="radio1" required> Radio Button 3

            <input type="checkbox" id="checkbox1" style="display: none;" required> Label for Checkbox
            <input type="checkbox" id="checkbox2" CHECKED style="display: none;" required> Label for Checkbox
            <input type="checkbox" CHECKED id="checkbox3" style="display: none;" required> Label for Checkbox

            Closing Remarks
            <textarea id="remarks" name="remarks" placeholder="Leave your remarks here."></textarea>

            <button type="submit" class="medium button green">Submit</button>

      </fieldset>
    </form>

Comment: Is it data-abide in your <form opening tag?
Answer: It is difficult to find the problem with only the HTML snippet (which looks ok, so the problem has to be somewhere else).
Did you make sure that:

You included ALL the CSS from the zurb foundation framework and DID NOT accidentally overwrite some of the CSS properties (check with the DIV inspector that the 'error' class is indeed hidden by default)
Make sure that you do not have a JS snippet that is trying to send/verify the form upon page load

If you want a more detailed analysis, publish the whole thing somewhere so that we can see the complete setup that you have there (HTML + CSS + JS)
Comment: I did not have it setup correctly, you were correct. Thank you.


How does a max-pooled layer connect to the second Convolution layer?

Question: Let's say I have 100 32x32x3 images.
I have an input layer which would would have one 32x32x3 array as the input.
 I connect to a convolution layer with 16 filters, which gets convoled and then
  regularized and max pooled, resulting in 16 activation maps.
Let's say I connect to another convolution layer, convo 2. If I have 8 filters in conv2, would each filter connect with all 16 activation maps from the previous layer?  That would result in 16x 8 activation maps in the next layer.
If not is there a way to do so in Keras?
Answer: The depth of your filter is generally going to be the number of activation maps from the previous layer. So your 8 filters in conv2 will be NxNx16 where N is the size of the filter you want to use.
The 2nd layer will produce 8 activation maps, one that corresponds to each filter. The number of activation maps is equal to the number of filters in that layer.
The first layer's filters have a depth of 3 because you can think of the original input image being 3 different activation maps (although they aren't actually "activations") - one for red, green, and blue. You could potentially have some 4th input channel, brightness at each pixel for example, and then your first layer filters would have a depth of 4.
Comment: I'm confused. Wouldn't the depth of each filter be only 3 dimensions?
I'm more interested in how many activation maps the conv2 would create if I used 8 filters.  Thanks.
Comment: I added some more information to my answer. I know most diagrams you can find don't really explain this part so well. Hope that helps
Comment: I asked on quora as well and this is the response I got from someone who had a PHD in computer vision:

'Its because each convolutional filter is also three dimensional.  In this case every filter is of size 32x32x3."

I asked him to expand on this, but I'm assuming that each of the 12 activation maps end of having a depth of 3?
Comment: The filters are 3 dimensional, but the activation maps produced by them are not. Each individual map is 2D. After these 2D maps are produced you can think of them all stacked on top of each other as a 3D volume, and then your 3D filters in the next layer convolve with your entire set of maps, and each filter produces one new 2D activation map. It wouldn't make any sense for your filters to all have a depth of 3 (except the first layer) because then either your activation maps would have to be 3D or you would have to assign certain filters to certain maps. neither is the case though
Comment: Actually there is a paper where they had filters with a depth that was less than the number of activation maps at the given layer and they chose certain maps to be convolved with certain filters. This is by no means the standard though
Comment: So each filter will only multiply with one of the RGB planes?  If we are in the first layer,
Having 6 filters would result in 2 filters x R plane, the other two  filters x G plane, 2x B plane?   Am I getting this right? This is really confusing. I
Comment: Each filter in the first layer will multiply with all three channels in the input image. What you might be overlooking is that they are summed into one value, and that is the value you put for that pixel in the resulting activation map. Think of both your input and your filter as a 3D volume. Say the filter is of dimension (w, h, d). You multiply this with a chunk of your input of the same dimension (w, h, d) and get a new volume of values (w, h, d). now you sum across all w*h*d values to get one single value, put that through your activation func, and this is your activation value for yourmap
Comment: And I know it is confusing. I asked nearly the same question on here almost 2 years ago. There isn't much out there that explains the finer details
Comment: Thanks, I was missing the summed into one value part.. So they are essentially taking the dot product it seems.  I understand now.  I actually was able to find explanation here (just now)

'http://xrds.acm.org/blog/2016/06/convolutional-neural-networks-cnns-illustrated-explanation/'

Thanks for you patience. I asked this question on quora as well, and everyone was giving me vague answers. They could have just said they are taking the dot product or what you said "What you might be overlooking is that they are summed into one value."


Exibindo Bitmap em aplicação C#

Question: Tenho o seguinte codigo na minha aplicação :
Bitmap original = new Bitmap("C:/Projeto/Imagens/antialiasing.jpg");
Graphics g = Graphics.FromImage(original);
g.DrawImage(original, 100, 100, 390, 390);

Porém não está sendo exibido na tela a imagem original. O que devo fazer para exibir.
Answer: Raphael, esta é uma aplicação winform? voce está mostrando esta imagem em um PictureBox?
Se for o caso, você pode colocar a imagem diretamente o pictureBox.
Bitmap original = new Bitmap("C:/Projeto/Imagens/antialiasing.jpg");
pictureBox1.Image = original;

Bom, não sei se era esta a sua duvida.
Comment: Isso mesmo que era minha duvida.

Exibindo Bitmap em aplicação C#


References for injectivity/surjectivity $\pi_i(U(n-k)) \rightarrow \pi_i(U(n))$

Question: I have read (Theorem 29.3, line 7, pg 144) the following statement:

$\pi_i(U(n-k)) \rightarrow \pi_i(U(n))$ is surjective for $i \le 2(n-k)+1$ and injective for $i \le 2(n-k)$.

and

Therefore $\pi_i(St_k(U(n))=0$ for $i \le 2(n-k)$.

What results are used here and where can I find a reference of this?
Comment: This follows from my answer to your question https://math.stackexchange.com/questions/3095144/extending-a-monomorphism-of-bundles-lemma-7-3-atiyah-shapiro/3096116#3096116, since $Gl(\mathbb{C}^n)$ deformation retracts onto $U_n$ by polar decomposition of matrices.
Comment: Oh my... I have completely forgotten to look back at the question! Thank you so much.
Answer: I think they have made an off-by-$1$ error, they might have been thinking about the map $BU(n-k) \to BU(n)$. (The notes you're reading are unpublished and there is a disclaimer at the start about not being proofread, so an off-by-$1$ error isn't so bad.)
A great reference for a lot of results like this is Mimura and Toda's "Topology of Lie Groups " I and II. They compute a lot of homotopy and homology groups for classical Lie groups and their classifying spaces. For this particular example, look at Corollary 3.17 on page 68. The statement includes:

... For $i < 2n$, $\pi_i U(n) \cong \pi_i U(n+1)$, ...

Changing variables we see $\pi_iU(n-1)\cong \pi_i U(n)$ for $i < 2(n-1)$, and their argument actually also shows surjectivity in degree $2(n-1)$.
Typically you use the long exact sequence of the fibration
$$ U(n - 1) \to U(n) \to S^{2n - 1}$$
to compute the connectivity of $U(n - 1)\to U(n)$. The degree $i = 2n - 2$ is the last degree where $\pi_i S^{2n-1} = 0$ so our map induces surjective homomorphisms in all degrees $i \leq 2(n-1)$, and moreover they are injective if $i < 2(n-1)$. The (correct) connectivity of your map, and hence the Stiefel manifold, follows.


Create shortcut on android with Ionic

Question: I'm developing an app and want to add a feature, create an shortcut on Home screen with user image getting from web
is there any way to make it with Ionic Framework.
I know there is a way to work around with native code but I'm looking for the idea for ionic.
Answer: Confirmed that when you upload the apk on google's playstore that's taken care of.
Make sure it's not in debug mode, otherwise you won't be able to upload it
Comment: thanks, this function already solved by myself, making icon from web images

Create shortcut on android with Ionic


$L^{2}(\mathbb R)$ and $L^{\infty}(\mathbb R)$-norm

Question: If both the $L^{2}(\mathbb R)$ norm and $L^{\infty}(\mathbb R)$ norm of a function $f$ are finite, is there any relation between the two norms in this case?
I know that there is a relation in case of a set of finite measure $S$, (i.e., $L^{2}(S)$ norm and $L^{\infty}(S)$), but what about the $\mathbb R$ case? If in general there is no relation, when we could have it?
Comment: Since your comment indicates that David's answer doesn't settle it for you, could you please make your question more precise?  What do you mean by relation?
Comment: Finite measure gives you $\|f\|_2\leq C\|f\|_\infty$ (for some constant $C$ independent of $f$).  If there is a positive lower bound on the measures of sets of positive measure, then $\|f\|_\infty\leq C\|f\|_2$ (for some constant $C$ independent of $f$).  For $\mathbb R$ you have neither of these, as David's answer shows.
Answer: No. Consider characteristic functions on sets of the form $[-n,n,]$. These have infinity norm of one but large $L_2$ norm.  On the other hand, consider functions of the form $f=n^{1/4}\chi_{[0,1/n]}$. These have small $L_2$ norm but large infinity norm.
Comment: Okay, I didn't think of examples of sequences, but if I allow sequences then I should assume that the limit of $\infty$ norms and the limit of the $L^2$ norms are both exists and finite. So, do we still have no relation?


How to have dynamic virtual hosts for all domains except localhost

Question: A bit of background: I've been wanting to set up dynamic virtual hosts for a while now. Basically I just wanted to be able to plop a folder into my virtual hosts folder and just have it work without any other configuration. I found out that in chrome any subdomain of .localhost will behave the same as localhost. This means that I can use .localhost as a TLD for all my projects and I don't have to edit my HOSTS file for every new virtual host I want to add.
I read the documentation on https://httpd.apache.org/docs/current/vhosts/mass.html and figured out how to have dynamic virtual hosts based on the host header.
After reading through that page and other resources on the internet, I came up with the following configuration in my httpd-vhosts.conf file. This uses the part before the .localhost to determine the folder name.
<VirtualHost *:80>
    ServerAdmin admin@localhost

    # Get the server name from the Host header
    UseCanonicalName Off

    # Log
    LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
    CustomLog logs/vhost_access.log vcommon
    ErrorLog logs/vhost_error.log

    # Match domain name against a folder
    VirtualDocumentRoot "C:/vhosts/%-2+"

    <Directory "C:/vhosts/*">
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        Allow from all
        Require all granted
    </Directory>
</VirtualHost>

So with this set-up, I can create a new folder in the vhosts folder that contains all my html documents. So, for instance, if I create a folder named project1 I can access that folder by going to http://project1.localhost.
Now my main problem is that I don't always need to create a new virtual host. I just want to create a random php file and access it by going to http://localhost/index.php. However, with the above configuration, just using http://localhost caused an error. Probably because the pattern in the VirtualDocumentRoot directive is using the subdomain of localhost and there is no subdomain when I just use http://localhost.
tl;dr:
Given the configuration above, is there any way to have a hard-coded virtual host for localhost and have dynamic virtual hosts for subdomains of localhost?
Or, how can I create a configuration that would allow me to have this:
http://localhost -----------------> C:/vhosts/
http://project1.localhost --------> C:/vhosts/project1
http://project2.localhost --------> C:/vhosts/project2
http://blog.project2.localhost ---> C:/vhosts/project2/blog

Answer: I have a solution but it just solve part of your problems.
<VirtualHost *:80>
   ServerAdmin admin@localhost
   ServerName  localhost
   DocumentRoot "C:/vhosts/"
</VirtualHost>
<VirtualHost *:80>
   ServerAdmin admin@localhost
   ServerName  longnameyoullneveruse.blabla.localhost
   ServerAlias  *.localhost
   DocumentRoot "C:/vhosts/"
   RewriteEngine On
   RewriteMap lowercase int:tolower
   RewriteCond %{HTTP_HOST} ^(.*)\.localhost$
   RewriteRule ^(.*)$ "C:/vhosts/${lowercase:%1}/$1"
   # you should use all lowercase for subfolders' name
</VirtualHost>

This should meets the first three lines of your requirement:
http://localhost -----------------> C:/vhosts/
http://project1.localhost --------> C:/vhosts/project1
http://project2.localhost --------> C:/vhosts/project2

Comment: Thank you very much. I just needed to add the extra virtual host item for `http://localhost`.


grok regex in logstash to parse and extract field

Question: I am trying to extract certain fields from a single message field. I am trying to achieve this by grok regex on the logstash so that i could view them in kibana.
My log events is as below:
[2021-01-06 12:10:40] ApiLogger.INFO: API log data:  {"endpoint":"/rest/thre_en/V1/temp-carts/13cEIQqUb6cUfxB/tryer-inform","http_method":"GET","payload":[],"user_id":0,"user_type":4,"http_response_code":200,"response":"{\"pay_methods\":[{\"code\":\"frane\",\"title\":\"R2 Partial redeem\"}],\"totals\":{\"grand_total\":0,\"base_grand_total\":0}}
The entire log has more information into different key value store-
Basically, I needed these information -

time stamp (i am able to get this)
log level (I am able to get this) => on loglevel, i just want the info not the entire Api.INFO
endpoint
http-method
user_id
user_type
http_response_code
response

I am not able to get the information from 3-8 ... i tested it. it is due to the semi colon(:)
this is what i tried through grok debugger
%{SYSLOG5424SD:logtime} %{JAVACLASS:loglevel}: (?<API>\w+ \w+ \w+):
i tried uri and other but it did not work, may be due to the colon.
Comment: I could only test using your sample input. If there are more fields in the payload try adding more `.*?` in between, `%{SYSLOG5424SD:logtime} %{JAVACLASS:loglevel}: (?\w+ \w+ \w+):\s*\{"endpoint":"(?[^"]*)","http_method":"(?[A-Z]++).*?"user_id":(?[0-9]++).*?"user_type":(?[0-9]++).*?"http_response_code":(?[0-9]++).*?"response":"(?.*)"`
Comment: why don't use the json filter instead of parsing by hand? You can extract the json part with a grok pattern like this: `%{SYSLOG5424SD:logtime} ApiLogger.%{LOGLEVEL:loglevel}: API log data: %{GREEDYDATA:json_field}` and then you call the json filter on the json_field.
Comment: Yes, `%{SYSLOG5424SD:logtime} ApiLogger.%{LOGLEVEL:loglevel}: (?\w+ \w+ \w+):\s*%{GREEDYDATA:json_field}` and then parse the `json_field` with [JSON filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html).
Comment: It looks as if the open quote after `"response":` has no matching closing quote, is it correct? Did you forget to include it?
Comment: It has but I did not share the entire content as it is very long. Yes, I forgot to include that.
Comment: Try `%{SYSLOG5424SD:logtime} %{JAVACLASS:loglevel}: (?\w+ \w+ \w+):\s*\{"endpoint":"(?[^"]*)","http_method":"(?[A-Z]+).*?"user_id":(?[0-9]+),"user_type":(?[0-9]+),"http_response_code":(?[0-9]+),"response":"(?.*)"`
Comment: i am able to get till http_method, not beyond that.
Comment: Thank you, @WiktorStribiżew. I already tested. it was really helpful. both the suggestions.
Comment: Thank you @baudsp. It was really helpful.
Answer: You can use
%{SYSLOG5424SD:logtime} ApiLogger.%{LOGLEVEL:loglevel}: (?<API>\w+ \w+ \w+):\s*%{GREEDYDATA:json_field}

Then, you can parse the json_field with JSON filter.
If you want to play around with regex, you should remember that regex engine parses a string from left to right by default. If you want to capture several fields with one regular expression, you should make sure the regex engine can "walk" all the way from one part to another. If you know what patterns there are, what types of chars there are between the two, it is great. If not, you can only rely on a .* (%{GREEDYDATA}) or .*? (%{DATA}) patterns.
So, as an excercise, you might have a look at
%{SYSLOG5424SD:logtime} %{JAVACLASS:loglevel}: (?<API>\w+ \w+ \w+):\s*\{"endpoint":"(?<endpoint>[^"]*)","http_method":"(?<http_method>[A-Z]++).*?"user_id":(?<user_id>[0-9]++).*?"user_type":(?<user_type>[0-9]++).*?"http_response_code":(?<http_response_code>[0-9]++).*?"response":"(?<response>.*)"

Check the ++ in [0-9]++ and .*? patterns between each field. The ++ possessive quantifier make sure the engine does not retry matching with the pattern that is modified by the quantifier again if the subsequent patterns fail to match. The [0-9]++ grabs a sequence of digits and does not give them away and if the subsequent patterns fail, the whole match fails. .*? simply matches any zero or more chars other than line break chars, as few as possible. The last .* is greedy, because it must match as many chars other than line break chars as possible.
See the regex demo.


Impute binary outcome variable for GLM using Stan in R

Question: My outcome variable is a series of Bernoulli trials where some values are missing
y $\in$ {0, 1, NA}
How do you impute NA values for an outcome variable in rstan in the context of a GLM, assuming they are missing at random?
Reproducible example, this runs with y. How do I modify the code to impute NA values in y_miss?
library(rstan)
set.seed(123)

#generate data
y <- rbinom(20, 1, 0.5)
N <- length(y)
x <- rnorm(20)

#outcome variable with missing data
y_miss <- c(y[1:17], NA, NA, NA)

#data as a list
data <- list(y=y, x=x, N=N)

#stan code
model_code <- '
data {
int N;                         //number of observations (#20)
int<lower=0, upper=1> y[N];    //Bernoulli distributed outcome variable
vector[N] x;                   //continuous explanatory variable
}
parameters {
real a;                        //intercept
real beta;                     //slope
}
model {
vector[N] p;                   //vector to hold values of linear model
a ~ normal(0,5);               //prior for intercept
beta ~ normal(0,5);            //prior for slope
for (i in 1:N) {
p[i] = a + beta*x[i];}         //linear model
y ~ bernoulli_logit(p);        //likelihood with link function
}'

#run model
m <- stan(model_code=model_code, data=data, iter=4000)

Answer: Each value of y_miss can either be 0 or 1, so you need to marginalize over them with a statement such as
for (i in 1:n_miss) {
  real eta = a + beta * x_miss[i];
  target += log_mix(inv_logit(eta), bernoulli_logit_lpmf(1 | eta),
                                    bernoulli_logit_lpmf(0 | eta));
}

where x_miss is a vector of observations on the predictor of size n_miss when the outcome is missing. This is an important conceptual point, rather than merely a code exercise: You need to obtain a posterior distribution that is differentiable with respect to the unknowns a and beta in order to draw from their posterior distribution with sufficient efficiency in order to make reliable conclusions with a finite number of draws.  Then, in generated quantities you can draw realizations of the missing values with
generated quantities {
  int y_imp[n_miss];
  for (i in 1:n_miss) {
    real eta = a + beta * x_miss[i];
    y_imp[i] = bernoulli_logit_rng(eta);
  }
}

Comment: When I try this with simulated data, I get biased inferences of `beta`, and the bias grows with the amount of data. If you have any ideas, I have detailed the issue here: https://stats.stackexchange.com/questions/563129/marginalizing-out-discrete-response-variables-in-stan
Comment: The accepted answer to this question is wrong because it provides very biased estimates of `beta`, as I detail here: https://stats.stackexchange.com/questions/563129/marginalizing-out-discrete-response-variables-in-stan
Answer: I cannot comment, but I agree that the accepted answer here seems problematic. Per this discussion on the Stan forums, it is clear that the provided code amounts to incrementing the log probability by:
$$\log(p_i^2 + (1-p_i)^2)$$
For each missing observation $y_i$, where $p_i = \mathrm{invlogit}(\eta_i)$,
In other words, letting $M(y_i)$ be an indicator variable indicating that $y_i$ is missing, this implies that
$$P(M(y_i) \mid p_i) = p_i^2 + (1-p_i)^2$$
for each missing $y_i$.
But this makes the presence of any missing values $y_i$ strongly informative about the corresponding $p_i$ and thus about $a$ and $\beta$. Specifically, it will cause the model to prefer values of $a$ and $\beta$ that make $\eta_i$ of large absolute value (and therefore $p_i$ close to 0 or 1) for all missing observations $y_i$, as a plot of the quadratic $P(M(y_i) \mid p_i) = p_i^2 + (1-p_i)^2$ makes clear:
Answer: As another answer mentioned, you can do this in the generated quantities block with Stan. However, I believe that answer is incorrect in marginalizing over the unknown values. The unknown values shouldn't affect target i.e. the model log probability. I think this model should unbiased estimates of model parameters while inferring distributions for missing y values.
N_mis <- 3
data <- list(y=y, x=x, N=N, N_mis=N_mis)

model_code <- '
data {
  int N;
  int N_mis;                           //number of observations (#20)
  array[N] int<lower=0, upper=1> y;    //observed variable with NAs at the end
  vector[N] x;                         //continuous explanatory variable
}
transformed data {
  int N_obs = N - N_mis;
}
parameters {
  real a;                        //intercept
  real beta;                     //slope
}
model {
  a ~ normal(0,5);               //prior for intercept
  beta ~ normal(0,5);            //prior for slope

  y[1:N_obs] ~ bernoulli_logit(a + beta*x[1:N_obs]);
}
generated quantities {
  array[N_mis] int<lower=0, upper=1> y_mis = bernoulli_logit_rng(a + beta*x[N_obs+1:N]);
}
'

Comment: The code example makes your answer considerably better. Just fyi: my comment above is an auto-generated comment, and it occurred, obviously, before your edit. So I would say it no longer applies. Cheers.
Comment: This is incorrect. The Stan manual shows how to impute missing continuous parameters, not discrete parameters, because HMC cannot sample from them (unlike BUGS/JAGS etc).
Comment: @user1454550 This works if the assumption is that the data is MCAR, yes. But if one wants to marginalize over the missing data, then it needs to be done in the model.
Comment: This does not provide an answer to the question. Once you have sufficient [reputation](https://stats.stackexchange.com/help/whats-reputation) you will be able to [comment on any post](https://stats.stackexchange.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/317478)
Comment: What part of the question did I not answer? I've updated my answer with a code example though.
Comment: @user_15 ah, very true. My apologies, yes this code would only work for continuous parameters not discrete.
Comment: @user_15, I think I may have fixed the answer, could you take a look? I moved the discrete parameters from the parameters block to generated quantities, I think that would work.


Conflicting interpretation of an angle?

Question:
This Figure is from an exercise to calculate the height of Blair Cuspid #5.
I seem to obtain conflicting interpretations from the given condition.  In particular, I am not sure it is which angle the description

the ground sloped downward $6.9^\circ$ in the direction of this shadow

is referring to.  My gut feeling was it refers to $\angle DBE$.  However, Question a) claims that $\angle CDB = 4^\circ$.  But shouldn't $\angle CDB = 6.9^\circ$, as the alternate interior angle of $\angle DBE$?  The only sensible explanation is that the angle measured $6.9^\circ$ is actually $\angle BDE$, but then I can not make sense of the description quoted above (English is my second language).
I Googled about Blair Cuspids and found a research article that included some calculation.  On page 33, it goes:

The only way for me to make sense of this text is to interpret "effective elevation angle" as $\angle CDB$, and "local slope" as $\angle BDE$.  (Otherwise, I would infer conflict again.)  But I cannot get my head around these interpretations.
Am I missing some background knowledge, such as effective elevation angle and local slope?  But Google did not return anything quite useful.
Comment: I wish I hadn't clicked on the link to that website (https://mysteriousuniverse.org/2016/02/the-blair-cuspids-a-legitimate-lunar-anomaly/): it seemed to do a hell of a lot of things to my computer without asking permission.
Answer: It makes no sense to me either. Maybe it's a mistake: for the information given, I would conclude that angle $\angle BDE$ is $4^\circ.$

Conflicting interpretation of an angle?


Why aren't all Normal and Rademacher variables independent?

Question: In Wikipedia page: https://en.wikipedia.org/wiki/Normally_distributed_and_uncorrelated_does_not_imply_independent, we find a classical counter-example showing that two normally distributed and uncorrelated random variables may not be independent.
This implies  $X$ having a normal distribution $N(0,1)$ and $W$ having the so-called Rademacher distribution.
This is OK, but what troubles me is the sentence "and assume $W$ is independent of $X$".
Do we really have to assume $X$ and $W$ are independent? Is it not obvious? Can we imagine such an $X$ so that $X$ and $W$ would not be independent?
Comment: It depends on what you mean by "such an $X$:" are you asking whether there exists a bivariate random variable $(X,W)$ with Normal and Rademacher marginals for which $X$ and $W$ are not independent? (In that case consider any measurable subset $\mathcal{A}$ of the line for which $\Pr(X\in\mathcal A)=1/2$ and define $W$ in terms of its indicator function.)  Or, as strongly suggested by your title, are you trying to ask something about the counterexample on the Wikipedia page?
Comment: whuber, you are right; actually, the title is not as accurate as it should be...And yes, I was asking how it comes that ``X`` following ``N(0,1)`` and ``W`` following ``w=+1`` and ``W=-1``, each with probability ``1/2``, could be not independent.
Answer: The tickets-in-a-box model of random variables described at https://stats.stackexchange.com/a/54894/919 provides a helpful way to think about this.
Imagine you have a box full of tickets on which are written various numbers in such a way that a blind draw of one ticket acts like observing $X.$ $W$ is a second number found on every ticket: half the tickets display a $1$ and the other half display a $-1$ (that's the definition of a Rademacher variable).
$X$ and $W$ are independent when observing $X$ gives you no clue about what $W$ is and vice versa.  Here is the tickets-in-a-box translation of that intuitive description of independence: no matter what the value of $X$ may be, the proportions of tickets with any particular values of $W$ are always the same.  In the terminology of conditional probability this says
$$\Pr(W\in\mathcal{A}\mid X\in\mathcal{B}) = \Pr(W\in\mathcal{A})$$
for any events $\mathcal{A}$ and $\mathcal B.$  Equivalently, multiplying both sides of this equation by $\Pr(X),$ we obtain
$$\Pr(X\in\mathcal{A}\text{ and }W\in\mathcal{B}) = \Pr(X\in\mathcal{A})\Pr(W\in\mathcal{B}).$$
That's the mathematical criterion of independence.
Thus, one way to create dependence is to write $W=1$ on some special half of the tickets.  In fact, almost any half will do, because the other half--unless you chose these halves very carefully--will exhibit a different distribution of $X$ values.
As a concrete example, let $X$ have a standard Normal distribution and set $W=1$ when $X$ is positive and $W=-1$ otherwise.  $X$ tells us exactly what's on $W,$ so $X$ and $W$ cannot be independent.
Comment: whuber, this is exactly the kind of answer I was looking for: detailed and precisely explained in all details. It's now clear why we have to "assume" independence. Thanks for having taken time to write.


Migration issues with UIManagedDocument

Question: I started using CoreData in my application following Stanford CS193P lessons regarding the use of iOS 5's new class UIManagedDocument. The approach itself is quite straightforward but I can't understand how to deal with model modifications i keep making. This is how I instantiate my UIManagedDocument object (inside the appDelegate, so that every other class can use it):
<code>if (!self.database) {
    NSURL *url=[[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject];
    url = [url URLByAppendingPathComponent:@"AppName"];

    UIManagedDocument *doc = [[UIManagedDocument alloc] initWithFileURL:url];
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], NSMigratePersistentStoresAutomaticallyOption,
                             [NSNumber numberWithBool:YES], NSInferMappingModelAutomaticallyOption, nil];
    doc.persistentStoreOptions = options;
    self.database=doc;
    [doc release];
}
</code>
The issue I have is that every time I change even a little bit of my .xcdatamodel, I am unable to get all the content previously stored in the document as well as to create any new instance. As a matter of fact doing this generates the following exception:
<code>*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'This NSPersistentStoreCoordinator has no persistent stores.  It cannot perform a save operation.'
</code>
I thought setting the "options" property of the managed document would have solved the problem, but apparently this isn't enough.
Anyone can help? Couldn't' find other questions that actually fit my precise needs.
Comment: I followed Stanford CS193P too, and I am experiencing the same error.
Answer: Before you modify your Core Data model, you should "Add Model Version".
1.
Select the original model file. (e.g. YourProject.xcdatamodel)
2.
"Editor" -> "Add Model Version...". Then add a new model version (e.g. 2.0)
3.
You will get a new model file. (e.g. YourProject 2.0.xcdatamodel). Modify it.
4.
Change the current model version. Select the top .xcdtatmodel file -> "View" -> "Utilities" -> "Show File Inspector". Find the tag "Versioned Core Data Model" and choose the right version you want to modify.
It also disturbed me for a long long time. Hope this way can help you ^^
Comment: Hi. Thank you for the answer. Seems like the good way to go. Anyway, after following your suggestion, I keep getting migration issues:

    *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Can't merge models with two different entities named 'ManagedChild''

where ManagedChild is one of the entities in my model. I found this [entry](http://stackoverflow.com/questions/3205783/core-data-error-cant-merge-models-with-two-different-entities-named-foo) but it doesn't use the samy UIManagedDocument approach as I need...
Comment: I'm getting the same problem.  All the documentation says that adding the "persistentStoreOptions" is the way to go.  But I get a nil return from UIManagedDocument along with the same error messages.  Deleting the store is not a valid option for me, since I'm supporting a released app.
Comment: Just another data point... I created a sample project that's initializing Core Data via UIManagedDocument exactly the same way I'm doing in my app, but the sample project seems to migrate just fine (even without any persistentStoreOptions).
Answer: There is a really simple solution to your problem. Simply erase the app from the simulator or device (touch the app icon for a few seconds, and touch the cross that appears when the app icons start wiggling). That way, Xcode updates your app's UIManagedDocument.
Comment: and what if the app is already in the store. you display an alert view and ask users to reinstall your application?
Comment: I think pH-k's answer is what the questioner needs since he is following a tutorial, not for publishing an app to the store:)
Comment: this didn't help me. Persistent store is not available it says, but I clearly have it!
Answer: Make sure that you did not mistype NSDocumentDirectory as NSDocumentationDirectory.
<code>   NSURL *url = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject];
</code>
NSDocumentDirectory is right!

Migration issues with UIManagedDocument


Prevent AWS load balancer serving on IP address

Question: I've configured a Amazon Webservices Load Balancer for my site.
It's all good the domain name resolves to the IP address of the Load Balancer Endpoint, and on the internal LAN it communicates with my two front end webservers.
I've noticed in webmaster tools that Google has picked up the IP of the load balancer and is serving the site up on the IP address as well as when the host name is used.
E.g.  54.76.x.x
If it were IIS I could use the host-header setting to block it serving up on the IP as well but there appears to be no control over this in the AWS admin panel?
Answer: Do it in your appserver.  The AWS load balancer is pretty dumb; it doesn't do much of anything other than take requests and forward them.  Configure your appserver to respond "appropriately" (404, 403, whatever) to requests that don't have an appropriate Host header, and Google will get the message pretty quick.
Comment: The AWS load balancer will send the Host-Header through?  Haha its me that being dumb.  I check..
Comment: For some reason I thought it would strip the header and just pull on whatever port was there.  Yes setting the host headers in IIS on both machines - actually I am using port 83 for this particular site locked it down.


Why does casting out $9$ and $11$ work to compute remainders?

Question: I found two examples that I'd like to understand:
Example 1: Find $112358132134 \mod 9$.
Solution is:
$1+1+2+3+5+8+1+3+2+1+3+4 \mod 9 = 7 \mod 9 = 7$
Example 2: Find $62831853 \mod 11$.
Solution is:
$(-6)+2+(-8)+3+(-1)+8+(-5)+3 \mod 11 = -4 \mod 11 = 7$
My question is: Based on what property we sum the digits of the number ( as Example 1) or subtract (as Example 2)?
Comment: $10\equiv 1\pmod{9}$, so since $112358132134 = 4 + 3\cdot 10 + 10^2 + 2\cdot 10^3 + ...$, it follows that $112358132134$ mod $9$ is the sum of the digits. Similarly, $10\equiv -1\pmod{11}$, so you get a factor of $-1$ every time you have an odd power of $10$ coming from the decimal expansion.
Comment: Lookup [divisibility rules](https://en.wikipedia.org/wiki/Divisibility_rule#Divisibility_rules_for_numbers_1.E2.80.9330).
Comment: Those are specific to the number 9.  And the number 11.  9=10 - 1.  And 11 =10+1.
Comment: See also : http://math.stackexchange.com/questions/328562/divisibility-criteria-for-7-11-13-17-19
Answer: Usually, we cannot compute the remainder of a number mod $n$ by summing the digits (example 1), or by subtracting alternately (example 2). This is a special trick that works for mod $9$ and $11$. It also works for mod $3$.

To compute the remainder of a number mod $3$ or mod $9$, you may sum the digits and then compute the remainder of that result.
Why it works: we write numbers in base $10$, and $10 \equiv 1$ mod $3$ or mod $9$, so multiplying by $10$ is really multiplying by $1$.
To compute the remainder of a number mod $11$, you may alternately add and subtract the digits and then compute the remainder of that result.
Why it works: we write numbers in base $10$, and $10 \equiv -1$ mod $11$, so multiplying by $10$ is really multiplying by $-1$ (changing the sign).

You can read more about these divisibility rules and others at this Wikipedia page.
Answer: Hint:
$$1\bmod9=10\bmod9=100\bmod9=1000\bmod9=\cdots 1,$$
$$1\bmod11=100\bmod11=\cdots 1,$$
$$10\bmod11=1000\bmod11=\cdots-1.$$
See the decimal expansion as a linear combination.
Answer: Radix notation has $\color{#c00}P\!$$\textit{olynomial form}$ $\, n = \color{#c00}{P}(10) = d_k 10^k +\cdots + d_2\cdot 10^2 + d_1\cdot 10 + d_0\, $
so $\, {\rm mod}\ 9\!:\, \color{#0a0}{10\equiv 1}\,\Rightarrow\, P(\color{#0a0}{10})\,\equiv\, P(\color{#0a0}1)\,\equiv\, d_k+\cdots + d_1 + d_0 = $ sum of digits $\,d_i,\ $ and
$ {\rm mod}\ 11\!:\, \color{#90f}{10\equiv -1}\,\Rightarrow\, P(\color{#90f}{10})\equiv P(\color{#90f}{-1})\equiv (-1)^k d_k+\cdots - d_1 + d_0 = $ alternating digit sum.
In both cases we employed the  the $ $ Polynomial Congruence Rule, $ $ i.e. $\,a\equiv b\,\Rightarrow\,P(a)\equiv P(b),\ $ for any polynomial $\,P(x)\,$ with integer coefficients, and any integers $\,a,b.$

Remark $\ $ These tests are special cases $\,(x = 10)\,$ of results true for any polynomial, namely
$ {\rm mod}\ x\!-\!1\!:\,\ \color{#0a0}{x\ \equiv\ 1}\ \Rightarrow\  P(\color{#0a0}{x})\,\equiv\, P(\color{#0a0}1)\,\equiv\, d_k+\cdots + d_1 + d_0 = $ sum of coef's $\,d_i,\ $ and
$ {\rm mod}\ x\!+\!1\!:\,\ \color{#90f}{x\equiv -1}\,\Rightarrow\, P(\color{#90f}{x})\equiv P(\color{#90f}{-1})\equiv (-1)^k d_k+\cdots - d_1 + d_0 = $ alternating coef sum.
Thus if the coef sum $\,P(1) = 0\,$ then $\,P(x)\equiv P(1)\equiv 0\,\pmod{x\!-\!1}\,\ $ so $\,\ x\!-\!1\mid P(x),$
and, similarly, if  $\ P(-1) = 0\,$ then $\,P(x)\equiv P(-1)\equiv 0\pmod{x\!+\!1}\,\ $ so $\,\ x\!+\!1\mid P(x).$
Both are special case of the well-known $ $ Factor Theorem $\,\ x\!-\!a\,\mid\, P(x)-P(a).$
In particular, for $\,x = b = \,$ radix, we get the base $b$ analog of casting out $9$ and $11$ in radix $10$
Answer: Remainder after division by $11$
The OP's question asked for the properties that 'force' the technique. They should be aware that there are often several ways to tackle math problems.
Working in $\Bbb N = \{0,1,2,3,\dots\}$ with numbers presented in their base ten representation.
Problem: For any $m$ find the remainder $r$ after division by $11$: $\;m = 11q + r$.
Without using modular arithmetic and negative numbers, you can employ symbolic homotopy rules to transform $m$ down to $r$. Since this is recreational math, we leave it to the reader to formalize these rules; here we only give some examples.
This method is useful for an organized approach where you can carefully check each step for errors.
Example 1: If $m \lt 11$ then $m \sim [m]$
Example 2: $20]. \sim [9]$
Example 3: $30]. \sim [8]$
Example 4: $90]. \sim [2]$
Example 5: $55] \sim [0]$
Example 6: $58] \sim [3]$
Example 7: $85] \sim 30]. \sim [8]$
Example 8: Find the remainder $r$ for $\frac{62831853}{11}$:
$\quad 62]831853 \sim 408]31853 \sim 12]31853 \sim 13]1853 \sim 21]853 \sim 108]53 \sim 95]3 \sim 403] \sim [7]$
Example 9: $29]0 \sim 70]. \sim [4]$
Example 10: $100]0000 \sim 100]00] \sim 100] \sim [1]$
Example 11: $11]11111 \sim 11]111 \sim 11]1 \sim [1]$


file_create_url printing different addresses on live and local installations

Question: I'm using the following code to print out the path of an image uploaded through theme-settings.php:
<img src="<?php print file_create_url(file_build_uri(theme_get_setting('my_setting')); ?>" />

Locally, everything works fine -- the path renders as I expected:
http://localhost/drupal/sites/default/files/image.jpg

When the site is online, however, the same code renders the following address:
http://www.mydomain.com/sites/default/files/public%3A//image.jpg

Does anyone know why public:// would be printed out on a live server but not on a local installation? My guess is that the problem is related to the use of file_build_uri.
I'd like to find some code that prints out the URL correctly on both live and local installs. Perhaps I'm not using the best approach with the API to achieve this end?
Answer: The following code worked on both the local and live install:
<?php print file_create_url(theme_get_setting('my_setting')); ?>


Using a Pascal Library in XCode

Question: I am developing an iOS app (OSX 10.7.4, XCode 4.3.3, and targeting iOS version 5.1).  I have the interface already built, but the core functions the app will be using are written in Pascal.  Rather than porting the Pascal functions to C (and having to maintain two copies), I was looking for a way to create a Pascal library that can be imported into XCode and called by the Objective-C code.
The Pascal functions are nothing more than calculations (no interface code, etc.). An example of how I would like it to work would go something like: the user pushes a button, the Objective-C code collects the entered data, the appropriate Pascal calculating function is called, and then the result is returned and displayed--Pascal only being used to provide the result.
I found FreePascal, and am assuming that is where I would start, but I have not been able to find any clear information regarding how to accomplish what I want to do.  I also found this walkthrough, but the end result is that you can use Pascal to actually write your applications--which seems like overkill compared to what I need.
My question is: what is the the easiest way to use my core Pascal functions within my already built iOS app?  Is integrating FreePascal into XCode and importing my current application storyboard and code into one of the Pascal templates absolutely necessary, or is it possible to create a compatible Pascal library using another method and reference it from my code?
I would very grateful for any help provided (even just a link).  I've done a lot of searching, and would really like to get this problem solved.
Answer: The simplest mechanism is to create a pascal Static Library and include it in the Objective-C iOS project, otherwise you would have to integrated the build process into the xcode environment; which is a little bit more work rather than just exposing the API via a header and then linking to the <code>.a</code> file.
Comment: Thank you very much for your comment and link!  It was exactly what I was looking for (and a lot easier to setup than I was first anticipating).

Using a Pascal Library in XCode


Is there a way to define the docstring for a python object that defines __call__?

Question: I have a class that defines a __call__, but the when the user sees requests help for this "function" on the help return, they get the entire object and methods and it can be a bit daunting...I would like the help to look more like a function's help if possible.
I followed the trail in to the inspect module, but it seems that since I cannot inherit from function type and/or method type, then this may not be doable. Am I missing something or do I have to re-arrange the code such that I am dynamically building a true Python function object?
Answer: What you want is not doable no. The help() information (and inspect information, etc.) will show it as a class with a __call__ method, not a function.
If the object you produce doesn't have to support anything other than calling, you could use a closure to return an actual function:
def produce_callable(*args, **kw):
    instance = YourClass(*args, **kw)
    def callable_function(*args, **kw):
        """Docstring here"""
        return instance(*args, **kw)
    return callable_function

So rather than return your custom class instance, you return a wrapper function that'll call the class instance. help() information will be given for that function.
I used the *args and **kw wildcard arguments in the callable_function() signature, but if you know what arguments are accepted by the __call__ method, you can hard-code those in the function you produce.


Looking for a new backup solution to replace dying tape drive

Question: We're running Windows Server 2003 SBS and another machine with Server 2003 Standard on it.
The SBS server is about 7 years old running pretty much 24/7 - a HP server of some description. We have an Ultrium 448 cycling LTO2 400GB tapes daily and incrementally backing up approximately 100gb worth of data (20gb C:\ and system state, 40gb exchange, 40gb database for some crap marketing software) on BackupExec 10D. As of 5 months ago, the backups have been consistently failing with IO errors, bad reads and some write errors.
When I say consistent, I mean every time and we haven't had a proper backup for the entire 5 months - So if the server explodes tomorrow, 7 years worth of data will just cease to exist. I've only just recently rejoined the company and am looking at rectifying the more concerning problems, so the first thing I did was try a backup to an USB2.0 external drive. It was excruciatingly slow. In fact it was so slow it took 40 hours and it still wasn't finished. I ended up cancelling it and reconfiguring the selections again to reduce file size. This, however, isn't a permanent solution.
I concluded that the IO error was either from a faulty tape drive (which has a tape stuck in there right now and not coming out) or from a dying SCSI controller. Neither of them are good news and both are extremely expensive to fix. I'm operating on an extremely low budget so have been looking at outsourcing the backups.
A company in Sydney (where I'm located) offer incremental online backups via a NAS. It costs almost double a new tape drive but offers monthly repayments which will let us get through times when cash flow is minimal. It seems like a sweet deal but it is still a little bit pricey. So I'm looking for a cheaper, yet reliable solution. Maybe some in-house NAS or something offsite? The idea is to avoid using tapes.
Are there any recommendations for rectifying my current situation? Or are tapes the only way to go? I'm concerned that the server will die one day in the near future and I must be able to restore it to another server with different hardware.
Answer: There are some good discussions on backup already on Server Fault. I'd have a look at them:

my local backup request
Is our planned backup strategy adequate for my new server insfrastructure?
Recommended Backup Media for Circa 2009?

I notice that you're already ruling out tape. I'd do a cost-benefit analysis of the alternatives pitted against tape before dismissing tape out of hand. It sounds to me like you've either got a bum SCSI controller, cable, or bum tape drive and it's unfortunate that it has soured you on tape because tape can be very reliable and robust.
I did some calculations a few months ago (posted on Server Fault in one of the questions above, but not updated) and found that tape wasn't necessarily the most expensive route. Call me a traditionalist, but tape has worked very well for me and my Customers, and has proven to be reliable and suitable for disaster recovery. I tend to think that a lot of the "horror stories" associated with tape being unreliable often stem from backup strategies that don't involve testing of the backups after-the-fact. As is often said, backup isn't about backup-- it's about being able to restore.
Having a maintenance agreement for your hardware is, obviously, a component of any robust solution as well. You state that you need a backup you can restore to other hardware, but I'd caution you that your SBS server's "System State" backups aren't restorable by any Microsoft-supported means to alternative hardware. Hopefully your other server is a domain controller, since you'll be losing Active Directory (and, thus, making your Exchange data fairly difficult to get at) if you lose that SBS Server computer.
Having a maintenance agreement on the tape drive and / or server would get you out of the mess that you're in, in this case. It's particularly critical since you've currently got a backup, assuming you don't have another domain controller, for which you really need the identical server hardware available for restore.
You shouldn't center your whole backup regime around writing backups to an on-site storage device. Backup is off-site and offline. Anything less isn't backup.
I'd question if you can restore fast enough to get back up and running in a timely fashion if your main backup is an off-site device and accessible only through your Internet connection. If your renmote backup provider has a provision to ship you a physical storage device I could see that working. If you have to download your entire backup corpus over a consumer Internet connection, though, I'd think that you'd be talking about at least a couple of days to get everything back up and running. Ouch!
Comment: What do you mean by "aren't restorable by any Microsoft-supported means"? Are there ways around this?
Comment: Here's what Microsoft has to say about restore scenarios for restoring the System State: http://support.microsoft.com/kb/249694
Comment: +1, almost like you read mah mind!
Answer: Unless you have a compelling reason to change your strategy (and I can't see anything in your post to suggest that you do - You're well within capacity. LTO2 is still widely available and used, throughput speed/window problems weren't mentioned so I assume not a problem), I recommend sticking with your current system. If a component has failed, replace it like-for-like. LTO 2 drives are still widely available.
Tape backups are not a legacy concept. They're still 100% relevant today, and widely used in most medium and enterprise businesses.
I hate to ask the obvious questions, but lets cover them off just in case: Has the tape drive been taking a cleaning tape regularly (every couple of weeks)? Has the server been rebooted/backup exec services been restarted? Have any Windows patches been applied recently? Have any other configuration changes been made to the server? Is the SCSI cable terminated properly? Have you tried swapping the SCSI cable for a replacement?
I'm always very wary of using a component failure as the business driver to move to a 'new and improved' system. The quickest way of getting a successful backup is likely to be replacing the tape deck with another tape deck.
Comment: +1 for "Tape backups are not a legacy concept". If only hard drive technology was as advanced as modern tape drive technology.
Answer: As I stated in an answer to another post, I don't like the idea of using any type of hard drive based media for permanent backups. I think they're fine for short term, near-line backups but I wouldn't want to rely on them for my long term, off-site, off-line, or archival backup needs. My recommendation would be to utilize them (if you must) for your daily backups combined with backups to tape of the same data for your off-site, long term backup needs (backup to disk, then back up the disk backups to tape).
I certainly wouldn't rely on any "online" backup provider to fill my primary backup needs. They'd be fine for maintaining a copy of the backup for off-site and archival needs as an augmentation to tape backup. How long would it take to backup to their service? what happens if they go out of business or their facility burns down? What if you need access to your data at 3AM?
Comment: I definitely wouldn't trust an online backup that seemingly exists only virtually. I met with these online backup people and they have a local presence. Basically, the server backs up to a NAS, the NAS sends encrypted data incrementally over to their servers, they transfer the data onto DVD/tapes, I can go and pick these up or they can hold it for me. Best thing is that they're only 15 minutes away from the office and I even had a tour of their facilities. Downside is that they are quite costly in aggregation.
Comment: @E3: That sounds pretty nice, actually. I wonder how the cost compares to doing your own disk-to-disk-to-tape in-house, over the long haul.
Answer: Buy a blu-ray external USB drive, and make some backups there.
It is cheap, and with 50GB in each disk (double layer) each of your savesets will comfortably fit in only one disk.
After you have that set up (it is a bit uncomfortable to change disks, but it is a temporary solution) you can think on better alternatives with more time ahead and no pressure.


Update all subdocuments of a sub-document (not an array)

Question: My documents are of the following format:
{
  _id: 'something',
  topLevelProp: 'top'
  obj: {
    prop1: { a: 'a', b: 'b' },
    prop2: { a: 'x', b: 'y'},
    prop3: {a: 'x'}
  }
}

I want to update all the subdocuments inside obj to include the value of topLevelProp.
So, the result would be:
{
  _id: 'something',
  topLevelProp: 'top'
  obj: {
    prop1: { a: 'a', b: 'b', anotherProp: 'top' },
    prop2: { a: 'x', b: 'y', anotherProp: 'top'},
    prop3: {a: 'x', anotherProp: 'top'}
  }
}

The closest I could get was something like
db.update({_id: "something")}, {$set: {'obj.$.anotherProp': '$topLevelProp'}}) which doesn't actually work, giving this write error: Cannot apply array updates to non-array element obj.
Guess this is because $ is meant for use in arrays, not objects. using $[] produces similar error, as expected.
Is there a way to do something like this?
Answer: Use $objectToArray to convert obj into array of k-v tuples. $mergeObjects to add the key. Finally use $arrayToObject to revert back to original object form.
db.collection.update({},
[
  {
    "$addFields": {
      "obj": {
        "$objectToArray": "$obj"
      }
    }
  },
  {
    "$addFields": {
      "obj": {
        "$map": {
          "input": "$obj",
          "as": "o",
          "in": {
            k: "$$o.k",
            v: {
              "$mergeObjects": [
                "$$o.v",
                {
                  anotherProp: "$topLevelProp"
                }
              ]
            }
          }
        }
      }
    }
  },
  {
    "$addFields": {
      "obj": {
        "$arrayToObject": "$obj"
      }
    }
  }
],
{
  multi: true
})

Mongo Playground
Comment: amazing! thanks for providing the right answer so quickly :)


appropriate way of using json to return array from php and displaying contents in html

Question: below is the ajax code.
questions are in comments
 $.ajax({
            type: "POST",
            url: "dbtryout2_2.php",
            data:datastr,
            dataType: "json",
            success: function(arrayphp)
            {
                              //ARRAYPHP is a json array and iam displaying the
                              //contents of the array one by one as link
              $.each(arrayphp, function(i, element) {
              var link = $('' + element + '');
              var image='<input type="image" src="img4.jpg" height="30px" width="30px">';

                 $(".searchby .searchlist").append(image).append(link);
                });

            }
         });

the above code is not working.am i not using json array properly
.below is the php code
/PHP CODE/
if(!strcmp($_REQUEST['id'],'a2'))
{
 $lang=$_REQUEST['lang'];$type=$_REQUEST['type'];
 $query="select distinct(cat_name) from song where lang='$lang' and category='$type'";
 $result=mysql_query($query,$con) or die("Query failed".mysql_error());

 $arrayphp=array();
while($row=mysql_fetch_array($result))
{
    $element=$row['cat_name'];
        array_push($arrayphp,$element);
}
    //here is the json array that is being sent to the above html code
  echo json_encode($arrayphp);
}

Answer: You need to convert the string you pass from PHP to a proper JSON Object/Array before it is usable as a javascript object/array.
success: function(arrayphp) {
    arrayphp = jQuery.parseJSON(arrayphp);  // will make proper json from a json string
                          //ARRAYPHP is a json array and iam displaying the
                          //contents of the array one by one as link
    $.each(arrayphp, function(i, element) {

    ....

Comment: @ riggs adding the line has no effect
Comment: @ riggs u have any other suggestion.look above i have written my problem to @ jantimon.
Comment: @Nitishkumarthakur Do you know how to use the debugger in your browser?
Comment: @ riggs no.How to use debugger?I have chrome browser
Comment: @Nitishkumarthakur F12 normally loads it.


Is any Half-space closed? (Since affine sets are closed)

Question: I came across the following property:

Any Affine set is closed.

But take any open half-space. I believe that it is an affine space.
However, It is not closed.
Can someone clarify?
Comment: I'm 90% sure that affine *set* and affine *space* are not the same thing, and 100% sure that open half-spaces aren't affine spaces. However, addressing this would require me to look up the definition, which I don't want to do.
Comment: You should edit your question to include which **definition** of an affine set you use.
Comment: Can you tell me why an open half-space isn't affine ( Or any half-space in general, if that is the case?) @SaucyO'Path
Comment: (Point in the half-space) + (vector in the vector space) may give you a point outside the half-space.
Comment: @frenchstudent00 No, I can't, because I don't understand what the sentence "An open half-space isn't affine" means. In point of fact, I can't even understand if I've ever claimed it to be true.
Answer: For the purposes of clarity I am using the definition that the set $A$ is affine if for any two points $x,y\in A$ the line through $x$ and $y$ is completely contained in $A$.
To answer your objection to the statement consider the open half space  $A=\{(x,y)\in\mathbb{R}^2:y> 0\}$ and note that the points $(0,1)$ and $(0,2)$ are both in $A$ and the line through these points is given by $x=0$ yet the line $x=0$ is certainly not contained in $A$.


How to sort language select list on current language context

Question: I have a form with a language field with the language select list widget.
The options of this field contains an array with a pattern like
<code>['langcode' => t('language')]
</code>
Now I want to sort these options in the right order.
The default sorting is on the value but not on the translated one.
e.g. ['de' => t('german')] which I translated into "Deutsch" will now sort under "g" instead of "d".
So I did try to do it in a form alter hook and a preprocess hook but this is not the right place to do it, because the values are still untranslated here.
So my question is how can I sort these options if the values are translated?
Answer: i think it can be done in a hook_form_alter()
if the options are in $form['languages'], you can do something like :
<code>$language_options = $form['languages'];
</code>
then manipulate the array $language_options, until it has the order you need
then override it in form :
<code>$form['languages'] = $language_options;
</code>
Comment: The problem is that the values are untranslated in a form_alter_hook.
I could sort it by key but this also doesnt work because the langcode differs from the value.
Comment: I need to render the translatablemarkup first than build a new array and sort its values.
Answer: Now I did this in a form_alter_hook.
This worked for me.
<code>/**
 * Implements hook_form_FORM_ID_alter().
 */
function arph_profile_form_FORM_ID_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id) {
  $language_options = $form["field_languages"]["widget"][0]["value"]["#options"];
  $new_language_options = [];
  foreach($language_options as $langcode => $language) {
    /** @var \Drupal\Core\StringTranslation\TranslatableMarkup $language */
    $new_language_options[$langcode] = $language->render();
  }
  asort($new_language_options); // We need to keep the keys here.
  $form["field_languages"]["widget"][0]["value"]["#options"] = $new_language_options;
}
</code>


Chebyshev's Inequality: given probability, find $k$

Question: Edit with Context: Book says the % of data captured within k standard deviations $= 1 - \dfrac{1}{k^2}$. Dug a bit deeper and found it was derived using Chebyshev's but no direct derivation found$\ldots$
Within how many multiples of standard deviation will capture at least $\boldsymbol{75}$% of the data in a distribution with a mean $\boldsymbol\mu$?
I derived the formula below and got that $k$ must be equal to or less than $k$2. This doesn't make sense to me as a larger $k$ would capture more and more data, so it should be the other way around.
Work is shown below.
The inequality derivation:
Let $v = |X-\mu|$.
Let $y = k\sigma$. Then
$$P(v \geq y) \leq \frac{1}{k^2} = 1 - P(v < y) \leq \frac{1}{k^2}.$$
So
$$k \leq \sqrt\frac{1}{1-P(v<y)}.$$
Comment: I did not know that, I thought it was an equivalent definition so no need to reverse inequalities?
Comment: No problem, I am still a novice at this stuff so I had to double check.
Answer: Your intuition is correct about what Chebyshev inequality says. It's just a minor confusion in the algebra.
The RHS of the inequality on your last line, the $y$ contains $k$ as well. You cannot directly interpret/solve in that fashion (see the edit below for more details).
If you take the probability $P(v<y)$ as given, that is $P(v < k \sigma) = p$ with everything (e.g. the density and $\sigma$) known, then in principle one can directly solve for the value of $k$.
For example, if $X$ is normally distributed and given $P(v < k \sigma) = p = 0.800$, the equation is equivalent to $\Phi(k) = \frac{1+p}2 = 0.900 $ and the solution to is $k \approx 1.28155$.
However, this is often not easy and we bound the probability of interest $P(v < y)$ (which is an increasing function of $k$) by the $1 - \frac1{k^2}$ on the RHS (which is also an increasing function in $k$).
Thus, the inequality to solve as per the question statement to capture at least 75% of the data becomes
$$ 1 - \frac1{k^2} \geq \frac{3}4 \qquad \textbf{so as to guarantee} \qquad Pr\bigg\{~ |X- \mu| < k\, \sigma ~\bigg\} \geq 1 - \frac1{k^2} \geq \frac{3}4 $$
and this gives the desired correct direction of the inequality for $k$.

----------- Below is esp. in response to the comment----------

The Chebychev inequality if written this way:
$$ Pr\bigg\{~ |X- \mu| < k\, \sigma ~\bigg\} \geq 1 -\frac{1}{k^2} \tag*{Eq.(1)}$$
then from the original question statement to capture at least 75% of the data, the correct inequality to solve is
$$ Pr\bigg\{~ |X- \mu| < k\, \sigma ~\bigg\} \geq \frac{3}4  \qquad \textbf{but NOT} \qquad \frac{3}4 \geq  1 -\frac{1}{k^2} \quad \text{(which gives $k \leq 2$)}$$
Similar goes with the complement statement not capturing at most 25% of the data applied directly to $P( v > y ) \leq 1/k^2$.
In conclusion, the counter-intuitive $k \leq \sqrt{1/(1-p)}$ stems from the deceivingly inviting direct application of the inequality. I hope this answers your question of "how this happened".
Comment: @Mid Does the revision address your question better now?
Comment: Thanks for the answer, though I am still confused on the "no longer an inequality aspect". Wouldn't it just be $1 - \frac{1}{k^2}$ less than or equal p? I get your point on since k is used to solve for the probability p in the first place, but I am confused on the algebra rules behind it. Could you enlighten me please?
Comment: Yes. I cannot solve the left and middle part of the inequality because they are tied together by k, but by bounding it from below by 75% then by solving k in the middle equation I am finding the minimum solution to the question I was trying to answer.  It is crystal clear now, this is **exactly** the understanding I needed. Thanks for the followup!
Answer: It seems like you got some great help by @Lee, maybe I can add some more insight by demonstrating the usage of this theorem in a made up problem..
Given something like:
$\mu =124$
and
$\sigma =7$
One could find the minimum probability that the number of values falls between 110 and 138. Note that this is 2 deviations away. For the sake of the problem lets say that 124 represents the amount of crimes per week in some neighborhood
the formula to this theorem looks like this:
$P(\mu-k \sigma <x<k \sigma +\mu)\geq 1-\frac{1}{k^2}$
where k is the number of deviations, so since above I noted that the values between 110 and 138 are 2 deviations away then we will use k = 2.
We can plug in the values we have above:
$P(124-2 \sigma <x<2 \sigma +124)\geq 1-\frac{1}{2^2}$
=
$P(124-2 \sigma <x<2 \sigma +124)\geq 0.75$
=
$P(110 <x< 138)\geq 0.75$
We would conclude that there is at least a 75% chance that the amount of crimes per week is between 110 and 138..
You can use this theorem for any deviation you like, such as 1.5 or 2.5 and so on.. It also appropriate in using non-normal data, pretty convenient.
Comment: Great example. I had in the back of my mind the Normal distribution, but I was surprised to learn that the power of the inequality is due to the generality.


How Can I Take Many Chars and Put Them Into One String

Question: I suppose I just never learned this. I have never done this before. I have seen the use of strcat(S1, S2), but that isn't applicable here, is it?
Can I do something like
string all_possible_strings[10];
char jumbled_chars[] = "ABCDEFG";
all_possible_strings[1] = jumbled_chars[0] << jumbled_chars[1]
                              << jumbled_chars[2] << jumbled_chars[3]
                              << jumbled_chars[4];

What I'm trying to do is make a program that can unscramble a word into all of its possible permutations.
Comment: Yes, but when the tags say `c++` and `string`, I assume an `std::string` until further notice. Question is ambiguous at best right now.
Comment: Your question implies that you don't understand how strings in C++ work. Maybe you should start at a more basic, general, level? If you have a specific scenario you need help with, provide the details.
Comment: Where are the chars to begin with?  If they're already in an array, then all you'd have to do is append a null character (\0), I think.
Comment: all_possible_strings[counter] = jumbled_chars[0] << jumbled_chars[1] << jumbled_chars[2] << jumbled_chars[3] << jumbled_chars[4];
Comment: Although I know I don't use the << stream operator
Comment: `str += c;` is how you append a char to a string, or use the `append` function.
Comment: Don't expand your question in the comments like that. Edit instead. I've done this one for you.
Comment: @anthony-arnold: `std::string str("whatever"); str+=c;` is how you expand a *standard string*, but fails horribly if he is using c-string.
Comment: What are the types of `all_possible_strings` and `jumbled_chars`?
Comment: string all_possible_strings[]
Comment: whenever I use += and cout the strings, it comes out with gibberish ascii values
Comment: Can you update the question with what you're trying?
Comment: @anthony-arnold I assumed you did, but you should be explicit when addressing a user who may be struggling with the basics.
Answer: #include <iostream>
#include <string>

using namespace std;

int main()
{
        string theString = "";
        char a = 'a';
        char b = 'b';
        const char* c = "cdefghijklmnopqrstuvwxyz";

        theString += a;
        theString += b;
        theString += c;

        cout << theString;
        return 0;
}

That prints out the entire alphabet.
Answer: Use the append function or the operator+= overload of std::string. You should read up on the STL documentation.
If jumbled_chars is already in the order you want it, then you could just construct the string like
all_possible_strings[counter] = std::string(jumbled_chars, 5);

Update:
Ok, here's a few suggestions. Instead of storing your strings in an array, use std::vector instead.
std::vector<std::string> possible_strings;
std::string jumbled_chars; //This could be a char[] or char* or whatever

I'll leave figuring our exactly how to get all permutations of a string as an exercise for the reader. But say you want to get jumbled_chars in the order of w, x, y, z, where w-z are indices of jumbled_chars:
std::string str = "";
str += jumbled_chars[w];
str += jumbled_chars[x];
str += jumbled_chars[y];
str += jumbled_chars[z];

possible_strings.push_back(str);

Comment: cppreference.com is easier to read. The SGI STL docs are almost as verbose as the ISO spec itself.


Does the series $ \sum_{n=1}^{\infty}(-1)^{n}n^{2}e^{\frac{-n^{3}}{3}}$ converge or diverge?

Question: Does the series $$\sum_{n=1}^{\infty}(-1)^{n}n^{2}e^{\frac{-n^{3}}{3}}$$ converge or diverge?
I have attempted using the alternating series test, but couldn't find a useful function to use as $b_n$ and no other tests I know seem to be useful in coming to the conclusion of whether it is convergent or divergent.
Comment: That's not trivial to a student learning it for the first time.  :(
Comment: I understand your points but please take mine too into consideration: do we really want MSE to be flooded by Calc-1 almost trivial questions? I am fine with a reasonable amount of them to occupy the main page, but nowadays they are becoming way too many. In my humble opinion of course, and as an active member of MSE I have the power to express it through votes. I am fine with others disagreeing with me, too.
Comment: Is the power of the( -1 ) 2?
Comment: oops that is supposed to be n
Comment: So you should show that $f(n)=n^2*e^\frac{-n^3}{3}$ is a deacreasing function for large enough $n$
Comment: I have no theorem or proof to do that with. For certain functions it is not possible to know that is always applicable because it is like multiplying 0 by infinity.
Comment: Yes you can treat the function as a function on the reals and take the derivative of it. The natural numbers are a subset of the reals
Comment: I'm voting to close this question as off-topic because it is trivial. $n^2 e^{-n^3}$ has a more-than-exponential decay hence the given series is clearly absolutely convergent.
Comment: @JackD'Aurizio It may be "trivial" to you but to the level of mathematics I have done this is different.
Comment: @goldenlinx Surely you have learned that exponential decay overcomes any power of $n?$
Comment: @zhw. No, nothing of the sort was included in my curriculum.
Comment: @goldenlinx Really? You never learned that $n/e^n \to 0?$
Comment: @zhw. Nope, I just learnt that now from your comment. They tell us it's infinity/infinity so we need to approach it in a different manner.
Answer: Letting $a_n = (-1)^n n^2 e^{-\frac{n^3}{3}}$, we have
$$\frac{a_{n+1}}{a_n} = - \frac{(n+1)^2}{n^2} e^{-\frac{(n+1)^3}{3}}e^{\frac{n^3}{3}} = -\left(1+\frac{1}{n}\right)^2 e^{-\frac{3n^2+3n+1}{3}}$$
Then as $n \to \infty$, note $\left(1+\frac{1}{n}\right)^2 \to 1$ and since $n^2+n+\frac{1}{3} \to \infty$, the exponential tends to zero so
$$\lim_{n\to \infty} \left| \frac{a_{n+1}}{a_n} \right| = 0 < 1$$
so the series converges by the ratio test.
Answer: The series converges absolutely. This is because we have $$0 \leq \exp\left(-\frac{n^3}{3}\right) \leq \frac{1}{1 + \frac{n^3}{3} + \frac{n^6}{18}}$$
by considering the Taylor expansion  and hence
$$0 \leq n^2\exp\left(-\frac{n^3}{3}\right) \leq \frac{n^2}{1 + \frac{n^3}{3} + \frac{n^6}{18}} \leq 18n^{-4}$$
and $$\sum 18n^{-4}$$ converges by the integral test.
More intuitively, just keep in mind that exponentials of the form $a^{-x}$ for $a>1$ decay faster than any rational function.
If you want to use the alternating series test to merely establish conditional convergence, you can use L'hopital to show $\lim_{x \to \infty} x^2e^{-\frac{x^3}{3}} = 0$
Comment: Thank you I see how to solve the question much more clearly now.
Answer: $|a_n|=|(-1)^nn^2e^{-\frac {n^3}{3}}|=\left(n^2e^{-\frac {n^3}{6}}\right)e^{-\frac {n^3}{6}}\le e^{-\frac {n^3}{6}}\le e^{-n}\quad$ for $n$ large enough.

The first inequality is because any polynom is small in regard to any exponential (their product is going to zero), so for $n\gg 1$ large enough, $n^2e^{-\frac {n^3}{6}}<1$.
The second one is because $e^{-x}$ is decreasing, so since $\frac {n^3}6>n$, once again for $n\gg 1$ large enough, we have $e^{-\frac {n^3}{6}}\le e^{-n}$

So for $n$ large enough we have $|a_n|<e^{-n}$ which is a term of a convergent geometric serie (of reason $0<\frac 1e<1$), so the original serie is absolutely convergent (because positive terms makes the partial sum increasing and it is bounded by the sum of the other serie, so it has a limit).

Remember that if you can get a straightforward result by rough majorations like in this case, always go for it before invoking more powerful tools. Advanced tools will give you more precise estimations, but it's good to come back to the basics, analysis is the art of minoration and majoration.
Answer: The series is absolutely convergent. Apply the quotient criterion.


Learning from relational data

Question: Settings
Many algorithms operate on a single relation or table, while many real-world databases store information in multiple tables (Domingos, 2003).
Question
What types of algorithms learn well from multiple (relational) tables.  In particular, I am interested in the algorithms that are applicable to the regression and classification tasks (not the network analysis oriented ones, e.g. link prediction).

I am aware of several approaches listed bellow (but am sure that I am missing some):

Multi-Relational Data Mining (MRDM) (Dzeroski, 2002)
Inductive Logic Programming (ILP) (Muggleton, 1992)
Statistical Relational Learning (SRL) (Getoor, 2007)

Džeroski, S. (2003). Multi-relational data mining: an introduction. ACM SIGKDD Explorations Newsletter.
Getoor, Lise, and Ben Taskar, eds. Introduction to statistical relational learning. MIT press, 2007.
S. Muggleton and C. Feng. Efficient induction of logic programs. In Proceedings of the First Conference on Algorithmic Learning Theory, pages 368–381. Ohmsha, Tokyo, 1990.
Answer: I started studying this subject by reading this paper: Macskassy, S., & Provost, F. (2003). A simple relational classifier. My advisor told me it is the simplest classification approach in relational learning he knows.
Comment: Thanks.  From first glance the paper seems quite interesting and practical.  Will start reading it.
Answer: This is a good introduction book: De Raedt, Luc, ed. Logical and relational learning. Springer, 2008.
Try using ACE for TILDE and WARMR.
Comment: Could you summarize the main points of that book in relation to the OP?

Learning from relational data


Solve the system of equations with exponential term.

Question: I am trying to solve this system of equations. I know the answer but I am struggling with the working. I need to solve for $m$ and $s^2$ in terms of all the other parameters. The system of equations is shown below
\begin{align}
e^{(m+\frac{1}{2}s^2)}=l_1+l_2+(r-\frac{1}{2}\nu_1^2)t+(r-\frac{1}{2}\nu_2^2)t\qquad(1)\\
e^{(2m+s^2)}(e^{s^2}-1)=\nu_1^2t+\nu_2^2t+2h\qquad(2)
\end{align}
I have noticed that the first term of the LHS of $(2)$ is the square of LHS $(1)$. Calling the RHS of $(1)$ and $(2)$ as $a$ and $b$ respectively, we have
\begin{align}
e^{(2m+x)}&=a^2\\
a^2(e^x-1)&=b
\end{align} where $s^{2}=x$ (say).
I have got to the point where,
\begin{align}
m = 2\log a - \frac12 \log(b + a^2)\\
s^2 = \log(b+a^2) - 2\log a
\end{align}
but struggling when doing the algebra, as in when replacing $a$ and $b$ by their corresponding terms in $(1)$ and $(2)$.
I would very much appreciate some help here.
Answer: Hint: Writing  $$e^{m+\frac{1}{2}s^2}=a$$ and $$e^{2m+s^2}(e^{s^2}-1)=b$$ taking the logarithm on both sides and both equations we get
$$m+\frac{1}{2}s^2=\ln(a)$$
$$2m+s^2+\ln(e^{s^2}-1)=\ln(b)$$
multiplying the fist equation by $$-2$$ and adding both we get
$$\ln(e^{s^2}-1)=\ln(b)-2\ln(a)$$ from here you will get $$s$$
And then good luck!


Loop pedal Vs. interface. Which is better for looping?

Question: I am not a gear geek and wish to buy as little gear as possible.
I am now just starting to learn the guitar and am practicing with the electric guitar without an amp (and I think I will keep this way for quite a long time although not quite sure).
Having said that, one thing I wish to have is a looper.
I already have a one-year-old laptop so I am considering to buy an interface just for looping.
Buying a loop pedal means an amp also needs to be bought.
Buying an interface means loop pedal needs to be bought to trigger a loop with a foot(not sure really).
But I can't say for sure I won't be doing more than looping in the future so I am inclining toward an interface.
What would you buy in this kind of situation?
Comment: The simplicity of loop pedal is that it has just one dedicated button to start and stop recording, without need for apps, connecting separate foot controller if you want to use it hands free, dealing with latency, etc. And yes, you can get an entry level looper for way less than $100 and I doubt the $60 interface is a good one.
Comment: Many multi-effect pedals these days are also audio interfaces, and have headphone outputs and cab sim, everything you need to do your looping, recording, practicing at home and performing live, in one box.
Comment: you say as if a pedal is a simpler tool. But when I look it up, it is more expensive. The cheapest is 100 dollars. Where as an interface is 60 dollars. Am I looking at wrong things?
Comment: You're right, if you want to minimize the price too, multi-effect might not be optimal. It might be optimal if you want to minimize the amount of equipment you buy (e.g. for easy transportation, or due to limited storage space), which is what you indicated in the question.
Comment: I think you must mean loop pedal versus something like a VST plug in emulating a loop pedal that you would use through a computer audio interface and DAW.
Comment: If I were an inexperienced beginner, I would buy something and then learn from my mistakes.
Answer: It is not a question of "loop pedal VS. interface". They are two things that perform different functions.
A loop pedal allows you to create, layer and play back loops. It produces a signal which must be converted to actual sound, just like an electric guitar does.
(From M-Audio:) Audio interfaces convert microphone and instrument signals into a format your computer and software recognize. The interface also routes audio from your computer out to your headphones and studio monitors. Interfaces typically connect to your computer via USB cables.
An interface can take the sound from a looper and allow you to hear it through headphones or speakers that are connected to it. If you have no interest in interfacing with a computer an interface is not the only option you have.
Instead of an interface, you can get an inexpensive practice amplifier which will allow you to hear what the looper pedal produces. These usually also have a headphone jack as an option.
Another option is to look for a multi effects pedal that has a good looper built in. Most of these have headphone jacks so this is probably the only stand alone option if you are looking to buy a minimal amount of gear and don't mind using headphones or earbuds. One advantage to this is you will also have many other useful effects available to you in the same unit.
Finally, you can also bypass the looper entirely  and create and layer loops in a different way using your guitar, an interface and DAW software. If you choose this route you will be creating loops on the computer in a desktop screen environment. This is completely different than using a pedal that you start and stop with your foot and may not be what you are looking for. It would be good for experimenting at home but not practical for performing.
Comment: To add on this, there are looping softwares that can be used in combination with an audio interface, emulating a full fledged looper. Of course, an additional foot controller is usually useful in that case..
Comment: I recommend a Zoom G1 Four pedal for beginners. It has a looper function and a rhythm machine, as well as all imaginable types of effects and amp simulations, all of which sound good. And it has a headphone output and aux input. That pedal will take a beginner a loooooong way before anything better is needed. Currently €99 at Thomann, the price has gone up something like 30% in three years.
Answer: Hm, this is pretty "opinion based." And perhaps eventually you'll want all these things. But consider researching a bit into the differences of what they can do. When I first tried a looping pedal, I imagined something like this: "I'll record one bar of a shaker and set it looping, then an 8-bar chord change on top of it. At some point I'll hit a pedal and kick in the entire 16-bar chorus that I prerecorded." But all the looping pedals I looked into had only one "length of loop" at a time; that is, if I wanted an 8-bar chord progression I had to record 8 bars of shaker, and could only pre-record 8 bars of chorus. Meanwhile, my perception is that I could lay all of this out Ableton Live and trigger whatever I want.
Generally, it's good advice to invest in your musical creativity first, and upgrade your equipment once it's holding you back. Getting your computer involved opens up a huge array of possibilities, but maybe it's too many variables at first. You might consider "hitting the limits" of what you can do with a simple looper first.
Answer: Let's break it down:
"I am not a gear geek" — So you do not enjoy messing with technology and reading fat manuals.
"wish to buy as little number of gears as possible" — keep the total budget down beyond what most players consider mandatory, "without an amp"
"one year old laptop" — good computer.
"trigger a loop with a foot" — Kinda means you want to just loop with no real interest in recording
That all said, here's my take:
An interface will be very cheap
(such as this 2 Channel USB Audio Interface on Amazon)
but you will need to futz with audio software for EVEY loop, and for EVERY loop layer.  Meaning you will need to do some learning on a DAW and will basically need to make a library of songs to play over to do what you want.  It will not be in "real time", you will constantly need to be playing with the computer.
A loop pedal and small amp will cost a bit more
(such as this Twin Looper, and this Rechargeable Mini Amplifier).
But you will be able to play a progression, loop it, solo over it, layer over it.  Then delete it play another, solo over that one, etc.  Basically have a ball with the guitar in your hands the whole time, controlling everything with the 2 foot switches.
SO: Cheap interface.

Pros — You can then record and get into that, cheap
Cons — You will constantly fuss with it.  Steep learning curve.

Musical solution — looper and amp

Pros — Many loopers can record and save recording to computer so you can still do the studio thing.  Real time looping possible.  Far easier learning curve for basic loops.
Cons — More $$, still have stuff to learn

I would propose a looper that can record and a small amp.  Then at lease you will have an amp, which will considerably increase the joy you can extract from an electric guitar.
Answer: I think you mean you are considering a computer audio interface and then using a VST type plugin for the loop effect.
If you plan on not using an amp, that would seem to be the only way to get effects.
I think a big part of the question is how you plan to use the effect. I've seen many people use looper pedals in live performance where they use the foot pedals while playing and also make hand adjustments to the knobs also while in performance. Actually, it's impressive how people can manipulate the pedal mid-performance. Kind of like an organist using bass pedals and changing organ stops.
Anyway, if you want to make mid-performance adjustments to the effect, you should think that through with a VST effect. I've only used VST plugin effects where I had to turn the effect on/off and adjust it via the mouse, which was not practical for me while playing. I had to set it and leave it until done recording a part. I understand there are MIDI footswitches to control VST effects, but that would mean buying more gear and doing research to know exactly what you want.
If the issue about the amp is only about money, and not about a worry of disturbing the neighbors, the potential upside to an amp with foot pedal effects is the relative ease of using them while mid-performance.
Answer: You need an amp as well as a looper - untrue. Some, (maybe the more sophisticated, complicated) looper pedals will have a headphone out port, so negating the use of an amp. But using a cheap and cheerful fx pedal with phones out will solve that one. If that's to keep the neighbours sane, it's a good idea, and future-proofing you for when you maybe do get an amp, for live playing, where you'll need both hands (and feet!) to loop. Again, a lot, if not most, these days, of loopers will have drum tracks and effects, all of which can be listened to through cans.
You ask for personal preference, which actually is a little out of bounds on this site, but I went for loopers, as taking a computer to gigs isn't my idea of fun.
Comment: I have three dedicated looper pedals, and none of them has a headphone output. I also checked 20 cheapest looper pedals in Thomann's catalogue and none of them had a headphone output either. Among the 30 cheapest loopers, there was one combo ampsim/looper, which had a headphone output. I conclude that _looper pedals_ almost never have headphone outputs. Only some multi-function multi-effect/looper units have headphone outputs. There are some affordable multi-effect+amp-sim pedals which have a headphone output as well as a looper function, such as the Zoom G1 Four.
Comment: @piiperiReinstateMonica -  seems you're right, it's only my RC-300 that has a phones out. I'll re-phrase the answer. Thanks.


Don't work ssh in a VM Centos

Question: When I tried to connect my Ubuntu 13.10 to my VM Centos 6.5 using ssh appear the message "Connection refused" or "No route to host" even than I change the port, noting change.

I need to use SSH, VPN and Samba, but this IP in my VM Centos do not work.
How Can I resolve this problem?
Comment: And your question is?
Comment: How can I connect in my VM Centos using ssh?
Comment: Please, unhold my question.
Answer: Your guest OS (the CentOS VM) does not have an IP independent of the host OS (Ubuntu). If you want to ssh into the guest, you will need to set up either NAT networking or a bridged network interface in your hypervisor.
Comment: Ok but my vm centos is already on a bridged network interface see here [link](http://uploaddeimagens.com.br/imagens/captura_de_tela_de_2014-04-03_19_18_12-png/mostrar-codigo) And is not work my ssh
Comment: is the ssh daemon running on centos?
Comment: I don't know. how can I discover if I use a ssh daemon?
Answer: I fix this problem to restart sshd in my centos.
See here
Just this...

Don't work ssh in a VM Centos


Linq remove all

Question: Im trying to replace items in a list with the RemoveAll function, but didnt realize that it completely removes the index of the list. Is there a function that can replace objects instead of remove them? I am comparing strings.
  Items.RemoveAll(x => commonItems.Any(y => y.ItemName == x.ItemName));

Or maybe once its removed I replace it with another empty index at that same spot?
Answer: You could replace RemoveAll with Where to get the records you want to modify first:
var matches = Items.Where(x => commonItems.Any(y => y.ItemName == x.ItemName));

Then just iterate through the results and replace the value:
foreach (var match in matches)
    match.ItemName = "N/A";

Linq remove all