Build falhando em projeto PHP no Travis-CI

Question: Estou tentando fazer builds noturnas de um projeto em PHP no Travis-CI mas estou apanhando. Minhas builds estão falhando porque o Travis-CI diz não encontrar o composer. Há mais detalhes no log abaixo.
Using worker: worker-linux-docker-c22a70ed.prod.travis-ci.org:travis-linux-3

Build system information
Build language: ruby
Build image provisioning date and time
Thu Feb  5 15:09:33 UTC 2015
Operating System Details
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.5 LTS
Release:    12.04
Codename:   precise
Linux Version
3.13.0-29-generic
Cookbooks Version
a68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e
GCC version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

LLVM version
clang version 3.4 (tags/RELEASE_34/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Pre-installed Ruby versions
ruby-1.9.3-p551
Pre-installed Node.js versions
v0.10.36
Pre-installed Go versions
1.4.1
Redis version
redis-server 2.8.19
riak version
2.0.2
MongoDB version
MongoDB 2.4.12
CouchDB version
couchdb 1.6.1
Neo4j version
1.9.4
RabbitMQ Version
3.4.3
ElasticSearch version
1.4.0
Installed Sphinx versions
2.0.10
2.1.9
2.2.6
Default Sphinx version
2.2.6
Installed Firefox version
firefox 31.0esr
PhantomJS version
1.9.8
ant -version
Apache Ant(TM) version 1.8.2 compiled on December 3 2011
mvn -version
Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)
Maven home: /usr/local/maven
Java version: 1.7.0_76, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-oracle/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix"

$ git clone --depth=50 --branch=master git://github.com/rgiaviti/laravel-correios-track.git rgiaviti/laravel-correios-track
Cloning into 'rgiaviti/laravel-correios-track'...
remote: Counting objects: 69, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 69 (delta 14), reused 0 (delta 0), pack-reused 20
Receiving objects: 100% (69/69), 11.17 KiB | 0 bytes/s, done.
Resolving deltas: 100% (14/14), done.
Checking connectivity... done.
$ cd rgiaviti/laravel-correios-track
$ git checkout -qf e2c8293aa5a996f5d8bbdef6788e4e2d84c15c1e

This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.
If you require sudo, add 'sudo: required' to your .travis.yml
See http://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details.
$ rvm use default
Using /home/travis/.rvm/gems/ruby-1.9.3-p551
$ ruby --version
ruby 1.9.3p551 (2014-11-13 revision 48407) [x86_64-linux]
$ rvm --version
rvm 1.26.10 (latest-minor) by Wayne E. Seguin <firstname.lastname@example.com>, Michal Papis <email@example.com> [https://rvm.io/]
$ bundle --version
Bundler version 1.7.6
$ gem --version
2.4.5
$ composer self-update
/home/travis/build.sh: line 41: composer: command not found

The command "composer self-update" failed and exited with 127 during .

Your build has been stopped.

O que eu percebi é que o Travis-CI está setando o ambiente para Ruby ao invés do ambiente em PHP. Mesmo eu configurando o arquivo .travis.yml para usar o ambiente PHP, parece o que na hora da build, o Travis-CI ignora minha configurações. Abaixo está o meu arquivo .travis.yml.
language: php

php:
  - 5.4
  - 5.5
  - 5.6
  - hhvm
  - hhvm-nightly

before_script:
  - composer self-update
  - composer install --prefer-source --no-interaction --dev

Já vasculhei toda a documentação do Travis-CI e postei essa mesma questão no Stackoverflow em inglês. Ainda não achei solução. Também configurei o Travis-CI para somente fazer a build caso o .travis.yml estivesse presente e ele faz o build. Sendo assim creio que o Travis-CI está lendo o .travis.yml.
Outro ponto é que verifiquei em outros projetos para ver como deve ser o arquivo .travis.yml e aperentemente está tudo correto.
Não faço idéia se o problema está no próprio Travis-CI ou no meu arquivo ou alguma configuração que eu não verifiquei.
Answer: Entrei em contato com o suporte do Travis-CI explicando meu problema. Eles checaram meu arquivo .travis.yml e viram que havia um caracter oculto antes da tag language: php. Isso fazia o Travis-CI não detectar a linguagem e então era iniciado a build com ambiente Ruby.
Recriei o arquivo .travis.yml removendo o caracter oculto e a build funcionou perfeitamente.


Query a postgres database for the SQL which generates the tables

Question: We want to replicate our local test DB automatically, given the structure of the production DB, on the fly / on demand.
What would be nice, would be to query the prod DB and retrieve the SQL that generates the tables/views and then run that SQL against a cleaned out local DB.
Or perhaps there is a better way to replicate a production DB on a local machine?
What is the best way to do this?
Comment: Possible duplicate of [How to generate create script of table using SQL query in SQL Server](http://stackoverflow.com/questions/4213687/how-to-generate-create-script-of-table-using-sql-query-in-sql-server)
Comment: You are approaching this from the wrong end. Your production database should have been created from SQL scripts stored in a version control system - including all schema changes. If you need to setup a test database, run those scripts from the version control system. Tools like Liquibase or Flyway will help you organize and track the execution of those scripts
Comment: @johan855: Not a duplicate. Postgres is not SQL Server .
Comment: @horse i sort of disagree - the truth ultimate lies in the DB itself not in VCS, there is no need to put SQL in VCS, that might be overkill?
Answer: If I understood your question correct, you need to copy DB schema. You can use pg_dump with --schema-only parameter to do this. It'll only dump schema of the database, and you can import it locally.
$ pg_dump mydb --schema-only > mydb-schema.sql


Secure file transfer over Internet to Win2k3 Server/64-bit?

Question: I've been using FTPS via Filezilla on my Win2k3 server after a mandate for encrypted file transfer came through. I use Beyond Compare or other FTPS clients on my Windows 7 and XP boxes to connect.
Is this the fastest encrypted option for moving files to and from the Win2k3 box?
Answer: To get the fastest you're really going to have to run benchmarks. Different means produce different results depending on the actual data being transmitted. However, I'd personally be looking into a VPN of some sort between the systems. If the data being transmitted is a "refresh", rather than all new material you should also look into using rsync or similar, so as to only transmit the changes.
Comment: There is already a VPN that I'm going over, but it's enterprise wide, so I need to encrypt within that from server to workstation. It's largely script files.
Comment: @Caveatrob, as script files are generally quite small I don't think you'll see much difference regardless of which method you use but any method which offers compression should work best, as scripts are generally very compressible. I'd still be considering rsync (over SSH).


Set size of child items of arc menu

Question: I downloaded arc menu library from https://github.com/daCapricorn/ArcMenu.
Here is arc_menu.xml:
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
android:layout_width="wrap_content"
android:layout_height="wrap_content" >

<com.example.arcmenu.lib.ArcLayout
    xmlns:custom="http://schemas.android.com/apk/res-auto"
    android:id="@+id/item_layout"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:layout_alignParentBottom="true"
    android:layout_alignParentRight="true"
    app:childSize="44px"
    custom:fromDegrees="270.0"
    custom:toDegrees="360.0" >
</com.example.arcmenu.lib.ArcLayout>

<FrameLayout
    android:id="@+id/control_layout"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_alignParentBottom="true"
    android:layout_alignParentRight="true"
    android:background="@drawable/button_option" >

    <ImageView
        android:id="@+id/control_hint"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content" />
</FrameLayout>

here is FragmentActivity :
public class FregmentActivity extends FragmentActivity  {
  private static final int[] ITEM_DRAWABLES = {
        R.drawable.button_contact_us_option,
        R.drawable.button_back_bookmark, R.drawable.button_news_option,
        R.drawable.button_product_option,
        R.drawable.button_featured_option,
        R.drawable.button_about_us_option };
ArcMenu arcMenu;

@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);
    arcMenu = (ArcMenu) findViewById(R.id.arc_menu);

    initArcMenu(arcMenu, ITEM_DRAWABLES);
}

private void initArcMenu(ArcMenu menu, int[] itemDrawables) {
    final int itemCount = itemDrawables.length;
    for (int i = 0; i < itemCount; i++) {
        ImageView item = new ImageView(this);
        item.setImageResource(itemDrawables[i]);

        final int position = i;
        menu.addItem(item, new OnClickListener() {

            @Override
            public void onClick(View v) {
                Log.i("TAG", "IN ARC MENU");

            }
        });
    }
}
}

here is main.xml :
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res/com.example.vjmceramics"
android:layout_width="wrap_content"
android:layout_height="wrap_content" >

<LinearLayout
    android:id="@+id/content_frame"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:orientation="vertical" >
</LinearLayout>

<com.example.arcmenu.lib.ArcMenu
    android:id="@+id/arc_menu"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:layout_alignParentLeft="true"
    android:layout_alignParentRight="true"
    app:fromDegrees="180.0"
    app:toDegrees="270.0" />

Whenever I run my project to nexus4 it shows child icons very small.
and shows different size of icon on different device.
Is there any way to set child size as per device runs on???
please help me. Thank you
Comment: You can do that in **ArcMenu.java** (instead of in **arc_menu.xml**). Since you must set the size in **px** instead of **dp**, you must scale the px to dp in code.
Answer: 1.remove the icons from the drawable folder(ex. drawadle-ldpi,drawable-hdpi,drawable-xhdpi etc. )
2.keep only one copy of the icon (that u want to show) in the drawable-nodpi folder.
note- if the folder is not present in the res directory then create it

Set size of child items of arc menu


Euclid algorithm greatest common factor

Question: Let A be the greatest common factor of 9883529 and 759345
Find A using Euclid's algorithm, and hence find integers x and y so that A = 9883529 x + 759345 y.
how do you use Euclid's algorithm ?
Comment: There's an [example of Wikipedia](https://en.wikipedia.org/wiki/Euclidean_algorithm#Description) which might be helpful.
Comment: thank i have read it but i still don't get it much due to my english isn't that good
Comment: what do you mean by $g$?
Answer: $9883529=759345(13)+12044$
$759345=12044(63)+573$
$12044=573(21)+11$
$573=11(52)+1$
$11=1(11)$
Hence the greatest common divisor will be $A=1$. Now Look at the reverse procedure:
$1=573-11(52)$
$11=12044-573(21)$
$573=759345-12044(63)$
$12044=9883529-759345(13)$
So by substitution we have:
$1=573-(12044-573(11))(52)=$
$-626288+573(573)=$
$=-626288+573(759345-12044(63))=$
$=-36099(12044)+434478397=$
$=-36099(9883529-759345(13))=9883529(-36099)+759345(469287)$
Hence $x=-36099$ and $y=469287$

Euclid algorithm greatest common factor


Exceptions handle

Question: I need to know witch of this two handling of exceptions is better and why?? Or some other way to do better.
try{
                if (String.IsNullOrEmpty(filePath))
                {
                    throw new ArgumentNullException("The path is null or empty.", "filePath");
                }

or
 try{
                if (String.IsNullOrEmpty(filePath))
                {
                    Console.WriteLine("The path is null or empty");
                }

Comment: Your second code doesn't has exception at all.
Comment: You are not handling exceptions in the samples you provided, you are validating that file path exists.
Answer: I would recommend the second approach, except that you don't need the try/catch in this case. Exceptions are for handling exceptional cases. If you can prevent those cases with an if condition you should prefer this. If you are some performance optimization maniac you probably already know that throwing and catching exceptions could be more expensive compared to testing for conditions.
Answer: If not having the path breaks your whole application flow, throwing an exception is a good idea so that you can recover the state of the application for another try, or show a dialog to the user.
On the other hand, Console.WriteLine("The path is null or empty"); just notifies the user of the situation and moves on. If your execution flow is not harmed with, this will add minimal overhead to your application, thus preferable.
Answer: Use code contracts instead. Especially for things like preconditions (i.e., your example).
Answer: If the situation is an error to be expected, use return code or similar. Like a file not existing or such
If it is an unexpected situation (null as an argument ) use an excpetion.
Exception handling is expensive, so use wisely.
hth
Mario
Answer: Well, first of all in the second code you don't throw any exception, you simply write a message to the console. This means that the second one won't signal anything in a windows forms or wpf application or - worse - a web application.
Besides that, by throwing an exception you are able to intercept it on another stage of the application, and behaving correspondingly, whereas Console.WriteLine is only specific to the scope where you invoked it.
You might also want to consider whether throwing an exception is appropriate at all. This occurs in all cases where your code could gently handle the "exceptional" case, like by providing default values or such.
Answer: I can't say this for sure because the rest of your program isn't there; but I'm guessing the first option is going to kill your program, while the second is going to just print something to the console and then continue with the rest of your program.  So the answer to this question depends more on what YOU want it to do?  Want the program to die when this situation happens?  Throw the exception and don't handle it.  Want to reprompt the user when this happens?  Then use your second option, but turn your IF statement into a while loop, and just prompt until the condition is no longer true.

Exceptions handle


AWS sent me abuse report - apparently one of my servers was hacked - whats next?

Question: I got an abuse report from AWS with the following information:
* Log Extract:
<<<
suspicious looking connections on the Undernet IRC
Chat Network connecting from a netblock you control ( 18.104.22.168/17 ).
Log taken at 2010.Oct.17 11:47:04 PM PDT
firstname.lastname@example.com (Tampa)
[note- the address above is indeed of my server]
How can I detect what took over my server?
Answer: If your machine has been compromised and you didn't plan for that scenario then there's effectively nothing you can do. Reflect, repent, reboot, reformat, reinstall and restore your data only. Then apply all the vendor patches to bring your system up to current spec and harden your systems. Then learn how to use a host-based IDS effectively so you'll be able to recover more easily from any future attacks. Then carry out a careful review of the custom code /scripts you've deployed and try to make them more secure. Then maybe you can think about switching the service back on.


Making ClickOnce Updates Mandatory?

Question: Currently in an application I'm building I have it check for updates, and it gives the user the option to install or not to install the updates.
I want it to just automatically install the updates no matter what.
How can I make this change?
Comment: See the answer to my question here: http://stackoverflow.com/questions/7184141/clickonce-automatically-making-all-builds-required
Answer: Automatic Updates
Choosing a ClickOnce Update Strategy (Making Updates Required)
One options is to set up automatic updates to your software and mark them as required by setting the minimum required version in your deployment manifest.  You can set this in the properties dialog in Visual Studio, or through configuration with the following tag: <deployment install="true" minimumRequiredVersion="126.96.36.199">.  The minimum required version checks the version of your assembly, and if it is not at least what you specify here it will force an update.
Programmatic Updates
How to: Add On-Demand Programmatic Update
Another option that will allow you more control of when and how often the update occurs is to do the updates programmatically.  You can use the ClickOnce API to check for any updates on the deployment server and install them once your application has been run.  You can have far more control over what updates should be installed, how they are installed, and how you notify the user of the updates.  If your application is generally a long running instance, you could also set up timers to run in the background every so often to do the updates as well.
Here is an example implementation that polls on an interval: Example.
You can also combine the above two update methods.
Comment: Is there an way to automatically set the minimum required version to the most current version, or do you have to manually change this value each time you publish a new version?
Comment: @Derek: The original Example is no longer online, but it is in the [Internet Archive's Wayback Machine](http://web.archive.org/web/*/http://msdn.microsoft.com/en-us/library/cc558902.aspx).

There's also this other example on MSDN: [How to: Check for Application Updates Programmatically Using the ClickOnce Deployment API](https://msdn.microsoft.com/en-us/library/ms404263.aspx).
Comment: I believe setting the minimum required version will give the user the option to upgrade, or close the app.
Comment: I've appended the answer with another option for you to look at.  You could create build tasks that automatically update the minimum required version within the ClickOnce deployment manifest for you as part of your build / deployment process as well.
Comment: Nate -- it does not give the user any chance to say yes or no, it just installs the update.
Comment: Soo -- no, but that is a feature request I have turned in to the ClickOnce product team.
Comment: Great answer but unfortunately the links are dead
Answer: Simply you can achieve by adding below <target> tag in your project's .csproj file.
<Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

    // Other Tags ...

    <Target Name="AutoSetMinimumRequiredVersion" BeforeTargets="GenerateDeploymentManifest">
        <FormatVersion Version="$(ApplicationVersion)" Revision="$(ApplicationRevision)">
            <Output PropertyName="MinimumRequiredVersion" TaskParameter="OutputVersion" />
        </FormatVersion>
        <FormatVersion Version="$(ApplicationVersion)" Revision="$(ApplicationRevision)">
            <Output PropertyName="_DeploymentBuiltMinimumRequiredVersion" TaskParameter="OutputVersion" />
        </FormatVersion>
    </Target>

    // Other Tags ...

</Project>

It will automatically make your each publish/update mandatory without doing any extra stuff.
Answer: The "Auto Update Project's Minimum Required ClickOnce Version" project has a nuget package that updates the ClickOnce settings for you.
https://aupmrcov.codeplex.com/ [ archived ]
GitHub:
https://github.com/deadlydog/AutoUpdateProjectsMinimumRequiredClickOnceVersion
Nuget:
https://www.nuget.org/packages/AutoUpdateProjectsMinimumRequiredClickOnceVersion
Comment: This deserves more love.. and a GITHub page
Answer: If you specify minimum required version same as the current publish version, your application will update as soon as clients open it.
1-Just right-click on the project name
2-Select Properties
3-Click Publish (Note publish version)
4-Click on Update button
5-Select "Specify minimum required version for this application"
and
Finally, make version number same as the "Publish Version".

Making ClickOnce Updates Mandatory?


Use the notion of Integers to find a solution for $a+x=b$, $a,b,x \in \mathbb{N}$.

Question: In my syllabus for "Scientific Computing" it states the following idea.

Consider the case $a \leq b$, $a,b \in \mathbb{N}$. This is essentially the same as $a + x = b$, $x \in \mathbb{N}$. Therefore, the only solutions to this equation are when $a$ is less than or equal to $b$. However, when we require for there to be a solution we need to introduce the idea of pairs, i.e. $(a,b)$ that describes $x$. However, it is clear to see that $(a,b)$ is not the only way to represent $x$. We also have $(a+1,b+1)$ which describes the same number. Therefore, we introduce equivalence relations that comes directly from the equation $a+x=b$:
$(a,b) \sim (c,d) \iff a+d=b+c$.

I'm just confused as to what this even means. How does $(a,b)$ describe $x$? If we have $1 \leq 5$, then the equation becomes $1+x=5 \Rightarrow x=4$. Then our equivalence relation becomes $(1,5)$ - so essentially the relation describes $x=b-a$? But then how does $(c,d)$ come into play?
Comment: If you are only interested in solutions in $\mathbb{N}$, you don't need pairs. If you want to find solutions for the case that $a > b$, then ordered pairs are used to construct $\mathbb{Z}$. You need ordered pairs to show that arithmetic operations are well-defined.
Comment: Wow!  I have seen very few things as horribly written as that excerpt.  I don't blame you for being confused.
Answer: From the Peano axioms you have constructed ${\mathbb N}=\{0,1,2,3,\ldots\}$ with $+$, $\times$, and $<\,$; and you have proven the usual rules of computing in this system. Now you want to enlarge ${\mathbb N}$ to the system ${\mathbb Z}$, which also contains the negative numbers, whereby the known rules are still valid.
A simple idea would be to create for each $n\in{\mathbb N}_{>0}$ its negative clone $-n$, and put these clones into the bag. But it has turned out that proving all the rules now is a hopeless undertaking, because of all the "cases". Instead one uses a certain algebraic trick, and your quoted text tries to indulge in this trick.
The idea is to represent all integer numbers, positive, $0$, and negative ones, by pairs of natural numbers. A pair $(a,b)\in{\mathbb N}^2$ then represents the solution $x$ to the equation $a+x=b$, i.e., the number that we usually call $b-a$. When $a\leq  b$ then $b-a\in{\mathbb N}$ is already present in our world, and we would not need this pair-complication. But when $a> b$ then the pair $(a,b)$ is a new kind of thing, representing $a-b>0$ "unit steps backwards".
Introducing these pairs creates a new difficulty: The same "new number" $x$, being $\geq0$ or $<0$, is represented by infinitely many pairs, e.g., $3=(2,5)=(16,19)$. This means that an actual "new number" is an equivalence class of such pairs. Some work is then needed  to install good operations $+$ and $\times$ on the set ${\mathbb Z}$  of these classes, whereby these operations have to give the same result as the  already present operations on the "old numbers" $\geq0$.
Comment: Nice, clear explanation,


Using Regression to project outside of the data range ok? never ok? sometimes ok?

Question: What are your thoughts about using regression to project outside of the data range? If we are sure that it follows a linear or power model shape, couldn't the model be useful beyond the data range? For example I have volume driven by price. We should be able to project for prices outside the data range I believe. Your thoughts?
<code>VOL     PRICE
3044    4.97
2549    4.97
3131    4.98
2708    4.98
2860    4.98
2907    4.98
3107    4.98
3194    4.98
2753    4.98
3228    4.98
3019    4.98
3077    4.99
2597    4.99
2706    4.99
3000    4.99
3022    4.99
3084    4.99
3973    4.99
3675    4.99
3065    4.99
3407    4.99
2359    4.99
2802    4.99
2589    4.99
2476    4.99
2387    5
3265    5
2039    5.14
1842    5.15
2660    5.37
1796    5.46
1734    5.46
1881    5.46
2204    5.58
1477    5.77
1620    5.84
1909    5.87
1744    5.87
1247    5.87
1848    5.88
1641    5.88
1758    5.88
1718    5.88
1656    5.88
1822    5.88
1556    5.89
1643    5.9
1850    5.91
1901    5.91
1837    5.91
1773    5.92
1729    5.92
</code>
Comment: Even when you're really, really, sure the linear relationship continues beyond the range of the predictors $x_1, \ldots, x_n$ in your sample of $n$ observations, there's a term in the variance of the predicted response for a new observation $x^*$ - viz $\frac{(x^* - \bar{x})^2}{\sum_i^n(x_i - \bar{x})^2}$ - that should worry you.
Comment: Depends on how well your assumptions hold outside the data range.  Predicting $y$'s for unobserved $x$'s is the whole reason you do regression in the first place.
Comment: Ben I tend to agree, isn't predicting y's for unobserved X's the whole point? Else why even do a regression? Maybe setting a limit on how far I allow myself to venture away from the observed data range could be responsible.  Surely 10% would be safe.. no?
Comment: My favourite yardstick on this topic is http://www.bmj.com/content/317/7155/409.
Comment: @Ben,@Johnson - Perhaps a quibble.  There is another use for regression.  It can be used to explain rather than predict.  I think, particularly in the social sciences this is a major use of regression.  I've read lots of arguments like We think that (variables) A effect outcome B, we run a regression, find that the 95% confidence interval of the  coefficient(s) of A don't contain 0 and we conclude that there is a relationship of the form A causes B.  Incidentally, this is not something I ever do !
Answer: Almost all answers and comments warn against the dangers of extrapolation. I would like to offer a more formal way of seeing whether prediction is prudent. The method is based on the projection matrix on the space spanned by the columns of $\mathbf{X}$ which we assume full rank, i.e. we assume the column space is p-dimensional. As you might remember,
$$\mathbf{H}=\mathbf{X}\left(\mathbf{X}^{T}\mathbf{X} \right)^{-1} \mathbf{X}$$
It can be shown that the diagonal elements of $\mathbf{H}$ satisfy $0<\mathbf{H}_{ii}<1,\ i=1,\ldots,n$, this is a consequence of idempotence by the way, and they can be interpreted as distances from the centroid of the predictor space. This is true because there is a one-to-one correspondence between the leverages $\mathbf{H}_{ii}$ and the squared Mahalanobis distances. A way to spot hidden extrapolations would then be to see how far the new obsevation lies from the centroid, right? This can be done by computing the new diagonal element. Recalling some basic rules of matrix multiplication, we have
$$\mathbf{H}_{new,new} = \mathbf{x}_{new}^{T} \left(\mathbf{X}^{T}\mathbf{X} \right)^{-1} \mathbf{x}_{new} $$
If $\mathbf{H}_{new,new}$ is much larger than the rest of the diagonal elements, then this tells you that your new observation lies quite far from the centroid and prediction is probably a risky move. It takes some judgement to decide how large is too large so of course the technique is not foolproof. Its beauty nevertheless is that it works in all dimensions, when you cannot look at a simple scatter plot that is.
I am not sure which software you are using but almost all of them will return the hat matrix with the right command. So I suggest you take a look before making up your mind.
Comment: Well done JohnK, this is very helpful. FYI I am using Excel regression.
Answer: The prediction error increases quadratically with the distance from the mean. The regression equation and results allow you to gauge the size of the error over the observed range of data, and the model is only adequate over that same range.
Outside of that range a lot of things can happen. First, the prediction gets worse and worse due to the increase of the prediction error.
Second, the model may break down completely. The easiest way to see that is to try to project a model relating price to time: You can't make predictions for negative time.
Third, the linear relationship may be inadequate. In your example, there almost certainly are economies of scale, which would become very noticeable if you try to predict far outside of the range of observed values.
A humorous example of this same effect appears in one of the works of Mark Twain, where he attempts to model the length of the Mississippi river over time --- it it/was quite windy and shortens/ed each year due to erosion of some of the bends as well as man-made shortcuts --- and "predicts" that in so many years the distance between Cairo, Illinois, and New Orleans will have shrunk to about a mile and three quarters).
Finally, note that the range of observed values can be quite complicated if you have more than one predictor variable. (Due to correlations between the predictors you often cannot just take the box defined by the maxima and minima in each predictor.)
Comment: (+1) Though to say the model is adequate *only* over the range of the observed data is a bit strong - it's that the problems you describe become more & more concerning the further away you get from it.
Comment: @Scortchi. Point taken. In most situations the degradation of the model is gradual. However, occasionally there *are* hard boundaries, and trying to go beyond those is going to cause grief.
Comment: @JohnsonJason: There's no sense in looking for a rule of thumb. You can easily calculate prediction intervals, assuming your model can be extrapolated; the degree to which you can trust extrapolation depends on subject matter knowledge: what's acceptable varies from case to case.
Comment: Excellent points (+1). But there is no logical problem in predicting price for negative time. The real problem is if you predict negative price for some given time (usually in the past, in practice). Often that means the model is qualitatively wrong as much as that extrapolation is stretching a line (or curve) too far. A logarithmic link function for example always implies positive predictions.
Comment: So is there any work around how far is a safe distance to venture away from the observed data range? Less than 1 standard deviation alright?
Answer: You cannot make data driven decisions for areas where you don't have data. End of story. The data can very well support a linear shape for the range of which your data is collected but you do not have data-driven reasons to believe this shape continues to be linear outside your range. It could be any shape under the sun!
You may assume the linear shape continues outside your data range but this is a subjective assumption not supported by the data you've collected. I would suggest consulted a subject matter expert to see, based on their subject matter expertise, how safe this assumption is.
Comment: So what's the point really of doing regression if we can't predict Y's for unobserved X's
Comment: I think the point is that you can still predict inside the range, it's just not advisable to predict outside the range. Presumably most new data points would be inside the range, so the model would remain useful the vast majority of the time


Win32 C API for redirecting minimize animation

Question: I have seen RocketDock redirect the minimize animation in Vista so windows minimize to the dock, and am just curious how this was done. Is the actual minimize animation redirected to the dock, or is something like a hook to stop Windows from minimizing the window and RocketDock has a custom animation when the window is minimized?
Answer: I am working on an open source multi-monitor taskbar project called "OpenMMT." I've recently discovered (through many headaches) how to accomplish this.
The following explanation assumes that you know how to go about using RegisterShellHookWindow.
On the window procedure that will receive the shell hooks, look for HSHELL_GETMINRECT.
Now, from here on out is where I had problems. According to MSDN, the lparam member passed contains a pointer to a "SHELLHOOK" object. Which is true, however, I could not get it to work for the simple fact that the "rc" member of that structure, is a RECT that differs from the actual RECT structure in the Windows header files. The RECT in the header files uses LONG for its members, were as on here, we want SHORT.
Anyways, here's a snippet on how I accomplished this.
Structures to define:
typedef struct {
  SHORT left;
  SHORT top;
  SHORT right;
  SHORT bottom;
} REALRECT, *LPREALRECT;

typedef struct {
  HWND hWnd;
  REALRECT rc;
} DOCUMENT_ME_RIGHT_MICROSOFT, *LPDOCUMENT_ME_RIGHT_MICROSOFT;

Then on the Window Procedure:
case HSHELL_GETMINRECT:
{
  LPDOCUMENT_ME_RIGHT_MICROSOFT lpShellHook = (LPDOCUMENT_ME_RIGHT_MICROSOFT)lParam;
  // lpShellHook now contains all the info. If you want to change the location
  // of the animation, simply change the lpShellHook->rc members to point
  // to the right coordinates and then return TRUE;
  return TRUE;
}

When minimizing programs from my application I encountered some instances where the animation would default back to the original one. I resolved this by minimizing them like so:
void MinimizeApp(HWND hWnd) {
  SetForegroundWindow(hWnd);
  ShowWindowAsync(hWnd, SW_MINIMIZE);
}

If you want more info regarding my project or you just want to peek at the source, feel free to do so at https://github.com/Fafson/OpenMMT
Comment: OK, thanks for your help - I will look into that. I had really only asked out of curiosity. I will definitely look into your project.
Answer: The ptMinPosition member of the WINDOWPLACEMENT structure specifies the coordinates of the window when it is minimized, so SetWindowPlacement function can be used to that effect. But some testing shows the window should not have a task bar button for that to work (i.e. no WS_EX_APPWINDOW).
I don't know how RocketDock works, but I guess this could be achieved by installing a global WH_CBT hook, and acting upon (setting the ex_style and then setting minimized coordinates) HCBT_MINMAX notification.
Comment: You can have a taskbar button even if you don't have WS_EX_APPWINDOW, see http://stackoverflow.com/questions/4758013/how-could-i-create-a-process-with-hiding-the-process-window-from-the-task-bar-i/4758050#4758050
Comment: Thanks, this is exactly the answer I wanted.
Answer: You can use the AnimateWindow API function, and pass it e.g. AW_HOR_POSITIVE | AW_VER_POSITIVE to get a diagonal animation.
I'd start with a global hook catching WM_SYSCOMMAND/SC_MINIMIZE, and use AnimateWindow to target the top right corner.
If this doesn't provide the desired effect, the next step would be to use WM_PRINT/WM_PRINTCLIENT to get a copy of the window into a bitmap (I believe this is what AnimateWindow does internally), then hiding the window and doing my own animation.
Comment: But how does RocketDock use that function to animate the minimization to the dock?


Problems creating a custom blank page with my custom module

Question: I'm trying to upgrade old Drupal 7 modules to 8 and I am struggling to figure out how to create a blank page (by this I mean, a page with zero HTML from enabled themes) for my module. I added routing to my custom controller, my controller method takes care of all the logic and returns the following array:
[
        'showVideo' => [
            '#theme' => 'videoPage',
            '#content' => $content
        ]
]

I already defined "videoPage" in mymodule.module using hook_theme:
Function mymodule_theme($existing, $type, $theme, $path){
  return array(
    'videoPage' => array(
      'variables' => array('content' => NULL),
      'file' => 'mymodule.html.twig'
    )
  );
}

"mymodule.html.twig" is a full stand-alone HTML file from the opening HTML tag to the closing tag. I don't want any other content to show up there but instead I'm getting the content in that Twig file, then followed by the HTML for the currently selected Drupal theme. Also, my {{ content.value }} calls in Twig seem to be getting ignored. And by this I mean, I can literally see {{ content.xyz }} in the markup being printed. The $content being returned by the controller is an array and I'm sure all the values are set. It almost feels like the Twig file is being called while the Controller's logic is being ignored. (this last bit has been solved by Clive's response)
Any help? Am I totally off in this approach?
Comment: I noticed that if in my Controller I return a Response object then I will be able to get a blank page. Perhaps I need to render the Twig template using the variables generated on my Controller, set the rendered HTML as the content of my Response object, and return that Response object?
Comment: I don't understand what you are trying to do in your question, but you are right in your comment. You can use a Response object to return any html from a controller, without anything added by drupal.
Comment: Basically, I need to return a Response object with only the HTML generated from the ONE Twig template I defined earlier using hook_theme. When people visit this custom page, the controller will send the user only the HTML generated from that Twig template and nothing else.
Comment: OK, now i know what you are trying to do, see my answer.
Answer: You can output the html of one twig only in a symfony response:
use Symfony\Component\HttpFoundation\Response;

public function my_page_controller() {

  $build = array(
    'page' => array(
      '#theme' => 'videoPage',
      '#content' => $content,
    ),
  );
  $html = \Drupal::service('renderer')->renderRoot($build);
  $response = new Response();
  $response->setContent($html);

  return $response;
}

Answer: The file parameter is:

The file the implementation resides in.

That should be used if you're creating a theme function, rather than template, and it will contain the path to the file that function is in.
What you're looking for is the template parameter:

If specified, the theme implementation is a template file, and this is the template name. Do not add 'html.twig' on the end of the template name. The extension will be added automatically by the default rendering engine (which is Twig.)

(emphasis mine)
So your implementation might look something like this:
function mymodule_theme($existing, $type, $theme, $path){
  return array(
    'videoPage' => array(
      'variables' => array('content' => NULL),
      'template' => 'mymodule'
    )
  );
}

Assuming "mymodule.html.twig" is in the root of your module's folder.
In case you're interested in why your current implementation is doing what it's doing: as you've provided a file, Drupal performs an include on it, and anything not within <?php tags gets written straight out to the buffer.
Source
Comment: Thanks for your quick response. The changes you suggested have fixed all the problems I was having with Twig. Twig is now being parsed and my variables (generated in my controller) are displaying correctly. But I'm still having the problem where the page is still displaying Drupal theme wrappers. My Twig file was supposed to be a standalone HTML page so I would like to make it so that's the only thing printed out by Drupal. Is there a way to remove the other theme templates on this custom page? HTML / Page etc..?
Comment: Using mixed case in a hook_theme might just confuse things. If you leave out template it will just convert your hook_theme template_name to template-name.html.twig. I'm probably not explaining well but most of core doesn't specify 'template' and it's good to follow that convention. Keeps things much more straightforward.
Comment: I prefer to be explicit personally @Cottser (probably something I get from Python). I'd rather know in 6 months that I specifically intended to use a template file rather than a theme function with the same name - it sits better with my brain :) Convention is cool, I like that too, but too rigidly following someone else's ideas can slow you down in my experience
Comment: Sorry I wasn't clear. The convention I was talking about was to use the same template name as your hook_theme key. If you want to include 'template' go for it :)
Comment: My fault @Cottser, one too many xmas White Russians probably! That makes perfect sense


using order by with union in inner select queries

Question: I want to use order by clause in inner select queries with union operator i.e.
select colname from table where <cond1> order by colname
union
select colname from table where <cond2> order by colname

I can use order by with the complete result but I want to have the results from first query ordered by and then the result of second query ordered by.
Please give me some suggestions upon it.
Comment: You have to take off first query's order by
Answer: Following should do... i guess
select colname,1 as OrderCol from table where <cond1>
union select colname,2 as OrderCol  from table
where <cond2>
order by OrderCol

Comment: This resulting another problem as there are some records that satisfy condition 1 as well as condition 2,so this causing the repetition of these records as they are not unique now as OrderCol have made them different records.Can you please suggest the solution for it.
Comment: i don't understand why are you using union if you're selecting data from same table. Can't you write simple select stmt and put both the conditions there with OR. And use a distinct in the select stmt to ignore duplicate entries.
Comment: I have to use union as I need to give priority to that results which are coming from the first condition over the second condition.
Comment: You don't need to use Union for that. you can use case statement i.e.- select col1,col2, case when  then 1 when  then 2 end ordCol from tbl1 where  or  order by ordCol (upvote the comment if its helpful)
Answer: the correct usage of Order by in unioned table is
select colname from table where <cond1>
union
select colname from table where <cond2> order by colname

the explanation of the code above is, after the the two queries had been unioned, it then Sorts based on the column name. It doesn't sort the lower query first before the server union it with the first query. But there is an alternative for that, you need to wrap it in a subquery.
SELECT newTable.colname
FROM
(
    select colname, 1 as OrderThis from table where <cond1>
    union
    select colname, 2 as OrderThis from table where <cond2>
) newTable
Order by newTable.OrderThis, newTable.colname

if i got your idea correctly, you want to sort the columns first before you union the two queries right and maintaining their correct position (first query results remains on top while the second query results remain below the first query)
Comment: This resulting another problem as there are some records that satisfy condition 1 as well as condition 2,so this causing the repetition of these records as they are not unique now as OrderThis have made them different records.Can you please suggest the solution for it.
Answer: This may do the trick
SELECT * FROM ( SELECT colname  FROM table1 where <cond1> order by colname ) DUMMY_ALIAS1

UNION ALL

SELECT * FROM ( SELECT colname  FROM table2 where <cond2> order by colname ) DUMMY_ALIAS2


Is the closure of $\{p\}$ equal to $V(p)$ in Spec $A$?

Question: Let $A$ be a commutative ring with unity and consider Spec$A$. Does the closure of $\{p\}$, $p \in$ Spec $A$, equal to $V(p)$ which is defined to be the set of all primes ideals containing $p$? Thank you.
Answer: That's right: $\mathfrak q\in\overline{\{\,\mathfrak p\,\}}$ means that  any elementary open set $D(f)$ for the Zariski topology which contains $\mathfrak q$ contains $\mathfrak p$ – in other words, if an element $f\notin \mathfrak q$, then  $f\notin \mathfrak p$.
Indeed, suppose  $\mathfrak p\not\subseteq \mathfrak q$. This means there exists an element $a\in\mathfrak p$ which does not belong to $\mathfrak q$. Then $D_a$ in an open neighbourhood of $\mathfrak q$, whereas $\mathfrak p\in V(a)$.
Answer: Another way to think of it is: the Zariski topology is defined so that the closed sets are $V(I)$ where $I$ is an ideal (it's also enough just to look at radical ideals). Therefore $\overline{\{p\}}$, which is a closed set, is equal to $V(I)$ for some ideal $I$. To say that $p \in V(I)$ means that $p \supseteq I$.
In fact, if $I$ is any ideal such that $p \in V(I)$ then $p \supseteq I$. A closure is necessarily the smallest closed set containing $p$. Thus, we want $p = V(I)$ with $V(I)$ as small as possible. How do we achieve this? We make $I$ as big as possible because $V$ reverses inclusions. So what's the biggest ideal $I$ such that $p \supseteq I$? Clearly $I = p$. Thus $V(p)$ is the smallest closed set containing $p$.


Erro de arumentos posicionais no Python3

Question: estou fazendo um código teste para o python, e estou tendo o seguinte erro:

Traceback (most recent call last):   File "teste.py", line 7, in

      g= teste('Digite um numero:  ') TypeError: teste() missing 2 required positional arguments: 'y' and 'z'

Minha ídeia de resolver este problema é apenas para fins de estudo mesmo.
Basicamente o que eu quero que o script faça é mostrar as perguntas das variáveis g, h, i e salvar os resultados em a, b, c respectivamente.
Depois quero que minha função retorne os valores armazenados.
Se puderem me ajudar sem alterar a estrutura do código, apenas para que
eu deixe ele funcionar de forma adequada.
def teste(x, y, z):
    a= str(input(x))
    b= str(input(y))
    c= str(input(z))
    return a, b, c

#Programa principal
g= teste('Digite um numero:  ')
h= teste('Digite uma frase:  ')
i= teste('Digite um texto:  ')

teste(x=g, y=h, z=i)
print(f'{a} {b} {c}')

Comment: Sem alterar a estrutura não tem como, ela está errada e não faz sentido. Você fez a chamada da função 4x, sendo que nas três primeiras passou apenas um parâmetro enquanto são esperado 3, por isso o erro. Você conseguiria explicar linha a linha, como um [teste de mesa](https://pt.stackoverflow.com/q/220474/5878) o que o seu código está fazendo?
Comment: Anderson, não conheço o teste de mesa, mas achei interessante, vou estudá-lo e montar meu código em cima dele para ver onde estou errando. A propósito, entendi o que você quis dizer com chamar 4x a função, valeu!! Quando eu disse que não queria mudar a estrutura do programa, me referi a não adicionar linhas ou sintaxes que não conheço, claro que quero deixá-lo funcional e será necessário mudar algo nesse quesito
Answer: Aparentemente o que você quer é definir uma função que lê do usuário três valores e as retorne nas variáveis a, b e c. Se você lerá os valores do usuário não faz sentido você passar parâmetros para a função.
def le_valores():
  a = input('Valor 1: ')
  b = input('Valor 2: ')
  c = input('Valor 3: ')

  return a, b, c

O retorno da função será uma tupla com os três valores lidos. A partir dele você poderá utilizar a atribuição múltipla para definir os valores fora da função:
g, h, i = le_valores()

print(f'{g}, {h}, {i}')

Exibindo, assim, os valores lidos pela função.
Comment: Muito obrigado Anderson,  isto mesmo que eu queria fazer, valeu pela dica!


Discrete Fourier Transform & GCD

Question: While reading Wolfgang Schramm's original paper concerning the relationship between the discrete Fourier transform and gcd, I came across the following condensed argument for his more general theorem relating the discrete Fourier transform of a particular type of arithmetic function to a Dirichlet convolution with the arithmetic function and the Ramanujan sum (see bottom of pg.3 in linked pdf):

A corollary of this fact is given as follows (see top of pg.4 in pdf):

Corollary. If $f$ is a multiplicative function, then $F_{f}(m,n)$ is also multiplicative in variable $n$.
Proof. The Ramanujan sum $c_{n}(m)$ is multiplicative in $n$ and the Dirichlet convolution
preserves the multiplicativity of functions.

As I'm not a number theorist by training, an answer to this post should accomplish the following:

Very carefully explicate, in full detail, the chain of equalities we observe in the condensed proof.
Similarly, explain how the corollary follows from the theorem on pg.2 of the linked pdf.
Bonus points if you can elaborate on why the inverse Fourier transform takes the form it does in the theorem on pg.2
Answer: By definition,
$$F_f(m,n):=\sum_{k=1}^n f((k,n))\cdot \exp(-2\pi i k m/n),$$
$$c_n(m):=\sum^n_{k=1,(k,n)=1}\exp(2\pi i k m/n),$$
and the convolution product is given by
$$(f\ast g)(n) :=\sum_{d|n}f(d)g(\frac{n}{d}).$$
The third definition gives the third equality immediately, and it is not hard to see that the second definition gives the second equality, with $n\sim n/d$ and $j\sim k$. Note that the minus sign just means we are taking the sum backwards, $\exp(-2\pi i k m/n) = \exp(2\pi i k (n-m)/n)$.
The first definition also gives the first equality directly, but here there is more to keep track of. But there is nothing really number theoretic going on, it's only combinatorics. If you write out the first above definition and group terms according to what the paper tells you to do, you arrive at the desired formula directly. I can provide more details if you want, but it gets a little messy if one wants to be careful.
The corollary is proven in the following way: The theorem shows that the discrete Fourier transformation is really a convolution product. Now, if we assume $f$ is multiplicative, then its discrete Fourier transformation is hence a convolution product of multiplicative functions (since it is a separate fact that $c_n$ is multiplicative). Finally, it is a standard result, and not hard to show directly, that the convolution product of multiplicative functions are multiplicative. This implies the corollary.

Edit: I'll add some more details. I don't think I can be more explicit in the two latter equalities, it is literally just checking that the definition is satisfied, symbol for symbol basically.
To see the first equality, the only non-trivial step is regrouping the $k$ in families where the $\gcd$ with $n$ is the same. So for each $d|n$, we collect all the indices $k$ that satisfies $(k,n)=d$. For all these entries, we see that $f((k,n))=f(d)$ agree. This can also be thought of another way: for each such $k$, write $k=d\cdot j$, where $(k,n)=d$ and $j$ is relatively prime to $n$. Of course, the possible values of $j$ lies between $1$ and $n/d$, as $1\leq k\leq n$. So we don't have to run over all $k$ satisfying $(k,n)=d$, we can reduce to going over all $j\leq n/d$ with $(j,n)=1$. We get a double sum: check that you are convinced of the bijection $\lbrace 1,2,\dots, n\rbrace \leftrightarrow \bigcup_{d|n} \lbrace 1\leq j\leq n/d\mid (j,n)=d\rbrace.$ You can then pull out the $f(d)$ term, as that is independent of $j$ when $d$ is given, and the remaining exponential term is exactly what you see in the proof.
One can check that $c_n$ is multiplicative directly, but a reference is Arithmetical Functions by Schwarz and Spilker, see p. $16$ here https://books.google.dk/books?id=12hjjwEACAAJ. The proof uses that the convolution product of two multiplicative functions is multiplicative, which I suspect is proved in the full book, but left out in the Google Books version. A proof of this fact is given here http://mathonline.wikidot.com/the-dirichlet-convolution-of-two-arithmetic-functions.
Comment: Yes! Any more detail you are willing to provide to your already insightful response is most welcomed. In particular, if you can explicitly spell out the regrouping of the terms that goes on in the first equality, that'd be great; I'll try to work on the other two, but again more detail is always welcomed. Also, if you could point me towards a proof of the multiplicativity of $c_{n}$ as well as the result concerning the convolution product of multiplicative functions, that would be much appreciated.

Discrete Fourier Transform & GCD


WPF/LINQ data layer design question

Question: I'm working through the "Data Binding" chapter in Pro WPF in C# 2008.
They have you set up this class:
public class StoreDB
{
    public Contact GetContact(int id) {...}
    public List<Contact> GetContacts() {...}
}

The idea is to call these methods, getting either a Contact or a List<Contact> and bind these to the appropriate controls using LINQ on the latter method to filter/sort your objects.
This all makes sense.
However, what happens when you have 100,000 contacts and you want to get 3 of them? Your GetContacts() method gets 100,000 and your LINQ picks out three of them?
Isn't this super inefficient?
How are real-world applications built to avoid this?
Answer: I would recommend a design that does not have GetContacts returning List<T>. I'm not entirely sure that you would like to use List<T> because the collection you are returning is mutable. In WPF if you'd truly like a mutable list of items try ObservableCollection<T>.
However, I think the best interface would return an IEnumerable<T> which is the result of your SQL or LINQ query. This will provide an immutable collection of your contacts to the binder, which can be filtered with LINQ to Objects simply and easily.
Comment: Does IEnumerable return the data all at once, or does it act like IQueryable and only process the data when it is accessed?
Comment: IEnumerable data access is based on the implementation. It could be deferred or it could be immediate.
Answer: I dont know the book your using but the question here is what database are you using if its SQL then you can use LINQ to SQL and add a dbml file to your project through the VS designer, then just drag the tales you want to use from the Sever Explorer view onto the the dbml and all your bussness object classes will be auto generated.
From there you can use the dataContex created and work with linq to filter the results before they are fetched.
What you have been doing right now is filtering the objects you fetched through a diffrent means.
HTH,
 Eric
Answer: Since you have been asking may WPF and LINQ quetions i thought you would really want to take a look at some good vido traning for WPF and LINQ or ADO.net,
this is an excelt source to start learning very straight forward.
Vido Traning WPF
HTH,
  Eric
Answer: Welcome in the Linq world!
You want to return an IQueryable<Contact>:
public class StoreDB
{
    public Contact GetContact(int id) {...}

    // change to this:
    public IQueryable<Contact> GetContacts() {...}
}

Now you can say this without wasting cycles or bandwidth:
IQueryable<Contact> SerachByFirstName(StoreDB store, string searchParam, int limit)
{
    return store.GetContacts()
        .Where(c => c.FirstName == searchParam)
        .Take(limit);
}

In fact, this still won't dispatch a query to the database unless you start enumarting over the IQueryable. That is, you still can add more Where() or OrderBy() or what ever and they will be merged and sent to the source together when the query is finally evaluated.

Of course this doesn't help you to bind to the UI. But that is a hard problem. You either have to expose the underlying EntitySet to WPF or use a MVVM (or similar) architecture to separate data access and UI.
Comment: yes, this is what I want to do, but the above causes "The type System.Collections.Generic.IEnumerable cannot be converted to System.Linq.IQueryable". What do I need to use beside List, since IQueryable doesn't work because it's abstract.

WPF/LINQ data layer design question


How to get a runtime control id

Question: Using asp.net and c#
Asp.net code
<asp:Button ID="btnAddNew" runat="server" Text="Add New"
        onclick="btnAddNew_Click" />

            First Name :
            <asp:TextBox ID="txtFName" runat="server" ></asp:TextBox>

            Last Name :
            <asp:TextBox ID="txtLName" runat="server" ></asp:TextBox>

            Date Of Birth :
            <asp:TextBox ID="txtDob" runat="server" ></asp:TextBox>

            Age :
            <asp:TextBox ID="txtAge" runat="server" ></asp:TextBox>

                <asp:Button ID="btnSave" runat="server" Text="Save"  OnClientClick=" return validate()" type="submit"   />

                <asp:Button ID="btnClose" runat="server" Text="Close" OnClientClick ="return clear()" type="submit" />

C# Code
static void Insert()
    {
        try
        {
            string connectionString =
                "server=JEBW1011ZAHID;" +
                "initial catalog=employee;" +
                "integrated security = true";
            using (SqlConnection conn =
                new SqlConnection(connectionString))
            {
                conn.Open();
                using (SqlCommand cmd = new SqlCommand(
                    "INSERT INTO [NyhedTB] ([NyhedDato], [NyhedTitel], [NyhedTekst]) " +
                    "VALUES (@NyhedDato, @NyhedTitel, @NyhedTekst)", conn))
                {
                    cmd.Parameters.AddWithValue("@NyhedDato", txtfname.Text);
                    cmd.Parameters.AddWithValue("@NyhedTitel", txtlanme.Text);
                    cmd.Parameters.AddWithValue("@NyhedTekst", txtage.Text);

                    int rows = cmd.ExecuteNonQuery();  // Inserted rows number
                }
            }
        }
        catch (SqlException ex)
        {
            //Log exception
            //Display Error message
        }

    }

Showing error as txtfname does not exist.
Need code help
Answer: You Insert method is static. It doesn't have access to any page elements. I would recommend you passing those values as parameters to the method instead of coupling it with your UI:
static void Insert(string fName, string lName, string age)
{
    try
    {
        string connectionString =
            "server=JEBW1011ZAHID;" +
            "initial catalog=employee;" +
            "integrated security = true";
        using (SqlConnection conn = new SqlConnection(connectionString))
        {
            conn.Open();
            using (SqlCommand cmd = new SqlCommand(
                "INSERT INTO [NyhedTB] ([NyhedDato], [NyhedTitel], [NyhedTekst]) " +
                "VALUES (@NyhedDato, @NyhedTitel, @NyhedTekst)", conn))
            {
                cmd.Parameters.AddWithValue("@NyhedDato", fName);
                cmd.Parameters.AddWithValue("@NyhedTitel", lName);
                cmd.Parameters.AddWithValue("@NyhedTekst", age);

                int rows = cmd.ExecuteNonQuery();  // Inserted rows number
            }
        }
    }
    catch (SqlException ex)
    {
        //Log exception
        //Display Error message
    }
}

and then when calling this method from your code behind you could pass whatever values you want:
protected void SomeButton_Click(object sender, EventArgs e)
{
    Insert(txtfname.Text, txtlanme.Text, txtage.Text);
}

Now your Insert method is a little more reusable as it is no longer coupled to your UI.
Answer: Remove the static  in your method
Put this code public void Insert() instead of  static void Insert()

How to get a runtime control id


jQuery Hide/Show DIV on change of anchor text

Question: I have one page website which changes URL #id or anchor text on scrolling and on navigation click. I have an image positioned fixed at the center of the site. The site is divided in different sections and the image shows in all the sections. I do not want to show the image on the first section as it is unrelated to the content of the 1st section but will be showed in all next sections. How can I make it work?
What I have tried till now but not working:
$(document).ready(function(){
    $(".phone").hide();
    var idSample = window.location.href.split("#");
    var id = "#"+idSample[1];
    if($(id) == "second"){
        $(".phone").show();
    }else{
        $(".phone").hide();
    }
});

Rest in Fiddle: https://jsfiddle.net/shubhamjha1000/vh7bu32q/
I don't want that phone to be seen on 1st section but on rest of the sections. Please help me guys!
Comment: added fiddle.. please check..
Comment: Note that you didn't add jquery to the fiddle and it prints errors...
Comment: thanks for the correction.. added..
Answer: You can use the answer written in this question to listen to changes in the window see what url you are on and show or hide the phone accordingly....
Example:
    $(function(){

  // Bind the event.
  $(window).hashchange(hashchanged);

  // Trigger the event (useful on page load).
  hashchanged();

});

function hashchanged(){
 var hash = location.hash.replace( /^#/, '' );
 //your code
 if(hash == "#first") {
    // Hide element
  } else {
    // Show element
  }
}

But still even though what you are planning to do will work with that solution I think it would still look bad on the app it self and instead of a hovering phone maybe you can create the phone as an img inside the relevant containers and hide and show with id...
Answer: You can simply use a substring on your location.hash for get your hash tag
$(document).ready(function(){
    $(".phone").hide();

    var id = window.location.hash.substr(1);

    if($(id) == "second"){
        $(".phone").show();
    }else{
        $(".phone").hide();
    }
});


Group each "graph dimension" in the legend of Plotly

Question: I would like to add to the Plotly "standard legend" produced by plotly.express each of the "graph dimensions" grouping together all traces of such characteristics. As it is probably hard to understand what I want to do from the previous description, let me put an example. I have a code that produces a plot using the following line:
px.line(
    df,
    x = 'x values',
    y = 'y values',
    color = 'Device specs', # This is what I call "color dimension".
    symbol = 'Device', # This is what I call "symbol dimension".
    line_dash = 'Contact type', # This is what I call "line_dash dimension".
)

and the plot looks (for some specific data) like this:

I want to add below this legend one legend for each "dimension", i.e. one legend for the color grouping all the traces of each color, one for the symbol and one for the line_dash, something like this:

and, if possible, such that if I click in e.g. contact=dot it toogles the visibility of all the dashed traces together.
Is this possible with Plotly Express?
Comment: You linked this question, can you fix that?
Comment: Haha, yes! [This](https://stackoverflow.com/questions/67168232/can-a-plotly-visualization-show-separate-legends-for-color-symbol-size-etc/67169416#67169416) is the correct link.
Comment: Answering to your (deleted) question, `plotly.express.scatter` does what I want by default if you use a continuous color scale and markers. However that is a (very) particular case of my question, which is not of use to me in this application. I want to be able to do that in a general case, specifically here using `plotly.express.line` and having 3 "graph dimensions" (discrete color, line dash and symbol).
Comment: So now I have rechecked the question you linked and yes, it seems to be the same. However he seemed to be asking for that very specific case, and accepted an answer that does not answer the question; the answer only shows how to change the size of the color bar so it does not overlap the legend. The accepted answer is indeed more suited for [this question](https://stackoverflow.com/questions/61827165/plotly-how-to-handle-overlapping-colorbar-and-legends). I would thus appreciate if you leave this question open.
Comment: Certainly! Thank you for the clarifications.
Comment: do you have some sample data?  I get your plot,  but it's going to be some effort to generate a data set plus answer
Answer:
generate a data frame that has features that you have defined
creating core figure is simple - just uses your code
adding dimensions to legend.  This uses core concept that a legend shows all traces in a figure.  Hence to extend legend,  need additional traces
traces are built in a list comprehension with parameters that match your requirement.  These traces are deliberately given a y value that will not be included in y-axes range

import pandas as pd
import numpy as np
import plotly.express as px

SIZE = 10
# generate a dataset with all required attributes
df = pd.DataFrame(
    {
        "x values": np.tile(np.linspace(0, SIZE - 1, SIZE), SIZE),
        "y values": np.sort(np.random.uniform(1, 1000, SIZE ** 2)),
        "Device": np.concatenate(
            [np.full(SIZE, np.random.choice([52, 36, 34], 1)) for _ in range(SIZE)]
        ),
        "Contact type": np.concatenate(
            [np.full(SIZE, np.random.choice(["dot", "ring"], 1)) for _ in range(SIZE)]
        ),
        "Device specs": np.concatenate(
            [
                np.full(SIZE, np.random.choice(["laptop", "tablet", "console"], 1))
                for _ in range(SIZE)
            ]
        ),
    }
)
df.loc[df["x values"].eq(SIZE - 1), "y values"] = np.nan

# build the standard figure
fig = px.line(
    df,
    x="x values",
    y="y values",
    color="Device specs",  # This is what I call "color dimension".
    symbol="Device",  # This is what I call "symbol dimension".
    line_dash="Contact type",  # This is what I call "line_dash dimension".
)

# build additional traces for items wanted in legend
legend_traces = [
    px.line(
        df,
        x="x values",
        y=np.full(len(df), -1000),
        **param["px"],
    ).update_traces(**param["lg"], legendgroup=str(param["px"]))
    for param in [
        {"px": {"color": "Device specs"}, "lg": {"legendgrouptitle_text": "Spec"}},
        {"px": {"symbol": "Device"}, "lg": {"legendgrouptitle_text": "Device"}},
        {
            "px": {"line_dash": "Contact type"},
            "lg": {"legendgrouptitle_text": "Contact type"},
        },
    ]
]

for t in legend_traces:
    fig.add_traces(t.data)

# hide the dummy traces for extra legend entries (given y-value of -1000)
fig.update_yaxes(range=[0, df["y values"].max()])
fig.update_layout(height=500)

Comment: Thank you. So I guess this is not yet implemented in Plotly Express, i.e. I have to do it "by hand". I appreciate your example. Small thing, if you change `y = np.full(len(df), -1000),` to `y = np.full(len(df), float('NaN')),` you don't need to hide any dummy trace as they will be invisible.
Comment: good point... I did build it such that it could be made fully generic by using listcomprehensions and effectively a list of parameters for the legend groups required
Comment: Do you know if it is possible and how to implement the last part of my question? Link the added legend elements to the original traces such that when you click e.g. on laptop it toggles all the traces of type laptop, and so on? I think this should be done using the "legend group" but not sure how to do it.
Comment: with a callback mechanism yes... are you open to using **dash**, possibly `FigureWidget()`
Comment: No, I prefer to keep it simple. For the moment I am using Plotly as (very nice) a replacement for Matplotlib. Thanks anyway.


count entityreference multivalue field conicidences (make an attendance form)

Question: This question is similar to this one How to make an attendance form, but i solved by entity refrence view widget module, saving date in a node, and having a field entityreference user relation with unlimited values. Now i have a views table of nodes like:

and now i can not sum how many times a user entityreference appears, ofcourse at the bottom i need to have: John=3, Mary=2, Pete=2
I tryed views aggregartor plus with no luck, please make some suggestions how to achive this.
Answer: Solved. This is what I did. In order to not make heavy loads i enebleded Entity Construction Kit (eck) module, then by rules i added an entity (instead of a node) for every user entity reference value, within two columns, one for user name, and one for date in attendance. Now I can have a view (of entities) with two exposed filter on dates (beginning and ending dates) Then I override my views table tpl view with the following code. It displays a 2 column table, first: username, second: number of days that user has attendance.  the following code assumes that username is second field on your view ($fieldloop == 2 ). code:
<?php $names = array(); ?>

<?php foreach ($rows as $row_count => $row): ?>
   <?php $fieldloop = 1; ?>
    <?php foreach ($row as $field => $content): ?>
    <?php if ($fieldloop == 2 ) {$names[] = strip_tags($content);} ?>
    <?php $fieldloop = $fieldloop + 1; ?>
    <?php endforeach; ?>
<?php endforeach; ?>

  <?php $attendances = (array_count_values($names)); ?>

Alumno	# dias

<?php foreach ($attendances as $attendance => $value) : {
?>
  <?php print $attendance ; ?>
 <?php print $value; ?>

<?php } endforeach; ?>


JAXB/Moxy @XmlPath(".") conflicts with the XMLAdapter during the unmarshalling

Question: Was unable to find the solution even after trying many things so posting here hoping to get some workaround or fix for this issue.
Basically, if the @XmlPath(".") has been used on a Map and if there is XMLAdapter on it then it fails during the unmarshalling. The marshaling works perfectly only the unmarshalling fails.
In short, I would like to perform the unmarshalling as mentioned here but along with Map I will have one more @XmlElement. So one field is annotated with (Map field) @XmlPath(".") and another String field with @XmlElement and then I would like to perform unmarshalling.
Following is the XML that I am trying to unmarshal:
<Customer xmlns:google="https://google.com">
  <name>Batman</name>
  <age>2008</age>
  <google:sub1>MyValue-1</google:sub1>
  <google:sub2>MyValue-1</google:sub2>
</Customer>

Following is the Customer.class to which it will be unmarshalled:
@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "userExtensions"})
@XmlAccessorType(XmlAccessType.FIELD)
@NoArgsConstructor
@Getter
@Setter
@AllArgsConstructor
public class Customer extends Person {

  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  @XmlAnyElement
  private Map<String, Object> userExtensions = new HashMap<>();
}

Following is the Main class which will unmarshal:
public class Unmarshalling {
  public static void main(String[] args)
      throws JAXBException, XMLStreamException, FactoryConfigurationError, IOException, URISyntaxException, ProcessingException {
    final InputStream inputStream = Unmarshalling.class.getClassLoader().getResourceAsStream("customer.xml");
    final XMLStreamReader xmlStreamReader = XMLInputFactory.newInstance().createXMLStreamReader(inputStream);
    final JAXBContext jaxbContext = JAXBContext.newInstance(Customer.class);
    final Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    final Customer customer = unmarshaller.unmarshal(xmlStreamReader, Customer.class).getValue();
    System.out.println("Read XML : " + customer);
  }
}

When I run this code I will get the output as:
Read XML : Person(name=Rise Against, age=2000)

Its not reading the custom elements such as google:sub1 and google:sub2.

There is one more issue similar to this but the workaround mentioned there did not work for me.
I tried the @XmlAnyElement(lax=true) on MAP even that did not work for me.

Following is the TestAdapter that I am using for marshalling and unmarhsalling the Map<String,Object>
public class TestAdapter extends XmlAdapter<Wrapper, Map<String, Object>> {

  @Override
  public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("ADAPTER UNMARSHALLING");
    System.out.println(value.getElements());

    if (value == null) {
      return null;
    }

    //Loop across all elements within ILMD tag
    final Map<String, Object> extensions = new HashMap<>();
    for (Object obj : value.getElements()) {
      Element element = (Element) obj;

      //System.out.println("Node Name : " + element.getNodeName() + " Value : " + element.getTextContent());

      final NodeList children = element.getChildNodes();

      if (children.getLength() == 1) {
        extensions.put(element.getNodeName(), element.getTextContent());
      } else {
        List<Object> child = new ArrayList<>();
        for (int i = 0; i < children.getLength(); i++) {
          final Node n = children.item(i);
          if (n.getNodeType() == Node.ELEMENT_NODE) {
            Wrapper wrapper = new Wrapper();
            List childElements = new ArrayList();
            childElements.add(n);
            wrapper.elements = childElements;
            child.add(unmarshal(wrapper));
          }
        }
        extensions.put(element.getNodeName(), child);
      }
    }
    return extensions;
  }

  @SuppressWarnings("unchecked")
  @Override
  public Wrapper marshal(Map<String, Object> v) throws Exception {
    if (v == null) {
      return null;
    }

    Wrapper wrapper = new Wrapper();
    List elements = new ArrayList();
    for (Map.Entry<String, Object> property : v.entrySet()) {
      if (property.getValue() instanceof Map) {
        elements.add(new JAXBElement<Wrapper>(new QName(property.getKey()), Wrapper.class, marshal((Map) property.getValue())));
      } else {
        elements.add(new JAXBElement<String>(new QName(property.getKey()), String.class, property.getValue().toString()));
      }
    }
    wrapper.elements = elements;
    return wrapper;
  }
}

class Wrapper {

  @XmlAnyElement
  List elements;

  public List getElements() {
    return elements;
  }

I tried many things but nothing worked till now. Can someone please have a look at this issue and provide your solution. Happy to provide any more info with regards to this issue as I have spent nearly 10 days looking for answers.
I tried following things:

Created one more variable and tried to use it only during unmarshalling but that's also failing.
Tried to create one more field then used the @XmlAnyElement on it but that's also not working as I expected.

Tried few more things but nothing helped.
Everything is failing. I tried to debug the code from MOXY but I am unable to follow many things there. Any help with this issue would be really helpful for me.
Answer: I was able to get it by using the BeforeMarshal and AfterMarshal methods. Posting here so it can be helpful to someone in the future:
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, visible = true, property = "isA")
@JsonInclude(Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
@XmlRootElement(name = "extension")
@XmlType(name = "extension", propOrder = {"name", "age", "otherElements"})
@XmlAccessorType(XmlAccessType.FIELD)
@Getter
@Setter
@AllArgsConstructor
@ToString
@NoArgsConstructor
public class Customer {
    @XmlTransient
    private String isA;

    @XmlPath("customer/name/text()")
    private String name;

    @XmlPath("customer/age/text()")
    private String age;

    @XmlAnyElement(lax = true)
    @JsonIgnore
    @XmlPath("customer")
    private List<Object> otherElements = new ArrayList<>();

    @JsonIgnore
    @XmlTransient
    private Map<String, Object> userExtensions = new HashMap<>();

    @JsonAnyGetter
    @JsonSerialize(using = CustomExtensionsSerializer.class)
    public Map<String, Object> getUserExtensions() {
        return userExtensions;
    }

    @JsonAnySetter
    public void setUserExtensions(String key, Object value) {
        userExtensions.put(key, value);
    }

    private void beforeMarshal(Marshaller m) throws ParserConfigurationException {
        System.out.println("Before Marshalling User Extension: " + userExtensions);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        otherElements = extensionsModifier.Marshalling(userExtensions);
        System.out.println("Before Marshalling Final Other Elements " + otherElements);
        userExtensions = new HashMap<>();
    }

    private void afterUnmarshal(Unmarshaller m, Object parent) throws ParserConfigurationException {
        System.out.println("After Unmarshalling : " + otherElements);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        userExtensions = extensionsModifier.Unmarshalling(otherElements);
        System.out.println("Final User Extensions : " + userExtensions);
        otherElements = new ArrayList();
    }
}

The complete answer can be found here: https://stackoverflow.com/a/67923216/7584240


Prove that only triangles, quadrilaterals, and hexagons will Tesselate the plane

Question:
Prove that only triangles, quadrilaterals, and hexagons will Tesselate the plane

So, I have almost completed the proof, I will write it down all the way to the part I'm stranded in.
Theorem: only triangles, quadrilaterals, and hexagons will Tesselate the plane

So, the angles of a regular polygon with $n$ sides have a measure of $(\frac{180(n-2)}{n})°$ when several "$n$-gons" meet at one point, $a$ angles meet, so the measure of all this a angles totals $(\frac{180(n-2)}{n}(a))°$; if $(\frac{180(n-2)}{n}(a))°>360°$ the polygons will overlap, whilst $(\frac{180(n-2)}{n}(a))°<360°$ will result in empty spaces. So
$$(\frac{180(n-2)}{n}(a))°=360°$$
which has to be simplified.
$\frac{180(n-2)}{n}(a)=360$
$180(n-2)a=360n$
$(n-2)a=2n$
$na-2a=2n$
$na-2a-2n=0$
$(a-2)n-2a=0$
So, this is this the immediately aforementioned linear system :
$\begin{bmatrix} (a-2) & -2 \end{bmatrix}\begin{bmatrix} n \\ a \end{bmatrix}=\begin{bmatrix} 0 \end{bmatrix}$
Which I solve
$\begin{bmatrix} (a-2) & -2 & | & 0 \end{bmatrix}$
$\begin{bmatrix} 1 & \frac{-2}{a-2} & | & 0 \end{bmatrix}$
So turns out
$\begin{bmatrix} n \\ a \end{bmatrix}=\begin{bmatrix} \frac{2t}{a-t} \\ t \end{bmatrix}$
But I get stuck in here; I know that $n$ is the number of sides of the $n$-gons we are tiling the plane with, and $a$ is the number of angles that meet at a point, so I know that I have to make sure that $a∈ℤ∧n∈ℤ∧n≥3$; I already know that this only happens when
$\begin{bmatrix} n \\ a \end{bmatrix}=\begin{bmatrix} 3 \\ 6 \end{bmatrix}$
$\begin{bmatrix} n \\ a \end{bmatrix}=\begin{bmatrix} 6 \\ 3 \end{bmatrix}$
and
$\begin{bmatrix} n \\ a \end{bmatrix}=\begin{bmatrix} 4 \\ 4 \end{bmatrix}$
But, I only do, because I know that beforehand; the only thing I'm missing to complete the proof, is that those are the only combinations that won't result in a number of sides or angles that make no sense.
How would you do this?
Thanks in advance.
Comment: And pentagons as well
Comment: What do you even mean by $\begin{bmatrix} (a-2) & -2 & | & 0 \end{bmatrix}$? That's the only notation I don't know what is in your question.
Answer: (I'm assuming the scope of this problem only includes regular polygons.)
Most of your later steps are unnecessary. Once you have
$$a(n-2)=2n$$
or $(n-2)|2n$, we have
$$(n-2)|(2(n-2)+4).$$
$$(n-2)|4.$$
Comment: Yes, the scope is only regular polygons.


How to prove $x^2=-1$ has a solution in $\mathbb{Q}_p$ iff $p=1\mod 4$

Question: Let $p$ be prime and let $\mathbb{Q}_p$ denote the field of $p$-adic numbers.
Is there an elementary way to prove $x^2=-1$ has a solution in $\mathbb{Q}_p$ iff $p=1\mod 4$?
I need this result, but I cannot find a reference. Can some recommend a good book or a set of (easily available) lecture notes to help me out?
Answer: Are you familiar about number theory? This is equivalent to saying that $x^2 \equiv -1$ in $\mathbb{F}_p = \mathbb{Z}/p\mathbb{Z}$ iff $p \equiv 1$ (mod $4$). But this is a basic fact on number theory, since
\begin{equation*}
\Big( \frac{-1}{p} \Big) = (-1)^{\frac{p-1}{2}} = 1 \text{ iff } p \equiv 1 (\text{ mod }4)
\end{equation*}
Comment: IMO you should specifically list the need to use Hensel lifting (or Hensel's lemma) for the implication "there is a solution in $\Bbb{F}_p$" $\implies$ "there is a solution in $\Bbb{Q}_p$".
Comment: But $\mathbb Q_p$ is a completion of $\mathbb Q$ and thus has characteristic $0$, not $p$?
Comment: Yes, but since you are searching for solutions of $x^2 = -1$ in $\mathbb{Q}_p$, in fact you are searching solutions in $\mathbb{Z}_p$, since $|x|^2_p = |-1|_p = 1$. So, you are searching for $x \in \mathbb{Z}_p$ such that $x^2 = -1$. Finally, recall that there is the reduction map

\begin{equation*}
\mathbb{Z}_p \longrightarrow \mathbb{F}_p
\end{equation*}

given by $x = \sum_{i=0}^{\infty} a_i p^i \mapsto \overline{x} = a_i$ ($a_i \in \{ 0,1,...,p-1\}$).

You see you need a little of background about p-adic numbers.
Comment: Also, the argument fails for $p = 2$, in which case $(X+1)^2 + 1 = X^2 + 2X + 2$ is an Eisenstein polynomial and therefore $X^2 + 1$ is irreducible.
Answer: Following Euler Criterion
http://en.wikipedia.org/wiki/Euler%27s_criterion
$-1$ is a quadratic residue if $(-1)^{\frac{(p-1)}2}=1$
which happens only when $\frac{(p-1)}2$ is even
so $p-1$ is divisible by 4
so $p\equiv1 (mod4)$
the book i would follow for number theory is Introduction to number theory by Hua Loo Keng
this is a fairly comprehensive book can a bit a too much for elementary course so depending on your syllabus you may want to use an elementary text like Elementary number theory by Rosen.


Flink Shaded Hadoop S3 Filesystems still requires hdfs-default and hdfs-site config paths

Question: I'm trying to configure S3 as my state backend with Flink 1.6.0.
flink-conf.yaml
state.backend: filesystem
state.checkpoints.dir: s3://***/flink-checkpoints
state.savepoints.dir: s3://***/flink-savepoints

s3.access-key: *******
s3.secret-key: *******

https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/aws.html#shaded-hadooppresto-s3-file-systems-recommended
I'have moved the flink-s3-fs-hadoop-1.6.0.jar to the lib directory. The docs don't specify any need for hadoop configuration files for this particular method. Yet i'm facing this error complaining about missing hadoop configuration path.
2018-08-24 23:25:17,829 INFO org.apache.flink.streaming.runtime.tasks.StreamTask           - State backend is set to heap memory (checkpoints to filesystem "s3://***/flink-checkpoints")
2018-08-24 23:25:17,831 DEBUG org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.AbstractFileSystemFactory  - Creating Hadoop file system (backed by Hadoop s3a file system)
2018-08-24 23:25:17,831 DEBUG org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.AbstractFileSystemFactory  - Loading Hadoop configuration for Hadoop s3a file system
2018-08-24 23:25:17,872 DEBUG org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.util.HadoopUtils  - Cannot find hdfs-default configuration-file path in Flink config.
2018-08-24 23:25:17,873 DEBUG org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.util.HadoopUtils  - Cannot find hdfs-site configuration-file path in Flink config.
2018-08-24 23:25:17,873 DEBUG org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.util.HadoopUtils  - Could not find Hadoop configuration via any of the supported methods (Flink configuration, environment variables).
2018-08-24 23:25:17,878 INFO  org.apache.flink.runtime.taskmanager.Task                     - Source: Custom Source -> Map -> Sink: Print to Std. Out (1/1) (ee0eeb00ea0f01043d90f6b8d3c0cc2e) switched from RUNNING to FAILED.
javax.xml.parsers.FactoryConfigurationError: Provider for class javax.xml.parsers.DocumentBuilderFactory cannot be created
        at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:311)
        at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:267)
        at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:120)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2565)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2541)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2424)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.set(Configuration.java:1149)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.conf.Configuration.set(Configuration.java:1121)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.HadoopConfigLoader.loadHadoopConfigFromFlink(HadoopConfigLoader.java:101)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.HadoopConfigLoader.getOrLoadHadoopConfig(HadoopConfigLoader.java:80)
        at org.apache.flink.fs.s3hadoop.shaded.org.apache.flink.runtime.fs.hdfs.AbstractFileSystemFactory.create(AbstractFileSystemFactory.java:55)
        at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:395)
        at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:318)
        at org.apache.flink.core.fs.Path.getFileSystem(Path.java:298)
        at org.apache.flink.runtime.state.filesystem.FsCheckpointStorage.<init>(FsCheckpointStorage.java:61)
        at org.apache.flink.runtime.state.filesystem.FsStateBackend.createCheckpointStorage(FsStateBackend.java:443)
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:257)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
        at java.lang.Thread.run(Thread.java:748)

Am I missing something here? Any help is appreciated.
Answer: Screwed my dependencies and that's what caused this unrelated exception. Was trying out Bucketing and Rolling Sink connectors which required Hadoop dependencies. Added them in maven's provided scope and was not able to run them from IntelliJ IDEA. So switched them to compile and left it as it is. They got packed part of the artifact jar and caused this issue.
Lesson Learnt: Never add Hadoop dependencies in default (compile) scope. IntelliJ has an option in the Run Configuration to included dependencies declared in provided scope.


An algorithm similar to (or based on) K-means that do not require the 'k' number of clusters

Question: These days I'm using a lot (and discovering) nice ways to use k-means' clustering. For example, clustering word embeddings (word2vec vectors) to find synonyms or clustering doc vectors (doc2vec) to classify topics.
But for these tasks I always need to specify the number of clusteres (the 'k'). Is there some unsupervised algorithm that estimates a good number of clusteres and (but not must do) do the clustering task?
Comment: But any "automatic" - i.e. beforehand - methods to determine K will always lose to the old and good "afterwards" method - it is when you do clusterings (maybe on a subsample) with different K and then select the best K [from some point of quality](https://stats.stackexchange.com/q/195456/3277).
Comment: I'm surprised no one has mentioned expectation-maximization for Gaussian mixtures, for instance. You generally need to specify $k$, but if it's too high, the algorithm will probably reveal it. If you're just getting into the field, it might be diffcult to understand, and it is more of a framework than a specific algorithm. However, it's a good thing to keep in mind if you want to gain a better understanding of clustering in general.
Comment: This question mixes two different in one: (i) clustering methods that do not require to specify K. (ii) ... that attempt selection of  the "best" K automatically before or during the clustering process.
Comment: Given that you say "similar to k-means", you probably are interested in what I've marked as (ii). TwoStep cluster analysis tries to determine the best number K during its 1st step based on minimum spanning tree approach. The 2nd step is hierarchical clustering; however, one could implement to do k-means as the 2nd step, in principle.
Comment: For soft clustering text documents, there is Latent Dirichlet Allocation that uses a hierarchical Dirichlet process to identify K. It is quite advanced and very interesting (as far as i understand). For a more general k-means kind of clustering, some researchers in my Uni came up with [adaptive weights clustering](https://bayesgroup.github.io/bmml_sem/2016/AWC_2016_Repino.pdf)
Comment: Thanks for the suggestion @broncoAbierto. I have about 7 months that I came to the area so there's a lot to learn about. I will check that.
Answer: Yet another approach is to use the Mean Shift algorithm. You must specify a radius around points to explore, but it determines the number of clusters.
Comment: I found out that DBSCAN is giving better results for my case, but unfortunately I have to reduce dimension to 2 to give acceptable results. I'm doing this with t-SNE, do you advise another one? PCA was performing poorly...
Comment: t-SNE and PCA would be my first two choices. There is also Multi-Dimensional Scaling, but it is closely related to PCA.  I would expect similar performance for PCA & MDS.  But you have said little about your data or problem.  Is Feature selection practical?  Might it be that some features are more important than others and you can discard the less important ones before t-SNE/PCA type reduction?
Comment: No, I can't. That's vectors created by Doc2Vec and I don't have information about the relevant features. There's 200 features each document vector. I found out that PCA 99% of the features, t-SNE to 3 dimensions and then DBSCAN it gave good results. Still don't know if it will be always good when the quantity of data increase, and it will a lot.
Answer: The NASA Autoclass employs Bayesian approach to inferring not only the cluster centers but also the most likely number of clusters. A similar tool, NBCTK, uses more modern inference algorithms (e.g. Variational Bayes).
Some of the nice features include:

Support for mixed data types (e.g. discrete and real-valued features)
Handles missing values

It works fairly well on relatively small datasets (on my datasets). Some basic understanding how the algorithm works under the hood helps to use the tool to its best potential. Expect time to process even moderate size datasets to be significant.
Note: The most likely number clusters are found under the assumptions made by the model. One must not interpret the results as absolute truth.
Answer: If you mean clustering without specifying number of clusters up front, you have a couple of approaches

estimate $k$ - use elbow method or x-means. Also, bisecting kmeans doesn't need $k$ specified.
use different clustering scheme altogether

If you choose 2 then you have further types of methods

use hierarchical clustering
use density-based clustering algorithms like DBSCAN, or its newer cousin, HDBSCAN

Both hierarchical methods and density-based methods require some kind of distance parameter to be specified - the clusters are formed by grouping the points based on that distance.
Comment: (+1) an example of a different clustering scheme that provides an estimate of $k$ is [spectral clustering](https://arxiv.org/pdf/0711.0189.pdf) with the eigengap. If the amount of noise is low relative to the clarity of the clusters then this can work well, but in my experience if there's a decent amount of noise (as there tends to be) then the spectrum looks pretty smooth and it's not clear what to do. Also there are usually similarity function / graph parameters and they can be just as hard to set as $k$
Comment: Thank you for the suggestion. I found a Python implementation of x-means but it does require a 'k_min', is this normal? Check that: https://github.com/mynameisfiber/pyxmeans/blob/master/pyxmeans/xmeans.py Liked the 'sklearn' way of implementation.
Answer: DBSCN, Affinity Propagation are clustering algorithms that don't require that you specify k.
Alternatively, you can iterate through multiple values of k and see which one gives you the best results using some similarity measure.
A 3rd approach is ask a domain expert in whichever domain you are working in to give you a rough estimate of how many clusters you should be expecting.
Comment: Thanks for the suggestion. I will read more about DBSCN to see its behavior.
Answer: Density clustering methods (MeanShift, DBScan....) don't require to specify the number of clusters, you specify something like the order of magnitude of the distance of points considered as "connected". The parameter is (very) roughly speaking the expected radius of a cluster, and then it finds the best number of clusters.
Density algorithms tend to fail in large dimension because the space is almost empty everywhere (or alternatively the distances gather around a certain value in a way that is extremely difficult to compensate). Density algorithms thus often require dimension reduction first, unlike k-means. With word2vec you usually start with dimension 300, which might be a problem (to experiment). If the density algorithms fail, then you can reduce the dimension first, possibly drastically.
Comment: Ok so the vectors has 200 dimensions each, how much dimensions do you think it's ideal to apply the Density methods? Also, I'm using t-SNE but just for plotting. Do you have an opinion of other better dimension reduction algorithm?
Comment: t-SNE is not recommended except for plotting (honestly I never tried for something else). I'd recommend positive matrix factorization (PMF). What dimension: I don't know. Try 200 first anyway. Then try 10, try 2... Then adjust. The fail of density clustering most often results in all points collapsing in a single cluster, or one/few points by cluster. It's most often blatant: changing the distance parameter just takes you from one extreme to the other.


Create multiple files in XSLT

Question: my xml file
<?xml version="1.0" encoding="UTF-8"?>
<tests>
<testrun run="test1">
    <test name="foo" pass="true" />
    <test name="bar" pass="true" />
    <test name="baz" pass="true" />
</testrun>
<testrun run="test2">
    <test name="foo" pass="true" />
    <test name="bar" pass="false" />
    <test name="baz" pass="false" />
        <testrun run="test2.1">
            <test name="foo" pass="true" />
            <test name="bar" pass="false" />
            <test name="baz" pass="false" />
        </testrun>
</testrun>
<testrun run="test3">
    <test name="foo" pass="false" />
    <test name="bar" pass="true" />
    <test name="baz" pass="false" />
</testrun>
</tests>

my xsl file
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="2.0">

<xsl:output method="text"/>
<xsl:output method="html" indent="yes" name="html"/>

<xsl:template match="/">
 <xsl:for-each select="//testrun">
  <xsl:variable name="filename"
   select="concat('output1/',@run,'.html')" />
   <xsl:value-of select="$filename" />
    <xsl:result-document href="{$filename}" format="html">

     <xsl:value-of select="@run"/>

   </xsl:result-document>
 </xsl:for-each>
</xsl:template>
</xsl:stylesheet>

In output I get four documents (test1, test2, test2.1, test3). Is it possible to create only three files (test1, test2, test3) where test2 contains values test2.1?
Answer: Here's one way to do it:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="2.0">

  <xsl:output method="text"/>
  <xsl:output method="html" indent="yes" name="html"/>

  <xsl:template match="/tests/testrun">
    <xsl:variable name="filename"
                  select="concat('output1/',@run,'.html')" />
    <xsl:value-of select="$filename" />
    <xsl:result-document href="{$filename}" format="html">

        <xsl:value-of select="@run"/>

        <xsl:apply-templates select="test"/>
        <xsl:apply-templates select="testrun"/>

    </xsl:result-document>
  </xsl:template>

  <xsl:template match="test">

test: <xsl:value-of select="@name"/>

  </xsl:template>

  <xsl:template match="testrun/testrun">

testrun: <xsl:value-of select="@run"/>

    <xsl:apply-templates select="test"/>
  </xsl:template>
</xsl:stylesheet>

The output/test2.html file will contain:

   test2

test: foo

test: bar

test: baz

testrun: test2.1

test: foo

test: bar

test: baz

(If you edit the tests in test2.1 to change the names so that they don't repeat those of test2, you'll be able to clearly see that the foo, bar, baz after test2.1 are those of test2.1.)

Create multiple files in XSLT


Vue.js: best practice with managing JS for other components

Question: I have a simple website set up with Vue.js. I have a component for each page (About us, Contact us, etc). Each component fetches the markup for the page via ajax then injects it using <vue-router>.
So far so good.
My question is: what should I do with all the JS that's only used for specific components? My home page might have ~100 lines of JS just for setting up image sliders, hover events, etc. Once I load the About Us component, all that JS for the home page still exists in the browser but it's no longer relevant. Should I manually be nullifying my variables?
What about this case: User visits the Home page, then the About page, then back to Home. On the 2nd visit of the home page, all my old variables still exist, but aren't necessarily in the correct state anymore. I'll want to move sliders back to slide 1, reset any hovers... basically just reset the page.
I guess I'm asking for a very high level overview on how to best manage the JS for multiple components and how it all ties in with document.ready and window.load callbacks once they've already fired and won't fire again. Thanks.
Comment: This question as it exists is a bit broad. You shouldn't worry about javascript memory limits as you'll likely never run into them - instead, optimize for reducing the number of bytes on the wire. Variables and whatnot that need to be restored between different parts of your application should be taken care of Vue.js, no?
Comment: For the second one: [Is there a proper way of resetting a component's initial data in vuejs?](https://stackoverflow.com/questions/35604987/is-there-a-proper-way-of-resetting-a-components-initial-data-in-vuejs)
Comment: @Qix "Variables and whatnot that need to be restored between different parts of your application should be taken care of Vue.js" I'm not sure, this is my first Vue project............... yuruy683, thanks for the link!
Answer:
Should I manually be nullifying my variables?

No, this is not a good use of your time/work/effort. Don't worry about performance until it starts to become an issue.
Vue has analogous callbacks for document.ready in the forms of created and mounted, but these are per component whereas document is per.. document?
If you are using the keep-alive wrapper for your router-view, you should use the activated, deactivated callbacks to perform work, such as resetting component state.
If not, then components are destroyed upon leaving the route, and you don't have to reset anything.
As for resetting, i tend to define a function that returns the initial state, and then use this function whenever i need a state reset.

Comment: It looks like Vue's `created` and `mounted` are what I was searching for. I'm not using 'keep-alive' but I'm going to give it another look because in retrospect it might be what I need. Thanks a lot