How to add a color filter to a button on click?

Question: I have modified the following How to make Button highlight? to this application.
But it seems I cant set the color filter to the button the way that I have done it.The error I'm getting is The method setColorFilter(PorterDuffColorFilter) is undefined for the type Button ,which I understand means that I can't use it with a button object but is there a way around this? Does anyone know how I could implement this in a similar way?
I declared the filter here:
//button filter
    PorterDuffColorFilter greenFilter =  new PorterDuffColorFilter(Color.GREEN, PorterDuff.Mode.SRC_ATOP);

And tried to implement it on the button click here:
@Override
    public void onClick(View v) {
        // TODO Auto-generated method stub

        try {

            String getoffsetlength = offsetLength.getText().toString();
            String getoffsetdepth = offsetDepth.getText().toString();
            String getductdepth = ductDepth.getText().toString();

            double tri1,tri2;
            double marking1,marking2,marking3;

            double off1 = Double.parseDouble(getoffsetlength);
            double off2 = Double.parseDouble(getoffsetdepth);
            double off3 = Double.parseDouble(getductdepth);

            //vibrate button onClick
            myVib.vibrate(20);

                //change button filter on click
            calculate.setColorFilter(greenFilter);

        } catch (NumberFormatException e) {
            // TODO: handle exception
            System.out.println("Must enter a numeric value!");

        }
//
    }

Answer: Use an ImageButton and call .setColorFilter on the ImageView that is set to the source of the ImageButton.


$SSR = \sum(\hat y_t - \bar y)^2 = \sum\hat y_t^2 - n \bar y^2$

Question: From Statistical Methods for Forecasting by Abraham and Ledolter:

Let $\hat y = X \hat \beta$, where $\hat \beta$ is the least squares estimate $(X^TX)^{-1}X^Ty$.
Then $SSR = \sum(\hat y_t - \bar y)^2 = \sum\hat y_t^2 - n \bar y^2$.

I'm having trouble showing this equality.
I see that expanding and eliminating expressions similar to both sides we're left with showing:
$\sum(y_t - \hat y_t) = 0$, but I can't figure this either.
Anyone have any ideas?
Answer: Let's decompose
\begin{equation}
 SSR = \sum\limits_{t=1}^n(\hat y_t - \bar y)^2 =
 \sum\limits_{t=1}^n \hat y_t^2 - 2\sum\limits_{t=1}^n\hat y_t \bar y + \sum\limits_{t=1}^n \bar y^2
\end{equation}
Notice that $\bar y$ does not depend on the summation index $t$ and hence could be extracted outside the sum as
\begin{equation}
 SSR =
  \sum\limits_{t=1}^n \hat y_t^2 - 2\bar y\sum\limits_{t=1}^n\hat y_t  + \bar y^2\sum \limits_{t=1}^n 1
  =
 \sum\limits_{t=1}^n \hat y_t^2 - 2\bar y\sum\limits_{t=1}^n\hat y_t  + n\bar y^2
 \tag{1}
\end{equation}
It is true that (As you say) if $\sum(y_t - \hat y_t) = 0$, then we get the desired result. Notice that this happens when $X$ is a square and invertible matrix, because
\begin{equation}
 \hat{\beta} = (X^TX)^{-1}X^Ty
 =X^{-1}X^{-T} X^{T}y = X^{-1}y
\end{equation}
where $X^{-T} = (X^{-1})^T$. Now, you can write $\hat{y}$ as
\begin{equation}
 \hat{y} = X\hat{\beta} = XX^{-1}y = y
\end{equation}
This means that
$\sum \hat{y}_t = \sum y = n\bar{y}$. Replacing this in $(1)$, we get
\begin{equation}
 SSR =
 \sum\limits_{t=1}^n \hat y_t^2 - 2n\bar y^2+ n\bar y^2
 =
\sum\limits_{t=1}^n \hat y_t^2 - n\bar y^2
\end{equation}


Insert into com select

Question: Preciso fazer uma inserção do tipo
INSERT INTO your_table SELECT * FROM temp_table;

mas, preciso que o id, e a segunda coluna que é uma id_fk eu possa setar.
Preciso fazer uma transferência da tupla para outro usuário, copiando todas as informações mas, passando novos  id, id_fk e nome
algum macete?
Comment: Explique melhor, sua pergunta não está clara
Comment: Seria interessante adicionar a estrutura das tabelas envolvidas.
Comment: Se possível colocar algum exemplo, pois não ficou muito claro a dúvida
Answer: tente fazer da seguinte maneira:
INSERT INTO tabela (nome_campo1,nome_campo2) VALUES (select nome_campo1,nome_campo2 from tabela)

Observação: Apenas substitua o 'nome_campo' pelos campos em questão, lembrando que tanto após o 'INSERT INTO' e o 'SELECT', a ordem dos campos devem ser a mesma.
Observação 2: Como vc não especificou em qual database vc esta fazendo esse SQL (Mysql, Postgree, SQlServer), eu usei a sintaxe do PostgreSQL, talvez vc precise adaptar o código para o banco q vc está usando
Answer: basta você alterar o select, e não usar *
INSERT INTO your_table
              SELECT
                 '1' as novo_id,
                 '2' as nova_fk,
                 t.outros_campos
              FROM temp_table t;

Insert into com select


Set source address to use tun device does not work (Debian Squeeze)

Question: There have been similar questions on StackExchange but none of the answers helped me, so I'll try a question of my own.
I have a VPN connection via OpenVPN. By default, all traffic is redirected through the tunnel using OpenVPN's "two more specific routes" trick, but I disabled that. My routing table is like this:
220.127.116.11 192.168.2.1     255.255.255.255 UGH   0      0        0 eth0
10.30.92.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun1
10.30.92.1      10.30.92.5      255.255.255.255 UGH   0      0        0 tun1
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0         10.30.92.5      0.0.0.0         UG    0      0        0 tun1
0.0.0.0         192.168.2.1     0.0.0.0         UG    0      0        0 eth0

And the interface configuration is like this:
# ifconfig
eth0      Link encap:Ethernet  HWaddr XX-XX-
          inet addr:192.168.2.100  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::211:9ff:fe8d:acbd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:394869 errors:0 dropped:0 overruns:0 frame:0
          TX packets:293489 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:388519578 (370.5 MiB)  TX bytes:148817487 (141.9 MiB)
          Interrupt:20 Base address:0x6f00

tun1      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:10.30.92.6  P-t-P:10.30.92.5  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:64 errors:0 dropped:0 overruns:0 frame:0
          TX packets:67 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:9885 (9.6 KiB)  TX bytes:4380 (4.2 KiB)

plus the lo device.
The routing table has two default routes, one via eth0 through my local network router (DSL modem) at 192.168.2.1, and another via tun1 through the VPN's gateway. With this configuration, if I connect to a site, the route chosen is the direct one (because it has less hops?):
# traceroute 184.108.40.206 -n
traceroute to 22.214.171.124 (184.108.40.206), 30 hops max, 60 byte packets
 1  192.168.2.1  0.427 ms  0.491 ms  0.610 ms
 2  18.104.22.168  17.981 ms  20.137 ms  22.141 ms
 3  18.104.22.168  23.681 ms  25.009 ms  26.401 ms
...

This is fine, because my goal is to send only traffic from specific applications through the tunnel (esp. transmission, using its -i / bind-address-ipv4 option). To test whether this can work at all, I check it first with traceroute's -s option:
# traceroute 18.104.22.168 -n -s 10.30.92.6
traceroute to 220.127.116.11 (188.8.131.52), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
...

This I take to mean that connection using the tunnel's local address as source is not possible.
What is possible (though only as root) is to specify the source interface:
# traceroute 184.108.40.206 -n -i tun1
traceroute to 188.8.131.52 (22.214.171.124), 30 hops max, 60 byte packets
 1  10.30.92.1  129.337 ms  297.758 ms  297.725 ms
 2  * * *
 3  126.96.36.199  297.653 ms  297.652 ms  297.650 ms
...

So apparently the tun1 interface is working and it is possible to send packets through it.
But selecting the source interface is not implemented in my actual target application (transmission), so I would like to get source address selection to work.
What am I doing wrong?

Edit: Based on the answer by DerfK, here's my OpenVPN configuration:
In the .conf- or .ovpn-file, include these commands
# disable automatic route configuration
route-noexec
# enable executing scripts
script-security 2
# script to configure routes
up updown.sh
# script to deconfigure routes
down updown.sh

And the script updown.sh contains:
#!/bin/bash
case $script_type in

up)     echo routes up
        # reproduce OpenVPN's standard routes
        ip route add $trusted_ip via $route_net_gateway
        ip route add $route_network_1 via $route_vpn_gateway
        # but put the default route in an extra table
        ip route add default via $route_vpn_gateway table 200
        # and make sure it is activated by selection of the interface's address
        ip rule add from $ifconfig_local/32 table 200
        # flush routing cache (but everything seems to work without this, too)
        ip route flush cache
        ;;

down)   echo routes down
        ip route del $trusted_ip
        ;;

esac

With this configuration, the traceroute -s command gives the same output as the traceroute -i showing the route through the VPN to the target address.
Something I did not explain in my original post: The VPN in question does not provide access to a protected local network, but to the Internet, but from a different location / IP address. Otherwise, adding a default route would not have been necessary in the first place.
Answer: What you are looking for is called "Source based routing" and the old route program can't manage that. You'll need to use iproute2 and set up an alternate routing table based on source IP address.  Using the guide here you'll have something like:
# Designate "table 200" as "vpntunnel"
# - optional, but otherwise we'd have to use 200 everywhere.  Also, 200 is an arbitrary number, 0 253 254 255 are reserved.
echo 200 vpntunnel >> /etc/iproute2/rt_tables
# Packets from your tunnel address use this table
ip rule add from 10.30.92.6/32 table vpntunnel
# The default route for packets using the vpntunnel is...
ip route add default via 10.30.92.5 dev tun1 table vpntunnel
# Flush routing cache
ip route flush cache

I'm not sure if you'll need to remove the route from the regular route table before doing this, but it should be OK to leave it (in fact it might be necessary in order to get the kernel to select the right source IP to talk to 10.30.92.5 for apps that don't bind to an IP).
Comment: Thanks a lot, that did it! One thing I'm not clear about though, is the last line in your configuration example, `ip route flush cache`. If I understand the man page right, this deletes all the entries in the (main?) routing table. Why would I want to do that?
Comment: I'm not sure where you're getting that idea.  It deletes all the entries in the *cache*, not the actual table
Comment: Right, I overlooked the last word in the command. Thanks again!


Over which set of elements should I perform norm clipping of gradients for backpropagation?

Question: I want to normalise the gradients of my multi-layer perceptron in order to avoid the Exploding Gradients Problem, so I thought I would use l2-normalisation but am unsure about how to apply it to the individual layers and tensors within the network. Specifically, I am unsure about whether to calculate the norm over the set of all variables/tensors or over each layer separately, and why.
L2 normalisation of gradients is performed by the tf.clip_by_global_norm function in tensorflow, and it defines the global norm (by which the gradients are adjusted) as;
global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))

where t_list is the list of tensors and l2norm(t) is a function that computes the magnitude of the input vector t.
What I am uncertain about is whether to compute global_norm for all tensors in the network, or for each layer or weight/bias vector independently. In my particular example, should t_list be the list of all variables (ie. flattened set of all weight and bias derivatives of my entire network), or should it be performed for each vector of biases and weights (and for each layer) independently? And why is one preferable over the other?
This answer from Stack Overflow alludes to these two possibilities and suggests that clipping on the global norm is desirable as doing it on each parameter set individually would result in a different normalisation scale for each set:

Despite what seems to be popular, you probably want to clip the whole
  gradient by its global norm... Clipping each
  gradient matrix individually changes their relative scale but is also
  possible

But it doesn't really provide more context than this, so I am still not covinced I understand the trade-offs behind each approach.
Answer: What is the global norm?
It's just the norm over all gradients as if they were concatenated together to form one global vector.
So regarding that question, you have to compute global_norm for all gradient tensors in the network (they are contained in t_list).
As you already said, there are different approaches to whether clip by the global norm or local norm or directly by values:
https://www.tensorflow.org/api_guides/python/train#Gradient_Clipping
As you can imagine, if you have very large gradient for one parameter-array but all others gradients are relatively moderate, than you would reduce your weight updating feedback for those parameters if you use global_norm as you clipping method, as the global_norm will be pretty high due to the outlier gradient.
Whereas, if you use local norm, you could restrict to only this outlier gradient which has these extremely huge values and let all other gradients be as they are.
In the end this is another hyperparameter you have to experiment with.
What I like to do, is to observe the gradients, if ALL of them are large, I often go with global_norm else if I see a large gradient for only one or a few paramters, I prefer to clip only those gradients by looking at their norm-value and deciding what threshold could be appropriate (look when your network starts to output nan's and clip before that!)


statistics test for heteroscedastic and homoscedastic data

Question: Which statistics tests should be applied on heteroscedastic and homoscedastic data, respectively? What are the differences?
Comment: This seems like a very broad question - there are thousands of hypotheses tests. What kinds of hypothesis are you testing?
Answer: I suppose you are interested in knowing how you should test for if your data is heterocedastic or not, right?
In this case, the two most common tests are:
- the Breusch-Pagan test
- the White test
Check this and this questions to get some insight about there differences.


Microsoft query format function with SQL Server data

Question: How can I get the ISO week/year from a date using Microsoft Query. I used to use the following WHERE statement:
WHERE (format([Date],'ww',2,2)=format(?,'ww',2,2))

The ? is the date parameter I used. the ,2,2 extension is making sure the result is according to the ISO for dates.
This worked when I used tables where Access was the data source. Now I'm trying to get the same result but then with SQL Server tables as datasource. I tried using:
datepart(ISO_week, [Date]) = datepart(ISO_week, ?)

But parameters don't seem to work in functions when using SQL Server data.
I load SQL Server tables in Excel using Microsoft Query. I do this so I can use parameters in my querys.
Answer: When using SQL Server data tables, functions using the '?' parameter sign didn't seem to work. So instead I made a field in Excel where the function is performed so that I can use that value in the query


How to compile 7zip source code in visual studio 2017?

Question: After following tutorial from here. I can not generate executables using from lzma1900 SDK.
Here is the output from powershell.
PS C:\Users\testbench\Downloads\7ZipSDK\CPP> "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars32.bat"
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars32.bat
PS C:\Users\testbench\Downloads\7ZipSDK\CPP> nmake NEW_COMPILER=1
MY_STATIC_LINK=1 Build.mak

Microsoft (R) Program Maintenance Utility Version 14.16.27030.1 Copyright (C) Microsoft Corporation.  All rights reserved.

    if not exist "o" mkdir "o"
    if not exist "o/asm" mkdir "o/asm"
    link  -nologo -OPT:REF -OPT:ICF /LARGEADDRESSAWARE /FIXED:NO -out:o\
 oleaut32.lib ole32.lib user32.lib advapi32.lib shell32.lib
LINK : fatal error LNK1104: cannot open file 'o\'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual
Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86\link.EXE"'
: return code '0x450'
Stop.

And I'm using this nmake version and ml.exe version:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86

I have also installed windows SDK. why it is not being loaded when I run that bat script.? and How can I compile the latest lzma SDK  using visual studio 2017?
Answer: Posting answer for future reference, in case any one needs to look at it:
There are two source codes provided by 7-Zip's developer.

7-Zip application.
LZMA algorithm implementation in (C, C++ ,C#, java).

One can generate Executable for windows by using makefile located inside
7z1900-src/CPP/7zip/makefile.
I used MSBUILD commandpromt for 2015.(note: one more way to achieve this by running vcvarsall.bat and then executing "nmake" command
If you want to understand how 7Z works then "Alone.dsw " is what you're looking for. it is inside
7z1900-src/CPP/7zip/Bundles/Alone/Alone.dsw
You will need to change "Debug Information Format" to Program Database /Zi inside Project Property. it is under C/C++ -> General.
I hope it helps.
Comment: Attempting to build this with VS2019 16.5.0 Preview 5.0 also results in:

1>CVTRES : fatal error CVT1100: duplicate resource.  type:MANIFEST, name:1, language:0x0409

1>LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
Answer: Turn off the Embed Manifest option in Project -> Properties -> Configuration Properties -> Manifest Tool -> Input and Output. Change the value of Embed Manifest to No. Find more


MySQL query slow with FULLTEXT index

Question: I have 1 MyISAM table with 620,000 rows.  Im running XAMPP on a Dual Core Server with 2GB RAM.  Apache is installed as a Windows Service, MySQL is controlled from the XAMPP control panel.
The query below is taking 30+ seconds to run.
select  `id`,`product_name`,`search_price`,`field1`,`field2`,
       `field3`,`field4`
    from  `all`
    where  MATCH (`product_name`) AGAINST ('searchterm')
      AND  `search_price` BETWEEN 0 AND 1000
    ORDER BY  `search_price` DESC
    LIMIT  0, 30

I have a FULLTEXT index on product_name, a BTREE on search_price, auto increment on id
If I explain the above query the results are:
id select_type   table type     possible_keys                       key                   key_len ref rows Extra
1  SIMPLE        all   fulltext search_price,FULLTEXT_product_name  FULLTEXT_product_name 0 NULL 1 Using where; Using filesort

How can I speed up this query?  Should it be taking this long on a table of 620,000 rows?
Ive just noticed that this only happens when the database has not been queried for a while, so im guessing this is to do with the cache, the first query is taking 30+ seconds, then if I try a second time the query takes 1 second
Comment: should I be using MyISAM with cache, or Innodb with buffer pool?
Answer: MySQL will do the fulltext search first, then look up the rest of the info, filter on price, sort on price, and finally deliver 30.  There is essentially no way to shorten that process.
Yes, caching is likely to be the explanation for 30 seconds becoming 1 second.
Switching to InnoDB (which now has FULLTEXT) may provide some benefits.
If running entirely MyISAM, do you have key_buffer_size set to about 20% of available RAM?  If you were much lower (or higher) than this, that could cause performance problems.
If running entirely InnoDB, set innodb_buffer_pool_size to about 70% of available RAM.
Comment: I cant find `key_buffer_size` in my.ini but it is set as
16777216, I can see key_buffer under `[mysqld] key_buffer = 16M` `[isamchk] = 20M` and `[myisamchk] = 20M`.  I will try using Innodb with `innodb_buffer_pool_size = 400MB`
Comment: I tried using Innodb with the buffer pool at 400MB, I also tried setting the `key_buffer_size = 100MB` and the queries are still 10-15+ seconds unless they are cached.

Is there anything else I can try to speed this query up?  Should I be using a different database setup instead of MySQL?  Would upgrading my server help?
Comment: Sagacious has suggestions on how to change the setup.  I have no further suggestions.
Comment: Im running this setup on a Cloud server... could this be why MySQL is performing slowly? Ive run the same setup on my personal computer and the queries are much faster.  Would switching to a more powerful dedicated server help?
Comment: Unlikely.  Let's check another thing -- `SHOW VARIABLES LIKE 'key_buffer_size';` on both machines.
Comment: 100MB on the cloud server, 16MB on my computer
Comment: just tried the same query on both machines, 12 seconds on my computer and then 1 minute 11 seconds on the cloud server
Comment: Do you have SSDs, but the cloud has spinning drives?
Answer: MySQL's capability of dealing with FULLTEXT is somewhat limited when th size of the table goes above 300,000. And it will peform even worse if you use really common words as search keywords like (in,the,of, etc commonly marked as stop words). I recommend using Sphinx Full Text Search/ Apache Lucene
Stackoverflow links:
Comparison of the two
More Comparison

MySQL query slow with FULLTEXT index


Some issue with bufferedReader

Question: I have a java function as follows:
 public HashMap<String, ArrayList<Double>> embedWords(BufferedReader buffR1 {
   ArrayList<String > arrayList = new ArrayList<String>();
   arrayList = getWords(buffR1);
   System.out.println("Word size:"+ arrayList.size());

       ArrayList<ArrayList<Double>> arrList = getWordFeature(buffR1);
   System.out.println("Size of arrList:embedWords:"+arrList.size());
     }

Here , the problem is , the both of the function  getWords  and  getWordFeatures can't give the size value. When i comment  function getWords the function getWordFeature returns non-zero value .But when uncommented , the output is as follows:
Word size:15055
Size of arrList:embedWords: 0

Answer: So you basically tell us, that BufferedReader will "eat" your input and that, once digested, there is no "read it again"?
Well, this is not surprising, since streams and readers are designed like this.
Comment: How would I overcome this situation?
Comment: You could do whatever `getWords()` and `getWordFeature()` are doing in a single method, reading the input once.
Comment: Or, if `getWords` just reads and splits the reader into words, you could rewrite `getWordFeature(Reader r)` into `getWordFeature(List words)`.
Comment: Read the text first, then throw away the `BufferedReader` and use `getWords` and `getWordFeature` on the data.
Answer: The getWords function has already read the contents of the buffR1 stream by the time getWordFeature function is called. You will need to read the contents of the stream before calling them to avoid it.
You might want first read the contents into an array, let's call it barr.
ByteArrayOutputStream bout = new ByteArrayOutputStream();
int c;
while ((c = buffR1.read()) != -1) {
  bout.write(c);
}
byte[] barr = bout.toByteArray();

And then calling the functions using a ByteArrayInputStream with the contents of barr.
getWords(new ByteArrayInputStream(barr));
getWordFeature(new ByteArrayInputStream(barr));

I think this is the best solution if you can not change/refactor the methods. This solution gets pretty cool if you implement it properly as a class.

Some issue with bufferedReader


Making Github pages searchable

Question: I have made a github webpage using R studio that I'm more or less happy with. Yay! I used the following tutorial to make it. Trouble is I cannot seem to 'find' it when I google it (even though I know what I'm looking for).
StackExchange::Webmasters have suggested I may need to add a sitemap.xml.
Question: Do I really need a sitemap.xml? If so, how do I go about making one using R studio, and where does it belong in my github pages directory?
Comment: I'm voting to close this question as off-topic because this is Off-topic to StackOverflow since it is more suited to Webmasters.
Comment: Here is the link to [Webmasters](https://webmasters.stackexchange.com/).
Comment: To better explain the off-topic reason: Only programming related questions or queries about programming related tools are only allowed in StackOverflow, refer also to [MCVE](https://stackoverflow.com/help/mcve) for writing on-topic questions. Even though Github maybe considered as one of that tools the context of your question is about SEO which more suited to Webmasters.
Comment: My apologies. It's not clear to me where "creating a github webpage" ends and where "webmasters" (whatever those are) begin. Thanks for your patience.
Comment: @Enzokie Sorry about that - should I delete it then?
Comment: since you have done migrating your question to Webmasters you can delete this question to avoid further downvotes. I understand that it sounds demotivating but that's how this site works, incase you have programming question feel free to ask in this site. If you have some issue deleting this question you can contact a moderator by flagging the question and explain that you wanted to remove it since you already migrated it :)
Comment: @Enzokie no problem, Stackoverflow has saved my life a million times (one 0.01% of which have been from my own questions), and you have been entirely respectful and polite. What's demotivating is downvotes with zero explanation. Thanks for all your help. Will delete now.
Answer: Google crawlers index the web 4 times a day (at least they used to, maybe more now).  I see you last updated your repository a month ago.
If you search site:cpryan.github.io Google is supposed to return the pages it has indexed.
You can also claim your site on Google's WebMaster site (by putting a small file in your repo with a code Google give you) to see even more information about what Google has or maybe has not indexed.
Comment: Ok thanks. I'm still confused as to why none of that appears when I search my name, defeating the purpose of a website. Thanks for your help though, and apologies for the off-topic post.

Making Github pages searchable


Double exponential fit in R

Question: I am not yet very good friends with R, but I hope to be! Right now I'm having a lot of troubles with fitting a model to a set of data.
My data looks like:
data <- as.data.frame(cbind(c(0,5,10,15,30,50,60,90,120,180,240),c(0.48, 1.15, 1.03, 1.37, 5.55, 16.77, 20.97, 21.67, 10.50, 2.28, 1.58)))
colnames(data) <- c("times","RNA")

The column RNA is dependent on the time.
I have been trying to fit a double exponential curve to the data, but I don't think it is working so well. This is the code for the fitting:
library(minpack.lm)
fit <- nlsLM(RNA~z*(exp(-v*times)-exp(-u*times)),data=test_2,start=list(u=0.67,v=0.018,z=0.98))

plot(data,pch=19)
lines(data[1],fitted(fit),col=2)

which produces this:

I am not very happy with the fit, since it doesn't really include the "top" of the data (max?). Do any of you have any suggestions about what to do next? I have no clue about what to try else, but maybe another exponential?
Do you have some suggested reading for this, since I'm not looking for a direct answer but more for some ideas...if it's possible!
Comment: Hi, based on your code and the plot, you seem to have no obvious difficulty with the coding aspect of this problem, which means you might want to ask Cross Validated.  Judging from the curve, I think it is really not bad. Keep in mind that you don't want to overfit, as overfitting is typically associated with low generalizability. You could, though, try fitting several related models: https://en.wikipedia.org/wiki/Relationships_among_probability_distributions
Comment: @DavidC. Oh okay, thanks! Maybe I really don't trust my ability in R, so this was just to make sure everything is correct :) I will see if Cross Validated can help me then!
Comment: @Gliz May be you can check out splines. With enough degree of freedom, splines can fit all the points exactly, but as David said this may not be a good idea. [R code example](https://www.r-bloggers.com/splines-opening-the-black-box/) [Spline primer](https://cran.r-project.org/web/packages/crs/vignettes/spline_primer.pdf)
Comment: I often find it useful to have a go at manually changing the parameters in the model to see if I can get it to look better than the fit does. That can often tell you if the fit's got stuck in a local minimum or something like that. I don't know how to do it in R though.
Comment: You are asking essentially the same question here twice.
Comment: @MichaelChernick I didn't know you could migrate the question :) I have deleted the duplicate :)
Answer: The functional form of your double exponential curve doesn't capture the shape of your data. Notice that the best fit that you obtained is concave down at $t=0$, where you might expect a gentle slope and positive second derivative.
The "double exponential" functional form is usually associated with the Laplace distribution:
$$f(t):= a\exp\left(-\frac{|t-c|}b\right)
$$
where $a$ measures the height, $b$ the 'slope', and $c$ the location of the peak. Here's an R implementation using the nls function:
test <- data.frame(times=c(0,5,10,15,30,50,60,90,120,180,240), RNA=c(0.48, 1.15, 1.03, 1.37, 5.55, 16.77, 20.97, 21.67, 10.50, 2.28, 1.58))

nonlin <- function(t, a, b, c) { a * (exp(-(abs(t-c) / b))) }
nlsfit <- nls(RNA ~ nonlin(times, a, b, c), data=test, start=list(a=25, b=20, c=75))

with(test, plot(times, RNA, pch=19, ylim=c(0,40)))
tseq <- seq(0,250,.1)
pars <- coef(nlsfit)
print(pars)
lines(tseq, nonlin(tseq, pars[1], pars[2], pars[3]), col=2)

The resulting fit is OK, but it doesn't capture the right skew in your data set, nor does it do a good job near zero:

One way to fix this is to use a different slope for the left half and the right half of the peak, yielding a four-parameter model:
nonlin <- function(t, a, bl, br, c) {
    ifelse(t < c, a * exp((t-c) / bl), a * exp(-(t-c) / br))
}

nlsfit <- nls(RNA ~ nonlin(times, a, bl, br, c), data=test, start=list(a=25, bl=20, br=20, c=75))

With this modification the fit is quite a bit better:

Another approach is to take the log of the time values to remove the skew. In essence you're fitting a double exponential relationship between RNA and log(time):
nonlin <- function(t, a, b, c) { a * (exp(-(abs(log(t)-c) / b))) }
nlsfit <- nls(RNA ~ nonlin(times, a, b, c), data=test, start=list(a=25, b=1, c=4))

With this three parameter model the fit is closer to zero at $t=0$ (perhaps too close to zero), while the right tail has gotten thicker:

Finally, you can fit a lognormal distribution to your data. In this case you are positing a bell shape for RNA vs log(time):
nonlin <- function(t, a, b, c) { a * (exp(-(log(t)-c)^2 / b)) }
nlsfit <- nls(RNA ~ nonlin(times, a, b, c), data=test, start=list(a=25, b=1, c=4))

Here the resulting fit is perhaps overly aggressive: it doesn't become positive until well past $t=0$.

BTW, here is an R implementation of the fit to the Gumbel distribution, which is sometimes known as the double exponential. This is the functional form used in James Phillips' answer, and perhaps what you intended to code up.
nonlin <- function(t, a, b, c) { (a/b) * exp( -(t-c)/b - exp(-(t-c)/b) ) }
nlsfit <- nls(RNA ~ nonlin(times, a, b, c), data=test, start=list(a=1000, b=50, c=75))

Which functional form is the right one? Which one to adopt depends not only on the quality of the fit but also on the mechanism that generated the data. Is there a theoretical/biological reason for any of these functional forms? OTOH, maybe none of these forms is justified, and you just want to describe what you've observed using a small number of parameters.
Comment: I must say the form of the last plot fits the mechanism best, and that's why I started with a double exponential fit - and my supervisor suggested it :P
Comment: Can I ask if you have an idea of how to determine what fit is the best in R? is that the R^2 I should be looking at?
Comment: @Gliz Hang on, see my edit: the Gumbel distribution is what you probably intended.
Comment: @Gliz You should compute something like $R^2$, which R can return easily. I imagine the Gumbel fit might win on those grounds, and if your supervisor suggested it, then there may have been some theoretical justification for it as well :)
Comment: I think that the Gumbel looks very convincing! Thank you very much :D
Answer: I also tried fitting the data you posted to a double exponential equation with similar results.
If you might use a different equation, I was able to fit the data you posted to the Extreme Value peak equation with good results as shown in the attached graph.  It seems to fit well near the origin also.
y = a * exp(-exp(-((x-b)/c))-((x-b)/c)+1.0)
Fitting target of lowest sum of squared absolute error = 5.6978486427595030E+00
a =  2.3314874489295441E+01
b =  7.2755959004396104E+01
c =  3.2201742765438965E+01

Degrees of freedom (error): 8
Degrees of freedom (regression): 2
Chi-squared: 5.69784864276
R-squared: 0.993392917433
R-squared adjusted: 0.991741146791
Model F-statistic: 601.410929778
Model F-statistic p-value: 1.90563154145e-09
Model log-likelihood: -11.9903875636
AIC: 2.72552501157
BIC: 2.83404190415
Root Mean Squared Error (RMSE): 0.719712609484

a = 2.3314874489295441E+01
       std err: 4.62920E-01
       t-stat: 3.42673E+01
       p-stat: 5.74859E-10
       95% confidence intervals: [2.17459E+01, 2.48838E+01]

b = 7.2755959004396104E+01
       std err: 1.20647E+00
       t-stat: 6.62384E+01
       p-stat: 3.00249E-12
       95% confidence intervals: [7.02231E+01, 7.52889E+01]

c = 3.2201742765438965E+01
       std err: 1.57788E+00
       t-stat: 2.56355E+01
       p-stat: 5.74859E-09
       95% confidence intervals: [2.93051E+01, 3.50984E+01]

Coefficient Covariance Matrix
[ 0.64995768 -0.01513588 -0.67014552]
[-0.01513588  1.69393358  0.83602731]
[-0.67014552  0.83602731  2.2154065 ]

Comment: Thanks! This was exactly what I was looking for! What fitting command did you use in R? Just so I can read up on it :)
Comment: I did not use R.  Years ago I ran a online curve and surface fitting web site named zunzun.com, and this had a "function finder" with a genetic algorithm front end for initial parameter estimation.  I used the code to fit your data to over two hundred known equations with three or less parameters, and chose this one from the sorted results.  The Python BSD-licensed source code is here if you would like to try it: https://github.com/zunzun/zunzunsite3
Comment: Oh that is quite cool! I will check it out definitely

Double exponential fit in R


Android: Boolean

Question: I trying to draw a circle only if the button is clicked. And to move the circle, a button has to be clicked in order to do the moving. The initial value of the bclick is false however after clicking on the button, the value of bclick becomes true all the way. How to make it such that the bclick is always set to false every time after the the circle is drawn?
My View class:
<code>  @Override
        public boolean onTouchEvent(MotionEvent event) {

                x = event.getX();
                y = event.getY();

                switch (event.getAction()) {
                case MotionEvent.ACTION_DOWN:

                if (MainActivity.isButtonClicked() == true ){

                                      canvas.drawCircle(x, y, radius, paint)

                                 invalidate();
                              }

                           break;
                      case MotionEvent.ACTION_MOVE:

                          lastx = event.getX();
                          lasty = event.getY();
                if (MainActivity.isMoveButtonClicked() == true && Mainactivity.is ButtonCLicked() == false ){

                                      canvas.drawCircle(lastx, lasty, radius, paint)

                     invalidate();// if this is used, the moving is slow and doesnt move to the exact spot at (lastx, laxyy)
                              }

                           break;
            }
</code>
My MainActivity:
<code>boolean bclick =false;
    boolean mbclick =false;

drawbutton.setOnClickListener(new OnClickListener() {

            @Override
            public void onClick(View arg0) {

            bclick = true;

            }

        });

        bclick = false;

                movebutton.setOnClickListener(new OnClickListener() {

            @Override
            public void onClick(View arg0) {

                   mbclick = true;

            }

        });

               mbclick = false;

public static boolean isButtonClicked() {
        return bclick;
    }

public static void setButtonClicked(boolean torf)
    {
      bclick = torf;
    }

public static boolean isMoveButtonClicked() {
        return mbclick;
    }

public static void setMoveButtonClicked(boolean torf)
    {
      mbclick = torf;
    }
</code>
Answer: You could create a method to draw the circle and check it,
<code> if(circleIsDrawn){

      bClick = false;

}
</code>
Answer: <code> @Override
public boolean onTouchEvent(MotionEvent event) {

        x = event.getX();
        y = event.getY();

        switch (event.getAction()) {
        case MotionEvent.ACTION_DOWN:

        if (MainActivity.isButtonClicked() == true ){

                              canvas.drawCircle(x, y, radius, paint)
                              bClick=false;
                         invalidate();
                      }

                   break;
    }
</code>
Comment: I have tried this method however when I try to move the circle(after pressing another button), the moving of the circle became slow and does not stop at the last touched coordinates. I have updated this with more codes. Please advice thank you.

Android: Boolean


Sum Combination List

Question: I need an algorithm for this problem:
Given a set of n natural numbers x1,x2,...,xn, a number S and k. Form the sum of k numbers picked from the set (a number can be pick many times) with sum S.
Stated differently: List every possible combination for S with Bounds: n<=256, x<=1000, k<=32
E.g.
  problem instance: {1,2,5,9,11,12,14,15}, S=30, k=3
  There are 4 possible combinations
  S=1+14+15, 2+14+14, 5+11+15, 9+9+12.

With these bounds, it is unfeasible to use brute force but I think of dynamic programming is a good approach.
The scheme is: Table t, with t[m,v] = number of combinations of sum v formed by m numbers.
1. Initialize t[1,x(i)], for every i.
2. Then use formula t[m,v]=Sum(t[m-1,v-x(i)], every i satisfied v-x(i)>0), 2<=m<=k.
3. After obtaining t[k,S], I can trace back to find all the combinations.

The dilemma is that t[m,v] can be increase by duplicate commutative combinations e.g., t[2,16]=2 due to 16=15+1 and 1+15. Furthermore, the final result f[3,30] is large, due to 1+14+15, 1+15+14, ...,2+14+14,14+2+14,...
How to get rid of symmetric permutations? Thanks in advance.
Comment: Looks like a Knapsack problem (http://en.wikipedia.org/wiki/Knapsack_problem)
Comment: In particular, look like integer partitioning problem http://en.wikipedia.org/wiki/Partition_%28number_theory%29 without number k
Answer: You can get rid of permutations by imposing an ordering on the way you pick elements of x. Make your table a triple t[m, v, n] = number of combinations of sum v formed by m numbers from x1..xn. Now observe t[m, v, n] = t[m, v, n-1] + t[m-1, v-x_n, n]. This solves the permutation problem by only generating summands in reverse order from their appearance in x. So for instance it'll generate 15+14+1 and 14+14+2 but never 14+15+1.
(You probably don't need to fill out the whole table, so you should probably compute lazily; in fact, a memoized recursive function is probably what you want here.)

Sum Combination List


How would I push the button to the bottom of the CSS flexbox in such case?

Question: I have created multiple CSS flexboxes on this landing page and I would like to have the SELECT button to be always on the bottom of each box, no matter the size of textual content above. I tried to use multiple properties such as align-self, justify-content, align-items, align-text with various attributes but could not achieve the desired outcome.
Would could I possibly do in this situation to make it work?
Here is the screenshot of flexboxes:

Here is my HTML and CSS for this section:

#pricing-option-1, #pricing-option-2, #pricing-option-3 {
  display: flex;
  flex-direction: column;
  border-style: solid;
  border-color: rgb(230, 230, 230);
  border-radius: 25px;
  padding: 25px;
  margin: 10px;
  width: 30%;
  background-color: yellow;
}

.pricing-amount {
  background-color: blue;
}

BASIC ENBALI

$1200/month

The most popular option. You wouldn't need to worry about finding a place to live, commute to the classes, cooking or find a right place to eat. We got you!

80 hours of any classes

Accommodation on campus (4 people per room)

Meal plan (3 times a day)

Unlimited access to our library

Unlimited access to our online courses library

            <button type="submit">SELECT</button>

Answer: First you need a defined height for the container to avoid that the height adjusts to the contents (I added that in the snippet below, and I erased some of the contents to make it more obvious).
Then you can apply margin-top: auto; to the last element (the container for the button) wo automatically move that to the bottom (end of the flex order)

#pricing-option-1,
#pricing-option-2,
#pricing-option-3 {
  display: flex;
  flex-direction: column;
  border-style: solid;
  border-color: rgb(230, 230, 230);
  border-radius: 25px;
  padding: 25px;
  margin: 10px;
  width: 30%;
  background-color: yellow;
}

.pricing-amount {
  background-color: blue;
}

#pricing-option-2 {
  height: 500px;
}
.v {
   margin-top: auto;
}

BASIC ENBALI

$1200/month

The most popular option. place to eat. We got you!

80 hours of any classes

Accommodation on campus (4 people per room)

Unlimited access to our online courses library

    <button type="submit">SELECT</button>

Comment: @PeterDvukhrechensky As I wrote: to avoid that the height adjusts to the contents
Comment: I understand that, but why? What if the content takes more than the height value?
Comment: @PeterDvukhrechensky depends: If you set `flex-wrap`, it will wrap into a second column, if you use `overflow: visible`, it will grow, if you do nothing, a scrollbar will appear. But that's not part of the question – the image in the question shows elements of equal height, some of would be less high if no height were set. Since the OP marked my answer as accepted, I seem to have been right about height and margin-top: auto
Answer:
First make the .pricing-body block to be a flexbox.
Then set flex-grow: 1 on .select-option which will make it fill all of the free space
Finally align the content of .select-option with align-items: flex-end to keep the button to the bottom

Just add this piece of code to your CSS:
.pricing-body {
  display: flex;
  flex-direction: column;
}

.select-option {
  flex-grow: 1;
  display: flex;
  align-items: flex-end;
}

See this codepen example


Can install Enterprise app on all tested iPads except for one

Question: I have an enterprise app, that has been installed on about 40 iPads without problems, and suddenly a user comes along with an iPad where the app simply won't install.
I have tried:

Restarting the iPad
Deleting the profiles
Checking restrictions

But so fare without luck
I dont have physical access to the iPad so I cant see any log.
Any suggestions?
Comment: have you checked the CPU architecture of your device and in your build setting?
Comment: tried reseting the iPad?
Comment: No. I have a feeling the user would be a bit upset about this approch. I have, however, tried the hard restart.
Comment: How do you install the app on the iPad? With iTunes, over the air? What is the error message?
Comment: Over the air, using a simple webpage with a link to the app. We have 5 different apps using the same strategy, where it, so fare, just works. The error message is a pretty generic one: "The app could not be downloaded". It the same message you get when there is a problem with the provisioning.
Comment: @walinaqvi: A really good suggestion. Unfortunately the Architecture is correct. The app is running on several iPad4s and this is just another iPad4.
Answer: It turns out the problem was related to the time on the device. The clock was set 1 hour off.
Answer: Check the iPad system version is suitable for your xcode, Check if this device are in your support list, It is device are jailbreak?
Comment: The device is running a supported version of iOS and is not jail broken
Comment: It's there any other Bundle ID conflict or something?
Comment: I'm pretty sure it is somehow related to the the specific device's settings. It is only a problem on that unit


openann c++ compilation fail

Question: I have a problem compiling some c++ code with OpenANN.
I wonder if you can help me, here is the log of make:
log
I have installed he dependancies for OpenANN.

Errors from link:

g++ -std=c++11 -O3  main.cpp  -I/usr/include/eigen3 -L/usr/include/OpenANN -L/usr/lib -lutil -lboost_system -lboost_filesystem -lboost_iostreams -o Bot.out:

/tmp/cc78iiwc.o: dans la fonction « Brain::ComputeTarget(Brain::MemCell const&) »:
main.cpp:(.text._ZN5Brain13ComputeTargetERKNS_7MemCellE[_ZN5Brain13ComputeTargetERKNS_7MemCellE]+0xa7): référence indéfinie vers « OpenANN::Net::operator()(Eigen::Matrix const&) »
main.cpp:(.text._ZN5Brain13ComputeTargetERKNS_7MemCellE[_ZN5Brain13ComputeTargetERKNS_7MemCellE]+0x18d): référence indéfinie vers « OpenANN::Net::operator()(Eigen::Matrix const&) »
/tmp/cc78iiwc.o: dans la fonction « main »:
main.cpp:(.text.startup+0x2cf): référence indéfinie vers « OpenANN::Net::operator()(Eigen::Matrix const&) »
main.cpp:(.text.startup+0xa16): référence indéfinie vers « OpenANN::DirectStorageDataSet::DirectStorageDataSet(Eigen::Matrix*, Eigen::Matrix*, OpenANN::Evaluator*) »
main.cpp:(.text.startup+0xa2f): référence indéfinie vers « OpenANN::Learner::trainingSet(OpenANN::DataSet&) »
main.cpp:(.text.startup+0xa3c): référence indéfinie vers « OpenANN::StoppingCriteria::StoppingCriteria() »
main.cpp:(.text.startup+0xa8d): référence indéfinie vers « OpenANN::train(OpenANN::Net&, std::string, OpenANN::ErrorFunction, OpenANN::StoppingCriteria const&, bool, bool) »
main.cpp:(.text.startup+0xbf4): référence indéfinie vers « OpenANN::Net::save(std::string const&) »
main.cpp:(.text.startup+0xfd5): référence indéfinie vers « OpenANN::Net::operator()(Eigen::Matrix const&) »
main.cpp:(.text.startup+0x1064): référence indéfinie vers « OpenANN::Net::operator()(Eigen::Matrix const&) »
main.cpp:(.text.startup+0x11f6): référence indéfinie vers « OpenANN::DirectStorageDataSet::DirectStorageDataSet(Eigen::Matrix*, Eigen::Matrix*, OpenANN::Evaluator*) »
main.cpp:(.text.startup+0x120f): référence indéfinie vers « OpenANN::Learner::trainingSet(OpenANN::DataSet&) »
main.cpp:(.text.startup+0x121c): référence indéfinie vers « OpenANN::StoppingCriteria::StoppingCriteria() »
main.cpp:(.text.startup+0x126d): référence indéfinie vers « OpenANN::train(OpenANN::Net&, std::string, OpenANN::ErrorFunction, OpenANN::StoppingCriteria const&, bool, bool) »
main.cpp:(.text.startup+0x15f6): référence indéfinie vers « OpenANN::DirectStorageDataSet::DirectStorageDataSet(Eigen::Matrix*, Eigen::Matrix*, OpenANN::Evaluator*) »
main.cpp:(.text.startup+0x160f): référence indéfinie vers « OpenANN::Learner::trainingSet(OpenANN::DataSet&) »
main.cpp:(.text.startup+0x161c): référence indéfinie vers « OpenANN::StoppingCriteria::StoppingCriteria() »
main.cpp:(.text.startup+0x166d): référence indéfinie vers « OpenANN::train(OpenANN::Net&, std::string, OpenANN::ErrorFunction, OpenANN::StoppingCriteria const&, bool, bool) »
main.cpp:(.text.startup+0x1900): référence indéfinie vers « OpenANN::Net::~Net() »
/tmp/cc78iiwc.o: dans la fonction « Brain::Brain() »:
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x37): référence indéfinie vers « OpenANN::Net::Net() »
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x58): référence indéfinie vers « OpenANN::Net::load(std::string const&) »
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x115): référence indéfinie vers « OpenANN::Net::inputLayer(int, int, int) »
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x134): référence indéfinie vers « OpenANN::Net::fullyConnectedLayer(int, OpenANN::ActivationFunction, double, bool) »
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x153): référence indéfinie vers « OpenANN::Net::outputLayer(int, OpenANN::ActivationFunction, double, bool) »
main.cpp:(.text._ZN5BrainC2Ev[_ZN5BrainC5Ev]+0x16d): référence indéfinie vers « OpenANN::Net::~Net() »
collect2: erreur: ld a retourné 1 code d'état d'exécution
Makefile:6 : la recette pour la cible « test » a échouée
make: *** [test] Erreur 1
Comment: Those are not really compiler errors, but *linker* errors, and they say that you are missing many symbols from the OpenANN library, presumably because you don't link with it. (At least I'm guessing that, because I don't read French (I guess it is), few here do. Please change locale so you get English output in the future when you post questions.)
Comment: @JoachimPileborg: I confirm it's French, and it's indeed "missing reference to" that is translated.
Answer: Your compilation command is erroneous.

g++ -std=c++11 -O3  main.cpp  -I/usr/include/eigen3 -L/usr/include/OpenANN \
    -L/usr/lib -lutil -lboost_system -lboost_filesystem -lboost_iostreams -o Bot.out

The link options that are -L and -l have two different roles:

-L means to consider the directory when looking for libraries
-l means to link with this library

If you look exactly at the flags concerning OpenANN that you pass you will realize that you instructed g++ too look into the /usr/include/OpenANN directory when searching for libraries, but never actually instruct it to link against the OpenANN library.
If g++ is not instructed to link the program/library with the OpenANN library it will not, and you'll get undefined symbols.

openann c++ compilation fail


PL/SQL ORA-00902: invalid datatype

Question: I am trying to put this piece of code to a stored procedure. But the Update statement is throwing invalid datatype error. Any help or advice would be highly appreciated.
CREATE TABLE "TABLE1"
(
    "USER_ID" NUMBER,
    "COMMENTS" VARCHAR2(20 BYTE)
)

PROCEDURE pr_test
AS
    v_table UserRecordTable;
    v_user_id_list SYS.ODCINUMBERLIST;
BEGIN
    select USER_ID bulk collect into v_table
    from TABLE1;

    select USER_ID bulk collect into v_user_id_list
    from TABLE1
    where USER_ID in (select USERID FROM TABLE(v_table)); -- No problem

FOR i IN 1..v_user_id_list.COUNT LOOP
    DBMS_OUTPUT.PUT_LINE('Column1: ' || v_user_id_list(i)); -- printout ok
END LOOP;

UPDATE TABLE1    -- This line throws the error
SET COMMENTS= 'NO NO'
where USER_ID in (select USERID FROM TABLE(v_table));
END pr_der_test;

In Spec
   TYPE UserRecordType IS RECORD (
    USERID TABLE1.USER_ID%TYPE
  );
TYPE UserRecordTable IS TABLE OF UserRecordType;

Comment: Please show the definition of `v_table`, `v_user_id_list` and `table1`.  You provided the spec for `UserRecordTable ` but that appears nowhere in the code you've shared.
Comment: @rich I flagged it as dupe because you created 2 - the other one is now deleted
Comment: Does this answer your question? [PL/SQL ORA-00902: invalid datatype](https://stackoverflow.com/questions/77395892/pl-sql-ora-00902-invalid-datatype)
Comment: @Paul I have updated the sample code. Please have a look.  And USERID is Number type in table1.
Comment: @KoenLostrie wrong link?
Comment: @Rich, what about `PriorityRecordTable` and the DDL for `TABLE1` ? We still don't know anything about these.
Comment: @PaulW That was a typo. it should be UserRecordTable. And TABLE1 has USER_ID  which is Number type. I made some simplification for this sample case because I only use this one field for this user case.
Comment: [fiddle](https://dbfiddle.uk/LK6Ij_H4) demonstrating the error. @Rich - you should include a complete [MRE] in your question that we can just copy-paste and execute. If we have to try to reverse engineer your setup from the question then it increases the burden on the answerers and makes it less likely that you are going to get assistance. Help us to help you and provide us with everything we need in an usable format.
Comment: @MT0 I have updated the question with full example. And I have tested this simplified code which can be used for reproducing the error.
Answer: The problem probably relates to records being a PL/SQL data-type and that their support is limited in SQL statements (but increasing in more recent versions).
You can work around the problem using v_user_id_list in the query (which uses SYS.ODCINUMBERLIST, a data-type declared in the SQL scope and not in a PL/SQL scope of a package).
CREATE PACKAGE package_name
AS
  TYPE UserRecordType IS RECORD (USERID TABLE1.USER_ID%TYPE);
  TYPE UserRecordTable IS TABLE OF UserRecordType;
  PROCEDURE pr_test;
END;
/

CREATE PACKAGE BODY package_name
AS
  PROCEDURE pr_test
  AS
    v_table UserRecordTable;
    v_user_id_list SYS.ODCINUMBERLIST;
  BEGIN
    select USER_ID
    bulk collect into v_table
    from  TABLE1;

    select USER_ID
    bulk collect into v_user_id_list
    from TABLE1
    where USER_ID in (select USERID FROM TABLE(v_table)); -- No problem

    FOR i IN 1..v_user_id_list.COUNT LOOP
      DBMS_OUTPUT.PUT_LINE('Column1: ' || v_user_id_list(i)); -- printout ok
    END LOOP;

    UPDATE TABLE1    -- This line throws the error
    SET COMMENTS= 'NO NO'
    -- where USER_ID in (select USERID FROM TABLE(v_table))
    where USER_ID in (select COLUMN_VALUE FROM TABLE(v_user_id_list))
    ;
  END;
END;
/

Given the sample data:
CREATE TABLE table1(
  user_id  NUMBER,
  comments VARCHAR2(20)
);

INSERT INTO table1 (user_id, comments)
SELECT 1, NULL FROM DUAL UNION ALL
SELECT 2, 'YES' FROM DUAL;

Then:
BEGIN
  DBMS_OUTPUT.ENABLE();
  package_name.pr_test;
END;
/

Outputs:

Column1: 1
Column1: 2

And the table then contains:

USER_ID
COMMENTS

1
NO NO

2
NO NO

fiddle
Comment: @MTO Thanks so much. This works. But if I need other columns, the list is not enough. That was why I tried to user Table of Record. And one more question, I was wondering why would this works with the same where clause?

select USER_ID
    bulk collect into v_user_id_list
    from TABLE1
    where USER_ID in (select USERID FROM TABLE(v_table));
Comment: @Rich As I said in the answer, support for PL/SQL data-types is limited in the SQL scope. It may be that record types are supported in `SELECT` statements but have either limited or no support in `UPDATE` statements. I don't have a good reference for what support is available where and it would also depend on which version of the database you are using.
Comment: If you want to use multi-column data-types then declare an `OBJECT` data-type in the SQL scope and that should be supported in both the PL/SQL and SQL scopes.
Comment: @MTO Much appreciated! I will accept this answer and give it a try with OBJECT data-type.

PL/SQL ORA-00902: invalid datatype


Issue with Dynamics 365 for Outlook add ins

Question: we configured outlook 2016 with Dynamics 365 with help of  Dynamics 365 for Outlook add ins. The contacts are Syncing fine but every time when I open the outlook 2016 in my machine, its asking us to add the  Dynamics 365 for Outlook add ins to Enable. Please suggest us how the Add ins should stay connected or Enabled longer instead of adding the add Ins every time.
Thank you. Suggestions are appreciated.
Answer: Looks like the add-in was installed/registered twice. You need to remove the second instance from Outlook.
Microsoft Office applications can disable Add-ins that behave unexpectedly. If an application does not load your Add-in, the application might have hard disabled or soft disabled your VSTO Add-in.
Hard disabling can occur when an Add-in causes the application to close unexpectedly. It might also occur on your development computer if you stop the debugger while the Startup event handler in your VSTO Add-in is executing.
Soft disabling can occur when a Add-in produces an error that does not cause the application to unexpectedly close. For example, an application might soft disable a Add-in if it throws an unhandled exception while the Startup event handler is executing.
When you re-enable a soft-disabled Add-in, the application immediately attempts to load the Add-in. If the problem that initially caused the application to soft disable the Add-in has not been fixed, the application will soft disable the Add-in again. Read more about that in the How to: Re-enable a VSTO Add-in That Has Been Disabled article.


Setup 'trust-certificate-pem' property in YAML config format

Question: Im trying to setup a trust certificate in my Quarkus application for my postgres connection.
Sadly I wasted my time yesterday, by not noticing that I configured the property wrong.
That lead to a io.vertx.core.VertxException: SSL handshake failed error.
The main cause was that I converted the example config of application.properties wrong to the application.yml format, since I am using the YAML extension. After testing the application properties variant it worked.
Now the question: How do I set is up correctly in YML format ?
# Working application.properties format
quarkus.datasource.reactive.trust-certificate-pem=true
quarkus.datasource.reactive.trust-certificate-pem.certs=somefile.pem

# Wrong YAML formats, which I have tried so far

# V1
quarkus:
   datasource:
      reactive:
         trust-certificate-pem: true
         trust-certificate-pem:
            certs: somefile.pem

#V2
quarkus:
   datasource:
      reactive:
         trust-certificate-pem: true
         trust-certificate-pem.certs: somefile.pem

#V3
quarkus:
   datasource:
      reactive:
         trust-certificate-pem: true
         trust-certificate-pem.certs: somefile.pem

#V4
quarkus:
   datasource:
      reactive:
         trust-certificate-pem:
            value: true
            certs: somefile.pem

Anyone got an idea ?
I really wished the Quarkus team had a toggle switch to see a YAML example :/
Comment: That is not  the [YML](https://fdik.org/yml/) format, and the officially recommended extension for files containing YAML documents has been `.yaml` since at least Sept. 2006.
Answer: YAML specification has a miscellaneous tag for null. This is the ~ (tilde) sign.
The following config will work
quarkus:
  datasource:
    reactive:
      trust-certificate-pem:
        ~: true
        certs: somefile.pem

Comment: Thank you very much! Didn't knew that


How to retrieve App Service Plan instance name (RDXXXXX)?

Question: I am having an issue trying to retrieve all the instance names in an Azure App Service Plan.
In Azure Monitor, if you specify a "Scope" to "App Service Plan" and look at the Metric "CPU Percentage" and then add a filter to specify the "Instance" property, you can see which instance uses the most CPU.  I am trying to do a PowerShell script to get these values.  Unfortunatly I have not found any Azure REST Api that would give me this information so that my script would be 100% dynamic.  I looked at the AzureRm or Az PowerShell modules but did not find anything there.
Any ideas how I can retrieve this list?  The instances names looks like this :RD123456.
Thanks for you help !
Answer: Suppose you want to get the web instance name, if that's right you could get it from instance process with Web Apps - Get Instance Process.
And under the environment_variables there is a COMPUTERNAME suppose this is what you want.
Comment: is not it the same thing answered below?
Comment: Check the rest description, it's not same thing.
Comment: @Sajeetharan,what you give is list app plan and what I give is Get Instance Process. And did you check my picture content?
Comment: check the comment below, I have mentioned he needs to use List to get all instances and get method to get the name
Comment: @Sajeetharan,Did you try your get method? Yours is this [one](https://learn.microsoft.com/en-us/rest/api/appservice/appserviceplans/get) and check again what's mine. One is plan get and one is web instance get. If you insist they are same, I have nothing to say.
Comment: Thanks for the awnser.  I remember trying this out but the values where all nulls.  I will give it a try with another account and let you guys know.  It seems to be what I am looking for.
Comment: I actually tested this solution before without success (the environment variable was empty). I investigated and it seems that the account I was using did not have enough rights to view the environment variables.  Using a contributor account worked.  I will take a look at which authorization would be necessary other than "Contibutor"
Comment: After further testing, the "Contributor" role is necessary in order to be able to read Environment_Variables properties.  Otherwise, the values will be empty in the property.  Hope this helps.


How to assign cloned functions to new this instance

Question: I have following code, written in ES6 plus some Stage 3 proposals:
class Parent {
  constructor(x){
    this.x = x;
    this.otherProperty = "preserve across copy";
  }

  printX=()=>{
    console.log(this.x);
  }

  squareX=()=>{
    this.x = this.x *this.x;
  }
}

class Child extends Parent {
  constructor(x){
    super(x);
  }
}

const c = new Child(20);
const copy = {...c, x: 10};

console.log(JSON.stringify(c));
c.squareX();
console.log(JSON.stringify(c));

console.log(JSON.stringify(copy));
copy.squareX();
console.log(JSON.stringify(copy));

Live demo: https://jsbin.com/wuqenabaya/1/edit?js,console
The idea is to create a copy of the instance c while updating some of it's properties. The output of this code is:
{x:  20, otherProperty: "preserve across copy"}
{x: 400, otherProperty: "preserve across copy"}
{x:  10, otherProperty: "preserve across copy"}
{x:  10, otherProperty: "preserve across copy"}

So as you can see the copy.squareX() does not update the instance copy. The issue is that the function squareX() is still bound to the old instance c.
What I want is to have the last call to squareX() to update the instance copy. How can this be achieved?
EDIT: I'm using Babel with following plugins, to allow the use of the new JS features (spread, function props).
{
  "presets": [
    ["es2015", { "modules": false }],
    "stage-0",
    "react"
  ],
  "plugins": [
    "react-hot-loader/babel",
    "transform-object-rest-spread",
    "transform-class-properties",
    "transform-flow-strip-types"
  ]
}

Comment: Note: "ES6" (ES2015) doesn't have spread properties. In fact, they aren't even in ES2017. They may be in ES2018; currently a [stage 3 proposal](https://github.com/tc39/proposal-object-rest-spread).
Comment: is it an option to update the classes? i'm thinking of something like a static factory function that as argument accepts an class instance, reads the needed properties, generates a new instance, updates those properties, and returns the new instance. instead of cloning the old instance via object literal
Comment: Similarly, the [field declaration syntax](https://github.com/tc39/proposal-class-fields) you're using is *also* not yet in any spec, and also at Stage 3.
Comment: @T.J.Crowder: I'm actually using Babel. I have update the answer to reflect this fact.
Comment: Well [just don't use arrow functions when you don't want your methods be bound](https://stackoverflow.com/q/34361379/1048572) - the code would totally work with `function` expressions.
Answer: There are a few problems with trying to use spread properties for this, not least that you'll end up with an object whose prototype is Object.prototype, not Child.prototype.
You're also using field syntax for methods, which makes those fields own properties of the object, not prototype properties, which there doesn't seem to be any reason for in this case; just use method syntax.
To do the copy, give yourself a "copy constructor" branch in your constructor:
constructor(x, ...more){
  if (x instanceof Child) {
    // The copy constructor branch
    super(x.x);
    Object.assign(this, x, ...more);
  } else {
    super(x);
  }
}

Or if you prefer, just do it where you create copy:
const copy = Object.assign(new Child(c.x), c, {x: 10});

In either case, if you choose to keep using fields rather than methods, you'll have to adjust things since otherwise you'll copy squareX and printX as well.
Live Example using methods and the copy constructor:

class Parent {
  constructor(x){
    this.x = x;
    this.otherProperty = "preserve across copy";
  }

  printX() {
    console.log(this.x);
  }

  squareX() {
    this.x = this.x *this.x;
  }
}

class Child extends Parent {
  constructor(x, ...more){
    if (x instanceof Child) {
      super(x.x);
      Object.assign(this, x, ...more);
    } else {
      super(x);
    }
  }
}

const c = new Child(20);
const copy = new Child(c, {x: 10});

console.log(JSON.stringify(c));
c.squareX();
console.log(JSON.stringify(c));

console.log(JSON.stringify(copy));
copy.squareX();
console.log(JSON.stringify(copy));


a2enmod not working on debian (buster)

Question: Running a2enmod gives:
bash: a2enmod: command not found
I've hunted high and low and can't find any answers. Is there something I need to do/install?
Comment: Have you installed apache2?
Comment: You need to run the command as root. You can do this with `sudo a2enmod ...`
Comment: I've been running the command as root and I have installed apache2.
Answer: Also if you used to open your root session with "su", then after upgrading to buster make sure to use "su -" instead, otherwise some commands won't be available (including a2enmod).
Comment: This fixed it, nice one!
Comment: @hendr1x In Debian 10 you can `su` to root but if you want the most powerful features you have to go up a level by running `su -` instead. I hope that helps.
Comment: This worked for me as well.  Can you explain why this fixes the issue? Thank you.
Comment: BTW This change is documented in https://wiki.debian.org/NewInBuster
Answer: It's a PATH variable bug.
Try this:
apt-get install mlocate
updatedb
locate a2enmod

the output is:
/usr/sbin/a2enmod

then
 grep PATH /etc/profile

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/sbin"
PATH="/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"
Export PATH

As you can see, /usr/sbin in not in root $PATH
So, edit the first line to
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/sbin"

And then
# . /etc/profile

And then you are OK.
Comment: Actually `/usr/sbin` is the last entry in the first line.
Comment: the step to _"edit the first line to"_ matches the first line. what is the difference?
Comment: also the _"As you can see, `/usr/sbin` is not in root $PATH"_ line doesn't make sense as it is in the $PATH, even in your example.
Comment: However, I will give you a vote up because running `. /etc/profile` did fix my problem
Comment: @Oxymoron thankyou . /etc/profile solve my prob. too
Answer: You need to have installed apache2

sudo apt install apache2

and sudo a2enmod file
you can also do a symbolic link:
ln -s /rute/ /rute2/
Comment: He can't create a symbolic link if he's not root. And the syntax would be `cd /etc/apache2/mods-enabled; ln -s ../mods-available/foo`
Comment: I've read that I shouldn't be doing it that way and that I should be using the a2* functions instead which is why I'm trying not to revert to doing that.
Answer: just put the PATH in root's ~/.bashrc file

a2enmod not working on debian (buster)


What is the projection in the database which is made by osm2pgsql?

Question: I want to search in the database made by osm2pgsql to render map tiles. I found this table which saved the lat and long for nodes. But these numbers don't seem to be in degree system or degree minutes seconds.
This is result of a query to find Tehran. (The true values for this city are lat="35.6884243" lon="51.4179604" but fetched values are lat="426776659" and lon="573163530")
<code>    id            lat          lon            tags
----------------------------------------------------------
 600129599     426776659     573163530       (Null)
</code>
Answer: The projection that is being used is EPSG:900913. This is a spherical mercator projection and its units are in metres.
Here is some code to do the conversion to decimal degrees.
You should also look here for a discussion about the common projections used.
Comment: http://stackoverflow.com/a/14814664/1809671 see here how to conver latlon to epsg:900913


Simple random walk bound hitting problem

Question: I am trying to understand the example 2.2. from this course
It is below. However, I am struggling to understand a key step in the reasoning. Specifically, I don't understand why this statement is true.
<code>If we let f(−B + 1) = α, then it follows that f(−B + r) = αr for all r ≤ A + B.
</code>
given this the solution is obvious algebra, but I just don't understand this at all. Any help would be appreciated.

Suppose that a gambler plays the following game. At each turn the
dealer throws an un-biased coin, and if the outcome is head the
gambler wins $1, while if it is head she loses \$1. If each coin toss
is independent, then the balance of the gambler has the distribution
of the simple random walk.
For two positive integers A and B, what is the probability that the
random walk reaches A before it reaches −B? Let τ be the first time at
which the random walk reaches either A or −B. Then Xτ = A or −B.
Define f(k) = P(Xτ = A | X0 = k), and note that our goal is to compute
f(0). The recursive formula f(k) =
0.5 f(k + 1) + 0.5 f (k −1) follows from considering the outcome of the first cointoss. We also have the boundary conditions f(A) = 1,
f(−B) = 0. If we let f(−B + 1) = α, then it follows that f( 1 −B + r)
= αr for all r ≤ A + B. Therefore, α = A+B , and it follows that
Answer: You can prove this by induction, and rearranging the formula to the form:
$$f(k) = 2f(k-1) - f(k-2)$$.
Supposing $f(-B + s) = \alpha s$ for $1 \leq s \leq r-1$ then
$$
\begin{align}
f(-B + r) & = 2f(-B + r-1) - f(-B +r - 2) \\
& = 2\alpha(r-1) - \alpha(r-2) \\
& = \alpha r.
\end{align}
$$


How to interpret a 1% significant relation when adding a control diminishes the significance level at 5%?

Question: In rough terms, I am given a t-statistic of 3.00 of the coefficient of variable $X$, $\beta_1$,  after regressing the independent variable $X$ on the dependent $Y$:
$$ Y = \alpha + \beta_1 X.$$
I have previously formed the null hypothesis that $X$ explains $Y$.
The normal interpretation would be:
"At the significance level of 1% I cannot reject the null hypothesis that $X$ explains $Y$".
However, adding some controls in different multiple regressions, say $Z_1$ in one specification and $Z_2$ in another, the t-statistic of $\beta_1$ drops to roughly 2.00, significant at 5% only, for only a couple of the tested control specifications.
What do I say then for interpreting the null hypothesis?
"I cannot reject the null hypothesis that $X$ explains $Y$ at the significant level %1 or 5%?
I understand that the null hypothesis considers one specification only, but what is the common approach?
Added clarification
In a study, consider I form the variables based on answers on questions of a specific group of people.
I then run
$$Y=\alpha+\beta_1X ~~~~~H_0: \beta_1=0$$
and
$$Y=\alpha+\beta_1X+\gamma Z ~~~~~H_0: \beta_1=0$$
Next day, as an added robustness check and assuming nothing should/has change/changed regarding the answers for forming the variables $Y$, $X$ and $Z$, I ask the same group of people the same questions, and I run the same regression on the new dataset.
Now, $\beta_1$ of the simple model is still significant at 1%, but it is significant at 5% only for the multiple regression model.
How do I treat this case?
In rough terms, the two dataset should give the same results.
But now, the new dataset is not exactly replicating the result of the former dataset, in terms of statistical significance.
How do I interpret this result in my study as a concluding sentence?
I say
"I cannot reject the null hypothesis that $X$ explains $Y$ at the significant level %1 or 5%?
Comment: What is the new question asking?  Is it wondering why two models can get two different p-values or why two datasets can get different p-values?  (And is there any mystery at all in either case?)
Comment: I'm a bit confused on your added part. Are you now asking why a new dataset isn't replicating the result?
Comment: @frage_man Clarified it, thanks, and sorry for my seemingly silly questions, but I have no experience on how people deal with such subtle differences.
Comment: No worries just trying to help.
Comment: @pidosaurus I added content to the bottom of my answer to attempt to answer your added part.
Answer: Initially you estimated:
$Y=\alpha+\beta_1X~~~~~H_0: \beta_1=0$
Then you accept or reject this at some confidence level (you were able to do that at 99%)
Now:
$Y=\alpha+\beta_1X+\gamma Z ~~~~~H_0: \beta_1=0$
Now you fail to reject at the 99% level and you reject at the 95% level.
If those extra Zs belong in the true model, then you ignore the first regression as it was biased. The regression that includes more relevant variables will result in less omitted variable bias and you can have more confidence in the results.
Before you likely had a higher chance of type 1 errors because of omitted variable bias, while the newer specification is a better model.
To answer your added part: If you add extra data points and your results change, then you should maybe estimate the model for all the available data using the fuller model and then determine whether you reject or accept at 1% or 5%.
So suppose you reran the full regression on the whole dataset and now you can only reject at the 5%. You would say that you reject the null that X is unrelated to Y, conditional on all the other variables, at the 95% level.
Comment: I get what you're saying, and largely agree.  It strikes me, though, that the null hypothesis *has* changed.  That is because the meaning of "$\beta_1$" has changed.  In the first model it's the coefficient of $X$, whereas in the second one it's the coefficient of *the residual of $X$ after regressing it on $Z$*.
Comment: @whuber Yes - $\beta_1$ is now the marginal effect of the variation in X orthogonal to Z that is affecting Y.
Comment: Thank you! I added a clarification in the question, where as an additional robustness check I form a new dataset that should on principle be the same with the first.


MVC 3 Checkbox failing at model return value

Question: I've got my Partial view :
@foreach (var checkbox in Model.TimeLimit)
{

    @Html.Label(checkbox.Name)
    @Html.Hidden(checkbox.Id)
    @Html.CheckBoxFor(c=>checkbox.IsChecked)

}

I've got a ParentViewModel which contains :
public IEnumerable<URLTimeLimitViewModel> TimeLimit { get; set; }

and  URlTimelimitViewmodel contains :
public string Name { get; set; }
public int Id { get; set; }
public bool IsChecked { get; set; }

When I call submit button, Timelimit in ParentViewModel returns null, why?
Answer: Better solution is EditorTemplate for URLTimeLimitViewModel
It will looks like this
@model URlTimelimitViewModel

    @Html.Label(Model.Name)
    @Html.Hidden(Model.Id)
    @Html.CheckBoxFor(c=>Model.IsChecked)

and write following code instead of "foreach"
@Html.EditorFor(x=> x.TimeLimit)

Answer: You need to change your loop - the model binder cannot figure out what to bind the posted values to:
@for (var i = 0; i < Model.TimeLimit.Count; i++)
{

    @Html.Label(Model.TimeLimit[i].Name)
    @Html.Hidden(Model.TimeLimit[i].Id)
    @Html.CheckBoxFor(c=>Model.TimeLimit[i].IsChecked)

}

You'll also need to declare your TimeLimit property as something other than an IEnumerable, I'd recommend a generic List (LINQs ToList method is very convenient!):
public List<URlTimelimitViewModel> TimeLimit { get; set; }


Killed an amp through audio interface in the effect loop?

Question: I was just using someone else's guitar and it stopped working. I'm not sure if something I did damaged it
It was some kind of tube amp.
The signal was: guitar -> amp -> amp effect send -> input on audio interface -> laptop -> daw (intending to add a software reverb) -> audio interface send -> amp effect return -> amp speaker.
The speaker on the amp remained plugged in at all times. The input I used on the audio interface was the hi z, which may not have been the right one actually. But I would assume that would only affect the audio interface, if anything, (maybe audio quality) and not the amp?
The amp was seemingly working at first. After I did the above I was getting no sound. I tested it a bit afterwards, with just straight guitar -> amp and was getting nothing. Sometimes I would get some sound and then it would go away again.
Did anything I did have the potential to have killed this amp? It looked like an obscure amp which hadn't seen much use in a while.
Comment: What you did sounds OK  in general terms, what was the specific amp, interface, and cables used though?
Comment: I'm not sure what the amp was. A make I'd never seen before. The cables were just standard mono quarter inch audio cables. The interface is a Steinberg ci2+. It's a sort of lower end 2 inputs, 2 outputs.
Answer: You usually hear it when killing an amp.  They don't tend to go quietly.  An audio interface can produce enough of a racket for killing one.  However, it's pretty hard to do while it is still turned down.
But killing an amp is not a come-and-go experience as usually the thing that goes up in smoke is the speaker or the power amplifier.  It is more likely that you are dealing with bad contacts, either because your cable is broken or because the connectors are broken or because the cable plugs are not matching the sockets in diameter or because you are using a stereo cable for an instrument cable connection or because the house fuse has blown for some reason or because your power cable or socket are iffy.
Or because there is a bad solder point in the amp itself and you've been the unlucky one to find it.  Oh, or because some pot has developed bad contacts in which case turning it back and forth should be a noisy experience with intermittent tone.  There is pot spray for that.


Probability of class in binary classification

Question: I have a binary classification task with classes 0 and 1 and
the classes are unbalanced (class 1: ~8%).
Data is in the range of ~10k samples and #features may vary but around 50-100.
I am only interested in the probability of an input to be in class 1 and
I will use the predicted probability as an actual probability in another
context later (see below).
Am am wondering how to best model this problem.
My current approach isto use a random forest and predict_proba in scikit-learn and use ROC-AUC as a scoring function. The accuracy is 0.92 as it does not
predict any class 1 with proba > 0.5.
After reading into the subject I came accross many suggestions and terms and I
try to put a little structure in all of this. Specifically:

I saw a couple of other scorers which were suggested, i.e. Cohens kapa,
Matthews correlation coefficient, PC-AUC and some more.
Should I look at all of those or is there a favorite for my problem?
I just came accross the probability calibration subject in scikit.
As I am interested in acual probability I think it's quite relevant.
Am I right to assume that an additional CalibratedClassifierCV should be
included in my model as it's based on Decision Trees?
(is that done automatically in R?)
After looking at some kaggle competitions xgboost seem very promising.
Is that alsorithm well suited for my problem or do you have other suggestions
regarding the algorithm (stick to  the RF)?
Answer: You are considering different classifiers, but in fact this is not a classification problem. You are not interested in classifying your data as zeros and ones, but in predicting probabilities that individual cases are zeros and ones. In this case the usual method of choice, that is designed especially for such problems, is logistic regression. Contrary to popular beliefs, logistic regression is not a classifier, but rather it predicts probabilities, so it does exactly what you want.
Comment: Yes, naturally I tried the logistic regression first but it seems to work much worse than my current RF approach. I only did the grid search for a decent C value for it and it did not help much, thats why I switched to the RFin the first place. When I do 'predict_proba' in sklearn for LR, do I get actual probability values without using calibration? Also in the case of the RF, I thought, as it is a voting based system, that the probability given in sklearn would be the fraction of votes for class 1 and therefore does not need further calibration. Is that correct?
Comment: @jens0r much worse in what sense..? How did you evaluate it?
Comment: The main evaluation criteria I'm interested in is the output of a second model where I plug the probabilities in. But it would be good to have a direct, intermediate evaluation for the problem the question. For that  I look at the AUC right now and I get better values for the RF model compared to the LR, roughly 0.77 to 0.79 (which is highly correlated to the evaluation measure in the subsequent model).


Deletion residuals

Question: Please can you help me to solve this problem. It should be calculated by vectorization. The question is:

The i-th deletion residual $e_{(-i)}$ is defined as $e_{(-i)} = y_i - X^\top B_{(-i)}$
  where $(X^\top)$ is the i-th row of the design matrix $X$ and $B_{(-i)}$ is a column vector of least square parameter estimates calculated without the i-th observation. Write some annotated R code to calculate the deletion residuals when the linear model $y_i = B_0 + B_1 X_i + B_2X_i^2 + E_i$
  is fitted to the data in the file quadratic.txt. By drawing suitable plot, comment on the distribution of these deletion residuals.

Edit
I wrote some r codes , but it just gives me residual. I want to calculate deletion residual for i-th cases.
> quad<-read.table("quadratic.txt", header=T)
> quad

tha data is like this
x=(0.75078002, 0.70959645 ,0.07482854,0.60755927 ,0.55037327 ,0.55037327,
0.35458257 ,0.21994714,0.66369585,0.12381099, 0.12381099,0.12381099,
0.77869635,0.63917962)

and
y=(18.715191,17.394049,-2.346149,5.528978,6.765831,6.324425,13.803874,
15.007047,4.034973,12.383765,14.823395)

> quad.lm<-lm(y~x+I(x^2), data=quad)
> resid(quad.lm)
        1          2          3          4          5          6          7
3.7933593  3.2646946 -4.9080046 -6.6589585 -4.3477835 -1.1856694  8.7046506
        8          9         10         11
1.7549092  0.6237708 -3.0781575  2.0371889

Comment: Since this is homework, can you explain what work you've done so far or what part of the problem is giving you trouble? We prefer not to just solve your homework for you, since then you don't learn very much...
Comment: you can get studentized deleted residuals via `rstudent` or `MASS::stdres` (`?rstudent` or `?MASS::stdres` to see the help on them) - if you want *raw* deleted residuals you'd need to compute them directly or from the standardized versions
Comment: **This question should not be responded to, this is an assigned question in a degree programme.**
Comment: @nina If there is a serious problem with this question, please flag it for deletion instead of replacing it by $11111111111...$ These changes can be rolledback, anyway.
Comment: I added the 'homework' tag, but you should feel free to explain how this arises in a way that doesn't come under the tag if you think it doesn't belong.
Comment: Actually I am not good in R because I have not used it before. So I just need some explanation about vectorization in order to start it. Thank you so much
Comment: I have merged your two accounts, nina. You'll still need to [register](http://stats.stackexchange.com/faq#login) to benefit from all Stack Exchange facilities (notifications, comments, update, etc.).
Answer: This kind of jackknifing problem can be programmed straightforwardly, but that usually results in a tangle of for loops. Some linear algebraic manipulations go a long way in drastically reducing code length (if your computer's memory can handle it).

Consider the original regression equation
$$
Y_i= \boldsymbol{X}_i'\boldsymbol{\beta} + \varepsilon_i;\;i=1, \ldots, n
$$
where $\boldsymbol{X}_i$ is a $k\times 1$ vector of regressors.You can write this more compactly as
$$
\boldsymbol{Y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon}
$$
Then the model that you want to estimate can be stacked up as
$$
\begin{bmatrix}
\boldsymbol{Y}^{(-1)} \\
\vdots \\
\boldsymbol{Y}^{(-n)} \\
\end{bmatrix} =
\begin{bmatrix}
\mathbf{X}^{(-1)} & & \\
 & \ddots & \\
 & & \mathbf{X}^{(-n)}
\end{bmatrix}
\underbrace{\begin{bmatrix}
\boldsymbol{\beta}^{(-1)} \\
\vdots \\
\boldsymbol{\beta}^{(-n)} \\
\end{bmatrix}}_{\equiv \boldsymbol{\beta}^-}
+
\begin{bmatrix}
\boldsymbol{\varepsilon}^{(-1)} \\
\vdots \\
\boldsymbol{\varepsilon}^{(-n)} \\
\end{bmatrix}
$$
where $\boldsymbol{Y}^{(-i)} = [Y_1, \ldots, Y_{i-1}, Y_{i+1}, \ldots, Y_n]'$ is the $n-1 \times 1 $ vector with the $i$-th row (of $\boldsymbol{Y}$) deleted; and $\mathbf{X}^{(-i)} = [\boldsymbol{X}_1, \ldots, \boldsymbol{X}_{i-1}, \boldsymbol{X}_{i+1}, \ldots, \boldsymbol{X}_n]'$ is the $(n-1)\times k$ matrix with the $i$-th row (of $\mathbf{X}$) deleted. Note that the design matrix in this regression equation is block diagonal. Also, this is a very large system.
Once $\widehat{\boldsymbol{\beta}}^-$ in the system above is estimated (by least squares), the residuals you want to estimate ($Y_i - \boldsymbol{X}_i'\widehat{\boldsymbol{\beta}}^{(-i)}$) can be had simply as $\boldsymbol{Y} - \mathrm{diag}(\mathbf{X}\mathrm{ivec}(\widehat{\boldsymbol{\beta}}^-))$, where the $\mathrm{diag}$ operator extracts the diagonal and $\mathrm{ivec}$ is the inverse of the  $\mathrm{vec}$ operator which stacks the columns of the matrix.

R code
Here is some R code to show how this can be done. The only trick here is constructing the stacked, row-deleted matrices, and then you are left with one (very) large least squares problem to solve.
iN <- 50                # number of observations
iK <- 4                 # number of regressors (including constant)

mX <- matrix(rnorm(iN*iK), nrow = iN, ncol = iK)  # design matrix
vBeta <- c(1, 2, 3, 4)                            # coefficients
vY <- mX%*%vBeta + matrix(rnorm(iN))              # dependent variable
mXStacked <- (diag(iN)%x%mX)[-seq(from = 1,
                                    to = iN*iN, by = iN+1), ]  # Stacked design matrix
vYStacked <- vec(vY%*%t(rep(1, iN)))[-seq(from = 1,
                                            to = iN*iN, by = iN+1), ]   # Stacked outcomes
vBetaMinus <- solve(t(mXStacked)%*% mXStacked,
                    t(mXStacked)%*%vYStacked)           # estimated coefficients
mBetaMinus <- matrix(vBetaMinus, nrow = iK, ncol = iN)
vResid <- vY - diag(mX%*%mBetaMinus)                  # required residuals

matplot(vResid, type = "l")           # plot the estimated residuals

Given dataset
The above R code can be easily adapted to the given dataset. Your outcome vector appears to be shorter than the regressor vector, so I have clipped off some of the latter.
vY <- as.matrix(c(18.715191,17.394049,-2.346149,5.528978,6.765831,6.324425,13.803874,
   15.007047,4.034973,12.383765,14.823395))
x <- c(0.75078002, 0.70959645 ,0.07482854,0.60755927 ,0.55037327 ,0.55037327,
   0.35458257 ,0.21994714,0.66369585,0.12381099, 0.12381099)
mX <- as.matrix(cbind(constant = 1, x = x, x2 = x^2))
iN <- nrow(mX)
iK <- ncol(mX)
mXStacked <- (diag(iN)%x%mX)[-seq(from = 1,
                                    to = iN*iN, by = iN+1), ]  # Stacked design matrix
vYStacked <- vec(vY%*%t(rep(1, iN)))[-seq(from = 1,
                                            to = iN*iN, by = iN+1), ]   # Stacked outcomes
vBetaMinus <- solve(t(mXStacked)%*% mXStacked,
                    t(mXStacked)%*%vYStacked)           # estimated coefficients
mBetaMinus <- matrix(vBetaMinus, nrow = iK, ncol = iN)
vResid <- vY - diag(mX%*%mBetaMinus)                  # required residuals

matplot(vResid, type = "l", xlab = "Obs. index", ylab = "Residuals")           # plot the estimated residuals

This produces the following picture.
Comment: May I asked why you offered to delete all this reply with your suggested edit, @nina?
Comment: Thank you so much it was great. Well done.but this command (vYStacked) does not work in my r, please can you tell me what is the problem? Thanks
Comment: @nina You will need to install and load the `Matrix` and the `matrixcalc` libraries using `install.packages("matrixcalc")` and `library(matrixcalc)` (similarly for `Matrix`) before you run this code.
Comment: Theank you @Peter. Now it is working. Thank you once again.@Nina
Comment: @chl Something bizarre just happened. Thanks for saving my answer. :)
Comment: Please can you explain ( e_(-i)=y_i-X_i^(transpode)*B_(-i) in R code how did you do that? and is that ok if all residuals were positive? Thank you . @Nina
Comment: @nina Firstly, there was a small error in the code that I have now fixed. Secondly, I realise that this answer is probably not easy for an R novice, but it does exactly what you would like, except that it does it in a compact way. The steps are exactly as in the mathematical description above the code.
Comment: thank you so much. by fixing this code the residual has changed. I will appreciate your job thank you.
Comment: I think these deletion residuals have approximate t distribution. I am not quite sure that is correct or no?
Comment: @nina Yes, jackknife residuals are supposed to be distributed $t$ with $n-k-1$ degrees of freedom, where $n$ is the (full) sample size and $k$ is the number of regressors. See for example, [page 297 in Kleinbaum, Kupper, Nizam & Muller (2007)](http://books.google.com/books?id=LnYGgaLQ3MMC&pg=PA297&lpg=PA297&dq=Jackknife+residuals+kleinbaum&source=bl&ots=AEGV_1ZQkM&sig=YEH3dcR16NB1AamOdqQhvCWzXLA&hl=en&sa=X&ei=1Iy_UJTbIYrZrQfagYHIAg&redir_esc=y#v=onepage&q=Jackknife%20residuals%20kleinbaum&f=false).
Comment: @nina If you had more observations, I would recommend that you created a kernel density plot and overlaid the PDF of a t-distribution with $n-k-1$ degrees of freedom to illustrate this point, but that might be tough with only 11 observations.

Deletion residuals


Adding another HardDisk to my server

Question: I have a ubuntu server and I am planning to add another harddisk to it.
Currently installed harddisk is iSCSI drive and I can add new SATA2 harddisk using NAS or SAS.
I lack knowledge of linux in windows when a new harddisk is installed it is like a different partition on linux how would it work a partition or add up to current harddisk space automatically? wherein I can continue storing files in /var/www/data
Thank You.
Comment: I think you mean that your currently installed hard disk is SCSI, not iSCSI if it is physically inside of the server.
Answer: On Unix systems, which includes Linux, partitions are physical space that you dedicate to a path. If you had only one partition, it would necessarily be allocated to '/', since this partition is mandatory.
If you wish to allocate physical space to /var/www/data using a specific partition, here is what you should do:

Create a new partition (on the same disk or another disk) and format it ;
Stop the services using the directory you wish to use as a mount point ;
Move the data in this directory to another place that has enough space ;
Find the identifier of your new partition ;
Mount the new partition on the path you wish to use ;
Put your data back in the directory.
Add the mount configuration in /etc/fstab to make sure your partition is mounted at startup.

Step 3 is important. If you don't do it, this is what will happen:

Your data is stored on the original partition (let's say it's /dev/sda1 for example) ;
You mount the new partition on top of the directory (let's imagine this is /dev/sdb2 for example) ;
You see the directory being empty, because /dev/sdb2 doesn't contain anything. The data are still stored in /dev/sda1, but you can't see them or remove them anymore, unless you unmount /dev/sdb2 first.

This is why it's important to clean the directory before you mount a partition on it.
You can find more informations on using partitions on Linux on this page.

Adding another HardDisk to my server


Simplification of $\frac{\left(\frac{6\ln^22x}{2x}\right)}{\left(\frac{3}{2\sqrt{x}}\right)}$

Question: My textbook goes from
$$\frac{\left( \frac{6\ln^22x}{2x} \right)}{\left(\frac{3}{2\sqrt{x}}\right)}$$
to:
$$\frac{6\ln^22x}{3\sqrt{x}}$$
I don't see how this is right. Could anyone explain?
Comment: Just what "$\ln^2 2x$" means is problematic.  Sometimes it means $\ln(\ln (2x))$ and sometimes it means $(\ln(2x))^2$.  I wouldn't use that expression at all.
Answer: $$\frac{\frac{6ln^22x}{2x}}{\frac{3}{2\sqrt{x}}} = \frac{6ln^22x}{2x} \cdot \frac{2\sqrt{x}}{3} = \frac{3ln^22x}{x} \cdot \frac{2\sqrt{x}}{3} = \frac{6\sqrt{x}ln^22x}{3x} = \frac{6\ln^22x}{3} \cdot \frac{\sqrt{x}}{x}$$
$$\frac{6\ln^22x}{3} \cdot \frac{\sqrt{x}}{x} = \frac{6\ln^22x}{3} \cdot x^{1/2}/x^1 =  \frac{6\ln^22x}{3} \cdot x^{-1/2} =  \frac{6\ln^22x}{3} \cdot \frac{1}{\sqrt{x}} = \frac{6\ln^22x}{3\sqrt{x}}$$
Answer: Well...
$\dfrac ab=a\times \dfrac 1b$
In your case $a=\frac{6\ln^22x}{2x}$ and $b=\frac{3}{2\sqrt{x}}$
Then... $\dfrac 1b=\dfrac{2\sqrt{x}}{3}$.
Can you go from here?
Answer: $$\frac{\frac{6\ln^2(2x)}{2x}}{\frac{3}{2\sqrt{x}}}=\frac{6\ln^2(2x)}{2x}\cdot\frac{2\sqrt{x}}{3}=\frac{6\ln^2(2x)}{x}\cdot\frac{\sqrt{x}}{3}=$$
$$\frac{2\ln^2(2x)}{x}\cdot\frac{\sqrt{x}}{1}=\frac{2\ln^2(2x)\sqrt{x}}{x}=\frac{2\ln^2(2x)}{\sqrt{x}}$$
Answer: $$
\frac{\left( \frac{6\ln^22x}{2x} \right)}{\left(\frac{3}{2\sqrt{x}}\right)} = \frac{6\ln^2 2x}{2x} \cdot \frac{2\sqrt x}{3} $$
Then use $\dfrac 6 3 = 2$ and $\dfrac {\sqrt x} x = \dfrac{\sqrt x}{\sqrt x \cdot\sqrt x} = \dfrac 1 {\sqrt x}$.
Answer: The textbook has rationalised the denominator for $\frac{3}{2\sqrt{x}}$ before simplifying further
i.e.
$$\frac{3}{2\sqrt{x}} = \frac{3\times \sqrt{x}}{2\sqrt{x}\times \sqrt{x}}= \frac{3\sqrt{x}}{2x}$$
Now Simplify:
$$ \frac{\left(\frac{6\ln^2 2x}{2x}\right)}{\left(\frac{3}{2\sqrt{x}}\right)}= \frac{\left(\frac{6\ln^2 2x}{2x}\right)}{\left(\frac{3\sqrt{x}}{2x}\right)} = \frac{6 \ln^2 2x}{3\sqrt{x}}$$
Answer: Just multiply denominator and numerator by $2x=2\sqrt x\cdot\sqrt x$  to obtain the given result.


Three Envelopes Problem

Question:
Possible Duplicate:
The Monty Hall problem

Three envelopes are presented in front of you by an interviewer. One contains a job offer, the other two contain rejection letters. You pick one of the envelopes. The interviewer then shows you the contents of one of the other envelopes, which is a rejection letter. The interviewer now gives you the opportunity to switch envelope choices. Should you switch?
Comment: @Marvis Sounds about right.
Comment: Sorry didn't see that, I will vote to close the question.
Comment: It depends .. does the interviewer know which envelope contains what? Does the interviewer always open an envelope?
Answer: yea u should
P of correct envelope the first time you picked is 1/3
That 1/3 has not changed so by switching you essentially make a switch with a P of 1/2.
Or something along those lines.

Three Envelopes Problem


gspread OAuth 2 json file does not contain client_email or private_key

Question: I am trying to us gspread using OAuth 2. I have seen several people using the code below, however the json file from google developers ("client_secret.....json") does not contain client_email or private_key. Any ideas where I went wrong?
The json file that I have has the following options (not actual these are copied from googles example):
<code>{
  "web": {
    "client_id": "asdfjasdljfasdkjf",
    "client_secret": "1912308409123890",
    "redirect_uris": ["www.example/oauth2callback"],
    "auth_uri": "accounts.google/o/oauth2/auth",
    "token_uri": "accounts.google/o/oauth2/token"
  }
}
</code>
gspread code I am trying to use:
<code>json_key = json.load(open('gspread-test.json'))
scope = ['https://spreadsheets.google.com/feeds']

credentials = SignedJwtAssertionCredentials(json_key['client_email'], json_key['private_key'], scope)
gc = gspread.authorize(credentials)
</code>
Answer: I had the same problem.
Instead of you create an Client ID Oauth 2.0, you must create an Service Account.


Discussions and email notifications on TFS tasks

Question: We're new to TFS and have some tasks set up in Team Explorer. It seems the History pane is the right/best place to add notes/discussions (although it's very different from other tools I'm used to like bugzilla, jira, redmine)?
But how can we set it up so interested parties get notified of new comments on a task, preferably by email? We're all making it up as we go along with no prior TFS experience but I'd hope my experience with other tools (the project hasn't used a proper issue tracker before) would help me figure things out but it all seems rather confusing.
Answer: If you install the TFS 2010 Power Tools (this is removed for 2012 as it's setup in the web interface according to this link although I've not played with this in 2012), under Team > Alerts Explorer you can add alerts for email notifications.
It's pretty simple, you can get alerts for when work items are modified, created under a certain path, assigned to you etc, they are basically configured in a similar way to the work item queries so it's quite easy to setup what you want.
They can be setup by any users, so you might want to let your users setup their own custom rules as they like, then you logon as a generic user (such as your admin user) to setup team specific queries, or else you might end up with users complaining about getting emails no longer relevent to them or need changing, when the user that set them up leaves/moves etc...  Else you'll be hunting round to find who setup the original rules.