How to access the page of a bridged modem with a different default IP from the router?

Question: I set up a modem as a range extender as in this thread. Initially I set it to WDS-only and could connect only via ethernet, so I wanted to login to the modem and change to Hybrid to also connect via wifi. With the default IP of the modem, I go instead to the router page.
Following these methods, I looked at the DHCP table on the router, which does not list the MAC address of the modem. I also tried pinging the broadcast address and then checking the result of arp -a. It returned 5 IP addresses, none with the modem's MAC address. One
address timed out, all others gave an error Safari can't open the page.
My machine is connected to the modem and has an IP address that does not show on the router's DHCP table, hence I believe the modem is doing Network Address Translation, as in How to access an adsl modem/router in bridged mode?.
I turned off the router, restarted the modem, got into the router's page, and changed the mode to "hybrid", so now I can connect via wifi. I also set the LAN IP of the modem to 192.168.2.1 to have a different range than the router and left
the same netmask (255.255.255.0). I confirmed that I can access this page when the modem is not bridged.
When bridged, the page at 192.168.2.1 does not respond from my machine plugged in to the modem with an ethernet cable or on the modem's wifi.
How can I find the IP address of the bridged modem or access the modem's page when bridged?
Comment: Your devices must be in the same network and have different IPs. `192.168.1.1` with a `24` bit netmask (addresses from `192.168.1.1` to `192.168.1.254`) and `192.168.2.1` with the same netmask (addresses from `192.168.2.1` to `192.168.2.254`) are disjoint networks.
Comment: You are right. I changed the IP to 192.168.1.2 and now I can see the router's page. I can now also see my machine on the ARP Cache Table, I don't know why it wasn't showing before. Can you write an answer?
Answer: You had an IP conflict: both routers had the same IP. After you changed your modem's IP, you placed it in another network: a netmask of 255.255.255.0 indicates that all devices in the network have the first 3 numbers (or 24 bits) of the address equal. The ARP tables won't detect it, since they have only one entry per IP. You must change the IP of your modem to something like 192.168.1.2.
A solution that can help in the future: choose a random number <n> between 2 and 255 and setup all your devices to use different addresses in the network 192.168.<n>.0/24 (addresses of the form 192.168.<n>.*). Embedded devices have a default IP address of 192.168.0.1 or 192.168.1.1, so you'll always be able to join them, by adding a second IP address to your computer's network interface (just to change the device's IP).


Pyspark throws Java gateway exception if I open a file before getting a SparkContext()

Question: I am using pyspark on Jupyter Notebook with Spark 2.1.1 cluster running on IP "spark://remote:port" (spark master IP)
I am able to create a SparkContext successfully.
But, I want to read spark_master_ip and spark.cores.max from a .properties file (instead of hard coding it). When I try to read my custom spark properties file in 'myspark_config.properties' file (which I parse and read successfully), but I get the following Java gateway exception, when I try to create SparkContext().
Here is my code:
import pyspark
from pprint import pprint
from pyspark import SparkConf
def getproperties():
    """Get Spark configuration properties in python dictionary"""
    global properties
    properties = dict()
    with open('myspark_config.properties') as f:
        for line in f:
                if not line.startswith('#') and not line.startswith('\n'):
                    tokens = line.split('=')
                    tokens[0] = tokens[0].strip()
                    tokens[1] = "=".join(tokens[1:])
                    properties[tokens[0]] = tokens[1].strip()

        f.close()
    pprint(properties)
    return(properties)
properties = getproperties()

conf = (SparkConf()
            .setMaster(properties["spark_master_url"])
            .setAppName("testApp")
            .set('spark.cores.max',properties["spark_app_cores"])
            .set('spark.executor.memory',properties["spark_app_memory"])
            .set('spark.dynamicAllocation.enabled','true')
            .set('spark.shuffle.service.enabled','true')

            )
# conf = (SparkConf()
#             .setMaster("spark://remote:port")
#             .setAppName("testApp")
#             .set('spark.cores.max',"2")
#             .set('spark.executor.memory',"2G")
#             .set('spark.dynamicAllocation.enabled','true')
#             .set('spark.shuffle.service.enabled','true')
#         )
sc = pyspark.SparkContext(conf=conf)

I do not get any exceptions and my code runs smoothly if I don't read from file and hard code the spark master in SparkConf() (currently it is commented). "JAVA_HOME","SPARK_HOME" "PYTHONPATH" are set successfully and I am not using Anaconda. I am using Jupyter notebook on python 2.7 on ubuntu and spark-2.1.1
{'spark_app_cores': '"2"',
 'spark_app_memory': '"2G"',
 'spark_master_url': '"spark://remote:port"'}

Exception                                 Traceback (most recent call last)
<ipython-input-1-c893eaf079f2> in <module>()
     36 #             .set('spark.shuffle.service.enabled','true')
     37 #         )
---> 38 sc = pyspark.SparkContext(conf=conf)

/usr/local/spark/python/pyspark/context.py in __init__(self, master, appName, sparkHome, pyFiles, environment, batchSize, serializer, conf, gateway, jsc, profiler_cls)
    113         """
    114         self._callsite = first_spark_call() or CallSite(None, None, None)
--> 115         SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
    116         try:
    117             self._do_init(master, appName, sparkHome, pyFiles, environment, batchSize, serializer,

/usr/local/spark/python/pyspark/context.py in _ensure_initialized(cls, instance, gateway, conf)
    281         with SparkContext._lock:
    282             if not SparkContext._gateway:
--> 283                 SparkContext._gateway = gateway or launch_gateway(conf)
    284                 SparkContext._jvm = SparkContext._gateway.jvm
    285

/usr/local/spark/python/pyspark/java_gateway.py in launch_gateway(conf)
     93                 callback_socket.close()
     94         if gateway_port is None:
---> 95             raise Exception("Java gateway process exited before sending the driver its port number")
     96
     97         # In Windows, ensure the Java child processes do not linger after Python has exited.

Exception: Java gateway process exited before sending the driver its port number

I have looked through various links but found no solution:
pyspark java gateway error stackoverflow
Github issue on java Gateway
Answer:
But, I want to read spark_master_ip and spark.cores.max from a .properties file (instead of hard coding it).

That is awesome idea, but you're ignoring the fact, that this is what $SPARK_HOME/conf/spark-defaults.conf is for. Just put required properties there.

but I get the following Java gateway exception,

This looks wrong:
"=".join(tokens[1:])

Why would use = in properties?
Otherwise it should have no effect. Also Python provides property parser https://docs.python.org/3/library/configparser.html
Comment: I need my own properties file to set properties specific to my application (like api_key to call a REST API and table_names of databases). Example api_key = "shajkhsakj=sdi23" this is one entry in properties file. I am not allowed to use third party libraries like configparser or jproperties.                    But my main issue is why do I get an error if I try to open a file in python with open() command, before initializing SparkContext().


DFS Replication failure

Question: We have an active directory server as global catalog and DC, this one has operations master running.
We have a second ADDC Global catalog and exchange server.
I started getting the message below on the main ADDC

Log Name:      DFS Replication Source:        DFSR Date:
  2/1/2014 12:43:22 AM Event ID:      4012 Task Category: None Level:
  Error Keywords:      Classic User:          N/A Computer:
   Description: The DFS Replication
  service stopped replication on the folder with the following local
  path: C:\Windows\SYSVOL\domain. This server has been disconnected from
  other partners for 126 days, which is longer than the time allowed by
  the MaxOfflineTimeInDays parameter (60). DFS Replication considers the
  data in this folder to be stale, and this server will not replicate
  the folder until this error is corrected.    To resume replication of
  this folder, use the DFS Management snap-in to remove this server from
  the replication group, and then add it back to the group. This causes
  the server to perform an initial synchronization task, which replaces
  the stale data with fresh data from other members of the replication
  group.

In the exchange server I get this error

The DFS Replication service stopped replication on volume C:. This
  occurs when a DFSR JET database is not shut down cleanly and Auto
  Recovery is disabled. To resolve this issue, back up the files in the
  affected replicated folders, and then use the ResumeReplication WMI
  method to resume replication.    Additional Information:  Volume: C:
  GUID: F18E11A2-980C-11E2-93E8-806E6F6E6963    Recovery Steps
  1. Back up the files in all replicated folders on the volume. Failure to do so may result in data loss due to unexpected conflict resolution
  during the recovery of the replicated folders.
  2. To resume the replication for this volume, use the WMI method ResumeReplication of the DfsrVolumeConfig class. For example, from an
  elevated command prompt, type the following command:  wmic
  /namespace:\root\microsoftdfs path dfsrVolumeConfig where
  volumeGuid="F18E11A2-980C-11E2-93E8-806E6F6E6963" call
  ResumeReplication

I tried following this guy here, but I can't seem to find this in ADSI Edit:
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<the server name>,OU=Domain Controllers,DC=<domain>

Comment: When you double-click on the domain controller in ADSIEdite, you don't see "CN=DFSR-LocalSettings"?  What folders do you see under the domain controller?
Comment: Is your AD replication still happening? Why DFSR stopped replication for such a long time in the first place? I recommend to ensure AD replication working before fixing any DFSR issue. Next step is to followed instruction in event log, instead of going for a different KB.
Answer: You have three different problems here.

You have one server acting as a Domain Controller and an Exchange server at the same time. This is bad. This isn't the cause of your problem, but you should be aware that this is not how you should have this configured. Exchange and AD should be installed on separate servers that are dedicated to each role.
You had an unexpected shutdown on the Exchange server. This could be a power outage, or any other unclean shutdown. In Server 2008 R2 SP1 and Windows Server 2012, DFS-R will not resume replication automatically on unclean shutdowns. The error message that you've posted tells you the exact command to bring replication of the volume back online. Microsoft has released guidance for when and how to enable auto-recovery  so that you don't need to mess with the wmic command each time you have an unclean shutdown. Domain Controllers are one instance where auto-recovery should always be enabled. Do this.
Your Exchange/DC and FSMO-holding DC haven't replicated sysvol in longer than the MaxOfflineTimeInDays threshold - this is called Content Freshness Protection in DFS terminology. You will have to determine which server's SYSVOL share has the most up to date information and use it as the authoritative source for replication and follow this document to resume replication.

In short - restart replication on the Exchange/DC server with the wmic command in the error message. Then, configure auto-recovery to prevent this in the future, then force an authoritative replication of sysvol to get the replication group running again.
Comment: In terms of number 2, Unexpected shutdowns like power-loss aren't the only situation where the DFS database has a "dirty" shutdown. If the server is under heavy load, and the DFS Service doesn't manage to commit all of it's database to disk within 30 seconds on a regular shutdown, that is still marked as a "dirty" shutdown and will need auto-recovery or reinitialization. This is yet another reason to not have DC and Exchange on the same server too...
Answer: When Server 2008 R2 has an unexpected DFS-R disconnect, it does not automatically start replicating again - as per the 2nd message in your event log.
You can restart replication using the following command:
wmic /namespace:\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="F18E11A2-980C-11E2-93E8-806E6F6E6963" call ResumeReplication

(as printed in the bottom of your 2nd event log message). Also heed attention to the bit where it tells you to back it up in case of an unexpected conflict.
The other option is to do as your first event log message says - open the DFS-R management MMC and remove and re-add the server from the replication group.
Comment: Unfortunately, the Domain System Volume is a special case replication group and cannot be managed via the DFS snap-in.
Comment: Oh, right. Well you can always try re-starting I guess...
Comment: It's more of a PITA than that - you need to force an authoritative sync of sysvol using adsiedit. It's **fun**!

DFS Replication failure


Switch statement not compiling

Question: Hello I've been sitting here for quite some time trying to figure out why this isn't working but I haven't had luck succeeding!
#include <stdio.h>

main(void)
{
    int mark;

    printf("Please enter the mark\n");
    scanf("%d", &mark);

    switch(mark){
        case (mark <40):
            printf("That's a fail");
            break;
        case (mark >=40 && <60):
            printf("That's a pass");
            break;
        case (mark >=60 && <70):
            printf("That's a merit");
            break;
        case (mark >=70):
            printf("That's a distinction");
            break;
    }
}

Comment: For future reference maybe include the compiler error too. You can also google the error and perhaps find the answer too.
Comment: use if/else buddy, instead switch
Comment: When you say doesn't work - what exactly do you mean?
Comment: Welp, I've got to be honest :D
Comment: It's producing an error in the compiler
Answer: For comparisons where the exact value is not known, use if-else blocks. You are trying to use a switch statement like an if-else block, and it does not work properly. A proper switch would look like
switch(mark) {
case 40: //if mark is 40, no more, no less
  ...
case 60: //same
 ...
}

Because you don't want to write out 40 lines that all lead to the same statement, just use
if(mark < 40) { ... }
else if(mark < 60) { ... }

Answer: From K&R:
"The switch statement is a mult-way decision that tests whether an expression matches one of a number of constant integer values, and branches accordingly."
That is, you can't do what you're trying to do. Each case needs a constant statement, such a simple int or char value.
For what you want to do you'll need if/else. Also, see jaap's answer on correct syntax for your boolean statement.
Answer: It's working fine, the compiler is preventing you from compiling something that's not valid C.
The case labels must be actual integer constants, not expressions that do further calculations.
You should use if/else if to check your various conditions.

Switch statement not compiling


Run next cloud build trigger after two other finished successfully

Question: Let's say that currently my CI/CD pipeline runs these two triggers after merging. (0 and 1)

And in order to run Trigger production deployment I need both to finish successfully. Is there a way to run them (Cypress E2E and Python E2E) separately, and wait for each to "enable" the "1 of 2" and "2 of 2" required conditions to then execute the deployment?
Answer: Synchronizing Cloud Builds to trigger a 3rd one can be done but it's complex.
However, it exists an option on Cloud Build to run steps in parallel. According to the documentation, you can do that
steps:
  - name: ...
    id: cypress
    args: ...
  - name: ...
    id: python
    args: ...
    waitFor: ['-']

  - name: ...
    id: deploy
    args: ...
    waitFor: ['cypress','python']

If you need more CPU to run all the steps in parallel, you can configure the machine type
Comment: Fair enough, this seems to be what I'm looking for. Thanks for the suggestion.


Is this an example of where I shouldn't scale before doing PCA / PLS?

Question: I'm working with NMR spectra (it's a common chemical test).
There are various peaks of the signal across a range of ppm values.  I'm trying to relate the NMR spectra of various samples to a measured property of the samples.
PCA and PLS seem like valuable tools.  I'll just talk about PCA for this example, but my question also applies to PLS.
The data that I send to PCA has this structure: each column is a ppm value, each row is a sample, and the value is the sample's signal at that ppm.
For this data that I've made up, it seems like there should be one dominant principal component that describes the size of the peak.  This is indeed the case if I don't scale the columns.  But if I do scale the columns (which is always what I've done with PCA), the Scree plot makes no sense to me.  Why are so many principal components needed to describe the variance?
Is this a problem where I shouldn't scale the columns?  What about when I apply to real data, where there are multiple peaks of widely varying heights? Is there a "best practice" for doing PCA and PLS on spectral data? (I don't have a background in biostatistics or chemometrics).
Spectra

The matrix for PCA
<code>                  3         3.01         3.02        3.03        3.04 ...
sample1 0.002798792 0.0031477185 0.0090656919 0.008150071 0.005397167 ...
sample2 0.009681631 0.0006409005 0.0003475848 0.009898928 0.003882373 ...
sample3 0.001149334 0.0020842757 0.0057974388 0.008381035 0.004349227 ...
sample4 0.009143676 0.0003381718 0.0078882525 0.009483161 0.007021517 ...
</code>

R code
<code>library(dplyr)
library(reshape2)
library(ggplot2)
library(factoextra)

# make up some spectral data
D <- tibble(ppm = seq(3, 4, 0.01),
            sample1 = dnorm(ppm, 3.2, 0.03),
            sample2 = 0.9 * sample1,
            sample3 = 0.8 * sample1,
            sample4 = 0.5 * sample1)
Noise <- runif(nrow(D) * (ncol(D) -1), 0, 0.01)
D[, -1] <- D[, -1] + Noise  # add a little noise so things aren't perfectly correlated

# plot the spectra
Dm <- melt(D, id.var = "ppm", value.name = "signal", variable.name = "sample")  # long format
Dm %>%
  ggplot() +
  aes(ppm, signal, color = sample) +
  geom_line() +
  geom_point()

# Prepare data for PCA
Dc <- dcast(Dm, sample ~ ppm, value.var = "signal")      # each ppm's signal is a variable
Dmat <- as.matrix(select(Dc, -c(sample))) # drop the sample column; only want to work on the signals
rownames(Dmat) <- Dc$sample

# PCA with scaling
Ps <- prcomp(Dmat, scale. = TRUE, center = TRUE)
fviz_screeplot(Ps) + ggtitle("Scree plot with scaling")

# PCA without scaling
P <- prcomp(Dmat, scale. = FALSE, center = TRUE)
fviz_screeplot(P) + ggtitle("Scree plot without scaling")

</code>
Comment: The main problem issue here is that this is a $n \leq p$ PCA. By just looking at the one it is clear that there is only a single PC. We need more PCs if we scale the data because we artificially inflate their variability when centring and the peak at approximately 3.2 no longer dominates the variability of the sample.
Comment: Scaling is recommended when the variables have a significantly different ranges. For instance, fist variable ranging between 1-10 and second variable ranges between 0-1e6. This is usually not the case for NMR data. Also scaling makes the regression coefficients compariable among variables. But I feel like scaling adds another layer of modeling by adding stddev assumption in addition to mean assumption. Most of the time, applying scaling to spectral data slightly decreased my models' performance.
Comment: If you are after a predictive model, I would recommend testing scaling see what works better for your case.
Comment: Arthur, please check my edit. You can revert it if my suggested correction is actually wrong. And welcome to cross validated.
Answer: As @theGD already pointed out in the comment, scaling is often not needed for spectroscopic data as the features already have a common intensity axis.
Here's my guess what's happening when you scale:
You have spectra with very nice zero baselines. In other words, all those features outside your analyte signal are constant mean + some noise.
If you scale such a feature, this noise will be inflated/amplified until it has unit variance. Just like the scaled feature(s) of your analyte signal.
After scaling, you'll then have lots of noise features of similar variance (i.e. importance) to your analyte signal features. The only thing that can now make your analyte signal stick out of this "forest of noise" is if your analyte signal covers a sufficiently large number of features.
But then, if you have many noise features and not that many samples, the chance to have accidentally correlated noise features increases. This will lead to noise-only PCs that nevertheless have substantial eigenvalues (if you have many noise only features and few analyte features, your analyte signal can actually end up in a higher PC!).

Unfortunately, there is nothing that forces noise-only features to be accidentally correlated with noise-only features: we may have accidental correlation with analyte features. In that case the model becomes noisy (unstable).
To be sure, this correlation is present also in the unscaled data, but there only a small amount of noise enters the PC/latent variable with the analyte whereas after scaling the noise is about as large as the signal from a single analyte feature.
Fortunately, we do have the huge advantage in spectroscopy that we can check the loadings: if they look noisy (instead of look like nice difference or maybe derivative spectra), they usually did pick up noise. I.e. we can see when/where our models get unstable.

Together, these the amplification of noise and the sheer number of noise-only features in relation to few analyte (signal) features may explain your observation.

I'd expect PLS to be somewhat less disturbed by such scaling than PCA as PLS looks for variance that is correlated also with the analyte signal, but in general I'd not expect any improvement, neither, for the situation you describe.

If you want to scale your spectra, make sure you first exclude uninformative spectral regions. (But maybe you should anyways do this as this dimensionality reduction will help the modeling with small sample sizes)


single time series per participant and using random effects in mixed effect modelling

Question: My mixed effect model design involves assessing 42 different time series via a polynomial function. that is, fixed effects of time and time^2. I've incorporated a random slope of the individual time series.
My hypothesis is effectively does each time series fit a quadratic curve when I allow the coefficients to vary.
My question for here is, is there any issue with incorporating a random effect in this manner. Most documentation assumes multiple time series per random effect and I can't find much on if using it as I have is an issue?
Answer: This seems like a reasonable approach. The grouping factor for the random effects is the individual time series ID, and you have 42 of these. Each time series consists of repeated measures over time, and you are going to estimate fixed effects for an intercept, a linear term and a quadratic term. By specifying random intercepts, you allow each time series to have it's own intercept, and by fitting random slopes for the linear term and quadratic term you allow the shape and location of the curve to be different for each time series.
If the variances of the random effects are meaningfully above zero then this would be consistent with your hypothesis. Obviously they don't all have to be above zero and if one or more of them is close to zero then you may be justified in removing it/them.
When you say "multiple time series per random effect" this implies that the time series' are nested within some other factor - such as subjects, which doesn't seem to be the case here, but if it was, then you would just specify the nested structure appropriately
Comment: Another great response, Thanks!


100th derivative of $ g(x) = (x^3 -4x + 7)^{30} $. Where is the pattern?

Question: I was reviewing come Calc 1 problems, and came across one that asks us to explain what the 100th derivative of
$$ g(x) = (x^3 -4x + 7)^{30} $$
would be.
I computed the first three derivatives:
$$ g'(x) = 30(x^3-4x+7)^{29}(3x^2-4) $$
$$ g''(x) = 30*29(x^3-4x+7)^{28}(3x^2-4)+(6x)(30(x^3-4x+7)^{29})$$
Very quickly we can see we will have to continue doing chain rules and product rules every time we take a derivative. I also took the 3rd derivative, but the termas quickly expand and I do not see a pattern such that I can figure out what is supposed to happen at 100th derivative (especially because the chain rule and product rules really add terms). I calculated the answer online, and it is supposed to be zero, but this is not evident to me at all. Your help is much appreciated.
Comment: What is the degree of $g$? How much is the degree reduced when taking the derivative? Write $g(x)=x^n+a_1x^{n-1}+ \cdots $ and then differentiate $n+1$-times - or more.
Comment: The leading term of the polynomial is $(x^3 - 4x + 7)^{30} \sim (x^3)^{30} = x^{90}$. Hence the $100$-th derivative is zero.
Comment: Thank you for your input guys, feel free to add it as a solution so I may marked this question as solved.
Answer: The highest power of $x$ in $g$ is $(x^3)^{30}=x^{90}$ which vanishes on the $91$st derivative. So,
$$g^{(100)}(x)=0$$
Answer: If $g(x)=x^n+a_{n-1}x^{n-1}+\cdots +a_0$, then differentiating it $(n+1)$-times gives zero. Now we have
$$
g(x)=(x^3-4x+7)^{30}=x^{90} - 120x^{88} + 210x^{87} + \cdots + 22539340290692258087863249,
$$
to be precise (we do not need the exact coefficients, of course).


multiply websites and different websites on the same iis server

Question: I've got this kind of situation: I've got windows 2003 server with dns server on same machine. It is binded to adress for ex. siteA.com Now i want to add to this machine website which name will be siteB.com. I created a new website on IIS6 server with name siteB.com but I dont know how to set up a dns server. My primary DNS administrator created me an alias for my server and he describe it to me like this: 'site siteB.com is an aliase for siteA.com' and then he said that I have to configure my DNS server by my own. I've tried to add new alias in my existing DNS zone (for siteA.com) but it's binding FQDN name like this: siteB.SiteA.com which is wrong as I supose. Can anybody explain me how can I bind this 2 webiste to my server?
Answer: It sounds like you have 2 domain names siteA.com and siteB.com that you wish to point to the same location.  I gather this from your statement that one is an alias for another.
Doing this should be pretty simple.  In IIS6 you go to the properties of the website by right clicking on the website and selecting properties.  On the Web Site tab, next to the IP Address, select advanced.  Add an entry in the "Multiple identities for this Web site" with the same IP and port for siteA.com but use siteB.com.
That should get you 2 domain names pointed at the same location.
Comment: I set up the IIS server with this configuration before, but on my server I have got dns role which is set up to use siteB.com domain. So I think I should set up somehow the dns server to serve another domain (siteB.com) but I don't know how..
Answer: Using your litteral example of SiteB.com needing to be set up in DNS on your DNS server, then you need to create a new zone file called siteB.com.
Under siteB.com create two records.  One with either a blank name or "@", record type A, and either the same or another IP address of siteA.com.  The other record will be "www", record type CNAME, with a value of siteB.com, so www.siteA.com is an alias to siteA.com which is where the IP is defined.
In IIS as mentioned earlier, if both siteA.com and siteB.com will go to the same web site, then you should use the same IP addresses in DNS and make sure the IIS site has both DNS hostnames listed if the site requires host headers.  If the site doesn't care about host headers, and only needs the IP address, then you're done.
If siteB.com is a new, different site from siteA.com, then use a different, available IP address, add it to your server, then create the new IIS site with that IP address.


(How) can I Create products by the square foot?

Question: I have a product that I need to sell by the square foot, where the customer would enter the length and width, and the number of sq ft would be the unit amount on the product line. That unit number should then only be changed by updating the length/width fields (not by direct customer entry). Is there a commerce module or combination of mods for that. I've tried commerce custom product, and tinkered with rules, but can't make it work.
Maybe I need a custom module?
Any advice appreciated.
Answer: There isn't a complete solution, but there's a start and an open issue to make it do exactly what you need. I wrote a Physical Fields module that defines a dimensions field you can use to collect the length and width via a line item field exposed to the Add to Cart form. The issue is the dimensions field right now only supports collecting all three dimensions: length, width, and height. It needs to be updated to support using a subset of the dimensions, and I've just updated the appropriate issue that you can follow here.
Once you have that field on your line item type and have exposed it to the Add to Cart form, customers will be able to specify their dimensions when they add the product to the cart. With this data in fields on the line item, you can then make calculations based on the value of the field using Rules (or a custom Rules action). You can create a new data value by multiplying the length times the width (should be possible in Rules) and then set the product price accordingly. In such a case, I'd probably make the price of the product be the "sq. ft." price and then just multiply that times the square footage I calculate.


State not correctly refreshed due to asynchronous calls

Question: I'm new in React (and in Javascript in general...) and I struggle to manage filters correctly. I think the problem comes from the fact that the calls are asynchronous but I cannot figured it out.
I have an application where you have matches and pronostics. You can filter matchs with a search box, as well as with a calendar:

If I click on a day on the calendar, the list of match must be filtered, but not the calendar itself, otherwise I will not be able to click on any other day
If I use the searchbox, both the matchs and the calendar must be updated

Both are OK in the fact that the matches are correctly filtered according to the day and/or search box, but it is not refreshed correctly, and the values of the pronostics and the score are completely messy after filtering
I have these 3 functions:
handleSelectDay = (selectedDate) => {
  this.setState({selectedDate},this.handleFilters)
}

handleSearch = (search) => {
  this.setState({search},this.handleFilters)
}

handleFilters = () => {
  console.log(this.state.selectedDate)
  const filteredMatchsWithoutDate = this.state.matchs
  .filter(match => {
    const valArray = this.state.search.toLowerCase().split(' ');
    let matchBool = true;
    for(let i = 0; i < valArray.length; i++) {
      if (
        match.team_a.name.toLowerCase().includes(valArray[i])
        || match.team_b.name.toLowerCase().includes(valArray[i])
        || match.location.toLowerCase().includes(valArray[i])
        || match.type.toLowerCase().includes(valArray[i])
      ){
        matchBool = true;
      }else{
        matchBool = false;
      }
    }
    return matchBool
  });
  const filteredMatchs = filteredMatchsWithoutDate
  .filter(match => {
    if(this.state.selectedDate != null)
      return isSameDay(parseISO(match.schedule),this.state.selectedDate)
    return true
  })
}

and my render function:
render() {
    return (

        <Navigation />
        <Container>
          <Filters
            onSearch={this.handleSearch}
            onCheckbox={this.handleCheckbox}
          />
          <Row>
            <Col sm={0} md={0} lg={4} xl={4} className="col-xxl-3">
              <Calendar
                onSelectDay={this.handleSelectDay}
                matchs={this.state.filteredMatchsWithoutDate}
              />
            </Col>
            <Col sm={12} md={12} lg={8} xl={8} className="col-xxl-9">
              <Row>
                { this.state.filteredMatchs.map(match => <Match match={match} />) }
              </Row>
            </Col>
          </Row>
          </Container>

    );
  }

I think that the problem comes from this.state.filteredMatchs.map(match => <Match match={match} />) which is not correctly refreshed due to the state filter, but I cannot figure out how I could deal with it and the fact that handleSearch and handleCheckbox are already calling the handleFilters method ahead.
Thanks in advance for your help :)
Comment: Looks like you forgot to update the state after filtering the data.
Answer: You need to update your state variable filteredMatchs in your handleFilters method. Your logic is there to filter the data but you are not updating the state. Adding the setState line at the end of your handleFilters method will make it work.
this.setState({filteredMatchs})

Also, as @jazz and @sowam suggested you should also add key attribute to your  component.
Answer: It looks like this could be because you are not using key props.
<Match key={someUniqueValue} match={match} />

Without key props react cannot track changes in arrays mapped to elements correctly.
https://reactjs.org/docs/lists-and-keys.html
Edit/ PS. You should never use an array index to map these change and it defeats the purpose of key props - https://robinpokorny.medium.com/index-as-a-key-is-an-anti-pattern-e0349aece318
Answer: The fastest fix would be adding a key which you can take from map function like:
this.state.filteredMatchs.map((match, index)=> <Match key={index} match={match} />)

So react knows that it has unique key and knows which one to use.
In real big application you shouldn't use index as a key. Key should be 100% unique so the best is to set it as ID or uuid etc.
Comment: It seems to be working if I use the ID of the match as key: `this.state.filteredMatchs.map((match)=> )`. Thanks for your help!


java stop watch program - using System.nanoTime() and TimerTask together

Question: I am writing a mini program in Java to use as a stop watch but I am not sure if I am using the right methods in terms of efficiency and accuracy.
From what I have read on stackoverflow it it appears that System.nanoTime() is the best method to use when measuring time elapsed.  Is that right?  To what extent is it accurate as in to the nearest nanosecond, microsecond, millisecond etc.
Also, while my stop watch is running, I would like it to display the current time elapsed every second.  To do this I plan to use a TimerTask and schedule it to report the time (converted to seconds) every second.
Is this the best way?  Will this have any effect on the accuracy?
Lastly with my current design will this use up much of a computer's resources e.g. processing time.
PS  Sorry can't share much code right now cause I've just started designing it.  I just did not want to waste time on a design that would be inefficient and make an inaccurate timer.
Answer: Yes, you can use java.util.Timer and TimerTask that runs periodically and updates your view every second. However I do not think you have to deal with nono seconds while you actually need resolution of seconds only. Use regular System.currentTimeMillis().
Comment: Since nanoseconds is at least as precise as currentTimeMillis, why not use nanoseconds? (*This method provides nanosecond precision, but not necessarily nanosecond resolution (that is, how frequently the value changes) - no guarantees are made except that the resolution is at least as good as that of currentTimeMillis().*)
Comment: That's great thanks.  Just to clarify, if I am calculating the time based on when I first call the System.nanoTime()  How can I use the System.currentTimeMilis()?  Do I also need to make a separate startTime in milliseconds with the System.currentTime()? Thanks again
Comment: If I understand your task correctly you just have to create one timer task that knows to update your view. Then create one, global `Timer` and schedule this task to run every second. The task should retrieve current time using for example `System.currentTimeMillis()` and update the view.


Why is my terminal alias failing

Question: I'm reading this:
http://www.maclife.com/article/columns/terminal_101_creating_aliases_commands
I wrote in my bash profile:
alias workspace = 'cd Documents/workspace'
however I get the following:
<code>-bash: alias: workspace: not found
-bash: alias: =: not found
-bash: alias: cd Documents/workspace: not found
</code>
when I source the file. What's going on?
I searched on SO and found: .bash_profile aliases: command not found but I'm not using double quotes
Answer: Please consider removing the spaces around the = sign.
For reference:  http://www.gnu.org/software/bash/manual/bashref.html#Bash-Builtins
This is not on mac, but still bash:
<code>$ cat with-spaces.sh
alias workspace = 'cd Documents/workspace'
$ . with-spaces.sh
./with-spaces.sh: line 1: alias: workspace: not found
./with-spaces.sh: line 1: alias: =: not found
./with-spaces.sh: line 1: alias: cd Documents/workspace: not found
$ # checking result...
$ alias workspace
bash: alias: workspace: not found

$ cat without-spaces.sh
alias workspace='cd Documents/workspace'
$ . without-spaces.sh
$ # checking result...
$ alias workspace
alias workspace='cd Documents/workspace'
</code>

Why is my terminal alias failing


choosing one pair randomly from repeated pairs

Question: I have two vectors, in my case all states in the US, and another vector, which indicates the states that it borders. States in vector 1 repeat for each state it borders. For instance, there would be 3 observations for California: California-Nevada; California-Oregon; California-Arizona.
For each state, I would like to randomly choose one of these dyads and create a new dataset. Let's say the original data set was 300 x 2. The new dataset would be 50x2. Here is an example dataset.
state <- sample(state.abb,size=300,replace=T)
neighbor <- sample(state.abb,size=300,replace=T)
data <- data.frame(state,neighbor)

thanks!
Answer: There are (of course) any number of ways to do this.  The approach I apply here uses data.table and does what is desired in a single line of code that is readable if you are familiar with data.table.
library(data.table)
result <- data.table(data)[,.SD[sample(1:.N,1),],by="state",]

In short, we convert data into a data.table.  Using the nomenclature data.table(data)[i,j]...
We are processing data by subsets of each unique value for state (the by argument), as if we used state== for each unique value of state in position of the argument i (row selection).
The j argument (results) provides the output from each subset.  Here we want a random row from each subset of data.  Each individual subset is internally referred to as .SD in the position of j.  So, we can just randomly sample a row from that subset ([sample(1:.N,1),]) in the i (row selection arguement) where .N is internally defined for a data.table (in this case .SD, because this is a bracketed selection from .SD) as the number of rows in that data.table.
Comment: P.S. If you aren't already familiar with data.table you should be made aware that result is a data.table and won't behave quite like a data.frame, but you can just do `new.result <- data.frame(result)` and push the data.table back into a data.frame if you so desire.


Verifying ownership of a domain name purchased through Sqaurespace with AWS ACM

Question: I'm relatively new to the AWS ecosystem and have a question that I'm hoping I can properly articulate enough to get an answer.
I have an app that whose frontend is deployed through AWS Amplify and has a load balancer set up through EC2 for requests to the backend. When developing locally, all requests from the frontend to the backend have been made over HTTP and presented no problems. However, when deployed up into the AWS environment, the URL from the Amplify deploy is serving over HTTPS while the load balancer DNS uses HTTP, and so when the frontend makes a request to the server I get a Mixed Content error. I know that to get around this I need to essentially set up a domain for the load balancer and use ACM to generate an SSL certificate for it so that it can serve back responses over HTTPS.
However, to further complicate the situation, a domain has already been purchased by others on my team through Squarespace, which I'm required to use (my-app.co as an example here). I want to have "api.my-app.co" route to my load balancer so that I can make requests from the frontend to "api.my-app.co/items" to grab whatever I need from my database.
I haven't been able to find a definitive answer to my specific problem, but through various tutorials I've pieced together the following:

In my squarespace account, I've created a CNAME record for "api.my-app.co" and pointed it to my load balancer's DNS name

In ACM, I've generated a certificate for the "api.my-app.co" domain that is currently awaiting ownership validation.

I've also set up a Hosted Zone through Route 53 as described in this video: https://www.youtube.com/watch?v=ookzXuMr8eY&ab_channel=EndreSynnes, though I'm unsure of how to supply the "Value/Route traffic to" values to Squarespace.

I know that once I get the certificate validated I need to provide it to an HTTPS listener on my load balancer, but I haven't yet been able to get to this point.

My main question is: How do I confirm ownership of the Squarespace domain using the DNS method? Unfortunately nothing I've read in ACM's documentation itself has been of much help. Thank you.
Answer: I had the same problem, and I had meaningless error messages in squarespace (they didn't give any clue what was wrong), after many tests I managed to find the right combination.
NOTE: I think this was my biggest discovery, DO NOT click on the aws copy button, when you paste it into squarespace it will tell you that the domain is invalid and a couple of other things. Copy data manually (ctr+c)
requested domain name in aws: xxxyyy.com
Squarespace Host: put the aws CNAME Name without the domain at the end and without any period at the end.
Data Squarespace: put the aws CNAME Value without the period at the end
It should be validated within next 5 minutes
Comment: This worked for me - could not find it anywhere else.
Answer:
I've also set up a Hosted Zone through Route 53 as described in this video: https://www.youtube.com/watch?v=ookzXuMr8eY&ab_channel=EndreSynnes, though I'm unsure of how to supply the "Value/Route traffic to" values to Squarespace.

You shouldn't have done that at all. You are using Squarespace as your DNS server. Setting up another DNS server on AWS isn't going to solve anything. Forget about Route53.
All you need to do is go into the ACM domain validation, and find the two DNS records it is asking you to create in order to verify you own the domain. Then go over to your Squarespace DNS settings, and add those two DNS records. After that, the ACM service should soon detect that you have created the records and then it will issue you the SSL certificate.
Comment: If the host value that AWS Amplify provides is say `_3639ac514e785e898d2646601fa951d5.example.com.`, you need to enter only the first part `_3639ac514e785e898d2646601fa951d5` (i.e. omit the `.example.com.` part) into Squarespace. Ref https://support.squarespace.com/hc/en-us/articles/360002101888#toc-troubleshooting
Comment: Just to confirm I'm understanding correctly, the redacted CNAME records found here on ACM:

https://imgur.com/FiHHWkw

need to be added alongside the CNAME record I already created on squarespace here?:

https://imgur.com/PtBRb0o
Comment: Yes, that is correct. It looks like there's just one CNAME you need to create. For some reason I was thinking there were 2 needed to validate. The `Name` value in ACM will be the `Host` value in Squarespace, and the `Value` value in ACM will be the `Data` value in Squarespace.
Comment: So strange - Squarespace is spitting a very nondescript error back at me when I try to add that record: Custom record not saved - We were unable to save this record. Try again later. If this problem continues, contact Customer Support. Not sure what's up.


Calling an ASP function from an external ASP file

Question: For some reason, I can include an ASP file with the following code:

But, I have some functions in that file, and I can't call them into my main file. Can someone help? I'm calling them like this:
<% my_func() %>

..and all I get is an error message, with a link to another webpage. Thanks for your help in advance.
Comment: It's a function of the editor.  You can either select it and click Ctrl-K, or select it and click the Binary Format button just above the editor (the button with the 1's and 0's)
Comment: Thank you for updating my syntax. How did you do that?
Answer: In VBScript, you can call functions in a handful of ways;

returnval = functionYaddaYaddaYadda()
functionFoo()
Call functionBar()

You're essentially trying to #2 with shorthand ASP tags. Use:
<%= my_func() %>

The "=" is the shorthand way of doing:
response.write(my_func())

P.S. Check out the MSDN VBScript Reference. There's really not much to VBScript outside of this reference manual. Maybe more elegant tricks but the meat of VBScript can be found on that site.
Comment: Dude, you rock. Thanks man, I'm a major noob with ASP. I appreciate it.
Comment: No problem, dude. :) I should also note that *ASP* type information isn't found on the MSDN reference link I provided. Just VBScript. You can think of ASP as like a mini layer of frosting on top of the cake that is VBScript. For ASP type info, w3schools (http://www.w3schools.com/asp/asp_intro.asp) would probably be a good place to start if you need ASP-centric info. They too have VBscript programming info should you need it.


Cannot use daisyui in tailwind, and npm install not installing

Question: I was trying to install DaisyUI, by npm i daisyui, it gives me an error:
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: email@example.com
npm WARN Found: firstname.lastname@example.com
npm WARN node_modules/react
npm WARN   peer react@">=16.8.0" from @firstname.lastname@example.com
npm WARN   node_modules/@emotion/react
npm WARN     peer @emotion/react@"^11.0.0-rc.0" from @firstname.lastname@example.com
npm WARN     node_modules/@emotion/styled
npm WARN     4 more (@mui/material, @mui/styled-engine, @mui/system, the root project)
npm WARN   35 more (@emotion/styled, @mui/base, @mui/icons-material, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react@"^16.0.0" from email@example.com
npm WARN node_modules/react-reconciler
npm WARN   react-reconciler@"^0.23.0" from @email@example.com
npm WARN   node_modules/@react-pdf/renderer
npm WARN
npm WARN Conflicting peer dependency: email@example.com
npm WARN node_modules/react
npm WARN   peer react@"^16.0.0" from email@example.com
npm WARN   node_modules/react-reconciler
npm WARN     react-reconciler@"^0.23.0" from @firstname.lastname@example.com
npm WARN     node_modules/@react-pdf/renderer
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: firstname.lastname@example.com
npm ERR! Found: firstname.lastname@example.com
npm ERR! node_modules/react
npm ERR!   peer react@">=16.8.0" from @email@example.com
npm ERR!   node_modules/@emotion/react
npm ERR!     peer @emotion/react@"^11.0.0-rc.0" from @email@example.com
npm ERR!     node_modules/@emotion/styled
npm ERR!       peerOptional @emotion/styled@"^11.3.0" from @email@example.com
npm ERR!       node_modules/@mui/material
npm ERR!         peer @mui/material@"^5.0.0" from @email@example.com
npm ERR!         node_modules/@mui/icons-material
npm ERR!         1 more (the root project)
npm ERR!       3 more (@mui/styled-engine, @mui/system, the root project)
npm ERR!     peerOptional @emotion/react@"^11.5.0" from @email@example.com
npm ERR!     node_modules/@mui/material
npm ERR!       peer @mui/material@"^5.0.0" from @firstname.lastname@example.com
npm ERR!       node_modules/@mui/icons-material
npm ERR!         @mui/icons-material@"^5.8.0" from the root project
npm ERR!       1 more (the root project)
npm ERR!     3 more (@mui/styled-engine, @mui/system, the root project)
npm ERR!   peer react@">=16.8.0" from @email@example.com
npm ERR!   node_modules/@emotion/styled
npm ERR!     peerOptional @emotion/styled@"^11.3.0" from @firstname.lastname@example.com
npm ERR!     node_modules/@mui/material
npm ERR!       peer @mui/material@"^5.0.0" from @firstname.lastname@example.com
npm ERR!       node_modules/@mui/icons-material
npm ERR!         @mui/icons-material@"^5.8.0" from the root project
npm ERR!       1 more (the root project)
npm ERR!     peerOptional @emotion/styled@"^11.3.0" from @email@example.com
npm ERR!     node_modules/@mui/styled-engine
npm ERR!       @mui/styled-engine@"^5.8.0" from @email@example.com
npm ERR!       node_modules/@mui/system
npm ERR!         @mui/system@"^5.8.1" from @email@example.com
npm ERR!         node_modules/@mui/material
npm ERR!     2 more (@mui/system, the root project)
npm ERR!   34 more (@mui/base, @mui/icons-material, @mui/material, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^16.0.0" from firstname.lastname@example.com
npm ERR! node_modules/react-tradingview-widget
npm ERR!   react-tradingview-widget@"^1.3.2" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: email@example.com
npm ERR! node_modules/react
npm ERR!   peer react@"^16.0.0" from firstname.lastname@example.com
npm ERR!   node_modules/react-tradingview-widget
npm ERR!     react-tradingview-widget@"^1.3.2" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/alex/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/alex/.npm/_logs/2022-09-26T02_27_08_062Z-debug-0.log

Then I tried "npm i daisyui --force" I can see the dependency is in my package.json file.
However, when I tried to use it in the  "tailwind.config.js" by inset  plugins: [require("daisyui")], into the file, however vscode shows
Could not find a declaration file for module 'daisyui'. '/Users/alex/Maqro-Portal-Frontend/node_modules/daisyui/src/index.js' implicitly has an 'any' type.
Try npm i --save-dev @types/daisyui if it exists or add a new declaration (.d.ts) file containing declare module 'daisyui';ts(7016)
then i do npm i --save-dev @types/daisyui, the terminal gives me en error
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: firstname.lastname@example.com
npm WARN Found: firstname.lastname@example.com
npm WARN node_modules/react
npm WARN   peer react@">=16.8.0" from @email@example.com
npm WARN   node_modules/@emotion/react
npm WARN     peer @emotion/react@"^11.0.0-rc.0" from @firstname.lastname@example.com
npm WARN     node_modules/@emotion/styled
npm WARN     4 more (@mui/material, @mui/styled-engine, @mui/system, the root project)
npm WARN   35 more (@emotion/styled, @mui/base, @mui/icons-material, ...)
npm WARN
npm WARN Could not resolve dependency:
npm WARN peer react@"^16.0.0" from firstname.lastname@example.com
npm WARN node_modules/react-reconciler
npm WARN   react-reconciler@"^0.23.0" from @firstname.lastname@example.com
npm WARN   node_modules/@react-pdf/renderer
npm WARN
npm WARN Conflicting peer dependency: firstname.lastname@example.com
npm WARN node_modules/react
npm WARN   peer react@"^16.0.0" from firstname.lastname@example.com
npm WARN   node_modules/react-reconciler
npm WARN     react-reconciler@"^0.23.0" from @firstname.lastname@example.com
npm WARN     node_modules/@react-pdf/renderer
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: firstname.lastname@example.com
npm ERR! Found: firstname.lastname@example.com
npm ERR! node_modules/react
npm ERR!   peer react@">=16.8.0" from @firstname.lastname@example.com
npm ERR!   node_modules/@emotion/react
npm ERR!     peer @emotion/react@"^11.0.0-rc.0" from @firstname.lastname@example.com
npm ERR!     node_modules/@emotion/styled
npm ERR!       peerOptional @emotion/styled@"^11.3.0" from @email@example.com
npm ERR!       node_modules/@mui/material
npm ERR!         peer @mui/material@"^5.0.0" from @firstname.lastname@example.com
npm ERR!         node_modules/@mui/icons-material
npm ERR!         1 more (the root project)
npm ERR!       3 more (@mui/styled-engine, @mui/system, the root project)
npm ERR!     peerOptional @emotion/react@"^11.5.0" from @email@example.com
npm ERR!     node_modules/@mui/material
npm ERR!       peer @mui/material@"^5.0.0" from @email@example.com
npm ERR!       node_modules/@mui/icons-material
npm ERR!         @mui/icons-material@"^5.8.0" from the root project
npm ERR!       1 more (the root project)
npm ERR!     3 more (@mui/styled-engine, @mui/system, the root project)
npm ERR!   peer react@">=16.8.0" from @firstname.lastname@example.com
npm ERR!   node_modules/@emotion/styled
npm ERR!     peerOptional @emotion/styled@"^11.3.0" from @email@example.com
npm ERR!     node_modules/@mui/material
npm ERR!       peer @mui/material@"^5.0.0" from @firstname.lastname@example.com
npm ERR!       node_modules/@mui/icons-material
npm ERR!         @mui/icons-material@"^5.8.0" from the root project
npm ERR!       1 more (the root project)
npm ERR!     peerOptional @emotion/styled@"^11.3.0" from @firstname.lastname@example.com
npm ERR!     node_modules/@mui/styled-engine
npm ERR!       @mui/styled-engine@"^5.8.0" from @firstname.lastname@example.com
npm ERR!       node_modules/@mui/system
npm ERR!         @mui/system@"^5.8.1" from @firstname.lastname@example.com
npm ERR!         node_modules/@mui/material
npm ERR!     2 more (@mui/system, the root project)
npm ERR!   34 more (@mui/base, @mui/icons-material, @mui/material, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer react@"^16.0.0" from firstname.lastname@example.com
npm ERR! node_modules/react-tradingview-widget
npm ERR!   react-tradingview-widget@"^1.3.2" from the root project
npm ERR!
npm ERR! Conflicting peer dependency: firstname.lastname@example.com
npm ERR! node_modules/react
npm ERR!   peer react@"^16.0.0" from email@example.com
npm ERR!   node_modules/react-tradingview-widget
npm ERR!     react-tradingview-widget@"^1.3.2" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /Users/alex/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/alex/.npm/_logs/2022-09-26T02_33_39_239Z-debug-0.log

How can I fix this problem? thanks.
Answer: You can try deleting your node_modules folder and your package-lock.json, then do
npm install daisyui
Answer: I have 2 suggestions

delete node_modules and package-lock.json and re-install the DaisyUI or react-daisyui
You can try to install DaisyUi with version as  npm i email@example.com


Comparing four classifiers

Question: I have trained and tested four different classifiers, and I would now like to compare them. The classifiers have accuracies 95, 90, 81, 75.
I know that there is no unbiased estimator of the variance of K-fold cross-validation accuracy, which is going to make things difficult.
In this question/answer, it has been assumed that the overlap in training sets does not matter. How large can I expect the resulting bias to be?
Is there some other (simple) way to do this?
Answer:
I have trained and tested four different classifiers, and I would now like to compare them. The classifiers have accuracies 95, 90, 81, 75.

In order to say more, we at the very least need to know the number of test cases (see example below).

 I know that there is no unbiased estimator of the variance of K-fold cross-validation accuracy, which is going to make things difficult.

$k$-fold cross validation cannot measure the variance connected to the fact that your particular data set may not be totally representative for the problem at hand. In other words, the assumption that re-sampling is a good approximation to getting actual new samples breaks down. But another part of the variance of the models can be measured by iterated/repeated $k$-fold cross validation: the variance in the predictions caused by exchanging a few training cases (model instability).
The still unknown residual model variance matters if the question at hand is "Which classifier performs best for problems of this type of application?" I.e. in scenarios where the winning classifier is then trained on a completely new sample of the same type.
If the scenario is that from this particular data set a model should be trained and you then need an estimate how well that model predicts unknown cases later on, this never-to-be-known-by-resampling-validation variance is not important - the instability measurements should yield an estimate that in practice tells you all you need to know.
All this is about variance uncertainty on the model.
Note that the finite number of cases also means that the testing is subject to variance, which is typically assumed to follow a binomial distribution for performance measures of the fraction-of-tested-cases type, i.e. overall accuracy, sensitivity, specificity, predicitive values, etc.
You can account for this variance by constructing binomial confidence intervals. E.g. assuming you observe 17 correct out of 21 tests, the 95% confidence interval for the accuracy is roughly 60 - 93 % (point estimate: 81 %). Here, the actual number of independent (different) cases enters: iterating the cross validation does not change this number!
Practical considerations:

If the confidence intervals calculated just on the finite number of test cases overlap, or a paired test based on the number of test cases (look e.g. into McNemar's test) is not significant, you cannot claim a difference.
In that case, you don't need to worry about the not-yet-accounted for variance in the models as it can never lower the total variance uncertainty.
The other way round: do sample size (or power) calculations, e.g. with the observed accuracies. If these calculations yield sample sizes that are way above what you can get, report that data-driven model comparison is impossible and instead choose the final classifier by your knowledge about the type of classifier, the application and the data at hand.
Use iterated/repeated $k$-fold cross validation to check the stability of the predictions wrt. slight changes in the training set. If you detect instability, in practice you should probably reconsider the way you set up your classifier rather than trying to include this variance estimate into the comparison.
Decide whether you'd need to include the undetected variance due to resampling not being the same as getting new samples into your comparison (scenarios above). If not, you're fine. If yes, report that your results do not include this possible additional variance (you anyways cannot do more).
If you are training all classifiers in question, you can and should set up a paired comparison: for each particular split, calculate and test all classifiers.
This allows far more powerful statistical tests for the comparison.
Answer: Unhandled exception's suggestion amounts to a computationally expensive approach to optimizing n-fold cv. While I understand it intuitively, given large n, it could be prohibitive even across a lot of CPUs. But then, I'm with Emmanuel Derman in pointing to the always funky nature of data and, based on that, questioning the virtues of "optimization" a priori and from a philosophical POV. In other words, what are we optimizing -- really?
My suggestion comes from experience with the vicissitudes of "random" assignment to the k-folds. So, yes, use k-fold cv, but the classifiers need to be evaluated across the same fixed k-folds for the comparisons to be valid. It seems obvious but I've made this point with some pretty senior guys and it was a big "aha" since the normative assumption is that the random samples, with or without overlap, are equivalent. They are equivalent only in the theoretical asymptote, not in finite data samples.
And to the point made about overlap of data in the k-folds, I've known people to make categorical assertions that there can be absolutely no overlap in the k-folds for cv to be meaningful. Personally, I don't understand this requirement and would be interested in understanding the motivations for it, either theoretically or practically. Non-overlapping k-folds can be challenging when data is limited. It may not be a motivated statement and more like a rule of thumb being rigidly prescribed. But then, lots of the orthodoxy are unnecessarily rigid in their prescriptions.
@Erik, thanks for the excellent link.
Comment: " the classifiers need to be evaluated across the same fixed k-folds for the comparisons to be valid" => this allows for a paired comparison. Of course, comparisons on different test sets can be done, but they'll have less power.
Comment: @cbeleites  "Comparisons on different test sets can be done." This is true but for finite data samples you can't answer what the real differences are in classifier performance vs random noise or drift in the data unless those test sets are fixed. I came to this insight somewhat reluctantly after seeing answers change as a function of the underlying, randomly drawn test datasets.
Comment: While a paired design has more power, it is no magic wand. You're still using only the particular training and the particular test sets (= variance). If your answer changes depending on the underlying random draw I'd say you a) need to evaluate way more draws in order to get a good representation of this type of variance and b) you should be very cautious in your conclusions: this variation is just one type of variance that people often forget to measure.
Comment: How are you defining a "paired design?" How would "evaluat(ing) way more draws" improve the answer over fixed k-folds, particularly with small *n*? Is it possible to be specific about the "type(s) of variance that people often forget to measure" or that aren't controlled for by k-fold cv?
Comment: paired design: is what you get training all classifiers in question with exactly the same training data and testing them with exactly the same test data - i.e. your "fixed" k-fold.
Comment: Evaluating more draws: this covers variance due to model instability wrt. to slight changes in the training data. It is particularly helpful for small $n$ as too small training data leads to such instability. Ideally, you'd want to see no differences between the different draws' predictions for the same test case. If you do detect instability here, chances are that the data set does not represent the problem well because it is too small.
Answer: I would suggest to compare your classifiers via Leave-One-Out rather than X-Fold Cross Validation. Here you test the entire data, which make comparision more easier...
Comment: Leave-one-out is K-fold with $K=N$... And what do you mean by testing the entire data?
Comment: I know it mmh ;-)
Comment: Sorry I just don't understand how this answers the question. :)
Comment: According to http://ai.stanford.edu/~ronnyk/accEst.pdf Leave-One-Out is almost unbiased and hence should answer the OP's answer " How large can I expect the resulting bias to be?"...
Comment: @Unhandledexception This does not answer the question and is also bad advice. Leave one out might be unbiased but that is off-set by the high variance. See for example this excellent answer: http://stats.stackexchange.com/questions/61783/model-variance-and-bias-in-cross-validation.
Comment: Leave-one-out can actually have a high bias in particular cases. And the variance is typically the same as a single run of k-fold. BUT: you loose the possibility to at least the part of the variance of $k$-fold CV that can be measured. => still bad advise.

Comparing four classifiers


Probability of an event with probability 0 happening at least once in infinite trials

Question: This question here is confusing me a lot.
To summarize, let's say you have $\text{i.i.d. }X_i  \sim U(0, 1), i = 1,2,\ldots, n.$
The question shows that $Y_i = \max(X_1, \ldots, X_i) \rightarrow 1$ in probability, and intuitively (for what it's worth) I can see that, but the math approached this way doesn't seem to agree with me.
$$\lim_{i\rightarrow \infty} P(|Y_i - 1| \geq\varepsilon ) =0 \text{ ?}$$
Alternate Solution
\begin{align}
& P(|Y_i - 1| \geq\varepsilon ) = P\left(\bigcup_i(X_i < 1)\right) \\[8pt]
= {} & \prod_i [P(X_i \leq 1) - P(X_i = 1)] \\[8pt]
= {} & \prod_i[F_{X_i}(1) - 0] = \prod_i 1 = 1 \neq 0.
\end{align}
This shows that $Y_n$ does not converge in probability, which does not align with the answer to the question posted above.
Possible flaws in my solution
However, I know while $P(X_i = 1) = 0$. If I can say that its limit is 0 but is in reality infinitesimal i.e. $P(X_i = 1) = \varepsilon$. I can show that $\lim_{n\rightarrow\infty}(1-\varepsilon)^n = 0,$ $\varepsilon >0$. However, that seems wrong to me.
What's confusing me is that as defined, $X_i:  i = 1, 2, \ldots, n$ is a countably infinite number of "trials" whereas there is an uncountably infinite number of values a continuous RV could take. If I take a countably infinite number of "trials" when there are uncountably infinite possible outcomes. I know that it isn't almost sure that any single outcome will occur. It also seems to me that it won't occur in probability since the set of uncountably infinite $\gg$ countably infinite.
Thus I believe my solution is correct and $Y_n$ does not converge in p. However, the posted question's answer does not agree.
Would appreciate it if you could point out the flaw in my reasoning!
Thanks!
Comment: you need to formulate the question clearly, separate it from the statements. what's the probability of drawing exactly 0 from standard normal if you keep trying every second to the end of world? It's zero. is this what you're looking for?
Comment: Where you wrote "infinite trials", the correct term is "infinitely many trials." In standard mathematical terminology, "infinite trials" would mean trials each one of which is infinite. I don't know what an "infinite trial" would be, but if you had two of those, you'd have infinite trials, but you wouldn't have infinitely many trials.
Comment: @Aksakal if it is 0, then wouldn't the above show that it $Y_n$ does not converge in p?
Comment: @MichaelHardy thanks for your edits and correction!
Comment: $P(X_i)\ne\varepsilon$, it's exactly zero: $P(X_i)=0$
Comment: @Aksakal thank you for your time! I've edited the question to address what I mean. In short, I understand, your point that $P(X_i) = 0$ exactly and use it in my solution. However, this means that something else is wrong in my logic, or the earlier question's answer is wrong in that $Y_n \rightarrow 1$ in p
Comment: The earlier answer seems Ok on quick glance. I don't understand the first equality in your "alternative solution" section.
Answer: $$
P(|Y_i - 1| \geq\varepsilon ) = P\left(\bigcup_i(X_i < 1)\right) \text{ ??}
$$
Here you need
$$
P(|Y_i - 1| \geq\varepsilon ) = P\left(\bigcap_i(X_i \le 1 - \varepsilon)\right).
$$
To say that the maximum observation is no more than $1-\varepsilon$ is to say that all of the observations are no more than $1-\varepsilon,$ so you need an intersection here, not a union. And how $1-\varepsilon$ got replaced by $1$ is not explained.
By independence, you have
$$
P\left( \bigcap_i \big[ X_i \le 1 - \varepsilon \big] \right) = \prod_i (1-\varepsilon) = 0.
$$
Comment: I see, thank you so much! I appreciate the time you took. I apologize for the careless mistake!


Enviando string da view para a controller via Ajax. Codeigniter, javascript e php

Question: estou tentando enviar uma informação que está no bloco de código javascript para a minha controller, porem não estou sabendo como capturar a informação na minha controller.
Segue código Javascript da view:

Arquivo de rota:
$route['id_notification'] = 'post_notification/insert_id_post_notification';

Minha controller:

<?php

class Post_notification extends CI_Controller
{
    public function insert_id_post_notification()
    {
        $teste = $_POST['id'];
        var_dump($teste);
    }
}

?>

Nome da minha controller: post_notification.
Quero capturar o id com valor 1515 na minha controller. Valor de teste
Desde já muito obrigado pessoal.
Comment: Seu fetch está errado: https://googleweblight.com/i?u=https://github.com/github/fetch/issues/635&hl=pt-BR dá uma olhada
Comment: Olá amigo! realmente estava incorreto,
alterei e ficou da seguinte forma
Comment: fetch('http://localhost/app.realizaconstrutora.com.br/public/id_notification', {
    method: "POST",
    body: 1515
   })
   .then(function(response) {
    return console.log(response);
   });
Comment: Agora preciso saber como capturar esse valor 1515 na minha controller :/
Comment: assim: `fetch('localhost/app.realizaconstrutora.com.br/public/id_notification', { method: "POST", body: 'source=1515', headers: {
        "Content-Type": "application/x-www-form-urlencoded"} }) .then(function(response) { return console.log(response); });` e depois no seu código se busca pela `source` assim vai funcionar no seu backend...
Answer: Passa a url inteira sem a rota dessa  forma:

Então no seu Controller faça o seguinte
<?php

class Post_notification extends CI_Controller
{

    public function __construct() {
      parent::__construct();
    }
    public function insert_id_post_notification()
    {

        $teste = $this->input->post('id');
        var_dump($teste);
    }
}

?>

Comment: Coloquei e o Ajax está enviando com status Ok, porém, jogando a url do Ajax direto eu consigo acessar o método **insert_id_post_notification**, porém através do ajax não está acessando. Acabei de postar como está meu código atualizado :/
Comment: @FelippeSousa Poderia me Mostrar a resposta do servidor
Comment: Consegui fazer com o axios js, deu certo aqui. Muito obrigado!
Answer: Meu código js na view agora está da seguinte forma:

var formData = new FormData();
   formData.append('id', '1515');
            fetch('http://localhost/page.com.br/public/post_notification/insert_id_post_notification', {
                method: "POST",
                body: formData
            })
            .then(function(response) {
                return console.log(response);
            });

O POST com o Ajax está com status OK!

Minha Controller está da seguinte forma:

<?php
    class Post_notification extends CI_Controller
    {
        public function __construct() {
            parent::__construct();
        }
        public function insert_id_post_notification()
        {
            $teste = $this->input->post('id');
            var_dump($teste);
        }
    }
?>

Não está chegando no var_dump, então nem está chegando no metodo insert_id_post_notification da controller. Não sei o porque, pois jogando o diretorio que estou usando no ajax direto na url eu consigo acessar :/
Answer: Código com Axios js funcionou!

Valeu galera!
Answer: Utilize o jQuery para falicitar
// CDN : https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.js
$.post(
       'link',//url de destino
       {id: '151515'},//dados e como vao chegar no controller, para adicionar mais dados basta dar , e continuar o array
       function(data){
    //Aqui voce lida com o retorno do controller
});

Answer: Man manda manda da seguinte forma:
var formData = new FormData();
formData.append('id', 14);
formData.append('name', 'teste');
formData.append('valor', '1.4');

fetch('http://localhost/page.com.br/public/id_notification', {
                method: "POST",
                body: formData
            })
            .then(function(response) {
                return console.log(response);
            });

Comment: Olá, coloquei da seguinte forma:
`var formData = new FormData();
formData.append('id', '1515');
fetch('http://localhost/page.com.br/public/id_notification', {
method: "POST",
body: formData
            })
            .then(function(response) {
                return console.log(response);
            });`

O POST com ajax está enviando com status ok.
A rota esta correta! Jogando ela direto na url eu consigo acessar o método na controller, porém o ajax não está entrando no metodo **insert_id_post_notification** da controller, mesmo com o diretório correto


error: expected '}' before 'else'?

Question: I get these errors when I compile my code.
experiment.cpp: In function 'void convert(char*)':
experiment.cpp:47:5: error: expected '}' before 'else'
experiment.cpp:47:5: error: expected '}' before 'else'
experiment.cpp: At global scope:
experiment.cpp:53:3: error: expected declaration before '}' token

But I do not understand why. I checked all the bracing many times.
void convert(char *infix)
{
    for(int i = 0 ; i < m ; i++)
    {
       if(isalpha(infix[i]))
          printf("%c",infix[i]);
       else if(ispunct(infix[i]))
       {
           if(!isempty())
               push(infix[i]);
           else
           {
               if(check(infix[i]));
                  push(infix[i]);
               else
               {
                  printf("%c",pop());
                  push(infix[i]);
               }
           }
       }
    }
}

Comment: note: ugly code. learn to use `&&` and `||`. eliminate duplication.
Comment: Remove ; in the line 46
Answer: if(check(infix[i]));: remove the ; at the end.
Answer: The error is in:
               if(check(infix[i]));

Remove the trailing ;
               if(check(infix[i]))

Answer: Your code is like writing:
if(check(infix[i])) { }  //if(something); == if(something) { }
push(infix[i]);
else                     //See why there's a problem?

It's a good practice to have { even if you have only one line under the statement.
Doing that might help you to reveal possible problems, like this ; you have there.
Another note, use && for clarity instead of nested ifs..

error: expected ‘}’ before ‘else’?


Loading text/ng-template in AngularJS directives correctly

Question: I have a directive that I want to clear the content of a DIV and replace it with a template either in my current view or somewhere else in my app.
So say I have my template like so...

and in my directive I have the following:
link: function (scope, element) {
                element.bind('click', function () {

                    // clear out old template
                    angular.element(element).empty();
                    angular.element(element).html(document.getElementById('1.html'));

                });
            }

I seem to be able to load the template but I see the following instead of my content
[object HTMLScriptElement]

I wondering if I need to compile again or run a digest or if I am just going about this totally wrong. I also don't like using  document.getElementById in my directive code, for some reason it feels wrong. Can anyone provide me with an answer to why I only see [object HTMLScriptElement] after clicking my directive and wether using document.getElementById in my directive is acceptable or if there is a better command to load the content...
Here is a fiddle of the whole app... or a bin! https://jsbin.com/yizupa/edit?html,output
Comment: Do you only want the replacement to happen when you click? Normally you would a directive like this: https://jsbin.com/qudategoda/1/edit?html,output
Comment: Hi, Pretty much yeah, I am more concerned with loading the template into the div that anything else. My original directive will cover large areas of the interface, I can't use the template directive or the templateUrl in my current implementation. I just want to load the new content into the container...
Answer: In the meantime I have just came to the conclusion that my setup / implementation is wrong and that I should consider another way to implement such a feature.
Answer: here, i have create one running demo.. for route using **text/ng-template**..

index.html
-------------

<!DOCTYPE html>

    <meta charset="UTF-8">
    <title>AngularJS Routing Template..</title>

{{message}}

Home

About

--------
app.js
--------

var app=angular.module('singlePageApp',['ngRoute']);

app.config(function($routeProvider){

    $routeProvider
        .when('/home',{
            controller:'singlePageAppController',
            templateUrl: 'home.html'
        })
        .when('/about',{
            templateUrl: 'about.html'
        });

});

app.controller('singlePageAppController',function($scope){

    $scope.message="Hello Single Page Application..";

});


PIVOT values from two columns to multiple columns

Question: Table: Sample
ID      Day    Status   MS
----------------------------
 1       1       0      10
 1       2       0      20
 1       3       1      15
 2       3       1      3
 2       30      0      5
 2       31      0      6

Expected Result:
 ID  Day1  Day2  Day3....Day30  Day31   Status1  Status2  Status3...Status30  Status31
 ---------------------------------------------------------------------------------------
 1    10    20    15      NULL   NULL      0        0        1           NULL     NULL

 2    NULL  NULL  3        5      6        NULL     NULL     1            0         0

I want to get the MS and Status value for each day from 1 to 31 for each ID.
I have used PIVOT to get the below result.
Result:
ID  Day1  Day2  Day3....Day30  Day31
-------------------------------------
 1    10    20    15      NULL   NULL

 2    NULL  NULL  3        5      6

Query:
SELECT
     ID
    ,[1]     AS Day1
    ,[2]     AS Day2
    ,[3]     AS Day3
        .
        .
        .
    ,[30]    AS Day30
    ,[31]    AS Day31
FROM
(
    SELECT
        ID
        ,[Day]
        ,MS
    FROM
        Sample
) AS A
PIVOT
(
    MIN(MS)
    FOR [Day] IN([1],[2],[3],...[30],[31])
) AS pvtTable

How can I merge the Status column with the result?.
Comment: simply consider the result you got as one table 't1' and create same with the second option i.e. with Status 't2' and join it with id
Comment: I have tried what you said. But is there any possibility to do it with the single PIVOT.
Answer: Try this. Use Another Pivot to transpose Status column. Then use aggregate (Max or Min) in select column list with group by Id to get the Result.
CREATE TABLE #est
  (ID INT,[Day] INT,[Status] INT,MS INT)

INSERT #est
VALUES (1,1,0,10),(1,2,0,20),(1,3,1,15 ),
       (2,3,1,3),(2,30,0,5),(2,31,0,6)

SELECT ID,
       Max([Day1])    [Day1],
       Max([Day2])    [Day2],
       Max([Day3])    [Day3],
       Max([Day30])   [Day30],
       Max([Day31])   [Day31],
       Max([status1]) [status1],
       Max([status2]) [status2],
       Max([status3]) [status3],
       Max([status30])[status30],
       Max([status31])[status31]
FROM   (SELECT Id,
               'status' + CONVERT(VARCHAR(30), Day) col_stat,
               'Day' + CONVERT(VARCHAR(30), Day)    Col_Day,
               [status],
               ms
        FROM   #est) a
       PIVOT (Min([ms])
             FOR Col_Day IN([Day1],[Day2],[Day3],[Day30],[Day31])) piv
       PIVOT (Min([status]) FOR col_stat IN ([status1],[status2],[status3],[status30],[status31])) piv1
GROUP BY id


Serial Correlation - Fixed Effects Estimation Proof

Question: Assume we have a standard panel model. $y_{it} = \alpha_{I} + \beta{x_{it}} + u_{it}$ Also assume $Cov(u_{is},u_{it}) = 0$.
I would like to show the fixed effects estimation strategy entails $Cov(\ddot{u}_{it}, \ddot{u}_{is}|X) \neq 0$. In words, we need to cluster for serial correlation.
Here's how far I can get into this proof.
\begin{align*}
Cov(\ddot{u}_{i,s},\ddot{u}_{i,t}) &=  Cov(u_{is} - \bar{u}_{i}, u_{it} - \bar{u}_{i}|X) \\
&= E[(u_{is} - \bar{u}_{i})(u_{it} - \bar{u}_{i})|X] - \underbrace{E[u_{is} - \bar{u}_{i}|X]E[u_{it} - \bar{u}_{i}|X]}_\text{Zero because $E[\ddot{u_{i}}|X] =0$}\\
&= E[(u_{is} - \bar{u}_{i})(u_{it} - \bar{u}_{i})|X] \\
&= E[u_{is}u_{it} - u_{is}\bar{u}_{i} - u_{it}\bar{u}_{i} + \bar{u}^{2}_{i}|X]  \\
&= E[u_{is}u_{it}|X] - E[u_{is}\bar{u}_{i}|X] - E[u_{it}\bar{u}_{i}|X] + E[\bar{u}^{2}_{i}|X]
\end{align*}
The first term after the last equality is zero because we assume $Cov(u_{is},u_{it}) = 0$.
Yet from here I can't solve it any further. I suspect the result is $-\frac{\sigma_{u}^{2}}{T}$. Let me know how I should see this.
Edit: Yes, I am assuming $Var(u_{it}|X) = \sigma_{u}^{2} \; \forall \; i$.
Comment: It seems you are also assuming homoscedasticity in the sense that $var(u_{it})=\sigma_u^2$ for all $t$ such that you have classical errors $\sigma_u^2I_T = Var(u)$ with $u =(u_1,...,u_T)^\top$, is that the case?
Comment: @StopClosingQuestionsFast Yes, that is correct.
Comment: In that case the within estimator is efficient.
Answer: Let $u_i=(u_{it},...,u_{iT})^\top$ have zero mean then
$$Var(u_i) = \mathbb E[u_iu_i^\top] = \sigma^2I_T$$
Let $D$ be the first difference matrix and do first differences to get transformed model
$$Dy_i = DX_i\beta + Du_i$$
Then see that the variance is
$$Var(Du_i) = D\mathbb E[u_iu_i^\top]D^\top=\sigma^2 DD^\top$$
Since $DD^\top$ is known you can use generalized least squares known to be optimal. Applying this to the transformed model result in the estimator
$$\hat \beta = \left(\sum_i X_i^\top D^\top(DD^\top)^{-1}DX_i^\top\right)^{-1}\sum_i X_i^\top D^\top(DD^\top)^{-1}Dy_i$$
but the matrix $ D\top(DD^\top)^{-1}D = Q$ where $Q$ is the demeaning matrix.
The covariance matrix $Var(\ddot u_i) = Var(Qu_i) = Q \mathbb E[u_iu_i^\top]Q^\top = \sigma^2_u QQ^\top = \sigma^2_u Q$ due to symmetry and idempotency of $Q$ matrix.
It follows that diagonal terms are given as
$$Var(u_{it}) = \sigma_u^2 (1 - 1/T)$$ and that off-diagonal terms are given as
$$Cov(u_{it},u_{is}) = -\sigma_u^2/T$$
knowing the structure of the demeaning matrix here shown in a $3 \times 3$ version
$$Q := \begin{pmatrix} 1-1/T & -1/T & -1/T \\ -1/T & 1 - 1/T & -1/T \\ -1/T & -1/T & 1 - 1/T\end{pmatrix}$$
Witout matrix algebra the off-diagonal terms $s\not=t$ can be found by seeing that
$$Cov(\ddot u_{it},\ddot u_{is}) = \mathbb E[\ddot u_{it}\ddot u_{is}] = \mathbb E[(u_{it} - \bar u_i)(u_{is} - \bar u_i)]$$
first identity follows by definition of covariance and use of the fact that $\mathbb E[\ddot u_{it}]=0$ second from defintion of $\ddot u_{it}$. Continuing
$$= \mathbb E[u_{it}u_{is}] - \mathbb E[u_{it}\bar u_{i}] - \mathbb E[\bar u_{i}u_{is}] + \mathbb E[\bar u_i \bar u_i]$$
First term is $0$ by assumption, second and third term are $-\sigma_u^2/T$ because $u_{it}$ is uncorrelated with all terms in the average $\bar u_i$ except one which is $u_{it}$ hence $\mathbb E[u_{it}\bar u_i] = \frac{1}{T} \mathbb E[u_{it}u_{it}] = \frac{\sigma_u^2}{T}$. The final term $\mathbb E[\bar u_i \bar u_i] = \frac{1}{T^2} \mathbb E[(u_{i1} + ...+ u_{iT})(u_{i1} + u_{iT})] = \frac{T}{T^2} \sigma_u^2 = \frac{1}{T}\sigma^2_u.$ Insert these and get the result as stated above.
However this serial correlation is of little consequence for standard errors because
$$\sqrt N (\hat \beta - \beta) = \left(\frac{1}{N} \sum_i \ddot X_i^\top \ddot X_i\right)^{-1} \left(\frac{1}{\sqrt N}\ddot X_i^\top \ddot u_i\right),$$
and $\ddot X_i^\top \ddot u_i = X_i^\top Q^\top Q u_i = X_i^\top Q^\top u_i = \ddot X u_i$ showing that demeaning of errors can be removed such that
$$\sqrt N (\hat \beta - \beta) = \left(\frac{1}{N} \sum_i \ddot X_i^\top \ddot X_i\right)^{-1} \left(\frac{1}{\sqrt N}\ddot X_i^\top  u\right),$$
implying that the asymptotic variance is $\sigma^2_u \mathbb E[\ddot X_i^\top \ddot X_i ]$ under the assumption that $\mathbb E[u_iu_i^\top\lvert \ddot X_i] = \sigma_u^2 I_T$ as was assumed.
The standard errors are not clustered because the above removal of the $\ddot u$ in the expression for the asymptotic covariance implies that there is no need for estimation of $\hat \Omega = \frac{1}{N} \sum_i \hat{\ddot u}\hat{\ddot u}^\top$. As they say in the reference you post "In the setting without clustering, the key assumption is that $\Omega$ is diagonal." And although $\Omega := Var(Qu)$ is not diagonal this has no consequence for the asymptotic variance where the variance $Var(u)=\sigma^2I_T$ which is diagonal  is used. Offcourse if you want to calculate robust standard errors the story is different, then there is clustering by individual.
Comment: I have edited the answer to include derivation of $Var(\ddot u)$ and a comment on clustering.
Comment: The matrix algebra is much simpler, but I have added the calculation of covariance without matrix algebra.
Comment: Thanks for the reply. Everything you said is correct and useful. But that does not answer the question about serial correlation, unfortunately.
Comment: You say we need to cluster for serial correlation .... point is you dont when using the within estimator. I understood your question to be about the estimator. Is it the variance estimation used to compute standard errors you are thinking about-
Comment: Firstly, I am asking about the proof. It's something I did in a graduate school course but cannot remember to solve now. Secondly, you do cluster with fixed effects estimators. See https://economics.mit.edu/files/13927. I am trying to see why that is the case for FE. Edit: If you're right, then just solve the proof and show me that $Cov(\ddot{u}_{it}, \ddot{u}_{is})=0$.
Comment: I appreciate it. I follow your steps until you get to the diagonals and off-diagonals. I don't see how those follow. Again, if you could show me how to do it via the proof steps I posted, that would be great.
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/105701/discussion-between-juliusbilly-and-stop-closing-questions-fast).


How can I modify the return-path in a google workspace account that has the capability to send as another workspace account?

Question: I haven't been able to find an answer to this, so forgive me if it's been asked somewhere before.
I'm working with non-profit who has a google workspace for non profits account.  I'm working on getting our SPF/DKIM/DMARC records set up, and they work perfectly except for one situation.
We have two different domains, whedoncon.com, and thehellmouth.org.  Some of our users have an email on both domains (i.e. firstname.lastname@example.com and email@example.com are going to the same person).  I can send emails individually from each domain, and they pass SPF, DKIM, and DMARC fine.  The problem comes in when I set up the domains to be able to send from each other.
I've added the capability for firstname.lastname@example.com to be able to send mail as firstname.lastname@example.com.  The issue seems to be when I log in as firstname.lastname@example.com, and send a message as email@example.com.  Looking at the email headers, it seems that because I logged in as firstname.lastname@example.com, it sets the return-path to the whedoncon.com address regardless of what account I select to send out the email.
The problem with this, is it causes DMARC to fail whenever I send an email out as firstname.lastname@example.com, even though SPF and DKIM both pass.  It seems to be because the return-path is showing as email@example.com, but the DKIM is looking at hellmouth.org.
So, TL:DR, google seems to always default to the signed-in account for the return-path, and not the secondary account that it's actually sending from.  Is there a way I can change the return-path so it matches the account the email is coming from, and not the account that I'm signed in as?
Answer: Google Workspace has a primary domain and users are assigned a primary address under that domain. When you have a domain alias, users are assigned an alias address under that alias domain.
The envelope sender address (also known as the return-path address) and the From: address for a message can be different or the same.
If users send email from their alias address, the return-path address will be their primary address, while the From: address will be their alias address.
To pass DMARC, a message must pass at least one of these checks:

SPF authentication and SPF alignment
DKIM authentication and DKIM alignment

SPF typically uses the message envelope sender address for authentication. DKIM uses the message From: address for authentication.
When the domain alias is setup correctly, both the SPF and DKIM authentication will pass. However, only DKIM alignment will pass, SPF alignment will not pass. But that is okay because DMARC does not require SPF alignment to pass as long ask DKIM authentication and DKIM alignment pass.
Comment: This is good info but I don't see how it addresses the issue, there are many support threads on this, e.g. [this one](https://support.google.com/a/thread/69536504/spf-alignment-when-sending-from-a-domain-alias?hl=en). I have an aliased domain and, despite having an aliased user *and* the second domain fully set up with its own SPF and (separate) DKIM records, the headers of sent mail will still show `Return path email@example.com` when I send as `email@example.com`. Google isn't choosing the right records to authenticate the send.


Maximal Element of a subset

Question: This has probably been asked before but how do I show that every non-empty bounded above subset of $\mathbb N$ has a maximal element? Doesn't the set: $S =[0,1) $ disprove this as I can always pick a larger natural number within $S$ that is not the maximal element? Moreover the supremum $1 \notin S$.
Comment: The set $S$ you created is **not** a subset of $\Bbb{N}$
Answer: The only natural number in $S$ is $0,$ so that does not disprove it.
A set $S \subset\mathbb{N}$ can be seen as a subset of $\mathbb{R}$. So, since it is bounded above it will have a supremum in $\mathbb{R}$. Now, you need to prove that this supremum $s$ will be an element of $\mathbb{N}$. You can prove this by contradiction, assuming that it was not and choosing $\frac{s-\lfloor s \rfloor}{2}$ as $\varepsilon.$ We would have to have $x\in S$ such that $\lfloor s \rfloor<\frac{s+\lfloor s \rfloor}{2}<x\leq s$ which is impossible since $x\in \mathbb{N}$.
Then, you use the fact there has to be an element $x\in S$ such that  $s-\frac{1}{2}<x\leq s$. Since $x\in \mathbb{N}$ and $s\in \mathbb{N}$ we must have $x=s$. Hence, $s\in S$ and the set $S$ has a maximum.
Comment: Thank you so much, somehow I'm struggled seeing how a set in the natural numbers could be bounded but I forgot that N is is a subset of R, thank you.
Comment: @Christheyankee Only finite sets can be bounded, an infinite set will always be unbounded above in $\Bbb N$.

Maximal Element of a subset


Pedal design for scale shift in electric guitar before amplification

Question: I have magnetic pickups using Alnico magnets
I plan to shift scale of all strings by an offset, decided by a potentiometer
Is a 555 timer based circuit possible?
Comment: Are you talking about creating a pitch shifter effect?
Comment: The 555 is a timing chip. You would need an MPC (Multi-Processing Chip). Or you could get a picaxe and upload some code :)
Comment: Not 555-based, but a simple digital logic harmony generator is [here](http://www.experimentalistsanonymous.com/diy/Schematics/Vibrato%20and%20Pitch%20Shift/EMM%20Harmony%20Generator.PDF). (I built one of these in the '90s; I would guess the parts can be adapted as it's just basic digital PLLs and flip flops.)  It produces a simple square wave tone that is in harmony with the input signal.  It will only work on one note at a time - no chords.
Answer: The type of pickups is irrelevant - you have an electronic signal you wish to analyse and change the frequencies detected.
A 555 timer is not the way to go. You are looking at this the wrong way entirely.
Pitch shift circuits typically use ring modulation or other mechanisms to perform Fourier transformation in the frequency domain.


Hibernate User Type is not registering by default

Question: I have written a custom User type to handle Date when value is NULL and Used in my pojo as
@Entity
@Table(name="Person"
)
public class Person {

    private int id;

    @Column(name="dob", length=10)
    @Type(type="poc.datetype.DateTimeUserType")
    private Date dob;

    //... getters and setters
}

When try to persist Person in DB,custom type DateTimeUserType is not invoking.
But If I register custom user type to hibernate Configuration then it working.
DateTimeUserType dateTimeUserType = new DateTimeUserType();
configuration.registerTypeOverride(dateTimeUserType,new String[] { dateTimeUserType.returnedClass().getName() });

I went through many examples of using Custom user type,But no one registered to hibernate configuration.
My question is, is it mandatory to register custom user type to configuration or it should work with out registration ?
Hibernate version using 4.3.5.Final
Comment: Your code is looks fine, have you extended java.util.Date class?

    `public class DateTimeUserType extends java.util.Date { .. }`
Comment: No,My Custom type did not extends java.util.date.   public class DateTimeUserType implements UserType {...} ,    Do I need to extend java.util.Date ,If so what could be the reason ?
Comment: I have resolve this issue either by registering userTypes in dialect or to hibernate sessionFactory.
Answer: Whenever you have usertypes/CustomTypes used in pojo classes,then you must register UserTypes/CustomType to hibernate configuration.And it can be done in two ways,either register custom type at dialect or registering to hibernate configuration.
Below is the way to use UserType and registering user type to configuration.
POJO Class :
@TypeDefs({
                        @TypeDef(
                         name = "DateTime",
                         defaultForType = org.joda.time.LocalDateTime.class,
                         typeClass = com.common.data.type.WMPersistentLocalDateTime.class
               )
})

@Entity
@Table(name="TABLE1"
    ,schema="tr11"
)
public class Table1  implements java.io.Serializable
 {
@Type(type="DateTime")
private LocalDateTime column3;

...getters & setters ....
}

UserType registration :
sessionFactory.registerTypeOverride(new com.common.data.type.WMPersistentLocalDateTime(), new String[]{"DateTime"});

Answer: Please see my below post, UserType api menthods are not called from Criteria Builder query in hibernate
And my answer there, its a bug in hinerate
it has been fixed in the version 5.2.0.Final


How can I add a directory to my PATH in Mac OS X 10.7?

Question:
I created an alias for a command that uses a file from one of my directories.
This alias is loaded in .bash_profile in the following format:
alias NAME_OF_COMMAND='ssh -i NAME_OF_KEYFILE USER@IP_ADDRESS'
I want to be able to use the alias from anywhere on my system but the KEYFILE (a.k.a. NAME_OF_KEYFILE) is only present in one directory which is not part of my $PATH (it's a subdirectory of dropbox).
I tried to adding that directory to the path with the following command:
export PATH=$PATH:/NAME/OF/DIRECTORY

I could see this was nominally successful because when I enter $PATH the new directory shows up at the end of the $PATH variable.

However, when I try to carry out my ssh command I get the error:
Warning: Identity file KEYFILE not accessible: No such file or directory.
Answer: The path is generally used to search for executables, not data files in general.
Your best bet is probably to change the alias so that the keyfile is a fully qualified path name, something like:
alias myssh='ssh -i $HOME/ssh/mykeyfile.ppk firstname.lastname@example.com'

With a fully qualified key file, it won't matter where you currently are in the file system (assuming ssh itself is on your path somewhere, of course, but that's a different matter).