Routing and remote access VPN setup

Question: I am trying to set up VPN using routing and remote access.
I've tried two configurations, one using a single network card, and one using two network cards.
I can connect through my VPN and get assigned an IP address by the DHCP server, but I can't "SEE" anything. By this I mean the client appears completely blind to the office network, this means:

No ping to any office server including the VPN server (I've stopped ICMP being filtered in and out)
No DNS resolution (isn't surprising if I can't connect by IP address. I tried accessing the VPN Server Share and a web page hosted on it using the domain name and the IP address, e.g. http://host.com/abc and http://18.104.22.168/abc)
Can't access any network resources (also not surprising given the above)

I've not had any problems connecting the VPN at all (I thought I had, but this was to do with the test router I given).
This doesn't appear to be a firewall/router problem as I configured it to allow VPN traffic through and forward to the correct server.
This, I think, is confirmed as the VPN server event logs shows success audits (that is, logon events).
However it does show the following event directly after the success audit logon event occurs (and I don't know if this is normal but wouldn't expect it to be - the VPN client doesn't disconnect).
<code>An account was logged off.

Subject:
    Security ID:        [DomainName]\[UserName]
    Account Name:       [UserName]
    Account Domain:     [DomainName]
    Logon ID:       0x[xxxxxx]

Logon Type:         3
</code>
This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.
I am not sure if this is of any use, but Wireshark shows me connecting successfully and then a load of encrypted packets, which is what I would suspect:

There's a PPP LCP/CHAP configuration conversation which is the client connecting.
I don't see any ICMP on either side when I do a ping to the VPN public interface (I assume this is because it is encapsulated on the GRE packets).

However the following is interesting (which I can't explain):

Pinging the public interface (A) (of VPN Server) I see increased PPP and GRE traffic
Same if I ping the private interface (B) (of VPN Server)
If I ping another server (C) on the network I can see the ICMP packet requests but no replies (I'm not sure if (C) server is replying directly to the VPN IP of the client (D) or not - this does appear the case as I see the traffic in (A))

I can't explain points 1 or 2 - I'd expect to see ICMP, but the problem does appear to be sending traffic to (D). To check this I pinged (D) from (C) and RECIEVED a reply, however I DID NOT see correlating ICMP traffic on (D) (could this have been GRE traffic?). I find this strange, but (C) is definitely pinging the correct machine as it stops responding when I disconnect (D) from the VPN.
Also note, I don't see any traffic on (D) to the VPN network addresses, only to the routers which have the port forwarding set up. Could this be some sort of routing problem?
The VPN server does seem to have a problem pinging the (D)- says NO RESOURCES, PathPing shows it is using interface (A). and this problem only occurs when pinging (D), it pings everything else without problem.
I changed RRAS to single NIC setup and the no resources problem went away - I can seemingly ping the client from all machines, but can't ping anything from the client. I say seemingly as when I ping the client it takes less than 1 ms (bear in mind this is across the Internet and different ISP's) and I don't see any ICMP traffic on the VPN server - plus when I disconnect the client, it is still pingable!?! (As if the VPN server is replying to pings on the client's behalf.) Whilst this is happening the VPN server gets timeouts when it pings the disconnected client. VERY STRANGE!
After leaving it for some time (drinking tea, eating food sort of length) the VPN server has gone back to the No resources problem, and not I have no ping in either direction. Disabling RRAS and enabling it again put me back to where I was just now, I can now ping from LAN to client.
Comment: It would appear the VPN Server is swallowing stopping traffic getting back to the VPN client. For example if I ping the client from the VPN server, the VPN server itself responds as if it were at the clients IP address (this is confirmed when I ping -a).
Comment: After a while with the RRAS service started, pinging the client ip (which isn't the client ip -see above comment) results in NO resources. message being returned by ping.
Answer: Your terminology "...see anything on the LAN..." is imprecise. What do you mean by "see"? Do you mean that you couldn't PING or make TCP connections to hosts on the LAN? Do you mean that some "Network Places" or such functioinality didn't work?
What you're trying to do will work fine. You're probably not getting NetBIOS name resolution across the VPN because you're probably not using a WINS Server on the LAN. That would be my "psychic powers" guess as to why you're having problems.
Installing RRAS on a domain controller makes it multi-homed. It will work but Microsoft doesn't recommend it. You should think about preventing the RRAS adapter from registering in DNS and WINS.
Edit:
I don't think there's anything "contrived" about my answer. I'm trying to help based on your imprecise description of your problems (using the term "see" nstead of saying exactly what is failing when you're connected) and my experience with these types of problems. Your vague statement about using RADIUS gave me some feeling that you weren't a professional sysadmin (later validated by your comment re: your job) and that you were probably trying to use some graphical tool or application to access resources on the LAN but hadn't performed the basic troubleshooting steps of verifying layer 3 communication, name resolution, etc.
I've setup RRAS servers on domain controllers on LANs that are connected to the Internet behind NAT firewalls. I connect to them several times a week. What you're trying to do works fine.
Are you allowing the RRAS server to assign IP addresses to clients from DHCP, or have you specified an address range? If you've specified an address range is it a range that is within the LAN subnet, or is it a different subnet? Is the IP being assigned to the client when "connected" what you'd expect to see?
It's still unclear to me what you've tried doing once "connected" that makes you think you can't "see" the LAN. Can you PING the RRAS server's IP address? Can you make TCP connections to services hosted by the RRAS server or other servers on the LAN by IP address? Are you getting DNS resolution?
Finally, I did not suggest that moving RRAS to another server would make anything work. I suggested that Microosft doesn't recommend multi-homed domain controllers. RRAS will run fine on a domain controller, provided you understand the ramifications therewith.
Edit 2:
With the RRAS server setup to assign IP addresses from DHCP you're seeing a good LAN IP address being assigned to the client, then?
Assuming you are, and you can't PING the RRAS server's LAN IP address from the client, it's time to start sniffing traffic. I'd sniff on the RRAS server and on the client to see that the PING request is properly routing out the VPN connection (as an encrypted GRE payload-- presumably you're using PPTP). If sniffing is inconvenient you can watch the bytes transferred via the "Status" dialog for the connected client in the "Remote Access Clients" node in the "Routing and Remote Access" management console snap-in. I'd sniff, though-- there's no substitute for seeing the data on the wire.
The client's routing table looks like you'd expect after connection, too, I'd assume. By default, the Microsoft VPN client assigns your default gateway to the remote network (the "Use default gateway on remote network" checkbox in the "Advanced" TCP/IP properties for the VPN connection). If you turn that off, instead of seeing your default gateway change you'll see an entry for the remote network with a gateway of the IP address assigned to the client's VPN adapter. You don't mention what the client OS is, but the behaviour of the Microsoft VPN client changed slightly in Windows 7 (allowing you to disable the silly "classful" route addition behaviour explicitly).
It probably goes w/o asking, but the VPN server's LAN IP subnet and the LAN subnet where the client is connected are using different address ranges, aren't they?
Comment: You won't be able to decrypt the GRE without a *lot* of complicated work. You should be able to do a traffic analysis, though, to see if you're seeing GRE packets corresponding to PING requests that should be traversing the VPN. Wireshark will also let you sniff on the VPN interfaces themselves (the nomenclature differs in versions of Wireshark, but typically "PPP" or "WAN" is in the interface name). That would help you see if the requests are being decapsulated properly on the far end.
Comment: @Mr Shoubs: I've dropped on an edit.
Comment: I don't want WINs or NetBios name resolutions, I want my DNS Server to take care of that.
Comment: @Mr Shoubs: You're misunderstanding my desire for precision in terminology as hostility. Computer networks have a lot of "moving parts", and calling everything by the precise name helps us mutually understand what's going on. I've dropped on an edit with some additional suggestions. My suggestion re: the multi-homed domain controller was made as a suggestion aside from your question. I forsaw that you might not know that a multi-homed domain controller can be the source of other, unrelated problems, and I thought I'd bring it to your attention.
Comment: Your assumptions are correct. I was hoping I wouldn't have to sniff traffic and there'd be a simple solution. Guess it's time to fire up WireShark. Client OS=Win7.Pro/WinXP.Pro Using both.

Yes they are using different ranges and the routing table looks how I'd expect.
Comment: Your answer did come across as particularity hostile, but lets not worry about that, your helping me now and that's what counts.
Comment: I am seeing a lot of traffic between client and server, but can't see what any of it is as it is GRE or PPP Comp Protocols. Is there anything in particular that I should be looking for? I applied filters to show me traffic between the client ip and vpn ip (which is the external ip of the router due to port forwaring) but as always, I've not sniffed 'normal' vpn traffic before, so am not sure what I should be looking at.
Comment: I updated my question with more information
Comment: What a snooty answer
Comment: @reach4thelasers: I have no tolerance for imprecise trouble reports. Computers don't "see" things. You can't solve a problem effectively if you can't communicate effectively. I'm happy with my "snootiness". I get results and the community seems to be alright with my approach, too.
Answer: This sounds to me like your VPN connection isn't configured properly. In order for a VPN to work, the client has to change some of its IP configuration details, particularly:

your name server has to be able to
resolve names in the target network.
This is generally achieved by having
the VPN server forward the IP
address of a suitable DNS server.
Your VPN server has to forward the relevant routes that are required to get traffic flowing. Usually a VPN connection operates on a very small (/30) subnet and your VPN server will handle the distribution into the various tunnels it holds. However, your client needs to know that IP address of the target network are reachable through the tunnel, and therefore additional routes are required in the clients routing table.

You can check this things as follows:
while on the VPN, open a shell (DOS-prompt, whatever) on your client and run (under Windows)ipconfig /allor (under linux) cat /etc/resolv.confThis will show you which DNS server you are using. This DNS server either must reside on the target network (the most commonly used solution) or must be able to resolve names of machines in the target network to IP addresses.
The second test is to check your routing. Under Windows type route printUnder Linux, use sudo route -nvThis will give you output looking like this:

# route -nv
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.180.0.1      10.180.0.169    255.255.255.255 UGH   0      0        0 tun0
10.180.0.169    0.0.0.0         255.255.255.255 UH    0      0        0 tun0
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
10.172.0.0      10.180.0.169    255.255.0.0     UG    0      0        0 tun0
10.171.0.0      10.180.0.169    255.255.0.0     UG    0      0        0 tun0
0.0.0.0         192.168.10.254  0.0.0.0         UG    0      0        0 eth0

In the above, the networks 10.171.0.0/16 and 10.172.0.0/16 are routed the tunnel.
If either of these two is missing on the client, then check your VPN server configuration, whether it acutally is set up to forward the relevant bits to the client. This obviously depends on the type of VPN server you are using.
Comment: I added a route manually using route add, but it didn't seem to make any difference.
Comment: have removed it again. - I'm not convinced this is a client problem (I've tried multiple clients from a couple of different external networks) and I have connected as a VPN client before at another place  (separate to this setup entirely) which worked fine.
Comment: I've noticed that after pinging from the VPN Server a few entrys are added (on VPN server) where the interface is the client address. This sounds incorrect to me.
Comment: DNS is found ok and the routing table seems a little weird. I don't have the .0 - the destination is 192.168.254.x (I know I didn't set it up as class C, but it has to stay that way due to firewall/portforwarding config) but there is no 192.168.254.0 in the routing table, there is 192.168.254.255 though.
Answer: I suggest changing your IP subnet. If you are IP address is 192.168.x.100 on your local network and you are trying to VPN into a remote site which also uses the x subnet, your problems could make sense.
Change your IP subnet. Not so easy I know...hahah.
Answer: It would appear to be a problem with DHCP traffic. I don't know why.
To resolve this I MANUALLY added a static address pool (it won't work if you configure it like this when setting up RRAS).
Now I can ping. Though DNS seems to be a problem even though the correct DNS servers are picked up by the client - to solve this you need to configure the VPN network connection on the client to append DNS suffix.
Comment: dhcp is a layer 2 protocol, and cannot be passed over routers unless dhcp relaying is enabled.

Routing and remote access VPN setup


Access Kedro context from decorator

Question: I am trying to create a decorator in which I need some information about the project and/or catalog.  Is it possible to access the project context from inside of the decorator?  I am looking for things like project_name, catalog entry name, and pipeline node name.
I created a way to find the root directory of the by getting the functions file path inspect.getfile(func), then walking up the path until I find .kedro.yml, but this method breaks when using a function from a library.
Answer: Passing the context around in the nodes is not recommended and would not fit in with the Kedro architecture (see https://kedro.readthedocs.io/en/latest/06_resources/02_architecture_overview.html). The ProjectContext holds the a collection of nodes (i.e. pipeline), which would hold the ProjectContext, which then gets into dangerous recursive teritory.
Alternatives would be to either a) pass the values via parameters, and referencing them in the node inputs using params:abc, or b) passing the actual values in the decorator.
a)
For instance, project_name you could pass in via parameters. Either by entering it in parameters.yml or dynamically by overriding the method _get_feed_dict in the ProjectContext to add an entry {"project_name": self.project_name} to the returned dictionary.
b)
You could have a decorator that looks like this and pass it to Node.decorate():
from functools import partial, wraps
from kedro.pipeline import Pipeline, node

def print_metadata(func, **outer_kwargs):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print("METADATA: {}".format(outer_kwargs))
        return func(*args, **kwargs)
    return wrapper

p1 = Pipeline([node(...), ...])
decorated_nodes = [
    n.decorate(partial(print_metadata, node_name=n.name, catalog_entries=n.inputs))
    for n in p1.nodes
]
p2 = Pipeline(decorated_nodes)

Not super-pretty but can get you unstuck. What is your exact use case? Why do you need the node name and catalog entry name in the node?

Access Kedro context from decorator


How to remove caret cursor in Gtk entry

Question: I have a Gtk entry. I need to completely remove this caret cursor, how can I do this? I searched for information about this for a long time but found only how to remove the blinking of Gtk entry.
Answer: Until you find a better one use CSS.
Just set the caret-color background to the same background-color of the entry:
main.c
#include <gtk/gtk.h>

int main ( void )
{
    GtkWidget *window;
    GtkWidget *grid;
    GtkWidget *entry;

    /// ***
    gtk_init ( NULL, NULL );

    /// ***
    window = gtk_window_new ( GTK_WINDOW_TOPLEVEL );
    gtk_window_set_title ( GTK_WINDOW ( window ), "Hello There!" );
    gtk_window_set_default_size ( GTK_WINDOW ( window ), 200, 100 );
    g_signal_connect ( window, "destroy", gtk_main_quit, NULL );

    ///***
    grid = gtk_grid_new();
    gtk_container_add ( GTK_CONTAINER ( window ), grid );

    ///***
    entry = gtk_entry_new();
    g_object_set ( entry, "margin-top", 30, NULL );
    g_object_set ( entry, "margin-left", 20, NULL );
    gtk_grid_attach ( GTK_GRID ( grid ), entry, 0, 0, 1, 1 );

    /// ***
    gtk_widget_show_all ( window );
    gtk_main();
}

CSS:
window
{
    background-color: red;
}

entry
{
    background-color: yellow;
    caret-color:      yellow;
}

rezult:

How to remove caret cursor in Gtk entry


kubeadm upgrade plan warning when updaten from 1.23 to 1.24 on ubuntu

Question: A few weeks ago I upgraded kubernetes 1.22 to 1.23. I'm using containerd as container runtime a long time. Os is Ubuntu 22.04.3 LTS
Now when upgrading from 1.23 to 1.24 kubeadm upgrade plan show warning about runtime as
W0917 12:14:33.970744   27728 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!

I have made quite lot of research to find out, where to add/fix setting to get out of this warning. Config-changes I have made at least /var/lib/kubelet/kubeadm-flags.env and /etc/crictl.yaml files, but there is no success. Anybody?
This was copied from stack overflow, I made it there some reason :-(
Before it was closed, Ola Ekdahl add anwer to this - thank's. I add it here.
Run this command on your control plane and nodes.

sudo crictl config \
--set runtime-endpoint=unix:///run/containerd/containerd.sock \
--set image-endpoint=unix:///run/containerd/containerd.sock

This does not solve the problem.
Answer: After installing 1.24-->1.25 this message disappear. Now at version 1.29 and no messages at any step.


Change UINavigationBar Background Color of UISearchController

Question: At the minute, my search controllers "navigation bar" background is the standard default translucent however, all the other bars in my app are black translucent. When I print the navigationController property of the searchController, it returns nil and so does the navigationBar, but not the navigationItem. There are no other public properties that seem to let you change the color of the bar. I don't mind overriding private ivars as I wont be releasing this app on the app store.
Answer: You can use the barTintColor property, in Swift this would be:
searchBar.barTintColor = UIColor.redColor()


Mocking 3rd party React component in jest has no effect

Question: I'm using jest with react testing library to provide basic testing for my React app. I have mocked the REST calls with msw.
Unfortunately one of the 3rd party libraries I'm using doesn't seem to render output properly in those tests and as I am more interesting in testing the whole flow than precise results I've decided to mock it and replace with a dummy implementation.
The library is react-photo-album, and I'm importing it in my component as:
import PhotoAlbum from "react-photo-album";

My test looks like:
it('loads photos', async () => {

 //mocking goes here

  const { container } = render(<MemoryRouter><Gallery /></MemoryRouter>);

  screen.debug();

  await waitFor(() => {
      expect(container.getElementsByClassName('mockedComponent').length).toBe(1);
   });
});

I'm mostly interested in the result after msw responds to the rest calls providing actual data, but for now I'd be happy to at least see the PhotoAlbum component replaced at all in the debug view. Yet, it doesn't seem to be affected at all. Instead original component is rendered.
Now, what I've tried for mocking (already dropping any fancy logic):
  jest.doMock('react-photo-album', () => ({
    PhotoAlbum: () =>
,
  }));

  jest.mock('react-photo-album', () => ({
    __esModule: true,
    PhotoAlbum: () =>
,
  }));

  jest.mock('react-photo-album', () => ({
    __esModule: true,
    default: () =>
,
  }));

  jest.mock('PhotoAlbum', () => (props) => {
    return
;
  });

plus multiple variations of those, including mock and doMock and even simply:
  jest.mock('react-photo-album');

But my component seems to be untouched!. With mock method I mostly get error of:

The module factory of jest.mock() is not allowed to reference any
out-of-scope variables. Invalid variable access: _jsxFileName

but doMock seems to solve this problem.
So what am I doing wrong? I feel like I'm missing some detail.
For the record neither React or jest are the tools I'm normally used to work with. I've also seen answers like: How to mock a third party React component using Jest?, but it just doesn't seem to work.
Answer: The error you are experiencing is probably specific to your environment / project setup as one of the mock options that you mentioned in your question actually works.
Here is a working example:
import { render } from "@testing-library/react";
import Gallery from "../src/Gallery";

jest.mock("react-photo-album", () => ({
    __esModule: true,
    default: () =>
,
}));

describe("Gallery", () => {
    it("loads photos", () => {
        const { container } = render(<Gallery />);

        expect(container.getElementsByClassName("mockedComponent").length).toBe(1);
    });
});

Now, if you want to address the original issue of PhotoAlbum not rendering in jest tests, you can fix it by mocking the PhotoAlbum's container
 clientWidth:
import { render } from "@testing-library/react";
import Gallery from "../src/Gallery";

// mock PhotoAlbum container clientWidth
jest
    .spyOn(Element.prototype, "clientWidth", "get")
    .mockImplementation(function () {
        return (this.className || "").split(" ").includes("react-photo-album")
            ? 800
            : 0;
    });

describe("Gallery", () => {
    it("loads photos", () => {
        const { container } = render(<Gallery />);

        // test PhotoAlbum rendered 3 photos
        expect(
            container.getElementsByClassName("react-photo-album--photo").length
        ).toBe(3);
    });
});

You can find minimal working example in the following sandbox.
https://codesandbox.io/p/sandbox/stackoverflow-75817885-fs9y5m?file=%2Ftest%2FGalleryMock.spec.jsx


Remove set of items from ArrayList using LINQ

Question: I have Arraylist which contains all attribute's IDs (list1). Now I have another set of attribute's Ids (list2) which needs to be remove from first ArrayList (list1)
I am at beginner stage as LINQ developer so please suggest proper code snippet
Arraylist attributeIDs; // which contains Ids
int[] ids = { 1, 2, 3, 4 };
var id = ids.Select(s => s);
var sam = attributeIDs.Cast<IEnumerable>().Where(s => id.Contains(Convert.ToInt32(s)));
Arraylist filterAttributDs = Cast<Arraylist>sam;

After above code, I need to transfer output Arraylist to different methods so I need output in Arraylist only
Thanks in advance!
Comment: Do you *really* need to use `ArrayList` rather than a generic collection?
Comment: You could have used the "Except" if it was a List.
Comment: Linq is not made to remove something from a enumerable (collection). It is made to create a new enumerable in which you can exclude certain items. It usually (always) results in a new collection.
Answer: Try out the method
var sam = attributeIDs.Cast<IEnumerable>().Where(s => id.Contains(Convert.ToInt32(s)));
ArrayList myArrayList = new ArrayList(sam );

EDIT
int[] ids = { 1, 2, 3, 4 };
//var id = ids.Select(s => s);
List<int> id = ids.OfType<int>().ToList();
var list = attributeIDs.Cast<int>().ToList();
//or try
//List<int> list = new List<int>(arrayList.ToArray(typeof(int)));
var sam = list.Where(s => id.Contains(s));
//if you want to remove items than , !Contains() rather an Contains()
// var sam = list.Where(s => !id.Contains(s);
//also try out Intersect or Except instead of this as jon suggestion in comment
//var sam= attributeIDs.Except(id); for diffence between list
//var sam= attributeIDs.Intersect(id); for common in list

ArrayList myArrayList = new ArrayList(sam );

combine all
check this for : LINQ to SQL in and not in
Comment: I think you mean `Cast`, at which point you can remove the `Convert.ToInt32` call.
Comment: @JonSkeet - quetion is not that much clear to me but i think ...he wants to convert output of linq qunery in ArrayList so i tried above code
Comment: Well it's not clear why you've still got the first part, which definitely won't work. Also, consider `Intersect` and `Except`...
Answer: new Arraylist((from id in attributeIDs where !ids.Contains(id) select id).ToList())
Like Jon mentioned though, you should consider just using a simple array or a generic collection. Also note that the above query runs in O(n*m) where n is the number of items in the original list and m is the number of elements in the list that you are trying to remove. You should consider using a HashSet and then using a set difference operation here for better performance.
Answer: Slight tweaks to your example:
ArrayList attributeIDs = new ArrayList(){ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
int[] ids = { 1, 2, 3, 4 };
var sam = attributeIDs.Cast<int>().Intersect(ids);

A few notes:

The Cast casts the underlying item type, not the type of the collection.
foo = setOfThings.Select(a => a) <==> setOfThings
Intersect means "only select those elements that appear in both sets"

(as mentioned elsewhere, Intersect is non-optimal for large sets of data: consider using an appropriate structure, like a HashSet)
Answer: solution to your problem is :
ArrayList firstList =new ArrayList(new int[]{ 1, 2, 3, 7, 8, 9 });
ArrayList secondList =new ArrayList(new int[] { 1, 3 });
var result = from c in firstList.ToArray() where !(from n in secondList.ToArray() select n).Contains(c) select c;
foreach (var temp in result)
  Console.WriteLine(temp);
OutPut of the above is : 2 7 8 9 i.e. items of secondList are removed from firstList, also result is a ArrayList as you required to get.


Entropy of a Gaussian Process: Log(Determinant(CovarianceMatrix))

Question: I want to be able to compute the entropy of a Gaussian Process.  To that end, I have a simple example in GPFLow.  I have a latent function which I sample with two different levels of noise, L1 and L2 where Noise(L1)>Noise(L2).  I then regress the Gaussian process in a small range over my function and compute the covariance matrix, the determinant of this, and then the log of the determinant as the entropy.  I have found a few things.  Firstly, as I compute the covariance matrix for an increasingly dense set of points in a fixed interval, the determinant goes to zero (0.0, actually).  Secondly, as I decrease the noise from L1 to L2, the log of the determinant does not go toward zero.  Are my reasoning and computations correct?
Here some extra notes about the code I am using.
Here is the code I am using:
import gpflow
import numpy as np
N = 300
noiseSize = 0.01
X = np.random.rand(N,1)
Y = np.sin(12*X) + 0.66*np.cos(25*X)  + np.random.randn(N,1)*noiseSize + 3
k = gpflow.kernels.Matern52(1, lengthscales=0.3)
m = gpflow.models.GPR(X, Y, kern=k)
m.likelihood.variance = 0.01
aRange = np.linspace(0.1,0.9,200)
newRange = []
for point in aRange:
    newRange.append([point])
covMatrix = m.predict_f_full_cov(newRange)[1][0]
import math
print("Determinant: " + str(np.linalg.det(covMatrix)))
print(-10*math.log(np.linalg.det(covMatrix)))

Answer: So, first things first, the entropy of a multivariate normal (and a GP, given a fixed set of points on which it's evaluated) only depends on its covariance matrix.
Answers to your questions:

Yes - when you make the set $X$ more and more dense, you're making the covariance matrix larger and larger, and for many simple covariance kernels, this makes the determinant smaller and smaller. My guess is that this is due to the fact that determinants of large matrices have a lot of product terms (see the Leibniz formula) and products of terms less than one tend to zero faster than their sums. You can verify this easily:

Code for this:
import numpy as np
import matplotlib.pyplot as plt
import sklearn.gaussian_process.kernels as k

plt.style.use("ggplot"); plt.ion()

n = np.linspace(2, 25, 23, dtype = int)
d = np.zeros(len(n))

for i in range(len(n)):
    X = np.linspace(-1, 1, n[i]).reshape(-1, 1)
    S = k.RBF()(X)
    d[i] = np.log(np.linalg.det(S))

plt.scatter(n, d)
plt.ylabel("Log Determinant of Covariance Matrix")
plt.xlabel("Dimension of Covariance Matrix")

Before moving onto the next point, do note that the entropy of a multivariate normal also has a contribution from size of the matrix, so even though the determinant shoots off to zero, there's a small contribution from the dimension.

With decreasing noise, as one would expect, the entropy & determinant do decrease but not tend to zero exactly; they'll decrease to the determinant due to the other kernels present in the covariance. For the demonstration below, the dimension of the covariance is kept constant ($10*10$) and the noise level is increased from 0:

Code:
e = np.logspace(1, -10, 30)
d = np.zeros(len(e))
X = np.linspace(-1, 1, 10).reshape(-1, 1)

for i in range(len(e)):
    S = (k.RBF() + k.WhiteKernel(e[i])) (X)
    d[i] = np.log(np.linalg.det(S))

e = np.log(e)

plt.scatter(e, d)
plt.ylabel("Log Determinant")
plt.xlabel("Log Error")

Comment: Thanks very much for your help for these problems.  It has been very useful and great information!!!  I have a bounty over at stack overflow on the question above,  Why not copy this answer in there too and I will award the bounty.


Hiding parent when child label has a certain "for"?

Question: This is my HTML:

    ...

    ...

...

Now, I want to hide the first div, but it does not exist on every page, so it must be hidden in combination with the for="attribute300". Possible with plain CSS?
Thanks
Comment: `div:has([for="attribute300"])` will do the trick. (You could add the `.label` class or the tag name as well, but since the `for` attribute only occurs on labels, it is doubtful you'll encounter a situation where that additional distinction would actually be necessary.)
Comment: It is possible using [`:has()`](https://developer.mozilla.org/en-US/docs/Web/CSS/:has), but check browser support. (Not available in Firefox yet, unless explicitly enabled by the user.)
Comment: Thanks, how would it look in case if the "for"?
Comment: In the real situation might there be other elements in the div, before and after the label and/or text nodes (ie text not in an element of its own?). Could you give a fuller example of the possible HTML if so?
Answer: div.field >
  .label[for="attribute300"] {
  display: none;
}

You can make this way, but it will be hidden in all scenarios.
code: https://codepen.io/Suryaprakash-the-vuer/pen/dywBRwW
Comment: The attribute is not set on the div itself, but on the label it contains.
Comment: Thanks, but its not working ... sure this is the way to go? I do not only want to hide the label, but the parent of the label including the label.
Comment: @user4095519 Check the edited answer, This should work
Comment: It would hide the label, but like I said, I want to hide the parent of the label as well, including the label.
Comment: Try putting some additional content in the div (not in the label), it will still be visible.


Selectize.js com Tags, recarregar chave e valor em formulário de edição

Question:
Postei a mesma dúvida no SO Gringo também.

Estou fazendo um formulário de pesquisa (action=GET) em que um campo utiliza Selectize.js:
$("#selectize").each(function () {
        $(this).selectize({
            plugins: ['remove_button'],
            valueField: 'AdvogadoId',
            labelField: 'Descricao',
            searchField: ['AdvogadoId', 'Descricao'],
            create: false,
            persist: false,
            preload: true,
            initUrl: "/Advogados/PesquisarJson/",
            initData: true,
            load: function (query, callback) {
                $.ajax({
                    url: "/Advogados/PesquisarJson/",
                    type: 'GET',
                    error: function () {
                        callback();
                    },
                    success: function (res) {
                        callback(res);
                    }
                });
            }
        });
    });

Com o formulário vazio, digito o nome e a sigla do advogado. Vem tudo preenchido corretamente. Ao submeter a pesquisa, os valores vão para o back-end corretamente, separados por vírgulas.
Quando o formulário é recarregado com os valores do GET, o que acontece é que cada tag aparece apenas com a sigla do advogado preenchida, sem o nome. Ao verificar as opções, as opções escolhidas aparecem apenas como a sigla, sendo que o correto seria "Sigla - Nome".
Tentei algumas soluções como esta (não funciona, além de ser complexa pro que eu preciso) e esta (não uso data-data: uso apenas os valores do formulário).
Existe alguma forma (preferencialmente performática) de carregar esses valores corretamente em cada tag selecionada em tela?

Código no Back-end:
    public JsonResult PesquisarJson(String termo = "")
    {
        // Aqui não é EF. É Dapper em cima de Oracle.
        using (var repositorio = new AdvogadoRepositorio())
        {
            var registros = repositorio.Condicao(a => a.DataSaida == null).OrdenarPor(a => a.AdvogadoId).Selecionar();
            return Json(registros.Select(a => new { AdvogadoId = a.AdvogadoId, Descricao = a.AdvogadoId + " - " + a.Nome }).ToList(),
                JsonRequestBehavior.AllowGet);
        }
    }

HTML:

        <input type="text" autocomplete="off" tabindex="" style="width: 4px; opacity: 1; position: relative; left: 0px;">

Resultado do Rest:
[
  {
    "AdvogadoId": "A1",
    "Descricao": "A1 - Fulano de Araujo"
  },
  {
    "AdvogadoId": "A2",
    "Descricao": "A2 - Beltrano de Lima"
  },
  {
    "AdvogadoId": "A3",
    "Descricao": "A3 - Ciclano da Silva"
  },
  {
    "AdvogadoId": "A4",
    "Descricao": "A4 - Herculano Junior"
  },
  ...
]

Comment: Mas como você deseja obter "Sigla - Nome" se o `valueField` só permite um único atributo? considerando que os nomes suas respectivas siglas estejam em banco de dados e possuam um `ID`, a solução seria pegar o `ID` e obter o nome e sigla dos registros selecionados via backend.
Comment: Você também poderia customizar a source da selectize para receber um array de value fields e concatenar os respectivos no formato desejado
Comment: Então, o desafio faz parte da pergunta. Gostaria que essas ideias virassem uma resposta.
Comment: Edita a pergunta com o código do seu controlador por gentileza, assim a resposta não fica genérica
Comment: @FelipeAssunção Feito.
Comment: @FelipeAssunção Coloquei mais informações. Veja se ajuda.
Answer: Isso ocorre porque o <input> ao selecionar é preenchido com o AdvogadoId e não com Descricao, quando você seleciona por exemplo:

Isso que você "enxerga" (botão cinza com o x do lado) é na verdade é o "label" e não o valor do <input>, neste caso o label usa a Descricao e o input usa a AdvogadoId veja valueField: 'AdvogadoId',
A única maneira "pratica" que vejo de fazer como deseja seria isto e colocar o valueField também como Descricao:
 $("#selectize").each(function () {
    $(this).selectize({
        plugins: ['remove_button'],
        valueField: 'Descricao', //Adicionar descrição aqui também
        labelField: 'Descricao',
        searchField: ['AdvogadoId', 'Descricao'],
        create: false,
        persist: false,
        preload: true,
        initUrl: "/Advogados/PesquisarJson/",
        initData: true,
        load: function (query, callback) {
            $.ajax({
                url: "/Advogados/PesquisarJson/",
                type: 'GET',
                error: function () {
                    callback();
                },
                success: function (res) {
                    callback(res);
                }
            });
        }
    });
});

Caso queria realmente enviar as ids (AdvogadoId) terá que na próxima página refazer a consulta usando os ids pra trazer novamente as descrições.
No entanto talvez possa usar sessionStorage junto a API do Selectize.js pra manter os options e adiciona-los automaticamente, um exemplo:
$(function() {
    var target = $("#selectize");
    var keySession = "selectizeData2";

    var update = function(a,b) {
        var opts = this.options;
        setTimeout(function() {//Requer timeout
            var values = target.val().split(",");
            console.log(values);

            var list = [];
            for (var k in opts) {
                current = opts[k];
                if (values.indexOf(current.AdvogadoId) !== -1) {
                    list.push({ AdvogadoId: current.AdvogadoId, Descricao: current.Descricao });
                }
            }

            console.log(JSON.stringify(list));
            sessionStorage.setItem(keySession, JSON.stringify(list));
        }, 1);
    };

    var getTarget = function(selectize) {
        var data = sessionStorage.getItem(keySession);
        var parsedData = JSON.parse(data);
            console.log(parsedData);
        if (parsedData) {
            for (var k in parsedData) {
                var current = parsedData[k];
                selectize.addOption([current]);
                selectize.addItem(current.AdvogadoId);
            }
        }
    };

    target.each(function () {
        var handler = $(this).selectize({
            plugins: ['remove_button'],
            valueField: 'AdvogadoId', //Veja que mantive o ID como value
            labelField: 'Descricao',
            searchField: ['AdvogadoId', 'Descricao'],
            create: false,
            persist: false,
            preload: true,
            initUrl: "test.php",
            initData: true,
            "onItemAdd": update, //Adiciona o evento
            "onItemRemove": update, //Adiciona o evento
            load: function (query, callback) {
                $.ajax({
                    url: "test.php",
                    type: 'GET',
                    error: function () {
                        callback();
                    },
                    success: function (res) {
                        callback(res);
                    }
                });
            }
        });

        getTarget(handler[0].selectize);
    });
});

Seguindo o exemplo do @Felipe também pode usar um pré-consulta (se bem que pra performance isso não parece uma boa quando aumentar o numero de clientes cadastrados), ficaria assim  (é necessário limpar o input primeiro porque se não o Selectize.js considera que as ids já existem):
$(function() {
    var target = $("#selectize");
    var currentValue = target.val();

    target.val("");

    target.each(function () {
        $(this).selectize({
            plugins: ['remove_button'],
            valueField: 'AdvogadoId',
            labelField: 'Descricao',
            searchField: ['AdvogadoId', 'Descricao'],
            create: false,
            persist: false,
            preload: true,
            initUrl: "test.php",
            initData: true,
            onInitialize: function() {
                var self = this;
                var value = "," + currentValue + ",";

                $.ajax({
                  url: "test.php",
                  type: 'GET',
                  success: function(res) {
                    res.forEach(function(existingOption) {
                        self.addOption(existingOption);
                        if (value.indexOf("," + existingOption.AdvogadoId + ",") !== -1) {
                            self.addItem(existingOption.AdvogadoId);
                        }
                    });
                  }
                });
            },
            load: function (query, callback) {
                $.ajax({
                    url: "test.php",
                    type: 'GET',
                    error: function () {
                        callback();
                    },
                    success: function (res) {
                        callback(res);
                    }
                });
            }
        });
    });
});

Comment: De fato funciona, mas eu tenho que tratar a lógica no back-end, e não era bem isso que eu queria. Vou conceder a recompensa mas não vou aceitar a resposta.
Answer: Para resolver este problema você vai precisar fazer um fix utilizando o método onInitialize a solução não é bonita, mas resolve essa deficiência da library:
$(this).selectize({
  plugins: ['remove_button'],
  valueField: 'Sigla',
  labelField: 'Nome',
  searchField: ['Sigla', 'Nome'],
  create: false,
  persist: false,
  preload: true,
  initUrl: "/Advogados/PesquisarJson/",
  initData: true,
  load: function(query, callback) {
    $.ajax({
      url: "/Advogados/PesquisarJson/",
      type: 'GET',
      error: function() {
        callback();
      },
      success: function(res) {
        callback(res);
      }
    });
  },
  // Bugfix
  onInitialize: function() {

    var self = this;
    var value = self.getValue();

    $.ajax({
      url: "/Advogados/PesquisarJson/",
      type: 'GET',
      success: function(res) {

        res.forEach(function(existingOption) {
          self.addOption(existingOption);
          self.addItem(existingOption[self.settings.valueField]);
        });

        self.setValue(value);
        self.blur();
      }
    });

  }
});

Comment: Pelo que eu entendi, esse código apenas cuida de como inserir os dados na DropDown, mas ele não troca as opções somente das siglas pelas opções "Sigla - Nome".
Comment: Ele atualiza a exibição por consequência :)
Comment: Então, mas não funcionou. Isso que eu quero dizer.
Comment: @CiganoMorrisonMendez a lógica do felipe "funciona", mas é provavel que você teria que alterar o back-end pra suportar a consulta vazia e por isto talvez não tenha funcionado.
Comment: @GuilhermeNascimento Aqui não funcionou. Seria legal termos um Fiddle pra testar isto.
Comment: @CiganoMorrisonMendez tem razão não funciona, só olhei a lógica, faltou o comparativo entre o getValue e os items no foreach


The Blues Face: Origins

Question: I'm sure plenty of you guys know this situation well: Someone's playing some blues guitar lick, and right as things get real stanky, they make a weird face, seemingly in reaction to the lick they're playing (some seem to be unable to avoid doing this, actually, and still others do it on purpose!). My questions are asking when and how this tradition got started.
There's another question that kind of gets at this a little in its answers, but are there any explanations as to why this occurs?
Do we know whether early blues pioneers did this when they played, or is this a relatively recent development?
This facial twist occurs in other contexts/genres, too, especially during difficult passages. Is there any record of this being observed for other styles in the past? (the western musical tradition has a bunch of recorded history for classical music, so maybe there?)
Comment: Is it the result of trying your best (in the face of distraction) to concentrate on some difficult task? Have you observed it in people doing apparently difficult things other than playing music?
Comment: It's not just musicians, check out the faces athletes' faces at top effort.
Comment: I do exactly that - usually *after* a bum note's been played...
Comment: When playing music where i feel like being one with it, making every feeling that i have talk throug the music, i make the same face like sexual climax (lol). I think this might have to do something with releasing your tensions (physically and psychological) when playing music. This is something which happens - at least to me - automatically. It is very satisfying.
Comment: I think this is something - more or less - normal in playing music. I guess blues - as a very expressive genre - is kind of made to cause this. But i think other genres can cause this aswell.
Comment: But i dont think that this behavior is something that developed over time. I think its natural
Answer: Blues performers do tend to drift into thespianism to "enhance the experience" or some such. So do boy bands and opera singers.
By way of comparison, I find it amusing that Yo-yo Ma and J. duPre move and emote(d) a lot while performing, yet Starker and Feuermann hardly moved or changed facial expressions at all. (Starker famously told his students that any effort put into body motion was effort lost to producing great music.  so there you go).

The Blues Face: Origins


how do I implement a single-word auto complete using Elasticsearch 6

Question: I would like to implement an single word autocomplete using elasticsearch 6. I have seen a fair amount of posts on how to do this using lesser versions however, it seems that autocomplete has changed significantly in the last version.
I am using the standard mapping for autocomplete:
PUT advertising_tins
{
"settings": {
  "analysis": {
    "analyzer": {
      "completion_analyzer": {
        "type": "custom",
        "filter": [
          "lowercase",
          "completion_filter"
        ],
        "tokenizer": "keyword"
      }
    },
  "filter": {
    "completion_filter": {
      "type": "edge_ngram",
      "min_gram": 1,
      "max_gram": 24
    }
  }
}
  },
  "mappings": {
"item": {
  "properties": {
    "date": {
      "type": "long"
    },
    "id": {
      "type": "text"
    },

    "title": {
      "type": "text"
    },
    "suggest": {
      "type": "text",
      "fields": {
        "raw": {
          "type": "keyword"
        },
        "completion": {
          "type": "text",
          "analyzer": "completion_analyzer",
          "search_analyzer": "standard"
        }
      }
    }
  }
}
}
}

I am indexing like this:
POST advertising_tins/item/_bulk
{"index":{}}
{"date": 20180217, "title": "Vintage Spice Cardboard Tin of Mace  Dainty Brand St. Paul, MN  1 oz.","id": "305232814","suggest": [ "spice","cardboard","tin","mace","dainty","brand","st","paul","mn","oz"]}

And querying like this:
POST advertising_tins/_search?pretty
 {
  "size": 0,
   "query": {
     "term": {
       "suggest.completion": "car"
     }
   },
   "aggs": {
     "suggestions": {
       "terms": {
         "field": "suggest.raw"
       }
     }
   }
}

However my results return all terms in the suggest field instead of just single term "cardboard".
{
  "took": 4,
  "hits": {
    "total": 1,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "suggestions": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
    {
      "key": "brand",
      "doc_count": 1
    },
    {
      "key": "cardboard",
      "doc_count": 1
    },
    {
      "key": "dainty",
      "doc_count": 1
    },
    {
      "key": "mace",
      "doc_count": 1
    },
    {
      "key": "mn",
      "doc_count": 1
    },
    {
      "key": "oz",
      "doc_count": 1
    },
    {
      "key": "paul",
      "doc_count": 1
    },
    {
      "key": "spice",
      "doc_count": 1
    },
    {
      "key": "st",
      "doc_count": 1
    },
    {
      "key": "tin",
      "doc_count": 1
    }
  ]
}
 }
}

And idea how I fix this and get just a single term match?
Answer: You are almost there. It can be achieved with the default Completion Suggester, you only need to change the type of your completion field to "completion":
"mappings": {
    "item": {
      "properties": {
        "date": {
          "type": "long"
        },
        "id": {
          "type": "text"
        },
        "title": {
          "type": "text"
        },
        "suggest": {
          "type": "text",
          "fields": {
            "raw": {
              "type": "keyword"
            },
            "completion": {
              "type" : "completion", <--- here
              "analyzer": "completion_analyzer",
              "search_analyzer": "standard"
            }
          }
        }
      }
    }
  }

And add a "suggest" part into your query:
POST advertising_tins/_search
 {
  "size": 0,
   "query": {
     "term": {
       "suggest.completion": "car"
     }
   },
   "suggest" : {   <--- Here goes he suggest query
    "my-suggestion" : {
      "text" : "car",
      "completion" : {
        "field" : "suggest.completion"
      }
    }
  },
   "aggs": {
     "suggestions": {
       "terms": {
         "field": "suggest.raw"
       }
     }
   }
}

The response will look like this:
{
  // ...
  "hits": //... ,
  "aggregations": // ...,
  "suggest": {
    "my-suggestion": [
      {
        "text": "car",
        "offset": 0,
        "length": 3,
        "options": [
          {
            "text": "cardboard",   <--- here is the suggestion
            "_index": "advertising_tins",
            "_type": "item",
            "_id": "GLeUqGEBVrFe7u7pR5uA",
            "_score": 1,
            "_source": {
              "date": 20180217,
              "title": "Vintage Spice Cardboard Tin of Mace  Dainty Brand St. Paul, MN  1 oz.",
              "id": "305232814",
              "suggest": [
                "spice",
                "cardboard",
                "tin",
                "mace",
                "dainty",
                "brand",
                "st",
                "paul",
                "mn",
                "oz"
              ]
            }
          }
        ]
      }
    ]
  }
}

The response also includes the _source of the suggested document, so you might not even need to use "query" and "aggs" parts.
Hope that helps!


How to use is_transparent feature on a string literal with unordered_map from std::string?

Question: Looking around on cppreference, I found that std::unordered_map gets efficient lookup functions from "equivalent keys".
I take that to mean that the equivalent key must have the same hash value. How can I provide that for a string literal I get the same hash value as for std::hash<std::string> without temporarily constructing an std::string and thereby making the whole point about the equivalent keys for naught?
Comment: Are we sure that `std::hash` is getting this support in C++14?  The paper you linked only shows it on the associative containers, not the unordered ones.  cppreference, for all its benefits, also makes heavy use of internal templates to reduce duplication.  It appears that both `map` and `unordered_map` are using the same description block for `find`, which explains the use of `Compare::is_transparent` in the description.
Comment: @PlasmaHH ahh i see. something like this: http://coliru.stacked-crooked.com/a/f80fefd2c6c51c2e . Unfortunately, gcc and clang don't seem to support it yet :/
Comment: At first glance, I see 2 issues to overcome: 1) find (and the other `std::unordered_map` lookup functions) all take `Key const&`, which will create a temporary pretty much immediately.  2) There is no `std::hash` specialization for string literals.   I suspect the best answer is to write your own container, which is similar to `unordered_map`, but allows to search by string literals, and uses a custom hash function which can handle `std::string` (stored values) as well as `const char*` for lookups.
Comment: @DaveS so how come that cppreference says that `unordered_map` starting with C++14 has an overload for `find` that takes an "equivalent key"? I'm confused. The paper at http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3657.htm that I thought introduced `is_transparent` doesn't seem to introduce it for hash function objects. Also what is `Compare` that the cppreference talks about? I have the feeling I'm missing something important. I just find it smelly that even in C++14 I still have to write my own container for such simple concerns :(
Comment: Yes, the idea was that you write your own hash function that covers all the equivalent types. For this use case I always had some "pair" in mind anyways. So don't use an std::hash, but your own that has an op() for both types.
Comment: Sorry, I missed the C++14 tag, and hadn't seen that paper.  I should know better than to try to respond to questions right after I wake up.
Comment: @PlasmaHH I would be glad if you could provide an answer on how to use this feature. I wonder if it is implemented in libc++ or libstdc++ anyway. AFAIK, C++17 will come with `std::string_view`, so then they could add a `hash` and make `hash` rely on it.
Comment: @JohannesSchaub-litb: Yeah, that string view stuff would be nice. I can not really comment on how the actual standard wording came out, I can just say what the intention was: make your own comparator and hash objects (the comparator marked, by having some is_transparent typedef, what type is unimportant) both of which have op() overloaded for (in your case) taking string and charptr.
Comment: This answer sounds relevant: http://stackoverflow.com/a/20318064/453271
Comment: @DaveS Hmm, interesting point, given that `unordered_map` doesn't have an actual `Compare` while `map` does.
Comment: @DaveS ah i see. these days i avoid looking into the stdlib spec, because i feel like being lost in a big forest. too bad i was possibly tricked by cppreference. I would be glad if someone provided an answer showing that this is indeed the case.
Comment: @DaveS you guessed the root cause correctly, by the way. I used the wrong wiki template initially.
Answer: That was an error in cppreference; there are no templated finds for unordered associated containers.
Compare, from n3690,
from §188.8.131.52[unord.map.overview]
// lookup
iterator find(const key_type& k);
const_iterator find(const key_type& k) const;
size_type count(const key_type& k) const;

from §18.104.22.168[map.overview]
// 18.104.22.168, map operations:
iterator find(const key_type& x);
const_iterator find(const key_type& x) const;
template <class K> iterator find(const K& x);
template <class K> const_iterator find(const K& x) const;
size_type count(const key_type& x) const;

Comment: It would be nice if they added unorded associative containers as well.  It would require a transparent `KeyEqual` function, but also a `Hash` that supports both the Key type and the input to a templated `find`.  I think specifying the latter would be a bit trickier.
Comment: @DaveS for `unordered_set< T, Hash, Equal >`, iff `Hash::is_transparent` add `unordered_set< T, Hash, Equal >::find(const K& x)` that does a `Hash(x)` then `Equal( y, x )` where `y` is the element(s) found by the `Hash(x)`?  Now, `x` has to be `Equal`-able (either by conversion or transparency), but...
Answer: Heterogeneous lookup for unordered containers has been adopted for C++20 (p0919, p1690). It's already available in gcc 11.0 and clang 12.0.
Comment: still mostly useless since no support in std:hash
Answer: As others said, unordered associative containers don't support the is_transparent mode. Boost.MultiIndex hashed indices, on the other hand, allow for what you want, as explained in the documentation, in case it is an option for you to replace std::unordered_map with an equivalent construct based on a multi_index_container.


Fauna Search By Ref

Question: I have two documents in the products collection...
{
  "ref": Ref(Collection("products"), "300137558676865540"),
  "ts": 1622492331145000,
  "data": {
    "product_id": 1004,
    "display_name": "Product By ABC",
    "description": "Product Description ABC",
    "status": "in_stock",
    "price_current": 100,
    "supplier": Ref(Collection("suppliers"), "300137504766427654")
  }
},
{
  "ref": Ref(Collection("products"), "300137592998855170"),
  "ts": 1622492386360000,
  "data": {
    "product_id": 1005,
    "display_name": "Product By XYZ",
    "description": "Product Description XYZ",
    "status": "in_stock",
    "price_current": 150,
    "supplier": Ref(Collection("suppliers"), "300137513423471107")
  }
}

then I have also two documents in my suppliers collection...
{
  "ref": Ref(Collection("suppliers"), "300137504766427654"),
  "ts": 1622492279715000,
  "data": {
    "supplier_id": 205,
    "display_name": "Test Supplier ABC"
  }
},
{
  "ref": Ref(Collection("suppliers"), "300137513423471107"),
  "ts": 1622492287963000,
  "data": {
    "supplier_id": 206,
    "display_name": "Test Supplier XYZ"
  }
}

How do I search the products collection by the supplier reference field? Any help is appreciated!  Thanks!
Answer: All searching in Fauna is done using indexes. You need to create an index for your "products" collection, specifying "supplier" as the search field (aka the terms field in the index):
CreateIndex({
  name: "products_by_supplier",
  source: Collection("products"),
  terms: [{ field: ["data", "supplier"]}]
})

Then you can find all products where the supplier field matches the reference for a specific supplier:
> Paginate(
  Match(
    Index("products_by_supplier"),
    Ref(Collection("suppliers"), "300137504766427654")
  )
)
{ data: [ Ref(Collection("products"), "300137558676865540") ] }

If you want to verify that the results are correct, you can iterate over the results to fetch the associated product documents:
> Map(
  Paginate(
    Match(
      Index("products_by_supplier"),
      Ref(Collection("suppliers"), "300137504766427654")
    )
  ),
  Lambda("X", Get(Var("X")))
)
{
  data: [
    {
      ref: Ref(Collection("products"), "300137558676865540"),
      ts: 1622501576960000,
      data: {
        product_id: 1004,
        display_name: 'Product By ABC',
        description: 'Product Description ABC',
        status: 'in_stock',
        price_current: 100,
        supplier: Ref(Collection("suppliers"), "300137504766427654")
      }
    }
  ]
}

Note that the ts field in my result differs from yours, because I just created those documents.
See the search tutorial for more details: https://docs.fauna.com/fauna/current/tutorials/indexes/search

Fauna Search By Ref


Add new field to node form on validate

Question: I am performing a hook_node_validate() and would like to add a new field (e.g. a textfield) during the processing of this function.  Is this possible?  If so could someone let me know how to do it.  I am tried to following so far w/ no luck:
<code>function hook_node_validate($node, $form, &$form_state) {
    $form['content_type_node_form'] = array(
        '#title'    => t('Phone'),
        '#type'     => 'textfield',
        '#required' => TRUE,
    );
    form_set_error('time', t('Stop submission to add new field'));
}
</code>
Thanks,
Derek
Comment: to understand: does the field already exist in the content type? or are you trying to add a field to the content type while validating a node of that type?
Comment: Hi Geoff.  The field doesn't already exist.  I am trying to add a field to the content type while validating a node of that type. Thanks ~D
Answer: I'm guessing what you actually want (from a user story perspective) is if a certain condition is met another, otherwise hidden, field is shown. As @geoff points out that field would need to exist on the content type.
I believe the correct way to do this within the Form API is with the #states parameter in order to conditionally show/hide the phone field.
https://api.drupal.org/api/drupal/developer!topics!forms_api_reference.html/7#states

Add new field to node form on validate


Ship to multiple addresses on single order

Question: We have below requirement,
We can add multiple products to single order(which is normal).
Our client expect a functionality to ship to multiple address in single order. Which means we should have option to add multiple address in single order and based on client's need, different products should ship to different addresses.
Any one has any experience or idea about the implementation? Or we have to write our custom module for the functionality?
Answer: This is one of those tasks that take a lot more time on Drupal 7 than Drupal 8, since the Drupal 7 version of Shipping doesn't have native support for multiple shipments.
Your best bet is investigating the Commerce Shipment and Commerce Fulfillment modules, choosing one of them, and then coding the rest.


AttributeError: 'Operation' object has no attribute '_graph' in tf.keras.models.save_model()

Question: I have what would appear to be a pretty straightforward model fitting process in a Jupyter notebook:
model = tf.keras.Sequential([
  feature_layer,
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(7, activation='softmax')
])
model.compile(loss='sparse_categorical_crossentropy',
              optimizer=tf.keras.optimizers.Adamax(),
              metrics=['accuracy'])
model.fit(train_ds, validation_data=val_ds,
          epochs=1, callbacks=[tf.keras.callbacks.EarlyStopping(monitor='accuracy', patience=10)])

When fitting is complete, I call tf.keras.models.save_model(model) (I have also tried model.save(path)), which saves successfully only if the notebook kernel has been loaded from scratch, otherwise it fails. For example if I run:
tf.keras.backend.clear_session()
tf.compat.v1.reset_default_graph()
del(model)

And try to execute the first block of code again, without reseting the kernel, I will get:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-29-d8fda83a42df> in <module>
----> 1 model.save('test3')

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/keras/engine/network.py in save(self, filepath, overwrite, include_optimizer, save_format, signatures, options)
   1006     """
   1007     save.save_model(self, filepath, overwrite, include_optimizer, save_format,
-> 1008                     signatures, options)
   1009
   1010   def save_weights(self, filepath, overwrite=True, save_format=None):

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/keras/saving/save.py in save_model(model, filepath, overwrite, include_optimizer, save_format, signatures, options)
    113   else:
    114     saved_model_save.save(model, filepath, overwrite, include_optimizer,
--> 115                           signatures, options)
    116
    117

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/keras/saving/saved_model/save.py in save(model, filepath, overwrite, include_optimizer, signatures, options)
     76     # we use the default replica context here.
     77     with distribution_strategy_context._get_default_replica_context():  # pylint: disable=protected-access
---> 78       save_lib.save(model, filepath, signatures, options)
     79
     80   if not include_optimizer:

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/saved_model/save.py in save(obj, export_dir, signatures, options)
    897   # Note we run this twice since, while constructing the view the first time
    898   # there can be side effects of creating variables.
--> 899   _ = _SaveableView(checkpoint_graph_view)
    900   saveable_view = _SaveableView(checkpoint_graph_view)
    901

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/saved_model/save.py in __init__(self, checkpoint_view)
    185           #  variables on first run.
    186           concrete_functions = (
--> 187               function._list_all_concrete_functions_for_serialization())  # pylint: disable=protected-access
    188         else:
    189           concrete_functions = [function]

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/def_function.py in _list_all_concrete_functions_for_serialization(self)
    797     """
    798     if self.input_signature is not None:
--> 799       self.get_concrete_function()
    800     concrete_functions = []
    801     # pylint: disable=protected-access

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/def_function.py in get_concrete_function(self, *args, **kwargs)
    907       if self._stateful_fn is None:
    908         initializers = []
--> 909         self._initialize(args, kwargs, add_initializers_to=initializers)
    910         self._initialize_uninitialized_variables(initializers)
    911

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
    495     self._concrete_stateful_fn = (
    496         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 497             *args, **kwds))
    498
    499     def invalid_creator_scope(*unused_args, **unused_kwds):

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2387       args, kwargs = None, None
   2388     with self._lock:
-> 2389       graph_function, _, _ = self._maybe_define_function(args, kwargs)
   2390     return graph_function
   2391

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   2701
   2702       self._function_cache.missed.add(call_context_key)
-> 2703       graph_function = self._create_graph_function(args, kwargs)
   2704       self._function_cache.primary[cache_key] = graph_function
   2705       return graph_function, args, kwargs

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   2591             arg_names=arg_names,
   2592             override_flat_arg_shapes=override_flat_arg_shapes,
-> 2593             capture_by_value=self._capture_by_value),
   2594         self._function_attributes,
   2595         # Tell the ConcreteFunction to clean up its graph once it goes out of

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    976                                           converted_func)
    977
--> 978       func_outputs = python_func(*func_args, **func_kwargs)
    979
    980       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/eager/def_function.py in wrapped_fn(*args, **kwds)
    437         # __wrapped__ allows AutoGraph to swap in a converted function. We give
    438         # the function a weak reference to itself to avoid a reference cycle.
--> 439         return weak_wrapped_fn().__wrapped__(*args, **kwds)
    440     weak_wrapped_fn = weakref.ref(wrapped_fn)
    441

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/training/tracking/tracking.py in _initializer()
    242     @def_function.function(input_signature=[], autograph=False)
    243     def _initializer():
--> 244       self._initialize()
    245       return 1  # Dummy return
    246

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/lookup_ops.py in _initialize(self)
    179
    180   def _initialize(self):
--> 181     return self._initializer.initialize(self)
    182
    183   @property

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/lookup_ops.py in initialize(self, table)
    462         self._name, values=(table.resource_handle, self._keys, self._values)):
    463       init_op = gen_lookup_ops.lookup_table_import_v2(table.resource_handle,
--> 464                                                       self._keys, self._values)
    465     ops.add_to_collection(ops.GraphKeys.TABLE_INITIALIZERS, init_op)
    466     return init_op

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/ops/gen_lookup_ops.py in lookup_table_import_v2(table_handle, keys, values, name)
    700   _, _, _op, _outputs = _op_def_library._apply_op_helper(
    701         "LookupTableImportV2", table_handle=table_handle, keys=keys,
--> 702                                values=values, name=name)
    703   return _op
    704 LookupTableImportV2 = tf_export("raw_ops.LookupTableImportV2")(_ops.to_raw_op(lookup_table_import_v2))

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/op_def_library.py in _apply_op_helper(op_type_name, name, **keywords)
    740       op = g._create_op_internal(op_type_name, inputs, dtypes=None,
    741                                  name=scope, input_types=input_types,
--> 742                                  attrs=attr_protos, op_def=op_def)
    743
    744     # `outputs` is returned as a separate return value so that the output

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/func_graph.py in _create_op_internal(self, op_type, inputs, dtypes, input_types, name, attrs, op_def, compute_device)
    589       if ctxt is not None and hasattr(ctxt, "AddValue"):
    590         inp = ctxt.AddValue(inp)
--> 591       inp = self.capture(inp)
    592       inputs[i] = inp
    593     return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/func_graph.py in capture(self, tensor, name, shape)
    628       # Large EagerTensors and resources are captured with Placeholder ops
    629       return self._capture_helper(tensor, name, shape)
--> 630     if tensor.graph is not self:
    631       if name is None:
    632         name = tensor.op.name

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py in graph(self)
    414   def graph(self):
    415     """The `Graph` that contains this tensor."""
--> 416     return self._op.graph
    417
    418   @property

/usr/local/lib/python3.5/dist-packages/tensorflow_core/python/framework/ops.py in graph(self)
   2223   def graph(self):
   2224     """The `Graph` that contains this operation."""
-> 2225     return self._graph
   2226
   2227   @property

AttributeError: 'Operation' object has no attribute '_graph'

Any help would be appreciated.
Comment: Hi, I am experiencing a very similar issue. Any updates on a possible solution?
Comment: Can confirm with an anaconda tf.2.1 env
Answer: I have recreated your code and successfully ran it without any errors. Kindly use the latest stable release of TensorFlow in this case 2.1.0.
If the problem still persists, try updating your Jupyter Notebook, Python (to the latest supported version), and or try simulating it first in Google Colab to ensure that this is not related to any hardware issues.
Screenshot of Successful execution:

Here is my code:
import tensorflow as tf
print('TensorFlow Version: {}'.format(tf.__version__))
# Synthetic Data
x, y =  tf.random.uniform(shape=(1000, 7, 1), maxval = 1), tf.random.uniform(shape=(1000, 7,1), maxval =1)

# Model
feature_layer = tf.keras.layers.Input(shape = (7,1))

model = tf.keras.models.Sequential([
  feature_layer,
  tf.keras.layers.Dense(128, activation='relu', input_shape = (7,1)),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(7, activation='softmax')
])
model.compile(loss='sparse_categorical_crossentropy', optimizer=tf.keras.optimizers.Adamax())
model.fit(x, y, epochs=1)

# 1st Save
first_model_loc = '/tmp/tf_keras_1st_run/'
tf.keras.models.save_model(model, filepath= first_model_loc)
print('\nSuccessfully saved at {}\n\n'.format(first_model_loc)) # Progress Checking

#
tf.keras.backend.clear_session()
tf.compat.v1.reset_default_graph()
del(model)
#

# Model
feature_layer = tf.keras.layers.Input(shape = (7,1))

model = tf.keras.models.Sequential([
  feature_layer,
  tf.keras.layers.Dense(128, activation='relu', input_shape = (7,1)),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dropout(0.5),
  tf.keras.layers.Dense(7, activation='softmax')
])
model.compile(loss='sparse_categorical_crossentropy', optimizer=tf.keras.optimizers.Adamax())
model.fit(x, y, epochs=1)

# 2nd Save
second_model_loc = '/tmp/tf_keras_2nd_run/'
tf.keras.models.save_model(model, filepath=second_model_loc)
print('\nSuccessfully saved at {}'.format(second_model_loc)) # Progress Checking


Why isn't the default template used?

Question: I set Mime Mail as the sitewide default mailing system and overrode mimemail-message.tpl.php. My custom template is used correctly for e-mails like account activation, but when I send an HTML e-mail with Rules, the template isn't loaded: I simply get an unstyled HTML text.
Did I forget something (e.g. a configuration for Rules)?
Answer: Is your administration theme the active theme when the Rule fires? This documentation page about using a custom mail template with Mime Mail might be referencing the issue:

If you use sub-themes or multiple themes, especially if you use a different administration theme than your default theme, you might end up with a different theme to what you expect.

In the Mail System module configuration settings (admin/config/system/mailsystem), try changing the Theme to render the emails from Current to the Default theme or to the theme where your overridden mimemail-message.tpl.php lives.
Comment: Of course! Subtle difference between *Current* and *Default*...

Why isn't the default template used?


Using mle2() for age-period-cohort models

Question: I've followed guidelines for comparing models in Chapter 6 of Bolker's Ecological Models and Data in R, applying code used in this section to cancer count data. The models include parameters for age group (a) and birth cohort (c), and are as follows:
# Value of lambda is the same for all age groups and birth cohorts
poisfit.0 <- mle2(counts ~ dpois(lambda = a*c*pyrs), start=list(a=3,c=0.7),data=x)

Warning message:
In dpois(x, lambda, log) : NaNs produced
Coefficients:
   Estimate Std. Error z value     Pr(z)
a 3.1932757  0.0022079 1446.28 < 2.2e-16
c 0.8429316  0.0083643  100.78 < 2.2e-16
-2 log L: 17550.32
# Use coefficients from simplest model to build more complex models
start.ab <- coef(poisfit.0)

# Model with varying parameter values for age groups and birth cohorts
poisfit.ab = mle2(counts ~ dpois(lambda = a*c*pyrs), start = start.ab, data = x, parameters=list(a~x\$ageGroup, c~x\$bCohort))

# Same as above, but without intercept
poisfit.ab1 = mle2(counts ~ dpois(lambda = a*c*pyrs), start = start.ab, data = x, parameters=list(a~x\$ageGroup-1, c~x$bCohort-1))

The data look like this, and go on for 110 rows (14 age groups, 16 birth cohorts):
ageGroup bCohort pyrs   counts
1    13      1.4006      0
1    14      32.6925     1
1    15      49.7632     1
1    16      49.7059     0
2    12      1.2528      0
2    13      30.3879     3
2    14      47.7709     1
2    15      52.3699     0
2    16      55.7669     2
3    11      1.0257      0
A sample of coefficients for the model lacking intercept:
Estimate    Std.    Error   z   value   Pr(z)
a.x\$ageGroup1  0.101115    0.072032    1.4037  0.1603938
a.x\$ageGroup2  0.189811    0.078609    2.4146  0.0157517
a.x\$ageGroup3  0.335937    0.086470    3.8850  0.0001023
a.x\$ageGroup4  0.760921    0.117493    6.4763  9.400e-11
a.x\$ageGroup5  1.282271    0.141785    9.0438  <   2.2e-16
a.x\$ageGroup6  2.221877    0.180402    12.3162 <   2.2e-16
a.x\$ageGroup7  3.694786    0.227830    16.2173 <   2.2e-16
a.x\$ageGroup8  6.629915    0.313676    21.1362 <   2.2e-16
a.x\$ageGroup9  10.103253   0.400364    25.2351 <   2.2e-16
a.x\$ageGroup10 16.198128   0.537537    30.1340 <   2.2e-16
a.x\$ageGroup11 24.141987   0.686824    35.1502 <   2.2e-16
a.x\$ageGroup12 32.166317   0.811856    39.6207 <   2.2e-16
a.x\$ageGroup13 39.877367   0.941045    42.3756 <   2.2e-16
a.x\$ageGroup14 36.982237   1.169403    31.6249 <   2.2e-16
c.x\$bCohort1   0.058183    0.026073    2.2315  0.0256476
c.x\$bCohort2   0.135923    0.023366    5.8171  5.988e-09

There are two questions I have about this model:

The age group parameters blow out of proportion starting with age group 8. Why is this happening? Where did I go wrong?
The starting parameter values in the simple model poisfit.0 are both positive. They must both have the same sign in order for mle2() to accept them. If one is positive and the other negative, mle2() issues this error:

Error in optim(par = c(-0.2, 3.4), fn = function (p)  :
  initial value in 'vmmin' is not finite
I don't know enough about what goes on under the hood to understand why this happens. I'd appreciate insight into these questions.
Thank you,
Susana
Answer: There are several important problems here.

Are you sure you can't just use Poisson regression (i.e. glm(counts~ageGroup+bCohort,data=x,family=poisson) (i.e. see ch 9 of Bolker 2008, or any other book on GLMs, such as Faraway 2006)?
In your first model, a and c are jointly unidentifiable -- for example, a=1, c=2 would have exactly the same likelihood as a=2, c=1 (or any other combination with a*c=2)
you should probably use (a~ageGroup c~bCohort) instead of including the x$
I don't think the parameters are "blowing up", it's just that there is a progressive trend through age (you should seriously consider a model that treats age group as a continuous predictor)
the problem you refer to with the starting values is that you can't start with parameters that make the lambda value negative.  (mle2, or rather most of the underlying optimizers in optim, can handle parameters that return NA for the negative log-likelihood during the course of the optimization, but not right at the beginning of the optimization).  You could fit lambda on the log scale (i.e. loglambda=loga+logc), or use bounded optimization.


GradientBoostClassifier(sklearn) takes very long time to train

Question: I'm using dataset with 61879 datapoints and 102 features. On this dataset Randomforest(sklearn) takes less than 90s to train for 100 estimators while GradientBoostClassifier(sklearn) is taking forever to train using same numbers of estimators. Is there any way to speedup the training process for GradientBoostClassifier?
Answer: Might be a bit late... But.
1 - sklearn's Random Forest supports multithreading. GradientBoostingClassifier does not. This can be responsible for a 8 times speed up.
2 - sklearn's Random Forest works on a subset of the total number of features (at least, by default) whereas GradientBoostingClassifier uses all the features to grow each each tree.
If you set the argument max_features for GBC, you can observe a huge speed-up (but different results). From sklearn documentation :
<code>max_features : int, float, string or None, optional (default=None)
The number of features to consider when looking for the best split:
If int, then consider max_features features at each split.
If float, then max_features is a percentage and int(max_features *
n_features) features are considered at each split.
If "auto", then max_features=sqrt(n_features).
If "sqrt", then max_features=sqrt(n_features).
If "log2", then max_features=log2(n_features).
If None, then max_features=n_features.

Choosing max_features < n_features leads to a reduction of variance and
an increase in bias.
</code>
Option 2 is a matter of choice/performance. As for option 1, an implementation of GBC supporting multithreading is now available: xgboost, https://github.com/dmlc/xgboost. I used it with R, but the python implementation seems even easier to use.
Edit. Regarding the training time of various algorithm, you may be interested in learning more about complexities of machine learning methods.


On routerLinkActive="active" change the color of child mat-icon inside button

Question: I have a header on my webpage with an item list and every item has a mat-icon child in it. Below is one of the items.

   <button routerLink="['/admin']" routerLinkActive="active">
       <mat-icon color="warn">history</mat-icon>Audit
   </button>

The CSS on routerLinkActive="active" works fine on the button text, but I also want to change the color of mat-icon, but nothing seems to work
Comment: Try something like the following: [color]="condition ? 'primary' : 'warn'"
Answer: The RouterLinkActive exports a variable to the template, and has a property that tells you what the active state currently is.
https://angular.io/api/router/RouterLinkActive#properties
https://github.com/angular/angular/blob/ead169a4022e2a2f5520e1c1a58f30692b90f7cd/packages/router/src/directives/router_link_active.ts#L77
   <button routerLink="['/admin']" routerLinkActive="active" #rla="routerLinkActive">
      <mat-icon [color]="rla.isActive && 'warn'">history</mat-icon>Audit
  </button>

The above will set the color to warn if the property isActive of the routerLinkActive is true.
Answer: Easiest way is to add this in your CSS -- probably best is to add it globally:
.active .mat-icon{
   color: inherit;
}

Comment: Thanks, please do select it as the best answer if you believe it was.
Answer: Looking at this answer, you could do something like this

   <button routerLink="['/admin']" routerLinkActive="active" #rla="routerLinkActive">
       <mat-icon [ngClass]="{'color': rla.isActive ? 'warn' : 'primary'}">history</mat-icon>Audit
   </button>

or this

   <button routerLink="['/admin']" routerLinkActive="active" #rla="routerLinkActive">
       <mat-icon [ngStyle]="{'color': rla.isActive ? 'warn' : 'primary'}">history</mat-icon>Audit
   </button>

note that these answers are untested
Comment: `active` is *not a template* variable. It is the string to assign to the `class` attribute of the button.
Comment: Should the color change dynamically?
Comment: @al.koval assuming that `active` is a boolean, this should change dynamically
Comment: @Reactgular thanks for the clarity, I haven't used active router before. Using this [link](https://angular.io/api/router/RouterLinkActive) I updated my answer
Answer:

                group
                Clients

> add this sytle in global css

.active i {
        color: #28A745;
    }
    .active span {
        color: #28A745;
    }


Is my hypothesis testing approach correct?

Question: I have one dataset which was tested under different programming languages / versions of the same algorithm. I consider them to be the treatments.
I want to test the effect of the treatments, so I collected the runtime obtained from using a treatment with the same dataset having either 50k, 100k or 1M rows, each size was processed 5 times with each treatment (same dataset same treatment 5 times), except in some cases when the bigger dataset was tested only once per treatment. This makes my data to be paired (dependent).
I selected two tests, that can be applied to non-parametric, dependent data: Wilcoxon signed rank for two samples, and Friedman's test for more than two samples.
For Wilcoxon signed rank test I plan to do it this way:

a) Would that be ok to use on MatLab:
<code>[p,h] = signrank(java,python)
</code>
b) Does it affect in anything to have 5 runtimes for the exact same dataset, then another 5 for a bigger dataset with the same information except for the IDs (e.g. it continues from ID 50 000, to 50 001, 50 0002, etc.), then 1 runtime for the 1M dataset? or is it ok to give MatLab two vectors, each one formed by the data in columns?
For Friedman's test I plan to do it this way:

c) Would that be ok to use on MatLab:
<code>p = friedman(data,5)
</code>
where data is the whole matrix shown in the picture, and 5 because of the repetition of 5 runtimes per dataset size.
d) How can I address the repetition on a two-sample testing?
Final note is to say that the results of each test are going to be used separately, on an isolated way.
I struggled to reach the point in which I think I know which hypothesis testings are appropriate to the data I have, but still have some doubts on its application. Any help and comment is very much appreciated.
Comment: First, the differences between the runtimes are so large that you hardly need any test!  Second, I don't think this is a paired design, as it is difficult to see to which commom "experimental sybjects" the treatments are applied, so you could use the independent groups test.
Comment: .... if the different runs was done on different machines, with very diferent payloads, or something such, then maybe this could be a paired comparison. But nothing you told us indicates so.
Answer: First, the runtimes are so different (about a factor ten) that there is hardly any need of a formal test!  But there is still much of interest to say ...
You say you have a paired comparison, since the implementations were tested with different datasets (same datasets for each implementation, making it paired).  But then, for some of the datasets, you did multiple (five) runs, making it an independent samples comparison within the paired comparison! Maybe simply those independent runs should be averaged.
Now note that formal hypothesis tests for paired comparisons typically are based on models such as
$$
  y_{i,1} = \mu + \epsilon_{i,1}   \\
  y_{i,2} = \mu + \Delta +\epsilon_{i,2}
$$
where the $\epsilon$'s have some distribution we do not bother to specify,
and $\Delta$ is the difference between treatments.  That is, the tests depend on the asumption that the expected difference between treatments on the pairs are constant over the different experimental conditions indexed by $i$.  That cannot be the case here, since runtimes would be expected to be (close to) proportional when input dataset size varies. That could be treated by a regression model, or more simply by analyzing logarithms! I leave that for you to see.


Google API - Auth server to server

Question: I need a service to manipulate emails in a gmail account within a project developed in PHP Symfony...
I found this example : https://github.com/googleapis/google-api-php-client/blob/main/docs/oauth-server.md
But more confusing than helping...
I wrote this code :
src/Service/Gmail.php
<?php

namespace App\Service;

use Google\Client;

class Gmail
{
    private \Google\Service\Gmail $api;

    private function getGoogleClient(): Client
    {
        $credentialsPath = getenv('GOOGLE_APPLICATION_CREDENTIALS');
        if (empty($credentialsPath)) {
            throw new \Exception('You need to set env var GOOGLE_APPLICATION_CREDENTIALS');
        }

        if (!file_exists($credentialsPath)) {
            throw new \Exception('Credentials file path ' . getenv('GOOGLE_APPLICATION_CREDENTIALS') . ' set in GOOGLE_APPLICATION_CREDENTIALS does not exist');
        }

        $client = new Client();
        $client->useApplicationDefaultCredentials();
        return $client;
    }

    private function getApi(): \Google\Service\Gmail
    {
        if (!isset($this->api)) {
            $this->api = new \Google\Service\Gmail($this->getGoogleClient());
        }

        return $this->api;
    }

    public function getUserMessages($userId): \Google\Service\Gmail\ListMessagesResponse
    {
        return $this->getApi()->users_messages->listUsersMessages($userId);
    }
}

Then I followed the steps described in Google Workspace for developers :

Create a new project : https://developers.google.com/workspace/guides/create-project?hl=en
Enable Gmail API : https://developers.google.com/workspace/guides/enable-apis?hl=en
Create credentials for web application server-side : https://developers.google.com/workspace/guides/create-credentials?hl=en

But at this point I have no idea what type of credentials I need : "OAuth client ID" or a "Service account" ??? If I choose "Oauth client ID" I suppose I have to use the application type "Web application" with server side url ? Or do I choose "Service account" ?
With service account I got this error :
In REST.php line 134:

  {
    "error": {
      "code": 401,
      "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.
  google.com/identity/sign-in/web/devconsole-project.",
      "errors": [
        {
          "message": "Login Required.",
          "domain": "global",
          "reason": "required",
          "location": "Authorization",
          "locationType": "header"
        }
      ],
      "status": "UNAUTHENTICATED",
      "details": [
        {
          "@type": "type.googleapis.com/google.rpc.ErrorInfo",
          "reason": "CREDENTIALS_MISSING",
          "domain": "googleapis.com",
          "metadata": {
            "service": "gmail.googleapis.com",
            "method": "caribou.api.proto.MailboxService.ListMessages"
          }
        }
      ]
    }
  }

Answer: You have not proeprly authorized your service account to delegate to a user on your domain.
require_once('../../vendor/autoload.php');

// Some user within your workspace domain
$user_to_impersonate = "firstname.lastname@example.com";

$sender = $user_to_impersonate;
$to = 'email@example.com';
$subject = 'Hello';
$messageText = 'How are you doing?';

// The path to your service account credentials goes here.
putenv("GOOGLE_APPLICATION_CREDENTIALS=credentials.json");
$client = new Google_Client();
$client->useApplicationDefaultCredentials();
$client->setSubject($sender);
$client->setApplicationName("Quickstart");
$client->setScopes(["https://mail.google.com/"]);
$service = new Google_Service_Gmail($client);

Comment: Thank you for your answer... One step forward !

Now I have the error `Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested.` with the "service account" credential or error `json key is missing the type field` with oauth server-side. Any idea why ?

For the "service account" error I found in stackoverflow a solution you gave about "domain-wide delegation" but I don't have workspace account :/
Comment: you need to create service account credentials you don't need an API key
Comment: I'm not using API key, I tried with the 2 other credentials methods : the service account and oauth server-side credentials...
Like I said, with service account, I have the following error : `Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested`
Comment: You have not properly configured [domain wide deligation](https://developers.google.com/cloud-search/docs/guides/delegation)  the service account needs to be authorized by your workspace admin.

Google API - Auth server to server


Are visualization techniques useful when the predictive model is a highly flexible machine learning algorithm?

Question: I've always used highly flexible machine learning algorithms like boosted trees, support vector machines, and random forests that supposedly excel at identifying non-linear and irregular patterns and variable interactions. I've been building predictive algorithms for years with reasonable success, but I've never really invested much time in visualizing my data -- no scatter plots, charts, or bar graphs.
When working on a new problem, I try to understand my data by reading about the topic for many hours and absorbing other people's subject matter expertise. People have analyzed my data for decades, so there is no shortage of expert opinions. When coming up with new ideas for predictors, am I better off reading more expert opinions because machine learning algorithms will generally "outperform my eyes"?
If I'm going to test the inclusion of each candidate variable based on the performance of out-of-sample data anyway, do visualization techniques like box plots become redundant? If a box plot says that a certain variable is important, wouldn't a boosted trees model or support vector machine reach the same conclusion?
My models are generally boosted trees. Are there certain visualization techniques that would help me see things that boosted trees could not?
Comment: See this page [deltarho](http://deltarho.org/) for effective visualization of complex data!
Answer: Firstly, could you clarify what you mean by 'highly flexible'?
Also, YES YES YES. Always do exploratory data analysis. Feature engineering is what will take your algorithms to the next level. Good EDA/plots will help you figure out good predictors to add.
Comment: By flexible I mean that the models don't require normally distributed data, and that they can uncover variable interactions without the user having to identify the variable interactions explicitly. Boosted trees are particularly good at identifying variable interactions
Comment: Also, if I'm going to test the inclusion of each candidate variable based on the performance of out-of-sample data anyway, do visualization techniques like box plots become redundant? If a box plot says that a certain variable is important, wouldn't a boosted trees model reach the same conclusion?


RxJS Array of urls to GET requests merged

Question: I'm brand new to RxJS and want to do the following (in Angular 2):

Request some JSON from the server containing an array of urls
Make a request to each of those urls
Subscribe to the combined completion of all of those requests

At the moment my method looks like this:
    load():void {
        this.http.get(SVG_LIST)
            .map(res => {
                return <Array<string>> res.json().files;
            })
            .catch(this.handleError);
    }

What do I chain onto the map method?
Answer: You are looking for the equivalent of Promise.all, which is the operator forkJoin. You could try something like :
load():void {
        this.http.get(SVG_LIST)
            .map(res => {
                return <Array<string>> res.json().files;
            })
            .flatMap(files => Rx.Observable.forkJoin(files.map(makeRequest)))
            .catch(this.handleError);
    }

where files is an array of urls, makeRequest is a function which takes an url and returns an observable or a promise.


$T^3-4T^2+4T=0$: inference about $T$

Question:
Suppose $T$ is a linear operator on $\mathbb R^2$ such that $T^3-4T^2+4T=\theta$ where $\theta$ is the null transformation. Then, describe $T$, given that $T$ is diagonalizable.

My approach:

$T(T^2-4T+I)=\theta$ implies $T(T-2I)^2=\theta$. Thus $T$ may be the null transformation itself, and it satisfies the condition of diagonalizability.
Consider $f(t)=(t-2)^2$. Then $f$ may be the characteristic polynomial of $T$. Since $T$ is diagonalizable, eigenvalue of $T$ is $2$ and there are two linearly independent eigenvectors of $T$ corresponding to $2$.
Consider $f(t)=t(t-2)$ then $f$ may be the characteristic polynomial of $T$. The eigenvalues are correspondingly $0$ and $2$ and because of diagonalizability, there is one independent eigenvector for $0$ and one eigenvector for $2$.

Is it right?
Comment: It could also have two eigenvalues which are $0$.  All you can deduce is that the diagonal entries have to be picked from $0$ or $2$.
Comment: If it is the null transformation (which I mentioned) then it must have eigenvalues $0$. I think I took care of this case in that sentence. Thanks for your comment.
Comment: Sorry, I missed that.  Then it looks correct.
Comment: Could you kindly check my answer in https://www.google.co.in/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCUQFjAB&url=http%3A%2F%2Fmath.stackexchange.com%2Fquestions%2F1154251%2Fshow-t-is-diagonalizable-if-t-lambda-i-is-idempotent&ei=VrbmVKfkBpaTuASs6YBg&usg=AFQjCNFABHP255cjguNTg4psakJ_2ho78A ?? Please? I have posted a solution but nobody has verified it yet.
Answer: Your answer looks good, but there are a few missing details (which your solution leads me to believe you know, but left assumed, and which a peer reading the solution might not be able to fill in).  Here is a slightly more complete write up, which is clearer but uses essentially the same logic.
Since $T$ is a linear operator on a two dimensional space, its characteristic polynomial is of degree two, and so it's minimal polynomial is of degree at most two and dividing $f(t)=t^3-4t^2+4t=t(t-2)^2$.  Furthermore, because $T$ is diagonalizable, the minimal polynomial will not have any repeated roots, and is therefore equal to one of $t$, $t-2$, or $t(t-2)$.  Note, if $T$ were not necessarily diagonalizable, then the minimal polynomial could also be $(t-2)^2$.
If the minimal polynomial of $T$ is t, then $T=0$, and if it is $t-2$, then $T=2I$.  If the minimal polynomial is $t(t-2)$, then $0$ and $2$ are both eigenvalues of $T$, and since $T$ is a transform of a two dimensional space, each of these eigenvalues have multiplicity one.
Answer: If you plug in $T=\operatorname{diag}(\lambda_1,\lambda_2)$ then obviously you have
$$
\lambda_i^3-4\lambda_i^2+4\lambda_i=0
$$
so $\lambda_i \in \{0,2\}$.
If $\operatorname{rk}(T)=0$, then $T=0$ (trivial, or $\lambda_1=\lambda_2=0$)
If $\operatorname{rk}(T)=1$, then $T$ is projection on a one-dim subspace and multiplication by $2$
If $\operatorname{rk}(T)=2$, then $T$ is $2I$ (because $\lambda_1=\lambda_2=2$)

$T^3-4T^2+4T=0$: inference about $T$