Why does arp_ignore=1 break ARP on POINTOPOINT interfaces? (kvm guest)

Question: If I add the following settings on a kvm guest:
net.ipv4.conf.all.arp_ignore=1

Networking of the guest stops (highly intermittent).
The kvm host uses bridge-networking:
auto lo
iface lo inet loopback
iface lo inet6 loopback

auto {{ vbox_interface }}
iface {{ vbox_interface }} inet static
  address {{ vbox_main_ipv4 }}
  netmask {{ vbox_ipv4_netmask }}
  gateway {{ vbox_ipv4_gw }}
  pointopoint {{ vbox_ipv4_gw }}

iface {{ vbox_interface }} inet6 static
  address {{ vbox_main_ipv6 }}
  netmask 128
  gateway {{ vbox_ipv6_gw }}

auto  br0
iface br0 inet static
 address   {{ vbox_main_ipv4 }}
 netmask   255.255.255.255
 bridge_ports none
 bridge_stp off
 bridge_fd 0
 bridge_maxwait 0
{% for ip in vbox_addtl_ipv4 %}
 up route add -host {{ ip }} dev br0
{% endfor %}

iface br0 inet6 static
  address {{ vbox_main_ipv6 }}
  netmask 64

Why is that?
As Michael is pointing out in the comments, some context:

I do understand the basic gist of arp_ignore (http://kb.linuxvirtualserver.org/wiki/Using_arp_announce/arp_ignore_to_disable_ARP) - I mainly want to keep that setting as I am in a untrusted network
I do understand that the setup of network is maybe not standard, this is due to me following this: https://wiki.hetzner.de/index.php/KVM_mit_Nutzung_aller_IPs_-_the_easy_way/en - which allows me to make usage of all IPs of my given subnet
I do understand that setting arp_ignore=0 on the guest VM will solve the issue. However I would like to know why (because I do not need to do that on the host itself) and I can not accept that as answer because I do not believe setting it should cause the network to go offline (e.g. I want to solve the issue without making that setting a dependency)
I am assuming (thanks Peter Zhabin) that with netmask 255.255.255.255 it is safe enough that nobody within the untrusted network can take advantage of arp?

So my question I suppose is:
Why does arp_ignore=1 break the kvm-guest <> kvm-host networking when the kvm-guest uses pointopoint/255.255.255.255 ?
Edit 23/10/2018: guest interfaces config
# The loopback network interface
auto lo
iface lo inet loopback

auto ens7
iface ens7 inet static
  address IP from the pool {{ vbox_addtl_ipv4 }}
  netmask 255.255.255.255
  gateway {{ vbox_main_ipv4 }}
  pointopoint {{ vbox_main_ipv4 }}

iface ens7 inet6 static
  address xxx::xxx
  netmask 64
  gateway {{ vbox_main_ipv6 }}

Comment: Looks like it's because that's what you're asking for. Why are you trying to do that?
Comment: The only thing that does is to prevent the machine from responding to traffic for that machine that comes in on an interface other than the one to which the IP address is assigned. This may or may not be what you want. Almost always it is not. But you still haven't explained why you are trying to set this despite being asked, so it's impossible to say whether this really will solve [whatever the real problem is](https://meta.stackexchange.com/q/66377/189912).
Comment: OK, then you definitely do not want this.
Comment: Would it not be safer to have arp_ignore set to 1 in a "untrusted" network? Or in other words: what do I need to do so arp_ignore=1 can be set - to avoid vm guests losing connectivity if that setting is "accidentally" set.
Comment: Thanks Michael, you are right - I added some context and what I am really looking to solve. The main reason for me wanting to rollout `arp_ignore=1` is that I can not trust the network.
Comment: What exactly are you trying to avoid by setting this?
Comment: Spooler: As mentioned above, I am running the server in an untrusted network. I want to avoid the VM-host to ARP that is not coming from its own known interfaces.
Comment: Well, if you don't have any other ARP-capable interfaces on your guests (which for what is available on your posted configs you don't seem to have) there's no point of setting `arp_ignore` to 1, like Michael tried to explain to you. Your current setup creates guest's Ethernet interfaces as `POINTOPOINT`, so the real question is "why does arp_ignore=1 ruins ARP processing on Ethernet POINTOPOINT interfaces". That's an interesting question that requires diggin' linux kernel sources (BTW, what's your kernel version?) and may even yield some bugfixes. But "per se" it doesn't serve any purpose.
Comment: @PeterZhabin: thank you for your more detailed information. It is far more helpful than a simple "why do you need" or "why are you trying". I am obviously doing something wrong, but I would like to learn why it is wrong.

Ok, so it makes sense that with a netmask of 255.255.255.255 no arp will be happening on the vm-guest, meaning also the vm-guest should be safe (arp-wise) on an untrusted network.

> BTW, what's your kernel version?

4.9.110-3+deb9u6 / debian 9
Comment: Technically, an ARP-capable interface in P2P mode *should still use ARP*, as  there's no other way for it's endpoints to figure MAC address of the other end. The point is though that if your guest has only one Ethernet interface, setting arp_ignore to 1 effectively does nothing - the kernel should drop ARPs coming from other interfaces whereas you have just one.
Comment: Couldn't an interface in P2P mode simply send everything as Ethernet-level broadcast and enable promiscuous mode on the receive side, if the expectation is that there is only one other endpoint present on the link? Or am I missing something?
Answer: I have verified this behavior on Ubuntu 18.04 without any bridges, KVMs or Hetzner whatsoever and I think this is actually a kernel bug with regard to handling of arp_ignore on point-to-point Ethernet interfaces. Steps to verify:

Make sure netplan won't interfere by disabling it altogether.
Set up two systems with Ethernet interfaces connected to each other and IP addresses assigned like this:
Sever A:
ip addr add 192.168.100.1/32 peer 192.168.100.2 dev ens33
ip link set ens33 up
Server B:
ip addr add 192.168.100.2/32 peer 192.168.100.1 dev ens33
ip link set ens33 up
Observe a normal ping from Server A to Server B and ip addr show output that contains this line:
inet 192.168.100.1 peer 192.168.100.2/32 scope global ens33
Enable arp_ignore=1 with sysctl net.ipv4.conf.all.arp_ignore=1 on Server A and see pings stopping as soon as ARP entry times out. After certain short amount of time pings will resume for a while and then end again. This process will continue indefinitely in this manner.
Run a tcpdump on Server B and observe inbound pings and outbound endless who-has requests for the Server A IP address coming from Server B with no response. Occasionally, Server A will issue who-has request for the Server B IP, will get a reply, and Server B will temporarily cache source MAC-address obtained from this ARP request and this is when pings resume.

Now, here's what's going on. There's an in-kernel difference between point-to-point and standard broadcast interfaces in terms of how iproute2 (and similar tools) populate in_ifaddr structure defined in include/linux/inetdevice.h.
For normal interfaces it's ifa_address field gets populated with local interface address; for point-to-point interfaces ifa_address gets populated with remote peer address and ifa_local gets populated with local interface address.
This is normal and expected behavior, as in old-school days any point-to-point interface was considered an attached remote device itself; a local endpoint address corresponding to it may not even exist. So nothing is broken here.
What's really broken is that arp_ignore() handler defined in net/ipv4/arp.c indirectly uses confirm_addr_indev() function defined in net/ipv4/devinet.c that iteratively tries to match target IP address of the ARP request (that is, local interface address) with ifa_address field of all configured IP address records on an interface.
This works well on standard interfaces as they do have their local IP address recorded in ifa_address field, however with point-to-point interfaces it fails, 'cause they have peer IP address recorded in this field instead.
Now, the real question is does this issue really need a fix, as it has been pointed out already that there's no sense in using arp_ignore on p2p links. I think it does, as there might exist a situation where the host in question has more than one interface (and these interfaces are not p2p ones) and someone might enable this stuff with conf.all prefix only to see his p2p links going down for no apparent reason.
Comment: Thanks so much Peter for the detailed answer and replication I am quite (positively) surprised with your knowledge and effort on giving this information.
Comment: Can you please report this bug to kernel developers?
Answer: The answer is in the definition:

arp_ignore -
    0 - (default): reply for any local target IP address, configured
    on any interface
    1 - reply only if the target IP address is local address
    configured on the incoming interface
    2 - reply only if the target IP address is local address
    configured on the incoming interface and both with the
    sender's IP address are part from same subnet on this interface
    3 - do not reply for local addresses configured with scope host,
    only resolutions for global and link addresses are replied
    4-7 - reserved
    8 - do not reply for all local addresses

Essentially, because the point-to-point interfaces themselves do not have IP addresses assigned, there is no "target ip on the incoming interface".  The IP is assigned to a bridge interface... not to the actual link interface.  And as such, no ARP will be processed.
Comment: Well, this is only true for kvm **host**, whereas OP states that he's setting `arp_ignore=1` on kvm **guest**, where this link is just a standard Ethernet point-to-point interface *with local IP address*.
Comment: It would be useful if he posted a compy of the guest's config... from everything above, it is not unreasonable to assume he also has a bridge interface.  (even if there is only 1 interface in the bridge)
Comment: This might be the case. Few days ago I did a quick dig on a kernel source for any special handling of address scope assignment on p2p Ethernet links and found none, although it was very quick, so I'm might have missed something. The `arp_ignore` in itself is as simple as one `switch` statement and in case of `1` its outcome only depends on address scope.
Comment: I added the guest network interfaces conf.
Comment: I still believe that the host side is not responding to ARPs because of the bridge.  Check on the guest side with an `arp -a` and look for the mac-address of the host.


What is a name of a major scale with raised 2nd degree?

Question: Assuming a major scale that has a raised 2nd degree and no other alterations: 1 ♯2 3 4 5 6 7 (8) (for example, in C it will be C D♯ E F G A B (C)).
This is a bit unusual due to 2 consecutive semitones between 2nd, 3rd and 4th. Also, the ♯2 here sounds much like ♭3, which is a bit "jazzy" (for example, the 7th chord built on 2nd will be (in C) D♯-F-A-C, which is an enharmonic of F7 chord inversion (E♭-F-A-C)).
I've searched the name of this scale in many sources (just because I find it sounding good), but found nothing. Does it have a specific name?
Comment: In addition of the answer provided by @Shevliaskovic, I'd recommend looking into the Carnatic scales.  There are over 72 (more like well above 144) scales in that system including the 7 modes we have grown accustomed to seeing in Western music.  I point this out because when people ask "what is this scale" it is often from a Western POV, and other cultures may have invented them outside of Western influence and named them as well.
Comment: Do you have an example of that where `C` is actually a tonic, and little of no occurance of D natural?
Comment: Raga are not quite the same thing as scales, they describe expected movements between notes in addition to the notes themselves (and imply an Indian approach to pitch, which is different from our the-note-is-this-exact-frequency approach). I'd generally recommend avoid using Carnatic names to describe Western approximations being used to write non-Carnatic music.
Answer: If you start the scale from E, it sounds like a minor Neapolitan scale, which consists of the steps 1  ♭2  ♭3 4  5  ♭6 7  8. So your scale could be a minor Neapolitan scale starting from the 6th degree, which from the wiki article you can see it is simply called Ionian ♯2 *. It has exactly the same steps as you mentioned:

The scale degrees B C D# E makes you think of a harmonic minor scale with E as the tonic, but the second degree is flat, unlike the harmonic minor scale.

The scales are distinguished from the harmonic and ascending melodic
minor scales by the lowered supertonic or second scale degree
From the wiki article

* The Ionian mode has the same steps as the major scale.
Answer: I doubt it has a specific name. I think the best way to look at it is as a major scale with a passing note between the D and E. Some musicians talk about a Blues Scale which has D#/Eb in the key of C but in my experience there is no clearly defined Blues Scale and I think it just adds more confusion to music theory.
Scales are not strict and the major scale in C can still use all the chromatic notes without leaving the scale or confusing the tonality. You probably know this but if not you could look up "passing notes".
Answer: In Jazz theory this might be notated as F7 (dom.7) chord and probably part of the Blus scale: C,Eb,F ...). But the upper tetrachord is   the normal major scale.
In traditional terminology this chord is D#,F,A,C -> F,A,C,D# = German 6th chord.
Answer: C Mela Sulini
aka Raga Sailadesakshi, Raga Trishuli, Chulini, Houzam, Huzam, Dolian, Oprian.
You can answer most questions of this kind by using the scale finder tool and searching for the names you find there (although internet sources for some scales are not very good.)


Stacked Barplot with well placed Dates on x-Axis

Question: I am trying to build a stacked Barplot, without gaps between the bars and Dates on the x-Axis.
To achieve this, I set the width to the difference between the following dates and nudget the x position by half the datedifference.
So if the Date is 2018-01-31, I want the bar to start exactly at that date and end at the next date (2018-02-28). So far that works well, but when introducing that positioning logic, my bars stopped being stacked (picture: bars are overlapping but not stacked -> would sum up to 1). How can I restack them?
ggplot(weights_gg2, aes(x=Date, y=Exposure, fill=Bond)) +
  geom_bar(stat="identity", colour="white", width=rep(c(diff(weights_df$Date), 20), 13), position = position_nudge(x=rep(c(diff(weights_df$Date), 20), 13)/2)) +
  theme_bw(base_size=12)

Needed data: weights_df contains all data and the dates, while weights_gg2 is already melted for ggplot
Comment: I think we can use `hjust` for alignment purposes for example as shown [here](https://stackoverflow.com/questions/7263849/what-do-hjust-and-vjust-do-when-making-a-plot-using-ggplot)
Comment: I don't understand `my bars stopped being stacked (picture)`.. to me they look staked.
Comment: sorry I didnt make that clear, they're just overlapping each other, they should sum up to 1 if they were really stacked
Comment: Is it possible your calculation of the width is off? Admittedly I don't understand the logic behind this part of the code.
Anyways: This works (kind off) `ggplot(weights_gg2, aes(x=Date, y=Exposure, fill=Bond, width=rep(c(diff(weights_df$Date), 20), 13))) +
  geom_bar(stat="identity", colour="white", position = "fill") +
  theme_bw(base_size=12)` but raises a warning about overlapping x values..
Comment: yes that's kind of where I want to go, but without the position_nudge argument, the x axis alignment is off (there are gaps between bars, some are overlapping and the bars are centered around a given date, where I want the Bar to be right aligned at a specific date...
Comment: thanks, yeah sure you were right! The key was to manipulate the Date Data to match with the desired output and not to use position_nudge
Answer: darios answer was the most accurate to the problem:
Manipulate input data rather than changing the position afterwards:
hjust <- rep(c(diff(weights_df$Date), 20), 13)/2
ggplot(weights_gg2, aes(x=Date+hjust, y=Exposure, fill=Bond)) +
  geom_bar(stat="identity", colour="white", width=rep(c(diff(weights_df$Date), 20), 13))+
  theme_bw(base_size=12)


Maximizing Minimizing Probability

Question: I have to create an Array A, and for the sake of making things clear, I will create A = [a,b,c,d] with length n, 4 in this case. Then I feed A to the following algorithm;
For i in 1...n: do
     generate a random number j
     Swap(A[i], A[j])

Now how do I come up with this array, call it A (array contents decided by me), so the probability that the new array generated by this algorithm after giving it A as input, is closest to A. So I have to create an Array, A, feed it to the algorithm and the result should be as close as possible to the original Array, A.
Comment: So lost. I don't get this
Comment: Shouldn't your array be of length $n$? Call the array you are given $A$ and your array $B$.  It sounds like you do $n$ times swapping a random element of $A$ with the element that is $B(i)$.  In your example, if the starting $A$ is $[a,b,c,d]$ and the first random is $3$ you would swap the third element $c$ with $a$ and get $[c,b,a,d]$.  Then if the next random is $2$ you would swap $b$ with $c$ and get $[b,c,a,d]$.  OK so far?  But then you say we do the loop $4$ times, so what do we swap with for the last two?
Comment: Are we allowed to duplicate elements in $B$?  If not, $B$ must be a permutation of $A$.  I suspect that you can then prove it doesn't matter what the permutation is.
Comment: So sorry, not the best at explaining. I edited the Post.
Comment: What does "closest to A" mean?  How do you determine which of two permutations is closer to the original one?
Comment: @saulspatz means I should come up with an Array A, which when passed through the algorithm, will have a higher probability of not changing at all.
Answer: I assume that duplicates are not allowed.  That is, the initial arrangement must be a permutation of the number $1,2,3,4.$  According to my calculations, it doesn't matter what the initial arrangement is, as indeed seems true intuitively.
Here's a python script to test this:
from itertools import permutations, product

for p in permutations(range(4)):
    a = list(p)
    count = 0
    for d in product(range(4), repeat=4):
        for i in range(4):
            a[i], a[d[i]], = a[d[i]], a[i]
        if a == list(p): count += 1
    print(p, count)

This produces the output
(0, 1, 2, 3) 10
(0, 1, 3, 2) 10
(0, 2, 1, 3) 10
(0, 2, 3, 1) 10
(0, 3, 1, 2) 10
(0, 3, 2, 1) 10
(1, 0, 2, 3) 10
(1, 0, 3, 2) 10
(1, 2, 0, 3) 10
(1, 2, 3, 0) 10
(1, 3, 0, 2) 10
(1, 3, 2, 0) 10
(2, 0, 1, 3) 10
(2, 0, 3, 1) 10
(2, 1, 0, 3) 10
(2, 1, 3, 0) 10
(2, 3, 0, 1) 10
(2, 3, 1, 0) 10
(3, 0, 1, 2) 10
(3, 0, 2, 1) 10
(3, 1, 0, 2) 10
(3, 1, 2, 0) 10
(3, 2, 0, 1) 10
(3, 2, 1, 0) 10

so in all cases the probability is the same: ${10\over4^4}=0.0390625$
I'm sure that it can be proved that this is true for any $n:$ the initial permutation doesn't matter, although the technique escapes me.  As a way to discover the proof, I suggest modifying the above script to print out the random numbers (d in the script) that leave the permutations fixed for two specific permutations, and trying to see how one sequence maps to the other.  This will probably give you the pattern you need to prove.
Comment: I think your solution is right, just that having to generate the permutations consumes time. But I have fixed that. But I will like to implore your help one more time. What about minimum probability? How will I get a permutation farthest away from the original array?
Comment: @CarlsonBimbuh I don't understand what you mean by "farthest away."
Comment: I mean how will I generate a permutation such that, the probability for the algorithm to get it is minimal possible.
Answer: For the new formulation, if all the elements of $A$ are distinct, all that matters is the number of elements in $A$.  Shorter is better because there are less permutations to choose from.  Before you run your algorithm you can permute the elements into any order you want.  Now imagine the permuted one is $[1,2,3,\ldots n]$.  You have some probability to end with $[1,2,3,\ldots n]$.  That will be the same for any given length of $A$.

Maximizing Minimizing Probability


Preimage of sets, complement of sets, continuity of functions

Question: I just got some simple questions in real analysis regarding preimage and complement of sets and continuity.

Suppose $f:X\to Y$,
then does $f^{-1} (Y\setminus F)=f^{-1} (Y)\setminus f^{-1} (F)=X\setminus f^{-1} (F)$ always hold? But why the first equality is true?
Can we rewrite the definition of continuity of a function in metric space as  $f:X\to Y$ is continuous at $a \in X \Longleftrightarrow $ given $\epsilon >0$, there exists $\delta >0$ such that$f(B^{0}(a,\delta))\subset B^0(f(a), \epsilon)$, where $B^0$ represents open balls.

Thank you.
Comment: If I say your question is brilliant, that's a compliment (with an "i"), but if I add something to your question, thereby making it _complete_, then what I add is a complement (with an "e").  Just remember the resemblance to the spelling of "complete" to remember which is which.  I corrected the spelling in the question. $\qquad$
Comment: Oh yes.. sorry... Thank you :-)
Answer:
The first equality is true:
\begin{align*}
f^{-1}(Y\setminus F)&=\{x\in X:f(x)\in Y\setminus F\}\\
&=\{x\in X:f(x)\in Y\text{ and }f(x)\not\in F\}\\
&=\{x\in X:f(x)\in Y\}\cap\{x\in X:f(x)\not\in F\}\\
&=\{x\in X:f(x)\in Y\}\cap\left(X\setminus\{x\in X:f(x)\in F\}\right)\\
&=f^{-1}(Y)\cap\left(X\setminus f^{-1}(F)\right)\\
&=f^{-1}(Y)\setminus f^{-1}(F).
\end{align*}
The second equality is also true because $f$ being a function implies $f^{-1}(Y)=X$. Note that in general the preimage acts as we would wish with most set operations.
This is true. I guess your definition of continuity in a metric space goes like this: we say $f:X\to Y$ is continuous at $a\in X$ if
$$
\forall\epsilon>0,\exists\delta>0,\forall x\in X,d_X(x,a)<\delta\implies d_Y(f(x),f(a))<\epsilon.
$$
But the condition
$$
\forall x\in X,d_X(x,a)<\delta\implies d_Y(f(x),f(a))<\epsilon
$$
is readily seen to be equivalent to
$$
f(B^0(a,\delta))\subset B^0(f(a),\epsilon).
$$
Answer: Yes to both. To see why the first is true, observe that $x\in f^{-1}(Y\setminus F)$ if and only if $f(x)\in Y\setminus F$ if and only if $x\notin f^{-1}(F)$.


Centos BIND9 multiple NS with different IPS

Question: I just acquired a Root Server and configured OpenVZ for multiple containers. Each container has a different IP. What I am trying to do is to centralize all DNS related stuff in the main server os ( CentOS ).
I installed bind9 in CentOS and configured to name servers for it, which worked fine. Then I tried to add more name servers for each different IP. So what I am trying to do is:
ns1.domain.tld 86400 IN A 176.x.x.x
ns2.domain.tld 86400 IN A 176.y.y.y
ns3.domain.tld 86400 IN A 176.z.z.z

Then when I add the nameserver ns2.domain.tld to a certain domain, it should point to 176.y.y.y .
My current config is :
$TTL 10800
@ IN SOA ns1.palianos.ro.    postmaster.palianos.ro. (
    2014051300  ;serial
    21600    ;refresh after 6 hours
    3600    ;retry after 1 hour
    604800   ;expire after 1 week
    86400 )    ;minimum TTL of 1 day
@    3600    IN    A    18.104.22.168
mail    86400    IN    A    22.214.171.124
ns1    172800    IN    A    126.96.36.199
ns2    172800    IN    A    184.108.40.206
ns3 172800  IN  A   184.108.40.206
ns4 172800  IN  A   184.108.40.206
ns5 172800  IN  A   22.214.171.124
ns6 172800  IN  A   126.96.36.199
ns7 172800  IN  A   188.8.131.52
ns8 172800  IN  A   22.214.171.124
gitlab  178000  IN  NS  ns3.palianos.ro.
gitlab  178000  IN  NS  ns4.palianos.ro.
www    3600    IN    CNAME   @
@    86400    IN    MX    10  mail.palianos.ro.
@    172800    IN    NS    ns1.palianos.ro.
@    172800    IN    NS    ns2.palianos.ro.

When I try to dig the subdomain test.domain.tld it gives me no ANSWER SECTION. It works fine for domain.tld tho.
Thanks.
Comment: If you are prepared not to redact the domain name in question, we can probably give you better answers.
Comment: There is no trace of `test` in the zone data included in the question. That would seem like the obvious reason why you get nothing when you try to look up `test`. Also, I don't understand the overall goal of what you are doing. You say that you want to centralize all DNS related stuff to the main host, yet you "add more name servers for each different IP" (which appears to mean just adding `A` records with  `nsX` as the name), what is the purpose of this? Also, why do you have two `nsX` names for each IP? Fwiw, `ns8.palianos.ro` does resolve, if that is the goal (not clear from the question).
Answer: The problem at the moment appears to be that the two nameservers (well, one, really; ns3 and ns4 have the same IP address) to which you have delegated gitlab.palianos.ro aren't responding to any kind of requests about it:
[me@risby ~]$ dig any gitlab.palianos.ro @ns3.palianos.ro

; <<>> DiG 9.9.4-P2-RedHat-9.9.4-12.P2.fc20 <<>> any gitlab.palianos.ro @ns3.palianos.ro
;; global options: +cmd
;; connection timed out; no servers could be reached
[me@risby ~]$ dig any gitlab.palianos.ro @ns4.palianos.ro

; <<>> DiG 9.9.4-P2-RedHat-9.9.4-12.P2.fc20 <<>> any gitlab.palianos.ro @ns4.palianos.ro
;; global options: +cmd
;; connection timed out; no servers could be reached

This means that although the nameservers for palianos.ro are happy to provide those delegating records:
[me@risby ~]$ dig ns gitlab.palianos.ro @ns1.palianos.ro
[...]
;; QUESTION SECTION:
;gitlab.palianos.ro.        IN  NS

;; AUTHORITY SECTION:
gitlab.palianos.ro. 178000  IN  NS  ns3.palianos.ro.
gitlab.palianos.ro. 178000  IN  NS  ns4.palianos.ro.

;; ADDITIONAL SECTION:
ns3.palianos.ro.    172800  IN  A   188.8.131.52
ns4.palianos.ro.    172800  IN  A   184.108.40.206

when I do a more general query and let the recursion sort itself out, I get no answer:
[me@risby ~]$ dig ns gitlab.palianos.ro

; <<>> DiG 9.9.4-P2-RedHat-9.9.4-12.P2.fc20 <<>> ns gitlab.palianos.ro
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 3480
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;gitlab.palianos.ro.        IN  NS

;; Query time: 192 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri May 16 07:09:01 BST 2014
;; MSG SIZE  rcvd: 47

I think you need to fix ns3 and ns4, and get them to serve the subzone, before any of this is going to work.


WebDriver: Possible to Create x2 Seperate 'Cookie' Session with the same Browser?

Question: WebDriver: Possible to Create x2 Separate 'Cookie' Session with the same Browser?

I have altered code within my TestNG file which will execute two test classes at the same time by using x2 seperate threads.
The issue: within my test scripts 'A user will log into their account', but when the two scripts are running the test fails because they both use the same account/login details.
Is it possible to use the same login account details and same browser when executing more than one+ WebDriver test?

thanks for your help
Answer: No, if you want two sessions you will need to run separate browsers as two different services.


Is there a way of getting a file name and inserting into Matlab script?

Question: In a folder, I have both my .m file that contains the script and one imaging .dcm file that is the target of my analysis.
Folder structure:
Folder1/analysis.m
Folder1/meas_dynamic_123.dcm
Folder1/meas_123.dcm
Folder1/meas_345.dcm

My script (analysis.m) begins as follows:
target     =''; <== here should go only the filename that contains 'dynamic'
                    example: meas_dynamic_123.dcm

txt        = dir(target);

// etc

So I'm wondering if there is a way of when running analysis.m it will:

automatically search the folder it's in,
grab the filename of file containing string dynamic in the name,
insert its name into target variable
continue running the script

Does anyone have any pointers on how to achieve this? Using ffpath?
Answer: you should do:
f=dir('*dynamic*');
target=f(1).name;

Comment: this works on the command line but when I put it in the script I get an error `Function is not defined for 'struct' inputs.`
Comment: This is because you need to extract only the filename from the structure returned by `dir`. Try `txt=dir('*dynamic*');` `txt={txt.name};`
Answer: Assume you are not in Folder1 and you need to find it:
script_name = 'analysis';
script_full_path = which(script_name);      % get full path of the script
script_dir = fileparts( script_full_path ); % get the directory of the script
file_list = dir( [script_dir '/*dynamic*']);
name_list = {file_list.name};

name_list will be a cell array containing the file names with the keyword 'dynamic'.
Comment: thx will keep this for full path but for now using simpler approach from @oli
Answer: Instead of which(scriptname), consider using mfilename('fullpath'), which will directly give you the full path of the running m-file. Using which might break if you had multiple commands of the same name shadowed on the path.


Apagar o primeiro numero se caso for 0

Question: Como faço pra apagar o primeiro número se caso ele for igual a "0", por exemplo, tenho  o seguinte numero "0123" ficaria "123", se ele fosse "123" não aconteceria nada.
Answer: Você pode utilizar o ltrim, por padrão ele remove espaços no inicio do texto. Entretanto, especificando o último argumento, ele irá remover os caracteres especificados se aparecerem no inicio da string:
ltrim ('0123', '0');

Isto irá remover todos os 0 que sejam apresentados antes.

Exemplos:
Original: 00000123
ltrim:    123

Original: 101
ltrim:    101

Original: 014410
ltrim:    14410

Original: 100
ltrim:    100

Original: 00100
ltrim:    100

Teste isto

Caso o valor seja 0 ele irá também remover, ou seja 0, 00 ou 00000 (...) se tornará vazio, uma alternativa, caso seja int é utilizar (int)ltrim ($numero, '0')
Comment: Usei e resolveu meu problema. Obrigado!

Apagar o primeiro numero se caso for 0


Is $\mathbb{Z}[x_1,x_2,\ldots]$ the universal non-Noetherian ring?

Question: I recently learnt about the fact that $\mathbb{Z}[x_1, x_2,\ldots]$ is an example of a non-Noetherian ring. This example seems like it should in some sense be the "universal example" of a non-Noetherian ring. So, my question is if there is a result stating the existence of a morphism between $\mathbb{Z}[x_1,x_2,\ldots]$ and any given non-Noetherian ring, or something along those lines.
Comment: It can't correspond to a non-commutative ring, and a ring could have more than $\mathbb{Z}^{\mathbb{N}}$ elements. I'm not even sure if we could say for every commutative ring $R$, there is a ring homomorphism from $\mathbb{Z}[V]$ to $R$ for some set of variables $V$.
Comment: well, $\mathbb Z[|R|]\to R$ is exactly the counit of the free-forgetful adjunction, which is an epimorphism because the right adjoint -the forgetful- is faithful (if two ring homomorphisms coincide as set functions, they are the same homomorphism).
Comment: @aschepler Your second sentence is weird, did you mean something else?
Comment: I don't understand the "I'm not even sure": if $R$ is nontrivial, $V$ a singleton, the answer is yes?
Comment: @fosco I should have said surjective. The idea I was trying to get across isn't entirely baked. Also in that case, I guess one variable per element of $R$ works. (I was thinking  earlier infinite sums could be a problem, but they don't come up because of the definition of polynomial rings.)
Answer: The answer is No. Any ring $R$ admits a homomorphism $\mathbb{Z}[x_1,x_2,\dotsc] \to R$ (just send all $x_i$ to $0$, for example, and use the unique homomorphism $\mathbb{Z} \to R$). It is also not possible to add reasonable assumptions to this homomorphism which enforce $R$ to be Non-Noetherian or that such a homomorphism needs to exist when $R$ is Non-Noetherian. For example, there is a monomorphism of rings $\mathbb{Z}[x_1,x_2,\dotsc] \to \mathbb{C}$, but $\mathbb{C}$ is a field and hence Noetherian. If $I$ is some infinite set, then $\mathbb{Z}[(x_i)_{i \in I}]$ is a Non-Noetherian ring, which does not embed into or admit an epimorphism from $ \mathbb{Z}[(x_i)_{i \in \mathbb{N}}]$ unless $I$ is countable. The ring $\mathbb{Z}[x_1,x_2,\dotsc]$ is Non-Noetherian, the quotient $\mathbb{Z}[x_1,x_2,\dotsc]/\langle x_1 \rangle$ is still Non-Noetherian, but the quotient $\mathbb{Z}[x_1,x_2,\dotsc]/\langle x_1,x_2,\dotsc \rangle = \mathbb{Z}$ is Noetherian.  The category of Non-Noetherian rings is also badly behaved, there are no initial or terminal objects. This false impression that $\mathbb{Z}[x_1,x_2,\dotsc]$ is somehow universal among Non-Noetherian rings probably can be avoided by looking at more examples of Non-Noetherian rings, such as $\prod_{p \text{ prime}} \mathbb{F}_p$ or $C^{\infty}(\mathbb{R})$.
Comment: Excellent answer, thank you!
Comment: You're welcome.


Prove that $x$ is bounded.

Question: Suppose that $x: \mathbb R^+ \to \mathbb R^+$ and that
$$
\frac{d x}{d t}  \leq x^2 $$
$$\int_0^{\infty} x(s) d s  <\infty $$
Post edit:( from the valuable comments)
Prove that  $x$ is bounded. (Note: this is not obvious since the integral constraint does not eliminate singularities like $1 / \sqrt{t}$.
As a hint I am thinking $x^2$ as $x(t) x(t)$ and thinking about how to solve $x^{\prime}=a(t) x(t)$ but didn't get any help so far.
Comment: @Ri-Li: You write "all $t$", but is it $t>0$ or $t\in \mathbb R$ or what? Also, are you sure about the conclusion? Shouldn't it be $x(t)\le C$ for some (finite) constant $C>0$?
Comment: The question is ill-posed. If $x$ is a function $x :[0,\infty ) \to \mathbb R$ then $x(t)$ cannot take the value $\infty$ - there is no real number named $\infty$
Comment: What is $\frac {dx} {dt}$ is $x$ is not real valued?
Comment: @geetha290krm: it's written that $x(t)>0$, so in particular it is real-valued.
Comment: Maybe the question is ill-posed but that is what I got. Your comment makes sense as being a continuous solution it can't reach infinity. In ODE we always try for $t \in (-\infty, \infty). @GiuseppeNegro yes it should be $x(t) \leq C$ for all $t$.
Answer: Suppose $x:\mathbb{R^+}\rightarrow\mathbb{R^+}$ is of class $C^1$. Since $x\left(t\right)>0$ we have
$$
\frac{x'(t)}{x(t)}\leq{x(t)}
$$
for all $t\in\mathbb{R}$.
Then
$$
\int_{0}^{s}\frac{x'(t)}{x(t)}dt \leq
\int_{0}^{s}{x(t)}dt \leq
\int_{0}^{\infty}{x(t)}dt = c <\infty
$$
for all $s>0$. Since the integral on the left side equals $\ln{x\left(s\right)}$ we have $x\left(s\right)\leq{e^c}$ so that $x$ is bounded.
Answer: This is answer to the original formulation of the question, where $x:\mathbb R \to \mathbb R^+$.
In fact $x$ does not need to be bounded if indeed defined for all $t\in \mathbb R$. To see it, consider the function
$$
  x(t) = \begin{cases}
t^2, & \text{ if } t\leq 0 \\
0,& \text{ if } t \geq 0.
\end{cases}
$$
Its derivative is negative on $(-\infty,0)$ and zero on $[0,\infty)$, so the inequality $x'\leq x^2$ is satisfied  trivially. Likewise the integral condition is satisfied trivially as $\int_0^\infty x(s)ds = 0$.
Comment: I think you are right the time is $[0,\infty)$. Edited

Prove that $x$ is bounded.


Where and how to get `identityId` using AWS Amplify?

Question: So I'm trying to follow along the guide in the docs and I'm stuck here
Storage.get('test.txt', {
  level: 'protected',
  identityId: 'xxxxxxx' // the identityId of that user
})
.then(result => console.log(result))
.catch(err => console.log(err));

How does one get that identityId?
Answer: I found a hack to this from Auth.currentUserCredentials() by saving the identityId to a custom attribute on successful user login:
const CUSTOM_IDENTITY_FIELD = "custom:identityId";

if (attributes && !attributes[CUSTOM_IDENTITY_FIELD]) {
    await Auth.updateUserAttributes(currUser, {
      [CUSTOM_IDENTITY_FIELD]: (await Auth.currentUserCredentials())
        .identityId,
    });
    onInfo("Welcome to Tippify, " + currUser.username);
}

With this, I could also implement a Firebase first-time log in functionality.
Later when I wanted to query the user image from Storage:
Storage.get(user.picture, {
    level: "protected",
    identityId: user[CUSTOM_IDENTITY_FIELD],
})
  .then(setPicture)
  .catch(onError);

Answer: In case anyone's still looking for the answer, this worked for me (in Amplify V6):
import { fetchAuthSession } from 'aws-amplify/auth'

...

const credentials = await fetchAuthSession()
console.log(credentials.identityId)


Read unsigned chars into string

Question: I have an array of unsigned chars:
unsigned char buffer[BUFFER_SIZE];

I want to read in the first N unsigned chars from that array into a string however after looking at strncpy(), that seems to only take pointers to signed chars.
Should I be looking at memcpy()?
Comment: What is the format of the buffer? Is all full or half empty? All values are have the same size? **What have you tryied**?
Comment: @PaperBirdMaster- Bad Comment man. Doesn't makes sense to me. The question is simple and self-explanatory.
Answer: Not sure about the exact syntax but, if possible, you should use:
reinterpret_cast<char *>(buffer[i]);

Also see:
Is there a good way to convert from unsigned char* to char*?
Answer: strcmp and memcmp compare data, they don't copy it...
Anyway, just cast to char * and use whatever stdlib function makes sense.
Answer: If by string you mean a char array, the simplest way in c++ is just to use std::copy, no need to cast anything:
unsigned char buffer[BUFFER_SIZE];
char dest[BUFFER_SIZE]

std::copy( buffer, buffer + N, dest );

If you mean a std::string, just use the iterator style constructor:
std::string dest( buffer, buffer + N );

Read unsigned chars into string


Can I programmatically create a new node revision, without the new revision becoming the "current" one?

Question: Whenever my feeds import finds a new version of an existing piece of content, it creates a new revision of this node instead of overwriting data.
Normally the new revision becomes the "current" one right away. I would like the currently displayed revision however to remain the same, and let a moderator decide later if the new one should be published.
Comment: Have you tried using Workbench Moderation, Workflow, or any of the modules that do the same thing?
Comment: I haven't. I initially assumed that because I create the node in code, they didn't apply. Perhaps that's not the case.
Answer: A standard Drupal install does not allow you to create a "Pending" revision.  You have two options:

Programatically create a new revision but programatically revert back to the original revision (which creates an even newer revision, but it has the original content)
(Recommended) Use Workbench Moderation, Revisioning or Workflow which are well thought-out solutions for version control and/or access control.

For option 1:
You can add this code as a new Rule or use it in a new module
<?php
  // Programatically load the existing revision and save it
  // Taken from http://api.drupal.org/api/drupal/modules!node!node.module/function/node_save/7
  // Load the revision
  $original_revision = node_load($nid);
  $original_revision->revision = 1;
  $original_revision->log = t('Copy of the revision from %date.', array('%date' => format_date($original_revision->revision_timestamp)));

  $new_revision = node_load($nid);
  // Make any changes to the new revision here...
  $new_revision->revision = 1;
  $new_revision->log = t('Summarize your changes here');

  // Save the new revision first
  node_save($new_revision);

  // Save the original one again so that it is still the current revision
  node_save($original_revision);

  watchdog('content', '@type: reverted %title revision %revision.', array('@type' => $node_revision->type, '%title' => $node_revision->title, '%revision' => $node_revision->vid));
  drupal_set_message(t('@type %title was saved with a new revision, but reverting to original revision from %revision-date.', array('@type' => node_type_get_name($node_revision), '%title' => $node_revision->title, '%revision-date' => format_date($node_revision->revision_timestamp))));
  drupal_goto('node/' . $node_revision->nid . '/revisions');
?>

For option 2:
I would recommend Workbench over Revisioning or Workflow, but each is different depending on your needs.  Workbench is kind of the successor to Revisioning, and Workflow is a lot more than just version control, so it may or may not be a good fit for your needs.
Here's a quick breakdown on the differences between Workbench and Workflow.
Comment: The first option would be great, but how do I make the revert?
Comment: Added the code for option 1, but really option 2 is probably your best bet because it adds functionality for handling which version is current, instead of just duplicating the original over and over again
Comment: the link is broken
Comment: Where to write in hook_form_alter ?
Comment: link fixed.  @IcecreamJelly, try hook_node_update.
Answer: This is a guess, but I would give the Workbench Moderation sub-module in the Workbench module a shot.  I have found this much easier to set up than Workflow.
The fact that you are creating nodes in code shouldn't matter, as long as the node_save() runs as a user with the proper settings for the content type (the moderation states are handled with the node API).  This may mean, though, that you need to do some session shenanigans to get things done as the proper user when Feeds runs.
Comment: +1 workbench moderation is a superb module I've been using it for quite a while now


How to pass Scenario name from the Cucumber feature file as a parameter in the Cucumber steps?

Question: I need some help, would appreciate if someone can help?
I need pass the Scenario Name in the Cucumber feature file as a parameter in the steps..
In the Background step- I am launching the browser and logging in to the application, so that I do not have to repeat the same steps in every scenario. There is a JAVA method to start the video recording for the GUI which is being used in the Background test- and the video recording will be there for individual scenarios- So if there are 10 scenarios in the feature file- the video recording needs to give 10 outputs which shows the automation run for these 10 scenarios. The method for video recording capture saves the file name based on the argument which will be passed.
For example- My Scenario in the Feature file is:
Feature: Do Something

Background:
    Given I start the recording for the scenario "Pass the scenario name here"
    And I navigate to the login page
    When I submit username and password
    Then I should be logged in

Scenario: Scenario Name
    Given I start the test for "Scenario Name"
    Then it should do something
    And stop the recording

Scenario: Scenario Name 2
    Given I start the test for "Scenario Name 2"
    Then it should do something
    And stop the recording

How can i pass the Scenario name as a parameter in the Step?
Comment: In the Background step- I am launching the browser and loging in to the application, so that I do not have to repeat the same steps in every scenario. There is a JAVA method to start the video recording for the GUI which is under test- and the video recording will be there for individual scenarios- So if there are 10 scenarios in the feature file- the video recording needs to give 10 outputs which shows the automation run for these 10 scenarios. The method for video recording capture saves the file name based on the arugument which will be passed.
Comment: What language are you writing step definitions in?
Comment: Step Definition code is in JAVA
Comment: In the Background: I want to pass the Scenarios name as a parameter- my step in background is- Given i start the recording for "This is where i want to pass the scenario name as a parameter" ... In the respective scenarios- I want to add the last step as- And i stop the recording for "This is where again i want to pass the name of the scenario"... The argument is the scenario name which should be used by the two methods.. Does it make sense now?
Comment: I'm sorry, but this is not how cucumber was meant to be used. There is a philosophy behind it. I highly recomment reading [BDD 101: Introducing BDD](https://automationpanda.com/2017/01/25/bdd-101-introducing-bdd/) and [Writing Good Gherkin](https://automationpanda.com/2017/01/30/bdd-101-writing-good-gherkin/).
Comment: In the Background: I want to pass the Scenarios name as a parameter- my step in background is- Given i start the recording for "This is where i want to pass the scenario name as a parameter" ... In the respective scenarios- I want to add the last step as- And i stop the recording for "This is where again i want to pass the name of the scenario"... The argument is the scenario name which should be used by the two methods.. Does it make sense now?
Comment: It makes sense. I'm saying that this is not the way you should be doing it. Each scenario needs the proper `Given` steps to set things up. What do you mean by "start recording"? Maybe this is where I'm getting confused.
Comment: And in this case the argument needs to be the name of the scenario- so that the file gets saved with the scenario name
Comment: Ah, ok. This is vital information. Please add this to your question.
Comment: Thanks Greg, please let me know if there is any solution to this, thank you!
Answer: Starting a recording of the test execution is not something that belongs in your cucumber test. And as you have found, it is very difficult to accomplish what you want.
This is where Cucumber Hooks can help:
@Binding
public class TestRecorder {
    private final VideoRecorder videoRecorder;

    public TestRecorder() {
        this(new VideoRecorder(...));
    }

    public TestRecorder(VideoRecorder videoRecorder) {
        this.videoRecorder = videoRecorder;
    }

    @Before
    public void BeginRecording(Scenario scenario) {
        String scenarioName = scenario.getName();

        // do something with scenarioName and start recording...
        videoRecorder.start();
    }

    @After
    public void StopRecording(Scenario scenario) {
        String scenarioName = scenario.getName();

        // Stop recording, and use scenarioName to save to a file
        videoRecorder.stop();
    }
}

Before the scenario starts, begin the video recording. The TestRecorder class can declare private fields to hold a reference to the video recorder. The Scenario object passed in to the before and after scenario hooks (BeginRecording and StopRecording) give you information about the scenario, including the name. This should give you enough to save the video recording to a file using the scenario name as the file name.
And since this is just a POJO, you could write some unit tests for the video recording functionality too (if you really wanted).
Now your cucumber tests can stay focused on the system under test, and the not system monitoring the test:
Feature: Do Something

Background:
    Given I navigate to the login page
    When I submit username and password
    Then I should be logged in

Scenario: Scenario Name
    When I do the thing
    Then it should do something

Scenario: Scenario Name 2
    When I do the thing
    Then it should do something

No need to pass the scenario name from the feature file. It's all done "behind the scenes" in the cucumber hooks.
Comment: Thank you so much, this is great feedback! :)


Query nodes from multiple specific paths in cypher

Question: I'm playing with cypher and I have some simple aggregation going on for me.
MATCH (p:Person)-[:HAS_CAR]->(n:Car)
RETURN n, count(p)

MATCH (p:Person)-[:HAS_APARTMENT]->(n:Apartment)
RETURN n, count(p)

MATCH (p:Person)-[:HAS_HOUSE]->(n:House)
RETURN n, count(p)

The problem is that I have to make 3 trips to the database to get all those results together. The problematic thing about that is that those queries are the last MATCH statement in a much bigger chain. Like this:
MATCH (:City { Id: 10})<-[:LIVES_IN]-(p:Person)
WITH p
MATCH ...
WITH p
MATCH ...
WITH p
MATCH ...
WITH p
MATCH ...
WITH p
MATCH p-[:HAS_CAR]->(n:Car)
RETURN n, count(p)

After all those MATCH ... WITH statements, only a few person nodes are matched so the last part of the query is very fast, but the initial part is not. I can't help but think that this could be improved because all three queries share a lot of statements.
I came up with this:
...
MATCH p-[:HAS_CAR|HAS_APARTMENT|HAS_HOUSE]->(n)
RETURN n, labels(n), count(p)

And I can work with that. But what if I wanted to mix in something like this:
MATCH p-[:KNOWS]->(:Person)-[:HAS_BIKE]->(n:Bike)
RETURN n, count(p)

Or even:
MATCH p-[:KNOWS]->(:Person)-[:HAS_BIKE|HAS_BOAT]->(n)
RETURN n, labels(n), count(p)

Can all of this be done in a single query and how?
Answer: Sometimes you need to use collections instead of rows to merge aggregation queries together and pass them along. This strategy might help... For example:
MATCH (p:Person)-[:HAS_CAR]->(car:Car)
WITH car, count(p) carCount
WITH collect({car:car, count:carCount}) as carCounts
MATCH (p:Person)-[:HAS_APARTMENT]->(n:Apartment)
WITH n, count(p) as apartmentCount, carCounts
RETURN collect({apartment:n, count:apartmentCount}) as apartmentCounts, carCounts

Update (see comments)--this lets you pass along the results of a filter and do a quick id lookup to find them again:
MATCH (p:Person)
WHERE p.name = "John" // or whatever else you need to filter on
WITH collect(id(p)) as pids
MATCH (p)-[:HAS_CAR]->(car:Car)
WHERE id(p) IN pids
WITH car, count(p) carCount, pids
WITH collect({car:car, count:carCount}) as carCounts, pids
MATCH (p)-[:HAS_APARTMENT]->(n:Apartment)
WHERE id(p) IN pids
WITH n, count(p) as apartmentCount, carCounts
RETURN collect({apartment:n, count:apartmentCount}) as apartmentCounts, carCounts

Comment: Because you are not passing `p` in the `WITH car, count(p) as carCount`, the statement `MATCH (p:Person)-[:HAS_APARTMENT]->(n:Apartment)` is not respecting previous filtering chain and matches **all** Person nodes. If you add `p` like `WITH p, car, count(p) as carCount` then the `count(p)` aggregation is not correct. Is there any way to solve this?


Removing object element from array of object in state (React.js)

Question: I am trying to remove an object from my array but it always removes the last one.
const removeDuplicateHandler = (id) => {
    setCopyActions((copyActions) =>
      copyActions.filter((action) => action.id !== id)
    );
};

And this is the loop that Im running through
{copyActions.map((actionType, idx) => (
        <TargetInputBox
          key={idx}
          isDuplicate={true}
          actionName={actionType.label}
          contactPersons={store.allContactPersons}
          buttonAttrValue={actionType.value}
          duplicateAddHandler={() =>
            addDuplicateHandler(actionType.value, actionType.label)
          }
          duplicateRemoveHandler={() =>
            removeDuplicateHandler(actionType.id)
          }
        />
))}

Answer: A for loop is a better and more efficient way to do this:
const duplicate = [...copyActions]
for (let i = 0; i < copyActions.length; i++) {
    if (copyActions[i].id === id) {
        duplicate.splice(i, 1)
    }
    if (i + 1 === copyActions.length) {
        setCopyActions(duplicate)
    }
}

You create a duplicate of the array and loop through it. If the program finds an ID that is the same as id, it will remove that index from the array. Once the loop has finished, it will set the state to the altered array.


How to style (HTML) Monolog emails sent via Symfony Mailer

Question: Due to depreciation, I'm converting my Monolog email logging from Swiftmailer to Symfony Mailer (as recommended here).
I have successfully completed the conversion and am receiving the emails.
However, with Swift Mailer all Monolog emails were nicely formatted (see example).
With Symfony Mailer, I only have a text version of the email.
Is there a way to get similar HTML output with Symfony Mailer?
By the way, I'm only using "symfony/mailer" and "symfony/monolog-bridge", not the full framework
My code:
<code>$logger = new Logger('My Logger');

$transport = Transport::fromDsn('MYDSN');
$mailer = new Mailer($transport);

$message = (new Email())
    ->from(new Address('firstname.lastname@example.com', 'John Doe'))
    ->to(new Address('firstname.lastname@example.com', 'Jane Doe'))
    ->subject('My Subject');

$logger->pushHandler(
    new MailerHandler(
        $mailer,
        $message,
        Logger::ERROR
    )
);
</code>
Answer: Got it working, I had to add an HtmlFormatter (this was not necessary with Swift Mailer):
<code>$logger->pushHandler(
    (new MailerHandler(
        $mailer,
        $message,
        Logger::ERROR
    )->setFormatter(new HtmlFormatter())
);
</code>
Answer: Late to the party, but if you configure the monolog-mailer via yaml only, this is what you need to add to get the nice html-formatting:
<code>formatter:  monolog.formatter.html
content_type: text/html
</code>
(source: How to configure Monolog to Email Errors)


negation of boolean expressions with XOR

Question: I have this:
// returns true if both are equal (independent of scale) and also checks against null
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
        // 1. check: both will be null or both will be non-null.
        if (val1 != null ^ val2 != null) return false;
        // 2. check: if not null, then compare if both are equal
        return !(val2 != null && val1.compareTo(val2) != 0);
    }

I want to merge the boolean expressions into one. So I use this:
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
    return !(val1 != null ^ val2 != null) && !(val2 != null && val1.compareTo(val2) != 0);
}

However, I am worry if this is correct. Is this correct? Could this be simplified/shorten?
With the help of the answer the shorten solution is:
// returns true, if both are null or both are equal
// (independent of their numbers scales)
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
    return val1 == null ? val2 == null : val2 != null && val1.compareTo(val2) == 0;
}

Comment: _"I want to merge the boolean expressions into one."_  --  Don't.  I fully understand the desire to have more "compact" code, but the compiler is going to convert them both into the same bytecode, so you should stick to making your code as easy to read (and thus to debug and maintain) as possible.
Comment: So you want to check if two BigDecimal are equals?
Comment: What do you mean by "correct"? Does it accomplish the logic or functionality you want?
Comment: @TylerH yes as it was answered..
Comment: @nimo23 Sorry, that doesn't make sense. By "it" I mean the code in your question. If the code in your question works as you expect, then it is "correct". Otherwise you need to clarify what you mean (or the question risks being closed as unclear/opinion-based).
Comment: yes, the code in my question was already right but too complicated..the shorten solution simplifies it.
Answer: Using ^ as a logical XOR operator is highly unusual. It works, but I would avoid it for readability's sake. != is a good replacement.
return !((val1 != null) != (val2 != null)) && !(val2 != null && val1.compareTo(val2) != 0);

Now you can replace the double negation with ==. Nice.
return ((val1 != null) == (val2 != null)) && !(val2 != null && val1.compareTo(val2) != 0);

You can also distribute the remaining ! via De Morgan's laws:
return ((val1 != null) == (val2 != null)) && (val2 == null || val1.compareTo(val2) == 0);

It's better, but to be honest I still find it unwieldy. I'd go with something more straightforward even if it's not a single line statement:
if (val1 == null) {
    return val2 == null;
}
else {
    return val2 != null && val1.compareTo(val2) == 0;
}

You could use the ternary operator in place of if/else. Up to personal preference which you find more readable:
return val1 == null
    ? val2 == null
    : val2 != null && val1.compareTo(val2) == 0;

You mention that you need to use compareTo(). For others who might read this answer, if you don't have to use compareTo() I'd use equals() instead.
if (val1 == null) {
    return val2 == null;
}
else {
    return val1.equals(val2);
}

Then, as it so happens, you wouldn't even need to write this method at all. The built-in Objects.equals() method does exactly this: it returns true if two objects are equal, or if they're both null.
Comment: `a.equals(b)` behaves the same as `Objects.equals(a, b)` which is NOT what I want. I want to check if both are equal (**independent of scale**) and also checks against nulls..(so if both are null, both should be treated as equal.
Comment: @Rogue I dont understand: I want to use `boolean isEqual(BigDecimal val1, BigDecimal val2) {return val1 == null ? val2 == null : val2 != null && val1.compareTo(val2) == 0;}`. How would you shorten this?
Comment: @nimo23 I think the last example/code of this answer was meant, not your comment
Answer: You could use Comparator.nullsFirst (or Comparator.nullsLast):
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
    return Comparator.nullsFirst(BigDecimal::compareTo).compare(val1, val2) == 0;
}

Example:
    System.out.println(isEqual(null, null));
    System.out.println(isEqual(null, new BigDecimal(1.0)));
    System.out.println(isEqual(new BigDecimal(1.0), null));
    System.out.println(isEqual(new BigDecimal(1.0), new BigDecimal(1.0)));

Output
true
false
false
true

Comment: hmm..I like your solution..however I guess, the approach of the accepted answer is better suited for overiding `equals()/hashCode()` methods..no need to create objects Comparator..etc..only boolean operators..should be faster..
Answer: If scale wasn't an issue, I would utilize a ternary here, and take advantage of Object#equals returning false for null:
//Basically Objects#equals at this point
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
    return val1 == null ? val2 == null : val1.equals(val2);
}

Thus in every case:
[1: null, 2: null]: true
[1: null, 2:  num]: false
[1: num,  2: null]: false
[1: num,  2:  num]: Object#equals

However as you've noted, you want to compare irrespective of scaling as well. So we'll have to include another null check as #compareTo throws an NPE if you pass null:
public static boolean isEqual(BigDecimal val1, BigDecimal val2) {
    return val1 == null
            ? val2 == null
            : val2 != null && val1.compareTo(val2) == 0;
}

Thus in every case (again):
[1: null, 2: null]: true
[1: null, 2:  num]: false
[1: num,  2: null]: false
[1: num,  2:  num]: BigDecimal#compareTo

I would find this more readable, but it's still a little unweildly. At this point, a well-written and succinct comment (or, breaking up the line per the other answer) would probably do you better than code rewriting.
Comment: It really shouldn't, as you're returning a scalar which represents the difference between two values. There's no good number to show the difference between `14` and `null` that wouldn't either conflict with other similar values or introduce a magic value.
Comment: Thanks, pitty that JDK does not provide the null checks within `compareTo()` out of box..
Comment: But overwriting `equals()` / `hashcode()` with `BigDecimal` is actually a little risky because someone needs to know that 1.0 != 1.000 by default when using `Objects.equals(1.0, 1.000)` and then I must treat NPE ..so the code above is not easy to have when overriding equals/hashCode for BigDecimal properties..
Comment: first code snippet is exactly what `Objects#equals` does... no need to re-implement it
Comment: @CarlosHeuberger good point, I'll leave it up for posterity. An alternative solution may be setting the scales to match, but I don't like the side effects of that solution.
Comment: I don't like the idea of changing scale, to easy to mess up


How can I turn a list of Aliases into a menu structure?

Question: I'm in a massive migration from a home grown CMS. So far I've been successful at using Feed Import to slurp in CSV exports of the old MySQL database. The old database contained page URLs. So I just mapped this over to Page Alias!
Now I'm searching for a more automated way to take this
/program/science/thesis
/program/math/core_requirements
/program/science/projects
/program/literature/great_works_collection
/program/literature/great_authors_showcase

into a menu structure like this:
Navigation Menu
+- Program
  +-science
    +-thesis
    +-projects
  +-math
    +-core_requirements
  +-literature
    +-great_works_collection
    +-great_authors_showcase

Yes, I could do this manually, but I have over two thousand imported pages.
Is there an Alias URL to Menu structure script or module that may help me along?
Update:
Based on feedback below, I decided to pull a list of URLs from the CSV file using the following pipe line. That is, I pulled only 5th column (the URLs), sorted the output, filtered urls that had "/programs/", and cut off the trailing quote
awk -F ',"' '{print $5}' programs.csv | sort | grep programs | sed "s/\"//g"

After that was exported to a file, a ran the following to generate the child items of the menu:
cat programs.just.urls.txt | awk -F '/' '{print $3}' | egrep -v "php$" | uniq

Answer: I'd give the Menu Export/Import module a go.

This module helps to export and/or import menus. Using it you can
  easily copy menus from one Drupal site to another.
It also allows developers to quickly build a new Drupal site by
  creating empty content and menu items referencing it. The module may
  be used in ensemble with Node export that allows to add content to
  your Drupal site. In this case, menu import allows to batch-arrange
  existing content into a given menu structure, using a very simple text
  file.

I guess you could possibly use Excel or something to take your list of aliases and split the paths to columns which would give you the menu title (in the last column of the split).
From there you have the menu link titles and paths, so it's just a case of getting it into the format that the module expects.
Comment: Interesting, since I already have the data in a CSV (spreadsheet), I could pull that URL column in PHP and create that JSON like structure. That may be worth doing!


Unable to retrieve uploaded file in submit handler

Question: I'm creating a custom form with a file upload field that will have its contents read, processed then sent to an external API. The problem is whenever I upload the file the field for it appears to be empty in the submit handler and I cannot see why.
In this example I'm using it as an unmanaged file as I only need to process it then remove it but I have tried it as a managed file which does upload (I can see it in the directory) and has the file added to the file_managed database table. I'm not sure where unmanaged file are meant to go to check for their upload.
public function buildForm(array $form, FormStateInterface $form_state) {
    $form = [
      '#attributes' => ['enctype' => 'multipart/form-data'],
    ];

    $form['file_upload_details'] = [
      '#markup' => "" . $this->t('The KML File') . "",
    ];

    $form['my_file'] = [
      '#type' => 'file',
      '#description' => $this->t('KML format only'),
      '#upload_location' => 'public://my_files/',
      '#upload_validators' => [
        'file_validate_extensions' => ['kml'],
      ],
    ];

    $form_build = parent::buildForm($form, $form_state);

    return $form_build;
  }

  public function submitForm(array &$form, FormStateInterface $form_state) {
    $messenger = parent::messenger();

var_dump(
  $form_state->getStorage(), // Empty array
  $form_state->getValue('my_file') // Empty string
);
  }

Comment: I know the answer to this, but I learned it from @leymannx, so I'll let him answer it ;-) PS: It's normal for un-managed file to be empty on $form_state
Comment: @NoSssweat – What answer are you referring to? I couldn't remember that myself and had to dig really deep to come up with a new answer.
Comment: @leymannx from this [batch file upload](https://github.com/xurizaemon/csvimport/blob/8.x-1.x/src/Form/CSVimportForm.php)
Comment: @NoSssweat – Oh Jesus, you are so right! And I forgot to merge this PR where I just reminded myself on that: https://github.com/xurizaemon/csvimport/pull/12/files#diff-fe6a8d67c7d2dd996e0e2a2d4a971770R58
Comment: Have you tried `managed_file` instead of `file`?
Comment: @Kartagis I've tried that as well but had the same issue
Comment: Is it to access the $_FILES variable directly?
Answer: All what '#type' => 'file' form elements do is to provide a bare <input type="file"> element. You get a prompt to select a file from your machine and on form submit the browser sends a request to your server, server forwards it to PHP, PHP extracts the file from the request, PHP stores it to the server's temp dir and when the script ends it deletes it again. Not much Drupal involved.
If you need to handle the file somehow and be it just the filename you need to take care of it yourself. And that's what file_save_upload() is for. It expects the $form_field_name as first parameter. And it will return you the file information. When scanning core for '#type' => 'file' you will often see this function being used in the form validation handler. Where the return value is being passed back to $form_state. For example in \Drupal\system\Form\ThemeSettingsForm::validateForm(). Just this time the function is wrapped from _file_save_upload_from_form() to allow correct error handling in forms.

// Check for a new uploaded logo.
if (isset($form['logo'])) {
  $file = _file_save_upload_from_form($form['logo']['settings']['logo_upload'], $form_state, 0);
  if ($file) {
    // Put the temporary file in form_values so we can save it on submit.
    $form_state->setValue('logo_upload', $file);
  }
}

Answer: I have a custom module to import CSV. This works for me.
$form['#attributes'] = [
  'enctype' => 'multipart/form-data',
  'class' => ['csv-import-form', 'mb-50'],
];

$form['csv_file'] = [
  '#type' => 'file',
  '#title' => t('CSV File'),
  '#description' => $this->t('Upload file here.'),
  '#element_validate' => ['::csv_import_validate_file_upload'],
];

public static function csv_import_validate_file_upload(&$element, FormStateInterface $form_state, &$complete_form) {

$validators = [
  'file_validate_extensions' => ['csv CSV'],
];

if ($file = file_save_upload('csv_file', $validators, FALSE, 0, \Drupal\Core\File\FileSystemInterface::EXISTS_REPLACE)) {

  $csv_dir          = 'public://my_files/';
  $directory_exists = \Drupal::service('file_system')->prepareDirectory($csv_dir);

  if(!$directory_exists){
    \Drupal::service('file_system')->prepareDirectory($csv_dir, \Drupal\Core\File\FileSystemInterface::CREATE_DIRECTORY);
  }

  if ($directory_exists) {
    $destination = $csv_dir . '/' . $file->getFilename();

    if (file_copy($file, $destination, \Drupal\Core\File\FileSystemInterface::EXISTS_REPLACE)) {
      $form_state->setValue('csv_file', $destination);
    }
    else {
      $form_state->setErrorByName('csv_file', t('Unable to copy upload file to @dest', ['@dest' => $destination]));
    }
  }
}}

public function submitForm(array &$form, FormStateInterface $form_state) {
    $csv_uri = $form_state->getValue('csv_upload', FALSE);

if ($csv_uri) {
  $csv_data = processCsv($csv_uri);

  if (count($csv_data)) {
    $batch = $this->generateBatchImport($csv_data);
    batch_set($batch);

    $form_state->setRedirect('view.users.page_1');
  }
}}

Answer: I'm using Drupal 9.3.3
I came up with this same problem, while using custom form in a Block. Check this.
/**
 * Provides an 'CareerJobs' List Block
 * @Block (
 *   id = "careerjobs_block",
 *   admin_label=@Translation("CareerJobs Block")
 *   )
 */
class CareerjobsBlock extends BlockBase {

  /**
   * @inheritDoc
   */
  public function build() {
    // return \Drupal::formBuilder()->getForm('Drupal\careerjobs\Form\CareerjobsForm');
    $node = \Drupal::request()->attributes->get('node');
    return [\Drupal::formBuilder()->getForm('Drupal\careerjobs\Form\CareerjobsForm', $node, 'careerjobs_block')];
  }
}

For handling the file.
class CareerjobsForm extends FormBase {

  protected static array $file_ext = ['pdf doc docx'];

  protected static int $file_size = 2048000000;

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state);
    ...
    // buildForm
    $form['resume_upload'] = [
      '#title' => t('Upload your Resume file'),
      '#type' => 'file',
      '#description' => t('Allowed extensions: pdf doc docx.
 Filesize <= 2MB'),
      '#element_validate' => ['::fz_import_validate_file_upload'],
    ];
    ...
    return $form;
  }

  // You can do this in submitForm method also.
  public static function fz_import_validate_file_upload(&$element, FormStateInterface $form_state, &$complete_form) {

    $validators = [
      'file_validate_extensions' => self::$file_ext,
      'file_validate_size' => [self::$file_size],
    ];

    if ($file = file_save_upload('resume_upload', $validators, FALSE, 0, \Drupal\Core\File\FileSystemInterface::EXISTS_REPLACE)) {

      $csv_dir = 'public://careerjobs/resume/x';
      $directory_exists = \Drupal::service('file_system')
        ->prepareDirectory($csv_dir);

      if (!$directory_exists) {
        \Drupal::service('file_system')
          ->prepareDirectory($csv_dir, \Drupal\Core\File\FileSystemInterface::CREATE_DIRECTORY);
      }

      if ($directory_exists) {
        $destination = $csv_dir . '/' . $file->getFilename();

        if (\Drupal::service('file.repository')->move($file, $destination, \Drupal\Core\File\FileSystemInterface::EXISTS_REPLACE)) {
          $form_state->setValue('resume_upload', $destination);
        }
        else {
          $form_state->setErrorByName('resume_upload', t('Unable to copy upload file to @dest', ['@dest' => $destination]));
        }
      }
    }
  }
}

My form is too big to paste here. Only the required part is here.


php date_timestamp_get to seconds

Question: trying to get time in format of seconds in php
$date = date_create();
$now= date_timestamp_get($date);

output $now = 1454706668
i think this microseconds am not sure (http://php.net/manual/en/function.date-timestamp-get.php seem to be missing a bit of info)
i tryd few things non seem to be working right ( google failed me :) )
how do i convert it to seconds?
Comment: That is a unix timestamp, the number of seconds from 1970. Make it the second parameter of http://php.net/manual/en/function.date.php and you can format it how you want..
Answer: Just use, $now = time(); no need to create date.
Answer: use php function
time()

Returns the current time measured in the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT)
http://php.net/manual/en/function.time.php

php date_timestamp_get to seconds


Changing default locations on Git Bash

Question: I am new to using Git Bash and Git Hub. Upon installing Bash, I accepted all of the default settings. In doing so, made my network drive the default directory. I will be using this offline and need to change the default directory to my local drive. I have tried to change the location path of GitBash to the local drive but did this did not work. How can I change this? Also, is there any alterations that need to be made to the GitHub account?
Comment: What about doing a clean re-install of Git bash?
Comment: Possible duplicate of [How do I change the default location for Git Bash on Windows?](https://stackoverflow.com/questions/7671461/how-do-i-change-the-default-location-for-git-bash-on-windows)
Answer: In my case I had to take the following steps to change the default startup directory of the Git Bash in Windows:
1) Right click on the Git Bash icon in your Windows Start menu and select Properties. In Windows 10 you may have to right click on Git Bash -> More -> Open file location -> Right click on the shortcut -> Properties
2) In the Start in field, paste the path to your desired folder, e.g. D:\git.
3) Remove --cd-to-home from the Target field.
Comment: followed the same steps Not working for me any idea, thoughts
Answer: You can modify the shortcut to Git Bash in order to specify the starting directory. This is discussed here.
Alternatively you can create a file called .bashrc in your home directory. That script is executed when the shell starts (rc stands for run-commands), so you can make that file cd into whatever directory you like.
Comment: @siluverukirankumar why not? You should have write permission to your home directory.
Comment: I am able to create another file but I am not able to create .bashrc file
Comment: @siluverukirankumar what error does it give you? Does the file already exist?
Answer: You can also do it from CMD:
start /D "path_target_on_windows_here" git_bash_path\git-bash.exe

It is useful for configuring a shortcut command on Notepad++ :]
Answer: Try creating .bashrc, refer to this solution
https://shawnhymel.com/576/quick-tip-changing-default-directory-of-git-bash/

Changing default locations on Git Bash


Dependent Features and Naive Bayes

Question: Naive Bayes assumes that the features given their classes are independent, and hence :
$$P(y~|~x_1, \ldots, x_n)= \frac{P(y)P(x_1,\ldots, x_n~|~ y) }{P(x_1,\ldots,x_n)}$$
Will become :
$$ P(y~|~ x_1,\ldots,x_n) =\frac{P(y)\prod_{i=1}^n P(x_i~| ~y)}{P(x_1,\ldots,x_n)}$$
That is, due to the assumption that the features conditioned to their class are independent, then we have multiplied each feature conditioned to its class by the other. My question is, if the features are dependent then what is the right way to calculated the features conditioned to their class ?
Comment: There is no *right* or *wrong* way. *Naïve Bayes* is an approximation that is used to solve problems that may be intractable (or intractable in realistic time or with the available computational resources.)
Answer: Even under the Naive Bayes condition, features can be dependent. It is conditioned on the class that the features have to be independent. And if the class conditionals are not independent, your first formula cannot be transformed into the convenient second one. I.e., $p(x_1, \ldots, x_n | y)$ cannot be factorized and you have to try some of the many other classification techniques.

Dependent Features and Naive Bayes


Lombok @Builder not recognised by IntelliJ

Question: I updated IntelliJ IDEA to version 2018.2.7, and Lombok's @Builder annotation is not recognised anymore. The project already had Lombok annotations, which worked fine with previous (2017...) version of IntelliJ.
Only @Builder is failing.
I am using Lombok version 1.14.4.
And Java 11.
The problem is similar to this one:
Lombok not working with Intellij
But the solution provided there is not applicable for my project as my project does not have a submodule and the only pom.xml is referring to the correct project.
The project has Lombok plugin and Annotation processing enabled.

Project's POM where version is configured:
<properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <lombok.version>1.14.4</lombok.version>
        <hibernate-jpa-2.1-api.version>1.0.0.Final</hibernate-jpa-2.1-api.version>
        <assertj-core.version>3.8.0</assertj-core.version>
</properties>

The error message at code level is: Cannot resolve symbol @Builder.
If possible I would like to keep Lombok's and IntelliJ version.
Comment: Allow Annotation Processor in Intellij Idea.
Comment: If possible, can you update Intellij Idea to the lastest version ?
Comment: annotation processor is already enabled.
Answer: Update your lombok version.
Because @Builder gained @Singular support and was promoted to the main lombok package since lombok v1.16.0.
Source
I have
 <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.16.20</version>
            <scope>provided</scope>
            <optional>true</optional>
        </dependency>

and haven't that problem
Comment: I'm using lombok 1.18.0 and IntelliJ doens't recognize method generated by @Singular which is defined on constructor (not on property).
Comment: Did you get solution to this,@Builder was working fine on my intellij earlier, suddenly stopped working, I removed and again installed lombok plugin... all worked :)  using 1.18.16 lombok and latest plugin in intellij community addition
Comment: sure yes, https://plugins.jetbrains.com/plugin/6317-lombok the latest one, any other annotation recognized, only the Singular on Builder constructor doesn't work.
Comment: @emeraldhieu Did you add before that Lombok plugin to Idea?
Comment: This solved the problem for me, and I also found that it worked with Lombok 1.18
Answer: I did no specify the version on my pom.xml and it got the latest version by default(1.18.8 I see this under External Libraries)
<dependency>
  <groupId>org.projectlombok</groupId>
  <artifactId>lombok</artifactId>
  <optional>true</optional>
</dependency>

The Class.builder() was not working for me but after installing the plugin(https://github.com/mplushnikov/lombok-intellij-plugin) it worked.
no extra configuration required. :)
Answer: In my case this is a problem with the version. Honestly. I downgrade to Version 1 and re-sync the Maven repositories:
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.20</version>
        <scope>provided</scope>
    </dependency>

Answer: This suggests to me that your Maven didn't pull the dependency down from your central repository.
Here's a list of things you can try:

Validate that the desired Lombok dependency appears in the central repository.
Validate that the desired Lombok dependency appears in your local .m2.
Re-index your Maven project; make sure auto import is turned on.
Check External Libraries in the left Project tab to make sure that Lombok appears.
Close the project and re-open if there's any red in the Maven tab on the right.

If it's available, IntelliJ will find it.  If IntelliJ can't find it, it's likely that it's not available.
Answer: Upgrading to Lombok 1.18 helped fix the issue, but caused problems that showed  Contructor Not Found on some generated classes.
I had to create a blank lombok.config file in the root of my project that had
lombok.anyConstructor.addConstructorProperties=true
as the only line, then everything built like before.
Answer: Add Latest version of Lombok Dependency in pom.xml,
Go to IntelliJ Settings -> Plugins -> lombok -> Install/Update


Java RSA with exchanged keys in socket connection

Question: I am implementing an RSA Encrypted Socket Connection in Java, for doing that i use two classes the first is the Connection Abstract class which represents the real Socket Connection and the Second is the ConnectionCallback which is a class called when the Connection class receives data.
When data is received by the Connection class, the data gets Decrypted using a before shared public key coming from the connected endpoint (There can only be 1 connected endpoint).
ByteArray class:
package connection.data;

public class ByteArray {

    private byte[] bytes;

    public ByteArray(byte[] bytes){
        this.bytes = bytes;
    }

    public ByteArray(){
    }

    public void add(byte[] data) {
        if(this.bytes == null) this.bytes = new byte[0];
        this.bytes = joinArrays(this.bytes, data);
    }

    private byte[] joinArrays(byte[] array1, byte[] array2) {
        byte[] array = new byte[array1.length + array2.length];
        System.arraycopy(array1, 0, array, 0, array1.length);
        System.arraycopy(array2, 0, array, array1.length, array2.length);
        return array;
    }

    public byte[] getBytes(){
        return this.bytes;
    }
}

Connection class:
package connection;

import connection.data.ByteArray;
import connection.protocols.ProtectedConnectionProtocol;
import crypto.CryptoUtils;
import crypto.algorithm.asymmetric.rsa.RSAAlgorithm;
import protocol.connection.ConnectionProtocol;
import util.function.Callback;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.security.PublicKey;
import java.util.Base64;

public abstract class Connection implements Runnable {

    private DataInputStream in;
    private DataOutputStream out;
    ConnectionProtocol protocol;
    private Callback callback;
    private boolean isConnected = false;

    public Connection() throws Exception {
        this.protocol = new ProtectedConnectionProtocol(new RSAAlgorithm(1024));
        this.callback = new ConnectionCallback(this);
    }

    public Connection(ConnectionProtocol connectionProtocol, Callback callback) throws Exception {
        this.protocol = connectionProtocol;
        this.callback = callback;
    }

    @Override
    public void run() {
        while(isConnected){
            try {
                ByteArray data = new ByteArray();
                while(this.in.available() > 0){
                    data.add(this.read());
                }
                if(data.getBytes() != null){
                    callback.run(data);
                }
            } catch (Exception e){
                e.printStackTrace();
                break;
            }
        }
    }

    protected void openConnection(InputStream in, OutputStream out) throws Exception{
        this.in = new DataInputStream(in);
        this.out = new DataOutputStream(out);
        this.isConnected = true;
        new Thread(this).start();
        this.write(CryptoUtils.encode(((PublicKey) this.protocol.getPublicKey()).getEncoded()));
    }

    private void write(byte[] data) throws Exception{
        System.out.println(new String(data,"UTF-8"));
        this.out.write(data);
        this.out.flush();
    }

    private byte[] read() throws Exception{
        byte[] bytes = new byte[8192];
        int read = this.in.read(bytes);
        if (read <= 0) return new byte[0]; // or return null, or something, read might be -1 when there was no data.
        byte[] readBytes = new byte[read];
        System.arraycopy(bytes, 0, readBytes, 0, read);
        return bytes;
    }

}

ConnectionCallback class:
package connection;

import connection.data.ByteArray;
import crypto.CryptoUtils;
import util.function.Callback;
import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.spec.X509EncodedKeySpec;

public class ConnectionCallback implements Callback {

    private Connection connection;

    public ConnectionCallback(Connection connection){
        this.connection = connection;
    }

    @Override
    public void run(Object data) throws Exception {
        ByteArray bytes = (ByteArray) data;
        byte[] dataToBytes = CryptoUtils.decode(bytes.getBytes());
        if(this.connection.protocol.getSharedKey() == null){
            X509EncodedKeySpec spec = new X509EncodedKeySpec(dataToBytes);
            KeyFactory kf = KeyFactory.getInstance("RSA");
            PublicKey publicKey = kf.generatePublic(spec);
            this.connection.protocol.setSharedKey(publicKey);
        } else {
            //this.so = StrongboxObject.parse(new String(bytes.getBytes()));
        }
    }

}

RSAlgorithm class:
package crypto.algorithm.asymmetric.rsa;

import crypto.CryptoUtils;
import crypto.algorithm.asymmetric.AssimetricalAlgorithm;
import javax.crypto.Cipher;
import java.security.*;
import java.util.Base64;

public class RSAAlgorithm extends AssimetricalAlgorithm {

    private KeyPairGenerator keyGen;

    public RSAAlgorithm(int keyLength) throws Exception {
        super();
        this.keyGen = KeyPairGenerator.getInstance("RSA");
        this.keyGen.initialize(keyLength);
        this.generateKeys();
    }

    @Override
    public void generateKeys() {
        KeyPair pair = this.keyGen.generateKeyPair();
        super.setPublicKey(pair.getPublic());
        super.setPrivateKey(pair.getPrivate());
    }

    @Override
    public byte[] encrypt(byte[] message) {
        try {
            super.cipher.init(Cipher.ENCRYPT_MODE, (PublicKey) super.getSharedKey());
            return CryptoUtils.encode(super.cipher.doFinal(message));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return new byte[0];
    }

    @Override
    public byte[] decrypt(byte[] message) {
        message = CryptoUtils.decode(message);
        try {
            super.cipher.init(Cipher.DECRYPT_MODE, (PrivateKey) super.getPrivateKey());
            return super.cipher.doFinal(message);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return new byte[0];
    }

}

ProtectedConnectionProtocol class:
package connection.protocols;

import protocol.connection.ConnectionProtocol;
import crypto.algorithm.asymmetric.AssimetricalAlgorithm;

public class ProtectedConnectionProtocol extends ConnectionProtocol {

    private AssimetricalAlgorithm algorithm;

    public ProtectedConnectionProtocol(AssimetricalAlgorithm algorithm){
        this.algorithm = algorithm;
    }

    @Override
    public Object getPublicKey() {
        return this.algorithm.getPublicKey();
    }

    @Override
    public Object getPrivateKey() {
        return this.algorithm.getPrivateKey();
    }

    @Override
    public Object getSharedKey() {
        return this.algorithm.getSharedKey();
    }

    @Override
    public void setSharedKey(Object sharedKey){
        this.algorithm.setSharedKey(sharedKey);
    }

    @Override
    public byte[] decrypt(byte[] message) {
        return this.algorithm.decrypt(message);
    }

    @Override
    public byte[] encrypt(byte[] message) {
        return this.algorithm.encrypt(message);
    }

}

CryptoUtils class:
package crypto;

import java.util.Base64;

public class CryptoUtils {

    public static byte[] encode(byte[] data){
        return Base64.getEncoder().encode(data);
    }

    public static byte[] decode(byte[] data){
        return Base64.getDecoder().decode(data);
    }

}

UPDATE of 05/09/2019:
Code update same Exception:
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCcrbJGHqpJdhDbVoZCJ0bucb8YnvcVWx9HIUfJOgmAKIuTmw1VUCk85ztqDq0VP2k6IP2bSD5MegR10FtqGtGEQrv+m0eNgbvE3O7czUzvedb5wKbA8eiSPbcX8JElobOhrolOb8JQRQzWAschBNp4MDljlu+0KZQHtZa6pPYJ0wIDAQAB
    java.lang.IllegalArgumentException: Illegal base64 character 0
        at java.base/java.util.Base64$Decoder.decode0(Base64.java:743)
        at java.base/java.util.Base64$Decoder.decode(Base64.java:535)
        at crypto.CryptoUtils.decode(CryptoUtils.java:12)
        at connection.ConnectionCallback.run(ConnectionCallback.java:21)
        at connection.Connection.run(Connection.java:42)
        at java.base/java.lang.Thread.run(Thread.java:834)

Please help me i am exasperated with this and have only 2 more days of Bounty, i prefer to give my Bounty to someone who helped me finding the solution to this problem than to lose it.
Comment: Encryption: Base64-encode immediately _after_ encryption. Decryption: Base64-decode _before_ decryption. Concerning the order see also e.g. [here](https://stackoverflow.com/a/5760584/9014097).
Comment: Convert the RSA output base64 after encryption.
Comment: Where is the `message`-parameter of `RSAAlgorithm#encrypt` consumed? This should happen in `doFinal`, right? The same applies to the `encrypted`-parameter in `RSAAlgorithm#decrypt`. Also: Base64-encoding in the `Connection`-class is too late! It must _already_ be done in `RSAAlgorithm#encrypt`. Storing the encrypted data in a string doesn't work in general. [Why?](https://stackoverflow.com/a/9098905). Base64-decoding should be performed in `RSAAlgorithm#decrypt`.
Comment: The posted public key is Base64-encoded. The [`X509EncodedKeySpec`](https://docs.oracle.com/en/java/javase/12/docs/api/java.base/java/security/spec/X509EncodedKeySpec.html)-ctor in the `ConnectionCallback`-class expects the raw binary data. Obviously the Base64-decoding is missing! In addition, `RSAAlgorithm#decrypt` uses a wrong cast (use `PrivateKey` instead of `PublicKey`), which would create a `java.lang.ClassCastException` later during decryption.
Comment: @Stecco
In Byte[] encrypt(byte[] message) method, change the return statment and do base64encryption as shown below.

    encodeBase64URLSafe(super.cipher.doFinal(message));

Basically you are trying to base64decode before doing actual decryption.  But you not doing base64encoding after encryption.  Hence its throwing the error.
Comment: @Stecco Ok didn't see that,  Then please remove the below line from onReadFinished method in Connection class
bytes = Base64.getDecoder().decode(bytes);
Because I think you are twice decrypting with base64.  Please check
Comment: Can you show it anyways? isn't whole issue just caused by your read method? you are always trying to read 8192 bytes, so if you will receive bytes in few parts or just smaller chunks then you will end up with few huge byte arrays ending with a lot of 0.
Comment: And then you would try to use base64 on it, and get `Illegal base64 character 0` error
Comment: which is the correct way to do that? i just go with: Base64.getEncoder().encodeToString(encrypted.getBytes(Charsets.UTF_8));?
Comment: [Should be](https://stackoverflow.com/questions/13109588/encoding-as-base64-in-java) See also [What is base 64 encoding used for?](https://stackoverflow.com/q/201479/1820553)
Comment: Could you update your code?
Comment: @kelalaka the code is already updated i have modified the original Connection class inserting the code necessary to transform bytes into Base64 and viceversa.
Comment: @Topaco so do i encode in base64 immediately after to have encrypted/decypted?
Comment: You don't decrypt with a public key. You *encrypt* with a public key, and you decrypt with the corresponding *private* key.
Comment: @user207421 it's what i am doing
Comment: @VinothA thanks didn't see it, i will try!
Comment: @Stecco
Please try the with below statement.  The above statement  will work with apache commons library.  For java util, you have to use the statement as shown below
Base64.getEncoder().encode((super.cipher.doFinal(message)))
Comment: @VinothA wait a moment, in the Connection class in the method write() i am doing the encoding before sending the message should i take that part of the code inside the encrypt method? Also thanks for helping me
Comment: @VinothA yes i can't belive, that is probably the error because i am decoding on the onReadFinished and on the decrypt method i will check it as soon as i can thanks!
Comment: @VinothA no it's not even that the error i don't know why but when i enter in the ConnectionCallback and print the Key is received in Base64 format when i try to decode it it just stops and throws the IllegalArgumentException: Illegal base64 character 0 Exception, at this point if anyone can make the code work i'm fine with it, you can even rewrite it entirely that is not even a problem if it works.
Comment: What is this ByteArray class?
Comment: @GotoFinal it is a class that i have built, it works just like an ArrayList but only for byte[]
Comment: I will show the ByteArray class here as fast as i can today, i didn't thought that receiving small chunks of data would end up filling the array with 0s, how can i just solve this problem? My first attempt would be read the array and put the data in to another array till i find some 0s is this enough? Thanks anyways for the reply, this can probably be the solution
Comment: yep, copying array is enough, but might hurt performance a bit, but that depends if you need to care about performance a lot here. see my answer for me info, and I hope it will be it.
Comment: @GotoFinal if your solution solves the problem you will have my bounty i promise!
Answer: I looked into your code and figured out that the problem is with the add() method in the ByteArray class. Let me show you, (See the comments)
Original : ByteArray
public void add(byte[] data){
    if(this.bytes == null)
        this.bytes = new byte[data.length];
    byte[] bytes1 = new byte[this.bytes.length + data.length];
    for(int i = 0; i < this.bytes.length; i++){
        bytes1[i] = this.bytes[i]; // when this.bytes is null you are adding data.length amount of 0, which is not something you want i guess. This prevents the base64 decoder to decode
    }
    for(int i = 0; i < data.length; i++){
        bytes1[i] = data[i];
    }
    this.bytes = bytes1;
}

Solution: ByteArray
public void add(byte[] data){
    if(this.bytes == null) {
        this.bytes = data; // just store it because the field is null
    } else {
        byte[] bytes1 = new byte[this.bytes.length + data.length];
        for (int i = 0; i < this.bytes.length; i++) {
            bytes1[i] = this.bytes[i];
        }
        for (int i = 0; i < data.length; i++) {
            bytes1[i] = data[i];
        }
        this.bytes = bytes1;
    }
}

public void flush(){
    this.bytes = null; // Important
}

EDIT

After observing the codes that reads bytes in Connection class I found that it's reading unnecessary 0 bytes at the end. So I come up with the following workaround,
Refactor: Connection
...

public abstract class Connection implements Runnable {

...

@Override
public void run() {
    while(isConnected){
        try {
            ByteArray data = new ByteArray();
            while(this.in.available() > 0){
                byte[] read = this.read();
                if (read != null) {
                    data.add(read);
                }
            }
            if(data.getBytes() != null){
                callback.run(data);
            }
        } catch (Exception e){
            e.printStackTrace();
            break;
        }
    }
}

...

private byte[] read() throws Exception{
    byte[] bytes = new byte[this.in.available()];
    int read = this.in.read(bytes);
    if (read <= 0) return null; // or return null, or something, read might be -1 when there was no data.
    return bytes; // just returning the read bytes is fine. you don't need to copy.
}

}
Comment: I tried it is still throwing the Exception can you just download the code and test it please?
Comment: @Stecco I have updated my submission. I refactored the read() method in the Connection class which was reading some unnecessary 0 bytes at the end of the data. I hope this time your exception will be gone. Let me know if it works?
Answer: This is probably caused by your read method:
private byte[] read() throws Exception{
    byte[] bytes = new byte[8192];
    this.in.read(bytes);
    return bytes;
}

You are always reading into array of 8192 bytes, even if there isn't enough bytes in input stream. this.in.read(bytes) returns amount of bytes read, you should use that value and only use that amount of bytes from this array, ignoring the rest - as rest of array will be just 0, so when you try to decode base64 from it you will get java.lang.IllegalArgumentException: Illegal base64 character 0
So when reading your bytes you can just copy them to new array:
private byte[] read() throws Exception{
    byte[] bytes = new byte[8192];
    int read = this.in.read(bytes);
    if (read <= 0) return new byte[0]; // or return null, or something, read might be -1 when there was no data.
    byte[] readBytes = new byte[read]
    System.arraycopy(bytes, 0, readBytes, 0, read)
    return readBytes;
}

Note that reading like that is actually pretty bad idea for performance, as you are allocating a lot of stuff for each read. More advanced libraries like netty have own byte buffers with separate read/write positions and just store everything in single self-resizing array of bytes, but first make it work, and if you will have any issues with performance then remember that this is one of places you might find a solution.
Also in your ByteArray you are coping both arrays into same spot:
    for(int i = 0; i < this.bytes.length; i++){
        bytes1[i] = this.bytes[i];
    }
    for(int i = 0; i < data.length; i++){
        bytes1[i] = data[i]; // this loop starts from 0 too
    }

you need to use i + this.bytes.length in second one. (and it's better to use System.arrayCopy)
public byte[] joinArrays(byte[] array1, byte[] array2) {
    byte[] array = new byte[array1.length + array2.length];
    System.arraycopy(array1, 0, array, 0, array1.length);
    System.arraycopy(array2, 0, array, array1.length, array2.length);
    return array;
}

And then just:
public void add(byte[] data) {
    if(this.bytes == null) this.bytes = new byte[0];
    this.bytes = joinArrays(this.bytes, data);
}

Also like in that other answer - it might be good idea to change flush method to just set field to null, or even better, just remove that method as I don't see it being used, and you could just create new instance of this object anyways.
Comment: I don't know why this is still not working can you just try this code and see what happens? I have tried anything i have updated the question description with the ByteArray class so please check it out, thanks for the help man i appreciate it!
Comment: added note about error in your ByteArray class @Stecco
Comment: Check if the add method now is correct because it is still throwing me the error
Comment: @Stecco nope, you are still overriding one array with another
Comment: At this point i can't belive that is throwing me another time the same Exception, i reupdate the code and the Exception, as you suggested i have deleted the flush() method.
Comment: If you could please download the code and test it by yourself and give me the solution i would thank you A LOT, but don't  worry, i know that this requires time so if you don't want to do it just don't do it.
Comment: @Stecco oh my, I did a small mistake in my code, make sure to `return readBytes;` in your read method