R Reading in files with symbols in Windows

Question: I'm having trouble reading in data from a file with unusual symbols; there is no error message but it stops once it hits a line with a specific symbol.
temp = read.csv(filePaths[i], header=TRUE, sep="\t", comment.char="#")

The last field which is read in is
Familial Non-VHL Clear Cell Renal Cancer;Birt-Hogg-Dub

Reading the file in Excel, this actually reads:
Familial Non-VHL Clear Cell Renal Cancer;Birt-Hogg-Dub-> Syndrome

but the "->" is a symbol; I believe this actually is "Birt–Hogg–Dubé syndrome", and the last character is probably being interpreted as an EOF char.
I only have this problem on Windows.
I've tried using different encoding (encoding = "UTF-8" and encoding = "bytes", fileEncoding = "UTF-8") without any difference.  I've looked at Cannot read unicode .csv into R and searched but can't easily find an answer. Note that I probably can't use a specific language encoding. Thanks!
-- Update --
Created a file with one column, a header, 3 entries (problematic entry at #2), found here:
https://www.dropbox.com/s/3m2wak8rhyab6j2/test.txt?dl=0
Comment: I see. `\032` is making problems. `download.file("https://www.dropbox.com/s/3m2wak8rhyab6j2/test.txt?dl=1", tf <- tempfile(fileext = ".csv"));library(stringi);read.csv(text=stri_read_lines(tf), header=T)` will load it.
Comment: I saw this answered here: http://r.789695.n4.nabble.com/End-of-line-marker-td1577062.html
Comment: Can you post the file (or the relevant part of it) to make your problem reproducible?
Comment: Just did, thanks for any help!
Answer: As you've guessed, you need to change your file encoding when reading your file.

Determine what file encoding you have
Use read.table specifying that file encoding via fileEncoding
Comment: Thanks, via the suggestion, I used Notepad and found that encoding was "ANSI".  When I tried fileEncoding = "ANSI" it gave "Unsupported conversion from 'ANSI' to ''.  Then I tried saving the file changing the encoding from "ANSI" to "UTF-8" but am still encountering errors while using fileEncoding="UTF-8"
Comment: When you opened it in Notepad, was it properly displayed?
Comment: Yes, it displayed all data (past the problematic character) if that is what you mean.


dask how to define a custom (time fold) function that operates in parallel and returns a dataframe with a different shape

Question: I am trying to implement a time fold function to be 'map'ed to various partitions of a dask dataframe which in turn changes the shape of the dataframe in question (or alternatively produces a new dataframe with the altered shape). This is how far I have gotten. The result 'res' returned on compute is a list of 3 delayed objects. When I try to compute each of them in a loop (last tow lines of code) this results in a "TypeError: 'DataFrame' object is not callable" After going through the examples for map_partitions, I also tried altering the input DF (inplace) in the function with no return value which causes a similar TypeError with NoneType. What am I missing?
Also, looking at the visualization (attached) I feel like there is a need for reducing the individually computed (folded) partitions into a single DF. How do I do this?
<code>#! /usr/bin/env python

# Start dask scheduler and workers
# dask-scheduler &
# dask-worker --nthreads 1 --nprocs 6 --memory-limit 3GB localhost:8786 --local-directory /dev/shm &

from dask.distributed import Client
from dask.delayed import delayed
import pandas as pd
import numpy as np
import dask.dataframe as dd
import math

foldbucketsecs=30
periodicitysecs=15
secsinday=24 * 60 * 60
chunksizesecs=60 # 1 minute
numts = 5
start = 1525132800 # 01/05
end = 1525132800 + (3 * 60) # 3 minute

c = Client('127.0.0.1:8786')

def fold(df, start, bucket):
    return df

def reduce_folds(df):
    return df

def load(epoch):
    idx = []
    for ts in range(0, chunksizesecs, periodicitysecs):
        idx.append(epoch + ts)
    d = np.random.rand(chunksizesecs/periodicitysecs, numts)
    ts = []
    for i in range(0, numts):
        tsname = "ts_%s" % (i)
        ts.append(tsname)
        gts.append(tsname)
    res = pd.DataFrame(index=idx, data=d, columns=ts, dtype=np.float64)
    res.index = pd.to_datetime(arg=res.index, unit='s')
    return res

gts = []
load(start)
cols = len(gts)

idx1 = pd.DatetimeIndex(start=start, freq=('%sS' % periodicitysecs), end=start+periodicitysecs, dtype='datetime64[s]')
meta = pd.DataFrame(index=idx1[:0], data=[], columns=gts, dtype=np.float64)
dfs = [delayed(load)(fn) for fn in range(start, end, chunksizesecs)]

from_delayed = dd.from_delayed(dfs, meta, 'sorted')

nfolds = int(math.ceil((end - start)/foldbucketsecs))
cprime = nfolds * cols

gtsnew = []

for i in range(0, cprime):
    gtsnew.append("ts_%s,fold=%s" % (i%cols, i/cols))

idx2 = pd.DatetimeIndex(start=start, freq=('%sS' % periodicitysecs), end=start+foldbucketsecs, dtype='datetime64[s]')
meta = pd.DataFrame(index=idx2[:0], data=[], columns=gtsnew, dtype=np.float64)
folded_df = from_delayed.map_partitions(delayed(fold)(from_delayed, start, foldbucketsecs), meta=meta)
result = c.submit(reduce_folds, folded_df)

c.gather(result).visualize(filename='/usr/share/nginx/html/svg/df4.svg')

res = c.gather(result).compute()

for f in res:
    f.compute()
</code>
Comment: Can you produce a [mcve](https://stackoverflow.com/help/mcve)? I think it would be easier in order to receive help.
Comment: Hi @user32185, I have tidied up my main post and retained the fold function implementation. However, even over simplifying (scoop out everything but the return df at the end) the fold function also does not work. Throws the same TypeError.
Comment: Please do "scoop everything out", so that we are only left with the bare necessities to get to your exception
Comment: @mdurant. Can you please check now?
Answer: Never mind! It was my fault, instead of wrapping my function in delayed I simply passed it to the map_partitions call like so and it worked.
<code>
folded_df = from_delayed.map_partitions(fold, start, foldbucketsecs, nfolds, meta=meta)
</code>


Mass Email with Rails 3

Question: I'm not only new to Rails, but to development in general, so I apologize if this is a basic question. I have an app where I want to send mass emails (ie newsletter) to users that opt-in.  I have two concerns: (1) is my code the most efficient way to process a mass mail (ie do I need to use a for...each statement) and (2) I read other posts on Stack Overflow talking about the importance of the mail server and using delayed jobs to send email.
Other things to note:

I'm on Heroku
I was thinking about using SendGrid
I added delayed job code based on this tutorial

My questions are:

Do most people use delayed jobs even for single recipient emails (ie a user registration confirmation)?
Is my code the most efficient way to do a mass email?
If, for example, there's 1000 users. Does it make a big difference in resources (ie bandwidth, CPU, etc) if I send everyone the same email with a BCC.  If so, do I just pass in a hash to the mailer (ie delete the for...each and change opt_in_user parameter to @opt_in_users) and change the recipient to bcc?

Thanks!
Controller
commmunity_users = UserCommunityRelation.find('community_id = ? and status = ?,'      current_community, "approved" )
@opt_in_users = User.find('opt_in = ?,' true)

@opt_in_users.each do |opt_in_user|
  UserMailer.send_later(:deliver_announcement, opt_in_user)
end

UserMailer.rb
def announcement(opt_in_user)
  mail(:to => "#{opt_in_user.username} <#{opt_in_user.email}>", :subject => "Huge Announcement!")
end

Answer: For bulk sending, you should definitely send it via a service (We use Postbox, SendGrid is another option). In addition to the load on the servers, a service will also probably be better suited to bounces, spam blockers, etc, and it will probably offer extra benefits, such as tracking.

Do most people use delayed jobs even for single recipient emails (ie a user registration confirmation)?

For single user emails, I would go with direct emails in the controller.

Is my code the most efficient way to do a mass email?

You wouldn't want to have a controller action manage queueing the tasks, as it will affect your sites responsiveness. Use a cron job and rake task to do it in the background.

If, for example, there's 1000 users. Does it make a big difference in resources (ie bandwidth, CPU, etc) if I send everyone the same email with a BCC. If so, do I just pass in a hash to the mailer (ie delete the for...each and change opt_in_user parameter to @opt_in_users) and change the recipient to bcc?

I think BCC is fine on resource usage, but your server will be getting hit with any bounces, and it is an extra service to manage, hence, I'd go with a service as mentioned earlier
BTW, for the #each vs for...each question, the cost of the iterator is going to be so low that it isn't worth thinking about IMO.
Comment: I would still use SendGrid even for the one-off emails. Better delivery, primarily, analytics secondarily.
Comment: Also you want to use Delayed Job where possible since you are interacting with an external service and you don't know how long it will take to process - let it happen in the background and return control to the user.

Mass Email with Rails 3


How to calculate width and depth to determine measurements using math (geometry?) along a cone shape

Question: all. I get accused of being too wordy with my posts but I always feel like I need to provide all the details. If you want to go right to my question, just look for "So, my question is..." in bold toward the bottom to get right to it.
Quite embarrassed that I need a refresher on what is probably high school level geometry. I'm in my fifties and was never good at math to begin with. It all went in one ear and out the other. To get to it I'm constructing two identical 10' tall cone-shaped Christmas trees. It's for a vintage modern home and will be pretty much simple, and literally conical in shape. However, it's not a perfect circle. It's elliptical (to save living room floor space). I decided to use rebar because it's cheap and is the ideal length for my very tall trees. My idea is to create wooden elliptical rings and use four rebar rods per cone. I've cut out a notch where I am placing the rebar and holding it in place with galvanized wire. You can see it all in my photo I've provided here.
The bottom ring is 36" wide with a depth of 21.5. I want to place another four wooden elliptical rings, which I'll cut and wire at 2ft intervals going upwards.
Here's why I want to do math to determine the size of my other elliptical rings. At this point, the rebar rods are so flexible that they're sagging toward the middle so I can't get the outer measures to create my other elliptical rings. I also do have some type of straight edge that's long enough to pull the rights straight in order to take the measurements. Not to mention, it would probably take three people to hold the rights perfectly straight while the third person takes the measurement for both major and minor axes.
So, my question is what geometry calculation do I use to determine the sizes of my other rings? If the top (where I'll place the star) is considered to be zero and since my existing bottom elliptical's major axis is 36" and the minor axis is 21.5", and if I move up in 2ft increments, how do I figure out the sizes for my ellipses at 8ft, 6ft, 4ft, and 2ft from the top?
I'm not expecting someone to do all the math for me, but can someone point me in the right direction on how to create either the next ellipse at 8ft (2ft from the bottom) or the 2ft ellipse (measuring from the top downward)? A theorem, perhaps?
Image posted here to provide a visual of what I've created so far:
Comment: I'm voting to close this question because it's a request for geometry, and the math discussion would be a more appropriate place for that.
Comment: Apologies. I didn't know there was a separate department to ask my query.
Answer: You're very much on the right track with the 0 at the top and the 36"/21.5" at the bottom. The sides of the tree will be straight lines, so you can interpolate linearly to find the ellipse size at any height. For a 10' tree, each 2' in height will add 1/5 in width and 1/5 in depth.
So, your total widths/depths will be:

8', 7.2"/4.3"
6', 14.4"/8.6"
4', 21.6"/12.9"
2', 29.8"/17.2".

To be more precise, don't forget the width of the rods - they will add a constant to each layer that you must remove from your elliptical rings' thickness.
The geometry theorem to prove this is the "angle-angle similarity theorem," which states that two triangles with two identical angles will be similar. Corresponding sides of similar triangles always have the same ratio.
Comment: I'm guessing people don't do this but I wanted to share my end result of my project. Thanks to Jacob I was able to make my ellipses and I couldn't be happier with the end result. https://youtu.be/dhFYUs-CCP8?t=275
Comment: Wow. Thanks, Jacob, for doing all the math! Much appreciated. I can see now that it's not visible in my photo but I cut four notches at each axis point on my bottom ellipsis so that the rebar tucks in there without protruding beyond the edges. I also cut them in a "V" pattern to allow me room to tuck the wires inside, so to speak, so that they don't protrude beyond the ellipsis shape. Thanks again!
Comment: Good job, Jacob.
Answer: Can't answer the math easily. However, I have a suggestion:
Instead of working on the tree with the base on the floor, suspend it from the top - e.g., put a hook in the ceiling and hang it from that. Gravity should help keep the rods straight - if not, add a little weight to the bottom ring. Then measure at the desired height and you're done.
Comment: Hey, sounds like you're suggesting I make my tree into the "Modern Christmas Tree" that hangs from the ceiling. That is a cool look but four rebars bundled together at 10' each can weigh quite a bit. Not to mention, since the shape of the tree is an ellipsis it can spin and rotate out of position. I want the widest portion of the base to be parallel to the wall.  Plus, keeping it on the floor helps to hide all the lights and cords I'll have to deal with after I'm all done.
Comment: Hanging is only to make the measurements.


How does one hide the bottom UI bar?

Question: An example of what I'm talking about was seen during the large Roach-Hydra battle in the game between DIMAGA and JulyZerg as part of the GSL World Championship Team League.
Under what circumstances can you use this?
Comment: I noticed that too; it's clearly something built into the game but I haven't seen anything about how to engage it.
Answer: Try pressing Ctrl+W while observing a match. No clue if this will work when playing as it is mostly an observing/replay feature (and I can't test it right now).
Comment: It doesn't appear to work while playing (not that you'd need it to).

How does one hide the bottom UI bar?


Why "if (unsigned int == string::npos )" CAN be true?

Question: Let me start by saying: I'm sorry if this was explained, I tried reading about it and searching for answer but I didn't find one that answered this specific question.
So I'm writing beginner's level code in C++ and I wrote a very simple function that returns index of the next found character that isn't space. It was inspired by the method string.find().
If the function doesn't find anything it returns string::npos.
int findChar (const string &line, const int startPos){
    for (unsigned int i= startPos; i < line.length(); i++){
        if ( line[i] != ' ') return i;
    }
    return string::npos;
}

Now, because I'm working on strings and indices, I thought of using a unsigned int.
unsigned int where;
where = findChar(line, 0);

And because "where" is an unsigned int I thought comparison of "where == string::npos" is a bad idea because string::npos is initialized with -1, and unsigned int can't be equal to -1. But if the function doesn't find anything and it returns string::npos the condition can be true, the program recognizes that "unsigned int where" is string::npos, but when I try to print "where" out, it prints out as biggest number unsigned int can hold.
So my question is why, or how, the program can recognize unsigned int to be equal to string::npos, even though technically string::npos is initialized with value of -1.
I'm trying to learn so if you can point me to some reading material that also answers this I'd be grateful.
Comment: Print `std::string::npos`, you'll be surprised. Note that your code "works" only on 32-bit platforms, where `std::size_t` has the same size as `unsigned int`.
Comment: string::npos is defined as `static const size_t npos = -1;` but `std::size_t is the unsigned integer type of the result of the sizeof operator`  So when -1 is assigned to a size_t object it is converted to an unsigned value with the same bit pattern as -1.
Comment: [1.](https://en.cppreference.com/), [2.](https://www.google.com/search?rlz=1C1CHBF_deDE833DE833&biw=1600&bih=789&sxsrf=ALeKk03cdliNz4aF942LdWI1YSSozg6GNg%3A1611167052843&ei=THUIYN_6MuOMlwS1hoHoDA&q=site%3Astackoverflow.com+%22c%2B%2B%22+std%3A%3Astring%3A%3Anpos&oq=site%3Astackoverflow.com+%22c%2B%2B%22+std%3A%3Astring%3A%3Anpos&gs_lcp=CgZwc3ktYWIQAzoECAAQR1C4-awFWLKkrQVgsrStBWgAcAJ4AIABsQKIAfETkgEIMy4xMC4yLjKYAQCgAQGqAQdnd3Mtd2l6yAEIwAEB&sclient=psy-ab&ved=0ahUKEwjf0O7-kKvuAhVjxoUKHTVDAM0Q4dUDCA0&uact=5), [3.](https://stackoverflow.com/questions/ask) This should be kept in order please.
Comment: That function `findChar` is weird. It returns `int`, while `i` is `unsigned int` and `std::string::npos` is `size_t`.
Comment: @JerryJeremiah The bit pattern may be different (and is, on a 64-bit system, or a non 2's complement system). The conversion is defined in terms of value under modular arithmetic
Answer: If you read the documentation:

Although the definition uses -1, size_type is an unsigned integer type, and the value of npos is the largest positive value it can hold, due to signed-to-unsigned implicit conversion. This is a portable way to specify the largest value of any unsigned type.

Note: std::string::npos is of type size_t. Using anything else may be undefined behaviour and is not advised.
Comment: Using other types might not work, but I think it's not UB per se.
Comment: @tadman UB has a well-defined meaning (perhaps ironically), there can be code which does not behave as intended, but is not Undefined Behaviour.
Comment: @tadman it's not undefined behaviour, regardless of the answer to that question
Comment: @HolyBlackCat Isn't that precisely what "might not work" means in practice?
Comment: @M.M Unless you can find a place where storing `npos` in anything other than its intended type is *defined* then I'd suggest it is *undefined*. I'm sure there's some conversions allowed via other explicit rules, but there aren't many things that are able to represent `size_t` without issues.
Comment: @tadman I can indeed find such place -- out-of-range conversion from unsigned to signed integer type is specified as *implementation-defined*
Comment: @M.M Not sure how that applies in this case when going from, say, `size_t` to some arbitrary-length `unsigned int`. Compiler-specific outcome? I did say it "may" be undefined behaviour to cover this case.
Comment: Out-of-range conversion to unsigned integer is well-defined; the value is adjusted via modular arithmetic until it is in range
Comment: @M.M Not sure there's a situation where `X y = std::string::npos` still preserves `y == std::string::npos` for any `X` not nearly identical to the definition of `size_t`. You can crunch things into range, but can you uncrunch them to match the original out-of-range value during a comparison?
Answer: string::npos is a large positive number .
To avoid these issues, your function's return type should match the type of the value you are returning, e.g.:

return string::size_type instead of int, or:
return some other failure sentinel such as -1 in the first code.

The way you have it now, there can always be a clash when the "adjusted" value to fit in the return type happens to be the same as a valid return value.
Although npos might be initialized with -1 in an implementation header,  that doesn't mean its value is -1. The initializer is converted to the type of the object being initialized, which is string::size_type, guaranteed to be an unsigned type.  The result of this conversion is a large positive value.

NB. Your code will break miserably if called on a string longer than UINT_MAX characters, since the i loop never ends. The i should also be string::size_type rather than int;  and  similar considerations apply to the startPos parameter.


Subtraction problems in JavaScript

Question: So, in my JavaScript web app calculator, the addition works great. But, when I try to code the subtraction calculator, everything goes right, except for when I try the calculator out, the addition works fine, but when I try to subtract two numbers, it just says: NaN
Here's my calculator. The subtraction calculator begins at the paragraph that says "Subtraction Calculator!"
Here's  my code:
<!DOCTYPE html>

Addition Calculator!

    <input id="n1" value=""/>
    <input id="n2" value=""/>
<button type="button" onclick="myFunction()">Submit</button>

Subtraction Calculator!

    <input id="sn1" value=""/>
    <input id="sn2" value=""/>
    <button type="button" onclick="subtract()">Submit</button>

Comment: You probably want to [make a fiddle](http://jsfiddle.net/) to demonstrate this, instead of sharing code in Google Docs. That document may not be available for future visitors finding this question (you might delete it), whereas it will probably live a bit longer on JSFiddle.
Comment: Downvotes don't mean that someone "hates" your question. It means "This question doesn't show any research effort; it is unclear or not useful". Don't take it personally. As for code formatting: http://stackoverflow.com/editing-help#code. There was also a yellow `?` button for the help and a box with tips on the right hand side when you were typing the question.
Comment: @zerkms: http://stackoverflow.com/questions/5588649/how-did-this-person-code-hello-world-with-microsoft-paint
Comment: Why do you have to hate my post.. It's just a question...
Comment: Code in google docs - it's something I see for the first time
Comment: @zerkms I love writing programs in Microsoft Office PowerPoint. Sometimes, Word is better.
Comment: @Ian: initially read it as "MS Paint". PowerPoint isn't that extreme
Comment: @zerkms Well whatever! Maybe I use MacPaint!
Comment: Well, since only one person here kind of helped, guess I'll just go look somewhere else...
Comment: @user2506919: "guess I'll just go look somewhere else" - noooooooooooooooooooooooooooooooooooooooooooooooo
Answer: The problem is this line:
var sn1=document.getElementById("sn2").value;

You have a typo that you're creating a second sn1 variable instead of sn2. Which means on this line:
stotal = ( Number(sn1) - Number(sn2) );

...you might expect a reference error when you use sn2 but the browser is instead picking up the element with id="sn2" which gives NaN when run through the Number() function. And a subtraction involving NaN gives Nan.
So make it sn2 = ... and it'll be fine.
(Noting that as a general rule it is better not to use variable names that are the same as element ids, because then you don't accidentally get this sort of "weird" behaviour.)
Working demo: http://jsbin.com/ivobew/1/edit
Comment: ctrl+f, "typo"; mark as too localized.
Comment: Confirmed that it works after this is fixed http://jsbin.com/opuzoh/1/edit
Answer: Misnamed variable:
function subtract() {
        var sn1=document.getElementById("sn1").value;
        var sn1=document.getElementById("sn2").value;
        stotal = ( Number(sn1) - Number(sn2) );
        document.getElementById("sanswer").innerHTML=stotal;
    }

Second sn1 should be sn2.

Subtraction problems in JavaScript


Why does it take longer for the water in my kitchen to come on after the city worked on drains near my home?

Question: The city recently "did some work" on one of the drains around the corner in my neighborhood. Ever since then, my hot and cold water in the kitchen is taking longer and longer to come on. Started at 5 seconds, now it's 30 seconds.
What can I do to fix this?
Comment: Do you mean it takes a while for the water to get hot, or nothing comes out of the faucet for 30 seconds?
Comment: Does water + air come out of the pipework and then does it splutter?  and eventually only water comes out.  This question doesn't make sense.  You're going to have to clarify what you mean hot and cold water in the kitchen is taking longer to come on.
Answer: Because air has been introduced into the water supply system. This is caused by the pipes being drained because they were cut and mended. So when they turn the water supply on again air pressure stop water from filling the pipes. (Because pipes are airtight and so are your taps)
To remedy this you need to open cold water tap and let it run. Don't runt your hot water tap because you might introduce dirt into your heating system.  You will hear hissing and spewing noise. Once water starts to run let it run for another minute. You might notice muddy water. If you do let it run for another minute or two until the water is clear.
Sometimes taps have filters (aerators) on them and they can get clogged up causing pressure issues. Also if you did run warm water during priming then it is likely that other filters are blocked. You need to find out which ones, clean or replace them now. Most likely a plumber will be needed to find the problem.


ggplot2 Color Scale Over Affected by Outliers

Question: I'm having difficulty with a few outliers making the color scale useless.
My data has a Length variable that is based in a range, but will usually have a few much larger values. The below example data has 95 values between 500 and 1500, and 5 values over 50,000. The resulting color legends tend to use 10k, 20k, ... 70k for the color changes when I want to see color changes between 500 and 1500. Really, anything over around 1300 should be the same solid color (probably median +/- mad), but I don't know where to define that.
I'm open to any ggplot solution, but ideally lower values would be red, middle white, and higher blue (low is bad). In my own dataset, date is an actual date with as.POSIXct() in the ggplot aes(), but doesn't seem to affect the example.
#example data
date <- sample(x=1:10,size=100,replace=T)
stateabbr <- sample(x=1:50,size=100,replace=T)
Length <- c(sample(x=500:1500,size=95,replace=T),60000,55000,70000,50000,65000)
x <- data.frame(date=date,stateabbr=stateabbr,Length=Length)

#main plot
(g <- ggplot(data=x,aes(x=date,y=factor(stateabbr))) +
  geom_point(aes(color=as.numeric(as.character(Length))),alpha=3/4,size=4) +
  #scale_x_datetime(labels=date_format("%m/%d")) +
  opts(title="Date and State") + xlab("Date") + ylab("State"))

#problem
g + scale_color_gradient2("Length",midpoint=median(x$Length))

Adding trans="log" or "sqrt" doesn't quite do the trick either.
Thank you for your help!
Comment: my workaround has been to use a log scale (or something like it) for coloring when I have outliers.  However, I'd love to know if there is a better way!
Comment: Yeah, I had tried that, but it's still off for this example. Hopefully a better way comes up!
Comment: You can use ?cut, to create another variable to your preffered breaks and then set the `color=` aesthetic to that variable.
Answer: Here's one slightly tricky options:
#Create a new variable indicating the unusual values
x$Length1 <- "> 1500"
x$Length1[x$Length <= 1500] <- NA

#main plot
# Using fill - tricky!
g <- ggplot() +
  geom_point(data = subset(x,Length <= 1500),
             aes(x=date,y=factor(stateabbr),color=Length),size=4) +
  geom_point(data = subset(x,Length > 1500),
             aes(x=date,y=factor(stateabbr),fill=Length1),size=4)+
  opts(title="Date and State") + xlab("Date") + ylab("State")

#problem
g + scale_color_gradient2("Length",midpoint=median(x$Length))

So the tricky part here is using fill on points, in order to convince ggplot to make another legend. You can obviously customize this with different labels and colors for the fill scale.
One more thing, reading Brandon's answer. You could in principle combine both approaches by taking the outlying values, using cut to create a separate categorical variable for them, and then use my trick with the fill scale. That way you could indicate multiple outlying groups of points.
Answer: From my comment, see ?cut
x$colors <- cut(x$Length, breaks=c(0,500,1000,1300,max(x$Length)))

g <- ggplot(data=x,aes(x=date,y=factor(stateabbr),color=colors)) +
    geom_point() +
    opts(title="Date and State") +
    xlab("Date") +
    ylab("State")

Comment: In this case, I would have to supply "continuous-looking" colors to a discrete variable with scale_color_manual, right? I'm getting discrete coloring, which isn't bad, just an observation.
Comment: Yes, to fit your original question (red -> white -> blue). Try something like + scale_colour_manual(values=c("red","white","blue")). See here for more effective pallettes: http://learnr.wordpress.com/2009/04/15/ggplot2-qualitative-colour-palettes/ I think the colourspace pallettes designation would likely suit your need best. Just remember that you need a colour for each break you create with cut. It's not hard to fake it so it "seems" continuous, with a bit of clever pallette usage.
Answer: Get rid of the outliers. Quick and dirty, I know, but I think it was worth saying. You can always describe them in your text. Why let them ruin your analyses and graphs?
There's a paper referenced in this blog post which deals with ethically removing outliers:
http://psuc2f.wordpress.com/2011/10/14/is-it-dishonest-or-unethical-to-remove-outliers/
Another simple way of dealing with them would be to cap them:
df$Value[df$Value>1300]=1300
Again, you can describe that you did this in the text or even just edit the scale to say 1300+ instead of 1300


Firebase storage security rules 400 error issue "Permission denied. Could not access bucket ..." (on deployment, whilst working on localhost)

Question: I got an upload file functionality in my application and this is working perfectly fine on localhost. However, when will run firebase deploy and the application is deployed I got the following error with lack of permissions.
Edit: I can deploy, there's no problem with the CLI, just on deployment it doesn't work as on localhost.
Tried:

This answer: https://stackoverflow.com/a/50298931/11127383
Enabling Blaze plan, thus billing too
Test on both domains: xxx.web.app and xxx.firebaseapp.com
Checked Google Cloud Platform Storage, it has exactly the same uploaded files from localhost as the Firebase Cloud Storage

HTML code
<mat-form-field>

        <ngx-mat-file-input
          [accept]="[
            '.doc',
            '.docx',
            '.jpg',
            '.jpeg',
            '.pdf',
            '.png',
            '.xls',
            '.xlsx'
          ]"
          (change)="uploadFile($event)"
          formControlName="fileUploader"
          multiple
          aria-label="Here you can add additional files about your project, which can be helpeful for us."
          placeholder="Additional files"
          title="Additional files"
          type="file"
        >
        </ngx-mat-file-input>
        <mat-icon matSuffix>folder</mat-icon>
        <mat-hint
          >Accepted formats: DOC, DOCX, JPG, JPEG, PDF, PNG, XLS and XLSX,
          maximum files upload size: 20 MB.
        </mat-hint>
        <mat-error
          *ngIf="contactForm.get('fileUploader')!.hasError('maxContentSize')"
        >
          This size is too large,
          maximum acceptable upload size is
            {{
              contactForm.get('fileUploader')?.getError('maxContentSize')
                .maxSize | byteFormat
            }}
          (uploaded size:
          {{
            contactForm.get('fileUploader')?.getError('maxContentSize')
              .actualSize | byteFormat
          }}).
        </mat-error>
      </mat-form-field>

TypeScript (in Angular) code
/**
   * @description Upload additional files to Cloud Firestore and get URL to the files.
   * @param {event} - object of sent files.
   * @returns {void}
   */
  public uploadFile(event: any): void {
    // Iterate through all uploaded files.
    for (let i = 0; i < event.target.files.length; i++) {
      const randomId = Math.random()
        .toString(36)
        .substring(2); // Create random ID, so the same file names can be uploaded to Cloud Firestore.

      const file = event.target.files[i]; // Get each uploaded file.

      // Get file reference.
      const fileRef: AngularFireStorageReference = this.angularFireStorage.ref(
        randomId
      );

      // Create upload task.
      const task: AngularFireUploadTask = this.angularFireStorage.upload(
        randomId,
        file
      );

      // Upload file to Cloud Firestore.
      task
        .snapshotChanges()
        .pipe(
          finalize(() => {
            fileRef.getDownloadURL().subscribe(downloadURL => {
              this.angularFirestore
                .collection('files')
                .add({ downloadURL: downloadURL });
              this.downloadURL.push(downloadURL);
            });
          }),
          catchError((error: any) => {
            return throwError(error);
          })
        )
        .subscribe();
    }
  }

Storage rules
rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
        allow read; // Required in order to send this as attachment.
      // Allow write files Firebase Storage, only if:
      // 1) File is no more than 20MB
      // 2) Content type is in one of the following formats: .doc, .docx, .jpg, .jpeg, .pdf, .png, .xls, .xlsx.
      allow write: if request.resource.size <= 20 * 1024 * 1024
                   && (request.resource.contentType.matches('application/msword')
                   || request.resource.contentType.matches('application/vnd.openxmlformats-officedocument.wordprocessingml.document')
                   || request.resource.contentType.matches('image/jpg')
                   || request.resource.contentType.matches('image/jpeg')
                   || request.resource.contentType.matches('application/pdf')
                                     || request.resource.contentType.matches('image/png')
                   || request.resource.contentType.matches('application/vnd.ms-excel')
                   || request.resource.contentType.matches('application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'))
    }
  }
}

Comment: Deployment isn't affected by security rules.  If your deployment is failing, there's something else wrong.  If the error message coming from the CLI isn't helpful, you should reach out to Firebase support for assistance.  https://support.google.com/firebase/contact/support
Comment: @DougStevenson I can deploy it, but on the deployment it doesn't works. I think my second sentence was confusing, edited it.
Answer: Due to the message you're getting, your issue is related to IAM roles in Firebase.
Here a Quick overview of those.
Check the Error message you are getting and make sure that the instance you're deploying or the service account you're using has the needed roles that let them do the action you want to do.
Answer: Problem solved. The problem was I had GitLab CI pipeline and was injecting the wrong environment variable into the wrong build. If you would have staging and production environment with a similar issue take a look if by mistake you are not injecting staging firebase config into production or vice versa.

Firebase storage security rules 400 error issue “Permission denied. Could not access bucket ...” (on deployment, whilst working on localhost)


For any set $A\subseteq\mathbb{R}^n$, we have $ \overline{A^{\circ}} = \overline{\overline{A^{\circ}}^{\,\circ}}$

Question: I have to prove that for any set $A\subseteq\mathbb{R}^n$,
$$ \large\overline{A^{\circ}} = \overline{\overline{A^{\circ}}^{\,\circ}} $$
This is what I got so far: for any set $A$ I'm using these definitions:
Interior:
$$\exists r > 0\text{ such that }\{x \mid B_r(x) \subseteq A\}$$
Closure:
$$\{x \in \mathbb{R}\mid \exists (X_n) \subseteq A \land X_n \rightarrow x\}$$
Now what I don't get is, I think the right part of the equal, because, I have the interior of $A$, that is all the points that have a ball that is included in $A$, using this I know using the definition of closure that I can pick a sequence that converges to them (using $r$ and decreasing it with $\frac{r}{n}$,  $n\to\infty$ for example). But then I don't know how to take the interior of that, I mean what I'm getting at, is that the closure of the interior is the interior, and then the right part of the equation is trivial, as it is the same (m the interior of the interior is the interior, and its closure its the interior)
I think I'm missing something..
Comment: no, what im trying to proove is cl(int($A$)) = cl(int(cl(int($A$))))
Comment: I've corrected the notation in the question now.
Comment: An equivalent formulation that might be a little easier to work with: If $U$ is an open subset of $\mathbb R^n$, then $\mathrm{cl}(U)=\mathrm{cl}(\mathrm{int}(\mathrm{cl}(U)))$.
Comment: Related: http://math.stackexchange.com/q/266156/
Comment: Please do not use titles consisting only of math expressions; these are discouraged for technical reasons -- see [meta](http://meta.math.stackexchange.com/questions/8891/title-and-latex).
Comment: Yes, thank you and I will do it in the future
Comment: Wouldnt cl($U$)=cl(int(cl($U$))) proove directly that the equality is valid, as cl(int(cl(int($A$))) would be then cl(int(A)). How do i prove that equality (cl($U$) = cl(int(cl($U$))).
Comment: @Aram: "equivalent formulation" just means another way of saying the same thing.  I merely suggest that perhaps thinking of it in that way makes it a little simpler, but it is equivalent to the original problem, not a solution to the problem.  Perhaps it does not help, but it is only a comment.
Comment: A "dual" question: [Idempotence of the interior of the closure](http://math.stackexchange.com/questions/385774/idempotence-of-the-interior-of-the-closure).
Answer: First, we can rephrase the question: Prove that for an open set $U\subseteq\mathbb{R}^2$,
$$\overline{U}=\overline{\overline{U}^{\circ}}$$
For the direction $\subseteq$, we have $U\subseteq \overline{U}^{\circ}$ because the RHS is the maximal open set that is contained is $\overline{U}$. Since taking a closure is monotone (if $A\subseteq B$ then $\overline{A}\subseteq \overline{B}$), we get $\overline{U}\subseteq\overline{\overline{U}^{\circ}}$.
For the direction $\supseteq$, note that for any set $A$ we have $\overline{A}=\overline{\overline{A}}$. Since $\overline{U}\supseteq\overline{U}^{\circ}$ we get $\overline{U}=\overline{\overline{U}}\supseteq\overline{\overline{U}^{\circ}}$.
Comment: Thanks. Note that I only used the topological properties of closure and interior, therefore it is true in any topological space (I see you have mentioned it in your answer).
Comment: Nice. Not one single feather ruffled.
Answer: Interestingly, you can solve this problem by appealing to only a few abstract properties of interior and closure. See this article for more information. Here are three conditions satisfied by the closure operation:

$\mathrm{cl}(S) \supset S$ for all $S \subset \mathbb{R}^n$ (sets get bigger when you close them)
$\mathrm{cl} (\mathrm{cl} (S)) = \mathrm{cl} (S)$ for all $S \subset \mathbb{R}^n$ (the closure of a closed set is itself)
If $S \subset T \subset \mathbb{R}^n$, then $\mathrm{cl}( S) \subset \mathrm{cl}( T)$ (closure preserves containment)

The situation for interiors is nearly the same:

$\mathrm{int}(S) \subset S$ for all $S \subset \mathbb{R}^n$ (sets get smaller when you take their interiors)
$\mathrm{int} (\mathrm{int} (S)) = \mathrm{int} (S)$ for all $S \subset \mathbb{R}^n$ (the interior of an open set is itself)
If $S \subset T \subset \mathbb{R}^n$, then $\mathrm{int}( S) \subset \mathrm{int}( T)$ (taking interiors preserves containment)

Now, let $S \subset \mathbb{R}^n$ be given. We have $\mathrm{cl} (\mathrm{int}(S)) \supset \mathrm{int}(S)$. Now take the interior of both sides to get $\mathrm{int}(\mathrm{cl}(\mathrm{int}(S))) \supset \mathrm{int}(\mathrm{int}(S)) = \mathrm{int}(S)$. Taking the closure on both sides leads to
$$ \mathrm{cl}(\mathrm{int}(\mathrm{cl}(\mathrm{int}(S)))) \supset \mathrm{cl} (\mathrm{int}(S))$$
which is one half what we set out to prove. I encourage you to try to deduce the reverse inclusion using similar methods beginning from $\mathrm{int}(\mathrm{cl}(\mathrm{int}(S))) \subset \mathrm{cl}(\mathrm{int}(S))$.
Comment: I think it's an application of interior(1) to get $int(cl(int(S))) \subset cl(int(S))$, followed by an application of closure(3) and then closure(2) to arrive at $cl(int(cl(int(S)))) \subset cl(int(S))$. Sorry about doubling up on the labels. Software seemed to insist on resetting the count at 1.
Comment: Using the properties you listed i know that int(cl(int($S$))) $\subset$ cl(int($S$)), applying cl to both sides, using 1, and 2 of cl i get cl(int(cl(int($S$))) $\subset $ cl(int($S$)) Is this right?
Comment: Oh yes, i used (1) of int without saying it explicitly. Guess i got confused with all the definitions, because using what you listed it was really straightforward.
Answer: I like to take $ U = \operatorname{cl}(\operatorname{int}(A))$. Then $U$ is a closed set in $\mathbb{R}^n$.
So we have to prove $ U = \operatorname{cl}(\operatorname{int}(U))$.
$\operatorname{int}(U) \subset U $
As $U$ is closed and closer is smallest closed set containing $\operatorname{int}(U)$, we say $\operatorname{cl}(\operatorname{int}(U)) \subset U$.
Take any point $a \in U$.
If $a \in \operatorname{int} (U)$, trivially $a \in \operatorname{cl}(\operatorname{int} (U))$.
If $a \in \operatorname{bd}(U)$, $a$ shall be a limit point of $\operatorname{cl}(\operatorname{int}(U))$.
In any case $a \in \operatorname{cl}(\operatorname{int}(U))$.
Thus $U = \operatorname{cl}(\operatorname{int}(A))$.
Now replace $U$ by $\operatorname{cl}(\operatorname{int}(A))$, and get the result.
Replace notations only.
Comment: @ Sleziak thanks
Answer: Observation 1: for $U$ open, $\bar{U}^o$ is the largest open set contained in $\bar{U}$, so since $\bar{U}$ contains the open set $U$ we have $U \subset \bar{U}^o$. Observation 2: for $C$ closed, $\bar{C^o}$ is the smallest closed set containing $C^o$, so since the closed set $C$ contains $C^o$ we have $C\supset \bar{C^o}$.
Apply observation 1 to $U=A^o$ to get $A^o\subset \bar{A^o}^o$, and take closure on both sides to get $\bar{A^o}\subset \bar{\bar{A^o}^o}$. Now apply observation 2 to $C=\bar{A^o}$ to get the other inclusion.


Rotation matrix for non-isometry transformation

Question: Imagine that you have a sphere in $\mathbb{R}^3$ and a plane (that is parallel to the x,y plane) through the sphere.
Now you want to have a clockwise rotation in the x/y plane that does the following:
The point that ends up after the rotation at the top of the sphere should have been rotated by an angle $\alpha$. The one that ends up at the bottom of the sphere should have been rotated by an angle $\beta$. The ones that end up at the intersection between plane and sphere should have been rotated by $\frac{\alpha + \beta}{2}$. My question is: Can anybody here construct an interpolation of this rotation?(Or even give me a linear transformation that does that?)
Note that although my painting could suggest that the plane goes through the center of the sphere, this is not necessarily the case!
Answer: A few details aren't clear (to me) from context: (i) Are $\alpha$ and $\beta$ specified in advance? (ii) Is the "height" of the cutting plane specified in advance, or can it be chosen once $\alpha$ and $\beta$ are known? (iii) Do all four points in question lie on a great circle perpendicular to the cutting plane (in which case the word "sphere" may as well be replaced by "circle" in the question)?
In any case, assuming I understand the spirit of the question: If you let $P$ be the plane containing your four points and you introduce an angle coordinate $\theta \in [-\pi, \pi]$ in $P$, measured from the center of the sphere and with $\theta = 0$ at the top of the sphere, and if $\pm\theta_0$ denotes the angle where $P$ intersects the cutting plane, you're looking for a monotone function $f:[-\pi, \pi] \to \mathbf{R}$ whose values at four points satisfy
$$f\bigl(-\theta_0 - (\alpha+\beta)/2\bigr) = -\theta_0,\quad
  f(-\alpha) = 0,\quad
  f\bigl(\theta_0 - (\alpha+\beta)/2\bigr) = \theta_0,\quad
  f(\pi - \beta) = \pi.$$
This should be a straightforward interpolation (subject to any other criteria you wish).


Unable to retrieve tagged photo info using FQL

Question: I'm having a hard time running this FQL query.
SELECT pid, src_big FROM photo WHERE pid
IN (SELECT pid FROM photo_tag WHERE subject=me())

The photo_tag table returns a pid that is a much longer string than the usual photo pid.
The inner SELECT statement returns the right number of tagged photos. However, I'd like to get more info on those photos. Any ideas?
Comment: HELP! Tried everything, still the same result :(
Answer: which info do you need? if you want more infos, here is the full list of data you can get from the photo table:
http://developers.facebook.com/docs/reference/fql/photo/
about the pid: yes, that is not the "real" photo id. i think you have to select the column "object_id" for the real id.
"The pid is unique only for a given user." - so you cannot really work with it anyway, i guess.
just include those columns from my link you need, like this:
SELECT pid, src_big, object_id FROM photo...


Running asp.net core Kestrel server on linux as a service

Question: I've set up an Ubuntu 18.04 Linux server, and run my ASP.NET Core 2.1 app on it using Kestrel locally and Nginx as a reverse-proxy.
When I run the app using dotnet run it starts up without a problem and I can access the website, but if I try to create a service per this MS documentation, the server starts up normally (the same DLL, Database, etc) and seems to encounter some exceptions which when trying to access the web page.
    ov 14 07:43:05:       Executed action method AcademiWeb.Controllers.HomeController.Index (AcademiWeb), returned result Microsoft.AspNetCore.Mvc.ViewResult in 0.4463ms.
Nov 14 07:43:05: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Mvc.ViewFeatures.ViewResultExecutor[1]
Nov 14 07:43:05:       Executing ViewResult, running view Index.
Nov 14 07:43:06: #033[40m#033[32minfo#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[2]
Nov 14 07:43:06 :       Executed action AcademiWeb.Controllers.HomeController.Index (AcademiWeb) in 151.9161ms
Nov 14 07:43:06: #033[41m#033[30mfail#033[39m#033[22m#033[49m: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
Nov 14 07:43:06 ]:       An unhandled exception has occurred while executing the request.
Nov 14 07:43:06 : System.ArgumentNullException: Value cannot be null.
Nov 14 07:43:06 : Parameter name: connectionString
Nov 14 07:43:06:    at Microsoft.EntityFrameworkCore.Utilities.Check.NotEmpty(String value, String parameterName)

Is there anything I might be missing here? Something with the appsettings file? Searching online didn't come up with any relevant results.
Comment: What exceptions? Make that clear please. Kestrel has its logging, so enable it and read its log file.
Comment: I've added the exception, when it's checking the connectionString it appears to be null - probably a problem accessing the appsettings.json but I don't know why that problem would occur
Answer: Found the problem - the working directory I set in the service was of the entire solution and not the running project which contains the appsettings.json, so when it was trying to access the connectionString in it the value was null.


Winforms Loading Window doesn't load

Question: In my (C#) application, the user clicks a button in one window, a bunch of time-consuming calculations happen, then another window is shown. I want to put a loading window up during those time-consuming calculations so the user knows something is happening.
Most of these time-consuming calculations are not thread-safe, and much of what I've read says I should use the same thread and create a simple Winform to serve as my loading window. The problem is that the loading window doesn't load. It's a smallish window with a label that says "Loading...". When I call LoadingWindow.Show(); the label doesn't load (there's only a white box where it should be).
I stuck some dummy code in the Shown event handler, but it isn't firing.
Am I doing something wrong? Or is there a way to force the thread to wait until the loading window has loaded completely?
Code:
LoadingWindow loadingWindow = new LoadingWindow();
using (var firstDialog = new FirstDialog())
{
    var firstResult = firstDialog.ShowDialog();
    loadingWindow.Show();
    // do a bunch of time-consuming stuff
    loadingWindow.Close();
}

Comment: We cannot possibly help you without seeing your code.  Add the relevant code to the question.  Please see https://stackoverflow.com/help/mcve for more info.
Answer: You show the firstDialog as a dialog window when you try to show your loading screen. I suggest trying to show your loadingWindow first as a dialog window then the other. Code below.
LoadingWindow loadingWindow = new LoadingWindow();
using (var firstDialog = new FirstDialog())
{
    loadingWindow.ShowDialog();
    var firstResult = firstDialog.ShowDialog();
    // do a bunch of time-consuming stuff
    loadingWindow.Close();
}

Comment: I get the same result when I try that. If I move the `loadingWindow.ShowDialog()` outside the `using` block, it loads as expected.
Answer: I had to refresh the display of the application that is being used for all the time-consuming calculations.

Winforms Loading Window doesn't load


Git Extensions fails to push

Question: I have a local git repo (on Windows machine) with a remote on gitlab.
When I try to push from the Git Extensions GUI, I get this error:
git: 'firstname.lastname@example.com' is not a git command. See 'git --help'.

When I view the command that failed (in the Git Extensions command log) it's:
git push -v --recurse-submodules=check --progress "origin" refs/heads/master:refs/heads/master

But if I run this command from the command line (git bash), it succeeds.
I assume it's a setting/config problem in Git Extensions, but I am unable to fix it.
Two data points:

This used to work until I updated Git Extensions a few weeks back. The current version is 184.108.40.20651, git is 2.32.0.windows.

The URL in the .git/config is
url = firstname.lastname@example.com:the-path-to-my-repo
Answer: The only recent change regarding SSH went into Git Extension 3.5.1 (Jul. 2021)

9112 SSH broken after upgrade to 3.5.0 (due to breaking setting change)

So:

check if 3.5.0 works
check if GIT_SSH was set (PR 9182)

In 3.4 or earlier, the OpenSSH path was set in the registry, even if it was supposed to be empty (handled in #9149).
If the OpenSSH path is corrupted (as reported and discussed in #9112 when upgrading GE) it will just be ignored.
This will also recover from issues if the Git installation is changed.
Comment: Uninstalled 3.5.3, installed 3.5.0, push works now.
Comment: So should I file a bug on 3.5.3?
Comment: @AmirKatz It sure looks like it: you can report it on https://github.com/gitextensions/gitextensions/issues

Git Extensions fails to push


Why does a clean git clone fix a failing build?

Question: I have a project which depends on webpack to build. I have a new branch from a teammember which I checked out and did a git reset --hard on. After running the build, it errors. When I do a clean git clone and check out the new branch, it works. What gives? How can I make it so I don't have to completely clone the project again?
The errors I get are as follows:
ERROR in [at-loader] ./webpack.config.ts:312:5
    TS2339: Property 'replace' does not exist on type '{}'.

ERROR in [at-loader] ./webpack.config.ts:317:103
    TS2339: Property 'value' does not exist on type '{}'.

ERROR in [at-loader] ./webpack.config.ts:545:5
    TS2339: Property 'getProperty' does not exist on type 'LoDashExplicitObjectWrapper<NodeConfig>'.

ERROR in [at-loader] ./webpack.config.ts:545:17
    TS7006: Parameter 'n' implicitly has an 'any' type.

The steps are as follows: (this generates errors)
git checkout extract_css
git reset --hard
yarn
Or, with a clean repo: (this works)
git clone <url> new
cd new
git checkout extract_css
yarn
Comment: Without `-x`, `git clean` will specifically *not* remove explicitly-ignored files. Depending on the build system and the way you use it, that could leave many already-built files that are based on obsolete inputs. After `git clean -fdx` you should have the same situation as a fresh clone (but beware of submodules). (I know nothing of webpack in particular but if it cannot or does not get dependencies correct, this would explain the problem.)
Comment: What error do you get? Did you do the same steps after the `reset` as after the `clone`? One possible reason is that `node_modules` are different and need to be (re-)installed, but without the error, there is no way for us to tell.
Comment: @MichaelJungo I've updated the post! It may be important to mention that our `node_modules` contains a few custom libraries which are committed in our repository, the rest gets populated by `yarn`.
Comment: What does a `git status` give you after the `git reset --hard`. A git reset doesn't clean the directory of files that are not tracked. A `git clean` (https://git-scm.com/docs/git-clean) will set the working directory to be 'exactly' like your repository checkout.
Comment: `git status` = `working directory clean`. `git clean -f`, then `yarn`: same error...
Comment: @torek thank you so much. This has solved my problem. If you want to make this an actual answer I'll gladly select it as the fix!
Comment: @torek unfortunately, this doesn't fix my issue on the CI-part of the deployment cycle. I do the `git clean -fdx` on there, but the same errors appear... Any ideas?
Comment: @torek (again :p): it seems that the `git clean -fdx` did work. My latest issue was because of webpack, which used an older version than I needed. Thanks again for the great help!
Answer: You probably need -x.  The rest of this is expanded from my comment above
It's important to consider precisely which files git clean cleans.  As the documentation says, it:

Cleans the working tree by recursively removing files that are not
  under version control, starting from the current directory.
Normally, only files unknown to Git are removed, but if the -x option
  is specified, ignored files are also removed. This can, for example, be
  useful to remove all build products.

The first paragraph leaves undefined what it means to be "not under version control".  The second paragraph, however, gives us a strong clue, and the description of the -d argument cements it—provided, that is, that we have a clear definition of "untracked":

-d
Remove untracked directories in addition to untracked files. ...

There's a nice, simple Git-specific definition of tracked vs untracked: an untracked file is one that is not in the index (and hence of course a tracked file is one that is in the index).
Remember that Git's index is where you build the next commit.  The index starts out matching the current commit—the one you ran git checkout on, i.e., the tip of the current branch.  Every file that is in that commit, is also in the current index.  Git copied the file from the commit, to the index, and then on to the work-tree, which puts them into a form the rest of the computer can deal with (the internal form inside commits and in the index is essentially a Git-only form).
It's quite normal for the work-tree to hold additional files that are not in the index.  Any time we use programs that build things from our sources—for instance, .o or .so files from C or other compiled code, or .pyc or .pyo byte-compiled Python files, or anything else often called build artifacts these days—we end up with things that are not, and should not be, version-controlled.  We do not add them to the index, so they are untracked.
Git, however, likes to split these up in various ways.  Untracked files can be a nuisance: Git keeps pestering us, warning that this file and that one are untracked, should we maybe add them?  We know we should not, so we tell Git to please shut up about that, by listing the files—by path name, or file pattern such as *.o, or whatever it takes—in a .gitignore file.  That makes Git shut up.
But it also changes the way git clean treats these files.  By default, git clean only removes the ones it would complain about.  Since we mostly tell Git to ignore build artifacts, a plain git clean -f won't remove them.  We need git clean -fx.
Note that when a untracked files live within a directory (or "folder") that has no tracked files in it, Git normally lumps them all together for reporting purposes, and says that the entire directory is untracked.  In this case, git clean -fx will still leave those untouched: you need git clean -dfx to remove them, and then also the now-empty directory itself.  (Note that if there are any tracked files within that directory, this rule no longer applies.)
As the description mentions, this all has to be run from the top level of the work-tree, since it starts "from the current directory".  It's also important to note that any sub-Gits or submodules will prevent git clean from working within that sub-directory; for these cases, it's probably best to repeat the git clean in that separate Git repository.  (E.g., with submodules you could use git submodule foreach to visit each one and clean it.)
Comment: Excellent explanation of the behaviour of git, and why your answer works for my issue. I wanted to be sure I had a clean directory on our CI servers, because our provider doesn't seem to properly clear them once in a while. It's probably fixable without the `git clean -fdx`, but it works. Thanks a lot!


PHP Sessions problems - Checking Login

Question: I'm relatively new to PHP, but I know quite a bit. I am making a login form and I am checking whether they are logged in before they can enter pages and if they are not, then they are redirected to the login page, I am using sessions. Every time that a user logs in or registers it creates a session using:
$_SESSION['username'] = '$username';

On every page that the user is not allowed to access, I include, checklogin.php - a file I made, with this code:
<?php
function checklogin() {
session_start();
if(!isset($_SESSION['myusername'])){
    return false;
    header('location: login.php');
} else {
    return true;
}
}

checklogin(); //checks login

?>

However it doesn't seem to be working as you can access pages when not logged in. Also, there is an if statement:
if(!isset($_SESSION['myusername'])){

This is used to determine what is shown in the header.
None of this is working, so what am I doing wrong? Any help would be greatly appreciated.
Comment: Put `session_start();` at the top of your page, not in your function.
Comment: Give me a minute, will try.
Comment: `$_SESSION['username'] !== $_SESSION['myusername']`...
Comment: Header still isn't working and haven't checked the pages that you can't access yet.
Answer: You need to put session_start() at the top-side of your PHP file, outside of the checkLogin() function. Session variables need to use the same name, so pick between "username" and "myusername". You also need to swap the order of your statements if the user isn't logged in:
 header('location: login.php');
 return false;

The header call is never happening as the function returns before hand. You also do not need quotes around your variables, e.g. $_SESSION['username'] = $username; is correct, assuming $username is defined.
Answer: $_SESSION['username'] = $username;

this is the correct syntax use it..

PHP Sessions problems - Checking Login


Are my trusses capable of carrying the load of insulation and drywall?

Question: We are insulating our garage and need to build a ceiling to separate the attic and the main garage area. The span of the bottom chord of our trusses is 22 feet. Will our truss system support the weight of the drywall and insulation?
Here are some pictures of our garage roof, which was built in 1970.
Comment: Those are factory built trusses. The odds of them _not_ being able to support some insulation and drywall are _very_ low. You'd have to ask the builder (or whoever you bought the trusses from) to know if they're designed to support the _additional_ weight of storage. From what I can see, I'd guess not, but a wide pic showing the full (or at least a half) span would probably give the engineering/builder types here the ability to make a decently educated _guess_. Adding additional ceiling joists for hanging drywall is definitely overkill.
Comment: The main problem you will have is that the span between trusses is fairly large, and drywall is likely to sag (especially if insulation is laid on top of it).  This probably wouldn't be a structural problem but it would be ugly.
Answer: Yes, your trusses will support insulation and gypsum board.
The Building Code requires the bottom chord of trusses and ceiling joists to support a minimum of 10 lbs. per square foot. (See IRC Table R301.5)
Gypsum board weighs about 3 lbs. per square foot and insulation (depending on the thickness and type) weighs between .3 and .5 lbs. per square foot per inch thick.
Your trusses were made by a truss manufacturer certified in your state. This requirement is built into their software for the construction of trusses.
Comment: I would doubt that any builders were using software to design trusses in 1970. I would however, expect that the math used included enough slack for the load.
Answer: You have truss bottom chords not "rafters" nor "joists".
If you have the plans/documents they will undoubtedly list the bottom chord loading in pounds per square foot.
A 22 foot 2x6 will behave like a wet noodle if it's not part of a truss. On the other hand, a 22 foot 2x4 that's part of a truss can carry quite a lot of load.
Answer: Your engineered trusses are almost certainly designed for insulation and drywall loads. I've never encountered a situation where they weren't, and I've been involved in residential construction almost since that garage was built. I wouldn't hesitate to finish the ceiling.
Use no-sag 1/2" drywall to reduce weight if you like. Just use some sense when loading the attic for storage. Spread the load, keep heavier items near the wall, and don't go nuts. A few hundred lbs. is about what you can reasonably do. If you ever get a zillion feet of snow at a time consider removing some near the middle.
Secondly, a proper solution to a situation where you did need to bolster the roof system doesn't involve a bunch more joists. It would likely be a single large beam down the center, directly under the vertical truss members. You'd then still attach drywall to the existing ceiling framing and wrap the beam. No need to lower your entire garage ceiling.
That said, beams under monolithic engineered trusses are very unorthodox. This type of truss has an expected seasonal deflection (called "truss lift")--as they heat and cool and absorb moisture and dry out they rise and fall at the ceiling line. A beam would be likely to have a gap over it half the time.


Using markdown formatting on queried SQL in flask app

Question: In a flask app, I am retrieving some entries from an sqlite database, for display on my html page, code below:
@app.route('/')
def index():
    db = get_db()
    cur = db.execute('select title, text from entries order by id desc')
    entries = cur.fetchall()
    return render_template('index.html', entries=entries)

I would like to use markdown to format the content. I already have markdown installed, and would like to use it on my sql query just like it is used below on raw data.
import markdown
from flask import Flask
from flask import render_template
from flask import Markup

app = Flask(__name__)
@app.route('/')

def index():
  content = """
Chapter
=======

Section
-------

* Item 1
* Item 2
"""
  content = Markup(markdown.markdown(content))
  return render_template('index.html', **locals())

app.run(debug=True)

Which works to yield the chapter/section/item stuff as marked down when it is pulled up in an html template. I do not want to install Flask-Markdown I just want to do this with regular markdown, if possible.
Answer: There are two options:

You can render the markdown before passing it to render_template:
@app.route('/')
def index():
    db = get_db()
    cur = db.execute('select title, text from entries order by id desc')
    entries = [Markup(markdown(entry) for entry in cur.fetchall()]
    return render_template('index.html', entries=entries)

You can register a template filter and then use it in your template:
@app.template_filter("markdown")
def render_markdown(markdown_text):
    return Markup(markdown(markdown_text))

Then, in your template you can just call the markdown filter:
{% for entry in entries %}

    {{entry | markdown}}

{% endfor %}

Comment: Thanks a million.

Simply registering a template filter, and then calling that filter in the template like so {{ entry.text |markdown }} does the trick.

I don't even have to do the entries = [Markup(markdown....] stuff

It works fine with entries =cur.fetchall()


Animate a sprite by using two images

Question: I would have thought this was a common desire, but I'll be damned if I can find this.
I simply want to Animate an image by swapping two images back and forth. I tried StoryBoard, but apparently you can't change the source in a storyboard.  The only answer I can come up with is a Timer, which I don't think is the best way.
Edit: How do I animate image content in WPF? Came up in the "Similiar Questions" window.  And while this will do what I want, it is "hackish" as the OP of it says, and it only swaps two images, what if I wanted a whole sequence of 10 or 20 or 100.
Answer: I found another post here on SO whose answer sounds like it could solve your problem: Change an image during animation using storyboard
The problem with this approach is that it also doesn't satify your request for supporting an arbitrary number of images.
I was going to propose a different method that involves databinding the Image Source to the image path, and then in code behind simply changing the path, but I haven't gotten it to work yet.
Here's an article that seems to have an elegant way to handle your requirements: http://www.codeproject.com/Articles/364529/Animation-using-Storyboards-in-WPF
Hopefully that will work for you.
Comment: The article at the bottom looks good, I will need to try it and see if I can get it to work.

Animate a sprite by using two images


AMP HTML & GTM/Analytics - Tracking iFrame Form Submission

Question: the setup: AMP HTML Page with GTM Integration. The AMP HTML page loads my own form wizard via amp-iframe.
Now i want to track the submissions (from the iframe) via Google Analytics.
Is there any way?
EDIT: On the parent page, GTM is integrated with AMP via the new AMP Support/ AMP Container.
The Question is: How can i track the submissions in the amp-iframe. I can use GTM on the iframe content/child but then GA Tracks 2 Times: ones when AMP Page is loaded with the AMP GTM and then when iframe loads with the "normal" GTM?
Thanks,
Daniel
Comment: Would it be possible to migrate your form wizard to AMP? This would solve your problem and the UX would be better without iframes as well.
Answer: i found a solution: Track the form submission via Measurement Protocol https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#events
and make sure that the cliend id are the same via https://www.simoahava.com/analytics/google-analytics-client-id-amp-pages/
Best,
Daniel
Comment: This is one of the way to do it and ensure that it tracks the action correctly. However, you can also set analytics in the iframe, not track the pageview and only track the form submission event.
Answer: Tag Mangager (with Analytics) ist now Supported in AMP without Iframe. But not with 100% Support.
I think for future Support you should use this solution.


After viewing logs with journalctl, how do I exit the screen that says "lines 1-2/2 (END)"?

Question: When you use the command journalctl -p err -b for example, you get an answer that ends with "END". What command do I use to end this and get the opportunity to enter the next command without having to close the window and open a new one?
erik@server ~ $ journalctl -p err -b
-- Logs begin at sön 2019-09-22 20:17:42 CEST, end at sön 2019-09-22 20:20:01 CE
sep 22 20:17:51 server iscsid[1289]: iSCSI daemon with pid=1290 started!
lines 1-2/2 (END)

Answer: A smooth way to end that command is to hit q (for quit). It looks like it is viewed with the viewer less.
You can quit from this command and several other text mode programs with  q. In this case and several other cases you can also quit with the ctrl C interrupt, but it is 'more brutal'.
Comment: It's using `less`. With `$SYSTEMD_PAGER` and `$PAGER` unset, `journalctl` tries some commands, including (as journalctl(1) says) `less`. But it tries `pager` first. In Debian and Ubuntu, `/usr/bin/pager` is a symlink to `/etc/alternatives/pager`, which is a symlink to `/bin/less` (which users rarely change). So `pidof less` doesn't work but `pidof pager` does. `journalctl` resets `$LESS`, by default to `FRSXMK`, but you can set `$SYSTEMD_LESS`. `FRSXMK` contains `F` but the `F` option only disables paging when neither vertical *nor horizontal* scrolling is needed; see the `S` option. @wjandrea
Comment: It seems like it's actually not using `less` itself but a `less`-related library, cause `pidof less` outputs nothing while it's running, and I tried to disable the paging with `LESS=F journalctl -p err -b` but it didn't work. Compare to `git` where both of those work.
Answer: Read man journalctl. In the Description section, it says:

The output is paged through less by default, and long lines are "truncated" to screen width. The hidden part can be viewed by using the left-arrow and right-arrow keys. Paging can be disabled; see the --no-pager option and the "Environment" section below.

So, you should read man less to learn about this useful tool.
One of the things you can learn from man less is:
   q or Q or :q or :Q or ZZ
          Exits less.

Comment: Worth noting that `man` also runs `less` usually, so you'll need `q` to exit it, too!
Comment: @interfect Per Eliah Kagan's earlier comment on another answer, it would indeed just show the output without paging if it would fit without scrolling, but line wrapping is turned off as well (to avoid ambiguity I guess), and although the example output is not taller than the output terminal, it is wider.
Comment: Do they document why they page 2 lines of output? Git for example is clever enough to tell when it is only going to show you less than a screen of text and not invoke a superfluous pager.
Answer: As mentioned in the other answers you can hit q to exit the less pager.
Assuming that the output is short, another option is to directly require the command not to use the pager. In the case of journalctl this is done with the option --no-pager:
journalctl -p err -b --no-pager


Still struggling with continuity

Question: A function is continuous iff it is continuous at all points in its domain. What confuses me about this definition is that for undefined points in a function, those points are removed from the domain.
Does this mean a function $f(x) = x$ with domain $(-\infty, -10) \cup (3, \infty)$ is continuous even though there's a big blank space between the two pieces?
Does this mean $f(x) = \frac{x-3}{x-3}$ is continuous even though it has a removable singularity at $x=3$?
Would a single point be continuous even though it lacks limits on either side?
Does this mean $f(x) = \tan(x)$ and $f(x) = 1/x$ are continuous even though they have asymptotes where the function goes to infinity?
Does this mean the only functions that are discontinuous are ones with jump discontinuities?
Comment: @HarryAlli The second one is $f(x)=1$ for all $x \neq 3$. At $x=3$ it isn't defined since the denominator is $0$.
Comment: Your first one is continuous. Your second one is just f(x)=1 which has no singularity. If you are referring to a normal rational function, then this function is continuous for all points except at the singularity.
Answer: Terence Tao's book Analysis II makes the following definition (p. 420):

Let $(X,d_X)$ be a metric space, and let $(Y,d_y)$ be another metric
  space, and let $f:X \to Y$ be a function. If $x_0 \in X$, we say that
  $f$ is continuous at $x_0$ iff for every $\epsilon > 0$, there exists
  a $\delta > 0$ such that $d_Y(f(x),f(x_0)) < \epsilon$ whenever
  $d_X(x,x_0) < \delta$. We say that $f$ is continuous iff it is
  continuous at every point $x \in X$.

Folland gives a definition similar to Tao's.
So, using the definition in Tao or Folland, it is correct to say that the functions you mentioned are continuous. For example, using Tao's definition, it is correct to state that the function $f(x) = \tan(x)$ is continuous. (But the Wikipedia article quoted below warns us that not everyone uses Tao's terminology, so we should be careful that the meaning of this statement is clear in context.)
The answer to your final question is no. Picture the function $f$ defined by  $f(x) = \sin(1/x)$ if $x \neq 0$ and $f(0) = 0$.

By the way, we should note that the Wikipedia says the following:

There are several different definitions of continuity of a function.
  Sometimes a function is said to be continuous if it is continuous at
  every point in its domain. In this case, the function $f(x) = \tan(x)$,
  with the domain of all real $x \neq (2n+1)\pi/2$, $n$ any integer, is
  continuous. Sometimes an exception is made for boundaries of the
  domain. For example, the graph of the function $f(x) = \sqrt{x}$, with the
  domain of all non-negative reals, has a left-hand endpoint. In this
  case only the limit from the right is required to equal the value of
  the function. Under this definition $f$ is continuous at the boundary $x= 0$ and so for all non-negative arguments. The most common and restrictive definition is that a function is continuous if it is
  continuous at all real numbers. In this case, the previous two
  examples are not continuous, but every polynomial function is
  continuous, as are the sine, cosine, and exponential functions. Care
  should be exercised in using the word continuous, so that it is clear
  from the context which meaning of the word is intended.
Comment: "The most common and restrictive definition is that a function is continuous if it is continuous at all real numbers." Really?!? I don't think I've ever seen that. The standard definition is "a function is continuous iff it is continuous at every point in its domain", so you should really trust Tao more than Wikipedia here.
Comment: Those are essential discontinuities because they oscillate forever, and that's fine, I did forget to mention those, but I'm mostly interested in the other categories, and more broadly, when a function is considered discontinuous
Comment: @HansLundmark Thanks, I edited the answer to emphasize the Tao / Folland definition and deemphasize the Wikipedia passage.
Answer: Usually continuity is defined for those points at which the function is defined and what occurs outside the domain is not relevant.
Anyway note that in some context a function is defined to be continuos if its domain is an interval, and it is continuous at every point of that interval.
Here you can find a good reference from MIT for a full classification Continuity and Discontinuity.
With respect to your examples, following the definition proposed in the reference

$f(x) = x$ with domain $(-\infty, -10) \cup (3, \infty)$ is
continuous on each interval but it is not a continuos function
$f(x) = \frac{x-3}{x-3}$ has a removable discontinuity at $x=3$ which can be eliminated by definining $f(3)=1$
a single point $(x_0,y_0)$ is a continuous function since $f(x_0)=y_0$
$f(x) = \tan(x)$ and $f(x) = 1/x$ are continuous in the intervals of definition and have infinity dicontinuities
Does this mean the only functions that are discontinuous are ones
with jump discontinuities? No that's only a special kind of discontinuity
Comment: This one says $\tan(x)$ is not continuous for example?
Comment: @user525966 yesexactly, in the sense that it is not continuos in a whole interval but it is continuos in $(-\pi/2,\pi/2)$ for example.
Comment: I would say the definitions in that document are most nonstandard. (Requiring the domain to be an interval? What? And requiring points of discontinuity to be isolated? What?!? Surely everyone would say that every point is a point of discontinuity for the Dirichlet function $\mathbf{1}_{\mathbb{Q}}$?)
Comment: And example 7 ("if $f(x)=|x|$ then $f'$ has a jump discontinuity at the origin") is misleading. Since $f'$ isn't even defined at the origin, one shouldn't even talk about continuity or discontinuity there to begin with (although I confess that I might have committed this sin myself from time to time). Doing so, one contradicts the Corollary to Theorem 5.12 in Rudin's *Principles* (a derivative can't have jump discontinuites).
Comment: @HansLundmark Always there are a lot of discussion on this kind of topic, I really think that it is mainly a fact of definitions which depends upon the particular context. Thus it is important to keep in mind that we need to be aware about this as we can have different definitions and related interpretation.
Answer: It is because we get a little bit loosey-goosey with the concept of domain.
We had too many questions in Algebra class / pre-calculs class were we were asked what is the domain of $f(x).$  i.e. the domain of $f(x) = \frac 1x$ is $\mathbb R - \{0\}.$  But, it is really the responsibility of whoever it is that defines the function to also define the domain. It could be that there are points in the domain where the function is not defined.  When we get to these continuity questions, we are often being asked to include points in the domain where the function may, in fact, be undefined.
$f(x) = \frac 1{x}$ is continuous on $(-\infty,0)\cup (0,\infty),$ but it is not continuous on other domains such as $[-1,1]$ or $\mathbb R.$
Comment: "It could be that there are points in the domain where the function is not defined." No, the domain of a function is (by definition!) the set of points where it's defined. It doesn't make sense to talk about "$f(x)=1/x$ on the domain $[-1,1]$".
Comment: Does it even make sense to use a different domain though, since this can include points that aren't even defined?
Comment: If I define my function $f:\mathbb R \to \mathbb R$ then the domain is $\mathbb R$ even if there are points where the function is infact undefined.  We say a function is a mapping from the domain to the co-domain where every input has "at most one" output.  There could certainly be no output for a given input and still be a valid function.
Comment: So if we don't explicitly give the domain then we generally assume the domain is wherever it's defined -- and that it's probably going to be a continuous function granted no jump or essential discontinuities, but if we explicitly make the domain include undefined points, then it won't be continuous anymore for sure?
Comment: That is exactly how I think about it.

Still struggling with continuity


Drupal 6 content reference

Question: I have sports contents, so I create 2 content types season and teams, I added a reference field to the season content type which get the list of teams to select the teams in this season...
Now I need to create a view to list teams under specific season, so I know I should use relationship and argument but I failed to do it
So I need the steps to create a view to list teams under specific season
Answer: View should look something like this:
filter:

node:type = season

Relationship:

Relation to team

fields:

node: title (for season)
node: title (for team) + check to use relation to team

now about selecting season you can use: exposed filter for season OR you can use arguments
Settings for arguments:

In arguments field select content(season)
In path set structure of your URL with % in place of argument (season/%)

Hope this helps

Drupal 6 content reference


ng-route not working. I dont know what's the error because nothing is showing in the console

Question: So here's my boot.php where I have set the links

Valley

beach

<ng-view>
</ng-view>

Now my angular module file is in js/angular-app.js
var loc = angular.module('loc', ['ngRoute']);

Now in js/router.js
loc.config(function ($routeProvider) {
    $routeProvider
          .when('/', {
              redirectTo: '/valley'
          })
          .when('/valley', {
              templateUrl: 'js/angular/index.php',
              controller: 'js/angular/restaurantController.js'
          })
          .when('/beach', {
              templateUrl: 'js/angular/index.php',
              controller: 'js/angular/restaurantController.js'
          })
});

But when click on the links nothing is happening.
Comment: check the net panel from console there may be a continues loop i guess because of this `redirectTo: '/valley'`. If it happens change it as `redirectTo: '/#/valley'`
Comment: What do you mean nothing is happening? Does it redirect to `/valley`?
Comment: no it is not redirecting to valley
Comment: why are not using `.otherwise` to redirect `'/valley'` instread of `.when`
Answer: Check your templateUrl and controller path if they are correct. You can try both absolute and relative path.


Absolute positioning and Z-Index stacking

Question: I am trying to recreate so called skill-cirle which is popular in CV templates.
Like this

I am trying to achieve this using following code, I hope you will get an idea.
CSS
/* +++++ SKILL MODULES +++++ */
.skill-item {
  width: 100px;
  height: 100px;
  text-align: center; }
  .skill-item__title {
    color: #494949; }
  .skill-item__circle {
    border-radius: 50%;
    position: relative;
    display: table;
    height: 100%;
    width: 100%;
    background-color: #52b3d9;
    overflow: hidden; }
    .skill-item__circle-progress {
      position: absolute;
      bottom: 0;
      width: 100%;
      height: 66px;
      background-color: #68c3a3; }
    .skill-item__circle-percent {
      display: table-cell;
      width: 100%;
      vertical-align: middle;
      color: #494949; }

/* ----- SKILL MODULES ----- */

HTML

66%

HTML5 + CSS3

Here is my result

And JSFiddle Link
As you can see my absolute positioned block overlays text block, but I need this to work vice versa. Percent text with stacked on top of the background progress block.
How can I achieve this ?
I would be grateful for any help or suggestion how to do this better.
Comment: From this question alone, I feel like your circle is a little too full for HTML5 + CSS3.
Answer: I think this is what you want. If you just add position/z-index you can place anything on anything.
https://jsfiddle.net/znc6wk36/1/
.skill-item {
  width: 100px;
  height: 100px;
  text-align: center; }
  .skill-item__title {
    color: #494949; }
  .skill-item__circle {
    border-radius: 50%;
    position: relative;
    display: table;
    height: 100%;
    width: 100%;
    background-color: #52b3d9;
    overflow: hidden; }
    .skill-item__circle-progress {
      position: absolute;
      bottom: 0;
      width: 100%;
      height: 66px;
      background-color: #68c3a3; }
    .skill-item__circle-percent {
      position: relative;
      display: table-cell;
      width: 100%;
      vertical-align: middle;
      color: #494949;
      z-index: 10;}

66%

HTML5 + CSS3

Comment: Thanks, it works. By the way `z-index` doesn't work without `position` property.
Answer: Just add postion: relative; to .skill-item__circle-percent and there you go!
.skill-item__circle-percent {
  display: table-cell;
  width: 100%;
  vertical-align: middle;
  color: #494949;
  position: relative;
}

Please let me know your feedback. Thanks!

.skill-item {
  width: 100px;
  height: 100px;
  text-align: center;
}
.skill-item__title {
  color: #494949;
}
.skill-item__circle {
  border-radius: 50%;
  position: relative;
  display: table;
  height: 100%;
  width: 100%;
  background-color: #52b3d9;
  overflow: hidden;
}
.skill-item__circle-progress {
  position: absolute;
  bottom: 0;
  width: 100%;
  height: 66px;
  background-color: #68c3a3;
}
.skill-item__circle-percent {
  display: table-cell;
  width: 100%;
  vertical-align: middle;
  color: #494949;
  position: relative;
}

66%

HTML5 + CSS3

EDIT:
This solution works as positioned elements (that are not static) are displayed above the elements that are not (static).
Comment: Thanks, the `position` property is the key.
Comment: Could you please explain, why I got such behavior, without specifying `position`, this because of default `static` ?
Comment: I guess so because positioned elements (that are not static) are displayed above the elements that are not (static).


What operations have represented 2^2?

Question: I knew = 2.2 and 2.2= 2+2
What operations have represented 2^2?
? =
3+3+3=9 ; 3.3.3=81 and continue is .
 what is representation for operations  ? I assume to have a operation, it is called f. I have 3f3= ; =3.3.3 ; 3.3= 3+3+3; what is representation for operations f ?
Comment: Refer to [this question](http://math.stackexchange.com/questions/1849528/what-is-this-operator-called/1849531#1849531)
Comment: $3^2 - 5{}{}{}{}$
Comment: Maybe the link to this question can help you?
http://math.stackexchange.com/questions/76156/who-introduced-the-notation-x2?rq=1
Comment: You need to learn to format your posts. Formatting tips [here](http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference).
Comment: May be it's the [tetration](https://en.wikipedia.org/wiki/Tetration) operation? Seems to me like the person is asking for $\text{plus}(\text{plus}(3,3),3)$ being multiplication, $\text{multiplication}(\text{multiplication}(3,3),3)$ being exponentiation and then moves on to asking what $\text{exponentiation}(\text{exponentiation}(3,3),3)$ is called.
Answer: I am not sure that I understand your question. The expression 2^2 is the same as $2^2$ and itmeans multiplying $2$ by itself $2$ times. So
$$
2^2 = 2\cdot 2 = 4.
$$
Now it happens that $2$ plus $2$ is also equal to $4$. It is not true in general that $n^m = n + m$. For example $3^2 = 3\cdot 3 = 9$ and $3 + 3 = 6$.
Comment: Yes i want to ask if 3+3+3=9 ; 3.3.3=81 and continue is 3^3^3. what is representation for operations 3^3^3 ? I assume to have a operation, it is called f. I have 3f3=3^3^3 ; 3^3=3.3.3 ; 3.3= 3+3+3; what is representation for operations f ?
Comment: @QChíNguyễn Tetration.

What operations have represented 2^2?


How to convert datasets from CSV to JSON with nested elements?

Question: I have the following csv file:
LID,Name,age,CID
122,David,29,ECB4
122,Frank,31,ECB4
567,David,29,ECB4
567,Daniel,35,ECB4

I want to group the data first by the CID and later by the LID and save them as json so that they have kind of that structure:
{
    "CID": "ECB4",
    "logs":[ {
            "LID":122,
            "body":[{
                   "name":"David",
                   "age":29
                  },
                  {
                   "name":"Frank",
                   "age":31
                  }
                 ]
            },
             "LID":567,
              "body":[{
                   "name":"David",
                   "age":29
                  },
                  {
                   "name":"Daniel",
                   "age":35
                  }
                 ]

              }

         ]
}

I have already defined a schema and loaded the data into a dataframe:
 sparkSession.sqlContext.read.format("csv")
  .option("delimiter",",").schema(someSchema).load("...")

But I have no idea how to group the dataframe in the wanted way. The groupBy functions returns a RelationalGroupedDataset which I can not save as json.
A sql query wants that I use an aggregation after grouping.
I would appreciate any help.
Answer: groupBy defines the groupings alone that you can later use to run aggregations upon. In order to have the result saved in JSON format you have to define the final action that will act on the groupings.

groupBy(col1: String, cols: String*): RelationalGroupedDataset Groups the Dataset using the specified columns, so that we can run aggregation on them.
See RelationalGroupedDataset for all the available aggregate functions.

In other words, you have to execute aggregations using RelationalGroupedDataset interface from which you can use the most generic agg operator.

agg(expr: Column, exprs: Column*): DataFrame Compute aggregates by specifying a series of aggregate columns.

If I'm not mistaken (by looking at the output JSON file), you do groupBy to collect the name and age fields per LID.
You should do the following then:
// Load your dataset
val cids = spark.read.option("header", true).csv("cids.csv")
scala> cids.show
+---+------+---+----+
|LID|  Name|age| CID|
+---+------+---+----+
|122| David| 29|ECB4|
|122| Frank| 31|ECB4|
|567| David| 29|ECB4|
|567|Daniel| 35|ECB4|
+---+------+---+----+

With the dataset you have to first struct the columns name and age as they participate in the aggregation.
val name_ages = cids.withColumn("name_age", struct("name", "age"))
scala> name_ages.show
+---+------+---+----+-----------+
|LID|  Name|age| CID|   name_age|
+---+------+---+----+-----------+
|122| David| 29|ECB4| [David,29]|
|122| Frank| 31|ECB4| [Frank,31]|
|567| David| 29|ECB4| [David,29]|
|567|Daniel| 35|ECB4|[Daniel,35]|
+---+------+---+----+-----------+

Now, it should be fairly straightforward.
val logs = name_ages.groupBy("CID", "LID")
  .agg(collect_list("name_age") as "logs") // <-- that's the missing piece in the puzzle
scala> logs.show(truncate = false)
+----+---+-------------------------+
|CID |LID|logs                     |
+----+---+-------------------------+
|ECB4|122|[[David,29], [Frank,31]] |
|ECB4|567|[[David,29], [Daniel,35]]|
+----+---+-------------------------+

Save away...(left as a home exercise :))

Hint: You may want to use struct once more.


How to check Qt is working or not

Question: I installed Qt version 4.8.5 in my ubuntu machine and how can i test a sample program. Can anyone provide me a sample .pro file and .cpp file ? or the sample of needful files to test Qt ?
Answer: You can find many examples from Qt's website:
http://doc.qt.io/qt-4.8/all-examples.html
QML examples:
http://doc.qt.io/qt-4.8/qdeclarativeexamples.html

How to check Qt is working or not


Align types in type annotation Python

Question: I want the arguments of a function to be a list and a Callable that can be called upon the type of list elements, i.e.:
import typing as tp

def foo(list_: list[T], func_: tp.Callable[[T], tp.Any]):
    func_(list_[0])

I don't care what the type T is, I only care if func_ can be called upon T.
Is there a way to annotate the function and not require a specific type of list elements?
Comment: If you add `T = tp.TypeVar("T")`, what you've posted [_works_](https://mypy-play.net/?mypy=latest&python=3.10&gist=db7ba1936bb3f501ee338fcd6a49b6dd). That _is_ how you describe the relationship between the elements in the list and the parameter of the function, it _doesn't_ require a specific type of the list elements.
Answer: You need to make T a TypeVar:
from typing import Callable, TypeVar

T = TypeVar("T")
U = TypeVar("U")

def foo(arr: list[T], func: Callable[[T], U]) -> None:
    func(arr[0])

Within a given call to foo, the arr and func arguments must have the same T:
foo(["asdf", "foo"], lambda x: x.replace("a", "b"))  # ok
foo([1, 2, 3], lambda x: x + 1)                      # ok
foo([1, 2, 3], lambda x: x.replace("foo", "bar"))    # error

I suggest using another TypeVar as a placeholder for the return value because Any can easily lead to uncaught errors in the future, whereas a TypeVar with no bound will error if you make bad assumptions about what it supports:
def foo(arr: list[T], func: Callable[[T], Any]) -> None:
    x = func(arr[0]) + 2  # ok  (danger!)

def foo(arr: list[T], func: Callable[[T], U]) -> None:
    x = func(arr[0]) + 2  # error  (good!)

If you intend for func to always be a function that returns None, just specify None instead of either U or Any.

Align types in type annotation Python


What Do I Get If I Connect a Uplay Account?

Question: Just Dance 2014 (on the Xbox 360 if it matters) wants me to create a Uplay account (for Ubisoft) and connect it to the game.  What will be the in-game benefits if I do this? Will I get free songs or other DLC? Are there other ways to get those goods?
Answer: According to Uplay's about page you can earn Units while playing games which you can exchange for in-game or real life rewards.
This page says that on the Xbox 360 you can earn (Click on the XBox 360 logo):

Follow the leader sweat version
Just Dance 2014 Theme
Just Dance 2014 Avatar
Fine China - Dance Mash-up


Let $T:V\rightarrow V$ be a linear transformation such that $T^2=2T$

Question:
Let $T:V\rightarrow V$ be a linear transformation such that $T^2=2T$.
Prove the following statements:

If $B=(b_1,...,b_k)$ is a basis of KerT and $C=(c_1,...c_{n-k})$ is a basis of ImT, then $(b_1,...,b_k,c_1,...,c_{n-k})$ is a basis of V. (I assumed dimV=n)
Show that there exists a basis of V, A, such that the representing matrix of T- $[T]_A$ is diagonal and the diagonal entries are all in {0,2}

Hi all. I've only just started my Linear Algebra course and we are yet to learn diagonalizing matrices (and shouldn't use this method for this problem).
for 1, I assumed $(b_1,...,b_k,c_1,...,c_{n-k})$ is a linear dependent set and therefore there exists $\beta_j\neq 0$ such that $-\sum_{i=1}^{k}\frac{\alpha_i}{\beta_j}\vec b_i-\sum_{i\neq j}^{n-k}\frac {\beta_i}{\beta_j}\vec c_i = \vec c_j$. I've reached the point where I show that $T(\vec c_j) = \sum_{i\neq j}^{n-k}\frac {\beta_i}{\beta_j}T(\vec c_i)$
How do I continue from here? Would like some help. (and also on question 2)
thanks in advance :)
Answer: Let $v\in V$ and you want to write down $$v=a_1b_1+\cdots+a_kb_k+d_1c_1+\cdots+d_{n-k}c_{n-k}$$
for some scalars $a_i,d_i$ in base field of vector space.
As $v\in V$, we have $T^2(v)=2T(v)$ i.e., $T(T(v)-2v)=0$ i.e., $T(v)-2v$ is in kernel of $T$.
So you have
$$T(v)-2v=a_1b_1+\cdots+a_kb_k$$
for some scalars $a_i$ in base field.
Can you continue from here?
Suggestion : If you are not into Physics completely, it is good to write $v,w$ for elements of vector space than $b_i,c_i$ and not to write those arrows above elements of vector space.
Answer: I like to do things as geometrically as possible as long as possible. That means writing things in terms of vectors and linear maps, not in terms of bases, coordinates and matrices.
Take an $x\in V$ such that $x$ is both in the kernel and in the image of $T$. Then there is a $y\in V$ such that $x = Ty$, and we have $$0 = Tx = T^2y = 2Ty = 2x$$so $x = 0$. Since $x$ was arbitrary, we must have $\ker T\cap \operatorname{im}T = \{0\}$. Because the $b_i$ are linearly independent, and the $c_i$ are linearly independent, the entire collection is linearly independent. Since we have a collection of $n$ linearly independent vectors, we have a basis of $V$.
For the second point, note that $Tb_i = 0$ and $Tc_i = 2c_i$. Also, recall that the matrix representation of a linear map is given by the columns being the images of the basis vectors, expressed in that basis. Thus we can simply use the basis consisting of $b_i$'s and $c_i$'s.
Answer: Indeed, we need to show that the set $B \cup C$ is linearly independent.  It seems that you wanted to prove this by contradiction (i.e. starting by assuming that $B \cup C$ is linearly dependent), but I don't like this approach.
A nicer approach here is to simply show that if $x \in \ker T$ and $y \in \operatorname{Im}(T)$ are such that $x + y = 0$, then it must be that $x = y = 0$ (to put it another way: we'd like to show that $\ker T \cap \operatorname{Im}(T) = \{0\}$). With this shown, we can conclude that if
$$
\sum \beta_i b_i + \sum \gamma_i c_i = 0
$$
then it must be the case that $\sum \beta_i b_i = 0$ and $\sum \gamma_i c_i = 0$, which (by the linear independence of the individual bases) implies that all $\beta_i$ and $\gamma_i$ are $0$.  So,

Claim: $x \in \ker T$ and $y \in \operatorname{Im}(T)$ are such that $x + y = 0$, then it must be that $x = y = 0$

Proof of claim: Let $v$ be such that $y = Tv$. Note that $x + y = 0$.  Thus, $T(x + y) = T(0) = 0$.  So, we have
$$
0 = T(x + y) = Tx + Ty = 0 + T(Tv) = T^2 v = 2Tv = 2y
$$
So, $2y = 0$, which means $y = 0$.  Since $x + y = 0$, conclude that $x = 0$ as well. $\square$
Next, we need to show that every vector in $v \in V$ can be written in the form $v = x + y$ where $x \in \ker(T)$ and $y \in \operatorname{Im}(T)$.  There are several approaches that work here, but there are two I like: once is to simply argue that $\ker(T) \cap \operatorname{Im}(T) = \{0\}$, which means that $\ker(T) + \operatorname{Im}(T)$ is necessarily an $n$-dimensional subspace of $V$.  The second approach is to note that every $v \in V$ can be decomposed as
$$
v = \underbrace{\left(v - \frac 12 Tv\right)}_x + \underbrace{\frac 12 Tv}_{y}
$$

Part 2 is easy if you understand what it means to write the matrix of a transformation with respect to a particular basis.  In particular, you should argue that the matrix of $T$ with respect to $A = (b_1,...,b_k,c_1,...,c_{n-k})$ is of the desired diagonal form.
Answer: For the first item you need to prove that Ker$T$ $\cap$ Im$T=\{0\}$. Let $\{b_1,\dots, b_k\}$ be a basis for Ker$T$, and let $v\in$ Im$T$. Suppose $v\in$Ker$T$ $\cap$ Im$T$, then $v=a_1b_1+\cdots+a_kb_k$, Hence
$$T(v)=\sum_{i=1}^k a_i T(b_i)=0$$
On the other hand, since $T^2=2T$ and $v\in$ Im$T$, there exists $u\in V$ such that $T(u)=v$, hence $T^2(u)=T(v)=0$ and $T^2(u)=2T(u)$, that is $T(T(u)-2u)=0$. This implies that $T(v-2u)=T(v)-2T(u)=0$, hence $2v=0$, so $v=0$.
For the second point, the matrix associate to the basis $\{b_1,...,b_k,c_1,..,c_{n-k}\}$ is diagonal, because $T(b_i)=0$ and $T(c_j)=2c_j$.