Survival analysis if one group is all censored?

Question: I'm analyzing 2 groups where all of the subjects in one group are still alive, i.e. one group is entirely right-censored. The other group is a mix of censored and uncensored. Is a Cox proportional hazards model valid to use here? I suspect it fails the proportional hazards assumption, plus Matlab's coxphfit function fails in this case, saying it can't calculate the baseline cumulative hazard. Is there a better alternative? Any insight would be appreciated.
Answer: It is not so much that proportional hazards fails in this case, but rather, proportional hazards becomes an untestable assumption if all the data is censored.  As a general principle, you estimate the hazard function by looking at failures over time; if there are no failures in the observation period then it is not possible to make robust inferences about the "shape" of the hazard function with respect to the explanatory variables, and thus it is not possible to robustly diagnose whether the proportional hazards assumption holds.
In relation to the secondary matter you mention, the absence of any failures in the observation period means that the MLE for the hazard function occurs in an edge case --- i.e., hazard being zero for that group, or equivalently, coefficients for the explanatory variables being infinite.  This is probably what is leading to the failure of convergence of the fitting function.  There are a number of ways you could deal with this; either you could change your fitting method (e.g., to Bayesian estimation) or you could consider the properties of the fitted function at the edge case.


How can I troubleshoot a U0208 DTC code on a 2009 Land Rover LR2?

Question: How can I troubleshoot a code U0208 on a 2009 Land Rover LR2? It has a 3.2L engine. Thanks!
Comment: @cdunn I don't think this is really that broad we have a code, engine, make and model. Maybe resolve isn't the correct word for the question, but it's fairly specific.
Comment: Your question has come up for deletion because it is very broad and answering it is beyond the scope of what we do here. What you can do, is do a web search for your make and model of car and this DTC and see what you find. When you have specific questions about what you find, then the question would be accepted. And thanks for contributing and we look forward to seeing more information on this so you can get the resolution you need.
Comment: So called "Universal" codes usually involve faults with the communication bus,and the OBD system itself - an auxiallary system unrelated to vehicle control, emissions, or fueling.   It's a rare beast.  Verify that you *really do have this code* before you spend too much effort.
Answer: According to this site, this is a "lost connection" issue with "Seat Control Module A." Find the location of the control module, make sure it's plugged in and that there's no corrosion at the connector.
Comment: Seat Control Module A is the Driver Seat Module. You should also verify power, ground and CAN Bus integrity.
Answer: I found this resource by searching on the terms "u0208 land rover." The linked document provides a step-by-step troubleshooting procedure. Try working through it and post a more specific question if you get stuck.
Somethings to keep in mind / look for:

Assume that it is a single problem until you've exhausted that option.
Check the fuse.
Look for corroded connections, signs of water/liquid getting near the controller and its connectors.
Look for physical damage to the wires serving the controller, the seat, and the door (for the mirror).
See if there are any other codes set (see the procedure notes).


Relation between BQP-Complete and BQP \ PH

Question: Recently, the oracle separation between BQP and PH has been proven. Does this result tell us something about the relation between BQP-complete problems (e.g. approximation Jones polynomial solved by AJL algorithm) and BQP\PH? In particular, is the latter included in the former?
Answer: Let me abstract the question a bit. We have two complexity classes: $\mathrm{C}=\mathrm{BQP}$ and $\mathrm{D} = \mathrm{PH}$, as well as a promise problem $A$ that's complete for $\mathrm{C}$. We also have an oracle relative to which $\mathrm{C}$ is not contained in $\mathrm{D}$. If I'm understanding the question correctly, it's asking if we can conclude that $A$ is not contained in $\mathrm{D}$. Certainly we cannot, which shouldn't come as a surprise given that even proving that computing the Jones polynomial is outside of $\mathrm{P}$ would separate $\mathrm{P}$ from $\mathrm{BQP}$.
So, nothing that we currently know definitively rules out the possibility that approximating the Jones polynomial is in $\mathrm{PH}$. What we can say is that if computing the Jones polynomial is in $\mathrm{PH}$, then $\mathrm{BQP} \subseteq \mathrm{PH}$, so if this containment is not true then obviously computing the Jones polynomial is not in $\mathrm{PH}$ — but this conclusion is independent of any oracle separations. We can also say that if it were to be proved that $\mathrm{BQP} \subseteq \mathrm{PH}$, by this or any other method, then it would necessarily be a so-called non-relativizing proof, meaning one that fails to work when the underlying machines are able to make oracle queries.
However, this is not so far fetched. For an actual counter-example, take $\mathrm{C}=\mathrm{PSPACE}$, $\mathrm{D} = \mathrm{IP}$ (problems with interactive proof systems), and $A = \mathrm{QBF}$ (the quantified Boolean formula problem). There is an oracle (due to Fortnow and Sipser) relative to which $\mathrm{PSPACE}$ is not contained in $\mathrm{IP}$ (and in fact even $\mathrm{co}$-$\mathrm{NP}$ is not contained in $\mathrm{IP}$), but nevertheless $\mathrm{QBF}$ is contained in $\mathrm{IP}$ (as proved by Shamir). What exactly is the non-relativizing technique here? In essence it comes from the Cook-Levin theorem, which leads to the completeness of $\mathrm{QBF}$ for $\mathrm{PSPACE}$. In short, Boolean formulas don't make oracle queries.
But the same basic ideas (with a lot of added detail) are at play when showing that approximating the Jones polynomial is complete for $\mathrm{BQP}$. In particular, as far as I know there's no known reduction from an oracle problem like forrelation to approximating the Jones polynomial. We can go gate by gate in a quantum circuit and construct braid diagrams and so on to prove that approximating the Jones polynomial is complete for $\mathrm{BQP}$ — but how do you get queries to an oracle to be properly reflected in this process?
Answer: Indeed I think it follows from (1) showing that evaluating the Jones polynomial is (Promise)BQP-complete, and (2) the existence of an oracle separation between BQP and the polynomial hierarchy PH, that (3) evaluating the Jones polynomial is not likely to be in PH. That is, evaluating the Jones polynomial is precisely in BQP\PH as you had proposed.
In detail, Raz and Tal had shown that the forrelation problem of deciding whether two Boolean functions $f$ and $g$ given by black-boxes are "forrelated" (e.g., whether the Fourier transform of $f$ is correlated with $g$) is, as an oracle problem, outside not just P, not just NP, not just AM, but even outside the entirety of the polynomial hierarchy PH.  But, previously Aharanov, Jones, and Landau had provided an algorithm showing that evaluating the Jones polynomial is PromiseBQP-Complete.
Thus, because the Jones polynomial is BQP-Complete, there is a reduction from any problem in BQP (such as forrelation) to the evaluation of some Jones polynomial.  Hence, we can relate forrelation to this evaluation, and use AJL's BQP algorithm to evaluate the Jones polynomial as a vehicle to evaluate the forrelation, but we can't evaluate the forrelation in PH because that was ruled out by Raz and Tal.  Thus the particular Jones polynomial related to forrelation cannot be evaluated in PH, and therefore evaluating the Jones polynomial is in BQP but not in PH.

 This is all rough and imprecise intuition about reductions relative to oracle separations, and also with respect to promises about the gap between the forrelations of $f$ and $g$ and the accuracy of the Jones polynomial evaluation. For me at least these require some effort to be particularly correct about, and I often get tripped up into the precise way to make the statements!


How do I ask the people I work with to write a complimentary note for my team to show appreciation for our efforts?

Question: My team has been working very hard to fufil all different types of (IT Application related) requests and urgent requests from this group of users who are very demanding. One day, I received a phone call from one of them to say thank you and that's all. I told him instead of thanking, he should at least write a complimentary email (like an email to say that they appreciate our efforts and highlight what are the things we have done for them and how satisfied they are with our performance, etc) to the team so that we will feel motivated to keep up the good work.
I mean, I also have told my manager that this guy sent his heart felt thanks to us through the phone. My manager told me that he would like to see an email from  him for his compliments instead, so that we could show the email to the rest of the team or department members as a form of motivation and encouragement to keep up the good work too. Imagine, those who have been working overtime and slogging to get things done, they don't even get to hear or see a thank you from the user himself. They only get to hear it through me a fellow team member.
And when I told this guy about sending us a complimentary email, he rejected with a rude joke saying it will become none of his business soon. (what?!?)
:(
Comment: Sounds like you mishandled this and your manager is a potential loon who got his management knowledge from a book instead of a human being. What is wrong with telling your team: "*X called to thank us for [going above and beyond / fixing Y so quickly / helping him out with Y]?*"
Comment: "My manager told me that he would like to see an email from him for his compliments instead" - why not get your manager to talk to him then, or better still his manager, to write the email you want? But I don't think you can just fish for compliments in general - people either give them or they don't. If it is a team morale issue (or even just your morale) then get your manager to sort it out, or publicise your success himself, or organise some morale-boost activity or something for you all to say thank you from the company.
Comment: I guess its like a restaurant business. Customers can tell a waitress "thank you" or not but reality a waitress/waiter is only interested in the tip left behind.
Answer: This is for your manager to push if he feels the need. But I see no need for it. You got a thank you phone call, your manager could congratulate the team himself via email or in the next meeting saying something like.

"Well done guys, we have a happy client, keep up the excellent work."

You can't force people to compliment your team, a thank you might be appropriate, but at the end of the day you were doing your job. A thank you is not a requirement.
As for the remark from the client, it's not constructive to try and psychoanalyse his issues, it could mean a number of things. Take it in your stride and move forwards.
Answer: I think you missed the opportunity on the initial phone call.  You want something for him, yet you are telling him what he should do, and implying he should be doing more.  Asking nicely for the e-mail would have been more likely to succeed.
"Thank you for calling to express your appreciation.  I'd love to share this with my team, but I think it would be so much nicer as a message from you.  Do you think you could possibly send me a quick email with your thoughts that I can share with them?"
Answer: It's reasonably normal, particularly in customer service, to ask people for a thank-you letter once they give you a compliment. In the pas, businesses would hang those on the wall to help entice new customers, which is the paper and tape equivalent of yelp.
Of course, it's possible he misinterpreted your tone. It does take certain sales/people skills to make this palatable to most, and I've personally felt it's always met with a certain resentment in all but the most devoted fans.
An effective pitch is to say that it would really help motivate the team, and justify the level of effort to management on your side. In effect, you're saying "If you put these words in an email, you'll continue to get great service".
I think there's something else here if you squint, and might be why the caller didn't want to put anything in writing. Saying that "it will become none of his business soon" might mean he is on his way out. If so, the call may have been his way of "apologizing" for the demanding behavior of an employer he doesn't agree with. I've made exactly that phone call to a vendor once I'd decided to leave a position. I couldn't do it earlier, because it would undermine the company on my side, but I felt strongly that I should re-affirm that the treatment the vendors were receiving wasn't acceptable, and that I had done what I could do. It was also a warning that once I left, the firewall would be down, and the treatment would get worse. It's a rare company that treats its vendors like slaves, and its employees well.
Answer: So you told him he should at least write a complimentary email to share with the team and he did not send an email.  You did not ask - you told him what he should at least do.  You talked to your boss and he wanted an email.  So you called the guy back and told (not asked) him to send a complimentary email.  You ask nicely once and if you don't get it then let it go.

Thank you for the expressing your appreciation.  If you would send an
  email I can share it with the team.


How does magnetic field know the particle's velocity?

Question: The force on one charged particle due to a uniform magnetic field is $F=qvB\sin\theta$. According to this formula, the faster the particle's velocity, the greater the force of the magnetic field is going to exert on the particle. However, how does the magnetic field know how fast the particle is moving?
Comment: What does it mean for the field to "know" how fast the particle is moving?
Comment: I only mean that why is the force on a charged particle created by the magnetic field have to do with the velocity of the particle? Why is it not a constant given that the charge and the magnetic field is constant? Sorry if this question seems too elementary, because I am just starting to learn about electromagnetism.
Comment: @MichaelWang The interaction of charged particles with magnetic fields is basically an interaction of magnetic moments and fields. A moving charged particle has a field in the reference frames in which it is not at rest. The strength of that field depends on how fast it's moving. That field interacts with the static field. E&M gets "weird" when you include special relativity because there are frames in which magnetic fields can become zero.
Comment: How is the question unclear?
Answer: Fields don't "know" anything.  "Knowing" is a anthropomorphic concept, based around how humans believe they operate.
The effects of a magnetic field on a charged particle are proportional to the derivative of position with respect to time.  This is simply the equation used to predict the effects a magnetic field has.
Perhaps even more interesting is the Coriolis effect.  The Coriolis effect fields a pseudo-force which is a function of the velocity of a particle.  This is true, even though the Coriolis effect is merely an effect of viewing the trajectory of a particle in a rotating reference frame -- there is nothing physical to which we could even attribute its effects.
Comment: See also: [Resemblance between Coriolis force and magnetic part of Lorentz force](https://physics.stackexchange.com/q/155093/21441)
Answer: The closest argument to a reason why the magnetic force depends on the velocity is that, fundamentally, the the four potential $A^\mu = (V, A^x, A^y, A^z)$ is coupled to the charge current $j^\mu = (q, j^x, j^y, j^z)$, which depends on the velocity of the charges.


PHP performance hampered by require()

Question: I ran my code through xdebug profiler and saw more than 30 percent of the time is spent on the require() calls.  What is the best way to improve on this?  I saw some posts about using __autoload, but there were conflicting statements about it's affect on APC (which we use), and doubts about it's use to improve performance.
Comment: Have you tried include() instead?
Comment: isn't require() just a hard-fail version of include() ?
Comment: In theory, yes. I wondered if it might have a huge impact, thus the question in a comment and not an answer :)
Answer: The reason why requires consume time is disk IO speed. You can try using autoloading, as you may be requiring files that aren't actually used. Another approach to reduce the disk IO overhead is to combine your PHP files into one large file. Requiring a big file which contains the code you always need is faster than including the same code in multiple small files.
Also, APC has a feature which speeds up requires called apc.include_once_override which you can try enabling.
Comment: are you referring to apc.stat ?
Comment: It's apc.include_once_override
Answer: Make sure your includes use absolute instead of relative paths. Easiest way to do this is by prepending your paths with
dirname(__FILE__)  // for php < 5.3
__DIR__            // for php >= 5.3

Answer: You can improve speed of your code by using PHP compiler, like http://eaccelerator.net/.
Such compiler makes everything work faster, also including files.
Answer: APC and autoload had some problems once. That's a long time ago. In general, APC can speed up you require statements, since it caches the parsed files. By default APC will still stat the file to see if it has changed on disk. You can prevent this by using absolute paths and turning the apc.stat setting off. Note that this means that you have to restart the server to clear the cache.
Answer: how many items are in your include path? and is the order of the locations sensible for your application? if you're using relative paths then it will check the include-path locations in order looking for a matching file.

PHP performance hampered by require()


What is the origin of "sleep till I wake him"?

Question: In King Lear, the phrase "If our father would sleep till I waked him" is used in Edmund's fake letter to Gloucester. Apparently it means "if our father were dead"[1][2]. What is the origin of the phrase? Why does "sleep till I waked him" mean dead?
Comment: @Gal That is what somebody on the internet suggested: https://en.wikipedia.org/wiki/Wikipedia:Reference_desk/Archives/Language/2015_February_15
Comment: 'Apparently it means'. Do you not think you should document, in your research, your reasons for stating this. And a link to the quotation would be helpful. You are asking other people to do all the work, here.
Comment: Pretty much any annotated Shakespeare mentions it without any additional citations. I know what it means, I'm wondering why it means that. I'll add links.
Comment: I would guess that the origin is Shaky.
Comment: @Hot Licks But why would "sleep till wake" mean death? Unless the word wake here is used in the sense of funeral, as opposed to waking up, and Shaky is just loving his puns, which to be fair, would not be uncharacteristic.
Comment: @HotLicks Shaky? The association between sleep and death is much older.
Comment: @ChristopheStrobbe - Like I said, the origin is Shaky.
Answer: The phrase occurs both in the 1608 quarto The History of King Lear and the 1623 First Folio, where the play's title is The Tragedy of King Lear. This can be seen in e.g. King Lear: A Parallel Text Edition. Second edition. Edited by René Weis. Harlow: Pearson Education, 2010.
René Weis adds the following annotation: "Edmund would not wake his father from sleep, i.e. 'If our father were dead'."
In the New Penguin Shakespeare, G. K. Hunter adds the following annotation: "if our father were put into my power to decide his sleeping or waking (that is, death or life)."
The association of sleep and death is not unusual in Shakespeare; it also occurs in Hamlet's "to be or not to be" soliloquy:

To die, to sleep –
  No more; and by a sleep to say we end
  The heartache and the thousand natural shocks (...)
  (...): to die, to sleep.
  To sleep, perchance to dream. (...)

In the Oxford Shakespeare edition of Hamlet, editor G. R. Hibbard comments on the fist line: "i.e. dying is no more than sleeping."
The association predates Shakespeare. See for example "The Parson's Tale" in Geoffrey Chaucer's The Canterbury Tales (quoted from The Riverside Chaucer page 292, emphasis added):

The riche folk, that embraceden and oneden al hire herte to tresor of this world, shul slepe in the slepynge of death; (...).

("oneden" means "united".)
The origin of the association between sleep and death in Western culture is probably biblical. See for example Daniel 12:2 (emphasis added):

And many of those who sleep in the dust of the earth shall awake, some to everlasting life, and some to shame and everlasting contempt.

See also the concept of soul sleep to refer to the time between physical death and resurrection.
Answer: It cannot be that "wake" refers to the noun we today define as "a gathering of people to pay respects to a dead person who is laid out in their presence." Why? Because it would make nonsense, then, of the word "sleep." Why would someone necessarily have to be waked here (as in the noun sense I have just offered)? If Edmund wants his father Gloucester dead, then he is saying simply, "If we can imagine a scenario in which my father would go to sleep and stay asleep forever, unless or until I wake him up, then that sleep would be his death, for I would NEVER wake him up. I'd like not to have to actually kill him with my own hands, but if he could go to sleep and stay asleep and wake up only when I made him wake up (as if by some magic power I might be imagined to have) then that sleep would be his death, for I would never end it by waking him."
So "sleep until I laid him out for others to pay their respects at a wake" makes no sense here. If Edmund is saying he would end Gloucester's sleep by laying Gloucester out in a casket for others to pay their respects, he is not telling us by what means Gloucester would be made to change from sleeping to being dead. Obviously, we cannot "wake" a living person (again, "wake" used here as "the ceremony of laying out a dead person . . . ), so he could not be saying that Gloucester would just go on sleeping during his own wake!


javascript anonymous function

Question: function (){
    alert('a function');
}

when i put it on the firebug javascript control. it shows function statement requires a name
(function (){
    alert('a function');
}())

when i put the above it shows ok.
function (){
    alert('a function');
}()

it also shows  function statement requires a name and doesn't execute the function. why?
Comment: http://stackoverflow.com/a/442408/1113426
Comment: possible duplicate of [What do parentheses surrounding a JavaScript object/function/class declaration mean?](http://stackoverflow.com/questions/440739/what-do-parentheses-surrounding-a-javascript-object-function-class-declaration-m)
Answer: function (){
    alert('a function');
}

is a function statement, so it requires a name.
(function (){
    alert('a function');
}())

The () change the statement to an expression, so it's OK.
And you could also use the below ways.
(function (){
    alert('a function');
})();

!function (){
    alert('a function');
}();

+function (){
    alert('a function');
}();

Comment: They just change the statement to an expression.

javascript anonymous function


FME - what is the difference between "generate workspace" and "create workspace"?

Question: as a newbie to FME I can´t really understand the basic difference between the two ways of making a workspace. Generate or Create. Can someone explain the difference?
Answer: Create Workspace is like a dashboard that allows you to access existing templates, your own saved templates, training, or pre-built demos via FME Store.  Generate Workspace does not have the dashboard option if you do not want to use any of the options listed above.
Answer: Also, Generate Workspace is a single dialog. If you go through the Create Workspace wizard, it's the same thing but a series of dialogs. I always recommend Generate Workspace.
Incidentally, the shortcut key is Ctrl+G. The shortcut for Create Workspace is Ctrl+N.


Is there a bot or other method to guide potential question-askers to the most appropriate SE site?

Question: I see that each SE site suggests related questions on that specific site, but I sometimes have questions and am not sure about which SE site is the most appropriate scope for them. Is there a bot or something that can analyze my question and give me the best guess SE site (maybe with confidence values or something of the sort)?
Comment: This question seems belongs to [meta.stackexchange](https://meta.stackexchange.com/).
Comment: @Enzokie I agree. I didn't know that site existed before writing the question.
Answer: There is no bot for this. Making such a bot would be quite an ambitious project (but also an exciting one!)
What you can do, is post on Meta Stack Exchange using the site-recommendation tag. Show the important parts of the question you want to ask, and show what research you've already done to find the appropriate site.
Comment: @George Posting and hoping for migration is risky. It can take a long time before something gets migrated. It is an option, but not one I would advise.
Comment: I agree it is risky as it's more likely that it'll get closed/downvoted but the sense I got from this question was that it was about a question with may have overlapping subjects which would be on topic on both sites so it _shouldn't_ be that bad. That being said, your answer is without a doubt the safer way to go about it.
Comment: Either that or post on _what you think_ is the most relevant site and your post will be migrated by the community. I agree that bot would be cool though.
Answer: You should be able to look at the list of sites and drill down pretty quickly to a couple of possibilities, just based on the broad topics/names. This site even allows you filter by category, so you can quickly narrow down the number of sites you have to consider.
Once you have that, you need to read the Help Center for those candidate sites to see exactly what is on topic there, since different sites have different rules (and some of them may be non-obvious, like Software Recommendations).
A bot could conceivably be built to help with the first part, I guess, but that's really the easy part. (Easy part for a human, that is; still hard for a bot.)
Answer: In addition to the other answers, it should be noted that topicality is not mutually exclusive.  In other words, there are questions which are on-topic at multiple sites.  When this happens, you get to choose which site to ask.
For example, you can ask questions about the Mass Effect franchise's storyline on both Arqade and Science Fiction & Fantasy.  For this case, I would recommend Arqade for questions more oriented towards player choices and gameplay ("How do I make [character] like me more?"), and Sci-fi & Fantasy for questions more oriented towards world building and storytelling ("Why is [character] so hostile towards [protagonist]?").  These areas are not totally equivalent; if you want to ask a pure gameplay question ("How do I beat the boss battle at the end of [level]?"), that's only going to work on Arqade.  But there is a lot of overlap, and we trust that questioners will choose a site wisely.
Of course, you may get a different style of answer, depending on which site you choose.  The best way to learn which questions are appropriate for which sites is to join more sites, and participate in them regularly.  You can also learn a lot by just reading Hot Network Questions, but bear in mind that these are sometimes unrepresentative of the site as a whole.


Creating Kernel Module without Sudo

Question: How do I create a kernel module without the use of sudo? I know the normal way(sudo insmod, sudo rmmod), but we're not allowed to use sudo in school.
Answer: First off, what module do you want to insert and why? Normally there is no need to manually remove / insert modules. By the way, creating a module is a different story, insmod and rmmod are for loading / unloading a particular module (created by someone else).
Once you got your reasons straight, and are sure that there is no other way... then go to your system administrator and ask politely. Explain clearly your reasons, and you have a chance to succeed. No joke: unless you want to go against all rules (and some laws, probably) and hack the machine belonging to your school, you will not acquire privileges granting you such a low level operation.
Comment: You need to be root to play with the kernel. Sudo is just a way of doing that. Logging in as root is another. Having a suid insmod is  another. And `su` is yet another. Yet I'm still wondering as to what kernel module you are referring to. Finding out the page size is as simple as typing `getconf PAGESIZE`. You might be on a wrong track...
Comment: Actually I'm just going to print the value of the page size of the Linux machine I am currently using. We're asked to demonstrate it with the use of kernel module. Aren't there any other way besides hacking and sudo?
Comment: One module in particular that it would be nice to remove and insert without sudo would be the pcspkr module to allow beep sometimes but not all of the time as it can get annoying.
Answer: Perhaps an easier way forward is to do the kernel development inside a virtual machine; I use virt-manager + kvm-qemu, but there are other options such as VirtualBox. This allow can then allow you to do all the development in a safe virtualized environment and maybe allowed by your system administrator.

Creating Kernel Module without Sudo


Cannot boot after upgrade to 18.04 grub-probe: error: cannot find a GRUB drive for /dev/sdb1

Question: I had upgraded my Ubuntu 16.04 to Ubuntu 18.04.
The upgrade completed and the system tried rebooting, but after that my PC no longer boots.
I get a black screen, with no disk activity. If I press the power button my PC turns off immediately (no need for long press).
I tried Boot-Repair but the process completes with errors. I think the last one is the most relevant.
http://paste.ubuntu.com/p/XFvWjsGCxz/
Comment: If by errors you mean the first two comments then no, not a problem. You have a UEFI system, Ubuntu installed in the same mode, a EFI partition as required. You aren't supposed to have a bootloader installed in the MBR. First thing to check is the boot order in UEFI and making sure "ubuntu" is still selected. If you find more than one entry saying "ubuntu", try the others just in case. If this doesn't solve it then something very wrong happened with the version upgrade.
Answer: I fixed it by disabling SecureBoot from UEFI. I don't know why after the update this had become a problem (since with Ubuntu 16.04 SecureBoot has always remained activated) and currently I do not have too much time to investigate.
In any case, thanks to GabrielaGarcia for the clarifications.


Duplicating the game analysis output on lichess

Question: I'd like to automatically generate move by move analysis (based on my past games) like they have on lichess for imported games.  My guess is that I need to run a full stockfish analysis for each move (via UCI) and then glue those outputs together.   I was just wondering if I was missing an easier way?  Is there a "full" analysis option that let's you get all of the scoring/annotation in one go?
Answer: If you know a little python or shell programming you could try and use uci for each move and grab stockfish output, I will try and see if I can get this sorted at some point and share the code.
If just being offline is your concern, then try the stockfish app, there is offline analysis.
Again, from a programming angle, there is trying to add such a feature to scid.
Answer: Stockfish, like the UCI protocol in general, does not have a single "annotate all" command.
From your question, you seem not to be interested in a GUI option like Chessbase or Scid, but instead in how to create a program of your own.
I've found that the script posted here: https://github.com/rpdelaney/python-chess-annotator provides a good start at a model. It's written in Python, which has a great chess library.
There are few tricks it uses. It makes two passes: the first to identify key moves, and the second to focus on those. It works backwards so that it will already know the evaluation of the move actually played when it is calculating the optimal move. It doesn't spend time evaluating ECO positions.
I can't guarantee that this is the best approach, but it seems to work.
Answer: There are a few good approaches to your problem:

Upload your game to lichess, you'll get your computer analysis for free.
The software used to analyse games on lichess is fishnet, you may freely download it and try to figure out which parts can be used for your purpose.
CRAFTY -- this is well known chess engine, freely available everywhere, that can analyse your whole game. Once you have crafty started in the command line mode, just give it an annotate my_game.pgn wb 1-999 0.5 2 5 command, the parameters are:

my_game.pgn -- your game file, "wb" -- if you want the analysis for White and/or Black, "1-999" -- the move range to analyse, "0.5" -- the score threshold, if the score change below that, the analysis is not printed -- this is used to make the engine to react to every small problem or to big blunders only, "2" is analysis time per move (in seconds), "5" -- the number of variations output.


Confusion about definition of ramification points in Forster

Question: In the book "Riemann surfaces" by Forster a ramification point of $f:Y\to X$ is defined as a point $y\in Y$ such that there is no neighborhood $V$ of $y$ such that $f|_V$ is injective. On the other hand given that $X$ and $Y$ are compact Riemann surfaces and $f$ holomorphic, if for an open set $V\subset Y$ the map $f:V\to f(V)$ is injective (hence bijective), then the degree of $f$ being equal to one on an open set, would force $degf=1$ and hence $f$ is an isomorphism. So by definition of ramification point in Forster, one could imply that any holomorphic map between compact Riemann surfaces is isomorphism (take a point $y$ which is not a ramification point then from the definition it follows that there exists a neighborhood $V\subset Y$ of  $y$ such taht $f|_V:V\to V$ is bijective and then apply the above degree argument) which is of course non-sense. I don't know where exactly my mistake is but I believe that I am making a mistake somewhere and don not see where.
Comment: The degree of a map is something global. Take $f\colon \widehat{\mathbb{C}} \to \widehat{\mathbb{C}}; \; z \mapsto z^n$ for $\lvert n\rvert > 1$ to see that there are non-constant holomorphic maps between compact Riemann surfaces that are not isomorphisms. Yet, except at $0$ and $\infty$, these maps are of course _locally_ injective. The degree is in some sense "local on the range"; if $\operatorname{card} f^{-1}(x)$ is equal to $n$ on some (punctured) neighbourhood of $x\in X$, then the map has degree $n$, but you are looking at a neighbourhood of an $y\in Y$ in the domain.
Answer: Your mistake is here:
...then the degree of $f$ being equal to one on an open set, would force $\deg f = 1$  and hence f is an isomorphism.
As you wrote you have a neighborhood $V$ in which $f: V \to f(V)$ is injective and somehow you wrongly concluded that $f : Y \to X$ must be injective. Have a look here to get more insight: https://en.wikipedia.org/wiki/Covering_space


When android fires ACTION_BATTERY_LOW

Question: In my application, I want to do something when the battery is low. When battery is low android fires ACTION_BATTERY_LOW and when the battery again reaches to its good health it fires intent ACTION_BATTERY_OKAY. So, I have three questions regarding this:
1.At what battery percentage android actually fires ACTION_BATTERY_LOW?
2.Does it fire that same event repeatedly if the battery gets even lower?
3.Can we configure the battery percentage at which android will fire ACTION_BATTERY_LOW intent?
I am more concerned about the third point.
Answer: No, you cannot set when the ACTION_BATTERY_LOW threshold will be sent.  That is a system level intent that is specified by the Android ROM. Here is the code where it sets the value in the Battery Service:
mLowBatteryWarningLevel = mContext.getResources().getInteger(
            com.android.internal.R.integer.config_lowBatteryWarningLevel);

See the code below which is cut from the Android system code in the update method of the Battery Service:
/* The ACTION_BATTERY_LOW broadcast is sent in these situations:
         * - is just un-plugged (previously was plugged) and battery level is
         *   less than or equal to WARNING, or
         * - is not plugged and battery level falls to WARNING boundary
         *   (becomes <= mLowBatteryWarningLevel).
         */
        final boolean sendBatteryLow = !plugged
            && mBatteryStatus != BatteryManager.BATTERY_STATUS_UNKNOWN
            && mBatteryLevel <= mLowBatteryWarningLevel
            && (oldPlugged || mLastBatteryLevel > mLowBatteryWarningLevel);

        sendIntent();

        // Separate broadcast is sent for power connected / not connected
        // since the standard intent will not wake any applications and some
        // applications may want to have smart behavior based on this.
        Intent statusIntent = new Intent();
        statusIntent.setFlags(Intent.FLAG_RECEIVER_REGISTERED_ONLY_BEFORE_BOOT);
        if (mPlugType != 0 && mLastPlugType == 0) {
            statusIntent.setAction(Intent.ACTION_POWER_CONNECTED);
            mContext.sendBroadcast(statusIntent);
        }
        else if (mPlugType == 0 && mLastPlugType != 0) {
            statusIntent.setAction(Intent.ACTION_POWER_DISCONNECTED);
            mContext.sendBroadcast(statusIntent);
        }

        if (sendBatteryLow) {
            mSentLowBatteryBroadcast = true;
            statusIntent.setAction(Intent.ACTION_BATTERY_LOW);
            mContext.sendBroadcast(statusIntent);

Answer: That intent is fired from the BatteryService.  You'll have to analyze the code a bit, but I'm pretty sure it does not fire repeatedly:
http://gitorious.org/android-eeepc/base/blobs/fda6fae156e31a287e3cfbf66e51ea1405cdf479/services/java/com/android/server/BatteryService.java

The actual values that it fires at are setup in the android resources, so it's configurable only during a system build.  This is what we have for our hardware, but this will likely be different for each hardware platform that Android runs on:

<integer name="config_lowBatteryWarningLevel">15</integer>

<integer name="config_lowBatteryCloseWarningLevel">20</integer>

Unless you're developing a custom hardware platform, I wouldn't make any assumptions about what these values are set to.
Comment: @KalaJ if these integers are going to be used *only* in java files then store it say, `CONST.java` (for constants) or `CONFIG.java` (for configuration parameters). If they are for the UI, then you store it `values.xml`. Note: `values.xml` is like any other res file and you can have multiple versions based on localizations etc.
Comment: Just a coding standard question, where do you store your integers? Is it in a separate constants file?
Comment: It's been a while since I looked at that, but I'm pretty sure those are already defined in the Battery Service resources.  We simply adjusted them.
Answer: There is another way that detect "config_lowBatteryWarningLevel" from "com.android.internal.R.integer" field.
enter code here

    try {
        Class clazz = Class.forName("com.android.internal.R$integer");
        Field field = clazz.getDeclaredField("config_lowBatteryWarningLevel");
        field.setAccessible(true);
        int LowBatteryLevel = _context.getResources().getInteger(field.getInt(null));

        Log.d("LowBattery","warninglevel " + LowBatteryLevel);
    } catch (ClassNotFoundException | NoSuchFieldException | IllegalAccessException e) {
        e.printStackTrace();

    }

Comment: Note that this uses reflection to get info from Android classes that are explicitly marked as `internal`, so there's no guarantee this will work in the future!

When android fires ACTION_BATTERY_LOW


Movie about aliens disguised as humans that turn out to be allergic to latex

Question: I saw this movie (or possibly mini-series) on TV (grandma had cable) in the mid to late eighties.  I was young, so I don't remember much.  Aliens were hiding as humans so it took a while to figure out what was going on.  There was a scene with a baby, sick in an incubator, but then his skin split open and it was a green alien baby.  I also remember a scene where there was a big balloon release because the aliens were allergic to latex, but then it seemed like the aliens went from being bad guys to good guys.
Comment: Makes me think of V
Comment: This is just plain funny, given the material of the usual alien costume/prosthetics in these shows and films.
Comment: One of the plots later in the run of V (1984), IIRC: A human woman and a reptilian alien "visitor" disguised as a human fall in love and conceive a child. Concerns about the fetus arise when her lover is revealed to be an alien. Everyone is relieved when the child is born as an apparently normal human, but while it is in an incubator, its human skin splits open to reveal the reptilian alien within. I don't specifically recall anything about a latex allergy.
Comment: @Danny3414 : I got that vibe too (a really muddled memory of it there if it is that though) : accept they were never "_**hiding**_ as humans" in V : I don't remember anything about a latex allergy either?
Comment: I don't remember a latex allergy either. IIRC the Resistance tried to release a red dust bacteria but it didn't work in sunny climes (maybe!)
Comment: @FuzzyBoots : I know, I did sort of wonder if that might be part of the years later muddle in memory from watching V (for example) as a child & remembering their human disguises being torn & such.
Answer: Could this be a confused mash-up memory of V?
V (1983 miniseries)

A race of aliens arrive on Earth in a fleet of 50 huge, saucer-shaped motherships, which hover over major cities across the world. They reveal themselves on the roof of the United Nations building in New York City, appearing human but requiring special glasses to protect their eyes and having a distinctive resonance to their voices. Referred to as the Visitors, they reach out in friendship, ostensibly seeking the help of humans to obtain chemicals and minerals needed to aid their ailing world, which is revealed to be a planet orbiting the star Sirius. In return, the Visitors promise to share their advanced technology with humanity. The governments of Earth accept the arrangement, and the Visitors, commanded by their leader John and his deputy Diana, begin to gain considerable influence with human authorities.

V The Final Battle
Part II of the second series ends with Robyn (a human character impregnated by an alien as an experiment) giving birth to surprise twins. The first baby appears mostly human, but unexpected baby number two crawls out of the womb shocking everyone with its reptilian appearance. Birth scene on YouTube
Near the end of the second series the human resistance used hundreds of Hot Air Balloons around the world to spread a virus/bacteria/toxin that was harmless to humans but deadly to the Visitors, to drive them off earth. They were able to spread it to the Visitor Mothership over LA and capture it. (Its been too long but I believe they also spread it to many/most of the Visitor ships too, so only a couple or a few would survive to the trip back to their home world.)

V (1984 TV series)
I don't recall anything about an allergy to latex but that might simply be a confused memory of their latex disguises if you were young, otherwise the elements do fit & the time period it was aired is right.
They were green reptilian aliens disguised with latex skin as humans invading earth, they initially appeared to be benevolent & here to help us before it became an overt invasion & some of them did switch sides to join or help the human resistance in later episodes.
Google searches
Images
Videos
Comment: I was expecting Danny3414 to drop an answer, so as he was first in comments with V I wasn't going to, but he hasn't so I went ahead & posted it.
Comment: nah, you're ok. I've put a couple of answers in this week... don't wanna be a teacher's pet!


Koi deaths after hail storm

Question: I have had my pond for 8 years and have never had any problems until a hail storm a few days ago.  I went out and found them all dead.  I then googled it and think it was due to the change of temperature and then it depleted their oxygen.   Does anyone know or have a cover that would work for hail?
Comment: where do you live? In terms of climate
Comment: Cold water holds more oxygen then warm water.  How deep is your pond? It takes a LOT of energy to change water temp significantly, particularly if it is deep.
Comment: Same deal here, year 6 in this house and we have lost koi over time probably for various reasons (probably not as dedicated pond care taker as I should be) but last night large hail (up to a 1.75 inches said the weather alerts) 3 hours without power and my koi pond was a ghost town when I got home from work.
Answer: I have a pond with koi. We have had two hail storms (in ten years) that required replacement of our roof shingles twice which the insurance adjuster authorized; so serious hail. No problem with koi or the tropicals (mollies, danios, etc.)
I have inadvertently killed koi by covering most of the pond, slowing oxygen exchange. Hail hitting the pond would increase dissolved oxygen content by agitation and cooling the water. I strongly recommend using no cover. I also doubt it was temperature change, as my tropicals would be much more sensitive to cold than koi. The only thing I can think of is a very fast temperature change and that is hard to accept.
What about a lightning strike on the pond (grasping at straws)?

Koi deaths after hail storm


Simple compile error in class constructor

Question: I have a very simple class that wont compile because of a default parameter in the constructor. The language is C# and the class will be used in a Unity3d game. So this compile error is occurring in Unity's "IDE"
Compile Error:

Assets/Utilities/GenericClasses.cs(30,94): error CS1736: The expression being assigned to optional parameter `nName' must be a constant or default value

public class Element {
    public static readonly string NULL_NAME = "___NULL_NAME___";
    public enum elementType {E_ELEMENT, E_MODEL, E_VIEW, E_SUPER, E_ARC};
    private string name;
    private elementType type;

    // line 30 is below
    public Element(elementType nType=elementType.E_ELEMENT, string nName=NULL_NAME) {
        type = nType;
        name = nName;
    }
}

What am I doing wrong?
Answer: You should use a constant value instead of a static readonly:
public const string NULL_NAME = "___NULL_NAME___";

The reason why your code doesn't compile is because you attempted to assign a non constant default value to the nName parameter in the constructor which is not supported.
Answer: NULL_NAME is not a constant, it is readonly.
A readonly variable can be initialized in the constructor so it is not known at compile time which is required for a default value for an optional parameter because the compiler will substitute the value where it is omitted in a call.
Answer: Try this one
public const string NULL_NAME = "___NULL_NAME___";


What are the alternative uses of unlimited wants and scarce resource?

Question: Actually I want to know what does it mean by alternative uses of this two terms of economy.
Answer: Scarce resource : is a (physical) limitation which forces us to make choices accordingly. It is what is available under current technology and cost of extaction. We could say its a supply side perspective and generally refers to material.
Unilimited desires: Is a demand side perspective. Individuals or organizations usually want more tomorrow than today. Even if you get satiated of one good you will want more. For instance if you have enough foood you can't eat more but maybe you would wish a special meal in a fancy restaurant. So even if cooking home was sufficient to narch your need, after an increase of income you will buy more services... bigger house... There are diminishing returns to consume more of the same good but imagination makes it possible to always find a new object of desire. Thus unlimited wants.
By combining these two phenomenons, a scarcity situation arises and humanity must make choices hopefully to fullfill the most pressing needs.


How to hand pollinate Narajilla (Lulo)?

Question: I have been poking a little at the flowers but they are not what I am used to. I've hand pollinated many other plants with success but this flower structure is more closed off and I can't seem to locate the pollen.

https://en.wikipedia.org/wiki/Solanum_quitoense
Comment: you could try with your phone on the highest vibrate setting
Comment: Which type of tool did you end up using? Did you try any tool that didn't work?
Comment: @csk So far no success but I ordered this tool to try out: https://smile.amazon.com/gp/product/B01D5G88ZE - In the meantime I have been flicking (attempting to vibrate the flowers/stems)
Comment: I have been able to pollinate 5 flowers so far using the above tool. Timing and method appear to be significant factors for success - many more flowers have fallen.
Answer: Many Solanums use a special pollination mechanism called buzz pollination. It requires a bee to land on the flower and vibrate its wings at a specific frequency before the anthers release their pollen. In nature, this increases the plants reproductive success rate, by only releasing pollen onto the exact kind of bee that will visit other plants of the same species.
In gardening, this is a bit annoying if you want to hand pollinate. You have to re-create the vibration of bee wings to get the flower to release its pollen. This article recommends using a vibrating implement such as an electric toothbrush. Hold a spoon or bowl under the flower to collect the pollen. If the toothbrush doesn't work, they suggest trying a tuning fork or vibrator.

How to hand pollinate Narajilla (Lulo)?


How powerful is Aria T'Loak?

Question: We know that she ruled Omega, and is probably an extremely powerful biotic, but in ME3 it was shown that even the Asari councilor was at her beck and call despite having lost her seat of power.  Was it ever explained in any of the mini comics how she came to have so much influence or power?
Answer: Basically she worked her way up from the bottom, proving that she is capable of being the leader of Omega.
She started as an exotic dancer in the Afterlife club while she plotted how to kill the owner. Once he was dead she claimed ownership and began building alliances and influence on Omega. She eventually overthrew, but did not kill, the former Omega leader, a krogan that goes by the name The Patriarch. She kept him alive and working for her so that others would see him as an example of her power and influence.
Over the years since then she has just built more alliances and influence.
Comment: yes, but its quite a step to go from ruler of omega to bending a council member to her will
Comment: Omega is one of, if not, the largest underworld hubs in the galaxy. Described as "the Terminus Systems' dark, twisted counterpart to the Citadel."

Its not so hard to believe that those in control of heart of the underworld would have considerable influence within the heart of the galaxy.

How powerful is Aria T'Loak?


Can you compare different clustering methods on a dataset with no ground truth by cross-validation?

Question: Currently, I am trying to analyze a text document dataset that has no ground truth. I was told that you can use k-fold cross validation to compare different clustering methods. However, the examples I have seen in the past uses a ground truth. Is there a way to use k-fold means on this dataset to verify my results?
Answer: The only application of cross-validation to clustering I know of is this one:

Divide the sample into a 4 parts training set & 1 part testing set.
Apply your clustering method to the training set.
Apply it also to the test set.
Use the results from Step 2 to assign each observation in the testing set to a training set cluster (e.g. the nearest centroid for k-means).
In the testing set, count for each cluster from Step 3 the number of pairs of observations in that cluster where each pair is also in the same cluster according to Step 4 (thus avoiding the cluster-identification problem pointed out by @cbeleites). Divide by the number of pairs in each cluster to give a proportion. The lowest proportion over all clusters is the measure of how good the method is at predicting cluster membership for new samples.
Repeat from Step 1 with different parts in training & testing sets to make it 5-fold.

Tibshirani & Walther (2005), "Cluster Validation by Prediction Strength", Journal of Computational and Graphical Statistics, 14, 3.
Comment: can you further explain what a pair of observation is (and why are we using pair of observations in the first place) ? Furthermore, how can we define what is a "same cluster" in the training set compared to the test set? I had a look to the article, but did not get the idea.
Comment: @Tanguy: You consider all pairs - if the observations are A, B, & C the pairs are {A,B}, {A,C}, & {B,C} - , & you don't try to define "the same cluster" across train & test sets, which contain different observations. Rather you compare the two clustering solutions applied to the test set (one generated from the training set & one from the test set itself) by looking how often they agree in uniting or separating members of each pair.
Comment: ok, then the two matrices of pair of observations, one on the train set, one on the test set, are compared with a similarity measure?
Comment: @Tanguy: No, you only consider pairs of observations in the test set.
Comment: sorry I was not clear enough. One should take all pair of observations of the test set, from which a matrix filled with 0 and 1 can be built (0 if pair of observation do not lie in the same cluster, 1 if they do). Two matrices are calculated since we look at pair of observations for the clusters obtained from the training set and from the test set. The similarity of those two matrices is then measured with some metric. Am I correct?
Comment: After re-reading your answer, I understand that we actually only build one matrix, where in each cell 1 indicates that the pair of observations lie together in the cluster for both clustering trained from the test set and the training set (I was being misled in my previous comments).
Answer: Resampling such as done during (iterated) $k$-fold cross validation generates "new" data sets that vary from the original data set by removing a few cases.
For ease of explanation and clarity I'd bootstrap the clustering.
In general, you can use such resampled clusterings to measure the stability of your solution: does it hardly change at all or does it completely change?
Even though you have no ground truth, you can of course compare the clustering that results from different runs of the same method (resampling) or the results of different clustering algorithms e.g. by tabulating:
km1 <- kmeans (iris [, 1:4], 3)
km2 <- kmeans (iris [, 1:4], 3)
table (km1$cluster, km2$cluster)

#      1  2  3
#   1 96  0  0
#   2  0  0 33
#   3  0 21  0

as the clusters are nominal, their order can change arbitrarily. But that means that you are allowed to change the order so that the clusters correspond. Then the diagonal* elements count cases that are assigned to the same cluster and off-diagonal elements show in what way assignments changed:
table (km1$cluster, km2$cluster)[c (1, 3, 2), ]

#      1  2  3
#   1 96  0  0
#   3  0 21  0
#   2  0  0 33

I'd say the resampling is good in order to establish how stable your clustering is within each method. Without that it doesn't make too much sense to compare the results to other methods.
* works also with non-square matrices if different numbers of clusters result. I'd then align so that elements $i,i$ have the meaning of the former diagonal. The extra rows/columns then show from which clusters the new cluster got its cases.

You're not mixing k-fold cross validation and k-means clustering, are you?
Answer: I'm trying to understand how would you apply cross validation to clustering method such as the k-means since the new coming data will change the centroid and even the clustering distributions on your existing one.
Regarding the unsupervised validation on clustering, you may need to quantify the stability of your algorithms with different cluster number on the re-sampled data.
The basic idea of clustering stability can be shown in the figure below:

You can observe that with the clustering number of 2 or 5, there are at least two different clustering results (see the splitting dash lines in the figures), yet with the clustering number of 4, the result is relatively stable.
Clustering stability: an overview by Ulrike von Luxburg might be helpful.


Relation between $T=0$ and $(Tx,x)=0$

Question: Let $X$ be a vector space and (.,.) be an inner product on $X$ also if we have a linear operator $T:X\rightarrow X$, then in both cases real and complex for inner product what is the relation between $T=0$ and $(Tx,x)=0$?
Answer: If $V$ is complex, then if $\langle Tv,v\rangle=0$ for all $v\in V$, then $T=0$. The proof comes from the identity:
$\langle Tu,w\rangle=\frac{\langle T(u+w),u+w\rangle-\langle T(u-w),u-w\rangle}{4}+\frac{\langle T(u+iw),u+iw\rangle-\langle T(u-iw),u-iw\rangle}{4}i$
For all $u$ and $w$ it is $\langle Tu,w\rangle=0$. This gives $T=0$.
In the real case you need to assume for example that $T$ is self-adjoint (the above identity then just lacks the imaginary part).
Comment: This identity, I think, assumes conjugate linearity in the second argument. Beware that you need to switch some minus signs around if your conjugate linearity is in the first argument.
Answer: Hints: If $T=0$, what is $Tx$? what is $(0,x)?$, what do you conclude? In the other direction, what is the geometric interpretation of $(x,y)=0$? Can you think of a linear transformation $T:\mathbb R^2 \to \mathbb R^2 $ such that $(Tx,x)=0$ without $T$ being $0$?
Comment: :If $T=0$ then $Tx=0$ and $T(x,0)=0$.(for both case real and complex inner product). Now in the other direction what happen؟
Answer: $T$ is a bounded linear operator on a complex vector space with $<Tx,x> = 0$ $\forall$ $x$ gives $T = 0$. This is not true in real vector space. Consider rotation in $\mathbb{R}^2$ in an angle $\frac{\pi}{2}$. Then $<Tx,x> = 0$ $\forall x$, but $T \neq 0$.
Comment: what about such a rotation in complex plane?  I can't conclude that it is non zero.

Relation between $T=0$ and $(Tx,x)=0$


.NET UML generation from code?

Question: Is there anything out there that for .NET that can generate UML diagrams from code. Preferably an addin for Visual Studio. Starting work on a mature project that has little architectural documentation can be painful at first. Eventually you get the ins and outs of the code but helping to see how the code all fits together from the get go would be wonderful.
Answer: In visual studio 2005/8 you can right click on a class and then select View in Class Diagram which will create a new ClassDiagram containing the selected class and any related classes.
Comment: Thank you, James.  In Visual Studio 2010 (Premium), 1. View > Class View.  2. Right-click on class > View Class Diagram.
Answer: If you have Visio and select Project->Visio UML->Reverse engineer you will get a uml of the project.
Sparx Systems has made a product called "Enterprise Architect" that should be able to do the trick as well.
Answer: If you generate UML class diagrams for a big project the result is going to be quite chaotic.
Sometimes I use the class diagrams in Visual Studio. I manually add the classes I think deserve some extra explanations. The diagrams are not UML, but it is close enough. They are always up-to-date and you can change the diagram and the code is updated automatically.
To convey the bigger picture of a design I use these UML stencils and draw the diagram by hand. For my points to come across to the people I am communicating with, I find it best to omit irrelevant details, so we can focus on what I think is important. No automatic UML generation tool can figure out which irrelevant details to omit.
Answer: Enterprise Architect does this and has an add-in for Visual Studio.
It will also do sequence diagrams which can be very useful.
Answer: Visual Studio 2010 Ultimate supports UML class, sequence, component, use case, and activity diagrams. It also supports creating sequence, dependency graphs, and layer diagrams from code. Regarding your question about generating UML diagrams from code, there's a response here in the VS Architecture & Modeling tools forum: Is it possible to reverse engineer C# code into an UML Class Diagram?
Other tools include Architecture Explorer, which lets you browse and explore your solution.
For more info, see the following links:
To download the RC release, visit: Microsoft Visual Studio 2010 Ultimate RC
To see the RC documentation, see Modeling the Application.
To discuss these tools, visit the Visual Studio 2010 Architectural Discovery & Modeling Tools forum.
Answer: Class diagram doesn't always work. I often find it wont display the classes for some reason. Pen & pencil or talking with people who work on the project is what I have to rely on.

.NET UML generation from code?


How are the penalized splines defined here?

Question: Based on 'Semiparametric Regression with R' (https://link.springer.com/chapter/10.1007%2F978-1-4939-8853-2_1), a penalized spline
$$
f(x)=\beta_{0}+\beta_{1} x+\sum_{k=1}^{K} u_{k}\left(x-\kappa_{k}\right)_{+}
$$
leads to the penalized least squares:
$$
\text { minimize } \sum_{i=1}^{n}\left\{y_{i}-f\left(x_{i}\right)\right\}^{2} \text { subject to } \sum_{k=1}^{K} u_{k}^{2} \leq C
$$
for some $C>0$. Or, with the lagrangian multiplicator:
$$
\operatorname{minimize}\left[\sum_{i=1}^{n}\left\{y_{i}-f\left(x_{i}\right)\right\}^{2}+\lambda \sum_{k=1}^{K} u_{k}^{2}\right].
$$
My question is: What does $u_k$ stand for? Are these equivalent to the coefficents in regression splines? And why is $u$ squared in the minimization problem?
Comment: I think this could be useful https://stats.stackexchange.com/questions/444325/representing-a-gam-with-truncated-power-basis-as-a-mixed-model/444466#444466. The u-coefficients are shrunk to ensure smooth estimates.
Answer: There are two terms in those equations,

a measure of the fit of the function $f()$ to the data
$$\sum_{i=1}^n (y_i - f(x_i))^2,$$ and

a measure of complexity for the function $f()$
$$\sum_{k=1}^K u_k(x - \kappa_k)_{+}$$

The complexity of $f()$ could be measured in a number of ways, but it's helpful if it can be written in terms of the coefficients of the function. That's what the $u_k$ are, they are the estimated value of the coefficient for the $k$th basis function in the spline basis used to represent $f()$.
I don't have access right now to the reference you mention, but from what you show, the basis functions are the $(x - \kappa_k)_{+}$, where the $\kappa_k$ are the knots of the spline and we are only taking the positive part of the function. (I think this is known as a truncated power spline basis.)
The $u_k$ are the coefficients for each basis function but they can also be thought of as the weight applied to each basis function; large weights for all basis functions implies a very wiggly function because each of the basis functions has a large "effect" in order to fit the data and that large "effect" must mean the data vary a lot locally and the spline has to adapt a lot locally to achieve good fit. Small weights indicate a simpler, less wiggly function because they imply the opposite, small local effects. Why local? As we're taking only the positive part of the function $f_k = (x - \kappa_k)_{+}$, each function only operates over part of the range of $x$, the bit to the right of the $k$th knot $\kappa_k$.
The coefficients $u_k$ are squared in the first formulation because a large negative value implies a large "effect" but which would cancel out some of the positive "effects" and thus we wouldn't be measuring the complexity of the spline if we only summed the $u_k$ – I can't quite remember now why the coefficients are squared and summed (although it likely has something to do with a Euclidean norm or measure of distance somewhere in the math that proves that constraining the squared coefficients minimises a particular measure of complexity of the function $f()$) instead of summing the absolute values $\sum_{k=1}^K |u_k| \leq C$.


Find the relationship between $n$ and $m$ (both natural numbers) such that $m^{1/n}$ is a rational number.

Question: I know how to show that specific numbers such as $2^{1/2}, 2^{1/3}, 3^{1/2}, etc.,$  are irrational, but what about the general form $m^{1/n}$?
Answer: Such  numbers are either natural or irrational.
Suppose $m^{\frac{1}{n}}=\frac{p}{q} \quad \Rightarrow m=\frac{p^n}{q^n}$.
If $m$ is not a $n$-th power of any natural number, there is a prime factor such that the highest power dividing $m$ is not a multiplie of $n$.
Now, what can we say about how often that prime occurs in the factorization of the left hand side of this equation? What about the right hand side?
Answer: Hint $ $ By the Rational Root Test, $\,m^{1/n}\,$ is rational iff it is integral. If so then $\,m^{1/n}= a\,$ for an integer $\,a\,$ so $\,m = a^n\,$ is a perfect $n$'th power; equivalently, the power of each prime in $\,a\,$ is a multiple of $\,n,\,$ i.e. $\,a = 2^{e_1} \cdots p_k^{e_k}\,$ $\Rightarrow\, n\mid e_i,\,$ for all $\,i.\,$ Or, said contrapositively, $\,m^{1/n}\,$ is irrational if some $\,e_i$ is not a multiple of $\,n.\,$ The prior inferences depend crucially on the Fundamental Theorem of Arithmetic, i.e. the existence and uniqueness of prime factorizations.


Managed Property for Site/Subsite that document lives on?

Question: Is there a managed property that will contain the URL for the site or subsite that a document lives on? I want to pull all documents from a site collection and have them contain a link to the Document library that they live in, but NOT the document itself. Due to what im using, i would rather not have to parse the Path variable and would like to see if there is an easier solution. Thanks
Answer: I have found the property im looking for. With SPWebURL i can pull the url of the site or subsite the document lives in.


which live/rescue disk it is which gives you an option to boot into your GNU/Linux OS without grub?

Question: I remember a live CD/DVD where I could just boot into my GNU/Linux partition in case your grub was shot. The simplest thing was once in your GNU/Linux partition, install grub-pc and its brethen and simply do update-grub and all would be well. I tried grml and know it's not grml, maybe knoppix or some other live-rescue distro ?
Comment: When you have a UEFI system you don't need grub when the kernel is built with EFISTUB (which current Debian kernels are). So in case your grub installation is broken you can directly boot the kernel from the UEFI and fix your grub afterwards.
Comment: what if it's NOT an EUFI system, such old systems still are there you know :)
Answer: the simplest way is to use isohybrid_bios_efi_grubrescue image meaning you can boot either as cd or usb on either oldPal_biosPC or on newer uefiPC
and
           you are in grub commands in any of these cases
see demo and explanation here
https://sourceforge.net/projects/toysbox/files/bios-efi_GrubRescue/
tested on "old i686 2GB" and on "Acer Nitro efi 4Gb"
enjoy


MacBook Pro - Which color profile is more realistic and like life?

Question: On the MacBook Pro, which color profile is more realistic and life like?
Color LCD or SMPTE RP 431-2-2007 DCI (P3)?
Answer: Unless you have a colorimeter to make an accurate profile, the best profile is the default profile, which will depend on your exact MBP model.
In System Prefs > Displays > Colour check the box for 'Show profiles for this display only' which will include system defaults plus any custom profiles specifically made for that display.
Answer:
Which color profile is more realistic and like life?

There is no way to know without calibrating your display. If color accuracy matters, buy a colorimeter. Older models are cheap and better than not calibrating at all.
According to How to Color Calibrate Your Mac's Display, the default profile is probably Color LCD, but depends on the specific display model.


Does reset do the same thing as shutting off the power to this device and turning it back on?

Question: I am trying to control a serial dataflash memory through an mbed. It is an AT45DB021B and the datasheet is here. It has a reset pin that I have wired to the mbed so that a software reset can be done. I heard that a software reset makes things easier so that a hardware reset, i.e. shutting off power and turning it back on, is not needed. I noticed that when I do a hardware reset, whatever is in the buffer is overwritten by random values. However, when I do a software reset, the buffer is unaffected. Nothing seems to change on a software reset. Is this normal?
Answer: Yes this is normal. If you do a hard reset(random powerup) you can be pretty sure that the buffer will be some garbage. If you do a soft reset the contents should be the same as before. Sram is designed to behave that way. But it depends on the design of the architecture. If the internal hardware is desinged in such a way that, on receiving a reset signal, the power to the RAM to be removed. Then, in that case, the contents of the RAM will be cleared on the reset.
Answer: After a power-on reset there's no guarantee as to what's in your buffer.  It may be the same as what was in it before, or all zeroes, or all ones, or a checkerboard, or most of the opening sentence of "A Tale of Two Cities", or anything else; it may be consistent, or random, or mostly consistent except for the third Tuesday of months containing the letter "a".  Unless specified otherwise, it's probably wise to figure there's no guarantee of buffer contents after a soft reset.  The proper approach is to assume that the buffer is going to contain whatever data would be most vexing to you, so you should be prepared to accept anything that might be there.
Answer: The reason memory and other chips have reset lines is not neccessarily to "clear" them, but to to disconnect them from the bus during low power states.
The problem is that, during power-up and shutdown, the strobe, address and data lines are still connected to the bus while power is rising from 0v to Vcc (or draining away), and this may manifest as digital noise that causes random "garbage" to be written to memory during bootup.   This is especially problematic for memory-mapped IO devices, and flash memories.
You can use a "brown-out reset" chip to ensure that your device is held in RESET state until power has stabilized and the bus values can be trusted.  Many modern microcontrollers have built-in brown-out detectors.
Answer: Datasheet covers this:

RESET: A low state on the reset pin
  (RESET) will terminate the operation
  in progress and reset the internal
  state machine to an idle state. The
  device will remain in the reset
  condition as long as a low level is
  present on the RESET pin. Normal
  operation can resume once the RESET
  pin is brought back to a high level

So you don't lose buffer values but any operation in progress is stopped and the state machine is reset.  So the data in the buffer is invalid (does not line up with current state and may not line up with whats actually in flash, it should be treated as garbage).
If you trip the software reset while a write or read from flash is occurring the operation stops and upon release of the software reset the chip is in its default state.  This means that the software reset can corrupt flash data without changing the buffer values.  When you write to the flash you have to make sure you allow enough time for the chip to actually write the data to the flash before using the software reset or your data will be not make it to the flash.  These times are listed in section 8.2 of the datasheet.
Strictly speaking it is also not safe to read from the data buffer as if it were valid after a software reset, you must re-issue a command to copy data from flash to the buffer before you can read it and be assured that it is valid (equivalent to whats actually in flash).
Additionally:

The device incorporates an internal
  power-on reset circuit, so there are
  no restrictions on the RESET pin
  during power-on sequences. If this pin
  and feature are not utilized it is
  recommended that the RESET pin be
  driven high externally

and

When power is first applied to the
  device, or when recovering from a
  reset condition, the device will
  default to SPI Mode 3. In addition,
  the SO pin will be in a high-impedance
  state, and a highto-low transition on
  the CS pin will be required to start a
  valid instruction. The SPI mode will
  be automatically selected on every
  falling edge of CS by sampling the
  inactive clock state. After power is
  applied and VCC is at the minimum
  datasheet value, the system should
  wait 20 ms before an operational mode
  is started

So there is no need for an external controller to bring the chip online on power up.  Unless your power on operation takes more than 20ms to stabilize supply voltage you shouldn't have any power on garbage issues.
Answer: In general, no, a reset is not the same as a power cycle.  You have to analyze the system on a case by case basis to determine what happens on a system wide and/or localized reset.  From Mark's answer it sounds like you are fine for this target, press reset and not have to power cycle.  Saying that before deploying or releasing a version of software though, test it using power cycles as well.
With sram or dram.  If you dont lose power and the reset is short enough the memory may very well look like nothing changed, but dont rely on that.  Some systems, dram in particular, parity checked, ecc checked memories will go through an initialization process which can include wiping out the memory.  Unless the system was designed to preserve the memory through a reset, do not rely on it being the way you left it.  Programs should always write before they read a memory location (or the location should be written by someone/something before it is read the first time) anyway.


Should I report hacking attempts?

Question: I am running a small (Windows-based) server. When I check the logs, I see a steady flow of (unsuccesfull) password-guessing hacking attempts. Should I try to report those attempts to the owners of the source IP addresses, or are these attempts nowadays considered completely normal and nobody would bother doing anything about them, anyway?
Answer: While the answer can depend greatly on the agency you are attempting to inform, I believe that in general you should. In fact, since monitoring and responding to the abuse mailbox for our organization is one of my primary job duties, I can positively say, 'Yes Please!'. I had this same conversation with members of other security organizations and the answers seemed to largely consist of:

If the whois information on the IP shows a business or university, then report
If the whois information on the IP shows an ISP, then don't bother

I, of course, won't tell you to follow those rules, but I would recommend erring on the side of reporting.  It usually doesn't take much effort, and can really help out the guys on the other end. Their reasoning was that ISPs aren't often in positions to take meaningful actions, so they will file the information away. I can say that we will aggressively pursue the matter. We do not appreciate hacked machines on our network, as they have a tendency to spread.
The real trick is to formalize your response and reporting procedure so that it can be consistent between reports, as well as between staff. We want, at minimum, the following:

IP address of the attacking system
Time stamp (including time zone) of the event
The IP addresses of the systems on  your end

If you can also include a sample of the log messages that tipped you off, that can also be useful.
Normally, when we see this kind of behaviour, we  also institute firewall blocks of the most appropriate scope at the most appropriate location. The definitions of appropriate are going to depend significantly on what is happening, what kind of business you're in, and what your infrastructure looks like. It may range from blocking the single attacking IP at the host, all the way up to not routing that ASN at the border.
Comment: Thanks - good to know.  Are there any simple-to-deploy tools to help automate such reports?  Identifying the sorts of abuse that are useful to report, finding who to report to, including the useful information, dealing with reports that get bounced, etc?
Comment: @Nealmcb - there are fun expensive IDS systems that can sum up all of this. I've seen Cisco MARS do it. I dunno if there are cheap/free options that make this easy, but if your set of logs are small, you can probably write up a logscraper to present you with an easy-to-use report.
Answer: This is password-guessing attack known as a brute force attack. Best defense is to make sure that users passwords is strong. Another, solution is to lock out an IP address with multiple failed logins. Brute force attacks are difficult to stop.
Answer: As what lynxman said all you really can do is contact their ISPs Abuse department and inform them. I would block that IP both in the Firewall and on the server. Second I would also setup attempt based lockout in Group policy(if you have AD). As long as your Passwords are strong I wouldn't worry about it, I have Servers that I run to learn and I get login attempts all day long.
Comment: Contacting their ISPs is what I meant (nothing important happened, the attack was not successful, therefore contacting the ISP is all I might want to do) – should I do it, or is it a waste of time?
Comment: @mormegil It depends I usually do but if its in russia or a country in the old soviet bloc I don't bother. They can null route to you which will get traffic from him to you to stop.
Answer: We get hundreds of hacking attempts on our web servers every day.  Perhaps that is because we have had a web presence with about 40 sites for over 20 years.  The hacking is bad enough for us to have written a 404 filter that searches for vulnerability patterns that generate 404's.  When a vulnerability pattern is detected, we blackhole the IP for 3 days.  We can see that hackers use lists of vulnerabilities and sequentially try them one at a time.  We generate a daily report of the hack attempts, usually about 400 per day, and look up the netblock of the ISP originating the hack attempt.  If the attempt is obvious and egregious, we add the netblock to a DB table with their abuse reporting address.
There are both responsible and irresponsible ISPs.  In our experience, the following really want to receive your abuse reports in an email message:
<code>- Microsoft (Azure)
- Sendgrid
- Comcast
- Amazon (AWS)
</code>
The following do not want to receive your abuse reports, but they respond back:
<code>- Charter (insists that it is not from their IP)
</code>
The following ignore your abuse reports:
<code>- Digitalocean (netblock had to be firewalled permanently)
- ovh.net, ovh.ca, ovh.us (netblock had to be firewalled permanently)
</code>
The following countries host ISPs, and they ignore your abuse reports:
<code>- Russia
- China
- Indonesia
- Netherlands
- India
- Pakistan
- Bulgaria
- Vietnam
</code>
There are, of course, many others, but these are the most egregious.
Answer: Unfortunately it's completely normal, most of this attempts are generated through other servers that have been hacked as well.
The best you can do is that if you see these attacks coming persistenly from a unique IP address and you have suspicion that the server got hacked is to email the abuse/sysadmins at that server so they can fix the situation, it's quite easy to lose track of a server when you're overloaded and maintaining hundreds of them.
In any other case firewalling, filtering or ignoring is mostly a good practice.
Answer: Your problem here is that the vast number of these are likely to be coming from compromised machines, in various countries, that are probably home users' PC's and are probably on dynamic addressing schemes.
Which means that the owners of the machines don't know they are forwarding attacks, and don't care, they may be in countries where the law really doesn't care, and the ISP's probably don't care and in any case won't want to trawl logs to see who was using that IP address.
Best plan is a combination of lynxman's, Jacob's and packs' - generally block them, but set up a script to see if there are common culprits and specifically send your comms to the Abuse departments of those ISP's.
Better use of your time that way.

Should I report hacking attempts?


Scheduling a task in PHP MVC Codeigniter

Question: I want to schedule a task which is a controller method in MVC CodeIgniter. Also, I want to add the option in my web application for the admin to change the schedule instead of working around windows task scheduler manually. I found some examples for PHP Web Form scheuler but not for MVC CodeiIgniter! Could you please guide me in this regard?
If my question is not clear please let me know which part you need more clarification!
Many thanks in advance,
Comment: Have you ever looked at http://atrigger.com/ ? Platform independent.
Comment: Sorry I forgot to mention I need it for a Windows server! Cron is for Linux, right?
Comment: Oh nice buddy! It seems nice! I'll try it! :-)
Answer: First of all scheduling a task is a OS base Operation, so you cannot control it through your application unless you have full rights to your server, however there are some ways where you can achieve this
Check this article it will help you about how CI controls the cron job.
If you want to change/create the time of the cron job through your application then you can use
exec function which will execute your passed command in the shell of windows but remember this function will only work when your hosting provider has enabled it or you have rights to this function and this answers will tell how you can use schedule task (CronJob) on Windows through command line.
Answer: Do this,
add this on config/routes.php
<code>$route['my_batch'] = 'your_controller/function_name';
</code>
In your your_controller:
<code>public function function_name(){
        your task....
        }

    }```
now go to c-panel and search for cron job.

1.select custom command
2. put this command "curl --silent http://your_domain.com/your_controller/function_name" (without quotes).
3. select timings and save it.
</code>
Comment: apologies put  $route['your_controller'] = 'your_controller/function_name'; on routes.php
Answer: I am not sure if I understand your question, but to schedule a task if working on a linux server you can use a cron job.
Suppose you have a controller called Welcome with a function called task, then you can use a cron job similar to this:
Cron_configuration path_to_bin_php index.php  welcome task
Here is a link with more details on cron:
http://www.thegeekstuff.com/2009/06/15-practical-crontab-examples


Why is fog in games exponential and has a start distance?

Question: In most tutorials I see on how to achieve fog with shaders the fog is exponential and has a minimum / start distance from the camera. Why? Both seem counterintuitive, especially the minimum distance. Is this an artistic / design choice or is this how our eyes see fog in real life?
Comment: Does not seem counter-intuitive to me. In fog you see pretty clearly just in front of you, then in becomes a bit foggy and not far after that you cannot see much anymore
Answer: This is a legacy from fixed pipeline fog equations, where you typically had one of three options available:

Linear blended between a start distance and an end distance.
Exponential based on density.
Exponential-squared based on density.

For example, the documentation for glFog and for Direct3D 9 fog equations.
A sample tutorial making reference to these three equations (you can cross-check with the documentation and determine that they are the same).
Another sample tutorial referring to fog in the fixed pipeline versus implementing it yourself in a shader.
GLSL 1.10.59 specification containing documentation of built-in uniforms for fog, allowing glFog calls to be used with shaders.
Using shaders you are of course not restricted to these equations, and you can adapt them, modify them, or write your own as you see fit.  However, tutorial material does tend to either emulate the older equations or take them as a starting point, because they are a familiar reference (and early versions of GLSL even provided access to glFog parameters via built-in uniforms).  You see the very same with lighting, where tutorials typically follow the old fixed pipeline lighting models too (and likewise early versions of GLSL provided access to lighting state too).
(Interesting aside: fixed-pipeline fog did not support specifying a start distance with exponential fog which suggests that some tutorial authors may not have paid as close attention to the documentation as they could have).
Comment: Fog formula use an exponential because exponential is close to the physics of what happen in real life. Tho, with shaders and powerful GPU, we can do better simulations of the atmospheric haze and light scattering in homogeneous media :)


How to get +HHMM offset of TZ="Asia/Tokyo" compared to current time-zone using bash?

Question: I am taking input for TZ and I want to convert it into a (+/-)HHMM format.
but not compared with UTC, I want it to be compared with the local time.
How can I do it?
Answer: This isn't done using bash.
This is a fairly trivial exercise in the use of the date command and the (widely available but ironically not standardized for date even though it is for strftime()) %z format specifier.  TZ="Asia/Tokyo" date +"%z"
This obtains the current UTC offset, accounting for daylight savings time switches, of course.
Various implementations of date allow one to display arbitrary timestamps instead of the current time, without attempting to set the system clock, and so obtain the UTC offset for other times of the year.
For example: FreeBSD date has a -j option and takes input timestamps (by default, if -f is not used) in a strictly descending form.
% TZ="Europe/Lisbon" date -j +"%z" "202401060000"
+0000
% TZ="Europe/Lisbon" date -j +"%z" "202407060000"
+0100
%
There are also tools such as zdump, but the date command is the obvious one here.
Further reading

date.  2018-08-04.
BSD General Commands Manual.  FreeBSD.
How can I have `date` output the time from a different timezone?
Why does TZ=UTC-8 produce dates that are UTC+8?
Need to get the Timezone details
"Utilities: date".  Shell Command Language.  Single UNIX Specification.  Issue 7.  IEEE 1003.1.  2016.  The Open Group.
Comment: Thank you @JdeBP , sorry for troubling again but I want the offset in comparison with local time. :(


Python Glob without the whole path - only the filename

Question: Is there a way I can use glob on a directory, to get files with a specific extension, but only the filename itself, not the whole path?
Answer: Use os.path.basename(path) to get the filename.
Comment: This is not exactly the same as relative glob though. Imagine the pattern is `"sql/*.sql"`. The `os.listdir` function would actually list file paths relative to the given directory, though it does not work with patterns. EDIT: Ah, I was looking for this answer: https://stackoverflow.com/a/44994860/1317713 The trick is using `os.path.relpath` rather than `basename`.
Comment: thanks for the answer. How to get rid the extension part..such as .csv, .dat, .nc, etc
Answer: This might help someone:
names = [os.path.basename(x) for x in glob.glob('/your_path')]
Comment: Alternatively they would use [os.path.relpath](https://docs.python.org/3/library/os.path.html#os.path.relpath) instead of [os.path.basename](https://docs.python.org/3/library/os.path.html#os.path.basename) if they want a relative path to `'/your_path'` rather than just the filename.
Comment: relpath or just `old = os.getcwd(); os.chdir(dir); files = glob.glob('**/*.txt'); os.chdir(old)`
Answer: map(os.path.basename, glob.glob("your/path"))

Returns an iterable with all the file names and extensions.
Answer: os.path.basename works for me.
Here is Code example:
import sys,glob
import os

expectedDir = sys.argv[1]                                                    ## User input for directory where files to search

for fileName_relative in glob.glob(expectedDir+"**/*.txt",recursive=True):       ## first get full file name with directores using for loop

    print("Full file name with directories: ", fileName_relative)

    fileName_absolute = os.path.basename(fileName_relative)                 ## Now get the file name with os.path.basename

    print("Only file name: ", fileName_absolute)

Output :
Full file name with directories:  C:\Users\erinksh\PycharmProjects\EMM_Test2\venv\Lib\site-packages\wheel-0.33.6.dist-info\top_level.txt
Only file name:  top_level.txt

Comment: You have mixed up your variable names: absolute means full path; relative means only the base name.
Answer: Or using pathlib:
from pathlib import Path

dir_URL = Path("your_directory_URL") # e.g. Path("/tmp")
filename_list = [file.name for file in dir_URL.glob("your_pattern")]

Answer: I keep rewriting the solution for relative globbing (esp. when I need to add items to a zipfile) - this is what it usually ends up looking like.
# Function
def rel_glob(pattern, rel):
    """glob.glob but with relative path
    """
    for v in glob.glob(os.path.join(rel, pattern)):
        yield v[len(rel):].lstrip("/")

# Use
# For example, when you have files like: 'dir1/dir2/*.py'
for p in rel_glob("dir2/*.py", "dir1"):
    # do work
    pass

Answer: If you are looking for CSV file:
file = [os.path.basename(x) for x in glob.glob(r'C:\Users\rajat.prakash\Downloads//' + '*.csv')]

If you are looking for EXCEL file:
file = [os.path.basename(x) for x in glob.glob(r'C:\Users\rajat.prakash\Downloads//' + '*.xlsx')]

Answer: None of the existing answers mention using the new pathlib module, which is what I was searching for, so I'll add a new answer here.
Path.glob produces Path objects containing the full path including any directories.  If you only need the file names, use the Path.name property.

If you find yourself frequently converting between pathlib and os.path, check out this handy table converting functions between the two libraries.
Answer: for f in glob.glob(gt_path + "/*.png"):  # find all png files
      exc_name = f.split('/')[-1].split(',')[0]

Then the exc_name is like myphoto.png
Answer: Use glob.glob("*.filetype") to get a list of all files with complete path and use os.path.basename(list_item) to remove the extra path and retain only the filename.
Here is an example:
import glob
a=glob.glob("*.pkl")

It returns a list with the complete path of each file ending with .pkl
Now you can remove the path information and extract only the filename for an list item using:
import os
b=os.path.basename(a[0])
# This is an example to extract filename for only one list item

If you need to create an entire list with only the filename:
bb=[os.path.basename(list_item) for list_item in a]


Tag: Avatar: The last Airbender

Question: In this question, I mentioned that the tag avatar-the-last-airbender may not the best choice for the question, as the tag only covers the first series, Avatar: The last Airbender. The question, however covers both, Avatar: The last Airbender and The Legend of Korra. Should there be a second tag the-legend-of-korra, or should the Avatar-Tag be generalized? A new tag can make searches more specific, however both series are simply referred to as Avatar, most times.
Comment: Seeing as how it's not even anime, it should, IMO, not even be allowed on this site.
Comment: @coleopterist: That was already [discussed](http://chat.stackexchange.com/rooms/6697/conversation/third-chat-cast-expanding-our-scope-2013-02-09) and it was decided to allow it as a pilot.
Answer: I think that in this case, the avatar tag should be created and used for both The Last Airbender and The Legend of Korra.  Madara's example of having a naruto and naruto-shippuuden tag to tell the two series apart isn't a very good one since everybody just uses the plain Naruto tag for both series.
To deal with differentiating the two series, I propose that avatar be used for both with related synonyms for avatar-the-last-airbender and legend-of-korra.  This way searching "avatar" will yield results for both series as they're both commonly referred to as Avatar and searching for either of the synonyms will give results for those particular series.
Comment: Are you sure that [tag:avatar] will not be used for anything else? Other than The last Airbender or Legend of Korra? Is there no other anime which includes "avatar" in the name?
Comment: I'm not exactly a fountain of knowledge when it comes to names of anime/manga but I don't know of any off the top of my head.  I'm sure it could be used for other animes though if they use the word avatar in the name.
Comment: Using a single tag to reference a series and a contiuation of that series is fine. Using a single tag for a lot of different series is not.
Answer: I think a second tag is in order. If we have naruto and naruto-shippuuden, we should have both tags.
Questions from both tags can be searched for by searching for [avatar*].
Comment: I should learn how to use the search ...
Comment: I'm not entirely certain your example was a very good one considering nobody uses the shippuuden tag.  All Naruto questions have been asked under the generalized Naruto tag whether they're from the original series or shippuuden.  In fact, looking at that tells me that it might be better to just have a generic avatar tag and use the other two tags as synonyms.
Comment: @MCeley: Hmmm, didn't think of that (*Y U NO DOWNVOTE?!*). That may be true. Post an answer detailing it, and see how many votes you get.
Comment: I use the main sites a lot but rarely use meta. My bad yo!

Tag: Avatar: The last Airbender


Punctuation of the Expression "The Works"

Question: I'm feeling really sick today, and I wanted to text my friend to tell him why I won't make it to our meeting. I wanted to say "I'm feeling pretty sick. Fever, headache, cough, the works.", but I've never used or even seen the expression in print before. What is the correct punctuation for this expression?
Comment: Looks and sounds good the way you have it.
Comment: There's nothing really special about "the works".  In the above context it's part of a list and hence is separated by a comma, but you could order a hamburger "with the works" or some such and have no commas near the phrase.
Answer: The way you have it: "the works" is fine.
Answer: I agree it looks / sounds fine as you have it. Obviously you would only write this in an informal context.
You could also combine it into one sentence, i.e., "I'm feeling pretty sick -- fever, headache, cough, the works."


Is Order-preserving encryption part of the functional encryption family?

Question: I believe I know quite well OPE and ORE, but I'm unsure about what family to put them in. Can we consider them as a sub family of Functional Encryption, like Attribute Based Encryption or Inner Product Encryption or are they a new paradigm of their own ?
Answer: No, order-preserving encryption is not a subclass of functional encryption as usually defined, that is with public-key encryption.
Argument: In such functional encryption, encryption uses a public key. But we can't have secure public-key order-preserving encryption, because knowledge of the public key allows adversaries to encrypt, and ability to encrypt combined with the order-preserving property allows to decipher by dichotomy.
As noted in comment, that argument does not apply to Secret Key Functional Encryption, a notion I'm unfamiliar with. So I reserve my opinion on if OPE can be obtained in the framework of SKFE.
Comment: [Secret-key functional encryption is a thing.](https://eprint.iacr.org/2021/1018) So while I agree with your conclusion, your argument doesn't quite track.
Comment: @Maeher I was going to comment that... FE for inner product suffers from more or less the same problem fgrieu pointed out: given a functional key for a hidden vector $v$, if one is able to encrypt several vectors $u_i$ by themselves, then they can decrypt several inner products $u_i \cdot v$ and then recover $v$ by linear algebra... So this type of FE for inner product is also secret key FE.


Git on Raspberry Pi?

Question: I need to get git (https://git-scm.com/downloads) on my computer so I can download a repository onto my Raspberry Pi from GitHub. Does anyone know how I can do this?
Answer: Assuming you are using Raspbian on your Pi, the folllowing command will install git:
sudo apt-get install git

depending on the version of Raspbian you are running this may already be installed.
You will then want to set some basic configuration values by entering the following at the command line:
git config --global user.name "John Doe"
git config --global user.email firstname.lastname@example.com

substituting your name and email in the above commands.
You should then be able to git clone the repository with the following command:
git clone URL_of_the_repository

replacing URL_of_the_repository with the URL of the repository you want to clone. You can find this URL on the homepage of the repository.
You can learn more about git from the following resources:

Pro Git book.
Git tutorial code school.
Answer: I thought git was already included in Raspbian (you don't actually tell us what you are using). If not sudo apt install git
Comment: Well, it would make sense if it was. I did not know if so, because im having some trouble with turning on my pi, so i was planning to use git to get my work on there and test from there. I'll try it tomorrow when i should get it working. https://www.youtube.com/watch?v=3YmMNpbFjp0

Git on Raspberry Pi?