Scraping a Website only for its Text

Question: I am helping remodel a website and was wondering if it was possible to scrape just the Text out of the entire site. Doing a page one at a time using DATA SCRAPER is possible but there is hundreds of pages that need to be worked on. Is there a way to get them all in one scrape? Or further suggestions?
Answer: If I understand your question correctly, there is a standalone program called HTTrack (https://www.httrack.com/) that will download an entire website to your local computer. I've used it successfully in the past when I need to grab everything.
Edit: This is my first answer on stackoverflow. Why was this voted down? I'd like to know so I don't do it again.
Comment: No idea why you were voted down, but you do understand for the most part. The problem is the way the site was written. I will give httrack a go and get back to you.
Answer: For those who landed here,
If you want to scrape an entire website and get only its text content, you can use scrapy.
The below github repository does the job.
https://github.com/mnk-reddy/Website-Crawler
You can change the depth and max pages to crawl using the DEPTH_LIMIT and CLOSESPIDER_PAGECOUNT in settings.py
The output is saved as a csv file with the url and text content in each record. Note this scarpes only the domain given to it. Hyperlinks to other domains are not scraped.

Scraping a Website only for its Text


How to widen the terminal window in Cloud9 IDE?

Question: Is there a layout in the Cloud9 IDE that will widen the terminal window to the full width of the browser? (similar to how Nitrous.io used to look, picture attached). Wondering if this is possible in the C9 settings.
Answer: If you click on the Workspace panel on the left side, you can hide your files, thus allowing the terminal to be nearly full screen. Otherwise, the terminal is confined to be nearly full-screen.
Answer: you can use the following init script to change the cloud9 layout
var layout = services.layout
var s = layout.getElement("searchRow")
var c = layout.getElement("consoleRow")
var cp = c.parentNode
var h = layout.getElement("hboxMain")
var hp = h.parentNode

cp.removeChild(c)
s.parentNode.removeChild(s)
cp.insertBefore(s, cp.firstChild)
hp.removeChild(h)
hp.appendChild(c)
c.insertBefore(h, c.firstChild)

use Cloud9>Open your init script menu item to edit it.
Then save and press ctrl-enter (cmd-enteron mac).


Program output that is neither STDOUT or STDERR?

Question: I'm getting some really strange behavior when trying to capture the output of django's manager.py
<code>> ./manage.py runserver 0.0.0.0:2869
Validating models...

0 errors found
Django version 1.3.1, using settings 'polling.settings'
Development server is running at http://0.0.0.0:2869/
Quit the server with CONTROL-C.
Error: That port is already in use.

> ./manage.py runserver 0.0.0.0:2869 >stdout 2>stderr

> cat stderr
Error: That port is already in use.

> cat stdout

>
</code>
Why am I getting the empty string when I try to capture the output on the second run?
Answer: Any program could detect if its STDOUT and/or STDERR connected to terminal or not: man isatty(3). Python also has such a functionality: sys.stdout.isatty().
Probably your python script or its logging subsystem prints the lines that missing in the second run only when running on terminal i.e. in interactive mode. It's a common and proper practice. For example, there is no sense to print <code>"Quit the server with CONTROL-C"</code> if the output is redirected to a log file.
Below is a simple illustration:
<code>#!/usr/bin/python

import sys

print "Print always."
if sys.stdout.isatty():
  print "Print only on tty!"
</code>
Here we go:
<code>$ ./test.py
Print always.
Print only on tty!
$ ./test.py > log
$ cat log
Print always.
$
</code>
Comment: How could I detect if this is the case? Is there a way that I can call the program and set isatty() to true? (I assume that it's something like Bash that sets this)
Is there a way to at the very least "snoop" in on the output?
Comment: I'm pretty sure, this is the case. You could check it by reading the script: for example, find string `"server is running at"` and look, how it is printed. Is there some test for `sys.stdout.isatty()` or does it uses some kind of log() instead of print(). If it does, then look for that `log()` code and so on.

You can not set isatty() to anything - the system does it. And there is nothing you can or should "snoop" - the script just **does not** print that lines when its output is redirected to a file. It just could detect whether the output is redirected or not.
Comment: What I meant is that if I set the output to STDOUT then I still get the output, is there a way to still somehow capture what is sent to STDOUT in this case?
Comment: What do you mean by "set the output to STDOUT"? STDOUT **is** the standard output of the program. That means the standord print function(s) will output to it. At runtime that STDOUT could be "connected" to a terminal, to a regular file, to a pipe, etc. Program could determine, whether its STDOUT connected to the terminal or not and handle it according to its logic. For example, program could print or not print something to STDOUT. In this case, if you want to get all the output, you should change the program (script in your case). But I don't really understand, what for.


Expatriates sucks because most questions are about Immigration

Question: As an expatriate, I was really hoping for interesting questions and dialog about the expatriate life; that is, best practices in getting settled with a new culture, how to balance the responsibilities of dual citizenship, even how to approach renunciation, or alternately how to vote remotely, etc.
Instead, nearly every question is about immigration and generally, how to become an expatriate or worse, how to game the host country's laws or customs in order to immigrate.
It is my opinion that the Expatriates experiment is unsuccessful and to make it good, we should first consider creating an "Immigration" question board to offload all these questions which are uninteresting from the perspective of ex-pat life.
Comment: As StrongBad says in his answer, I think the best way to change this would be if you'd ask some interesting questions yourself!
Comment: Immigration is a huge *huge* piece to the puzzle of being an expat.  If you've ever had the experience of working in another country, the laws and the amount of paperwork you must create, compile, photocopy, collate, print, staple, sign, and stamp can be staggering, especially in India.  With that said, I'd be interested in information on how to vote remotely because that will be something that will come up with me. So if you don't ask it eventually, I will. :)
Comment: I've seen a few instances where questions like "how does this particular thing in the French healthcare system work" are voted to be closed because it's not about emigrating. Except that these are, I feel, the kinds of questions this community is about - maybe you don't know the local language or the local administration very well because -- you've guessed it -- you're an expat. Surely that's on topic! But I've been surprised here more than once.
Comment: I've been a transnational for more than ten years now. I would rather answer questions for others. BUT: I am not an immigration lawyer.
Comment: Hi jmort253, you need to register in a state that believes you are still a resident somehow. https://vote.usa.gov/
Comment: I'm having trouble understanding your proposed solution, and feel like I'm missing something that you might be able to spell out better. You feel an almost-empty site would be more welcoming for interesting questions than one containing boring questions?
Comment: Dan that's a very good question. If I could get to the heart of it, I am here because I have learned a lot in the years since I left my home country, and I would like a chance to help others by answering questions. But, I am unqualified qualified to answer 100% of immigration questions; and I can only offer helpful advice in a narrow 0.1% (*). I think that immigration questions - unless our site is full of immigration lawyers willing to give advice for free - are going to attract low quality answers based on guesswork.
Comment: (*) where the asker's situation is identical to mine, and I know for a fact that the law underlying the question has not changed (in my country, the immigration law is redesigned with every new government change)
Comment: @jmort253 actually, as a US expat you are supposed to vote in the state you last lived in.  The state does not need to "believe you are still a resident."  But that's off topic for meta.  Ask on the main site!
Comment: @phoog - I didn't ask on meta, I said I'd be interested in asking, as an example of an on-topic question. ;)   If I have time, I may ask on the main site.  Time is not something I feel I have a lot of at the moment. Cheers.
Comment: Yeah, how can we get rid of all these brown "immigrants" and their visa problems and all these newcomers who want to partake in our lifestyle?
Comment: Note that asking a question to be able to post your own answer is perfectly fine (if that question follows the guidelines).
Comment: I'd like to follow up on my observation from two years ago.  Apparently, Travel.stackexchange.com ALSO sucks now, for the same reason. I just scrolled through a page of 30 travel questions, and all but 19 of them concerned visa applications, including multiple people asking how to make the best of their situation after having violated the immigration rules of their host countries.

I think it is a sign that people are transnational now, and governments have really not caught up with this trend, except for the occasional minor panic about a "migrant crisis".
Answer: While "dialog" doesn't work well in the SE Q/A model, questions that go beyond immigration and getting settled would be great. You should ask some.
We are already a small community and doing things to actively make us smaller would be a bad idea.
Comment: @DouglasHeld we do not need to be large, but there is a minimum size that a community needs to be to be viable. Currently we are averaging 2.6 questions per day. That is pretty small and we should be looking for ways to grow and not shrink. I do not see immigration and expat life questions as being in competition with each other. We just do not get too many expat life questions.
Comment: Why does the community need to be large? I think it would be more important to be focused; this would in turn lead to better quality content.
Comment: @DouglasHeld In practice, there are three reasons why we need a larger community: (1) to have enough high-rep users to complete basic maintenance tasks like voting to close questions in a timely manner without too much burden on the moderators or a handful of key users, (2) for new questions to have any chance of reaching an actual expert (that's what this site is about, not free-wheeling discussion in which everybody can in principle participate) and (3) to avoid dying down and being closed (SE won't let a site with a question a day and nobody to edit them go on forever).
Answer: I find it a bit annoying to see people who don't have this kind of problems complain about, in effect, having to see a handful of questions they personally don't find interesting. Java questions do not prevent anyone from talking about Excel or Python on Stackoverflow. And we have a lot of great answers to visa/residence permit questions, it's not like they create special difficulties as your comments would seem to imply. Objectively, that's the one expertise we do have at the moment (at least for some part of the world).
It's true that we could do with more questions about other aspects of life in another country but I have not seen many good questions or answers about that. One very clear requirement of the Stackexchange platform is that we don't want to turn this into a forum. "Dialog" is not what we want and the best questions should in principle have an objective answer (a solution to a problem, not various opinions on a topic). That's why questions about dealing with a foreign culture are actually more problematic than visa questions.
Most importantly, what we have is clearly too few rather than too many questions, what actual problem would yet another split solve? Visa and immigration questions are explicitly on-topic and this site was created in large part to handle them (because travel.SE did not like them). Anybody who wishes for something else is free to go through the usual site-creation process and see if they can get anything done but why should people who are interested in visas and the like take care of this or let you hijack this site? I don't think getting rid of the bits that actually work is going to make it any better.
Incidentally, questions about voting abroad, dual citizenship, etc. are in fact very similar to visa questions in that they require some legal expertise (or at least raise legal problems) and we actually have plenty of those! In practice, the ones that we don't really know to handle right now are questions about taxes and running a business between two or more countries.
Answer: Questions can be closed when "it's not about an issue a local wouldn't have as well.". I guess this partly explains why immigration is one of the most common topic here, since many non-immigration questions that expats would ask can be closed under that reason.
Comment: The policy on this is incredibly confusing and is the reason I've never even tried to use this site. That question suggests non-legal questions are okay, but [this one](http://meta.expatriates.stackexchange.com/questions/247/) suggests they're not, and (as always) people vote based on the strictest, most pernickety interpretation. I don't use this site, despite being an ex-pat and a keen SE user, precisely because life is too short to waste time grovelling and begging for permission and re-open votes every time I want to ask an ex-pat question that's not about visas. I'm sure I'm not alone.
Comment: Yes, it's frustrating that so many non-immigration questions attract close votes. See http://meta.expatriates.stackexchange.com/q/42/2037 for more discussion.
Comment: An issue a local wouldn't have is not allowed?  That would eliminate not only immigration questions, but many others.  If I don't ask my immigration questions, I'll never get there to have other questions.  And if I can't ask how to cope with how things work there just because it's how things work there, ....  last but not least is the frustration of Travel telling me it's an Expat question and vice versa.


Scaling to millions of users with a Twitter tool

Question: How do Twitter tools like Buffer scale to millions of users where they have to write thousands of tweets every second?
I have a similar service where I am running a cron script however just going through 50 users takes about 10 seconds - so I'm wondering how this sort of scale is possible.
Comment: There's no one glib answer other than that they pay people like us to do this kind of thing, do that.
Comment: Sure: Find your most severe bottleneck. Eliminate it. Rinse and repeat.
Comment: granted but less then 10 users a second to thousands? There must be some basic tactic.
Answer: Tasks like what Buffer does are easily paralleled - you don't have to wait for one person's posts to be processed to post another user's.  Because of this, it's trivial to scale horizontally - multiple threads and/or servers processing multiple users at a time.
Some basic tricks they might use are:

Multiple database servers - the post content doesn't have to be available to all servers, just the ones that need to process it.  So you could have one main database that holds account and login info, and a bunch of separate ones that queued posts are added to, so the load is balanced out across them.  There is no need to setup a database cluster or syncing between the servers, since the data doesn't need to be available across all servers.
Multiple posting servers - each server would look through one of the databases for things ready to post and deal with them.  Probably with multiple processes/threads, and some way to control which thread is handling which database record, so duplicates don't get posted.
Optimization.  If you are handling thousands of posts a second, shaving 1/100th of a second off the posting time is a big increase in speed.  So much work is likely done to improve the performance of the parts of their code that get called often.  Careful profiling helps determine which parts of the code need to be optimized, and which don't get called frequently enough to bother.

Basically, instead of processing one post from one user at a time like your script likely does, process as many of them in parallel as possible.  Using these techniques you can just add more servers as you grow.  You can also automate cloud server scaling - when you get too much of a backlog, a new server automatically gets started up to handle the extra load.  When things die down, the extra servers get shutdown too.


Curvature in $\mathbb{R}^2$ and curvature in $\mathbb{R}^3$

Question:
For a unit speed curve $\beta(s)=(x(s),y(s))$ in $\mathbb{R}^2$ , the unit tangent is $T=(x',y')$ as usual , but the unit normal $N$ is defined by $N=(-y',x')$ . Prove that if the curvature $k$ does not change sign , then $|k|$ is the same as usual $\mathbb{R}^3$ curvature $\beta(s)=(x,y,0)$ .

The exercise above was in Barrett O'Neill's Elementary Differential Geometry , Page$_{68}$ . For $k$ in $\mathbb{R}^2$ , we find that$$(x'',y'')=k(-y',x')$$So we have $|k|=||(x'',y'')||$ since $\beta$ has unit speed . For $k$ in $R^3$ , we have $k=||\beta''(s)||=||(x'',y'',0)||=||(x'',y'')||$ .
So we have $|k|$ in $\mathbb{R}^2$ equals to $k$ in $\mathbb{R}^3$ , however , I cann't see why we need $k$ does not change sign in $\mathbb{R}^2$ .
Comment: We usually impose nonvanishing of curvature for space curve so normal, binormal and torsion can be defined.  Didn't it say so in the text?
Comment: @user10354138 They do. At the beginning of the chapter, they restrict their attention to strictly positive curvature.
Answer: Let's call the plane curvature ${\widetilde{\kappa}}$ and the standard curvature to be $\kappa.$ If $\widetilde{\kappa}$ does not change sign, that intermediate value theorem guarantees the it is non-zero over the interval that it is defined on. Calculate that $$\kappa=\| T'\|=\|\widetilde{\kappa} N\|=|\widetilde{\kappa}|\|N\|=| \widetilde{\kappa}|,$$ using that $T'=\widetilde{\kappa} N$ (see the first part of the problem) and that $\|N\|=1$.
You'll see that, at the beginning of the chapter, there was an assumption that $\kappa>0$. If $\widetilde{\kappa}$ could change sign, then this would be violated. In particular, we need $\kappa>0$ to define $N=T'/\kappa.$


Why are concert bands the standard in U.S. public schools?

Question: School concert bands are often the first opportunity for formal musical training that american children encounter. They, along with marching bands,  make up the bulk of many music programs across the country, and yet they represent a miniscule portion of its actual music production and consumption.
How did this dominance come about historically, and what are the modern explanations for keeping it this way?
Comment: Are you contrasting this with individual instruction, or other groups like orchestra (with strings) and jazz bands?
Comment: Viewed from the UK, the "school marching band" looks like a very US-specific thing, associated with US field sports. The only "indigenous" marching bands in the UK are in the armed forces and associated with some political organizations and (historically) with trade unions - none of which are relevant to school age players. And If you have a marching band, you already have most of a concert band. For orchestras, I suspect that string players need more learning time than wind and brass, before they reach a standard where they can perform in public.
Comment: .... Note, the UK *brass* band movement is something very different - as the name implies there are only brass instruments, the size and composition is rigidly fixed for competitive purposes, and the size is relatively small - maximum 25 players + percussion. Brass bands do sometimes "march", in the sense of "walking in step with each other from point A to point B while playing", but they don't perform choreographed routines like US marching bands.
Comment: I'm not sure about the premise of your question. A sampling of local school districts in two different states shows that A) the programs are partly guided by "federal standards" and B) they all offer band, orchestra, chorus, and Jazz ensemble. I'm assuming many, if not most or even all schools also have marching bands. What other contexts for music instruction are you expecting to see or thinking would be more appropriate? Even LAUSD, possibly the most maligned school district in the country, offers choir, string orchestra, band, guitar/"modern band", and piano instruction.
Comment: I'm most directly asking about bands as opposed to orchestras. While I realize that most schools have orchestra and band, it seems like band is given special emphasis considering its limited repitiore and popularity.
Answer: I don't think this is restricted to The USA and this form of musical education has many pluses.

The band culture  requires each musician to behave in an cooperative fashion.  While there are rules-and-regulations, the  fundamental structure is based on the individual self-discipline contributed to achieve the group's goals.
School bands can allow musicians with differing skills to play together.
Probably the most important being cost. Individual tuition is expensive. In the UK schools may have individual tuition, usually on Saturday morning, but there is normally a charge for this.
Comment: Isn't there overlap between concert band and marching band? Marching band is **huge** at many universities in the US and students can get significant band scholarships, so at least high school marching band has value in prepping students for that. I also wonder if the literature for concert band is educational, accessible to beginners and intermediate students alike, and widely culturally acceptable, unlike other options like symphony orchestra or punk rock.
Answer: Compared with what?
Orchestras are relatively rarer, primarily because fewer kids are encouraged to take up string instruments than wind instruments -- and because a school band can handle a near-arbitrary number players in each section, while an orchestra will only use a few winds and brass players.
Nearly every high school I know of (Eastern USA) has a jazz band of one sort or another as well, so it's not strictly "concert band or nothing."
As for "why is it still this way,"  well, just try changing any tradition in a school system (double sad face)
Comment: Are orchestras actually rarer? I can't find any support for that.
Comment: @ToddWilcox I have to admit this is a highly parochial study -- high schools in my general environs all have bands but quite a few do not have an orchestra.  I don't have a statewide or countrywide study at hand.
Comment: Hmm.. I wonder if it's a case of **districts/counties** having orchestra and other non-band programs available but not all **schools** have enough interest in those programs in their student body. If so, that would change the nature of the question to being, "why are so few students interested in music programs besides concert band?"
Comment: Where I grew up, orchestras and string quartets were the norm, with small brass groups also in existence.


Why was USA disqualified from the 4x100 men's relay final in the Rio Olympics?

Question: After the mens 4x100 relay final races in Rio on August the 19th the commentators said they exchanged the baton outside the box.
What does that mean?
Answer: 4x100m relay are composed by 4 runners that run approximately 100m each. To avoid that a faster athlete run a longer section are imposed limits for the passage of the baton.
In the image below you can see those takeover zones (red boxes) on a lap

Yesterday (August 18th 2016) Gatlin has touched his mate out of those takeover zone
From IAAF rule book rule 170.7

The baton shall be passed within the takeover zone. The passing of
  the baton commences when it is first touched by the receiving
  athlete and is completed the moment it is in the hand of only the
  receiving athlete. In relation to the takeover zone, it is only the
  position of the baton which is decisive. Passing of the baton outside
  the takeover zone shall result in disqualification
Comment: The key, as you've cited, is "The passing of the baton commences when it is first touched by the receiving athlete..." because the touch, in this case, was accidental but illegal, like a basketball player inadvertently stepping out of bounds.
Answer: To add to Ale's thorough answer: in the case of any disqualification in a major athletics event, the official results will usually cite a specific rule in the IAAF competition rules (which Ale linked). If you look up that rule you see what violation they're being disqualified for. In this case it wasn't immediately obvious which exchange was the problem from the results, but often the rule cited in the results gives you the clue you need to explain the disqualification.
Answer: They have a set margin of distance called the exchange zone to hand off the baton. The USA team handed the baton off before the exchange zone started therefore being disqualified.
Comment: Well, sort of. They didn't complete their pass outsize the zone; however, according to the rules, they *began* the pass (see my comment on Ale's answer) due to an accidental touch outside the zone.


Does a small perturbation push the minimizing point of continuous function too far?

Question: Let $(M,\,d)$ be a compact metric space and $f,\,g: M \rightarrow \mathbb{R}$ two continuous functions.
For $|ε|<1$ we define a perturbation of $g$: $$g_ε(x) = g(x) - εf(x)$$
For every $ε>0$, there exist some $x_ε\in M$ such that $\displaystyle g_ε(x_ε) = \min_{x\in M}g_ε(x)$, since $g_ε$ is continuous and $M$ is compact.
The same argument applies for $g$, too, but suppose that the according minimizer $x_0$ is unique.
Can we argue that for small $ε$ we can find a $x_ε$ $δ$-close to $x_0$ for $δ\xrightarrow{ε\rightarrow 0} 0$?
How can we get around it?
Every help is appreciated! Thank you in advance!
Answer: It is straightforward to show that any accumulation point taken from $\operatorname{argmin}  g_{1 \over n}$ will be an element of $\operatorname{argmin}  g$:
Suppose $g_{1 \over n}( x_{1 \over n}) \le g_{1 \over n}(x)$ for all $n,$ and $x \in M$. Suppose $x^*$ is an accumulation point of $x_{1 \over n}$, then taking limits along an appropriate subsequence gives: $g(x^*) \le g(x)$ for all $x$.
Hence if $\operatorname{argmin}  g$ is a singleton, any sequence taken from $\operatorname{argmin}  g_{1 \over n}$ must converge to this point. To see this note that any subsequence of $x_{1 \over n}$ must contain a further subsequence that converges to $\operatorname{argmin}  g$. Hence $x_{1 \over n}$ converges.
Comment: @SotirisSimos: Let me know it the above edits are sufficient.
Comment: There is an analysis result that says that $x_n \to x^*$ **iff** every subsequence contains a further subsequence that converges to $x^*$. It is a surprisingly useful result.
Comment: Thank you for your answer! Can you elaborate a little more?
Comment: sorry, I fell asleep, I'm not really sure I understand the second argument. You're saying that there is a subsequence of $x_{1/n}$ that converges to some $x^* \in M$ and for that point we have $g(x^*) \leq g(x)$ due to uniform convergence of $g_{1/n}$ to $g$, so that $x^* \in \operatorname{argmin} g$. But I can't follow the rest of the argument!
Comment: Thank you once again! You were very insightful! Let me ask you something, though! You did prove something more, right? I mean, in my case, we have that every sequence has a subsequence which converges to some point which turns out to be the unique minimizing point of $g$. So, we can choose a point from the subsequence as close to it as we want. Am I right? It's not that I don't appreciate that you proved something more, on the contrary! I'm just clarifying things!
Comment: If $g$ has a unique minimiser $x^*$ then any sequence of $g_\epsilon$ minimisers will converge to $x^*$. If $g$ does not have a unique minimiser, then you can find a subsequence of the $g_\epsilon$ minimisers that converges to one of the $g$ minimisers.
Comment: Another useful result, for sure! Unfortunately, it does not answer my question! What I want to learn is if it's enough to show that for some converging subsequence of minimizers, its limit equals the unique minimizer of $g$, which is your first argument, since then we can take some point from the subsequence (which is a point from our original sequence) to be close to the minimizer of $g$.


FTPLIB error socket.gaierror: [Errno 8] nodename nor servname provided, or not known

Question: I'm trying to download a file from an FTP server using FTPlib but i keep getting the error below. I tried several methods found on SO namely editing the etc/host file and adding the computer name after the local host IP but that doesnt work either.
Any help is much appreciated. (running a mac, python 3.7, macOS Mojave 10.14.2)
Code im trying to run:
from ftplib import FTP

ftp = FTP('myftpurl')

Error:
Traceback (most recent call last):

File "", line 1, in
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ftplib.py", line 117, in init
    self.connect(host)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ftplib.py", line 152, in connect
    source_address=self.source_address)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 707, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
Answer: ftp = FTP('myftpurl')

It is not clear what myftpurl actually contains in your specific code.
But given its name I assume that you've tried something like ftp://example.com. Only, as clearly documented FTP(...) does not expect a URL but instead a hostname or IP, i.e. FTP('example.com') and not FTP('ftp://example.com').
Given a URL like you probably do will result in treating that URL as hostname, i.e. doing a lookup for the hostname ftp://example.com. Since such a host does not exist you get an error.


Prove that $1+20+20^2+20^3+....+20^{21} \equiv 0\bmod 23$

Question: I am in search of a correct and shortcut techniques to prove this.
Otherwise I have calculated each remainder according to the power of $20$ to prove this :-
$1 \equiv 1 \bmod 23$
$\Rightarrow 20 \equiv -3\bmod 23$
$\Rightarrow 20^2 \equiv (-3)^2 \equiv 9\bmod 23$
$20^{2n}\equiv (-3)^{2n}\bmod 23$
$20^{2n-1}\equiv (-3)^{2n-1}\bmod 23$.
Therefore
$1+20+20^2+20^3+....+20^{21} \equiv 0\bmod 23$
For $\sum_{n=1}^{11} 20^{2n-1} \equiv r \bmod 23$
we have  $$-3-4+10-2+5-1-9+11+7-6-8 \equiv 0 \bmod 23$$
For $\sum_{n=0}^{10} 20^{2n} \equiv s \bmod 23$
we have   $$1+9+12+16+6+8+3+4+13+2-5 \equiv 0\bmod 23$$
Therefore $$\sum_{n=1}^{11} 20^{2n-1}+\sum_{n=0}^{10} 20^{2n} \equiv r+s \bmod 23$$
$$\Rightarrow 1+20+20^2+20^3+....+20^{21} \equiv 0+0 \equiv 0\bmod 23$$
If possible just show any short-cut correct way to prove this. Any help is appreciated.
Comment: If you still want parentheses around the "mod 23" parts, try `\pmod{23}`.
Answer: HINT
Note that by geometric series
$$1+20+20^2+20^3+....+20^{21} \equiv \sum_{k=0}^{21}(-3)^k=\frac{1- (-3)^{22} }{4} \pmod{23}$$
then use FLT for $(-3)^{22} \pmod{23}$.
Answer: Bracket expansion and Fermat's little theorem gives
$$
(1-20)(1+20+\cdots + 20^{21}) = 1-20^{22}\equiv 1-1 = 0\pmod{23}
$$
And since $-19$ multiplied by your sum is congruent to $0$ modulo $23$, and $-19$ is coprime to $23$, that means the sum itself must also be congruent to $0$.
Answer: $\sum_{i=0}^{21}{20}^i=\frac{1-{20}^{22}}{1-20}$...  But ${20}^{22}\cong 1\pmod{23}$ by Fermat's little theorem...
Answer: Sum is equal to $\frac{20^{22}-1}{20-1}=\frac{20^{22}-1}{19}$,
but $20^{22}\cong 1 \mod 19,23$.


Python: Format string to appear as plaintext in Markdown or HTML?

Question: I am using a Telegram bot to send messages from my Python program. Telegram requires all bot messages you send to be formatted either in Markdown or HTML.
All I want is the strings in my Python program to appear exactly the same way on the receiving end of a Telegram message.
Problem is, the text I'm trying to send is drawn from the public, so it could be anything, including special characters that have meanings in those formats, which screws up the message completely.
Is there a way I can format this message string into one of those formats so it will appear just as plain text on the other end?
edit: I've tried a bunch of things. As mmiron suggested, I tried escaping my string into html, which I have not been able to get to work. What seems to happen is that special (<,>,#) characters completely screw up the message EVEN IF I replace them with character references like &
I also tried escaping my string into Markdown, which had this very strange result. Unlike HTML using markdown seems more likely to actually send the text, but special characters (especially #) seem to screw up the result
Here is the starting text with Markdown
>>Bravo: Priyanka Chopra, Navya Naveli Nanda praise Jharkhand girl who got #Harvard University scholarship https://url
"Educate a girl you can change the whole community"
- - - - - - - - - - - - - - - - - - - - - -
Sunchartist
(@sunchartist)
                                                     j1.1
- - - - - - - - - - - - - - - - - - - - - -
9:09PM +43seconds    23-4-2021
[Chopra]
(Balance: $3.43)
-----------------------------------
<https://twitter.com/sunchartist/status/1385777945248030723>
-----------------------------------
<https://www.url.com>

After escaped into markdown and sent, here is telegram message I receive:
'\>Bravo: Priyanka Chopra, Navya Naveli Nanda praise Jharkhand girl who got Harvard University scholarship https://url
"Educate a girl you can change the whole community"
\- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \-
Sunchartist
\(@sunchartist\)
                                                     j1\.1
\- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \- \-
9:09PM \ 43seconds    23\-4\-2021
[Chopra\]
\(Balance: $3\.43\)
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
<https://twitter\.com/sunchartist/status/1385777945248030723\>
\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
<https://www\.url\.com\>

Obviously what's happening is the escape function is adding a backslash before every special character. What's strange is it doesn't seem to be parsing that correctly as markdown EXCEPT in the case of the first bracket in
[Chopra\]

and the first < that appeared in the original text before the links
<https://www\.url\.com\>

Which is strange because for some reason it only seems to be working for the opening character in [] or <> but not the closing.
I've also tried wrapping with
...
 using the HTML flag but that doesn't seem to achieve anything except change the text color on telegram IF it sends, which it will only do if I remove the offending special characters
If at any point in the text there is a hashtag (#) it seems to send only the text leading up to the hashtag, but none after it. This is true for both markdown and html for some reason
Comment: If you don't need any other styling like bold or links,  you could just send the text as code, so surrounded by 3 backtick.
Answer: import html
escaped = html.escape(EXTERNAL_INPUT_STRING)

Then send escaped as HTML.  Also see https://core.telegram.org/bots/api#sendmessage
Comment: For some reason when I try this I'm getting. {'ok': False, 'error_code': 400, 'description': 'Bad Request: message text is empty'} when I do this. Despite the fact that the 'escaped' variable is a long string I can print out
Comment: `The API currently supports only the following named HTML entities: <, >, & and ".`  Try wrapping your message in ` ... ` tags instead.
Comment: Seem to be getting closer as playing around with  actually let me send and receive a message, which I wasn't able to do before. However, this hasn't solved it. I can only get it to send if I remove the offending characters like >,< and #. If I do it exactly as you describe the response is {'ok': False, 'error_code': 400, 'description': "Bad Request: can't parse entities: Can't find end tag corresponding to start tag pre"} It at least seems to acknowledge the first , but not the second for some reason
Comment: You are including the `/` in the close tag, right?  The open is ``, and the close is ``.
Comment: Yes I am: new_text="".join(['',original_text,''])
Answer: Ok I finally solved it. Posting here for future reference.
I could never get HTML to work so I'm still unsure about that.
So first, I had my parse_mode=Markdown Instead it needs to be parse_mode=MarkdownV2
Next, there are a few specific characters that using the \ operator does work to display as a literal. Instead, you need to use Percent-Encoding to retain those symbols.
Here is the code I used to fix that portion.
message_body=message_body.replace('%', '\\%25')
message_body=message_body.replace('#', '\\%23')
message_body=message_body.replace('+', '\\%2B')
message_body=message_body.replace('*', '\\%2A')
message_body=message_body.replace('&', '\\%26')

Which fixes for %, #, +... I could probably make this more elegant/faster but this works for now.
Finally, there's a group of characters that the \ operator DOES work to create a literal for. Here's the code I used to fix those
    message_body = re.sub(r"([_*\[\]()~`>\#\+\-=|\.!{}])", r"\\\1", message_body)

Answer: To add a specific for Telegram escaping you can just use PlainText from telegram-text:
from telegram_text import PlainText

element = PlainText("Your non-escaped text!")
escaped_text = element.to_markdown()
escaped_text
'Your non\\-escaped text\\!'


Não consigo adicionar inteiros em um arquivo java

Question: Não estou conseguindo adicionar inteiros em um arquivo .txt em Java. Não consigo entender o que estou fazendo de errado. Como por exemplo adicionar esse número 10 do codigo abaixo. Gostaria de saber, como faço para adicionar o inteiro?
public static void salvar(){
    File arq = new File("C:\\Users\\giova\\Documents\\arq.txt");

    try{
        if (!arq.exists()) {
            arq.createNewFile();
        }
    }catch (IOException ex) {
        Logger.getLogger(main.class.getName()).log(Level.SEVERE, null, ex);
    }

    FileWriter fw;
    try {
        fw = new FileWriter(arq, true);
        BufferedWriter bw = new BufferedWriter(fw);
        bw.write(10);
        bw.close();
        fw.close();
    } catch (IOException ex) {
        Logger.getLogger(main.class.getName()).log(Level.SEVERE, null, ex);
    }
}

Comment: Por favor, esclareça seu problema ou forneça detalhes adicionais a fim de destacar exatamente o que você precisa. Da forma como está escrito atualmente é difícil dizer com exatidão o que você está perguntando.
Answer: Primeiramente, FileWriter já cria o arquivo caso ele não exista, então não precisa verificar isso antes. Aliás, verificar se o arquivo existe para só depois criá-lo pode cair em uma condição de corrida (ver mais aqui). Em vez disso, simplesmente tente criar o FileWriter e trate as exceções que ele lança caso o arquivo não possa ser criado.
E a partir do Java 7 você pode usar o try-with-resoruces, que garante que o arquivo é fechado ao final (mesmo em caso de erro).
E quanto ao write, veja na documentação que o número que você passa é o código numérico equivalente ao code point do caractere. No caso, valores menores que 128 correspondem à tabela ASCII, e o número 10 equivale ao caractere NEW LINE (ou LINE FEED), também conhecido como a "quebra de linha".
Se você quer escrever o número 10, terá que escrever os caracteres 1 e 0 explicitamente, como char ou então String:
// try-with-resources
try (BufferedWriter bw = new BufferedWriter(new FileWriter("nomeArquivo.txt", true))) {
    bw.write(Integer.toString(10));
    // ou ainda
    // bw.write("10");
} catch (IOException e) {
    // tratar erros (coloque aqui o seu código de tratamento de erro)
}
// não precisa chamar close(), o bloco try-with-resources já faz isso automaticamente

Assim, ele escreve os caracteres 1 e 0, em vez do NEW LINE.
E no código acima, ele cai no catch caso o arquivo não possa ser criado (por exemplo, por falta de permissão ou disco cheio), e também em caso de erros na escrita. E o bloco try-with-resources garante que ele é fechado ao final, sem precisar chamar close explicitamente.

Outra alternativa é usar um PrintWriter, pois esta classe sim possui um método que aceita um inteiro diretamente:
try (PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter("nomeArquivo.txt", true)))) {
    pw.print(10); // escreve o número 10 no arquivo
} catch (IOException e) {
    // tratar erros
}

E você também pode usar as classes do pacote java.nio, como java.nio.file.Files e java.nio.file.Paths. Mas neste caso ele também não aceita o número 10 diretamente:
// não esqueça de colocar em um bloco try-catch
Files.write(Paths.get("nomeArquivo.txt"), "10".getBytes(), StandardOpenOption.APPEND);


How do I convert this SQL inner join query into LINQ syntax?

Question: I'm not that familiar with LINQ. I need this query converted into a LINQ statement for use inside my C# project.
Thanks
SELECT Galleries.GalleryTitle, Media.*
FROM Galleries
INNER JOIN Media ON Galleries.GalleryID = Media.GalleryID
WHERE (Galleries.GalleryID = 100)

Answer: var query = from g in db.Galleries
            join m in db.Media on g.GalleryID equals m.GalleryID into gm
            where g.GalleryID == 100
            select new { g.GalleryTitle, Media = gm };

Property Media will contain list of joined media entities. Also if you have navigation property defined in Gallery entity, then:
var gallery = db.Galleries.Include("Media")
                .FirstOrDefault(g => g.GalleryID == 100);


Why is dns.lookup working on these incorrect URLs?

Question: I'm currently trying to create a URL shortening microservice with Node.js. I'm passing in incorrect strings to dns.lookup() to check if they are valid URL's, however instead of returning an error, they are returning an address and a family.
dns.lookup('hello', (err, address, family) => {
                if(err) {
                    res.send('invalid')
                    return
                } else {
                    console.log(`error = ${err}`)
                    console.log(`address = ${address}`)
                    console.log(`family = ${family}`)
                    next()
                }
            });

Console:
error = null
address = 220.127.116.11
family = 4

I'd like it to pass an error if the URL is invalid.
EDIT: Adding full code if that helps?
var express = require('express')
var app = express()
var cors = require('cors')
var dns = require('dns')

            dns.lookup('hello', (err, address, family) => {
                if(err) {
                    console.log(`error = ${err}`)
                    //res.send('invalid')
                    //return
                } else {
                    console.log(`error = ${err}`)
                    console.log(`address = ${address}`)
                    console.log(`family = ${family}`)
                    //next()
                }
            });

var listener = app.listen(3001, function(){
    console.log('working on port 3001')
});

Comment: Seeing the function invocation is nice and all, but how about showing the function definition?
Comment: i am getting Error: getaddrinfo ENOTFOUND hello
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:60:26) {
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'hello'
}
Comment: Thanks Ashish, I was getting your response a while ago, but it's now returning the ip above and I don't know why. The ip is for something called 'Barefruit Ltd.' in London.

https://whatismyipaddress.com/ip/188.8.131.52

So weird!
Comment: JDunken, not sure what you mean by this? I've put the rest of my code in an edit above.
Answer: It turns out my ISP (Virgin Media) has a third party called Barefruit Ltd that intercepts dns errors and sends back advertising instead: https://www.barefruit.com/
For the meantime, I've found a workaround by adding if(err || address == '126.96.36.199') to my error handling.


Quando uso o método __construct não preciso usar o get; set?

Question: Tenho o método __construct na classe, devo criar os métodos set e get?
    <?php
class Produto{
    //atributos
    private $descricao;
    private $preco;

    //método construtor
    public function __construct($descricao, $preco){
        $this->descricao = $descricao;
        $this->preco = $preco;
    }

    //métodos

    public function getDetalhes(){
        return "O produto {$this->getDescricao()} custa {$this->preco} reais";
    }

    /*
    public function setDescricao($valor){
        $this->descricao = $valor;
    }

    public function setPreco($valor){
        $this->preco = $valor;
    }
    */
    public function getDescricao(){
        return $this->descricao;
    }

    public function getPreco(){
        return $this->preco;
    }

}

$p1 = new Produto('livro', 50);
//$p1->setDescricao('livro');
//$p1->setPreco(50);

//var_dump($p1);
echo $p1->getDetalhes();

?>

Até cheguei a esta outra dúvida mas não ficou claro para mim
Answer: Não temos como saber. E essa é a única resposta correta para essa pergunta específica.
Uma coisa que as pessoas não entendem sobre orientação a objetos é que não é uma maneira de aplicar receitas de bolo e tudo ficar lindo e funcionará maravilhosamente. Não são os mecanismos de OOP que fazem o código ficar bom, como aplicá-los é que faz o código ser bom ou ruim.
Não tem como dizer se algo é o mais adequado ou não sem conhecer o contexto, sem estar se referenciando a um caso concreto. Por isso que sempre falo que esse negócio de "melhores práticas" não funciona. Se fosse uma lista de checagem sempre considerando opções seria uma boa, mas na forma como costumam ser expostas e propagadas elas costumam causar mais mal do que bem. Elas não consideram contexto e, em geral, "empurram" uma ideia que se quer "vender". E não se engane, estão sempre querendo vender algo, eu estou fazendo isto agora. Só você pode discernir o que é melhor e saber quando aplicar o que em cada caso.
A pergunta não tem requisitos, não diz o que precisa fazer, então tanto faz o que vai fazer. É o que eu falo, antes de saber a resposta é preciso saber a pergunta. Quando a pergunta é errada (parte de premissa errada), incompleta (não tem todos os dados necessários), ou é irrelevante (aquilo não importa), a resposta nunca será adequada.
Em exemplos abstratos pode-se falar do mecanismo, mas não pode-se falar da adequação dele. E a pergunta é sobre a adequação e não sobre o mecanismo.
Eu não sei se precisa do construtor, não sei se ele é suficiente. Não sei se deve ter como acessar ou permitir modificar cada um dos membros isoladamente. E se puder, não sei se não deveria ter algo mais nele. Não sei se um campo público é suficiente, em geral no PHP é. Não sei se o correto deveria usar o __get() e __set() padrão da linguagem. Não sei se deveria ser uma classe ou um array associativo, quem sabe com algumas funções soltas, é suficiente.
Se o código fosse esse, eu iria pelo caminho mais simples que é um array associativo com uma função que entregue os dados mais formatados. Se tivesse um contexto aí eu poderia dizer como eu faria. Mas já adianto que em PHP eu ainda faria de forma muito simples, não criaria coisas desnecessárias. Não adoto o cargo cult programming.
Coloquei no GitHub para referência futura.
Comment: Acho que entendi o que quis dizer, complexo então.. Basicamente devo aprender OOP porém nem tudo que diz é o que é?
Comment: É complexo mesmo.
Comment: Na minha opinião o __construct e alguns outros métodos nativos OOP só servem par complicar, já que vc pode optar por métodos mais simples e práticos.
Comment: E pode optar por não fazer OOP que também complica. Mas se for fazer OOP, eles são bem úteis.
Answer: Se você quiser acessar os atributos $descricao e $preco, sim. Se eles fossem públicos, você poderia ler e modificar eles direitamente usando
$p1->preco = 10;

mas como na sua classe os atributos são privados, você tem que usar os métodos get e set para poder ler e modificar eles, como você fez nos comentários.
É bom usar métodos get e set para controlar a visibilidade dos seus atributos e os valores que eles podem ter. Por exemplo, pode fazer testes como esse antes de atribuir o valor no atributo:
public function setPreco($valor){
    // Não podemos ter um preço negativo
    if($valor >= 0)
        $this->preco = $valor;
}


while extending any class, which class (child or parent) size get larger logically

Question: i m in a confusion that while extending any class, which class (child or parent) size get larger logically.
thanks in advance
Answer: The parent class itself stays the same size.
The size of an instance of the child class is greater than or equal to the size of an instance of the parent class.
Answer: I think you mean the object (the instance of that class)
an object that extends a class will have memory allocated for the subclass like normal, as well as any methods/variables used in the superclass.
as a rule of thumb, you can just think of a subclass object containing all of the memory from it and all superclasses.
Answer: The child would have a larger memory footprint.  The parent doesn't know about the child.
Answer: The parent class pretty well has to stay the same size.  Otherwise, you'd change the results of a routine using the parent class just by defining a child class, and that would be bad.  For one thing, it would violate the Principle of Least Astonishment, and also it would introduce very undesirable coupling.
The child class has to have everything from the parent class and presumably some extras (or it wouldn't be a child).  However, while it will normally be larger than the parent, it doesn't actually get larger.  Its size is normally determined when it is defined.  (At least for all languages I'm familiar with.  If this question is truly language-agnostic, well, there's lots of object-oriented systems out there.)
Comment: hi david,
thnx for replying,
then why cant we make object of super class object as child contains all of the code in it but the reverse way round is possible.


3-way interaction Stata

Question: I have a panel data in the following form:
<id> <city> <treated> <time> <after>

where id identifies the individuals in my panel, city is the location where the individual live (non-time varying), treated is a dummy indicating those individual that are eventually treated (0: non-treated, 1: treated), time is a year-month variable, and after is a dummy (0: before, 1: after) indicating the period in which the treated unit are under treatment.
With this data, I am running a simple difference-in-differences model. I am including individual fixed effects and year-month fixed effects. To relax the parallel assumption, I include a treatment-city specific time trend -- people often include treatment specific trends, but in my settings the outcome can vary a lot across cities for treatment and control units -- so my specification should be:
xtset id time
xtreg depvar i.treated i.after c.treated#c.after i.time i.city#c.treated#c.time, fe cluster(id)

where
i.city#c.treated#c.time

is the treatment-city specific linear trend.
Generally, in this kind of models I use to include only a treatment-specific trend, and not a 3-way interaction. The first question is whether this approach makes sense.
Second, I thought that adding:
 i.city#c.treated#c.time

or
 i.city#i.treated#c.time

was exactly the same (note that the variable treated in coded as 0/1), but apparently is not. Can someone explain me the statistical difference between including one term or the other in my regression?
Morever, some tests I did suggest that adding
 i.treated#c.time

or
 c.treated#c.time

is the same. Why with a 2-way interaction, treating the variable treated as continuous or as a factor does not matter?
P.S. Thanks to Dimitriy for the current answer!
Comment: I think there's a statistical question lurking in there about whether treatment-specific trends are appropriate, so it's not purely about software.
Comment: Hi All, why my question is still on hold? I edited and it seemed that the edits where approved. Thanks!
Comment: I cleaned up your question a bit. You omitted the command name, which I filled in as `xtreg`. Please edit with the correct one if that is not the case.
Comment: Hi @Davide, there is a difference between the "edit approval" process and the "reopening" process. I approved your edit (the problem was an account problem and you should have been able to edit your own post automatically), but didn't take part in the process to reopen/leave closed. So hopefully someone who did can explain. It's likely they felt that the edits didn't sufficiently clarify the extent to which this question is *statistical* (on-topic) rather than merely about *programming* (off-topic).
Answer: This is edited a bit in response to revisions.
It is strange to me that treated and after are dummies, yet you are treating them as continuous variables by using the c. prefix. I would have used the i. prefix. I will assume that is what you had intended below.
This will not matter in simple models, but once you add interactions, Stata might choose a different city as the base in your specification (for reasons which elude me). This means the parameters will be different for that reason alone. Fixing the base with something like ib4.city and ib0.time will remedy this. I will add an example below.$^*$
Also, I might be inclined to cluster at the city level with this setup.
Also, note that i.treated#c.time and c.treated#c.time are not equivalent. Here's example with the cars data:
sysuse auto, clear
reg price i.foreign#c.mpg
reg price c.foreign#c.mpg

The first spec allows for two separate effects for mpg on price, one for domestic cars and one for foreign. The second allows for mpg to alter price for foreign cars only.
Now to answer your main question. As a way to relax the parallel trends assumption, people will often include time dummies and a parametric time trend (linear or quadratic) for the treated only in the estimating specification. You are going further and making the time trend city-specific in your first command. There's a nice SJ paper by Mora and Reggio, where they discuss this approach (take a look at equations (3) and (4)). If I understand their notation, that corresponds to i.city#c.treated#c.t or i.city#1.treated#c.t with i.t. They also have a WP with a section on common types of specifications from the literature. All the linear/polynomial trends are for the treatment group only, not both.
Your goal can be accomplished by i.city#c.treated#c.t since treating a binary variable as continous is equivalent to putting in a dummy. When you use i.city#i.treated#c.t, that is equivalent to putting in two trends for each city: one for treated and one for control observations. I don't think this is something you want.
Personally, I think the clearest way to achieve the former goal is with i.city#1.treated#c.t.
Here's an example with the Card & Krueger unemployment data, where I am using the fast food chain instead of city (Wendy's is the base category). This data only has two periods, so there's no separate time dummy. It's a bit confusing since X.chain#c.treated#c.t coefficient is the same parameter as X.chain#1.treated#c.t in the table, but it has different labels:
. set more off

. estimates clear

. use http://fmwww.bc.edu/repec/bocode/c/CardKrueger1994.dta, clear
(Dataset from Card&Krueger (1994))

. drop if id==407
(4 observations deleted)

. xtset id t
       panel variable:  id (strongly balanced)
        time variable:  t, 0 to 1
                delta:  1 unit

. gen chain = ""
(816 missing values generated)

. foreach var of varlist  bk kfc roys wendys {
  2. replace chain="`var'" if `var'==1
  3. }
variable chain was str1 now str2
(342 real changes made)
variable chain was str2 now str3
(158 real changes made)
variable chain was str3 now str4
(198 real changes made)
variable chain was str4 now str6
(118 real changes made)

. sencode chain, replace

. eststo: qui xtreg fte i.treated##i.t i.chain#c.treated#c.t, fe cluster(id)
(est1 stored)

. eststo: qui xtreg fte i.treated##i.t i.chain#1.treated#c.t, fe cluster(id)
(est2 stored)

. eststo: qui xtreg fte i.treated##i.t i.chain#i.treated#c.t, fe cluster(id)
(est3 stored)

. esttab *, noomitted drop(0.treated 0.t 0.treated#0.t) varwidth(25)

-------------------------------------------------------------------------
                                   (1)             (2)             (3)
                                   fte             fte             fte
-------------------------------------------------------------------------
1.t                             -2.523*         -2.523*         -2.577
                               (-2.02)         (-2.02)         (-1.04)

1.treated#1.t                    3.517           3.517           3.571
                                (1.63)          (1.63)          (1.17)

1.chain#c.treated#c.t            0.268
                                (0.14)

2.chain#c.treated#c.t           -0.225
                               (-0.12)

3.chain#c.treated#c.t           -2.439
                               (-1.28)

1.chain#1.treated#c.t                            0.268           0.268
                                                (0.14)          (0.14)

2.chain#1.treated#c.t                           -0.225          -0.225
                                               (-0.12)         (-0.12)

3.chain#1.treated#c.t                           -2.439          -2.439
                                               (-1.28)         (-1.28)

1.chain#0.treated#c.t                                           -0.791
                                                               (-0.23)

2.chain#0.treated#c.t                                            4.804
                                                                (1.87)

3.chain#0.treated#c.t                                           -1.291
                                                               (-0.41)

_cons                            17.69***        17.69***        17.69***
                               (79.93)         (79.93)         (79.99)
-------------------------------------------------------------------------
N                                  797             797             797
-------------------------------------------------------------------------
t statistics in parentheses
* p<0.05, ** p<0.01, *** p<0.001

$^*$Fixing The Base:
estimates clear
eststo: xtreg fte i.treated##i.t ib4.chain#c.treated#c.t, fe cluster(id)
eststo: xtreg fte c.treated##c.t ib4.chain#c.treated#c.t, fe cluster(id)
eststo: xtreg fte c.treated##c.t i.chain#c.treated#c.t, fe cluster(id)
esttab *, noomitted drop(0.treated 0.t 0.treated#0.t) varwidth(30)

Comment: Thanks Dimitriy, your answer was really helpful. I edited the question because I still have a couple of concerns about how to proceed with the identification strategy.
Comment: @ Dimitris; I understand that with i.treated#c.time I have two effects, but the coefficients for after, treated, and their interaction are exactly the same independently of what I use, i.treated#c.time and c.treated#c.t. However, this is not case for the  3-way interaction, and I don't understand why.
Comment: You can try with the Card&Krueger data, you'll see that 1.t, 1.treated#1.t and the constant are exactly the same in both cases.
Comment: @David You are not being clear about the exact spec you have in mind here. As I explained above, with the 3-way interaction, this happens because when you write it one way, the included chain effects are KFC, Roy Rogers, and Wendy's (with Burger King as the omitted base) and the other has BK, KFC, and Roy Rogers (with Wendy's as the base). This changes the parameter being estimated. If you fix the base to Wendy's (as I did in the footnote code snippet) or by using `fvset` (even better), you will get the same answer. The simpler 2-way interaction spec doesn't have this problem.
Comment: Thanks Dimitriy and sorry about being not very clear. I missed that part -- so it is just Stata setting different levels. I actually reached the same conclusion yesterday night playing with a small example.
Comment: @Davide I added a WP that you might find interesting.

3-way interaction Stata


How call web service in GpioPinListenerDigital (Pi4J / RaspberryPi)?

Question: I'd like to call a webservice when my PIR sensor is active.
I created the web service call with wsimport:
wsimport -keep -s src http://webservicex.net/geoipservice.asmx?WSDL
I wrote a test application to my Raspberry Pi with pi4j (http://pi4j.com/) project library. In this code call the service on the start, and call again when PIR sensor gettig active.
The first calling is right, but when i try to call on handleGpioPinDigitalStateChangeEvent method of GpioPinListenerDigital object i've got a NullPointerException.
package me.losadrian.Pirel;

import com.pi4j.io.gpio.GpioController;
import com.pi4j.io.gpio.GpioFactory;
import com.pi4j.io.gpio.GpioPinDigitalInput;
import com.pi4j.io.gpio.GpioPinDigitalOutput;
import com.pi4j.io.gpio.PinPullResistance;
import com.pi4j.io.gpio.PinState;
import com.pi4j.io.gpio.RaspiPin;
import com.pi4j.io.gpio.event.GpioPinDigitalStateChangeEvent;
import com.pi4j.io.gpio.event.GpioPinListenerDigital;

import net.webservicex.GeoIP;
import net.webservicex.GeoIPService;
import net.webservicex.GeoIPServiceSoap;

public class App {
    public static void main(String[] args) {

        GeoIPService service = new GeoIPService();
        GeoIPServiceSoap client = service.getGeoIPServiceSoap();
        GeoIP geoIp = client.getGeoIP("18.104.22.168");
        System.out.println(geoIp.getCountryName());

        final GpioController gpioPIR = GpioFactory.getInstance();
        final GpioPinDigitalInput sensorPir = gpioPIR.provisionDigitalInputPin(RaspiPin.GPIO_07, PinPullResistance.PULL_DOWN);

        final GpioController gpioREL1 = GpioFactory.getInstance();
        final GpioPinDigitalOutput rel1 = gpioREL1.provisionDigitalOutputPin(RaspiPin.GPIO_08, "REL", PinState.HIGH);
        final GpioController gpioREL2 = GpioFactory.getInstance();
        final GpioPinDigitalOutput rel2 = gpioREL2.provisionDigitalOutputPin(RaspiPin.GPIO_09, "REL", PinState.HIGH);

        rel1.high();
        rel2.high();

        sensorPir.addListener(new GpioPinListenerDigital() {

            public void handleGpioPinDigitalStateChangeEvent(GpioPinDigitalStateChangeEvent event) {
                if (event.getState().isHigh()) {
                    System.out.println("Motion!!");
                    GeoIPService service = new GeoIPService();
                    GeoIPServiceSoap client = service.getGeoIPServiceSoap();
                    GeoIP geoIp = client.getGeoIP("184.108.40.206");
                    System.out.println(geoIp.getCountryName());
                    rel1.low();
                    rel2.high();
                }
                if (event.getState().isLow()) {
                    System.out.println("Quiet...");
                    rel1.high();
                    rel2.low();
                }
            }
        });

        try {
            for (;;) {
                Thread.sleep(3000);
            }
        } catch (final Exception e) {
            System.out.println(e.getMessage());
        }

    }
}

The console output is:
pi@raspberrypi ~ $ sudo java -jar Pirel.jar
United States
Motion!!
Exception in thread "pi4j-gpio-event-executor-0" java.lang.NullPointerException
    at com.sun.xml.internal.ws.client.ClientContainer$1.getResource(ClientContainer.java:45)
    at com.sun.xml.internal.ws.assembler.MetroConfigLoader.locateResource(MetroConfigLoader.java:220)
    at com.sun.xml.internal.ws.assembler.MetroConfigLoader.locateResource(MetroConfigLoader.java:230)
    at com.sun.xml.internal.ws.assembler.MetroConfigLoader.init(MetroConfigLoader.java:125)
    at com.sun.xml.internal.ws.assembler.MetroConfigLoader.<init>(MetroConfigLoader.java:104)
    at com.sun.xml.internal.ws.assembler.TubelineAssemblyController.getTubeCreators(TubelineAssemblyController.java:78)
    at com.sun.xml.internal.ws.assembler.MetroTubelineAssembler.createClient(MetroTubelineAssembler.java:103)
    at com.sun.xml.internal.ws.client.Stub.createPipeline(Stub.java:328)
    at com.sun.xml.internal.ws.client.Stub.<init>(Stub.java:295)
    at com.sun.xml.internal.ws.client.Stub.<init>(Stub.java:228)
    at com.sun.xml.internal.ws.client.Stub.<init>(Stub.java:243)
    at com.sun.xml.internal.ws.client.sei.SEIStub.<init>(SEIStub.java:84)
    at com.sun.xml.internal.ws.client.WSServiceDelegate.getStubHandler(WSServiceDelegate.java:814)
    at com.sun.xml.internal.ws.client.WSServiceDelegate.createEndpointIFBaseProxy(WSServiceDelegate.java:803)
    at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:436)
    at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:404)
    at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:386)
    at javax.xml.ws.Service.getPort(Service.java:119)
    at net.webservicex.GeoIPService.getGeoIPServiceSoap(GeoIPService.java:72)
    at me.losadrian.Pirel.App$1.handleGpioPinDigitalStateChangeEvent(App.java:42)
    at com.pi4j.io.gpio.tasks.impl.GpioEventDispatchTaskImpl.run(GpioEventDispatchTaskImpl.java:66)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Quiet...

Does someone have any idea how to change this code?
Thank you in advance for any help you can provide.
Answer: Maybe I've found solution:

The service call works if I create static GeoIPService and GeoIPServiceSoap class variable.
The service call works if I create final GeoIPServiceSoap local variable before the handler in the main.

.
package me.losadrian.Pirel;

import com.pi4j.io.gpio.GpioController;
import com.pi4j.io.gpio.GpioFactory;
import com.pi4j.io.gpio.GpioPinDigitalInput;
import com.pi4j.io.gpio.GpioPinDigitalOutput;
import com.pi4j.io.gpio.PinPullResistance;
import com.pi4j.io.gpio.PinState;
import com.pi4j.io.gpio.RaspiPin;
import com.pi4j.io.gpio.event.GpioPinDigitalStateChangeEvent;
import com.pi4j.io.gpio.event.GpioPinListenerDigital;

import net.webservicex.GeoIP;
import net.webservicex.GeoIPService;
import net.webservicex.GeoIPServiceSoap;

public class App {
// Works here
//  static GeoIPService service = new GeoIPService();
//  static GeoIPServiceSoap client = service.getGeoIPServiceSoap();

    public static void main(String[] args) {
        // Works here too
        GeoIPService service = new GeoIPService();
        final GeoIPServiceSoap client = service.getGeoIPServiceSoap();

        final GpioController gpioPIR = GpioFactory.getInstance();
        final GpioPinDigitalInput sensorPir = gpioPIR.provisionDigitalInputPin(RaspiPin.GPIO_07, PinPullResistance.PULL_DOWN);

        final GpioController gpioREL1 = GpioFactory.getInstance();
        final GpioPinDigitalOutput rel1 = gpioREL1.provisionDigitalOutputPin(RaspiPin.GPIO_08, "REL", PinState.HIGH);
        final GpioController gpioREL2 = GpioFactory.getInstance();
        final GpioPinDigitalOutput rel2 = gpioREL2.provisionDigitalOutputPin(RaspiPin.GPIO_09, "REL", PinState.HIGH);

        rel1.high();
        rel2.high();

        sensorPir.addListener(new GpioPinListenerDigital() {

            public void handleGpioPinDigitalStateChangeEvent(GpioPinDigitalStateChangeEvent event) {
                if (event.getState().isHigh()) {
                    System.out.println("Motion!!");
// Not works here
//                  GeoIPService service = new GeoIPService();
//                  GeoIPServiceSoap client = service.getGeoIPServiceSoap();
                    GeoIP geoIp = client.getGeoIP("184.108.40.206");
                    System.out.println(geoIp.getCountryName());
                    rel1.low();
                    rel2.high();
                }
                if (event.getState().isLow()) {
                    System.out.println("Quiet...");
                    rel1.high();
                    rel2.low();
                }
            }
        });

        try {
            for (;;) {
                Thread.sleep(3000);
            }
        } catch (final Exception e) {
            System.out.println(e.getMessage());
        }

    }
}


Ler e interpretar arquivos JSON

Question: Estou desenvolvendo um programa no qual eu preciso ler e interpretar um arquivo JSON, que na verdade é um arquivo XML transformado em JSON. Preciso selecionar e pegar algumas informações deste arquivo JSON e guardá-los em variáveis, como por exemplo o número da nota, seu valor, cnpj do cliente etc.
Segue abaixo o arquivo JSON (editei alguns valores).
{
    "ConsultarNfseResposta":{
        "ListaNfse":{
            "CompNfse":[
                {
                    "Nfse":{
                        "InfNfse":{
                            "Numero":"12651",
                            "CodigoVerificacao":"ECSV-EJZZ",
                            "DataEmissao":"2017-07-25T17:51:12",
                            "NaturezaOperacao":"1",
                            "OptanteSimplesNacional":"1",
                            "IncentivadorCultural":"2",
                            "Competencia":"2017-07-25T00:00:00",
                            "Servico":{
                                "Valores":{
                                    "ValorServicos":"2350",
                                    "IssRetido":"2",
                                    "BaseCalculo":"2350",
                                    "Aliquota":"0.02",
                                    "ValorLiquidoNfse":"2350"
                                },
                                "ItemListaServico":"0107",
                                "CodigoTributacaoMunicipio":"6209100",
                                "Discriminacao":"TAXA: SERVIÇO DE VOTAÇÃO ELETRÔNICA",
                                "CodigoMunicipio":"2611606"
                            },
                            "PrestadorServico":{
                                "IdentificacaoPrestador":{
                                    "Cnpj":"41069964000173",
                                    "InscricaoMunicipal":"2427745"
                                },
                                "RazaoSocial":"INFORMATICA LTDA",
                                "Endereco":{
                                    "Endereco":"RUA 241",
                                    "Numero":"241",
                                    "Bairro":"Exemplo",
                                    "CodigoMunicipio":"2611606",
                                    "Uf":"PE",
                                    "Cep":"52030190"
                                },
                                "Contato":{
                                    "Telefone":"33254854",
                                    "Email":"email@example.com"
                                }
                            },
                            "TomadorServico":{
                                "IdentificacaoTomador":{
                                    "CpfCnpj":{
                                        "Cnpj":"00085803000196"
                                    }
                                },
                                "RazaoSocial":"EXEMPLO - AMBR",
                                "Endereco":{
                                    "Endereco":"ST 06",
                                    "Bairro":"Asa Sul",
                                    "CodigoMunicipio":"5300108",
                                    "Uf":"DF",
                                    "Cep":"15425845211"
                                },
                                "Contato":{
                                    "Email":"firstname.lastname@example.com"
                                }
                            },
                            "OrgaoGerador":{
                                "CodigoMunicipio":"2611606",
                                "Uf":"PE"
                            }
                        }
                    }
                }
            ]
        }
    }
}

No script PHP eu estou conseguindo decodificar o arquivo JSON e jogá-lo para uma variável, assim como exibir as informações na tela com o print_r de forma organizada, mas já tentei de tudo e não consigo colocar um foreach para funcionar, pra eu conseguir pegar certos valores nesse arquivo.
Segue meu código em php:
<?php

$json = file_get_contents('arquivo.json');

$json_data = json_decode($json,true);

echo '
' . print_r($json_data, true) . '
';

//logica do foreach

?>

Coloquei só um item, na verdade, são quase 50.
Comment: Isso aí é um XML e não json.
Comment: Com o foreach você vai percorrer determinados arrays. Pelo que entendi do seu problema, você tem que entrar nas chaves do seu JSON decodificado e chegar até o array que você quer listar. Por exemplo:

echo '' . print_r($json_data['ConsultarNfseResposta']['ListaNfse']['CompNfse'][0]['Nfse'], true) . '';
Comment: Poste o JSON também, já que esse XML foi convertido.
Comment: @RORSCHACH pode até ser dup de alguma pergunta que já existe no site, mas creio que não seja desta que postou, parece que usar o json_decode ele já sabe, o que ele não sabe é trabalhar com arrays e/ou objetos (`->`).
Answer: O que você precisa entender é que quando você decodifica um arquivo JSON em uma variável, esta variável se torna um objeto, e como todo objeto em PHP, você precisa acessar cada elemento através do ->.
Para acessar primeiro o array e iterar através dele, você precisa primeiro acessar objeto por objeto até chegar na variável que representa a lista que contém todos os outros objetos. O script abaixo provavelmente funcionará para o seu caso, você só precisa complementar com as variáveis que deseja acessar.
<?php

    $json = file_get_contents('arquivo.json');

    $json_data = json_decode($json);

    for($i=0; $i < count($json_data->ConsultarNfseResposta->ListaNfse->CompNfse), $i++){

        echo $json_data->ConsultarNfseResposta->ListaNfse->CompNfse[$i]->Nfse->...
        echo $json_data->ConsultarNfseResposta->ListaNfse->CompNfse[$i]->PrestadorServico->...
        echo $json_data->ConsultarNfseResposta->ListaNfse->CompNfse[$i]->TomadorServico->...
        echo $json_data->ConsultarNfseResposta->ListaNfse->CompNfse[$i]->OrgaoGerador->...
    }

?>

Teste o script e veja se funciona, caso haja algum problema, comenta aqui que eu ajudo.
Comment: Obrigado Cloud, eu só tive que tirar o "true" ali do decode que deu certinho.

Ler e interpretar arquivos JSON


If $ a+ \frac{1}{b}= b+ \frac{1}{c}= c + \frac{1}{a} = k$ , how to find the value of $k$?

Question: If $a,b,c$ be distinct (non-zero) such that
$ a+ \frac{1}{b}= b+ \frac{1}{c}= c + \frac{1}{a} = k$
how do I find the value of k ?
Comment: Possibly related : https://math.stackexchange.com/questions/1917588/algebra-problem-a-1-b-b-1-c-c-1-a-t
Comment: Please rewrite the equations with MathJax. I, at least, don't understand what you mean by the numbers. What is "1b" and "1c", etc ?
Comment: Welcome to MSE. Your question is phrased as an isolated problem, without any further information or context. This does not match [many users' quality standards](http://goo.gl/mLWc8), so it may attract downvotes, or be put on hold. To prevent that, please [edit] the question. [This](http://goo.gl/PlJyVQ) will help you recognise and resolve the issues. Concretely: please provide context, and include your work and thoughts on the problem. These changes can help in formulating more appropriate answers.
Comment: thanks everyone, I'll try .I need time to get used to this
Comment: If you negate all of $a,b,c$, you get the opposite value of $k$ too. And $k=0$ is clearly impossible since $a,b,c$ would then need to have pairwise opposite signs...
Comment: from value of $abc$ , how to find $k$?
Comment: @user, my answer to that question *starts* by finding your $k$.
Answer: Hint: From the second equation we get $$b=k-\frac{1}{c}$$ so we get
$$a(kc-1)+c=k(kc-1)$$ and wwith $$c=k-\frac{1}{a}$$ and we get
$$a(k(k-\frac{1}{a})-1)+k-\frac{1}{a}=k(k(k-\frac{1}{a})-1)$$
simplifying and factorizing we get
$$(k-1)(k+1)(a^2-ka+1)=0$$
Can you proceed?
Comment: Thanks so  much , helpful.
Comment: Is your problem solved now?
Comment: Yes.  solved . after problem solved, is there something I have to do? this is my first time MSE.
Comment: No,no, but you can give me some points, if you want.
Comment: I tried. but MSE says "Votes cast by those with less than 15 reputation are recorded " .... I just came here and this is the first question. so I have only one reputaion. so I can't vote..
Comment: Ok, then have a nice day, with systems of equations.


How to get an a-tag to fullsize

Question: it would be great, when I can click the
 container and in it is the  ..
So my container is width: 100px and height: 100px but my  tag only width: 20px and height: 15px.
So I can only click on the text in the  tag.
Can I change that?

SEM

CSS:
.menu ul li {
    background-image: url("../images/menu_background.png");
    background-position: 1px center;
    background-repeat: repeat-x;
    border-color: #000000;
    border-right: 1px solid #000000;
    float: left;
    font-family: 'Open Sans',sans-serif;
    font-size: 16px;
    height: 38px;
    list-style-type: none;
    padding: 0 34px;
    vertical-align: middle;
}

.menu ul li a {
    color: #FFFFFF;
    display: block;
    line-height: 35px;
    text-decoration: none;
}

that doesn't work..
Comment: By default the `` tag is an inline element.
Answer: Set a block context property on it.
li a {
display:block;
}

Comment: @AlexKante Try adding the padding to the `` tag and not the ``
Comment: Then you have something else going on. Perhaps you could provide a link or a JSFiddle
Answer: You should apply display: block to your 'a' tag
An example : http://jsfiddle.net/e49Tg/
Comment: Provide us a demo link to debug the issue
Answer: You can try li > a { display: block; } in your stylesheet. Make sure you prefix the li with the appropriate ul class or selector, just so you keep your change isolated to that occurrence.
Answer: I can't clearly get what your problem is, if you want an  tag to be sized, the solution is as proposed: the "display" property should be either "block" or "inline-block".
In case you want it to have the same size as the parent it should also have the size directives, like:
li a { display: block; width:100%; height: 100% }

and if you want it to be so when clicked, a pure css solution would be
li a:active {
  display: block; width:100%; height: 100%;
}

If this isn't answer to your question, explain more.

How to get an a-tag to fullsize


How does the reparameterisation trick work for multivariate Gaussians?

Question: I understand that for sampling from a univariate Gaussian, we can use $x = g(\epsilon) =  \mu + \epsilon \sigma $ and then differentiate this transformation with respect to $\mu, \sigma$. How does this work for sampling a multivariate Gaussian with a covariance matrix $\Sigma$?
In this paper (p19) they say $g(\epsilon) = \mu + L\epsilon$, with $LL^T = \Sigma$. What is this $L$? It can't be a Cholesky factorisation, for the terms to add it must be a vector, but covariance matrices won't decompose this way.
What is the transformation for the reparameterisation trick on a multivariate Gaussian? And then what is its gradient with respect to $\Sigma$?
Answer: I am being stupid --  it is in fact the Cholesky, $L \epsilon$ is a vector for a vector $\epsilon$. Found my answer here in case it helps anyone else who is confused


Como adicionar mais linhas a uma classe em Ruby?

Question: Comecei a me aventurar no mundo das rubygems. Estou desenvolvendo uma gem para gerar os assets da minha aplicação ao estilo twitter bootstrap, já que pretendo reaproveitar o meu código front-end. Tudo que eu preciso fazer é colocar os assets nos lugares corretos, adicionar alguns helpers e efetuar algumas configurações. Várias gems que funcionam dessa forma, fazem adições ao arquivo application.js/.css no momento da instalação. Por exemplo, o bootstrap, quando o comando rails g bootstrap:install é executado, os assets são transferidos para o devido path e um require é automaticamente adicionado ao application.css e ao application.js.
Quero fazer algo semelhante, só que em um ponto específico do arquivo application.rb. Sendo mais exato, o que eu quero é adicionar mais pastas ao path de assets. O arquivo é mais ou menos assim:
class Application < Rails::Application

end

Eu preciso que fique tipo assim:
class Application < Rails::Application
     config.assets.paths << 'path/para/a/pasta'
end

Mas como eu faria para garantir que sempre esse conteúdo será adicionado dentro da definição da classe? Digo, simplesmente fazer adições no arquivo é simples, mas nesse caso não posso adicionar fora do escopo da classe.
Answer: Tem alguns exemplos bem legais de como fazer isso, dos quais eu destacaria um da gem responders: https://github.com/plataformatec/responders/blob/master/lib/generators/responders/install_generator.rb
Nesse caso, ele usa a funcao inject_into_class, que vem de Rails::Generators::Base, e o seu codigo ficaria assim:
def add_assets
  inject_into_class "config/application.rb", "Application", <<-RUBY
    config.assets.paths << 'path/para/a/pasta'
  RUBY
end

Acho que bastaria colocar isso em um generator...
Comment: Valeeeeeu! É isso mesmo que eu preciso!


How do I make my table tennis paddle's rubber less sticky?

Question: I've only been playing table tennis/ping-pong for about ~3 months now at work. Some of these guys are brutal when it comes to spin. While I have gotten a lot better at handling top/bottom spin, I am still trying to get a handle on some of the side spins.
I started off with a nice paddle but it had a lot of spin control and I was just shooting the balls all over the place. I swapped to a cheaper, lower end paddle with a lower spin control rating and it was working very well.
I did a bit of research on paddles, techniques, etc. and found that rubber with no spin control could be used to return the ball with the original spin so the opponent has to deal with the spin. I decided my strategy was to use a paddle with no spin control and work on my angles and attacks. This way I'm not always trying to defend against spin. It was working out very well and I started to win some games.
Then, I cleaned my paddle. Now the rubber is tacky and I have the same issue, defending against spin. I've tried wiping my finger oils on the rubber, I've tried olive oil (which I just found out does the opposite of what I wanted!) and even tried dusting it.
How can I get my rubber to not be sticky/tacky again?
Comment: It would be really help full if you can share manufacturer name of your paddle/rubber.
Answer: I realize this is an older question, but I think my answer might be useful to other people with the same or similar questions.
Anti-spin rubber can take the spinniest of serves and return it virtually deadened.
Several rubber manufacturers create anti-spin rubber.  It is designed for exactly what you're looking for, however there are some disadvantages as well.  Most users will put the anti on their backhand side of the paddle.  I don't know of anyone who uses anti on both sides.
In my own personal opinion, I believe it's better to learn how to handle spin.  However many top players have (and continue) to use anti-spin.
Danny Seemiller, for example, with his own namesake grip, has the distinct advantage of being able to put fast, spinny rubber on one side of his paddle, and anti on the other.  Because of his unique grip, the same side of the paddle is used for both backhand and forehand shots.  This allows him (and others using that grip) to be able to "flip" the paddle to anti, say for spinny serves, then flip the paddle again for attacking.
Those of us who use the more traditional shakehands grip can also flip the paddle, but there's always going to be one side of the table, forehand or backhand, that you'll have to play with your anti, the other with the regular rubber.
Answer: Try a thin layer of black charcoal dry lubricant (Home Depot has them in a spray can.), test it and see if it's slippery enough (it would not smear after it drys in 3-5 min. Tape/cover the areas you don't want to cover. Use it on the black side only or your want the red side to turn black. To remove wipe with WD40, it will be a bit messy.
Answer: As far as I know the only way you can get off the tackiness of your paddle's rubber is to let natural dust cover it as you play.
I have Hurricane 3 (I use it on backhand) and it was quite tacky in the beginning. But as the time passes, its tackiness get reduced as dust clots on it (And I have to clean it to get the tackiness back).


10,000+ nodes vs. 1 node with 10,000+ field collection items

Question: I need a sort of term database with around 3 fields that won't be used during normal site operations.. Only when I, the administrator, add content.
I'm wondering about how to minimise performance impact for regular users.. My intuition tells me that a field collection would be better so the nodes table is smaller.
I will be likely be using some code to add data from APIs and Rules to fetch the data by property.
Which should i use? I don't have much interest in creating a custom entity type unless it's really easy.. I want image and audio fields and I don't know if that complicates things.
Comment: Bonjour Niall! Not sure I fully get it, but how about making each of those 3 fields an entity reference field, pointing to some other content type (1 such type for each of those 3 fields). That way you don't "need" to use field collections, and have a variation of that custom entity type you want to avoid. And you can probably keep your rules relatively easy.
Comment: Hi Pierre! See there's no overlapping data so having entity references would result in like 40,000+ nodes. Eg. ["Airplane" - definition, picture, translation, audio]. It's best to keep it all together. I already use field collections for other small parts of this site that I don't mind being slower and am comfortable with coding for them. It's really just a performance thing if using field collections will make impact regular nodes less. I don't know enough about databases to know for sure.
Comment: Pierre, I am going to delete this question.. I think a second database would be better. I just found a goldmine of open-source data free to use commercially so my initial estimate of 10k will grow to more like 100k.
Comment: @NiallMurphy - Quite interesting, general question. When using the external database will you import the content into Drupal then? This sounds like a good job for Feeds https://www.drupal.org/project/feeds
Comment: Yeah, I'm already using Feeds / Feeds Field Collection and a module to make other API calls. I have never looked at using a second database before but I think it would be perfect for what I want to do.
Answer: Loading 1 node with like 10,000+ field collection items sounds like performance suicide if you don't limit the amount of items being loaded and displayed initially. And having to open a node with 10,000+ field collection items only to maybe add or edit one item will be really big pain then.
With nodes, much likely you'll never display all of them at once. You very likely will have them organised in Views. Limited display, pagers, filters, lots of caching possibilities. When you want to add a node, you'll have a fresh node form, loaded in milliseconds. Plus you will have a much larger number of contrib modules available dealing with sortability, searchability, orderability and a lot of other -abilities that come in handy when organising content. Also when it comes to content import.
Go for 10,000+ or even 1,000,000+ nodes. That's what Drupal is made for.
Comment: @NiallMurphy - That sounds reasonable as you already are on track and you have the proven power of a very stable D7 and all it's reliable contrib modules available. Good luck!
Comment: The thing about Field Collection is this - every item constitutes 1 entity_load. So if you had a FC with 100 items, thats 100 entity_loads + all listening hooks or events _on top of_ the page request itself. My advice, if you go with FC, is find a way to pre-render and cache the HTML before users visit the page. The performance otherwise will kill you without some sort of strategy.
Comment: That all makes a lot of sense! At what number of nodes would you think a second database specifically for this would be worth it? All of these nodes will basically be used for data when creating regular nodes.
Comment: @NiallMurphy - I'ld try to circumvent a second database as much as possible. You'll run into big problems sooner or later. Think of users, think of permissions, think of sessions, think of caching, think of updates. You'll have to deal with a lot of issues. One Drupal database is fine for millions of nodes.
Comment: @NiallMurphy - And one last recommendation as it's about performance and you still seem to be in the starting moves of an interesting great project. Start with Drupal 8 and with PHP 7. This probably will make your life much easier in the end. Much more high-performance at least.
Comment: I've put over 2.5 years into this project so far so can't change now. I'll just stick with one database then and throw money at server power instead if there are issues. Thanks!


How to load field value to custom form?

Question: I'm using Drupal 7 for my website.
I created custom settings page with custom form in it, but the problem is I don't know proper way to load inserted value from database to my form after saving it, and every time I write something in it, information is written to database, but form is empty.
My code looks like this for this:
function registration_pdf_form($form, $form_state) {
    $form['registration_body_twoside'] = array(
        '#type' => 'text_format',
        '#title' => t('Two-sided PDF'),
    );
    $form['registration_body_threeside'] = array(
        '#type' => 'text_format',
        '#title' => t('Three-sided PDF'),
    );
    $form['actions'] = array(
        '#type' => 'actions'
    );
    $form['actions']['submit'] = array(
        '#type' => 'submit',
        '#value' => t('Save settings')
    );
    return $form;
}
function registration_pdf_form_submit($form, $form_state){
    $values = array(
        'reg_twoside' => $form_state['values']['registration_body_twoside']['value'],
        'reg_threeside' => $form_state['values']['registration_body_threeside']['value'],
    );
    $insert = db_update('pdf_settings')
        -> fields(array(
            'reg_twoside' => $form_state['values']['registration_body_twoside']['value'],
            'reg_threeside' => $form_state['values']['registration_body_threeside']['value'],
        ))
        ->execute();
    drupal_set_message(t('Settings have been saved'));
}

How do I get values to my fields from database?
Comment: Query the database and set the default value of the field.
Comment: I tried using db_query for that, but ended up with this error: Warning: htmlspecialchars() expects parameter 1 to be string, object given - check_plain(). and dont get any values to a field.
Answer: You can try using system_settings_form which is simple way of storing settings in Drupal..
function registration_pdf_form($form, &$form_state) {
   $form['registration_body_twoside'] = array(
        '#type' => 'text_format',
        '#title' => t('Two-sided PDF'),
        '#default_value' => variable_get('registration_body_twoside', ''),
    );
    $form['registration_body_threeside'] = array(
        '#type' => 'text_format',
        '#title' => t('Three-sided PDF'),
        '#default_value' => variable_get('registration_body_threeside', ''),
    );
  return system_settings_form($form);
}

you can read values using
$twoside = variable_get('registration_body_twoside', '');
$threeside = variable_get('registration_body_threeside', '');

How to load field value to custom form?


Fixed point generalisation

Question: If a continuous function  defined on an open interval in $R$ and the range is a proper subset of domain, then there exists a fixed point. Is this statement true? How about making domain and range compact and either of domain or range contains the other?
Answer: Try the function $f(x) = x/2$ on $(0,1)$.
It's true for a function mapping a compact interval into itself by the Brouwer fixed point theorem.
Comment: I am sorry, by proper subset I meant something like (0.1    0.9) I mean closure of range being subset of domain)
Comment: If so, there is a compact subinterval of the domain that contains the closure of the range, and you can use the Brouwer fixed point theorem.

Fixed point generalisation


Is there any drupal module to export/import all ubercart products with product details(attributes)

Question: I am looking for the module which having the option to import all ubercart products including all information like product images, price, sale price and other attributes. After exporting the information how we can import these to other drupal setup.
If there is no single module to do this then what will be the right approach to do the same.
Answer: I would use start by having a look at UberCart Feeds with  uc_feeds_product_importer  they should help you get the job done.


Liferay JavaScript Function is not defined

Question: I am new to liferay and I have been trying to do Ajax but my script does not load inside the browser. I also tested it by inserting a simple alert.
I've been getting the "function is not defined" error.
Below is my liferay aui script:
Liferay.provide(window,'refreshFunction',
    function(param){
        alert('param: ' + param);
        A.one('#divToBeRefreshed').plug(A.LoadingMask);
        var mask = A.one('#divToBeRefreshed').loadingmask;
        mask.show();
        var dTime = new Date();
        var dynamicURL = '<%=resourceURL %>&t=' + dTime.getTime().toString();

        A.io.request(dynamicURL, {
            method: 'GET',
            cache: false,
            data : {
                param: param
            },
            on: {
                success: function(event, id, obj) {
                    var response = this.get('responseData');
                    mask.hide();
                    $('#divToBeRefreshed').html(response);
                },
                fail: function(event, id, obj) {
                    mask.hide();
                }
            }
        });
    },
    ['aui-base']
);  </aui:script>

Is there something wrong in my script? what might be the problem? HELP!
Answer: Your script should be wrapped in
;(function(A, Liferay) {
<<your script>>
}

This will allow the use of A and Liferay. you can remove <aui:script>
Answer: in tag  of popup add
onclick="window.parent.MyFunctionInPageParents()"


How to use the LINQ where expression?

Question: I'm implementing the service \ repository pattern in a new project.  I've got a base interface that looks like this. Everything works great until I need to use the GetMany method.  I'm just not sure how to pass a LINQ expression into the GetMany method.  For example how would I simply sort a list of objects of type name?
nameRepository.GetMany( ? )
public interface IRepository<T> where T : class
{
    void Add(T entity);
    void Update(T entity);
    void Delete(T entity);
    void Delete(Expression<Func<T, bool>> where);
    T GetById(long Id);
    T GetById(string Id);
    T Get(Expression<Func<T, bool>> where);
    IEnumerable<T> GetAll();
    IEnumerable<T> GetMany(Expression<Func<T, bool>> where);

}

 public virtual IEnumerable<T> GetMany(Expression<Func<T, bool>> where)
 {
    return dbset.Where(where).ToList();
 }

Comment: Note that you can use generics like : GetById(U id) , instead of getting string or long .
Answer: Assuming that you had an implementation of IRepository<MyClass>, you would make a call to GetMany like so:
IRepository<MyClass> repository = ...;

IEnumerable<MyClass> filtered = repository.GetMany(
    mc => true);

Note the mc => true lambda expression.  mc in this case is the parameter (in this case, of type MyClass) passed to the Expression<Func<T, bool>> which will be evaluated (one would assume through an IQueryable<T>) and the mc.SomeProperty == someValue is an expression which returns true, which is the second type parameter of the Expression<Func<T, bool>>.
Once you have the filtered, you can use the order by clause (or the OrderBy extension method, they're the same) to sort the results, like so:
var filteredAndOrdered = filtered.OrderBy(mc => mc.MyProperty);

Note that GetMany returns an IEnumerable<T>, not an IQueryable<T>; this is important if your result set is large, as the ordering will happen on the client, and not be sent to the server.
For an OrderBy operation, this can be expensive because it has to go through the entire sequence (to order) before returning the first result.
Comment: Thanks for the quick reply. The example provided is with a condition, I'm simply looking for a way to return all object sorted using the get many.
Comment: Thanks, it was the mc => true part that was throwing me off.

How to use the LINQ where expression?


How to get the site name in drupal 6?

Question: In some situations, I need the site name of a Drupal 6 site which is given while installation of the site.
How can I achieve it?
Answer: In both Drupal 6 and Drupal 7, you can use variable_get.
The CMI initiative will likely deprecate this function in Drupal 8.
<code>$name = variable_get('site_name', '');
</code>
Notice the blank '' at the end, which is the "default" value, in case a variable isn't defined.
Comment: In Drupal 6, [`variable_get()`](http://api.drupal.org/api/drupal/includes!bootstrap.inc/function/variable_get/6) has a required second parameter: `$default`.  If you leave this required parameter off, you may get PHP warnings (depending on the PHP error level configured on your server).
Comment: Good point, added this.

How to get the site name in drupal 6?