how to delete all files from a directory structure in a HDFS?

Question: I am looking to clean up the environment. I want to execute the following command recursively, witout hard coding for each sub folder. As I have hundreds of subfolder in a HDFS directory.
hadoop fs -rm -r /a/b1/c1/d1.txt

hadoop fs -rm -r /a/b2/c2/d2.txt

hadoop fs -rm -r /a/b3/c3/d3.txt

hadoop fs -rm -r /a/bn/cn/dn.txt

I want to delete all the files but not the directory or the sub-directories. I am not sure how to use a recursive command to keep the directory structure in HDFS while delete all the files in it.
Comment: You can't keep the directories from the command line, you need to iterate the directories in a script (Java would be easiest)
Comment: So I am using Talend big data tool so I can use java but not sure about the logic.
Comment: On a high level, import the `FileSystem` object from Hadoop, then iterate over folders and delete(path, false) on them... but make sure the directories are kept https://hadoop.apache.org/docs/r2.8.2/api/org/apache/hadoop/fs/FileSystem.html#delete(org.apache.hadoop.fs.Path,%20boolean)
Answer: hadoop fs -ls -R /directory/structure/ | grep "\.json" | awk '{print $8}' | xargs -I{} hadoop fs -rm "{}"


ImportError: cannot import name 'randint' ... partially initialized module 'random' (most likely due to a circular import)

Question: I am trying to use random library in my VS code. This code is running well on Pycharm though. But I want to use VS Code anyway because it is less complicated. Below is my code
import random
number = random.randint(5,6)
print (number)

This is the error I'm getting.
from random import randint
ImportError: cannot import name 'randint' from partiaom.py", line 1, in <module>lly initialized module 'random' (most likely due to a circular import)

Comment: Your error does not match your code.
Answer: I think many people make the same mistake I did. Creating a file called random.py and getting errors and trying to find the reason...
Do NOT name the file random.py :) That was the lesson


Ratio of training/testing data different to real life?

Question: My company wants to build a model that will be used to predictive conversion that is usually about 2%.
However every sample we purchase (converted or unconverted) is expensive. So my question is:

How many converted samples do i need, 500 or 1000 ok?
How many unconverted to I need? The same number? I can't have as many as possible.

If i build a model using a 50/50 split, will that be OK to use on a real world sample of 98/2? Or do I have to do something like resample the unconverted to get a more real-world split?
Just wondering if there is any rule of thumb here? I'm not even sure the name for my problem, Domain adaptation or sampling bias?
Thank you.
Comment: I typically purchase as many samples as I need to equal 2000 if the thing I'm trying to predict. This rule if thumb is based on the fact that I made plenty of explanatory variables. In your example, this means I would be buying 2000/0.02=100,000. Regarding your concerns about splitting and oversampling, if you can save money by purchasing less but still getting 2000 y=1, then go for it, just remember you'll have to adjust your final model probabilities down to 2% to match the real world. I've personally not heard of such a thing, I'd be stuck buying 100K.
Comment: Thanks Josh, I'm told I cant afford to do that. So i was wondering what a alternative approach might be. If i can afford 5,000, I assume an even split is best.
However if only 1,000 positive records exit, will it be ok to purchase these and 4,000 negative. If i can fight to get 1,000 positive and 9,000 negative, will it really be much better?
I realise how subjective this is but appreciate any feedback!
Comment: Yes, always more the better. Since you're  limited by budget, buy what you can and you can always address the challenges during the modeling process. If you only have the money for 4000, then try to get 2000 of each. If you can only get 1000 positives, then get as many negatives as possible. Your advantage is that you already know the rate is around 2%, so you don't need to worry about the sample rate as much, and can just focus on getting as much as possible. After you finish the modeling process, you can adjust the predicted probabilities down to near 2%.
Comment: Thank you very much. Can i just ask about "adjust the predicted probabilities down to 2%". What exactly do you mean. If i train a model based on a 50-50 split, I can't just divide the predicted probabilities by 25 to get an idea of the real probability right?!!
Sorry for asking the basics.
Comment: After you build your model on a 50/50 dataset, let's pick an average observation and see that it produces a predicted probability of 50%. In real life, that person would only have a 2% probability, but it would indeed still produce a predicted p of 50% This could be a problem if you're using your probabilities for something where you need them to be accurate. Therefore, they all need to be adjusted. You can't simply subtract 48% from each one either, because you might get negatives. You can use this equation:

P_i** = ( P_i*  x  R_0 x  P_1) / ( (1-P_i*) (R_1)(P_0)  +  (P_i*)(R_0)(P_1) )

where
Comment: You can use this equation:

P_i** = ( P_i*  x  R_0 x  P_1) / ( (1-P_i*) (R_1)(P_0)  +  (P_i*)(R_0)(P_1) )

where:

P_i* is the unadjusted probability you get from your model

R_0 and R_1 are the sample proportions of 1 and 0 respectively

P_0 and P_1 are the original event and non_event rates (population rates)

P_i** is the true probability
Comment: Thank you for taking the time to write all that out Josh. I'm a beginner at all this. I get ML concepts, but best practises in real life are always a bit confusing to me and difficult to get an answer on. Thanks again.
Answer: The number of samples you need depends on how complex a model you want to use (and on the distribution of the samples in your feature space). There is no simple rule here. How many features do you anticipate your model will use as input? If it is only a few, then ~1000 samples should be adequate.
50/50 split should be fine for training the model, but for your test set you want the distribution to be similar to the distribution you would see in the real world. But it depends on what you want the model to do. Look into the precision/recall tradeoff.
Comment: Thanks for the response. I always assumed training and testing had to be the same ratio because people always talk about splitting it randomly.
So if i can only get 1,000 converted applications, but can get more than 1,000 unconverted (say 5,000), there is still benefit in doing that, right?
Comment: Random splits are usually performed, but are not always 'best practice'. For example, if you are concerned with the external validity (~ out of sample performance) of a prediction model, you might specifically want to test your model in a poopulation which has some characteristic different compared to the development set (think of sampling from a different region or timeperiod). The same goes for the occurence of outcome; if you want to see whether it affects model performance, you might opt to specifically have different ratios in develop and training sets.
Comment: Thanks, I will do that.
So in the testing set, if i can have, lets say 1,000 positive and 1,000 negative results. That's OK.
If i can have 1,000 positive and 5,000 negative that's better right?
And even if i had 1,000 positive and 99,000, would that even be ideal or actually overkill as i read that some predictive algorithms don't work well with such imbalance?
BTW the model i anticipate to be simple (<10 predictors)
Comment: This question may be relevant to your problem: https://stats.stackexchange.com/questions/9398/supervised-learning-with-rare-events-when-rarity-is-due-to-the-large-number-o


preference of the key of B or Cb (also G#min and Abmin)

Question: The debate of whether F# or Gb is a better key is very prominent, but I'd like to discuss the keys of B and Cb.
As a guitar player, I would much rather have the key of B to play in. As a trombonist, the same key makes me cringe wildly. And as a pianist I am impartial to both, although playing in Cb is more comfortable and easier with accidentals.
String players have told me that they would rather play in B, and that seems reasonable to me, as a guitar player myself I can understand that string players often prefer to play with sharps instead of flats in their repertoire.
The brass players I've talked to have told me that, while they fully expect a chart to be written in B major, they would much rather play in the key of Cb major.
I have also had this discussion with my piano teacher, when pitting G# minor against Ab minor. I don't know how common each key is, but he did say that he prefers Ab minor, and the biggest reason why is that when you have a dominant seventh in the key of G#m, it's D#7 which has a nasty Fx. Compare that to the more lenient Eb7 of Abm.
What are your thoughts? I assume that in a popularity contest, because Cb is such an obscure key, B will win, but I wanted thoughts on the playability, practicality and convenience of each.
** I am talking in the context of 12-TET, and not an other tuning where the difference between two semitones is varied for each key.

Quickly want to add that in most orchestral music that is in the key of B major, if there is a harp part for it it would be written in Cb, as it is considered the home key of the instrument.
Comment: By "playing" do you mean reading music, or is there a difference for playing by ear too ?
Comment: @Rosie I don't know, that's why I asked. Or does the OP use the word "play" as a synonym for "read"? Maybe.
Comment: @piiperi Seeing as the OP's established the context of 12-equal temperament, we're comparing different notations for the same pitches. So how would that affect playing by ear?
Comment: @piiperiReinstateMonica - playing by ear, I think most players won't necessarily be thinking in a key, but if they do, yes, good point.
Comment: For a lot of people, writing a piece in either Bb or C would be a better bet, but wonder how that would affect some listeners. It probably would affect players adversly. But if someone with absolute pitch was introduced to a new piece in Bb or C instead, what difference would it make? None?
Comment: @Tim If you have an awareness of where the tonic is, then you're playing in a key. But if you don't sense where tonic is, then how do you play anything, without a point of reference?
Comment: @piiperiReinstateMonica - it depends on what instrument I'm playing. On bass, for example, I may well establish 'the key' as 'that fret on that string'. It doesn't necessarily have to have a name. I know the name, but it doesn't come into the equation. Example - in key A, I feel things go up a 4th. I don't think '4th of E is A, so find A.' I just move across a string. Maybe different on keyboard.
Comment: @piiperiReinstateMonica I do indeed refer to reading music, but I guess there could be a diference for playing by ear too, if there was it would probably depend on how often they hear this specific key as "Cb" or "B". Just a guess, though.
Answer: Brass players are more at home in flat keys because their instruments are built that way: with fundamentals of F, Bb and Eb. (Yes, I know: trumpets in A, C and D, trombones in G and probably others also exist!) Guitars have a slight bias towards the sharp side. (Yes, I know: capos!)
On a piano any key is fine, but I can't think of a piece in Cb. Can anyone? Plenty of pieces visit Cb (Bolcom's Graceful Ghost springs to mind) but they don't often start and end in it. So it's rather an unfamiliar key. I could play scales of Cb faster thinking of them as B!
If it was at all interesting a piece of music in seven flats would soon need double flats, and along with the double-flat signs there would be the signs to cancel the double and return to single-flat. Big chords with mixtures of those accidentals are a pain to sight-read: the bars get cluttered, like a street full of bike lanes and No Right Turns.
It's odd that Cb major is such a warm, squashy key when B major is such a glittery one.
Comment: How to explain your last para? Would you have to *know* which key it was to get a warm, squashy OR a glittery feeling..?
Comment: A very interesting question @Tim. As a pianist, there's definitely a psychological difference between extreme sharp keys and their enharmonic flat equivalents. I was told by my teacher that a professor at one of the academies asked a teacher to get their pupils to learn the Bach C# major prelude and fugue - but to get half of them to learn it in Db (it's sometimes published in that key). He was able to tell when listening to their performances which key they had learnt it in without knowing in advance.
Comment: @Peter - Would it be because the pupils' mistake rate was higher in one of C sharp major or D flat major?
Comment: @Dekkadeci I doubt it - these were academy students. It's more to do with how you play it. In C# I would use a more staccato touch. But it was a long time ago - I heard about it in the early 1960s and it was old then.
Comment: @Peter Yes - I'd play it with more staccato too, though I'm not sure why. (Perhaps because C# is colder and more fragile, Tim!) I hadn't heard that story about the students and Bach since the early 70's. Excellent!
Comment: @Tim  I once saw a simplified version of the Moonlight Sonata. It was intended for beginners and had been transposed from C#m to Cm. It was oddly shocking to see. Beethoven loved Cm, but he would *never* have written that piece in that key. Cm was for other stuff. The reasons for seeing the keys as having unique qualities are many and varied. 'Classical' music often has the key in the tiltle, so you are told before it starts that, for example, Tchaik's 4th is in Fm before a note is played. Listen to - and play - enough music and you get a feeling for what the keys have meant over the years...
Comment: @Tim there is the concept of sharper keys being "brighter" and flatter keys "darker"; when traveling around the circle of fifths each on the sharp side the keys tend to get "brighter" and the opposite seems to get darker. But it definitely feels like more of a psychological concept.
Comment: @Lars - are you sure you meant psychological and not psychiatric (or most other words beginning with psy..? In 12tet all things are equal... So, when you finally get to the other side, I guess they cancel each other out!
Comment: @Tim ... Some people have suggested it all dates from the time before equal temperament, when one key really *was* different from another. I don't know if that's true, but musicians - jazz as well as classical - tend to agree about the different qualities of the keys. I haven't got perfect pitch. (Thank goodness! As a composer I regularly have to write for transposing instruments, which is easier without it) But all our lives we hear musicians playing with this understanding of, say, the bucolic quality of F major, and it stays with us. We associate bucolic music with F and vice versa.
Comment: @Lars Clearly music modulating to the sharp side generally feels brighter because a note has been sharpened: we are lifted into the new key. In the other direction, when a note is flattened, we relax or settle or feel resigned. Countless hymns do this, rising at the end of the second line and then falling back again. It's a different thing.
Comment: @OldBrixtonian - I'm sceptical. Many's te time, before digital tuners, we had to tune to whatever - sometimes a piano at a gig migt have been a quarter tone out. Where did that put the music? I'm not sure if we then played i sharp or flat keys, unknowingly. Point is, the numbers didn't sound or feel any different. And surely, in just temperament or other, one key would have been the best, with each successive key round the circle getting progressively more out of tune. *That* would make differences!
Comment: @Tim - Maybe it's because of my absolute pitch, but music in quarter-tone keys or close enough often comes off as unnerving to me. Boss themes like "Rude Buster" from *Deltarune* (key is between F sharp minor and G minor) and the *Final Fantasy Mystic Quest* regular boss theme (key is between B minor and C minor from what I've listened to) tend to be fine because unnerving boss themes are typical enough, but a studio recording of "Root Beer Rag" threw me for a complete loop because it was in between C major and D flat major (while I believe I first heard the piece in C major).
Comment: @Dekkadeci - interesting! Key question (please excuse pun...) What is your absolute pitch in - 440Hz, or something different?
Comment: @Tim - Yup, it's in A440.
Comment: @Tim: As an introduction to what historical tunings can do relating to Beethoven, I might suggest a listen to the CD "Beethoven in the Temperaments" (Enid Katahn, pianist). These are played on a modern concert grand, but using tunings that were common around Beethoven's time. Observe the perfect "flatness" and placidity of the opening of the Waldstein with a C major chord more in-tune than equal temperament, vs. the "buzz" and liveliness of the dim. sevenths at the opening of the Pathetique with more unequal thirds. Historical composers appreciated the differences among less "in-tune" keys.
Comment: @Tim Are you sceptical about musicians agreeing about the 'colours' of different keys? Or about composers writing certain types of music for certain keys? [About the quarter-flat insts:] I don't know what kind of band or orch you're talking about, or what kind of music they were playing, but I think orch and jazz musicians would still have played something written in E more brightly than something in Db. And 'A' wasn't always 440 of course. We don't know exactly what kind of 'well temperaments' Bach wrote for, but I'm sure, as Athanasus suggests, he enjoyed the differences between the keys.
Comment: @Athanasius It was quite hard to find but I just downloaded that CD from CDBaby. Thanks for mentioning it. Yes - the extreme in-tuneness has a
Comment: @OldBrixtonian: I think the end of your comment was cut off, but I hope you enjoyed it. I haven't listened to the CD in several years, but my recollection is that the performances were not necessarily my preferred ones, but the tuning was intriguing. Since the time that CD was released (late 90s), there have been a lot of releases of such music played on 19th century or 18th century pianos, usually in appropriate temperaments.  But putting those temperaments on a modern concert grand makes them stand out a bit more to my ear, which is why I thought of that specific album in this discussion.
Comment: @Athanasius Yes. I got interrupted!  I've listened to Moonlight and Waldstein so far and I think they gain a lot played with those tunings: there is a *reason* for Beethoven's wanderings. Yes - that C major chord is extremely still, isn't it? Inert. The minor 9th dissonances in the 1st mov't of the Moonlight are more beautiful and limpid. It's like hearing this music with shiny new ears!
Comment: @Athanasius Maybe you're right about the perfomances, I like its precision and its playfulness, but the pedalling is unusual in places. Btw,I would be interested in any recordings you know of using more extreme historical tunings. I used to be on a mailing list of academics and composers studying and experimenting with this stuff and have written a number of pieces in some outlandish tunings. There were two schools of thought among the composers there: 1) At the start of a piece, slide the most out-of-tune notes in *gradually* so that listeners can adjust to them. 2) Hit 'em between the eyes!
Comment: @OldBrixtonian: This discussion is perhaps going on too long in comments, but regarding recordings with more extreme historical tunings -- the easiest ones to find are often those played on old organs, which are often maintained in some variety of meantone tuning.  There are a lot of recordings of 17th-century repertory (e.g., Frescobaldi, Froberger, Buxtehude) played on various organs. Chromatic music often sounds quite wild, but when you get a cadence on a nice plain C major chord or something, the contrast is amazing.
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/103211/discussion-between-old-brixtonian-and-athanasius).
Comment: @OldBrixtonian I heard that story at school - in the early 1960s. It's all psychological. One thinks of flats as "sinking" and sharps as "rising".
Comment: @Peter  Hi Peter. Sorry - I'm not sure which of my comments you were commenting on! What did I say? What story?
Comment: @OldBrixtonian Jan 10 17:48. I think it happened before the 2nd World War somewhere in England.
Comment: @Peter Oh Ok. Thanks. Yes, that seems likely. Btw, when you say it's all psychological, do you mean you don't believe the story of the professor?
Comment: I believe it totally. I meant that it's the psychology of the performer. So the listener can pick up the different style of playing (smoother vs. spikier etc.).
Answer: I much favour keys with small accidental count.
In this case:
Cb has 40% extra accidentals with respect to B. Those extra accidentals are millions of times rustier inside my brain than for example the tone Bb, as I never use them (other than literally as accidentals, i.e. passing tones or other melodic devices). As a result, the difficulty increases much more than by this 40%.
As you add accidentals one by one (for example, keys C, F, Bb...), the difficulty delta is higher, because the accidental added is less and less common. The tone Bb is present in plenty of key signatures... the tone Db (for example) is present in very few of them.
That's me...
Answer: As a saxophone player (it's a monophonic instrument, so I am largely unaffected by chords, unless I am playing multiphonics or "virtual" chords in my head during solos), I found it slightly easier to play key signatures that have flats rather than sharps when I was a beginner, because of the mechanics of my instrument. To give you an example (pictures taken from The Woodwind Fingering Guide, I flipped them for your convenience, so that you can see which keys are higher and which are lower on the instrument when it is played; blacked-out keys on the diagram represent those that are pressed):
A4:
A♭4/G#4:
G4:
Explaining the example... If I am moving from A4 (a higher note in terms of sound) to A♭4 (a lower one), I have to press 2 more keys below A4, so my downward motion reflects the equivalent motion in sound. If I am going from G4 (a lower note) to G#4 (a higher one), I have to counterintuitively press 1 more key below G4, contrary to the motion in sound.
There are, naturally, some exceptions to this:

the more "unusual" notes like C♭4 (enharmonic of B4, upward motion on the instrument) and E#4 (enharmonic of F4, upward motion)
low-end notes requiring the left little finger (B♭3, B3 and C#3; the left hand is above the right on the saxophone, but it doesn't feel as much of an upward motion, because it's handled by the little finger when all the fingers on the right hand are already down, which makes it feel more like a "cherry on top" than an actual proper movement)
some upper-register and altissimo notes
some of the more unorthodox trill, alternative and microtonal fingerings

But, overall, as a saxophonist you quickly learn that playing a flat means going down the instrument, towards the bell, and playing sharps involves (at least for me) some mental gymnastics (they become less and less problematic with time) that translate sharps into flats. I still find it easier to play scales descending and play the occasional D4 instead of an F4 when I see an E#4 and am very tired/drunk. Your mileage may vary though.
I think that might be the case for many instruments that do not use a linear structure (like the piano) or a matrix (like guitar) to represent notes. To all oboe players out there: I can feel your pain.
P.S. My personal "favourite" on the saxophone is D#4 → E#4, which involves lowering/lifting 3 fingers to go down/up 1 tone, 1 "unnecessary" accidental (F natural is so much nicer than E#) and a note (D#) which likes to sound muffled/split in the 2nd octave of the instrument's "normal" range. If you are exhausted and trying to translate sharps into flats while playing a fast passage involving this switch, you can miss the timing or play something else note-wise entirely (if this switch is for the 3rd octave of the instrument's "normal" range, it becomes even more problematic and is considered one of the most difficult ones for the instrument, so you really don't want to play it with "unnecessary" sharps).
Update: this information is based on my personal experiences of playing "regular" B♭ and E♭ saxophones. There are horns out there that feature a linear fingering design.


How can I use environment variables in Nginx.conf

Question: I have a docker container running Nginx, that links to another docker container. The host name and IP address of the second container is loaded into the Nginx container as environment variables on startup, but is not know before then (it's dynamic). I want my nginx.conf to use these values - e.g.
upstream gunicorn {
    server $APP_HOST_NAME:$APP_HOST_PORT;
}

How can I get environment variables into the Nginx configuration on startup?
EDIT 1
This is the entire file, after the suggested answer below:
env APP_WEB_1_PORT_5000_TCP_ADDR;
# Nginx host configuration for django_app

# Django app is served by Gunicorn, running under port 5000 (via Foreman)
upstream gunicorn {
    server $ENV{"APP_WEB_1_PORT_5000_TCP_ADDR"}:5000;
}

server {
    listen 80;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    location /static/ {
        alias /app/static/;
    }
    location /media/ {
        alias /app/media/;
    }
    location / {
        proxy_pass http://gunicorn;
    }
}

Reloading nginx then errors:
$ nginx -s reload
nginx: [emerg] unknown directive "env" in /etc/nginx/sites-enabled/default:1

EDIT 2: more details
Current environment variables
root@87ede56e0b11:/# env | grep APP_WEB_1
APP_WEB_1_NAME=/furious_turing/app_web_1
APP_WEB_1_PORT=tcp://172.17.0.63:5000
APP_WEB_1_PORT_5000_TCP=tcp://172.17.0.63:5000
APP_WEB_1_PORT_5000_TCP_PROTO=tcp
APP_WEB_1_PORT_5000_TCP_PORT=5000
APP_WEB_1_PORT_5000_TCP_ADDR=172.17.0.63

Root nginx.conf:
root@87ede56e0b11:/# head /etc/nginx/nginx.conf
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
env APP_WEB_1_PORT_5000_TCP_ADDR;

Site nginx configuration:
root@87ede56e0b11:/# head /etc/nginx/sites-available/default
# Django app is served by Gunicorn, running under port 5000 (via Foreman)
upstream gunicorn {
    server $ENV{"APP_WEB_1_PORT_5000_TCP_ADDR"}:5000;
}

server {
    listen 80;

Reload nginx configuration:
root@87ede56e0b11:/# nginx -s reload
nginx: [emerg] directive "server" is not terminated by ";" in /etc/nginx/sites-enabled/default:3

Comment: This isn't a generic solution for environment variables, but if you're wanting to use environment variables for the hostnames/IP addresses of upstream servers, note that Docker (at least in recent versions) modifies /etc/hosts for you.  See https://docs.docker.com/userguide/dockerlinks/

This means, if your linked container is called 'app_web_1', docker will create a line in /etc/hosts in your Nginx container.  So you can just replace

`server $ENV{"APP_WEB_1_PORT_5000_TCP_ADDR"}:5000;`

with

`server app_web_1:5000;`
Comment: Thanks @mozz100 - that's incredibly useful - /etc/hosts entries are much more effective than env vars in this case. The only bit missing is what happens if the upstream container is restarted, and acquires a new IP. I am presuming that the child containers will still point to the original IP, not the new one?
Comment: Yes, if you restarted `app_web_1` it would get a new IP address, so you'd need to restart your nginx container too.  Docker would restart it with an updated `/etc/hosts` so you wouldn't need to alter the nginx config file(s).
Comment: Reading through the answers here, I am flabbergasted that something so basic hasn't been elegantly implemented by probably the most popular web server on the planet.
Answer: From the official Nginx docker file:

Using environment variables in nginx configuration:
Out-of-the-box, Nginx doesn't support using environment variables
  inside most configuration blocks.
But envsubst may be used as a
  workaround if you need to generate your nginx configuration
  dynamically before nginx starts.
Here is an example using docker-compose.yml:

image: nginx
volumes:
 - ./mysite.template:/etc/nginx/conf.d/mysite.template
ports:
 - "8080:80"
environment:
 - NGINX_HOST=foobar.com
 - NGINX_PORT=80
command: /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"

The mysite.template file may then contain variable references like
  this :

listen ${NGINX_PORT};

Update:
But you know this caused to its Nginx variables like this:
proxy_set_header        X-Forwarded-Host $host;

damaged to:
proxy_set_header        X-Forwarded-Host ;

So, to prevent that, i use this trick:
I have a script to run Nginx, that used on the docker-compose file as command option for Nginx server, i named it run_nginx.sh:
#!/usr/bin/env bash
export DOLLAR='$'
envsubst < nginx.conf.template > /etc/nginx/nginx.conf
nginx -g "daemon off;"

And because of defined new DOLLAR variable on run_nginx.sh script, now content of my nginx.conf.template file for Nginx itself variable is like this:
proxy_set_header        X-Forwarded-Host ${DOLLAR}host;

And for my defined variable is like this:
server_name  ${WEB_DOMAIN} www.${WEB_DOMAIN};

Also here, there is my real use case for that.
Comment: @shredding you can pass in variables names to be replaced - others are not touched: `command: /bin/bash -c "envsubst '$VAR1 $VAR2' < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"` works for me b/c I know what they are called ...
Comment: ok, forgot to escape the `$`, should be `command: /bin/bash -c "envsubst '\$VAR1 \$VAR2' < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"`
Comment: Escaping $ in docker-compose.yml should be done with another $. So, the command should look like this: `command: /bin/bash -c "envsubst '$${VAR1} $${VAR2}' < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"`
Comment: That kills nginx confs like `proxy_set_header Host $http_host;`
Comment: This is a great solution. Thanks. Also the link mentioned above https://github.com/docker-library/docs/issues/496 has a great solution for the issue.
Comment: With consideration for escaping this works in your own Dockerfile: `CMD ["/bin/sh","-c", "if [ -n \"${SOME_ENV}\" ]; then echo envsubst '${SOME_ENV}' with ${SOME_ENV} && envsubst '${SOME_ENV}' < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf; fi ; nginx -g 'daemon off;'"]`
Comment: [As per my answer](https://serverfault.com/a/1022249/184199), the official `nginx` image can handle `envsubstr` for you
Comment: @shredding could you elaborate? why?
Comment: @shredding ok, just ran into the same problem :( did you find a way around this? I opened an issue: https://github.com/docker-library/docs/issues/496
Comment: The problem is that envsubst thinks `$http_host` is an environment variable and replaces it with nothing. I haven't found a solution yet.
Comment: Is $VAR1 set in that context (e.g. within a `docker-compose.yml`)?
Comment: I love @pkyeck solution. /bin/bash -c "envsubst '$VAR1 $VAR2'
Comment: You actually have to copy template files to /etc/nginx/templates and then you don't need the command. That way your system variables won't be affected.
Answer: The official nginx image recommends using envsubst, but as pointed out by others it will replace also $host and other variables, which is not desirable. But fortunately envsubst can take as a parameter the names of variables to replace.
To avoid a very complex command parameter to the container (as in the linked example), you can write a Docker entrypoint script which will fill in the environment variables before executing the command. The entrypoint script is also a good place for validating the parameters and setting default values.
Here is an example of an nginx container which takes in API_HOST and API_PORT parameters as environment variables.
nginx-default.conf.template
resolver  127.0.0.11 valid=10s;  # recover from the backend's IP changing

server {
  listen  80;

  location / {
    root  /usr/share/nginx/html;
  }

  location /api {
    proxy_pass  http://${API_HOST}:${API_PORT};
    proxy_set_header  Host $http_host;
  }
}

docker-entrypoint.sh
#!/usr/bin/env sh
set -eu

envsubst '${API_HOST} ${API_PORT}' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf

exec "$@"

Dockerfile
FROM nginx:1.15-alpine

COPY nginx-default.conf.template /etc/nginx/conf.d/default.conf.template

COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]

Comment: I changed the script _slightly_ to clean up the template from conf.d. Final version:

```
#!/usr/bin/env sh
set -eu

envsubst '${API_HOST} ${API_PORT}' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf
rm /etc/nginx/conf.d/default.conf/template

exec "$@"
```

great script though thanks!
Comment: The official image now has one more environment variable to avoid replacing unwanted variables: `NGINX_ENVSUBST_FILTER`. It is a regex, and variables not matching it won't get replaced. You can set it to e.g. `API_` via `docker-compose.yml` or `Dockerfile` to leave $host and other variables intact. It eliminates the need for custom entrypoint script. See https://github.com/nginxinc/docker-nginx/pull/706 for more details.
Comment: im curious why nginx made this so complicated.
Answer: envsubstr is handled automatically by the nginx image now.
From the docs:

Out-of-the-box, nginx doesn't support environment variables inside most configuration blocks. But this image has a function, which will extract environment variables before nginx starts.

By default, this function reads template files in /etc/nginx/templates/*.template and outputs the result of executing envsubst to /etc/nginx/conf.d.
Comment: Docs are a bit sparse. The missing steps that got this working for me were: 1) creating a template, `default.conf.template` in a `templates/` directory 2) in my `docker-compose.yml` adding `volumes: - ./templates:/etc/nginx/templates` to copy the templates. Then just do `docker-compose up` and et voila.
Comment: As far as I can tell this only works if you have no `command`'s in your docker-compose. It seems those will prevent the `docker-entrypoint.sh` script from running. You can see that script if you exec into the running container. Posted about this [here](https://stackoverflow.com/questions/66732240/docker-compose-with-nginx-templates-to-pass-global-variables-adding-a-command-t).
Comment: @JohnL I didn't look thoroghly, but it seems as though this action is done via [a script included](https://github.com/nginxinc/docker-nginx/blob/master/stable/alpine/Dockerfile#L112) via the Dockerfile. You'll need to run that script to use that feature
Comment: This is great!  But, I don't see it happening. I put a `.template` file in `/etc/nginx/templates/`, but nothing is being written to `/etc/nginx/conf.d`. I am using the docker image `nginx:1.19.2`. I have a custom entrypoint script, but it ends up calling `nginx -g 'daemon off;'`.  Am I missing something?
Comment: Does this method require the official nginx docker image and all the files? I am trying this but its not working, My dockerfile is essentially
"FROM nginx:1.19, EXPOSE 80, CMD ["nginx", "-g", "daemon off;"]"

and my dir is just the dockerfile and a templates/default.conf.template and of course the docker-compose.yml filled from the official docs
Comment: @KentWong refer to https://serverfault.com/questions/577370/how-can-i-use-environment-variables-in-nginx-conf/1022249?noredirect=1#comment1342616_1022249 - there is a script included in the base image that must be run
Answer: Doing this with Lua is substantially easier than it sounds:
server {
    set_by_lua $server_name 'return os.getenv("NGINX_SERVERNAME")';
}

I found that here:
https://docs.apitools.com/blog/2014/07/02/using-environment-variables-in-nginx-conf.html
Edit:
Apparently this requires installing the lua module: https://github.com/openresty/lua-nginx-module
Edit 2:
Note that with this approach you have to define env variable in Nginx:
env ENVIRONMENT_VARIABLE_NAME

You have to do this in toplevel context in nginx.conf or it won't work! Not in server block or in config of some site in /etc/nginx/sites-available, because it is included by nginx.conf in http context (which is not top-level context).
Also note that with this approach if you try to make a redirect e.g.:
server {
    listen 80;
    server_name $server_name;
    return 301 https://$server_name$request_uri;
}

it won't work as well:
2016/08/30 14:49:35 [emerg] 1#0: the duplicate "server_name" variable in /etc/nginx/sites-enabled/default:8

And if you give a separate variable name to it:
set_by_lua $server_name_from_env 'return os.getenv("NGINX_SERVERNAME")';

server {
    listen 80;
    server_name $server_name_from_env;
    return 301 https://$server_name$request_uri;
}

nginx won't interpret it and will redirect you to https://%24server_name_from_env/.
Comment: you may need `env NGINX_SERVERNAME` somewhere in your nginx.conf.
Comment: @MarkWinterbottom didn't test this yet but it looks like you would not have to grant write access to the nginx config files, which you have to using `envsubst` and which is a no-go in my case
Comment: This did not work for me, although I have the lua module in my nginx docker image. Could this be related to the fact that I include a config file within my nginx.conf? I was trying to `set_by_lua` the variable in the included config file, while the `env MY_VAR` declaration was in the main nginx.conf, as suggested. What a shame, this would have been the cleanest solution!
Comment: Does anyone know the pros/cons to using this method of `envsubst`? I guess the pro is you don't need to run the `envsubstr` command before starting the server and the con is that you need to install the lua module? I wonder if there are any security implications on either approach?
Comment: When you say the `env` directive has to be done "in toplevel context in nginx.conf", what exactly is "nginx.conf"? Where should that file be?
Answer: Here another one with envsubst.
nginx docker image already runs envsubst through /docker-entrypoint.d/20-envsubst-on-templates.sh. You only need to place a template file in the right place: /etc/nginx/templates/my-file.conf.template.
Then, you can create a dedicated template file only for variables with one or more map clauses that we will use as our custom variables. You must choose a name for the conf.d file so that it is the first one that nginx takes.
This is the basic concept explained with docker-compose and three files:
docker-compose.yml:
version: "3.9"
services:
  nginx:
    image: nginx:1.23
    volumes:
        - ./template-variables:/etc/nginx/templates/10-variables.conf.template:ro

    environment:
        EXTERNAL_IP: "188.8.131.52"

template-variables:
map $host $external_ip {
  default "$EXTERNAL_IP";
}

nginx.conf:
server {
     location / {

          proxy_set_header X-Real-IP $external_ip;
      }
}

Comment: Marking this as the solution as although it's not the first to highlight the solution, it's the most useful as it contains all the various elements rather that pointing to another article.
Comment: why did you map $host to $external_ip I can see you are using $external_ip ion the other file but what is the purpose of $host?
Comment: PirateApp, It's not relevant to the subject. The point is to be able to use something similar to environment variables in nginx conf. But, to answer your question, this header allows a service behind a reverse proxy to know the real IP from the host instead of the IP in the docker internal network.
Comment: For this to work, make sure to keep `include /etc/nginx/conf.d/*.conf;` in `nginx.conf` - this is where the template gets written to.
Answer: I wrote something that may or may not be helpful: https://github.com/yawn/envplate
It inline edits configuration files with ${key} references to environment variables, optionally creating backups / logging what it does. It's written in Go and the resulting static binary can be simply downloaded from the release tab for Linux and MacOS.
It can also exec() processes, substitute defaults, logs and has sensible failure semantics.
Answer: What I did was to use the erb!
cat nginx.conf  | grep -i error_log

error_log <%= ENV["APP_ROOT"] %>/nginx/logs/error.log;

--After using erb
export APP_ROOT=/tmp

erb nginx.conf  | grep -i error_log

error_log /tmp/nginx/logs/error.log;

This is used in the Cloudfoundry staticfile-buildpack
Sample nginx configuration: https://github.com/cloudfoundry/staticfile-buildpack/blob/master/conf/nginx.conf
In your case
head /etc/nginx/nginx.conf
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
env APP_WEB_1_PORT_5000_TCP_ADDR;
upstream gunicorn {
    server $APP_HOST_NAME:$APP_HOST_PORT;
}

become
head /etc/nginx/nginx.conf
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
env <%= ENV["APP_WEB_1_PORT_5000_TCP_ADDR"] %>
upstream gunicorn {
    server <%= ENV["APP_HOST_NAME"] %>:<%= ENV["APP_HOST_PORT"] %>
}

#After applying erb

export APP_WEB_1_PORT_5000_TCP_ADDR=188.8.131.52
export APP_HOST_NAME=test
export APP_HOST_PORT=7089

erb /etc/nginx/nginx.conf

head /etc/nginx/nginx.conf
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
env 22.214.171.124
upstream gunicorn {
    server test: 7089
}

Answer: Referring to the answer on using erb, it can be done as below.
Write the NGINX config file as an erb file containing the environment variable, and evaluate it using the erb command to a normal config file.
erb nginx.conf.erb > nginx.conf

Inside the server block of the nginx.conf.erb file there could be
listen <%= ENV["PORT"] %>;

Comment: Do I need to install Ruby then, inside the container? (If not, how is `erb` able to do variable substitution ... after the container was started, right?) — `erb` is this Ruby stuff right: http://www.stuartellis.name/articles/erb/
Comment: It's a command line tool that comes with standard Ruby installation. Try other options if you don't have it in the container.
Answer: I accomplish this using a shell script.
Here is the nginx template:
server {

    listen 80;
    server_name ___MY_DOMAIN_NAME___;
    charset utf-8;

    location /proxy {
        proxy_pass http://___PROXY_IP___:___PROXY_PORT___;
        proxy_set_header Host            $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
    }

    location / {
        return         200;
    }

}

And the script to replace the environment variables is here:
echo sleep 3
sleep 3

echo build starting nginx config

echo replacing ___MY_DOMAIN_NAME___/$MY_DOMAIN_NAME
echo replacing ___PROXY_IP___/$LETSENCRYPT_IP
echo replacing ___PROXY_PORT___/$PROXY_PORT

sed -i "s/___MY_DOMAIN_NAME___/$MY_DOMAIN_NAME/g" /etc/nginx/nginx.conf
sed -i "s/___PROXY_IP___/$PROXY_IP/g" /etc/nginx/nginx.conf
sed -i "s/___PROXY_PORT___/$PROXY_PORT/g" /etc/nginx/nginx.conf

cat /etc/nginx/nginx.conf

if [ -z "$MY_DOMAIN_NAME" ]; then
    echo "Need to set MY_DOMAIN_NAME"
    exit 1
fi
if [ -z "$LETSENCRYPT_IP" ]; then
    echo "Need to set LETSENCRYPT_IP"
    exit 1
fi
if [ -z "$LETSENCRYPT_PORT" ]; then
    echo "Need to set LETSENCRYPT_PORT"
    exit 1
fi
if [ -z "$LETSENCRYPT_HTTPS_IP" ]; then
    echo "Need to set LETSENCRYPT_HTTPS_IP"
    exit 1
fi
if [ -z "$LETSENCRYPT_HTTPS_PORT" ]; then
    echo "Need to set LETSENCRYPT_HTTPS_PORT"
    exit 1
fi

nginx -g 'daemon off;'

Comment: Works perfect with docker-compose.
Answer: There are many ways. Some have been outlined in some answers.
If you use the ngx_http_js_module, there is also a way to do that with JS:
## /etc/nginx/fetch_env.js
function fetch_upstream_host(r) {
  return process.env.UPSTREAM_HOST;
}

function fetch_upstream_port(r) {
  return process.env.UPSTREAM_PORT;
}

and
## /etc/nginx/nginx.conf
load_module modules/ngx_http_js_module.so;

env UPSTREAM_HOST;
env UPSTREAM_PORT;

http {
  js_include fetch_env.js;
  js_set $upstream_host fetch_upstream_host;
  js_set $upstream_port fetch_upstream_port;

  server {
    ...

    location / {
      ...

      proxy_pass http://$upstream_host:$upstream_port;
    }
  }

Please make sure you use njs module 0.33 or newer
Comment: A neat approach, however limited in that nginx variables can only be used in run-time directives, during the processing of each request; http://nginx.org/en/docs/faq/variables_in_config.html. For example, the njs approach won't work for most config directives such as listen, server_name, etc.
Answer: Here's an example of using the sed approach, not necessarily better but it may be useful to some. First, add a custom keyword to be replaced within the conf file. Second, create a dockerfile that declares an ENV variable and then a CMD that uses sed to edit the config before running nginx explicitly.
So suppose your default.conf contains this with the keyword docker_host:
location /api { proxy_pass http://docker_host:9000/api; }

And write your Dockerfile similar to:
ENV docker_host localhost
ADD default.conf /etc/nginx/conf.d/default.conf
CMD sed -i.bak s/docker_host/$docker_host/g /etc/nginx/conf.d/default.conf &&   nginx -g "daemon off;"

Then build the image and run the container using
docker run -d -p 80:80 -e "docker_host=${env:COMPUTERNAME}" imagename

Comment: It could be stated slightly clearer `proxy_pass http://__DOCKER_HOST__:9000/api;`
Answer: I know this is an old question, but just in case someone stumbles upon this (as I now have), there's a much better way to do this.  Because docker inserts the linked container's alias in /etc/hosts, you can just do
upstream upstream_name {
    server docker_link_alias;
}

assuming your docker command is something like docker run --link othercontainer:docker_link_alias nginx_container.
Comment: You can get the host using this method but not the port. You have to either hard code the port or assume it's using port 80.
Answer: Like already explained in Martin's answer, the official Docker image supports parsing templates with envsubst nowadays. However, envsubst does not support default values like a regular shell does, as in ${MY_VAR:-My Default}.
To circumvent that, one can use a custom entrypoint, like docker-defaults.sh:
#!/usr/bin/env sh
set -eu

# As of version 1.19, the official Nginx Docker image supports templates with
# variable substitution. But that uses `envsubst`, which does not allow for
# defaults for missing variables. Here, first use the regular command shell
# to set the defaults:

export PROXY_API_DEST=${PROXY_API_DEST:-http://host.docker.internal:8000/api/}

echo "Will proxy requests for /api/* to ${PROXY_API_DEST}*"

# Next, let the original entry point do its work:

/docker-entrypoint.sh "$@"

Along with, say, some docker-nginx.conf:
# After variable substitution, this will replace /etc/nginx/conf.d/default.conf
server {
    listen 80;
    listen [::]:80;
    server_name localhost;

    location / {
        root /usr/share/nginx/html;
        # Forward all non-existing paths, such as /help and /file/..., to the app
        try_files $uri $uri/ /index.html;
    }

    location /api/ {
       # Proxy API calls to another destination; the default for the variable is
       # set in docker-defaults.sh
       proxy_pass $PROXY_API_DEST;
    }

    # Redirect server error pages to the static page /50x.html
    error_page 500 502 503 504  /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
}

In the Dockerfile copy it into /etc/nginx/templates/default.conf.template:
# Each time Nginx is started it will perform variable substition in the template
# file /etc/nginx/templates/default.conf.template and copy the result into
# /etc/nginx/conf.d/default.conf, so replacing the original configuration; see
# also https://hub.docker.com/_/nginx
COPY docker-nginx.conf /etc/nginx/templates/default.conf.template
COPY docker-defaults.sh /

# This will delegate to the original docker-entrypoint.sh
ENTRYPOINT ["/docker-defaults.sh"]

CMD ["nginx", "-g", "daemon off;"]

Now using, e.g., docker run --env PROXY_API_DEST=https://example.com/api/ ... will set a value, or in this example will default to http://host.docker.internal:8000/api/ if not set (which is actually  http://localhost:8000/api/ on the local machine).
Answer: nginx Docker image can extract environment variables before it starts, but it's a bit tricky. One solution is to:

Add env variables to your nginx.conf file.
Copy it to /etc/nginx/templates/nginx.conf.template in the container (as opposed to your normal /etc/nginx) in the build step or as a volume.
Set the NGINX_ENVSUBST_OUTPUT_DIR: /etc/nginx environment variable in docker-compose.yml.

This will cause the nginx.conf.template file to be copied to /etc/nginx as nginx.conf and the environment variables will be replaced with their values.
There is one caveat to keep in mind: using command property in docker-compose.yml seems to be disabling the extraction functionality. If you need to run a custom command to start-up nginx, you can use the Dockerfile version.
I created a repo with the full setup, but in case it's not available:
# docker-compose.yml

version: "3"
services:
  nginx-no-dockerfile:
    container_name: nginx-no-dockerfile
    image: nginx:1.23.1-alpine
    ports:
      - 8081:80
    volumes:
      - ./site/index.html:/usr/share/nginx/html/index.html
      - ./site/nginx.conf:/etc/nginx/templates/nginx.conf.template
    working_dir: /usr/share/nginx/html
    environment:
      NGINX_ENVSUBST_OUTPUT_DIR: /etc/nginx
      API_URL: http://example.com

  nginx-with-dockerfile:
    container_name: nginx-with-dockerfile
    build:
      context: ./site
      dockerfile: ./Dockerfile
    ports:
      - 8082:80
    volumes:
      - ./site/index.html:/usr/share/nginx/html/index.html
    environment:
      NGINX_ENVSUBST_OUTPUT_DIR: /etc/nginx
      API_URL: http://example.com

# site/nginx.conf

worker_processes auto;

events {
}

http {
  include /etc/nginx/mime.types;

  server {
    listen 80;

    root /usr/share/nginx/html;
    index index.html index.htm;

    location / {
      try_files $uri $uri/ /index.html;
    }

    location /example {
      proxy_pass $API_URL;
    }
  }
}

# site/Dockerfile

FROM nginx:1.23.1-alpine

WORKDIR /usr/share/nginx/html
COPY ./nginx.conf /etc/nginx/templates/nginx.conf.template

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Run no Dockerfile version: docker compose up nginx-no-dockerfile
Run Dockerfile version:
docker compose build nginx-with-dockerfile
docker compose up nginx-with-dockerfile

Make sure to also have index.html file in the site folder.
Answer: Another option... I just found this tool today: https://github.com/kreuzwerker/envplate ... Written in Go, it can be installed very easily. Using it is quite simple. Though you will need to place template variables in your nginx.conf.
For example ${SOME_ENV_VAR} will be replaced when envplate's ep command is called on the file. So should your Dockerfile fail to get that binary or should it not run for some reason, it would lave your config invalid. Just a little note compared to other solutions like using perl or lua extensions.
I really like how you can set default values too for when the environment variable is not set. Ex. ${SOME_ENV_VAR:default-value} (and you can escape values). Again, envplate must still successfully run.
One benefit of using an approach like this is that you don't end up with a Docker image that's larger than necessary because you went off installing all sorts of extra modules you don't otherwise need. It may also be easier than using sed if things start to get complex and it contains that default value functionality.
Comment: I advise https://github.com/gliderlabs/sigil as I ran into many bugs and problems with envplate .
Answer: Another possibility is to use the 'sed' command with regular expressions, then you won't have to mess with your config files at all!  That way you can use your config files normally, but when you run docker, it will swap the values out with the env variables.  None of this "add a string of text to your config files that you search and replace with."
You can create a run.sh file with replacement values using your environment variables.
To change the "7s" on this line:
client_body_timeout 7s;         #Default 60s

Sed Command using client_body_timeout as search line and $client_body_timeout as replacement env variable:
sed -i "s/\(client_body_timeout\).*\?\;/\1 $client_body_timeout;/" /usr/local/nginx/conf/nginx.conf

Copy/paste this line for every parameter you want to set and change the client_body_timeout with the config option and $client_body_timeout with the env variable it is associated with.  Use with existing config file and it will just work.
Answer: Proper way to do this with lua, since the answer above is stale:
server {
    set_by_lua $curr_server_name 'return os.getenv("NGINX_SERVERNAME")';
    server_name = $curr_server_name;
}

Answer: In addition to the lua and to the js method, there is a way with perl, especially, if nginx already compiled with --with-http_perl_module. Important parts are commented inline:
# declare variable
env ENDPOINT;

events {
    worker_connections 1024;
}

http {
    include mime.types;

    # store variable from env as nginx variable
    perl_set $endpoint 'sub { return $ENV{"ENDPOINT"}; }';

    server {
        listen 8080;
        root /usr/local/nginx;
        location / {
            sub_filter_once off;
            sub_filter_last_modified on;
            sub_filter_types application/javascript;

            # use variable, for example:
            sub_filter 'ENDPOINT' $endpoint;

            try_files $uri /testfile.js;
        }
    }
}

Be aware, that perl is a dynamic module and probably you need libnginx-mod-http-perl or nginx-extras to be installed:
apt-get install libnginx-mod-http-perl

Answer: I think everything is simpler here.
First of all, envsubst is better than lua block. envsubst creates configuration files before nginx starts and then all variables are static.
When using a lua block, each time a request is made, the block will be processed and the environment variables will be initialized each time,
and this is an additional burden.
But this doesn't matter if you have a workstation. At high loads I think this will matter.
Further, if you go a little deeper into the details, you can simplify everything. Environment variables can simply be included in a separate file, which is generated from the template.
In container /docker-entrypoint.d/20-envsubst-on-templates.sh script handles the templates.
All templates paths are saved in the created configuration files.
Example: /etc/nginx/templates/dir/my_vars.conf.template to /etc/nginx/dir/my_vars.conf
In *.conf.template templates you can include environment variables via the env directive - env VAR=${MY_VAR}.
Next, in any config file you include a variable file
include /etc/nginx/dir/my_vars.conf
I'll explain with an example.
First, let's copy the nginx configs from the container
docker run -d --rm --name test nginx:latest && docker cp test:/etc/nginx/ $(pwd)/ngninx_config ; docker stop test
($(pwd)/ngninx_config directory must not exist)
Next, let's launch the container for debugging
docker run \
-e NGINX_ENVSUBST_OUTPUT_DIR=/etc/nginx \
-e MY_VAR=Hello \
--mount type=bind,src=$(pwd)/ngninx_config,dst=/etc/nginx \
--name test_nginx \
-it --entrypoint bash --rm nginx:latest

NGINX_ENVSUBST_OUTPUT_DIR variable  see https://hub.docker.com/_/nginx => Using environment variables in nginx configuration (new in 1.19)
Сreate file
/etc/nginx/templates/env/nginx.conf.template
env VAR=${MY_VAR}

Change file /etc/nginx/nginx.conf
user  www-data;
worker_processes  auto;
include /etc/nginx/env/nginx.conf; # <===

...

run command in container
/docker-entrypoint.d/20-envsubst-on-templates.sh && nginx -t
Debug your settings until you succeed.
The env directive is used in the main block.
You can use the set directive. It is used in server, location, if blocks
(UPDATED)
Since setting environment variables in config files is a common difficulty (for example in mysql configs), I rewrote the /docker-entrypoint.d/20-envsubst-on-templates.sh file for general cases. The script in the templates replaces environment variables and creates config files.
https://gist.github.com/alexeyp0708/d244d0c6714d937bbcac55769c2bd392
See createFromTpl.sh -h command
Answer: If you need to keep it simple: bash eval sed
And follow two simple rules: ${ENV_IN_PARENTHESIS} and always use ' quotes in template
You can swap all environment variables in one generic call.
 eval "echo \"$(sed 's/\$/##/g; s/\##{/${/g;' some.template)\"" | sed 's/\##/$/g' > some.conf

Whats happening:
A. $(sed 's/\$/##/g; s/\##{/${/g;' some.template)
Run the template through sed and:
1. replace all $ characters with ##
2. replace all ##{ back to ${
B. eval "echo \"$(sed 's/\$/##/g; s/\##{/${/g;' some.template)\""
Runs the modified template through eval which replaces visible ${ENV_IN_PARENTHESIS}
C. ... | sed 's/\##/$/g' > some.conf
Pipes the result of the eval to another sed call that returns all original $ back from ## and output the final result to your shiny new conf file.
Answer: Taking up @omid's solution, here's a way of doing it without custom scripting and/or changing the default entry point script of the official Nginx docker image:
in Dockerfile:
FROM nginx

# override if necessary
ENV MY_VARIABLE='something'

# do NOT override this
ENV DOLLAR_SIGN='$'

COPY ./nginx.conf.template /etc/nginx/templates/default.conf.template

In the local nginx.conf.template file:
upstream upstream_server {
 server ${MY_VARIABLE}:8001 fail_timeout=0;
}

server {
 ...
 location / {
   proxy_set_header Host ${DOLLAR_SIGN}http_host;
   ....
   }
 }
}

Answer: Nginx 1.19 supports environment variables and templates in Docker
In this blog
Comment: Hello! Please post complete answers instead of links to blog posts. Links can rot or be not a direct answer to the posted question.
Answer: ex:
set_by_lua $curr_domain_name 'return os.getenv("DOMAIN")';
add_header Content-Security-Policy 'script-src  ${curr_domain_name}';

This worked for me.
Answer: If you're not tied to bare installation of nginx, you could use docker for the job.
For example nginx4docker implements a bunch of basic env variables that can be set through docker and you don't have to fiddle around with nginx basic templating and all it's drawbacks.
nginx4docker could also be extended with your custom env variables. only mount a file that lists all your env variables to docker ... --mount $(pwd)/CUSTOM_ENV:/ENV ...
When the worst case happens and you can't switch/user docker, a workaround maybe to set all nginx variables with their names (e.g. host="$host") in this case envsubst replaces $host with $host.
Answer: The offical Nginx Docker image has a hidden hook Supporting this.
See
https://email@example.com/nginx-docker-and-the-hidden-hooks-f5782261fae
Comment: Would be great to post the answer also here instead of only linking to an external resource.
Answer: For future reference, this is our local setup:
docker-compose.yml
# docker-compose.yml

version: "3.8"

services:
  # nginx is used to handle SSL termination. The upstream app location
  # is the Django app, and it can be set to run in docker or on the
  # host. It is set using the NGINX_UPSTREAM_APP env var which is
  # injected into the app.conf using the app.conf.template file.
  nginx:
    image: nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./docker/nginx/ssl:/etc/nginx/certs:ro
      - ./docker/nginx/templates:/etc/nginx/templates:ro
    environment:
      - NGINX_UPSTREAM_APP=${NGINX_UPSTREAM_APP:-django:5001}

app.conf.template
# Nginx host configuration for local development.
#
# The upstream app path is set dynamically when the nginx container is
# rebuilt, using a template and env var substitution.
#
# NB paths in this file are resolved inside the docker container.
upstream app {
    # host.docker.internal is the host machine
    server ${NGINX_UPSTREAM_APP} fail_timeout=0;
}

server {
    listen 80;
    listen 443 ssl;
    ssl_protocols             TLSv1.2 TLSv1.3;
    ssl_ciphers               HIGH:!ADH:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_certificate           /etc/nginx/certs/localhost.dev.crt;
    ssl_certificate_key       /etc/nginx/certs/localhost.dev.key;
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header Host $http_host;
        proxy_pass http://app;
    }
    access_log /var/log/nginx/access.log;
    error_log  /var/log/nginx/error.log;
}

Answer:
Create a template directory that should contain the .conf files (make sure the files have .template extension e.g default.conf.template)

mount the folder as a volume in the docker-compose file to /etc/nginx/templates (- ./templates:/etc/nginx/templates) ( see image below)

Define the .env variables keys in your docker-compose file without the values ( see image below)

run the docker compose up command and nginx will automatically create the conf files from the template directory before starting the engine.

Docker compose
Docker compose file
Default.conf.template
Default.conf.template
NB: No extra commands needed
Answer: According to @xyhhx and @ESV nginx can now handle ENV variables by creating a template, and placing it in directory defined in NGINX_ENVSUBST_TEMPLATE_DIR env variable.
TEMPLATE APPROACH - docker/nginx docs
However since approach with envsubst can still be a desired solution I've created easy approach that doesn't have drawback of having to modify variables in many places.
So if someone wants to create their own custom solution with envsubst, here's a concise example.
docker-compose
  nginx:
    image: nginx:stable-alpine
    env_file:
      - ./config/nginx/.nginx-env
    entrypoint: /etc/nginx/setup_conf.sh

entrypoint
#!/bin/sh

# LOAD ALL NGINX_* ENV VARS TO SHELL-FORMAT
envs=$(printenv | grep NGINX_ | awk -F '=' '{printf("$${%s} ", $1)}')

# LOAD TEMPLATE, INJECT ENV VARIABLES, OUTPUT CONF WITH INJECTED VARIABLES
envsubst "$envs" < /etc/nginx/template.conf > /etc/nginx/nginx.conf

# START NGINX IN THE BACKGROUND
nginx -g "daemon off;"

env file
NGINX_API_PATH=example.com

Answer: I will further expand David reply with a more complex example:
NGINX image do already have variable substitution; you can do it yourself with an envsub in your entrypoint (like i did before) or CMD but i do prefer using what it's already available.
I usually have an image that creates the right user and group (i usually run all my containers directly on 1000:1000 for security, with NGINX it's better to run the image as root to do variable replacing in config and make it read only for the user running NGINX) and various configurations with predefined config (ex. wordpress, laravel ,etc.) that i switch by changing the TEMPLATE env variable (ex. i set TEMPLATE="laravel" for laravel).
With the default NGINX image all the files ending with .conf.template under /etc/nginx/templates will be processed replacing variables and copied under /etc/nginx/conf.d/.conf; this works also with folders so /etc/nginx/templates/my-template.conf.template  will generate /etc/nginx/conf.d/my-template.conf and /etc/nginx/templates/my-folder/another-template.conf.template  will generate /etc/nginx/conf.d/my-folder/another-template.conf.
Let's start making a sample docker-compose file; this is just a sample to show where to mount the files and folders, you could also create a Dockerfile if you prefer
docker-compose.yml
version: "3.9"
services:
  nginx:
    image: nginx
    volumes:
        - ./templates:/etc/nginx/templates
        - ./nginx.conf:/etc/nginx/nginx.conf

    environment:
        USER_NAME: "user"
        USER_GROUP: "user"
        TEMPLATE: "my-template"
        MAX_BODY_SIZE: "1G"
        PROXY_TIMEOUT: "60"
        MY_VAR: "my-value"

We then create a nginx.conf file that will import all the configuration files in conf.d
nginx.conf
include /etc/nginx/conf.d/*.conf;

We then create a default config file that in our case only import the correct template based on the TEMPLATE env; this could not be done on nginx.conf.
templates/default.conf.template
include /etc/nginx/conf.d/templates/$TEMPLATE.conf;

then we will create our templates under /templates/templates
templates/templates/my-template.conf.template
user $USER_NAME $USER_GROUP;
events {
}
http {
    client_max_body_size $MAX_BODY_SIZE;
    fastcgi_read_timeout $PROXY_TIMEOUT;
    proxy_read_timeout $PROXY_TIMEOUT;
    #....
}

Answer: You should able to do what you want with Dockerize's templates.
Comment: Can you add some more details?


How to get "Show block on specific pages" working?

Question: I would like to show a certain block only on one specific page. I want to use the Show block on specific pages feature for it. I have tried to set the url of my page like content/hills-restoration-process  as well as the actual node number, like /node/36 but I don't seem to get it to work correctly. What am I doing wrong here? It seems like an easy and straight forward feature...
Thanks for hints and suggestions!
Comment: What is happening?  Is it showing on all pages or none?  Or something different?
Comment: Please check all block settings like region, content type(Not restricted), Roles(Not restricted).
Answer: When you use this field don't provide the leading slash.  Try node/36.
Not sure why the alias wouldn't work - perhaps there's a character out of place.  Node numbers work better because the block then doesn't disappear if the alias changes.
Answer: If this is a test site, first try making the block unrestricted.
If it does not appear, try assigning it to different content regions. If it appears in some but not others, this is probably a theming issue.
If it appears on every page as expected, you can then start applying your page restriction rule.
If this is a live site, follow the same approach but restrict the block to administrators until you have tracked down the problem.


Convert NSArray to Array in extension in Swift

Question: I am trying to make a Swift extension for NSArray that uses some Swift methods but am having trouble with the firsts step of creating a properly formatted Swift Array from the NSArray.
The incoming NSArray looks like
[@"Hello",@"Me",@"That",@"Me",@"Hello",@"Me",@"as",@"the"]

Here are different versions of the extension:
This version creates a valid array but because the array is an NSArray, not an array, fails on the line counts[item] with error

cannot subscript a value of type '[String : Int]' with an index of
  type 'Any'

public extension NSArray {
public var sortedArray: NSArray {
let array : NSArray = self
for item in array {
    counts[item] = (counts[item] ?? 0) + 1
    }
    print("counts",counts)

The next one works but is not taking the array as an input
public extension NSArray {
public var sortedArray: NSArray {
var array = ["Hello","Me","That","Me","Hello","Me","as","the"]
for item in array {
    counts[item] = (counts[item] ?? 0) + 1
    }
    print("counts",counts)

The following two methods compile but array2 and array3 end up being empty
public extension NSArray {
public var sortedArray: NSArray {
var array2: [String] = self.flatMap { $0 as? String } //This compiles but array is empty
var array3: [String] = self.compactMap({ $0 as? String }) //This compiles but
for item in array2 {
    counts[item] = (counts[item] ?? 0) + 1
    }
    print("counts",counts)//prints 0 as the array is empty

Note: when I insert a breakpoint at beginning of extension, self is a valid NSArray with lots of elements.
Thanks for any suggestions.
Comment: It might help if you provide your extension code.
Comment: What @nteissler said. Also, why are you messing with NSArrays in Swift. There should be no reason for that.
Comment: The reason for the NSArray is that most of the project is in Obj-C. However, what I want to do, create an an array of unique values sorted by incidence is easier in Swift. So I thought I would use a Swift extension.
Answer: It's worth noting that NSArray bridging has become very good in recent versions of swift. For example, all of the below works
let array: NSArray = NSArray(array: [3, 2, 1])
print(array)
print(type(of: array))
let newArray = array.compactMap { $0 as? Int }.compactMap(UInt8.init)
print(newArray)

print(array.sorted(by: { (left, right) -> Bool in
    guard let left = left as? Int, let right = right as? Int else {
        return false
    }
    return left < right
}))

I think NSArray gets compactMap and friends from conformace to the Sequence protocol. So the standard library is doing all it can to help you out here. Like @Rob mentioned, it's probably better to work with the bridging and not try to stay in NSArray-land


Does the ring of polynomial of infinitely many variables $\mathbb{R} [ x_1 , x_2 , \ldots ]$ have one maximal ideal or infinitely many?

Question: I know that $R:= \mathbb{R} [x_1 , x_2 , \ldots ]$ is non Noetherian, i.e. there is a increasing chain of ideals $\{ I_n \}_{ n \in \mathbb{N} }$such that $I_n \subsetneq I_{n+1}$ for all $n \in \mathbb{N}$ and furthermore, I know that given any ideal $I$, we can find a maximal ideal $M$ such that $I \subset M$.
Can I use this to answer this? Can someone drop me some hints?
Answer: Hint:  $\mathbb R[x_1]$ is a quotient, and the maximal ideals in this quotient correspond to some of those in your big ring.  Does $\mathbb R[x_1]$ have one maximal ideal or does it have infinitely many?
Comment: Thanks. An ideal generated irreducible polynomial in maximal in $\mathbb{R}[x_1]$ and there is a surjective homomorphism from $R$ to $\mathbb{R}[x_1]$.Since there are infinitely many irreducible polynomial in $R[x_1]$ , by the correspondence theorem, there must be infinitely many maximal ideals in $R$. Is this line of reasoning, correct?
Comment: @AshishK yes, well said.
Answer: There are infinitely many maximal ideals. Indeed, for any real number $r$, the ideal $(x_1+r,x_2,x_3,x_4,...)$ is maximal (of course, $r$ could be added to other variables also, while still yielding a maximal ideal).


CSS3 / Chrome: Anchors not clickable in CSS3 Cube

Question: Fiddle: http://jsfiddle.net/ab0Lf117/
I'm not sure why, but in Chrome I'm unable to click my anchors (2 and 4 are working for some reason), even though they're clearly there when inspecting.
I've tried using an  instead of a background image, I've also tried turning the whole face of the cube into an anchor instead of anchors within divs.
I'm guessing this is due to the transforms but I really haven't got a clue.
It works as expected in Firefox (Fiddle is a bit buggy)
Any help with this would be great as I'm well and truly stuck!
Comment: Ah, I see now.  Probably has to do with the z-index, or which display object is currently responding to events.  You might need to raise the z-index of the active span to a high value (e.g. 1000).
Comment: Works for me. Version 43.0.2357.130 (64-bit)
Comment: @PaulProgrammer I'm using V 43.0.2357.130 also, however, 32bit.  Do you have any ideas what the issue may be?  The cube "rotates" correctly, the links just don't seem to be clickable for me
Comment: @PaulProgrammer I've tried this too, to no avail :(
Answer: The problem is originated by the box element being perpendicular to  the view, so it is not "clickable" or "viewable", and then erroneously propagating this behaviour to the descendants.
This won't happen with an angle of almost - but not equal to - 90 deg
Set this
#box {
    position: relative;
    margin: 0 auto;
    height: 600px;
    width: 600px;
    -webkit-transform-style: preserve-3d;
    transform-style: preserve-3d;
    -webkit-transition: all 1s ease;
    transition: all 1s ease;
    -webkit-transform: rotateX(-89.99deg) rotateY(0deg);
    transform: rotateX(-89.99deg) rotateY(0deg);
    z-index: 10;
}

and this
$("#b1").click(function() { calcRotation(89.99, 0); });
$("#b2").click(function() { calcRotation(0, 0); });
$("#b3").click(function() { calcRotation(0, 89.99); });
$("#b4").click(function() { calcRotation(0, 180); });
$("#b5").click(function() { calcRotation(0, 269.99); });
$("#b6").click(function() { calcRotation(269.99, 180); });

and it will work.
I believe this is a new bug in Chrome, I am almost certain that in previous Chrome versions it worked ok (with the preserve-3d style applied)
fiddle
Comment: Thanks!  Much nicer than the fix I put in place!


Redimensionar Array Multidimensional/Associativo a partir de um ResultSet

Question: Estou tentando alimentar um array multidimensional e associativo a partir de um ResultSet, porém acusa o seguinte erro:
Microsoft VBScript compilation error '800a0401'
Expected end of statement
/beta/comercialBannersRelatorioGerado.asp, line 53
Redim preserve arrayRs("porIP", "mesPorIP") = rsRelatorio1("mesPorIP")
--------------------------------------------^

E aqui o código onde tento fazer isso:
Dim arrayRs()
while not rsRelatorio1.EOF
    Redim preserve arrayRs("porIP", "mesPorIP") = rsRelatorio1("mesPorIP")
    Redim preserve arrayRs("porIP", "viewsPorIP") = rsRelatorio1("viewsPorIP")
    Redim preserve arrayRs("porIP", "clickPorIP") = rsRelatorio1("clickPorIP")
rsRelatorio1.movenext
wend

Comment: http://meta.pt.stackoverflow.com/questions/297/quando-se-deve-colocar-o-nome-da-linguagem-no-t%C3%ADtulo/1911#1911
Comment: Alterado, obrigado!
Answer: O comando Redim não permite a atribuição de valor inicial.
A mensagem Expected end of statement informada no erro se refere que era esperado o fim da declaração no comando Redim após o ")".


How to pass struct with array attribute from Java to SQL stored procedure?

Question: I have some types in Oracle.
create or replace
TYPE r_telefone_cand AS OBJECT (
    candidato NUMBER(10,0),
    telefone VARCHAR2(15)
);

create or replace TYPE t_telefone_cand AS TABLE OF r_telefone_cand;

create or replace
TYPE r_candidato AS OBJECT (
    codigo NUMBER(10,0),
    rg VARCHAR2(15),
    cpf NUMBER(15,0),
    nome VARCHAR2(50),
    telefone t_telefone_cand,
);

And I have a procedure insert_candidato that parameter candidato is IN r_candidato
create or replace
PROCEDURE insert_candidato(
    candidato IN r_candidato
) AS ...

But I don't know how to pass the parameter candidato with the collection telefone.
Please, help me.
Answer: Give a try using JPublisher of Oracle.. !
http://download.oracle.com/docs/cd/B10500_01/java.920/a96658/intro.htm
It helps in interacting with user defined DB objects through Java.
Answer: You can use SQLData to map a java class to an oracle User Defined type. Check out the docs at http://download.oracle.com/javase/1.4.2/docs/api/java/sql/SQLData.html
In your case you'd need two types to match the db types.


Connected subgroup of a topological group.

Question: If $G$ is a connected topological group, and $H$ is a closed subgroup of $G$, such that
$G/H$ is simply connected, does it follow that $H$ is connected?
(In the particular class of cases I have in mind, $G$ is compact, and $G/H$ is homeomorphic to a convex set in $\mathbb{R}^n$. )
Answer: At least if $G, H$ are Lie groups, there is a long exact sequence
$$\dots \to \pi_1(G/H) \to \pi_0(H) \to \pi_0(G) \to \pi_0(G/H)$$
which shows that the answer is yes. In fact, the long exact sequence shows that if $G/H$ is contractible then the map $H \to G$ is a weak homotopy equivalence, and hence if $G, H$ are Lie groups it is a homotopy equivalence by Whitehead's theorem.


How to remove ".aspx" from url in asp.net c#?

Question: I want to remove ".aspx" from my web appliaction url. also I have using webservices.
If I use below code web services is not working.
Global.asax
protected void Application_BeginRequest(object sender, EventArgs e)
{
      String WebsiteURL = Request.Url.ToString();
      String[] SplitedURL = WebsiteURL.Split('/');
      String[] Temp = SplitedURL[SplitedURL.Length - 1].Split('.');

      // This is for aspx page
      if (!WebsiteURL.Contains(".aspx") && Temp.Length == 1)
      {
          if (!string.IsNullOrEmpty(Temp[0].Trim()))

              Context.RewritePath(Temp[0] + ".aspx");
      }
}

for Eg:-
Actual page is DEFAULT.aspx, but I want to show DEFAULT in address bar. So I used Global.asax to remove (.aspx). It's working fine. but Web service is not working(Default.asmx)
Comment: I'd consider using the [`URLRewrite`](http://www.iis.net/learn/extensions/url-rewrite-module) module for IIS.  Not necessarily the easiest to use, but very powerful and will do what I believe you want
Answer: There is a module that will handle this for you without having to directly manipulate the urls, as described here: http://www.hanselman.com/blog/IntroducingASPNETFriendlyUrlsCleanerURLsEasierRoutingAndMobileViewsForASPNETWebForms.aspx.

Install the package, Microsoft.AspNet.FriendlyUrls.
In your RouteConfig, the extensionless urls are enabled using:
routes.EnableFriendlyUrls();

You can generate friendly urls using extension methods, for example, to generate /Foo/bar/34, you can use:
<a href="<%: FriendlyUrl.Href("~/Foo", "bar", 34) %>">Click me


Erro na SQL QUERY

Question: SELECT COUNT(*) AS total
FROM table
GROUP BY (YEAR_MONTH FROM data)
ORDER BY (YEAR_MONTH FROM data) ASC

Está a dar erro no _YEAR_MONTH_ e não sei porque
Comment: Não está faltando a função EXTRACT? `EXTRACT(YEAR_MONTH FROM data)`
Comment: Esse SQL foi feito em qual banco? Coloque a mensagem de erro para facilitar a identificação do problema.
Answer: Pela sintaxe acredito que esteja usando mysql. Sendo assim, acredito que tenha esquecido o extract.
SELECT COUNT(*) AS total
FROM table
GROUP BY EXTRACT(YEAR_MONTH FROM data)
ORDER BY EXTRACT(YEAR_MONTH FROM data) ASC

Comment: Vi agora que [@EmanuelF](https://pt.stackoverflow.com/users/37939/emanuelf) comentou o mesmo que respondi aqui, então tenta isso mesmo.

Erro na SQL QUERY


System of divisibility constraints

Question: Let us consider integers $a,b$ and $c$ such that all of them are greater than 1. I am trying to figure out whether the following three divisibility conditions can be satisfied together

$a|b^{a-1}$
$ab|c$
${{ab}\choose{a}}|b^{a-1}c$

or, in other words, if we can find such integers. I started with the 3rd condition and plugged in a value for $a$, solved for $c$ and plugged in values for $b$ but $c$ turned out to be non-integer for the values I tried.
Answer: Correct me if I have misunderstood the question, but it seems quite straightforward when you observe the fact that $c$ never came in the divisor side. First, we can set $a=b=x$ so that the first condition holds. Next, we need $x^2 \mid c$ and $\binom{x^2}{x} \mid x^{x-1}c$. Both these conditions will trivially hold if $c=x^2 \cdot \binom{x^2}{x}$. Thus, such solutions exist and one example is:
$$(a,b,c)=\bigg(x,x,x^2 \cdot \binom{x^2}{x}\bigg)$$
It might be helpful to sometimes start from basic statements with few of the variables absent, rather than directly jumping to the crux of the arguments (3rd statement).
Comment: Examples are $(a,b,c)=(1,1,1),(2,2,24),(3,3,756)$ for $x=1,2,3$. If you want other examples than the general form suggested above, just follow the same trick and define $c$ at last.

System of divisibility constraints


Bootstrap navbar covers top of section when linked

Question: I've been working on my first website for learning purposes and run into this problem:
Whenever I link to a specific place on the same page it opens up as intended but the top is covered with navbar.

            <button type="button" data-target="#navbarCollapse" data-toggle="collapse" class="navbar-toggle">

            </button>

Home

About

Portfolio

Contact

      <un class="list-inline">

                Facebook

                GitHub

                LinkedIn

      </un>

Is there any way to make it link below the navbar so that it wouldn't cover the top of section?
Have tried playing with body's padding-top but to no use.
(I was able to solve this by linking to separate divs and adjusting them vie CSS, but after the implementation of bootstrap scrollspy divs that cover whole section had o be targeted, thus the problem returned.)
P.S.: no JS/Query has been used.
Comment: Share your code so that we can help you.
Answer: This might help or give you some ideas; you're essentially just using two navbars together under a navbar-fixed-top class so they're then together on scroll.

body {
  margin-top: 100px;
  position: relative;
}
.navbar.nav-lower {
  height: 20px;
  background: #f5f5f5;
  border-radius: 0;
  border: none;
  margin-bottom: 0;
}
.nav-lower .navbar-toggle,
.nav-lower .navbar-toggle:focus,
.nav-lower .navbar-toggle:hover {
  border: none;
  background: none;
}
.navbar.nav-lower > li {
  display: inline-block;
  width: 33%;
  text-align: center;
}
#section1 {
  padding-top: 100px;
  height: 500px;
  color: #fff;
  background-color: #1E88E5;
}
#section2 {
  padding-top: 100px;
  height: 500px;
  color: #fff;
  background-color: #673ab7;
}
#section3 {
  padding-top: 100px;
  height: 500px;
  color: #fff;
  background-color: #ff9800;
}
#section4 {
  padding-top: 100px;
  height: 500px;
  color: #fff;
  background-color: #00bcd4;
}
@media (max-width: 767px) {
  .nav-lower .navbar-collapse {
    text-align: left;
    background: #f5f5f5;
  }
}
@media (min-width: 767px) {
  .navbar.navbar-default.nav-lower .navbar-inner {
    padding: 0;
  }
  .navbar.navbar-default.nav-lower .nav {
    margin: 0;
    display: table;
    width: 100%;
  }
  .navbar.navbar-default.nav-lower .nav > li {
    display: table-cell;
    float: none;
  }
  .navbar.navbar-default.nav-lower .nav > li > a {
    text-align: center;
  }
}

<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="stylesheet" />

        <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false"> Toggle navigation

        </button>Brand

Home

About

Portfolio

Contact

        <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-lower-nav" aria-expanded="false"> Toggle navigation

        </button>

              Facebook

              GitHub

 LinkedIn

Section 1

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Section 2

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Section 3

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Section 4

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!

Try to scroll this section and look at the navigation bar while scrolling! Try to scroll this section and look at the navigation bar while scrolling!


Não consigo recuperar o JSON no jQuery

Question: Não estou conseguindo exibir o uf na seguinte situação:
$.post("<?php echo site_url('Welcome/viacep'); ?>",
    {cep: cep},
   function(dados){
       alert(dados.uf);
    },
    'json'
);

Mas quando eu mudo o alert para alert(dados), o JSON aparece completo.
O que será que eu estou fazendo errado?
Comment: como está retornando o seu json?
Comment: http://viacep.com.br/ws/30626620/json/{
  "cep": "30626-620",
  "logradouro": "Rua Joaquim Teixeira Dias",
  "complemento": "",
  "bairro": "Cardoso (Barreiro)",
  "localidade": "Belo Horizonte",
  "uf": "MG",
  "unidade": "",
  "ibge": "3106200",
  "gia": ""
}
Answer: Possivelmente tens de fazer parse do json:
$.post("<?php echo site_url('Welcome/viacep'); ?>", {cep: cep},
    function(response){
        var dados = JSON.parse(response);
        alert(dados.uf);
    }
, 'json' );

Comment: Quando eu chamo o JSON.parse, aí é que não funciona nada.
Comment: Meu json está chegando assim:
Comment: http://viacep.com.br/ws/30626620/json/{
  "cep": "30626-620",
  "logradouro": "Rua Joaquim Teixeira Dias",
  "complemento": "",
  "bairro": "Cardoso (Barreiro)",
  "localidade": "Belo Horizonte",
  "uf": "MG",
  "unidade": "",
  "ibge": "3106200",
  "gia": ""
}
Comment: aqui deu tudo certo.
Answer: Você precisa converter esse JSON que foi recebido como texto para um Object. Você pode usar o JSON.parse, do JavaScript nativo mesmo, da seguinte forma:
let obj = JSON.parse(dados);

A manipulação você usa o obj ao invés do dados.
alert(obj.cep);

O jQuery também tem uma função para deserializar JSON, $.parseJSON,  mas é considerada obsoleta, então você pode usar a do JavaScript puro como no exemplo dado anteriormente.

As of jQuery 3.0, $.parseJSON is deprecated. To parse JSON strings use
  the native JSON.parse method instead. Fonte
Answer: Tenta assim:
$.getJSON("//viacep.com.br/ws/"+ cep +"/json/?callback=?", function(dados) {

        if (!("erro" in dados)) {
            //Atualiza os campos com os valores da consulta.
            $("#logradouro").val(dados.logradouro);
            $("#bairro").val(dados.bairro);
        } //end if.
        else {
            //CEP pesquisado não foi encontrado.
            //limpa_formulário_cep();
            errosend("CEP não encontrado.");
        }
    });

Comment: Obrigado adventistaam! Funcionou perfeitamente.

Não consigo recuperar o JSON no jQuery


How do I tell what caused my Windows Server 2008 server to reboot?

Question: My server did a reboot today in the middle of the day, where should I look to see what may have caused this to happen?
It is a Windows Server 2008 machine
Answer: Definitely check the event log, but also depending on what brand of server it is, it may have its own internal logging system which is well worth checking.
For example, my HP servers have an inbuilt logging system that once told me a server rebooted in the middle of the night, and immediately before it did that there was a hard disk went bad and the RAID controller just was not happy and blue screened Windows. All I got from the Windows event log was "The last shutdown at 02:28 was unexpected".
Comment: +1 for proper server kit that tells you things like that. One more reason I facepalm when people say "Can't we just run this on an XP machine and save some money?"
Comment: Also worth checking that Windows Updates Settings are not set to automatically restart the server.
Comment: If you get an "unexpected restart" be srue to check the minidump folder, it may be worth running the debugger to see what caused the BSoD.
Answer: The Event Log - particularly the System log around the time the reboot happened.
You might find it was a BSOD (blue screen of death) - more information should have been logged to the event log.
Answer: I ran across a tool called "WhoCrashed" located at http://www.resplendence.com/downloads.
From the website:

Comprehensible crash dump analysis
  tool for Windows
  7/Vista/XP/2008/2003/2000 (x86 and
  x64)

This thing looks at your crash dump file to determine what might have caused the crash.


Command to insert/prefix characters in lines after searching for a phrase from a file

Question: Similar to a question posted by me here earlier, am looking for a help in Linux command that performs/does the following that is subtle different from my original question:
Searches for a particular word/phrase case-insensitively in a given file, and then insert/prefix with the character -- (in this case, this is an SQL comment) in the immediate next 'n' lines including the line where the word/phrase was matched in the given file.
EXAMPLE:
If I try to search for the phrase "CREATE FUNCTION plpgsql_call_handler" case-insensitively and if it matches at line no.102644, then I expect to insert/prefix the character -- in line no.102644 and the immediate next 2 lines in the given file.  In this case, I expect to insert/prefix line nos. 102644,102645,102646 with the character --.
Example file:
102644 CREATE FUNCTION plpgsql_call_handler() RETURNS language_handler
102645     AS '/usr/lib/pgsql/plpgsql.so', 'plpgsql_call_handler'
102646     LANGUAGE c;

I expect/want it to be changed into (in this case, it is an SQL comment):
102644 -- CREATE FUNCTION plpgsql_call_handler() RETURNS language_handler
102645 --    AS '/usr/lib/pgsql/plpgsql.so', 'plpgsql_call_handler'
102646 --    LANGUAGE c;

Answer: This is how you'd do it in-place with sed:
sed -r -i -e '/CREATE FUNCTION plpgsql_call_handler/I,+2 s/^/-- /'

(May require GNU sed.)
Replace 2 with the number of lines you want (or use any other sed address forms).
This is much simpler than that perl monstrosity. Also, perl could do it in place by using the -i as well:
perl -i -n -e horrible_perl_code /path/to/file

In both cases if you wish to keep a backup change -i to -i.bak.
Comment: Yes, it's working as I expected.  It's more easy & readable than `perl` syntax provided by @MadHatter.
Answer: And yet again, the ol' Swiss Army Chainsaw revs into life:
perl -n -e 'if (/create/i) { s/^(\d+)/\1 --/; print $_; $_=<> ; s/^(\d+)/\1 --/; print $_; $_=<> ; s/^(\d+)/\1 --/; print $_; } else { print $_; }' < /path/to/file

It's not as horrible as it looks!  perl -n -e does a loop around each line in the file, not implicitly printing the line, and running the little perl script supplied, on each line.
The little script supplied looks for a string (here it's create, but you can change it as you like) in each line, matching case-insensitively (/i).  When it finds the string, it:

replaces all the immediately-sequential digits at the beginning of a line ( ^(\d+) ) with themselves (\1) plus " --",
prints the modified line
reads the next line in ($_=<>)and repeats steps 1 and 2 on that line
reads the next line in and repeats steps 1 and 2 on that line

If the line doesn't contain that string, it prints it unmodified.
I'm sure this could be optimised further, but hopefully that'll do what you want.  It sends the modified file to STDOUT; capturing that to a temp file and moving it over the original is left as a (trivial) exercise for the OP!
Comment: @MadHatter: Could you provide `awk/sed` equivalent that does the same *in place*?
Comment: No.  awk won't do that, and sed's not my favourite tool.  Are you asking for that because this is a homework question, or do you have a genuine business need to do it in place which doing it to a temporary file and moving that into place won't satisfy?
Comment: @MadHatter: Only 2 reasons: 1) I thought awk/sed may have a simplified/ readable format. 2) In fact, I need to do this (modification) exercise in all our different onsite customer servers, so I'm seeing whether this could help me in making this operation in place (in the same input file passed), like the one I've asked before (on deleting).
Comment: With regard to point 2, I fail to understand the functional difference between operating on a file in place and operating on a file to a temporary copy and moving that copy into place, given appropriate checks on the success of each of the steps.  You have also been told in the comments to your earlier question (referenced above) that awk won't easily make changes *in situ*, either.  Perhaps someone with more sed experience can provide you with an incantation suitable for that tool, so we can compare complexity with regard to point 1?
Comment: @MadHatter: No issues.  I can follow your steps also.  But this is just for my understanding, can this not be performed *in place* using `perl`?
Comment: Not using the above script.  perl can, used correctly, do nearly anything; but I feel it would be much more complex to ( open up a file and insert characters into the middle of it ) than ( read a file through a filter, inserting additional characters as it goes ) + ( capture the modified stream and put it in place of the original file ).
Answer: AWK example:
awk 'tolower($0) ~ /create function plpgsql_call_handler/ {n=3;} // { if (n-->0) printf "-- "; print $0;}'


indenting specific parts of output in bash

Question: say I have the following bash script that finds all error.logs  gives them a header and then tails 5 lines below it..
#!/bin/bash
for ERR in `find -name "error.log"`;
do
    echo -e "---- LOG: $ERR ----";
    tail -n 5 $ERR;
done;

I would like to have it output like so:
---- LOG: foo/error.log ----
    [error ....]
    [error ....]
---- LOG: foo/error.log ----
    [error ....]
    [error ....]
---- LOG: foo/error.log ----
    [error ....]
    [error ....]

where the tail'd section is indented some amount of characters (e.g. 4 or whatever a tab character is)
How would I indent that specific section out output?
Answer: Pipe your tail command through sed:
tail -n 5 $ERR | sed -e 's/^/    /'

Comment: Also works with `cat`, and presumably anything else whose output can be piped through `sed`... this was just what I needed! :)
Answer: There are many ways.  One might be to simply pipe through sed or awk and add some space.
cat syslog | sed -e 's/^/    /'
#
cat syslog | sed -e 's/^/\t/'