Google OAuth 2.0 failing with Error 400: invalid_request for some client_id, but works well for others in the same project

Question: We have some apps (or maybe we should call them a handful of scripts) that use Google APIs to facilitate some administrative tasks. Recently, after making another client_id in the same project, I started getting an error message similar to the one described in localhost redirect_uri does not work for Google Oauth2 (results in 400: invalid_request error). I.e.,

Error 400: invalid_request
You can't sign in to this app because it doesn't comply with Google's
OAuth 2.0 policy for keeping apps secure.
You can let the app developer know that this app doesn't comply with
one or more Google validation rules.
Request details:
The content in this section has been provided by the app developer.
This content has not been reviewed or verified by Google.
If you're the app developer, make sure that these request details
comply with Google policies.
redirect_uri: urn:ietf:wg:oauth:2.0:oob

How do I get through this error? It is important to note that:

The OAuth consent screen for this project is marked as "Internal". Therefore any mentions of Google review of the project, or publishing status are irrelevant
I do have "Trust internal, domain-owned apps" enabled for the domain
Another client id in the same project works and there are no obvious differences between the client IDs - they are both "Desktop" type which only gives me a Client ID and Client secret that are different
This is a command line script, so I use the "copy/paste" verification method as documented here hence the urn:ietf:wg:oauth:2.0:oob redirect URI (copy/paste is the only friendly way to run this on a headless machine which has no browser).
I was able to reproduce the same problem in a dev domain. I have three client ids. The oldest one is from January 2021, another one from December 2021, and one I created today - March 2022. Of those, only the December 2021 works and lets me choose which account to authenticate with before it either accepts it or rejects it with "Error 403: org_internal" (this is expected). The other two give me an "Error 400: invalid_request" and do not even let me choose the "internal" account. Here are the URLs generated by my app (I use the ruby google client APIs) and the only difference between them is the client_id - January 2021, December 2021, March 2022.

Here is the part of the code around the authorization flow, and the URLs for the different client IDs are what was produced on the $stderr.puts url line. It is pretty much the same thing as documented in the official example here (version as of this writing).

OOB_URI = 'urn:ietf:wg:oauth:2.0:oob'

def user_credentials_for(scope, user_id = 'default')
    token_store = Google::Auth::Stores::FileTokenStore.new(:file => token_store_path)
    authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store)
    credentials = authorizer.get_credentials(user_id)
    if credentials.nil?
        url = authorizer.get_authorization_url(base_url: OOB_URI)
        $stderr.puts ""
        $stderr.puts "-----------------------------------------------"
        $stderr.puts "Requesting authorization for '#{user_id}'"
        $stderr.puts "Open the following URL in your browser and authorize the application."
        $stderr.puts url
        code = $stdin.readline.chomp
        $stderr.puts "-----------------------------------------------"
        credentials = authorizer.get_and_store_credentials_from_code(
            user_id: user_id, code: code, base_url: OOB_URI)
    end
    credentials
end

Comment: Please edit your question and include your code. I will ping someone at google
Comment: is your account a standard google account or a workspace account?  You can only set to internal if its workspace.   Can you double check that they are all desktop clients.
Comment: Yes, this is a workspace account.

The "December 2021" link in my answer is a real link generated by my app, and anyone can verify the `org_internal` error (which means that it works well).

Yes, I am looking at the client IDs right now and they are all "Desktop". For extra confirmation: The January 2021 client id is `566372898811-b89vfj17vaqpcd7bat5tdvbdcpbb41li.apps.googleusercontent.com`, it is Desktop, and gives the invalid_request error; December 2021 client id is `566372898811-htvr8md883tk25e1t8q7kabacpmh94dc.apps.googleusercontent.com`, it is Desktop and works.
Comment: I wonder if its related to this [manual-copypaste](https://developers.google.com/identity/protocols/oauth2/native-app#manual-copypaste)
Comment: @DaImTo The code is not really interesting, but I am adding it to the question. I see that there is some deprecation going on, which would mean that running such command line tools inside WSL or a headless Linux box may become... quite inconvenient.
Comment: Im going to email someone that might be able to clear this up.    This will not be optimal for my curl scripts either.
Answer: Please see https://stackoverflow.com/a/71491500/1213346 for a "proper" solution. This answer is just an ugly workaround that the community seems to like.
...
Here is a cringy workaround for this situation:
Replace urn:ietf:wg:oauth:2.0:oob with http://localhost:1/ in the code posted in the question. This makes the flow go through, my browser gets redirected and fails and I get an error messages like:
This site can't be reached

The webpage at http://localhost:1/oauth2callback?
code=4/a3MU9MlhWxit8P7N8QsGtT0ye8GJygOeCa3MU9MlhWxit8P7N8QsGtT0y
e8GJygOeC&scope=email%20profile%20https... might be temporarily
down or it may have moved permanently to a new web address.

ERR_UNSAFE_PORT

Now copy the code code value from the failing URL, paste it into the app, and voila... same as before :)
P.S. Here is the updated "working" version:

def user_credentials_for(scope, user_id = 'default')
    token_store = Google::Auth::Stores::FileTokenStore.new(:file => token_store_path)
    authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store, "http://localhost:1/")
    credentials = authorizer.get_credentials(user_id)
    if credentials.nil?
        url = authorizer.get_authorization_url
        $stderr.puts ""
        $stderr.puts "-----------------------------------------------"
        $stderr.puts "Requesting authorization for '#{user_id}'"
        $stderr.puts "Open the following URL in your browser and authorize the application."
        $stderr.puts url
        $stderr.puts
        $stderr.puts "At the end the browser will fail to connect to http://localhost:1/?code=SOMECODE&scope=..."
        $stderr.puts "Copy the value of SOMECODE from the address and paste it below"

        code = $stdin.readline.chomp
        $stderr.puts "-----------------------------------------------"
        credentials = authorizer.get_and_store_credentials_from_code(
            user_id: user_id, code: code)
    end
    credentials
end                                                                                ```

Comment: @DariuszDudziński this Ruby, but the only meaningful change is replacing `urn:ietf:wg:oauth:2.0:oob` with `http://localhost:1/`. Also changed the comments but these are cosmetic. If you have code that worked before, you can try changing the redirect URL to what I mentioned.
Comment: FYI for others: Note that you can't just change `redirect_uri` in the URL that was provided by the authenticator.  You have to set `http://localhost:1/` before making the request!  (We tried to just change redirect_uri in the url that had contained `urn:ietf:wg:oauth:2.0:oob` and Google didn't accept the "code".  I hacked my local copy of `googleads-php-lib-git/examples/Auth/GetRefreshToken.php` and set `const REDIRECT_URI = 'https://localhost:1';`, re-ran GetRefreshToken.php and it worked.)
Comment: Hi, what language is that? Do you have a similar snippet for Python?
Comment: can confirm this works. It sucks but if you need it for an internal quick project it is fine.

If anyone is using ruby its :
`authorizer.get_authorization_url(base_url: "http://localhost:1/" )`
Comment: I tried in Google Colab as follows: `credentials = flow.run_local_server(host="localhost", port=1,` and it didn't worked for me.
Answer: I sent off an email to someone on the Google OAuth team.  This is the gist of their response.
As I feared your issue is related to Making Google OAuth interactions safer by using more secure OAuth flows
The current recommendation from google is to move  to use localhost/loopback redirects as recommended here: instructions-oob or use the OAuth for devices flow if you are using non-sensitive scopes and need a headless solution.
Comment: OMG, `gcloud auth login` is not working anymore either. ♂
Comment: Thanks for digging this out. Too bad that I cannot use the device authentication flow as that seems to only support profile and drive scopes. Mighty nice of Google to not break the already existing applications :)
Comment: It also means to make it work as Google recommends : OAuth 2.0 Client IDs credentials you create to access application must be of type Web Application. Cause only then you can configure Authorised redirect URIs for that credential. You also must use that redirect URI you defined for any call to Google APIs. To make authentication seamless for desktop apps you need to run own server on that redirect URI as well to get the code sent by Google.
Comment: you can still use installed app the default redirect uri is http ://127.0.0.1
Comment: If you follow official documentation nothing will work for native apps described there: https://developers.google.com/identity/protocols/oauth2/native-app#request-parameter-redirect_uri . This answer pretty much updated official Google documentation. Thank you
Comment: I bet next step for Google is to oblige everyone to use SMS messages to get credentials for APIs calls :) Hope it never happens. Anyways I still must approve first call made with credentials on my phone which doesn't stop me from using API but it can I suppose block further calls if I answer that it wasn't me who made that API access call.
Comment: Just now tested AuthorizationCodeInstalledApp class in Java and it works pretty well but I had to still explicitly configure host and port for LocalServerReceiver, as well as allow it in  Allowed redirect URIs in https://console.cloud.google.com/apis/credentials. The last element is to use FileDataStoreFactory to persist credentials so that redirect URI can be skipped for subsequent APIs calls.
Comment: All of the client libraries should have bene updated prior to this change. if you have any issues make sure to post them over on the github issue forum for that library.
Comment: @DaImTo I guess Google then need to update an authorization docs for Google Ads API because they still refer the older `code` flow that results in the same error as described
https://developers.google.com/google-ads/api/docs/client-libs/python/oauth-desktop
Comment: its a process its going to take time for them to update all the samples.
Answer: A solution for python.
As google_auth_oauthlib shows, InstalledAppFlow.run_console has been deprecated after Feb 28, 2022. And if you are using google-ads-python, you can just replace flow.run_console() by flow.run_local_server().
Comment: I actually got it to work in colab by starting a server in python. `run_local_server()` still tries `localhost` but I was able to hack the URL at the end to point to the server running in my notebook.
Comment: @Fuhrmanator can you elaborate, please? - I'm trying using OAuth 2.0 in Google Colab, but, I'm unable to get it working. This is [my question](https://stackoverflow.com/q/72805453/12511801) describing my problem I'm currently facing in Google Colab and OAuth 2.0.
Comment: Indeed, this was my inspiration for the "proper" solution in https://stackoverflow.com/a/71491500/1213346
Answer: Let me post the "proper" solution as a separate answer, which is to actually follow the recommended procedure by implementing an HTTP listener in the ruby app. If this is running on an offline machine the listener will never get the code, but you can still paste the code from the failing URL.
require 'colorize'
require 'sinatra/base'

# A simplistic local server to receive authorization tokens from the browser
def run_local_server(authorizer, port, user_id)

    require 'thin'
    Thin::Logging.silent = true

    Thread.new {

        Thread.current[:server] = Sinatra.new do

            enable :quiet
            disable :logging
            set :port, port
            set :server, %w[ thin ]

            get "/" do
                request = Rack::Request.new env
                state = {
                    code:  request["code"],
                    error: request["error"],
                    scope: request["scope"]
                }
                raise Signet::AuthorizationError, ("Authorization error: %s" % [ state[:error] ] ) if state[:error]
                raise Signet::AuthorizationError, "Authorization code missing from the request" if state[:code].nil?
                credentials = authorizer.get_and_store_credentials_from_code(
                    user_id: user_id,
                    code: state[:code],
                    scope: state[:scope],
                )
                [
                    200,
                    { "Content-Type" => "text/plain" },
                    "All seems to be OK. You can close this window and press ENTER in the application to proceed.",
                ]
            end

        end
        Thread.current[:server].run!
    }

end

# Returns user credentials for the given scope. Requests authorization
# if requrired.
def user_credentials_for(scope, user_id = 'default')
        client_id = Google::Auth::ClientId.new(ENV['GOOGLE_CLIENT_ID'], ENV['GOOGLE_CLIENT_SECRET'])
        token_store = Google::Auth::Stores::FileTokenStore.new(:file => ENV['GOOGLE_CREDENTIAL_STORE'])
        port = 6969
    redirect_uri = "http://localhost:#{port}/"
    authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store, redirect_uri)

    credentials = authorizer.get_credentials(user_id)
    if credentials.nil? then
        server_thread = run_local_server(authorizer, port, user_id)
        url = authorizer.get_authorization_url
        $stderr.puts ""
        $stderr.puts "-----------------------------------------------"
        $stderr.puts "Requesting authorization for '#{user_id.yellow}'"
        $stderr.puts "Open the following URL in your browser and authorize the application."
        $stderr.puts
        $stderr.puts url.yellow.bold
        $stderr.puts
        $stderr.puts "⚠ If you are authorizing on a different machine, you will have to port-forward"
        $stderr.puts "so your browser can reach #{redirect_uri.yellow}"
        $stderr.puts
        $stderr.puts "⚠ If you get a " << "This site can't be reached".red << " error in the browser,"
        $stderr.puts "just copy the failing URL below. Copy the whole thing, starting with #{redirect_uri.yellow}."
        $stderr.puts "-----------------------------------------------"
        code = $stdin.readline.chomp
        server_thread[:server].stop!
        server_thread.join
        credentials = authorizer.get_credentials(user_id)
        # If the redirect failed, the user must have provided us with a code on their own
        if credentials.nil? then
            begin
                require 'uri'
                require 'cgi'
                code = CGI.parse(URI.parse(code).query)['code'][0]
            rescue StandardException
                # Noop, if we could not get a code out of the URL, maybe it was
                # not the URL but the actual code.
            end

            credentials = authorizer.get_and_store_credentials_from_code(
                user_id: user_id,
                code: code,
                scope: scope,
            )
        end
    end
    credentials
end

credentials = user_credentials_for(['https://www.googleapis.com/auth/drive.readonly'])

In short, we run a web server expecting the redirect from the browser. It takes the code the browser sent, or it takes the code pasted by the user.
Comment: @ZivKesten The colorize library monkey patches the string class. It adds color methods that will change its color on a terminal.
Comment: What is the 'yellow' method for user.id?
Answer: For headless Python scripts that need sensitive scopes, continuing to use run_console now produces the following (and the flow likely fails):
DeprecationWarning: New clients will be unable to use `InstalledAppFlow.run_console` starting on Feb 28, 2022. All clients will be unable to use this method starting on Oct 3, 2022. Use `InstalledAppFlow.run_local_server` instead. For details on the OOB flow deprecation, see https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html?m=1#disallowed-oob

The official solution is to migrate to a flow that spins up a local server to handle the OAuth redirect, but this will not work on remote headless systems.
The solution Google adopted in gcloud is to run a local server on the same machine as the user's browser and then have the user copy the redirect URL requested from this local server back to the remote machine. Note that this requires having gcloud installed both on the remote machine and on the user's workstation.
As a hack for situations where installing a script to echo back the redirect URL on the workstation is not practical, we can use a redirect URL that is guaranteed to fail and just have the user copy back the URL of the error page on which they will land after authorization is complete.
from urllib.parse import parse_qs, urlparse
from google_auth_oauthlib.flow import InstalledAppFlow

def run_console_hack(flow):
    flow.redirect_uri = 'http://localhost:1'
    auth_url, _ = flow.authorization_url()
    print(
        "Visit the following URL:",
        auth_url,
        "After granting permissions, you will be redirected to an error page",
        "Copy the URL of that error page (http://localhost:1/?state=...)",
        sep="\n"
    )
    redir_url = input("URL: ")
    code = parse_qs(urlparse(redir_url).query)['code'][0]
    flow.fetch_token(code=code)
    return flow.credentials

scopes = ['https://www.googleapis.com/auth/drive.file']
flow = InstalledAppFlow.from_client_secrets_file(secrets_file, scopes)
credentials = run_console_hack(flow)

We could also ask the user to pass back the code query string parameter directly but that is likely to be confusing and error-prone.
The use of 1 as the port number means that the request is guaranteed to fail, rather than potentially hit some service that happens to be running on that port. (e.g. Chrome will fail with ERR_UNSAFE_PORT without even trying to connect)
Comment: @GrishaLevit I cannot edit my original comment, but it was not working due to an error on my part. My issue was that I had `http://localhost` and not `https://localhost` in my authorized redirect URIs.
Comment: This is yet another cringy workaround which I would definitely not want to encourage when there are "proper" answers already: https://stackoverflow.com/a/71491500/1213346 for Ruby (since original question is about Ruby) and https://stackoverflow.com/a/71524639/1213346 for Python. P.S. Very creative that you parse the code from the URL.
Comment: I agree it's a crappy way to do it. But the local server approach requires that your browser redirect you to a server running on the same machine as the script, which is not possible in many scenarios.
Comment: This is exactly the situation I am in however, unfortunately this solution does not seem to work for me as of 07/30/22.
Comment: @CadeM what's not working about it for you? Note that you are expected to hit an error page after authorizing the request in your browser.
Answer: "Hello world" for this error:
Generating an authentication URL
https://github.com/googleapis/google-api-nodejs-client#generating-an-authentication-url
const {google} = require('googleapis');

const oauth2Client = new google.auth.OAuth2(
  YOUR_CLIENT_ID,
  YOUR_CLIENT_SECRET,
  YOUR_REDIRECT_URL
);

// generate a url that asks permissions for Blogger and Google Calendar scopes
const scopes = [
  'https://www.googleapis.com/auth/blogger',
  'https://www.googleapis.com/auth/calendar'
];

const url = oauth2Client.generateAuthUrl({
  // 'online' (default) or 'offline' (gets refresh_token)
  access_type: 'offline',

  // If you only need one scope you can pass it as a string
  scope: scopes
});

If something goes wrong the first step is to Re Check again the three values of the google.auth.OAuth2 function.
1 of 2
Compare to the store values under Google APIs console:

YOUR_CLIENT_ID
YOUR_CLIENT_SECRET

YOUR_REDIRECT_URL -

For example http://localhost:3000/login

2 of 2 (environment variables)
A lot of times the values store inside .env. So re-check the env and the output under your files - for example index.ts (Even use console.log).
.env
# Google Sign-In (OAuth)
G_CLIENT_ID=some_id_1234
G_CLIENT_SECRET=some_secret_1234
PUBLIC_URL=http://localhost:3000

index
const auth = new google.auth.OAuth2(
  process.env.G_CLIENT_ID,
  process.env.G_CLIENT_SECRET,
  `${process.env.PUBLIC_URL}/login`
);

SUM:
Something like this will not work
const oauth2Client = new google.auth.OAuth2(
  "no_such_id",
  "no_such_secret",
  "http://localhost:3000/i_forgot_to_Authorised_this_url"
);

Answer: I've fixed this problem with recreate my App in google console. And I think the problem was with redirect_url. I had this problem when I was using 'Android' type of App in google console (in this case you can't configure redirect url). In my android App I'm using google auth with WebView so the best option here use use 'Web' type for your app in google console.
Answer: In my case, had to update plugins. by running following command-
bundle exec fastlane update_plugins

With this redirect uri was getting created properly as
https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=563584335869-fgrhgmd47bqnekij5i8b5pr03ho849e6.apps.googleusercontent.com&include_granted_scopes=true&redirect_uri=http://localhost:8081&response_type=code&scope=https://www.googleapis.com/auth/cloud-platform&state=2ce8a59b2d403f3a89fa635402bfc5c4

Answer: steps.oauth.v2.invalid_request  400 This error name is used for multiple different kinds of errors, typically for missing or incorrect parameters sent in the request. If  is set to false, use fault variables (described below) to retrieve details about the error, such as the fault name and cause.

GenerateAccessToken GenerateAuthorizationCode
GenerateAccessTokenImplicitGrant
RefreshAccessToken

Google Oauth Policy
Comment: Welcome to stack please read [how-to-answer](https://stackoverflow.com/help/how-to-answer)  Im not sure how you think that answer would help it doesnt even address the fact that the author is using an internal app or that its desktop.   Not to mention your link is for Google cloud auth and not google apis auth.
Comment: As it's currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).


How can I create a content type in a custom module?

Question: I would like to create a content type called announcements for a module that I'm creating, but haven't found much to make this work. I tried this:
    function srl_dash_install() {
    //drupal_install_schema('srl_contactics_monthly_apps');
    $t = get_t(); // runs in both the installer and runtime

  /* Create and save a new content object */
  $announcement = 'announcement'; // machine name of the content type

  // define the node type
  $announcements = array(
    'type' => $announcement,
    'name' => $t('Announcement'),// Display name of the content type
    'base' => 'node_content',
    'announcement' => $t('Announcement'),
    'description' => $t('Add Contactics Announcements.'),
    'custom' => TRUE,
  );

  // set other node defaults not declared above
  $content_type = node_type_set_defaults($announcements);

  // add the body field
  node_add_body_field($content_type, $t('Body'));

  // save the content type
  node_type_save($content_type);
}

but I'm not sure how to make this hook run.
Any help would be appreciated.
Comment: Are you working in drupal 6 or 7?
Comment: hook_node_info()?
Answer: That hook is only run when the module is installed, maybe you added the hook during development of your module and when it was enabled?
The hook should be placed in a file called srl_dash.install.
Then disable your module and uninstall it, then enable it again. Now the hook will run.
Answer: If you are using Drupal 6 then please follow the link https://www.drupal.org/node/231019.
You have to properly place the functions in proper files.

In Module file create hook_node_info, hook_perm, hook_access.
To add new CCK fields to the newly created content type follow here.


How to put UILabel.textColor in Interface Builder

Question: I would like to put the font to my UILabel, i do it like this : myLabel.textColor = [UIColor colorWithRed:242.0/255 green:239.0/255 blue:236.0/255 alpha:1.0f]; My question is : how i can do exactly the same think with Interface Builder ? i know that i will use the inspector color in IB, but i don't know how to put the same think, for example how to put : red:242.0/255 in  IB, and the alpha :1.0f ? thanks for your answers
Answer: place a label in IB,
go to Attribute Inspector -> Text Color-> Other
In colors window, Go to Color Sliders (2nd tab on top)
Click on Grey Scale Slider and change it to RGB Sliders.
There you can put R/G/B values as you have done in code.
You dont need to put like: x/255
just set Red as : 242
Alpha can be set between 0 - 1 with Opacity slider at the bottom 0 - 100.
Comment: thanks, but if i put for example in IB : red :242 green:239 blue:236 and the opacité 100%, it gives not me the same think with [UIColor colorWith....]; thanks
Comment: it has to and it does give the same color - otherwise what is the use of providing RGB values input feature?


Is it ok to decrease the size of a dataset to speedup hyperparameters search?

Question: When training deep architectures, I want to be able to select hyperparameters in a feasible time. However, if my dataset is very big, this can be very time-consuming.
Is it OK to reduce the size of the dataset to check which hyperparameters perform better and then train the full dataset? Would this bias my hyperparameters selection since the training setups are not the same?
Answer: Depends. In your case, most probably yes
The hyper-parameters do not depend too strongly on the amount of data (as long as you have "enough") but on the complexity of the data (resp. its distribution). Reducing a dataset can mean to "change" the distribution of the data caused by statistical fluctuations.
On big datasets, you should be able to reduce the amount withouth real loss of the distribution shape. Try it! Use a classifier on the full dataset and one on the reduced. Are the metrics (Roc auc etc) the same? (maybe do that several times to get a good mean)
If they are, it should really not change the selection of the hyper-parameters.
Anyway, if you feel unsure whether the reduced dataset still has the same distribution, you may use it to get close to your best hyper-parameters and use the full distribution to find the best one in the end.
Comment: This is an interesting sub-optimization problem: what is an optimal subset to train the initial hyperparameters on?
Comment: I'm getting good results with a subset of size=0.1 in a fraction of training time (slightly better hyperparameter selection in a fraction of the time)
Comment: This really cannot be generalized in any way as it depends *only* on the size and the complexity of the data-set. Sometimes even your full data-set is way to small, sometimes a tiny fraction represents everything. Simple trial and error.


Calculating the CACE using instrumental variables

Question: In randomized trials with non-compliance among the treatment group, a common estimator is the Complier Average Causal Effect (also called the Local Average Treatment Effect), which (conditional on a number of assumptions) provides an unbiased estimate of the causal effect of the treatment. The idea is that both the intention-to-treat and per-protocol estimates of the treatment effect may be biased in the presence of non-compliance, so the CACE is, by my understanding, the causal effect among individuals who were randomized to the treatment and complied with it versus those individuals who would have complied with the treatment if they had been randomized to it.
A common approach to calculating the CACE is using instrumental variables methods, where randomization to treatment is the instrument. So, then, it should be possible to calculate the CACE using a two-stage regression approach, where the first stage we regress the instrument on the compliance indicator, and then we use the predicted values from that regression as a predictor in a regression on the outcome of interest. So, to use a very simplistic notation, the two-stage approach to calculating the CACE is then:
$$C_i = \gamma_0 + \gamma_1Trt_i + \gamma^T{X}$$
$$Y_i = \beta_0 + \beta_1\hat{C_i}+\beta^T{X}$$
where $C_i$ is an indicator for whether or not individual $i$ complied with the treatment, and $X$ is some vector of covariates, and $\beta_1$ should give you the CACE (I think? I am having a hard time following the literature on the subject, since it has a tendency to use notation and language I find rather esoteric).
I see this basic approach used and discussed frequently. However, what I don't understand, is that typically compliance is a "one-sided" issue ... that is, you only observe whether or not someone complies with the treatment if they are in the treatment group; in the control group, compliance status is unobserved.
So how can you even get any estimates from the first stage of the regression? $C_i$ is unobserved for members of the control group, so the only way to get any estimates to plug into the second stage of the regression is to assign them values for $C_i$, but I never see this done or discussed. Am I misunderstanding something about this approach?
Answer: This is an old question and I can't pretend to be able to help you with the maths. But, maybe this will help others looking to understand this conceptually...
The way I've seen CACE used runs like this:

Define compliers in the treatment group
Use wider data (that is available for the treatment and control arms) to find factors that determine compliance in the treatment group
Limit the control group using the wider factors that predict compliance
Compare between 'potential' compliers in the control group and actual compliers in the treatment group


Fotos não se alinham

Question: Estou desenvolvendo um app para absorver conhecimento e acabei com uma coisa meia estranha, é o seguinte a função do app é abrir uma lista de fotos (parseia uma xml, pega umas fotos da pasta assets) e aí quando ele vai amostrar a lista de fotos ele não fica um do lado do outro.
Classes
Comment: Davi, sempre adicione o código na pergunta, linkar ele em varios outros links, e ainda em site externo não é recomendavel no site, e ainda dificulta quem for te ajudar. leia [É uma boa idéia colocar o código no Pastebin ao invés de colocar o código na pergunta?](http://meta.pt.stackoverflow.com/q/4255/28595)
Comment: Obrigado pela dica
Answer: Tenta colocar dentro de um LinearLayout(horizontal), e verifica se o seu xml está LinearLayout ou RelativeLayout. Recomendo que user o RelativeLayout pois consegue trabalhar melhor.
Comment: Funcionou! obrigado!

Fotos não se alinham


Erro configuração de rotas no NodeJS com Express

Question: Estou com um problema na hora de iniciar o NodeJS, semelhante a este outro post
neste link, mas não teve mais feedback de quem perguntou e a solução apresentada na resposta não me serviu também.
O exemplo é de um livro, da Casa do Código, e são exemplos simples com erros de diretório mas que fazem me quebrar a cabeça pois estou começando com Node e Express.
Todo o exemplo foi utilizado o mesmo nome das variáveis do livro.
Eis o erro:
Error: Cannot find module './app/routes/home.js'
at Function.Module._resolveFilename (module.js:326:15)
at Function.Module._load (module.js:277:25)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (E:\projects\contatooh\config\express.js:2:12)
at Module._compile (module.js:398:26)
at Object.Module._extensions..js (module.js:405:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)

contatooh/config/express.js
var express = require('express');
var home = require('./app/routes/home')
module.exports = function() {
    var app = express();

    app.set('port', 3000);
    app.use(express.static('./public'));
    app.set('view engine', 'ejs');
    app.set('views','./app/views');

    home(app);

    return app;
};

contatooh/app/routes/home.js
var controller = require('./app/controllers/home');

module.exports = function(app) {
    app.get('/', controler.index);
    app.get('/index', controler.index);
}

UPDATE:
contatooh/app/controllers/home.js
module.exports = function(){
    var controller = {};
    controller.index = function(req, res){

        res.render('index'), {nome: 'Express'});
    };
    return controller;
}

Oraganização dos diretórios:
contatooh
    app
        controllers
            home.js
        models
        routes
            home.js
        views
            index.ejs
    config
        express.js
    node_modules
        ...
    public

    server.js
    package.json

Answer: Quando fazes . estás dizendo "nesta atual pasta". Por exemplo, fazendo ./app/controller/home, o que está na verdade a tentar importar é um file que se chama home que está situado numa pasta (ou repositório) que se chama controller que é uma sub-pasta da pasta app, que por sua vez se encontra na pasta donde estás a tentar importar. Mas acho que não é exatamente isso que querias fazer.
Se te encontras por exemplo no file contatooh/app/routes/home.js e queres importar app/controller/home (suponho que a pasta app seja a mesma nos dois casos), o que tens que fazer é fazer um "import relativo":
var controller = require("../controller/home");

.. significa a pasta mãe (ou anterior) da pasta onde atualmente encontras-te, neste caso essa pasta é app, porque te encontras em routes. Depois disso só tens que especificar o path partindo de app. Se queres ir duas pastas para atras, só tens que fazer ../.., e similmente se queres ir 3 ou 4, etc.
Penso que seja esse o teu problema, mesmo não tendo a certeza absoluta, porque não estou percebendo exatamente qual é a estrutura da tua aplicação.
Comment: Vou tentar alterar o caminho das pastas, retorno depois se funcionar
Comment: Fiz um update do post e do código.
Alterei a linha do 'express.js' de: `var home = require('./app/routes/home')`
para `var home = require('../app/routes/home');` . E também a do 'routes/home.js' de: `var controller = require('./app/controllers/home');` para `var controller = require('../controllers/home');` e parou de dar o erro de diretório. Valeu.


Would Xamarin help in porting win32 DLL to android shared library?

Question: I am working on a project in which I need to include NGSpice simulation library in an Android application and of course be able to use it.
I need to port the NGSpice windows DLL to my android application, how far would xamarin help me in this task? its a win32 dll, its not .Net based dll, please have a look at NGSpice link
Thanks.
Comment: I think, as long as the compile target is arm it should work
Comment: possible duplicate of [Can a shared library (.so) file built for linux be included/linked and used in an Android application?](http://stackoverflow.com/questions/19280391/can-a-shared-library-so-file-built-for-linux-be-included-linked-and-used-in-a)
Comment: The windows dll will not work. Since android is arm platform you need to compile the source for arm.
Comment: I am asking about using xamarin, would it be the solution? my project has thousands of files scattered into folders
Comment: No, your problem has nothing to do with Xamarin.  Xamarin helps a lot if you have existing .Net managed code (F#, C#, etc), but doesn't do anything special for C/C++.
Answer: Yes that is possible using the Android NDK.
Since reading through all the docs can be somewhat overwhelming at first, I'll give you a quick tour.

Install the NDK
Create a folder named jni inside your project directory
Inside that folder create a subfolder lib and copy the required source files of your library to that folder
Inside the jni folder create two files, wrapper.c and wrapper.h. These files should contain high level wrappers around your native library suitable for binding from the C# side using DllImport
Inside the jni folder create a file named Android.mk
Inside the jni folder create a file named Application.mk
run ndk-build
Once ndk-build completes successfully create subfolder lib inside your C# project directory and copy the generated native libraries including their surrounding folder to lib. Your folder structure should look like this: <project> -> libs -> armeabi and x86 (both folders contain an architure specific .so file). By establishing this folder structure, we ensure that MonoDroid will automatically load the matching library for the current runtime architecture.
Create new C# Class file Foo.cs inside your C# project. This class implements the binding glue between your native library wrapper and the C# world.
Profit!

Android.mk:
LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE    := libFoo
LOCAL_SRC_FILES := wrapper.c lib/source1.c lib/sourcen.c

include $(BUILD_SHARED_LIBRARY)

Note: lib/source1.c and lib/sourcen.c need to be substituted by the names of your actually library source files.
Application.mk:
APP_ABI := armeabi armeabi-v7a x86

This line tells ndk-build which architectures you wish to support. x86 is for the Intel Android HAXM Simulator.
wrapper.h:
#pragma once

typedef struct _InternalState
{
    FILE *m_pFile;
} InternalState;

//////////////////////
// Public API

#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */

InternalState *FooCreate(int foo);
void FooDoStuff(InternalState *pState);

#ifdef __cplusplus
}
#endif /* __cplusplus */

wrapper.c:
//////////////////////
// Public API

InternalState *FooCreate(int foo)
{
  // Initialize actual library here
  // return state if necessary
}

void FooDoStuff(InternalState *pState)
{
  // Call actual library here
}

Foo.cs:
public class Foo :
  IDisposable
{
  private IntPtr handle;
  private bool disposed;

  private Foo(IntPtr handle)
  {
    this.handle = handle;
  }

  ~Foo()
  {
    Dispose(false);
  }

  public void Dispose()
  {
    Dispose(true);
    GC.SuppressFinalize(this);
  }

  public void Dispose(bool disposing)
  {
    disposed = true;

    Close();
  }

  #region Native Imports

  [DllImport("libFoo")]
  private static extern IntPtr FooCreate(int seed);
  [DllImport("libFoo")]
  private static extern void FooDoStuff(IntPtr pHandle);

  #endregion

  private void VerifyHandle()
  {
    if (handle == IntPtr.Zero)
      throw new InvalidOperationException("Handle not initialized");
  }

  #region Public API

  public static Foo FromSeed(int seed)
  {
    IntPtr handle = FooCreate(seed);
    if(handle == IntPtr.Zero)
      throw new System.Exception("Failed to create native object");

    return new Foo(handle);
  }

  public void DoStuff()
  {
    VerifyHandle();

    FooDoStuff(handle);
  }

  #endregion
}

Comment: Thanks, but I am asking about xamarin tool, I don't want to get into the process of fixing errors .. especially those that are related to windows ..
Comment: There's no tool for this.
Comment: Actually, I was asking about this http://xamarin.com/ , would it really solve my problem?
Comment: According to the NGSpice documentation, the library is cross platform. So the steps outlined in my post should be applicable. But you don't really need MonoDroid unless you want to consume that library in C# _on_ Android.


Hypothesis Testing: Z-test or T-test? and how to test the null hypothesis?

Question: I have a question as below:
I have to generate two portfolio returns in R and compare their means.
However, I am not sure which test should be applied in this case. And how can I indicate at least 2% higher statement in the null hypothesis. I wrote this code:
<code>asset1 <- rnorm(100, 0, 1)
asset2 <- rnorm(100, 0.1, 1)
z.test(asset1, asset2, mu = 0.02, alternative = "greater", sigma.x = 1, sigma.y = 1)
</code>
I am not so sure about this code because I wrote mu = 0.02 but there should be an appropriate way to show the difference in means is at least higher than 2% of the mean of the first population. Furthermore, since I thought that I have the true population parameters here, I applied Z test and set sigma.x and sigma.y to 1, but is this the correct way to proceed?
Comment: Welcome to Cross Validated! Please add the self-study tag (in place of mathematical statistics) and read its wiki. A good place to start would be saying why you selected the parameters you did in z.test and why you're not sure about them.
Comment: I also recommend putting set.seed(2020) at the top of your code so that you can reproduce your results.
Comment: Thank you. I added extra information regarding your suggestions.
Comment: As a final clarification, are the "returns" supposed to be percent return and you're just supposed to test if $\mu_1-\mu_2>0.02?$
Comment: Yes, that was what I understood.
Answer: I would use the unequal variance t-test (default t.test in R) here, and I think the full-credit answer would go through why you elected against z-testing and equal-variance t-testing even though you know the variances and know that they're equal.
The reason why I'd go with t over z is that, yes, you know the population standard deviations, but you also know the population means! If you're going to take the simulation parameters as known, there's no hypothesis testing to do. You know, with certainty, that the first asset has lower average returns than the second. If you don't know that, then you shouldn't be able to say that you know the standard deviations.
Put another way, if your teacher just gave you two data sets that she generated form your code, but you did not know the generation process, you would proceed with an unequal-variance t-test (aka Welch's t-test).
Comment: Thank you! That made sense now.


Scale degree in minor key

Question: I am just beginning my ear training and I have a question about hearing minor scales.
Music theory tells me that minor scale degrees work similarly to major scale degrees. For example in the key of Am the subdominant chord would be Dm and the dominant chord would be Em (or E).
But quite often I stumble upon songs that tend to use the degrees of the relative major key.
For example, a song in Am would not use Dm or Em at all, but instead have some F and G chords that seem to function as subdominant and dominant (like they would in C, the relative major key)
Is such a song really more of a major song that uses its relative minor as a replacement for the tonic? Or what is going on here?
Comment: Could you link to an example, please?
Comment: @aparente001 A popular example would be Zombie by the Cranberries. Disarm by the Smashing Pumpkins seems to do a similar thing.
Answer: Subdominant and dominant are tonal music terms, they may or may not make sense in a modal context. In tonal music (like common practice era classical music) you are hard pressed to find an Em functioning as dominant for Am. E is used almost exclusively.
Today's popular music has more modal roots. A (non-raised) seventh degree major chord (G in Am) is a very popular way to connect to the tonic. I'm not sure whether labeling it as a dominant is 100% correct though; it doesn't have the strong tendency for resolution that E or E7 has. The lack of this strong tendency for resolution is what distinguishes aeolic (modal) minor from tonal (melodic/harmonic) minor.
F is the relative major of subdominant Dm. It's used quite often as a subdominant substitute in tonal music too. F -> E -> Am is a common cadence. F7 -> E7 -> Am (actually it's not F7, it's an augmented sixth chord) is even more common. So there's nothing surprising for it to be used for a similar function in popular music.
Comment: Interesting how different languages put different spins on concepts.
Comment: F is relative rather than parallel. Dmaj is parallel of Dmin.
Comment: @Tim You're right. I'll fix it. I've studied harmony from Turkish translations of German books. Here they call "relative" "parallel" and "parallel" "namesake". Sometimes I screw up when translating :)
Comment: Interesting and [confusing](http://en.wikipedia.org/wiki/Relative_key#Terminology)
Answer: Maybe it comes down to the Human predilection to label everything. In, say, Cmaj., all the white notes( aka keys) on a keyboard can be used; n Amin., the same. They're the diatonic notes. So if a melody is only using those white notes, any chords produced from their combinations will fit the melody at some point. When the melody seems to gravitate more towards the C, we like to say it's IN C, and vice versa. Sometimes fortified by the use of the raised leading tone, G#.
In Cmaj, there are 3 minor harmonies which are commonplace, and in Amin there are 3 major harmonies. So moving from one to another is very natural.A mild modulation may be construed if a tune in Cmaj has a few bars consecutively in using Am, Dm and Em, but has the key actually changed?
Look at 'Fly Me to the Moon. It starts on Am, but ends up on Cmaj. So which key is it in? It uses most of the chords from both keys, so to speak, so it could be argued that it's in Amin because there's a dominant E to get there, or it's in Cmaj because it finishes on that chord.
And that's all before we start considering the parallel minor...
Answer: There are three basic functions a chord can have in a song: Tonic Function, Subdominant function, or Dominant Function. They get their functions from their numerical position in a key.

Tonic Function (stability) I, III, VI
Subdominant Function (contrast) II, IV
Dominant Function (tension) V, VII

This works in Major or Minor, A III chord and a bIII chord are both tonic function chords. In Rock and Blues I, IV, V are the three most important, in Jazz I, II, and V are.
The bVII chord (G or G7 in A minor) does create tension that can be resoved by whole-step to the tonic, but its not as strong as V to I.
The bVI chord (the F in A minor) is normally a Tonic function chord, however you are correct it can take on the Subdominant function, usually when moving to the V (E). The easiest way to wrap your head around it is to think of it as a tritone substitution for the Subdominant II chord.
Given a basic progression you can create some variation in the chord progression for swapping out a chord that has the same function. The reason this works is because they share many of the same notes, and this process is often called Common Tone Substitution for that reason.
It is common in minor harmony to change the qualities of the chords as well. The normally minor V- chord is almost always changed to a dominant V7 or major to create stronger resolution to the I-. It is also common to change all the chords except the root chord to Major chords. House of the Rising Sun is a good example. You can also change all the chords to Major chords, from the perspective of popular music, it remains minor as long as the roots of the chords in the song, together add up to a minor scale. This is how rock and metal songs played only with power chords manage to be Major or Minor.
Here it is laid out in a table on my website:
Chord Functions and the Uppercase Roman Numeral System
Comment: I'm not sure this really address the question about minor keys using VI and VII chords to function as the IV and V chords from the relative major.
Comment: I read the question differently. "Is such a song really more of a major song that uses its relative minor as a replacement for the tonic" and asking about the G used in a dominant function, it seemed to me he was asking what chords can operate in which function in minor. *" I'm not sure whether labeling it as a dominant is 100% correct though"* I think I'm more along the lines of the text of the question, although you may know the poster better. You are correct I did forget to mention the use of the bVI chord as a subdominant function chord in minor, I will correct that.

Scale degree in minor key


What is the standard method for writing a continuously running app that is allowed on the Mac App Store

Question: I'm brand new to OS X and mac programming.  I'm looking to write an app that runs continuously or is woken up every 30 seconds to do some minor task.  It would do something like check the weather via a web service and shoot off an email if the temperature drops below freezing.  In Windows I would just use a service or put a repeating task in the task scheduler.  Is there a standard way to accomplish this in OS X which the mac app store allows?  I would like to write something that I can put in the store but Apple seems to have vague restrictions against running background processes.
I've learned a little about launchd but I can't tell if I am allowed to use it.  Ideally the process would get run even when no one is logged in, but it probably won't need root access.  If someone does happen to be logged in, it would trigger some popup if it sent an email temperature alert out.  I would also have some GUI which the user would be able to modify the configuration files that determine the behavior of the background process. I don't anticipate any app store restriction for that piece of the project though.
Answer: I'm pretty sure an app that runs invisibly like that would not be allowed into the App Store.  What you're going to want is to create a menubar-only application, which can be enabled by setting a key in your app's info, namely LSUIElement.  Although it won't be able to be run unless a user is signed in, it's your only way to get it into the store successfully.
Answer: The MAS guidelines have several prohibitions that would impact you. Go to the guidelines and read section 2. In particularly consider 2.15, 2.23, and 2.27. I don't know any way to get your application running prior to login without at some point requesting admin privileges (even if you don't use the privilege when you run).
I would probably make it a Login Item (System Preferences>Users>Login Items), probably as an LSUIElement app as sudo rm -rf suggests. Definitely if it's your first app, I wouldn't go diving into launchd in any case. It is one of the most infuriating system processes I've ever dealt with.


How to customise extjs remote extensible calendar example with MVC application?

Question: I have downloaded 'extensible-1.6.0-b1', I'm trying to understand the remote example to customise it with my application. I'm coding with extjs 4.0.7.
I want to know how to intgrate the example in mvc application? Is there an example with clear architecture: store+model+controller?
Edit:
This is the code I'm using now:
Ext.define('Webdesktop.view.calendar.Calendar', {
extend : 'Ext.tab.Panel',
alias : 'widget.calendar_calendar',
//autoShow : true,
paths: {
    'Extensible': 'extensible-1.6.0-b1/src',
    'Extensible.example': 'extensible-1.6.0-b1/examples'
},
requires:([
    'Ext.Viewport',
    'Ext.layout.container.Border',
    'Ext.data.proxy.Rest',
    'Extensible.calendar.data.MemoryCalendarStore',
    'Extensible.calendar.data.EventStore',
    'Extensible.calendar.CalendarPanel'
    ]),
initComponent: function() {
    var me = this;
    var calendarStore = Ext.create('Extensible.calendar.data.MemoryCalendarStore', {
        autoLoad: true,
        proxy: {
            type: 'ajax',
            url: 'app/data/calendars.json',
            noCache: false,
            reader: {
                type: 'json',
                root: 'calendars'
            }
        }
    });

    var eventStore = new Ext.data.JsonStore({
        autoLoad: true,
        fields: [
         {name: 'EventId', mapping:'id', type:'int'},
         {name: 'CalendarId', mapping: 'cid', type: 'int'},
         {name: 'Title', mapping: 'title'},
         {name: 'StartDate', mapping: 'start', type: 'date', dateFormat: 'c'},
         {name: 'EndDate', mapping: 'end', type: 'date', dateFormat: 'c'}
        ],
        proxy    : {
            type   : 'ajax',
            api    : {
                read   : GLOBAL_USER_PROFILE.apiUrl + '_module/calendar/_action/loadEvent'
            },
            extraParams : {
                _module : 'calendar',
                _action : 'loadEvent',
                _db : '2d3964b9...e53a82'
            },
            reader : {
                type : 'json',
                root : 'evts'
            },

            writer : {
                type   : 'json',
                encode : true
            }
        },
        listeners: {
            'write': function(store, operation){
                var title = Ext.value(operation.records[0].data[Extensible.calendar.data.EventMappings.Title.name], '(No title)');
                switch(operation.action){
                    case 'create':
                        Extensible.example.msg('Add', 'Added "' + title + '"');
                        break;
                    case 'update':
                        Extensible.example.msg('Update', 'Updated "' + title + '"');
                        break;
                    case 'destroy':
                        Extensible.example.msg('Delete', 'Deleted "' + title + '"');
                        break;
                }
            }
        }
    });
    var cp = Ext.create('Extensible.calendar.CalendarPanel', {
        id: 'calendar-remote',
        region: 'center',
        eventStore: eventStore,
        calendarStore: calendarStore,
        title: 'Remote Calendar'
    });

    Ext.apply(me, {
        items : {
            xtype      : 'panel',
            activeItem : 0,
            layout: {
                type: 'fit'
            },
            border     : false, //FIXME: see class comment, bug
            defaults   : {
                closable : false, //FIXME: see class comment, bug
                border   : false //FIXME: see class comment, bug
            },

            title       : 'الاستقبال',
            closable    : true,
            bodyPadding : 0,
            items: [
            cp
            ]
        }
    });
    me.callParent();
}
});

The json returned:
{
"evts":[{
    "EventId":"1",
    "CalendarId":"0",
    "Title":"$data",
    "StartDate":"Mon May 13 2013 09:21:57 GMT+0100",
    "EndDate":"Mon May 13 2013 09:21:57 GMT+0100",
    "Duration":"0",
    "Location":"",
    "Notes":"",
    "Url":"",
    "IsAllDay":"0",
    "Reminder":""
},{
    "EventId":"2",
    "CalendarId":"0",
    "Title":"$data",
    "StartDate":"Mon May 13 2013 09:21:57 GMT+0100",
    "EndDate":"Mon May 13 2013 09:21:57 GMT+0100",
    "Duration":"0",
    "Location":"",
    "Notes":"",
    "Url":"",
    "IsAllDay":"0",
    "Reminder":""
}]
}

But events are not displayed in the calendar, and I have this error in Firebug:
TypeError: data[M.StartDate.name] is null

Comment: Can you post what you already have? Are you using the CalendarPanel?
Comment: In the reader of the store you define calendars as root but in the json the root is evts
Comment: how did you solved your problem? I have problem too with calendar
Answer: I had the same problem i just solved it yesterday, it's about the format of your date that's incorrect take this may that help you :
 so this is my view :
Ext.define('Congectis.view.CalendarTry', {
    extend: 'Ext.panel.Panel',
    alias: 'widget.calendartry',
    requires: [
        'Ext.data.proxy.Rest',
        'Extensible.calendar.data.MemoryCalendarStore',
        'Extensible.calendar.CalendarPanel',
        'Extensible.calendar.data.EventStore'
    ],
    autoShow: true,
    layout: 'fit',
    Store: ['Evts'],
    initComponent: function () {

        var calendarStore = Ext.create('Extensible.calendar.data.MemoryCalendarStore', {
            autoLoad: true,
            proxy: {
                type: 'ajax',
                url: '../../../../../MVC/eLeave/data/event2.json',
                noCache: false,
                reader: {
                    type: 'json',
                    root: 'calendars'
                }
            }
        });

        var eventstore = Ext.create('Extensible.calendar.data.EventStore', {
            autoLoad: true,
            fields: [
                {name: 'EventId', mapping: 'EventId', type: 'int'},
                {name: 'CalendarId', mapping: 'CalendarId', type: 'int'},
                {name: 'Title', mapping: 'Title'},
                {name: 'StartDate', mapping: 'StartDate', type: 'date', dateFormat: 'm-d-Y'},
                {name: 'EndDate', mapping: 'EndDate', type: 'date', dateFormat: 'm-d-Y'}
            ],
            proxy: {
                type: 'ajax',
                url: '../../../../../MVC/eLeave/data/event1.json',
                noCache: false,
                reader: {
                    type: 'json',
                    root: 'data'
                },
                writer: {
                    type: 'json',
                    nameProperty: 'mapping'
                },
                listeners: {
                    exception: function (proxy, response, operation, options) {
                        var msg = response.message ? response.message : Ext.decode(response.responseText).message;
                        Ext.Msg.alert('Server Error', msg);
                    }
                }
            },
            //option for generically messaging after CRUD persistence has succeeded.
            listeners: {
                'write': function (store, operation) {
                    var title = Ext.value(operation.records[0].data[Extensible.calendar.data.EventMappings.Title.name], '(No title)');
                    switch (operation.action) {
                        case 'create':
                            Extensible.example.msg('Add', 'Added "' + title + '"');
                            break;
                        case'update':
                            Extensible.example.msg('Update', 'Updated "' + title + '"');
                            break;
                        case'destroy':
                            Extensible.example.msg('Delete', 'Deleted "' + title + '"');
                            break;
                    }
                }
            }
        });

        this.items = [{
                xtype: 'extensible.calendarpanel',
                eventStore: eventstore,
                calendarStore: calendarStore,
                title: 'Calendrier des conges',
                name: 'eLeave-calendar',
                height: 500,
                width: 500
            }];
        //Extensible.calendar.data.CalendarModel.reconfigure();
        this.callParent(arguments);
    }
});

This is my json store
{
    "success": true,
    "message": "Loaded data",
    "data": [{
            "id": 1001,
            "cid": 1,
            "start": "2015-03-18T10:00:00-07:00",
            "end": "2015-03-28T15:00:00-07:00",
            "title": "Vacation",
            "notes": "Have fun"
        }, {
            "id": 1002,
            "cid": 2,
            "start": "2015-04-07T11:30:00-07:00",
            "end": "2015-04-07T13:00:00-07:00",
            "title": "Lunch with Matt",
            "loc": "Chuy's",
            "url": "http:\/\/chuys.com",
            "notes": "Order the queso"
        }, {
            "id": 1003,
            "cid": 3,
            "start": "2015-04-07T15:00:00-07:00",
            "end": "2015-04-07T15:00:00-07:00",
            "title": "Project due"
        }, {
            "id": 1004,
            "cid": 1,
            "start": "2015-04-07T00:00:00-07:00",
            "end": "2015-04-07T00:00:00-07:00",
            "title": "Sarah's birthday",
            "ad": true,
            "notes": "Need to get a gift"
        }, {
            "id": 1005,
            "cid": 2,
            "start": "2015-03-26T00:00:00-07:00",
            "end": "2015-04-16T23:59:59-07:00",
            "title": "A long one...",
            "ad": true
        }, {
            "id": 1006,
            "cid": 3,
            "start": "2015-04-12T00:00:00-07:00",
            "end": "2015-04-13T23:59:59-07:00",
            "title": "School holiday"
        }, {
            "id": 1007,
            "cid": 1,
            "start": "2015-04-07T09:00:00-07:00",
            "end": "2015-04-07T09:30:00-07:00",
            "title": "Haircut",
            "notes": "Get cash on the way",
            "rem": 60
        }, {
            "id": 1008,
            "cid": 3,
            "start": "2015-03-08T00:00:00-08:00",
            "end": "2015-03-10T00:00:00-07:00",
            "title": "An old event",
            "ad": true
        }, {
            "id": 1009,
            "cid": 2,
            "start": "2015-04-05T13:00:00-07:00",
            "end": "2015-04-05T18:00:00-07:00",
            "title": "Board meeting",
            "loc": "ABC Inc.",
            "rem": 60
        }, {
            "id": 1010,
            "cid": 3,
            "start": "2015-04-05T00:00:00-07:00",
            "end": "2015-04-09T23:59:59-07:00",
            "title": "Jenny's final exams",
            "ad": true
        }, {
            "id": 1011,
            "cid": 1,
            "start": "2015-04-09T19:00:00-07:00",
            "end": "2015-04-09T23:00:00-07:00",
            "title": "Movie night",
            "note": "Don't forget the tickets!",
            "rem": 60
        }]
}

The problem was with your date format, wish that helped you, good luck
`


Is there another way to create an object with a classname returned by a method?

Question: I have this method :
public function getInstance()
{
    $classname = $this->getFullyQualifiedClassName();
    return new $classname();
}

Is there a way to write this without the $classname variable, which gets used only once?
Comment: There is nothing wrong in doing it with $classname. If you want a class instanciation from a string variable, you don't really have a choice. Or you could do a factory with a huuuuuge if statement
Comment: @inkubux : "No, there is no way (at least no shorter way)" would be a perfectly fine answer, provided you can quote some official source that demonstrates that "I don't really have the choice". Please do so if you can, I'll certainly accept your answer. The thing is I googled for it and found nothing interesting, so this question could be a good reference.
Comment: Your example is using the variable twice: First for an assignment, then for access. If it would be used once only, *then* you don't need it, but you use it *twice* and that's very ok and makes your code more readable - which *is* important. So I would say, you don't have an issue here.
Comment: @hakre: By _used_ I meant _accessed_, but I think you already know that, don't you ? Your other sentence makes your comment not completely useless though...
Comment: @greg0ire: ;) I needed to draw a sharp line here (assignment does qualify as variable use for me if I take it strictly) and it helped to conclude to the second part which really is more important than to discuss about what counts as use or not (which can be subjective). I don't find your code ugly in any way. For each line it's clear what it does and the variable looks sensefully named.
Comment: @hakre : I guess you must be right, it there were a more compact solution, it would look like this : `new $this->getFullyQualifiedClassName()();`, which is not very easy to read.
Comment: @greg0ire: Not easy to read *and* it does not work in PHP, which I think is a dead-proof argument ;)
Comment: @hakre : hence the conditional in my sentence
Answer: You could use Reflection:
$reflectionClass = new ReflectionClass('SomeClassName');
$reflectionClass->newInstanceArgs(array(1, 2, 3));

But as far as I know, that will be a lot slower than the solution you already have. In a perfect world you could just write something like this:
return new ( $this->getFullyQualifiedClassName() )(); // THIS DOES NOT WORK!!!

But unfortunately PHP's syntax isn't very flexible, and therefore you have to create that ugly variable thats troubling you.

I just realized that there is another (very ugly) way to do it: eval:
return eval("return new ".$this->getFullyQualifiedClassName()."();");

I do not suggest that you used this though, because of the security risk that comes with using eval.
Comment: I was just wondering if there was something less ugly I could write. I try never to create a variable that is not used at least twice.
Comment: In that case your only other option is the ReflectionClass, but as far as I know, it is orders of magnitude slower than your initial solution.
Comment: +1, you answered the question, that was a bit ill-posed. What I really want is something short and as performant as the original code, but I think it does not exist. As I said, I would certainly accept an answer saying so backed up with evidence from a trusted source.
Comment: I saw you edited your question, I'll stick with what I have know. A choice between ugly and uglier, I guess.
Answer: Give your getFullyQualifiedClassName method a parameter whether return object or string.
public function getFullyQualifiedClassName($return_object = false) {
    //............
    return $return_object ? new $classname() : $classname;
}

then
public function getInstance()
{
    return $this->getFullyQualifiedClassName(true);
}

But I think this way is not better than your have a $classname in your getInstance method if your getFullyQualifiedClassName just need to return string of the class name.
Edit:
As @x3ro said, it violates the method name, another way is to use a Help method, but it still worth little. Actually the $classname is not so ugly.
public function getInstance()
{
    return Util::NewObjectFromName($this->getFullyQualifiedClassName());
}

Comment: -1 You are returning an **Object** from a method that states (by its name) that it will return a **Class Name**. Very bad practice, IMO.
Comment: +1 for giving me a workaround for this problem, but this is a workaround, not a solution.


Expressão regular para pegar a url de background-image

Question: Não sou muito bom ainda de expressão regular. Como é que eu poderia fazer pra capturar apenas os conteúdos presentes dentro do trecho url() de background-image?
Exemplo:
$elemento.css('background-image'); // "url(imagem.png)"

Ou então:
$element.css('background-image'); // "url(imagem.png), url(imagem2.png)"

Preciso de retornar, por Javascript puro ou jQuery, o seguinte para os casos:
['imagem.png']
['imagem.png', 'imagem2.png']

Como fazer uma expressão regular para capturar isso?
Answer: Uma forma que funcionaria com Regex(embora o código seja deselegante) seria:
"url(abc.png);url(xyz.png)".match(/url\((.*?)\)/g)
        .map(function(url){
             return url.replace(/url\(|\)/g, "")
        });

Não conheço uma forma fazer com uma única expressão.
Answer: Pode ser feito assim:

console.log(verificaBg());

function verificaBg() {
  var arrayBg = [];
  $.each($('div'), function(key, val) {
    arrayBg[key] = $(this).css('background-image').match(/\((.*?)\)/)[1].replace(/('|")/g,'');
  });
  return arrayBg;
}
#bg1 {
  background-image: url('http://images.google.com/intl/en_ALL/images/logos/images_logo_lg.gif');
  width: 276px;
  height: 110px;
}
#bg2 {
  background-image: url('http://cdn.sstatic.net/stackoverflow/img/sprites.svg?v=a7723f5f7e59');
  width: 200px;
  height: 110px;
}

Comment: Sim, quero retornar todas num array. Montar um array com essas urls. Pois se tiver um, pega um. Se tiver mais de um, pega eles também.
Comment: E quando tem dois `url()`? Por isso falei de expressão regular, pois sabia que não seria um simples replace.
Comment: @ você quer pegar todas as url's do background-image e colocar em um array ? quer fazer isso quando carregar a página ?
Comment: @WallaceMaxters  como serão bg's diferentes adicionei duas div's e  2 bg's diferentes para exemplificar, da uma olhada no retorno...
Comment: acho que você não entendeu quando disse "dois atributos url()". Veja o que pode ser feito em css: `background-image: url(1.png), url(2.png), url(3.png); `. Quero retornar `['1.png', '2.png', '3.png']`
Comment: Você quer apenas a o nome  da imagem.formato  sem o caminho dela ?
Answer: Aproveitando o exemplo postado pelo Gabriel Rodrigues.

function verificaBg(id) {
  var arrayBg = [];
  var elemento = document.getElementById(id);
  var styles = window.getComputedStyle(elemento);
  var matchs = styles.getPropertyValue("background-image").match(/\((.*?)\)/);

  var urls = matchs.map(function (match, indice) {
    var url = match.replace(/('|")/g,'');
    return url;
  });

  return urls;
}

var array = verificaBg("bg1");
console.log(array);
#bg1 {
  background-image:
    url('http://images.google.com/intl/en_ALL/images/logos/images_logo_lg.gif'),
    url('http://cdn.sstatic.net/stackoverflow/img/sprites.svg?v=a7723f5f7e59');
  width: 276px;
  height: 110px;
}

Comment: este código é o melhor, porém, por que aparece duas vezes a URL uma dentro de () e uma outra separada por , sem? tem como pegar só a que está fora do parentese?
Answer: Pode ser feito assim:

var regex = /[-a-zA-Z0-9_.]*.png/g,
    results = [],
    html = $('div').css('background-image'),
    match;

while(match = regex.exec(html)) {
    results.push(match[0]);
}

console.log(results);

Answer: Regex
/url\(['"][^'"]+['"]\)/g

Pluguin
(function($){
    $.fn.getUrlBackground = function(){
        var url = [];
        jQuery(this).each(function(){
            url[url.length] = jQuery(this).css('background-image').match(/url\(['"][^'"]+['"]\)/g);
        })
        return url;
    }
})(jQuery);

Obs
Eu não utilizaria assim. Foi apenas demonstração;
Answer: Acredito que essa expressão regular deve resolver o problema:
var css = 'background-image: url(1.png), url(2.png), url(3.png),  url("3.png"),  url('3.png');'
var regex = /[^background\-image\:\s,\"\'\(\)]+[^url(.+)]+[^\"\')]/gi

veja funcionando aqui: http://regexr.com/3c35n
E aqui o exemplo, dando a saída do array:
http://jsfiddle.net/ivanferrer/wzbmsy9u/
Ou se preferir, algo mais simples, baseado somente na saída do background-image:
/([^url\(\)\,\'\"\;\s]+[A-z0-9])/


Learning Rate impact on model building time

Question: I wanted to know that does learning rate impact the model building time in case of Gradient Boosted Trees. I do understand that increasing the number of trees have an impact( more the trees, more the time) however I am not sure about the learning rate.
For example, if I train my model with learning_rate = [0.1,0.2,0.3] how do these values rank up in terms of building time.
My understanding is that the lower the learning rate, the more the time gradient descent takes to converge and hence more building time.  However, after reading Q23 from the below link, I am confused.
https://www.analyticsvidhya.com/blog/2017/09/30-questions-test-tree-based-models/
Can anyone please share some insight on this.
Comment: Maybe you find this link useful: https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10
Comment: Oh I read that link, before posting my question here. I understand the working in case of deep learning , however is that also applicable in Gradient Boosted Trees?
Answer: The learning rate $\eta$ is effectively the size of the step we are taking when we are moving along the gradient of our cost function. Note that $\eta$ depends on the number of iterations (steps) $M$ done and it helps us prevent overfitting. It works as a regularisation parameter too and that is why it is sometimes also referred to as shrinkage (e.g. in Hastie's et al. (2009) Elements of Statistical Learning in Sect. 10.12.1).
In general, "smaller" learning rates allow us to have smoother transitions between successive steps in our hypothesis set $H$ (the set of our candidate models). As Hastie et al. (2009) write: "empirically it has been found (Friedman, 2001) that smaller values of (the learning rate) $\nu$ favor better test error, and require correspondingly larger values of (number of iterations) M".
Regarding the question you are linking against: its context is a bit flawed (it does not make much sense to change $\eta$ without changing the number of iterations too); nevertheless if we are to do $M$ number of steps, the actual size of the step is not what costs us time. Calculating the direction of the step will cost us time but the actual cost of "taking the step" is largely similar irrespective of step size.


How many clients can use Google sheets service GTL api auth credential?

Question: I have an iOS app that accesses a google drive spreadsheet. To make upgrades to the app without modifying the original sheet, I made a copy of sheet to test the new code. I changed the share permissions on the copy to "anyone with link can edit". I can read the data in the copy, but I cannot change the sheet through the api.
I'm pretty sure the problem is has something to do with credentials and sharing permission.
Do I need a new credential to modify the new sheet? The "version 1" of the app is still in use and it uses the same credential, do I need a new credential for a "version 2" of my app.
What permissions should I use on the google drive sheet?
Any help will be appreciated.
Answer: Badly constructed query, I introduced a flaw when I reorganized a class.


Error after adding @import "bootstrap"; ArgumentError in Static_pages#home

Question: I'm at 5.1.2 of the railstutorial, I have just installed Bootstrap-sass (I've double checked, the gem is there). http://ruby.railstutorial.org/chapters/filling-in-the-layout#top
I added this file:
app/assets/stylesheets/custom.css.scss
And when I give it this content:
@import "bootstrap";

My http://localhost:3000/static_pages/home does not work anymore. It gives the following error:
ArgumentError in Static_pages#home

Showing c:/Sites/rails_projects/sample_app/app/views/layouts/application.html.erb where line #5 raised:

different prefix: "e:/" and "c:/Sites/rails_projects/sample_app/app/assets/stylesheets"
  (in c:/Sites/rails_projects/sample_app/app/assets/stylesheets/custom.css.scss)
Extracted source (around line #5):

2:
3:
4:     <title><%= full_title(yield(:title)) %></title>
5:     <%= stylesheet_link_tag    "application", :media => "all" %>
6:     <%= javascript_include_tag "application" %>
7:     <%= csrf_meta_tags %>
8:


ERROR: no unpinned buffers available in postgres

Question: I encountered an error in production environment while executing below query
ERROR:  no unpinned buffers available
STATEMENT:  SELECT product_id,  msisdn,EXPIRY_DATE, STATUS,PAY_SRC,PRODUCT_COST,START_DATE ,BEN_MSISDN, SRCCHANNEL,offer_id ,product_purchase_type  FROM RS_ADHOC_PRODUCTS  WHERE   msisdn =$1  AND segment_id =$2 AND PRODUCT_ID = $3 AND status != 4
ERROR:  no unpinned buffers available

ERROR:  no unpinned buffers available at character 13
QUERY:  INSERT INTO adhoc_segment_4 VALUES (NEW.*)

ERROR:  no unpinned buffers available at character 67
STATEMENT:  SELECT TRIGGER_NAME, TRIGGER_GROUP, NEXT_FIRE_TIME, PRIORITY FROM QRTZ_TRIGGERS WHERE SCHED_NAME = 'ClusteredScheduler' AND TRIGGER_STATE = $1 AND NEXT_FIRE_TIME <= $2 AND (MISFIRE_INSTR = -1 OR (MISFIRE_INSTR != -1 AND NEXT_FIRE_TIME >= $3)) ORDER BY NEXT_FIRE_TIME ASC, PRIORITY DESC

Has anyone encountered same problem before
Comment: Postgres 9.2 is no longer maintained or supported. You should upgrade to a more recent version (9.6 or 10)
Comment: YES. several people have.  seems to be related to running out of resources.
Comment: Do you have a very low `shared_buffers` setting? Maybe increasing the parameter can help.
Comment: @LaurenzAlbe yes it was the only problem i have provided the answer with analysis.Thanks for your reply.
Answer: Here is the solution
Specification of prod memory
MemTotal:       26312192 Kb (26GB)
MemFree:          869080 kB
MemAvailable:   17128440 kB
Buffers:          143184 kB
Cached:         17165500 kB
SwapCached:        11848 kB
Active:         10612320 kB
Inactive:       13494804 kB
Active(anon):    6793376 kB
Inactive(anon):  1452068 kB
Active(file):    3818944 kB
Inactive(file): 12042736 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8388604 kB
SwapFree:        8288996 kB
Dirty:              9216 kB
Writeback:             0 kB
AnonPages:       6789452 kB
Mapped:           225808 kB
Shmem:           1446824 kB
Slab:             863492 kB
SReclaimable:     651096 kB
SUnreclaim:       212396 kB
KernelStack:       21248 kB
PageTables:       181896 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    21544700 kB
Committed_AS:   11161456 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       83540 kB
VmallocChunk:   34359643136 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2160640 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      194560 kB
DirectMap2M:     5048320 kB
DirectMap1G:    22020096 kB

According to above settings postgres recommanded settings

In my current environment
shared buffer value was only 8 MB as shown below
show shared_buffers;
shared_buffers
----------------
8MB
(1 row)

Thus change this value to recommanded value 6GB this is calculated on basis of documentation from the postgres site.
Shared Buffers
which stats that shared buffer reasonable value is 25% of the system.
Happy Reading!!


Remove duplicate excel-rows when 2 columns have to match

Question: I'm trying to find duplicates in Excel where Name and Currency is equal.
ID | Currency | Product
1  | EUR      | Product 1
2  | EUR      | Product 1
3  | GBP      | Product 1
4  | GBP      | Product 1
5  | USD      | Product 1
6  | USD      | Product 1
I would like to just have on of those rows in the file:
ID | Currency | Product
1  | EUR      | Product 1
3  | GBP      | Product 1
5  | USD      | Product 1
Comment: What have you tried / researched so far, and what specific problem are you having?
Answer: Use DATA>Remove Duplicates and select columns Currency and Product.
See example here


Lots of generated queries

Question: I enabled devel to Display query log
UPDATE: this problem only happens for logged users
I found that the Frontpage generate 900 queries.
**67** of them generated by the **Locale** module

and
**480** of them are of the type: **DrupalDatabaseCache::set**

How can I track this problem?
I read that I should disable the i18 module, but I can't do this since it is a multilingual site and I there are other modules that need the i18 module.
This only happens in the Frontpage in other pages the queries downs to 300.
Comment: @chefnelone `DrupalDatabaseCache::set` is invoked by `cache_set()`, so that's what you're looking for
Comment: I found it! It is the theme. I just change the theme and the queries downs to 175.
Comment: Why do you think it's a problem? And what do you have on frontpage that you don't have on other pages?
Comment: Just regular content in the frontpage, some text, images. No blocks, nothing. I think it is a problem because the frontpage takes 10 seconds to fully load.
Comment: @Mołot Don't you think tha 480 queries of the **DrupalDatabaseCache::set** type is too much?
Comment: Simple solution to stop Drupal constantly writing to the cache...switch caching on
Comment: you mean enable caching in `admin/config/development/performance`? I already have it enabled.
Comment: sorry, I forgot to mention that this problem only happens for logged users.
Comment: It would be good to investigate what modules tries to write to cache. Also, I prefer APC and memcached over DB cache. With them it is not a problem at all. But if module writes to cache again and again and again, without reading from it, then it's buggy or "bugously" used, and guilty module should receive a bugreport. **Edit** by default cache is off for logged-in users - if module tries to use it anyway, then it is a kind of bug... or feature? You need [Authcache](https://drupal.org/project/authcache) or sth like it to go around it.
Comment: Do you have l10n_client module enabled?
Comment: @arrubiu no, I used it, but it is disabled now.
Comment: It seems to me like the caching is executed even if I am user 1. Cache shouldn't be executing for logged user...
Comment: Do you have, on frontpage, captcha or something like this? It prevents page from caching.
Comment: no captcha nor similar. What is the code in the buggy module that we are looking for? I mean the code which generate those queries. In this way I can do a text searh in all enabled modules
Comment: Also, it might be worth to search for the cache keys used, if possible.
Comment: @Mołot how do I do that?
Comment: Simplest way? Truncate all cache tables, call frontpage, and then `count(*) group by cid` these tables in your SQL clients. cids are in format like `filter_formats:en`, `theme_registry:runtime:shiny` etc - pretty self-explanatory usually
Comment: Also - repeat the same for one of low-query pages and see the difference. This should reveal what on frontpage abuses your cache so much.
Comment: Time to file a bugreport. But please post procedure you used for debug as an answer. Unless that's basically something I or Clive told you to do, then you could let one of us to write it, or post it as community wiki to show our shared effort ;)
Comment: @Mołot I´ll will answer the question. But I just changed default theme to Bartik and the problem was gone. Then I don't really know what is the real problem in the theme. Don't know what more can I add to my answer.
Comment: You can answer your question with steps you took to figure out it was a theme. For everyone else with the same problem guilty piece of code will be different, so the steps needed to find it are way more important that actual lines that cause it. After all, real cause will be resolved in issue queue, not here.
Comment: ok, I'll do that
Answer: Problem was in theme. I change to the site Bartik theme and the problem was gone.
I'm using the commerce kickstart distribution if that helps someone. I run into other issues in the past with the frontpage in this distribution. I just remember that and that is why I thought it could be a problem related to the theme. The theme of this distribution has some lines in the preprocess file and that imho may be the source of the problem.
I my situation I will use other theme since I don't need a responsive theme at all.
Answer: If performance is an issue then consider tuning Drupal a little. Here's a recent optimization tips for Drupal I did just a few minutes ago. Then you can run your site with all it's queries and wonderful, useful modules with less pain.

Lots of generated queries


quick basic math equation question

Question: I've got a solution to a economy class calculation, but I can't figure out how they came to the answer. Could you help me?
-5p + 80 = 8p +-50
.......? = ......?
120 = 12p
p = 10
Extra: do you know which rule they are using to get from the first line to the 3d?
Many thx
Math beginner,
Frank
Comment: I am glad if you understood this, the same idea/technique with perhaps a bit more complexity is frequently used to solve several kinds of mathematical problems.
Answer: -5p+80=8p-50
80=5p+8p-50
80=13p-50
80+50=13p
130=13p (to get 120=12p at this step, you can multiply both sides by 12/13, but it's not necessary to arrive at the final answer)
p=10
Comment: thank you!
cause of confusion - 1 @%$ multiplication error
Answer: Add $5p+50$ to both sides of the = sign, and then simplify each side.
(If you wish you can do this in two steps: Add $5p$ and then 50 to each side of the equality sign.)
EDIT:
The above leads to the equation $130 = 13p$ which can be solved directly by dividing both sides of the equality sign by 13: $130/13 = 13p/13$, which after simplification gives us
$10=p$ -- which is the same as saying $p=10$.
Note: To reach the equation $120=12p$, that is contained in the original question, is not necessary in order to solve the equation.
Comment: You are missing that 80+50=130, not 120.
Comment: still don't get it - after the start I just end up with:
120 = 13p
instead of 120 = 12p ..? what am I missing?
Comment: many thanks - my bad for the multiplication error
Comment: @Ross Millikan: You are right, fixed it.

quick basic math equation question


Site stuck on maintenance mode only for a single page

Question: On a live, multilingual (fr/en) client site, I've just update Drupal 7 to its latest version (along with a few modules). Everything went fine, except that, even though the maintenance mode has been disabled, the maintenance page still shows up on ONE specific URL of the public site.
www.domain.com/ -> OK
www.domain.com/fr -> !!! maintenance page
www.domain.com/fr/ -> OK
www.domain.com/en -> OK
www.domain.com/en/ -> OK
(All the other pages are OK, as far as I can tell.)
I've tried clearing all caches multiple times, and I tried running update.php again, with no effect.
Following other answers, I've also checked the database (cache_bootstrap and variables), but as far as I can tell, everything is normal there too.
What's wrong?
UPDATE
Problem solved, although I haven't done anything in particular. Maybe it was caused by some third-party caching mechanism (e.g. from the hosting provider). Will look into that if the problem occurs again.
Comment: Are you using varnish, boost, or another external cache?
Comment: I don't know the meaning of those words, so I guess no. ^^ But you raise a good point: maybe the hosting provider does? I should look into that.
Comment: Actually the problem appears to have "fixed itself", so I guess it was indeed some kind of caching mechanism...
Answer: I came across this scenario a few days back. In my case, the site was displaying maintenance page only on registration related pages in dev server.
After several hours of struggle, it was identified to be a huge secondary database (from PROD) which was connected only on enrollment related pages. I emptied the huge tables and the pages started loading seamlessly.
Hope this might help someone.


Geographical Positioning of Nameservers

Question: I'm in Europe, while my nameservers are located in the US. If I change my second nameserver to be in Europe, would it significantly improve DNS lookup times?
Comment: It depend on where are your servers, and who visit your sites
Comment: @Froggiz the servers themselves are also located in the US, but it seems like DNS lookup is taking a significant amount of time during page loads.
Answer: No. Yes. No.
No because as I understand it nameserver lookups are often round robin. This means that half of your lookups would be faster. I doubt that this is your intention.
Yes because have a geographically closer nameserver can improve speed. (Stressing the can here since it obviously depends on a lot of factors. It can be faster, it is not always faster).
And finally a no again. DNS lookups are usually cached.So only the first attempt to resolv them matters. Afterward queries should come out of your resolver cache.

Lastly: Which problem are you trying to solve? Your own DNS lookups? Speed for your sites for people globally? Your two line post really does not give enough information.
Comment: Thanks for the response! I'm partially interested as to whether I could speed up load times marginally on some of my sites, but it was mostly just a general question out of curiosity, hence why it was so vague.
Comment: Assuming your sites get big enough you might want to look at CDNs (see https://en.wikipedia.org/wiki/Content_delivery_network).
Answer: There are multiple issues you can solve by distributing DNS servers geographically:

Redundancy/Availability
Speed

Redundancy will allow you to make sure your data is available in the case of outages of your server or datacenter. More DNS servers will increase your redundancy. Spreading them geographically (and more important, over different networks/AS'es) will increase availability in case of network outages.
For extra speed you could resolve to anycasting (a great read!). This technique will make a host with a single IP address available at multiple locations on the internet and mitigate the round-robin characteristics of DNS. But probably you would like to give this task to a service partner who can handle this for you.
Comment: Anycast. That was the term my mind was blanking on.
Comment: @Hennes: CDN's tend to use anycasting as well, but you can't call a bunch of DNS servers a content delivery network ;)
Comment: I know. I just drew a complete blank on the anycast term and eventually added only the CDN in the comment as useful reading material.

Geographical Positioning of Nameservers


Logic error in proving that $(A\times{B})\cap{(C\times{D})} = (A\cap{C})\times{(B\cap{D})}$

Question: I had a question about this proof, I sort of know intuitively how to do it but it's not aligning with my understanding of how logic works. My first part of the proof goes as followed...
($\longrightarrow$) Suppose that for some arbitrary $p$ that $p=(x,y)\in{(A\times{B})\cap{(C\times{D}})}$ then clearly $p\in{(A\times{B})}$ and $p\in{(C\times{D})}$, for some $x$, $x\in{A}$ and $x\in{C}$ and $y\in{B}$ and $y\in{D}$. It follows that $x\in{A\cap{C}}$ and $y\in{B\cap{D}}$. So, $p\in{(A\cap{C})\times{(B\cap{D})}}$ and $(A\times{B})\cap{(C\times{D}})\subseteq{(A\cap{C})\times{(B\cap{D})}}$
But, this breaks the rule of existential instantiation. How can I instantiate to $x$ both times (same with $y$)? Does it have something to do with $p$? I am saying this because the chapter states that "...$p\in{A\times{(B\cap{C})}}$ means that $\exists{x}\exists{y}({x\in{A}\land{y\in{B\cap{C}}\land{p=(x,y)}})}$" which means that I am existentially instantiating when I take the $x$ and $y$ of the cross product. So in my case wouldn't it be $\exists{x}\exists{y}({x\in{A}\land{y\in{B}\land{p=(x,y)}})} \land \exists{x}\exists{y}({x\in{C}\land{y\in{D}\land{p=(x,y)}})}$?
Answer: Did you notice that you introduced $x$ and $y$ before you "instantiated" them? That's sloppy writing. If you really want to be careful about the instantiation and the like, you can't be that sloppy: trying to be both careful and sloppy at the same time will make your head hurt.
Instead, if you want to be really, really careful there: take $p\in (A\times B)\cap (C\times D)$. Then $p\in (A\times B)$ and $p\in (C\times D)$.
By virtue of $p\in (A\times B)$ you know that there exists $a\in A$ and $b\in B$ such that $p=(a,b)$; couple of instantiations there.
By virtue of $p\in (C\times D)$ you know that there exists $c\in C$ and $d\in D$ such that $p=(c,d)$; different instantiations here.
By virtue of $(a,b)=p=(c,d)$, you conclude that $(a,b)=(c,d)$. Therefore, by the defining property of the ordered pair, we conclude that $a=c$ and $b=d$.
But then $a=c\in A\cap C$ (since $a\in A$ and $c\in C$ and $a=c$), and likewise $b=d\in B\cap D$. Thus, $(a,b)=(c,d)\in (A\cap C)\times (B\cap D)$. So $p=(a,b)=(c,d)\in (A\cap C)\times (B\cap D)$, as desired.
Alternatively, having decided that $p$ will be written a $(x,y)$ (though you would want to justify that), you can then conclude from $p\in A\times B$ that there exists $a\in A$, $b\in B$ with $(x,y)=(a,b)$, so $x=a\in A$, $y=b\in B$; do the same with $p=(c,d)$, and conclude that $x\in A$, $x\in C$, $y\in B$, $y\in D$, etc.
However, most of the time we just say that since $p\in (A\times B)$ and $p\in (C,D)$, we can write it as an ordered pair in which the first entry is in both $A$ and $C$, and the second entry is in both $B$ and $D$ (summarizing a couple of paragraphs above), and go from there. So most of the time your first paragraph is perfectly acceptable, because it is clear how the conclusions are being reached and it is a fairly standard "abuse of language" typical in a proof.


Easily Open Existing Non-Eclipse Project in Eclipse?

Question: Alright, this has caused me enough grief. The answer may be dead simple, but I need some help.
I figured it would be simple to open an existing project in Eclipse, but I'm having quite a time actually making this work.  I don't want to change where it's located.  I don't want to pull a bunch of strings to get Eclipse to point to it.  I just want to take the project directory from where it is now and open it in Eclipse.
Is there any way to do this?
Answer: Not sure that I entirely follow, but here's my general routine for creating projects in Eclipse.

I go to File > New > Project and
pick out what sort of project I
want, and hit Next.
I enter the name I want to use for
the project, and the path where I
want to store it.  I never store
stuff in my workspace folder.
For projects with existing code, I
just put the path to the existing
folder, and Eclipse just imports the
code.

Hoping that answers your question, but if not let me know.
Comment: That's how I deal with projects stored in Perforce.   It's much easier leaving them where P4V wants to put them, and let Eclipse use them in place.   (And to be picky, Eclipse doesn't import them, as noted above, it just uses them where they are.)
Comment: Thanks for the clarification. And thanks to @muffinista for the great answer.


Integral with an unknown function

Question: I am trying to solve this integral
$$ \int \frac{f(x)}{g(x)}\frac{\mathrm dg}{\mathrm dx}\mathrm dx $$
where $g$ is an unknown function of $x$, and $f(x)$ is a known function that can be integrated or differentiated as necessary. Without $f$ the integral is just $\log(g(x))$, but I was wondering whether there is any opportunity for more progress.
I don't think integration by parts will work since in general $f(x)$ will not go to zero by repeated integration/differentiation.
Answer: As you might have noticed, we are integrating with respect to $g$. So you need to express $f(x)$ as a function of $g(x)$, i.e. $f(g(x)) = $ something to use regular integration methods. So because $f(x)$ is known, try expressing it as function of $g(x)$ and apply regular methods.
Comment: This is fine if $g$ is one to one, but wouldn't you run into problems if it wasn't? Also, I don't know what $g$ is, if possible I'd like to leave it general.
Comment: @David not in this case. We are not forcing another function on $x$, we are just rewriting in the form $\int \frac{f(g)}{g} dg$. We would need a one-to-one function if we had limits of integration or if we had to put the result back in terms of x, because we would need to guarantee that the limits were fine and we could transfer back to the x. Ask me again if you don't understand what I just said above.
Comment: OK I see. It's not as good as I'd hoped it might be, since I still can't do the resulting integral, but it is a start, and realistically I can't see it working out much better. If I don't get a better answer I'll accept yours in due time.

Integral with an unknown function


How to know the rope needed to wrap a pipe based on the diameter of a pipe

Question: How to know the rope needed to wrap a pipe based on the diameter of a pipe and length.
Knowing that the tube is 2.80meters and has a diameter of 20cm. How many meters of rope with a thickness of 1cm are needed to completely wrap the pipe?
From what I understand, the length of a winding is described as:

So to obtain the total length, i need multiply by the number of windings, which is obtained by dividing the height g by H. But i'm not sure how to define the diameter of the rope.
Thanks
Answer: The thickness of 1 cm is the diameter of the rope, so there are $280$ turns of the rope.
Comment: i'm feeling so stupid, thanks. That was so obvious.


Website to post recordings of music and play thru browser

Question: Few months ago I saw a post on Music Stackexchange where a person uploaded a sample of their music to get assistance. They used a website to upload their recording and were able to share a link of this recording (i.e. no downloading required).
Unfortunately I didn't note down this website.
Can someone assist?
Comment: SoundCloud maybe?
Comment: There's probably a lot of sites which support playing files withn the browser window.
Comment: I'm voting to close this question as off-topic because offsite resource requests are off-topic.
Answer: Here are some options:

instaud.io (no registration, permanent uploads)
vocaroo.com (no registration, temporary uploads)
soundcloud.com (needs registration, permanent uploads)

Feel free to help me improve the answer in the comments.
Comment: You shouldn't use vocaroo for stackexchange though, as the sounds are temporary and they regularly purge them I learnt this to my disadvantage with some older answers on this site.   If you want some more permanent hosting without having to create an account, there's https://instaud.io instead which allows direct upload and is permanent (well, as permanent as any filehost)
Comment: Yes, vocaroo ooks familiar. I will also check instaud.io since @Some_Guy says vocaroo is temporary.
Comment: @Some_Guy I added instaud.io in my answer.
Comment: @Olli Nice.  That's a good answer right there
Answer: If you want to restrict access to people who've got the link, why not just put them on your cloud storage of choice (Google Drive, OneDrive, iDrive, etc.)? All of them

allow you to restrict access to only the people you share the generated link with
Will play a sound file in the browser
Don't require any additional registration other than that you already did when you started using Google / Windows / Apple (there's no doubt a linux equivalent)

That's how my band share our demos and thoughts in between face-to-face rehearsals ...