In Ruby, is it possible to share a database connection across threads?

Question: I've got a small little ruby script that pours over 80,000 or so records.
The processor and memory load involved for each record is smaller than a smurf balls, but it still takes about 8 minutes to walk all the records.
I'd though to use threading, but when I gave it a go, my db ran out of connections. Sure it was when I attempted to connect 200 times, and really I could limit it better than that.. But when I'm pushing this code up to Heroku (where I have 20 connections for all workers to share), I don't want to chance blocking other processes because this one ramped up.
I have thought of refactoring the code so that it conjoins the all the SQL, but that is going to feel really really messy.
So I'm wondering is there a trick to letting the threads share connections?  Given I don't expect the connection variable to change during processing, I am actually sort of surprised that the thread fork needs to create a new DB connection.
Well any help would be super cool (just like me).. thanks

SUPER CONTRIVED EXAMPLE
Below is a 100% contrived example.  It does display the issue.
I am using ActiveRecord inside a very simple thread.  It seems each thread is creating it's own connection to the database.  I base that assumption on the warning message that follows.
START_TIME = Time.now

require 'rubygems'
require 'erb'
require "active_record"

@environment = 'development'
@dbconfig = YAML.load(ERB.new(File.read('config/database.yml')).result)
ActiveRecord::Base.establish_connection @dbconfig[@environment]

class Product < ActiveRecord::Base; end

ids = Product.pluck(:id)
p "after pluck #{Time.now.to_f - START_TIME.to_f}"

threads = [];
ids.each do |id|
  threads << Thread.new {Product.where(:id => id).update_all(:product_status_id => 99); }
  if(threads.size > 4)
    threads.each(&:join)
    threads = []
    p "after thread join #{Time.now.to_f - START_TIME.to_f}"
  end
end

p "#{Time.now.to_f - START_TIME.to_f}"

OUTPUT
"after pluck 0.6663269996643066"
DEPRECATION WARNING: Database connections will not be closed automatically, please close your
database connection at the end of the thread by calling `close` on your
connection.  For example: ActiveRecord::Base.connection.close
. (called from mon_synchronize at /Users/davidrawk/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/monitor.rb:211)
.....
"after thread join 5.7263710498809814"   #THIS HAPPENS AFTER THE FIRST JOIN.
.....
"after thread join 10.743254899978638"   #THIS HAPPENS AFTER THE SECOND JOIN

Comment: `I have thought of refactoring the code` - I suspect you actually will have to - 80,000 DB accesses, as you imply, is going to be slow whatever you do. Can you show some code?
Comment: If you're using ActiveRecord then it's already shared.
Comment: Slow is all about perspective. :)  80,000 in 8 minutes would be fine, if I wasn't building the system to handle a couple million. Sadly I there's nothing special about the code. Just a small script to connect and update records.
Answer: See this gem https://github.com/mperham/connection_pool and answer, a connection pool might be what you need: Why not use shared ActiveRecord connections for Rspec + Selenium?
The other option would be to use https://github.com/eventmachine/eventmachine and run your tasks in EM.defer block in such a way that DB access happens in the callback block (within reactor) in a non-blocking way
Alternatively, and a more robust solution too, go for a light-weight background processing queue such as beanstalkd, see https://www.ruby-toolbox.com/categories/Background_Jobs for more options - this would be my primary recommendation
EDIT,
also, you probably don't have 200 cores, so creating 200+ parallel threads and db connections doesn't really speed up the process (slows it down actually), see if you can find a way to partition your problem into a number of sets equal to your number of cores + 1 and solve the problem this way,
this is probably the simplest solution to your problem
Comment: Thanks for the connection pool link, that might be the ticket. My 300 line ruby app is already running in a Delayed_Job which is sort of why I was looking for something.  It sucks to have 250megs of ram and use 3megs, but at the same time have 10 minutes to get a job done, and take 8 (or more).  I want to trade speed for ram, and "core +1" threads seems to be one way to accomplish this. I couldn't even have two threads before your answer, because connections was my limit. Again thanks!
Comment: I gave the second link a go, and my simple little app crashed each time it attempted to connect in the thread.
Comment: event machine fires up distinct connections to the DB. :(
Comment: EM.defer uses a thread pool to execute its code (default is 10 threads iirc) which means that no more then number-of-threads connections will be up and your CPUs should still be pegged at 100% given an appropriate thread count for your system.
Comment: The other thing EM is good at is using the maximum out of your cpu, which means that if you make sure all your IO calls are non-blocking your app will not wait for db responses and other socket operations which would enable your processors to be used to the max.

This means if you have x cores, you can set up x EM reactors and assuming your reactor is well written you should be able to squeeze the last bit of processing power from your system.
Comment: @baash05

Also, the trade-off between computing power and memory isn't as obvious as you describe it.

You will get the best performance when you limit the number of worker threads so your CPUs are pegged at 100%. Once you start forking more then this you will actually *degrade* performance due to context switching overhead between threads.

However, this entire discussion applies to jruby and rbx, and **not to MRI ruby**, if you're using MRI then you need to fork processes (not threads) or you need to use EM because the GIL will put a hard limit on your processing that thread spam won't fix
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/42494/discussion-between-dosadnizub-and-baash05)
Answer: Okay, I understand that my response is inapplicable to your problem. However, I'd want to provide a secure method for DB work with multiple threads in a Rails application.
Number of Thread running should be smaller than Total DB connection pool size.
This is no.1 rule. So I made that busy running DB connection's size shouldn't be reach to half of total db connection pool size.
If you have 10 DB connection pool size, maximum thread count would be 5. So it's quite safe.
When you have less than 5 DB connection available, It will run as single thread, Then When you have enough DB connection, It will run it as multi thread again.
def call
  if enough_db_connection?
    run_as_multi_thread(works, max_thread_count)
  else
    run_as_single_thread(works, max_thread_count)
  end
end

private

def enough_db_connection?
  (ActiveRecord::Base.connection_pool.stat[:size] / 2) > ActiveRecord::Base.connection_pool.stat[:busy]
end

def max_thread_count
  (ActiveRecord::Base.connection_pool.stat[:size] / 2)
end

def works
  @works ||= [some array of work..]
end

## running methods

def run_as_multi_thread(works, max_thread_count)
  # puts "We have enough db connection. Lets running it as multi thread."
  works = works.take(max_thread_count)
  rest_of_works = works.drop(max_thread_count)

  works.map do |work|
    Thread.new do
      ActiveRecord::Base.transaction do
        work.do_some_work # Do some work
      end
    end
  end.each(&:join)

  if rest_of_works.size.positive?
    if enough_db_connection?
      run_as_multi_thread(rest_of_works, max_thread_count)
    else
      run_as_single_thread(rest_of_works, max_thread_count)
    end
  end
end

def run_as_single_thread(works, max_thread_count)
  # puts "We don't have enough db connection now. Cuurent Busy Running DB connection size is bigger than #{max_thread_count}. Let's running it as single thread."
  works = works.take(max_thread_count)
  rest_of_works = works.drop(max_thread_count)

  works.map do |work|
    ActiveRecord::Base.transaction do
      work.do_some_work # Do some work
    end
  end

  if rest_of_works.size.positive?
    if enough_db_connection?
      run_as_multi_thread(rest_of_works, max_thread_count)
    else
      run_as_single_thread(rest_of_works, max_thread_count)
    end
  end
end


Double line loop search in Python 3?

Question: I am using Python 3 and I need to extract data from a file. An example of the data is below:
ENERGY_BOUNDS
  1.964033E+07  1.733253E+07  1.491825E+07  1.384031E+07  1.161834E+07  1.000000E+07  8.187308E+06  6.703200E+06
  6.065307E+06  5.488116E+06  4.493290E+06  3.678794E+06  3.011942E+06  2.465970E+06  2.231302E+06  2.018965E+06
GAMMA_INTERFACE
     0
EIGENVALUE
  1.219034E+00
N,2N
  1.191994E+00  1.535081E+00  1.543891E+00  1.413861E+00  1.181815E+00  6.174152E-01  1.302440E-02  0.000000E+00
  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00
  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00
MACRO      1
SIGABS
 -3.826074E-03 -3.707513E-04  2.610351E-03  6.961084E-03  7.832982E-03  7.512567E-03  1.018417E-02  1.276596E-02
  9.148128E-03  8.828235E-03  8.527789E-03  7.514346E-03  7.544248E-03  7.801064E-03  7.724884E-03  7.047571E-03
  5.280749E-03  3.999751E-03  3.821688E-03  3.748186E-03  3.712753E-03  3.591795E-03  3.390300E-03  3.180354E-03
SIGTRAN
  7.513455E-02  8.061355E-02  8.377954E-02  8.787775E-02  9.114071E-02  9.170817E-02  9.440786E-02  9.535947E-02
  1.010975E-01  1.035364E-01  1.160553E-01  1.290131E-01  1.197249E-01  1.151962E-01  1.298934E-01  1.375417E-01
  1.428861E-01  1.715100E-01  1.627465E-01  2.026621E-01  2.007540E-01  1.644982E-01  1.781501E-01  1.624188E-01

The process needs to be:
Search the file line by line until the starting keyword (MACRO in this case) is found.
After this, continue searching line by line until the specific identifier is found.
Read each value in the lines after the identifier into an array or list.
Stop reading once another identifier is found.
So far this is what I have. The code works fine if the identifier is the first value after MACRO (e.g. if it is SIGABS) but not for any others (e.g. SIGTRAN). My results file has maybe 50 different identifiers in it so I need the code to be able to pick out one at a time.
def read_data_from_file_macro(file_name, start_macro, identifier):
    with open(file_name, 'r') as read_obj:
        list_of_results = []
        # Read all lines in the file one by one
        for line in read_obj:
            # For each line, check if line contains the string
            if start_macro in line:
            # If MACRO is found, start looking for the identifier read the next line
                nextValue = next(read_obj)
                if identifier in nextValue:
                # If identifier is found read next line
                    nextValue = next(read_obj)
                    while(not nextValue.strip().isidentifier()): #keep on reading untill next identifier appears
                        list_of_results.extend(nextValue.split())
                        nextValue = next(read_obj)
                        # Convert to float
                        for i in range(0, len(list_of_results)):
                                list_of_results[i] = float(list_of_results[i])
    return(list_of_results)

Comment: what is the integer that represents '-3.826074E-03'?
Comment: ...d'oh. Have taken that bit out, thank you!
Comment: Can you explain this line?    `while(not nextValue.strip().isidentifier()):`    How isidentifier() is defined?
Comment: iterate line of file until MAKRO found, set a boolean. iterate further lines until your keyword is found, set another boolean.  while line starts with a space, read lines into list. If line does not start with a space, reset both bools to false and process list. All your "data" is indented by 1 space ... keywords: are not.
Comment: isidentifier() checks whether the next value is a valid identifier - any letter, number, or underscore but it can't start with a number.
Answer: Try the following. It handles you file as text, separates the part that is included between your start_identifier and end_identifier and with some work it returns a list of floats that is finally extended in your list_of_results (which must preexist before the function is called, so you must firstly create it manually). You can run for any pair of identifiers. Let me know how it works
def read_data_from_file_macro(file_name, start_identifier, end_identifier):
    with open(file_name) as f:
    t=f.read()
    t=t[t.find('MACRO'):]
    t=t[t.find(start_identifier)+len(start_identifier):t.find(end_identifier)]
    t=t.replace('\n', '').split()
    t=[float(i) for i in t if not i.isidentifier()]
    list_of_results.extend(t)

Double line loop search in Python 3?


How to harmonize ^4 ^2 after a ii7 chord?

Question: I am busy trying to complete an exercise in which I have to harmonize a soprano melody using supertonic and subdominant harmony where the red stars are marked. I am allowed to use inversions. I got stuck in bar 5 where the blue arrow is pointing. Could anybody give me a hand and tell me what chords could go in this space? I only have the main diatonic chords to work with and have to follow the ii7 with some kind of dominant family chords. The problem I have is that the D in the soprano wouldnt resolve if I used a V chord... This is what confuses me. So the only other chord that goes there is a vii°6.. maybe that could go to a V7 chord?
Answer: Stick on the E chord for 2 beats.   The D DOES resolve to C♯, the B just delays it.  That's OK.   A very common cadential cliche in hymn-style writing!
Comment: Thanks. Wont the 7th be transfered to another voice if I keep the E7 on both beats? If the answer is yes then wont it need to resolve too?
Comment: See expanded answer above.
Comment: Thank you Lawrence. I had thought of that but the reason that I didn't use that is because I thought that going from a V7 to a V on the second beat before moving back to I was somehow not allowed. But if this solution is acceptable then fantastic!
Comment: You're not really going from V7 to V.   Just decorating the V7 a bit.  Look at the first two beats of that bar as all V7.


Debug an app runnin of AVD

Question: I added the file-transfer cordova plugin to my app to upload images, to test it i had to use the AVD since i can't do this on my browser.
the App run correctly on the AVD the UI showing but non of the function that needs to make HTTP calls on my server works, like login, Register or Upload an image. i made sure my server is up and running and things working so well on browser.
My question is how to debug my up while it's Running on AVD
Answer: Bring up a command prompt (Windows)/terminal (Mac) window and run adb logcat to view the logcat output of the AVD in real-time.
If you want to write this to a file, do something like adb logcat > debug.log
If the AVD is Android 4.4+, you can connect Chrome developer tools to it - navigate to chrome://inspect/#devices in Chrome. You can view the network activity of the AVD Webview this way.

Debug an app runnin of AVD


How to Setup two DNS servers with newly registered domain using IP as Nameserver?

Question: I have setup two VPS DNS Servers using Ubuntu 16.04.02 and PowerDNS with IP's
 DNS IP 1 184.108.40.206 [ns1.hostthrone.com Hostname]
 DNS IP 2 184.108.40.206 [ns2.hostthrone.com Hostname]

I Register new domain hostthrone.com using namecheap.com and point the name server to :
 18.104.22.168
 184.108.40.206

On First DNS Server I create Master Zone
 12 hostthrone.com  SOA n1.hostthrone.com hostthrone.com 2017073002 28800 7200 604800 86400     86400
 13  ns1.hostthrone.com  A  18.104.22.168
 14  ns2.hostthrone.com  A  126.96.36.199

on the second DNS Server i created a slave zone pointing to the master NS 220.127.116.11.
I also created Reverse DNS on my VPS Provider site
522 220.127.116.11 241.180.239.169.in-addr.arpa    ns1.hostthrone.com
521 18.104.22.168 220.127.116.11.in-addr.arpa    ns2.hostthrone.com

and also Reverse DNS in powerdns control panel for each record.
The domain does not get pointed to DNS Servers or DNS is wrongly configured Any help ?
Comment: Put in a support ticket with your registrar asking about glue records
Answer: A lot of things are going wrong here.  Firstly, your DNS servers don't seem to be set up right:
[me@risby ~]$ dig soa hostthrone.com @188.8.131.52
[...]
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1680
;; QUESTION SECTION:
;hostthrone.com.                        IN      SOA
[...]
[me@risby ~]$ dig soa hostthrone.com @126.96.36.199
[...]
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1680
;; QUESTION SECTION:
;hostthrone.com.                        IN      SOA

But your single biggest problem is that you haven't set the domain up with your registrar properly:
[me@risby ~]$ whois hostthrone.com
[Querying whois.verisign-grs.com]
[Redirected to whois.namecheap.com]
[Querying whois.namecheap.com]
[whois.namecheap.com]
Domain name: hostthrone.com
[...]
Name Server: dns1.registrar-servers.com
Name Server: dns2.registrar-servers.com

Whilst it is perfectly legitimate to have a domain use nameservers inside itself - that's what glue records were invented for - each registrar has a slightly different procedure for setting this up, and you have not done yours.  That means your registrar is not telling the internet that your servers are authoritative for your domain, so even if they were working, the domain would not be served properly.
Thank you for not obscuring your domain name and IP addresses; that makes the question much easier to answer.
Comment: Surely the `;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: n` line from the first set of dig outputs should not have been removed? The status code appears to be the single most telling thing in what little is returned?
Comment: @HåkanLindqvist I think everyone will have their own preferred set of diagnostic steps and things to look out for; the most significant element of the the response for me was `WARNING: recursion requested but not available`.  However, to my mind, the single most important thing to show is not what I got but **what I typed**.  With that, the OP can start to dig into this him/herself; can try a working lookup and see how the responses differ, and start to form his/her own hypotheses to test - and others who come later, with slightly different problems, can do likewise.
Comment: Fwiw, the `;; WARNING: recursion requested but not available` message is generated by `dig` when the message flags indicate that you asked for recursion (you can opt not to ask for recursion with `+norec` on the command line) but the server did not allow recursion. In the case here, with you querying an authoritative server, it's not a problem at all that it does not allow recursion, it's rather a good sign.
Comment: As for *what I got* vs *what I typed*, I agree to a point. However, I think a very important aspect is *what to look for*.
Comment: Any tutorial you can suggest to me please ? i added glue records to namecheap using this article  https://www.namecheap.com/support/knowledgebase/article.aspx/292/10/how-can-i-update-my-personal-nameservers-ip-address but still shows whois hostthrone.com
   Name Server: DNS1.REGISTRAR-SERVERS.COM
   Name Server: DNS2.REGISTRAR-SERVERS.COM
Comment: @Donno I'm sorry, but you should direct questions about your registrar's processes in the first case to your registrar.
Comment: @HåkanLindqvist I was going to disagree, but on reflection I think you have a point.  Feel free to edit my answer, if you think it could be improved, or try one of your own!
Answer: After following MadHatter answer I solved the whois hostthrone.com by using these namecheap settings :

My DNS server would only work for period of time after setting up hostthrone.com zone in PowerDNS

After a lot of googling the real problem was that that Ubuntu was installing an alpha version of PowerDNS
Here is the solution :
sudo nano /etc/apt/sources.list.d/pdns.list

deb [arch=amd64] http://repo.powerdns.com/ubuntu xenial-auth-40 main

sudo nano /etc/apt/preferences.d/pdns

Package: pdns-*
Pin: origin repo.powerdns.com
Pin-Priority: 600

curl https://repo.powerdns.com/FD380FBB-pub.asc | sudo apt-key add - && sudo apt-get update && sudo apt-get install pdns-server

If you have an error of this nature RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 then edit /lib/systemd/system/pdns.service
#RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6

Install pdns-backend-mysql
sudo apt-get install pdns-backend-mysql
service pdns restart
netstat -tap | grep pdns

tcp        0      0 *:domain                *:*                     LISTEN                                                             1145/pdns_server-in
tcp6       0      0 [::]:domain             [::]:*                  LISTEN                                                             1145/pdns_server-in

dig hostthrone.com SOA @184.108.40.206 returns an answer still after a few days


Difference between a control in a frame and a control not in a frame

Question: I am writing a Excel VBA program that examines VBA forms and converts them to HTML forms.  Most of it works, but I need to know how to tell if a VBA control is within a frame or not.  Because when I use absolute positioning in HTML, if (for example) the left attribute of a control in a frame needs the left attribute of the frame added to it.
Answer: It's not necessarily a frame, it can be any control that allows nested controls.
There are several ways to deal with it:

Recursively add all .Parent.Left until the last .Parent is the form.
Translate to HTML using nested blocks too, so that you don't need to add anything.
Comment: thanks GSerg..  I did a "? userform22.textbox19.parent.name" and it gave me "Frame1" which is what I want to know.
Comment: @LittleNorm That is a wrong approach. You should not rely on controls names. If you want to keep it simple and only support frames (and only one nesting level deep), you can check if `TypeOf textbox19.parent Is Frame`.


How to calculate the singularities of $x^4 + x^2 + 1$?

Question: Here is the integration I want to calculate (complex analysis):
$$\int_{-\infty}^{\infty} \frac{\cos x}{x^4 + x^2 +1}$$
But I do not know how to factorize the following equation $x^4 + x^2 + 1$ to get the singularities.
I first let $y = x^2$ and I used the quadratic formula to get $y = \frac{-1 \pm \sqrt{3}i}{2}$ but then what should I do to get $x$? Could someone help me please?
EDIT:
I think my professor did it using that any complex number can be written as $r e^{i \theta}$ but I do not know how he got the $\theta$, could anyone show me the solution by this method please?
Comment: You can compute it using polar coordinates. Singularities are at $y = \exp\left(\frac{2 \pi}{3}\right)$ and $y = \exp\left(\frac{4 \pi}{3}\right)$, thus $x = e^{i \phi}$ where $\phi \in \left\{\frac{\pi}{3}, \frac{5 \pi}{3}, \frac{2 \pi }{3}, \frac{4 \pi }{3}\right\}$.
Comment: @Secretly since $e^{i \phi} = \cos \phi + i \sin \phi$ I observed that $- \frac{1}{2} = \cos \left(\frac{2 \pi}{3}\right)$ and also $\frac{\sqrt{3}}{2} = \sin \left(\frac{2 \pi}{3}\right)$. Thus $\phi = \frac{2\pi}{3}$. Similarly for $-\frac{1}{2} - \frac{\sqrt{3}}{2}$ you get that $\phi = \frac{4\pi}{3}$.
Comment: If $y=x^2$, $x=\sqrt{y}$. Probably easier by using de Moivre on the polar form, $2 \text{cis} \dots$ whatever the angle is.
Comment: @AnneBauval no it does not, I think my professor did it using that any complex number can be written as $r e^{i \theta}$ but I do not know how he got the $\theta $
Comment: Then, use [this one](https://math.stackexchange.com/questions/742841).
Comment: @MrGhost How did you know these values?
Answer: You can multuply $x^4+x^2+1$ with $x^2 - 1$ and solve equation $x^6=1$ then remember that extrude root $x = \pm 1$.
Comment: Ummm. Is it $x^6=1$?
Comment: Yes, thank you.
Answer: You can compute the square roots of your $y_\pm:=\frac{-1\pm i\sqrt3}2$ directly, by the classical method indicated in the proposed duplicate. E.g. for $y_+$: let $z=a+ib$ with $a,b\in\mathbb R$.
$$\begin{align}z^2=\frac{-1+i\sqrt3}2&\Leftrightarrow a^2-b^2=-\frac12,\;a^2+b^2=1,\;2ab>0\\
&\Leftrightarrow a^2=\frac14,\;b^2=\frac34,\;ab>0\\
&\Leftrightarrow z=\pm\frac{1+i\sqrt3}2.\end{align}$$
Comment: How did you know that $a^2 + b^2 =1$? is that because our radius is assumed to be 1?
Comment: and why $ab>0$? is it because $2ab = \sqrt{3}/2$?
Comment: @Emptymind 1/2: $a^2+b^2=|z|^2=|y_+|=1.$ 2/2: Yes.
Answer: Method I
$x^4+x^2+1=x^4+2x^2+1-x^2=(x^2+1)^2-x^2=(x^2+x+1)(x^2-x+1)$
$=(x-a_1)(x-a_2)(x-a_3)(x-a_4)$
where $a_1,a_2,a_3,a_4$ are roots of those two quadratic equations.
Method II
Start from your $x^2=y= \frac{-1 \pm \sqrt{3}i}{2}=cos\pm\frac{2\pi}{3}+isin\pm\frac{2\pi}{3}=cos\pm\frac{4\pi}{3}+isin\pm\frac{4\pi}{3}$
Then you can get 4 $x$'s like
$x=cos\pm\frac{\pi}{3}+isin\pm\frac{\pi}{3}, cos\pm\frac{2\pi}{3}+isin\pm\frac{2\pi}{3}$


Point-wise defined equation

Question: Recently I have faced a problem of the following general form:
Find function f such that $f(x, y) = \alpha f(a_1x + b_1y + c_1, a_2x + b_2y + c_2)$ for all $0<x<1$, $0<y<1$, where $a_1$, $a_2$, $b_1$, $b_2$, $c_1$, $c_2$ and $\alpha$ are constants. Note that the value of the constants ensure that $0<a_ix+b_iy+c_i<1$. In other words, $f$ is defined on a $(0, 1) \times (0, 1)$ square.
I searched a bit and it seems that this problem has no well-established method of solving. It would be great if I find some insight here.
Answer: This is a fine, but difficult problem. The solution depends on the dynamics produced by the iterates of the map
$$T:\quad Q\to Q,\qquad(x,y)\mapsto\left\{\eqalign{x'&:=a_1x+b_1y+c_1\cr y'&:=a_2x+b_2y+c_2\cr}\right.\quad.$$
About $T$ we are only told that it maps the open square $Q:=\ ]0,1[\, ^2$ into itself.
There is a large number of morphologically different cases to consider. E.g., $T$ might or might not have an attractive fixed point in $Q$, etcetera.
Comment: Thank you for your prompt reply, Christian.
The original problem is a 3D one. However, I found it difficult to solve the 3D problem, so I started with the 2D form of the problem. That is why the question is posed on a square. The original problem is:
$2f(x, -x+1, -2x+1) = f(0, -x+1, -x+1)  ;  0<x<frac{1}{2}$
To solve it, first I tried to guess the solution! Next, I Taylor expanded the right-hand side around the point $(x, -x+1, -2x+1)$. The resulting equation is a PDE. However, it contains all possible orders of derivatives (up to $n$-th derivative, where $n\rightarrow \infty$).
Comment: Truncating the PDE to the terms with first or second derivatives does not work as the residual of the truncated terms is not negligible. So, it seems all orders of derivatives shall be considered; not an easy task. That is why I am looking for other methods.
I see your point. As $x=0$ is not in the domain, there is no fixed point.
I don't know if this type of problem has even a specific name in the scientific community. It would be great if you suggest some references or keywords toward the solution of this problem?
Comment: The "3D-problem" you sketch in your comment only refers to the values of $f$ on two selected lines, and poses no conditions whatsoever on the values of $f$ in the rest of the envisaged cube.
Comment: Generally, the domain of $f$ is the cube. The aforementioned relation between the values of $f$ on the two lines is the only condition that must be met in determining $f$. You raised a good point, indeed, as I am not sure if this condition would result in a unique solution. I'd expect a class of solutions.

Point-wise defined equation


single point of failure in IIS Web Farm Framework setting?

Question: ASP.NET WEB API
Windows Server 2008 R2/IIS 7.5/Web Farm Framework 2.5
I am planning to deploy application across 4 web servers.
Should i use shared content/configuration using DFS among web servers for web farm scenario?
Second option is to use Web Farm Framework for deployment.
Furthermore, is there chance of single point of failure in WFF? for example what if primary server goes down.
which option would be better? pros and cons of each of the above.
I appreciate your response.
Answer: The Web Farm Framework is easy.  The only SPOF exists when integrating with an external load balancer as opposed to using the built in Application Request Routing.  This is mainly because unless your load balancer is able to detect a failure on the server it may still try to route to that server.
If you are just deploying a single application the WFF is a perfect choice and MUCH easier to configure than the traditional shared config setup.


Why is typescript optional class property set to undefined for es2022

Question: Typescript compiler handling of optional class properties seems to have changed from es2021 to es2022.
class A {
    a?: string
    b?: string
    constructor() {
        this.b = 'asd'
    }
}
console.log(new A())

with tsconfig target=es2021 results in
A: {
  "b": "asd"
}

with tsconfig target=es2022 results in
A: {
  "a": undefined,
  "b": "asd"
}

I cannot find any documentation regarding this change. Is this really intended behaviour and why?
It is easy to reproduce in
ts playground
by changing TS Config->Target
Comment: ES2021? Surely you mean ES3. In the playground it is currently set to *ES3*, and that produces `A` without an `a` field. Maybe you need to edit your question?
Comment: Yeah, the generated js code isn't the same in ES2022 & ES2021.
Comment: @MatthieuRiegler Uhhh what? It is the same for me. I only notice the "2021" behavior when targeting ES3...
Comment: @vera For me the change happens going from es2021 to es2022. es3 has same behavior as es2021.
Answer: The difference is useDefineForClassFields :

This flag is used as part of migrating to the upcoming standard
version of class fields. TypeScript introduced class fields many years
before it was ratified in TC39. The latest version of the upcoming
specification has a different runtime behavior to TypeScript's
implementation but the same syntax.
This flag switches to the upcoming ECMA runtime behavior.

TS Documentation


Prove that a sequence has at least three partial limits.

Question: Let  $a_n$ be a sequence. suppose $|a_{n+1}-a_n|<2$ for every $n$.
given $lim\inf_{n\longrightarrow\infty}a_{n}\leq1$ and $lim_{n\longrightarrow\infty}\sup a_{n}\geq4$.
Prove:  the sequence has at least three Subsequential limits.
Answer: The idea is to form a subsequence that must alternate between being close-ish to the infimum and supremum. In order to get from one side to the other the subsequence must bounce in the interval. The points that fall in the middle have a subsequence that converges, and we will guarantee that this sequence does not converge to the infimum or supremum.
By definition of infimum, for all $\epsilon > 0$, for any $N$ (no matter how large) we may find a $k\geq N$ such that
$$a_k -\operatorname{lim\,inf}_n a_n < \epsilon$$
Rearranging and applying the inequality we get
$$\operatorname{lim\,inf}_n a_n \leq a_k < \operatorname{lim\,inf}_n a_n + \epsilon \leq 1 + \epsilon$$
Similarly, for the supremum, we find
$$4-\epsilon < a_k \leq \operatorname{lim\,sup}_n a_n $$
Fix some $\epsilon$ such that $\tfrac12 \geq \epsilon > 0$ (the decision of upper bound will become apparent shortly, but is necessary here as we are fixing $\epsilon$).
Using these guarantees we may construct an index sequence inductively over $k$ as follows:

For $2k$, define $n_{2k}$ to be a number such that $a_{n_{2k}} < 1+\epsilon$ and $n_{2k} > n_{2k-1}$.
For $2k+1$, define $n_{2k+1}$ to be a number such that $a_{n_{2k+1}} > 4-\epsilon$ and $n_{2k+1} > n_{2k}$.

We will show that for each $k$ there exists an index $r_k$ between $n_{2k}$ and $n_{2k+1}$ such that
$$1+\epsilon\leq a_{r_k}\leq 4-\epsilon.$$
By the definition of the subsequence, $a_{n_{2k}} < 1+\epsilon$, and so
$$a_{n_{2k}+1} = \underbrace{a_{n_{2k}+1}-a_{n_{2k}}}_{\leq |a_{n_{2k}+1}-a_{n_{2k}}|} + a_{n_{2k}} < 2 + (1+\epsilon) = 3+\epsilon \leq 4-\epsilon$$
where the right-most inequality follows from the upper bound on $\epsilon$.
Hence $a_{n_{2k}+1}\not \geq 4-\epsilon$, and if $a_{n_{2k}+1} \geq 1+\epsilon$ then we are done (take $r_k = n_{2k}+1$) otherwise $a_{n_{2k}+1} < 1+\epsilon$. Continuing inductively, we must find an $r_k$ as otherwise, for all $i\colon\, n_{2k}\leq i \leq n_{2k+1}$, $a_i < 1+\epsilon$, contradicting the definition of $n_{2k+1}$ (as $a_{n_{2k+1}} > 4-\epsilon$ by construction).
Finally, as $\left\{r_k\right\}$ is a bounded sequence, it has a convergent subsequence that converges to some limit $\ell\in[1+\epsilon,4-\epsilon]$ and
$$\operatorname{lim\,inf}_n a_n \leq 1 < 1+\epsilon \leq \ell \leq 4-\epsilon < 4 \leq \operatorname{lim\,sup}_n a_n$$
So $\operatorname{lim\,inf}_n a_n$, $\ell$, and $\operatorname{lim\,sup}_n a_n$ are distinct partial limits.
Comment: @Jneven I don't quite follow. It is perfectly possible that every single $a_n$ satisfies $1< a_n < 4$, e.g., $a_n = \begin{cases}1+1/n & \text{for $n\equiv 0 \operatorname{mod} 4$}\\
4-1/n & \text{for $n\equiv 2 \operatorname{mod} 4$}\\
2 & \text{otherwise}
\end{cases}$
Comment: Can you please explain how you concluded that ark is between 4-epsilon and 1+epsilon?
Comment: I have modified my answer to make this clearer. Essentially we are are always able to find an $r_k$ for which $n_{2k} < r_k < n_{2k+1}$ and $1+\epsilon \leq r_k \leq 4-\epsilon$.
Comment: is it possible to solve this question by saying: we shall want to find a sub-sequence of $a_{n}$, suppose it to be $a_{n_{k}}$ . If there's an infinite number of objects of $a_{n}$ in s.t: $1\leq a_{n_{k}}\leq4$, than we will get a contradiction to the claim assuming there's only infinite number of $a_{n} $ objects  for $1\geq a_{n}$ and $a_{n}\geq4$.
Answer: Recall this standard result:

$\limsup_{n \to \infty} a_n$ is the largest subsequential limit of the sequence $a_n$.
$\liminf_{n \to \infty} a_n$ is the smallest subsequential limit of the sequence $a_n$.

Since $\limsup_{n \to \infty} a_n \geq 4$ and $\liminf_{n \to \infty} a_n \leq 1$, this means that $a_n$ has at least one subsequential limit greater than or equal to $4$, and at least one subsequential limit less than or equal to $1$.
It remains to find a third subsequential limit.

To make progress, I would prove the following:

Lemma: For all $N \in \mathbb N$, there exists an $n \geq N$ such that $a_n \in [\tfrac 3 2, \tfrac 7 2]$.

Proof: Suppose, for contradiction, that there exists an $N \in \mathbb N$ such that for all $n \geq N$, $a_n \notin [\tfrac 3 2, \tfrac 7 2]$.

$a_n$ has a subsequence that tends to $\liminf_{n \to \infty} a_n$, which is less than or equal to $1$. Therefore, there exists an $n_1 \geq N$ such that $a_{n_1} \in (-\infty, \tfrac 3 2)$. By induction, one can prove that $a_{n'} \in (-\infty, \tfrac 3 2)$ for all $n' \geq n_1$. (You'll need to use the fact that  $|a_{m+1} - a_{m}| < 2$ for all $m \in \mathbb N$ to prove the inductive step.)
$a_n$ has a subsequence that tends to $\limsup_{n \to \infty} a_n$, which is greater than or equal to $4$. So there exists an $n_2 \geq N$ such that $a_{n_2} \in (\tfrac 7 2, \infty)$. By similar reasoning, one can prove that $a_{n'} \in (\tfrac 7 2, \infty)$ for all $n' \geq n_2$.

These two bullet points are contradictory. Indeed, defining $n := \max \{ n_1, n_2 \}$, we have $a_n \in (-\infty, \tfrac 3 2) \cap (\tfrac 7 2, \infty) = \emptyset$, which is absurd.

Our Lemma shows that $a_n$ has infinitely many terms that lie in the interval $[\tfrac 3 2, \tfrac 7 2]$. From this, we may deduce that $a_n$ has a subsequence $b_k := a_{n_k}$ such that $b_k \in [\tfrac 3 2, \tfrac 7 2]$.
So $\liminf_{k \to \infty} b_k$ and $\limsup_{k \to \infty} b_k$ are both in $[\tfrac 3 2, \tfrac 7 2]$. Therefore, $b_k$ (and therefore $a_n$) has a subsequential limit between $\tfrac 3 2$ and $\tfrac 7 2$, and we're done.


Item Response Theory (IRT) -- Problem (?) or unxepected result using the mirt package with a 2PL model for binary variables?

Question: I'm working with 5 variables related to "child and family health" using Item Response Theory to check the model and identify its parameters. My model is composed of items such as "Do you have a child with a learning or behavior problem?" and "Do you and your family members have health insurance or access to regular medical and dental care?"
My model is well fitted: TLI 0.92
However, the coefficients that I've obtained showed that one item (cf2) has the difficulty at 31.375. I'm not sure if something is going under my radar and I'm wondering if someone could explain these results.
This is the simulated data, just to give you the chance to run my code.
df2 = structure(list(cf1 = c(0, 0, 0, 10, 0, 0, 0, 10, 10, 0, 10, 0,
                                 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0),
cf2 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 10, 0, 10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0),
cf3 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0, 0, 0, 10, 0), cf4 = c(0, 10, 0, 0, 10, 0, 0, 10,                                                                                10, 0, 10, 0, 10, 10, 0, 10, 0, 10, 0, 0, 0, 0, 0, 10, 10, 10,                                                                                10, 10, 0, 0, 0, 0, 0, 10, 0, 0, 10, 0, 0, 0, 0, 0, 0, 10, 0,                                                                                10, 0, 0, 0, 10, 10, 0, 10, 0, 0, 10, 10, 0, 10, 0, 0, 0, 0,                                                                                0, 10, 0, 10, 10, 0, 10, 0, 10, 0, 10, 0, 0, 0, 0, 10, 10, 10,                                                                                0, 10, 10, 10, 0, 0, 10, 10, 10, 0, 0, 0, 10, 0, 10, 0, 0, 10,                                                                                0),
cf5 = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 10, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0,                                                                                0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0, 0, 0, 0, 10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,                                                                                0, 0, 0, 0)),
row.names = c(NA, -100L), class = c("tbl_df", "tbl", "data.frame"))

mod_2pl_cf <- mirt(data = df2, model = 1, itemtype = "2PL"
    )
    coef(mod_2pl_cf, simplify=TRUE,
         IRTpars= TRUE)

Comment: Can you explain your abbreviations, IRT etc, and as an edit to the post, not only as comments ... and please add relevant tags, the tag [tag:r] should never be the only one!
Answer: A couple of things. First, in an item response theory (IRT) context, a Tucker-Lewis Index (TLI) of 0.92 is not necessarily good. For example, Cai, Chung, and Lee (2023) found that adequate fit is obtained when the TLI is greater than 0.97-0.98. Second, and perhaps more importantly, the two items you provide do not look like typical items one would use to establish a psychometric scale using IRT (or other models for continuous latent variables, such as confirmatory factor analysis). Instead, they look more like covariates used in a structural equation model. I say this because it is important to remember the assumptions of IRT models. Perhaps the most important assumption is that of conditional independence, which is that an individual's item responses are independent of each other, conditional on their latent variable score $\theta$. It is hard for me to believe that this assumption can be reasonably made with the two example items you provide. For more information on IRT and its assumptions, see Bock & Gibbons (2021).
This is all to say that I am not surprised you obtained such an implausible difficulty value, as I do not believe IRT is the correct model for your application.
References
Bock, R. D., & Gibbons, R. D. (2021). Item response theory. John Wiley & Sons.
Cai, L., Chung, S. W., & Lee, T. (2023). Incremental model fit assessment in the case of categorical data: Tucker–lewis index for item response theory modeling. Prevention Science, 24(3), 455-466.
Comment: Thanks. I was thinking about your response today and it does make sense. I agree with you 100%. However, from a purely statistical framework, IRT models and CFA (with categorical estimators) are virtually the same. Strangely, the CFA model runs perfectly while the IRT gives this strange coef.  Any thoughts on that ? (By the way, I'm just asking for academic purposes, my team and I decided to use network analysis with this data instead of latent-based models). Thanks
Comment: @Luis. Glad it helped!


Resolving a probability problem: obtaining 3 cards of the same color amongst a set of 52 cards

Question: My question is about calculating the probability of the following event A: obtaining 3 cards of the same color when drawing 3 cards among a set of 52 cards (and without putting back the drawn card).
I tried resolving this problem by adding the probability of the two events: drawing 3 red cards and drawing 3 black cards which would give:
$P(A) =(\frac{26}{52}*\frac{25}{51}*\frac{24}{50}) + (\frac{26}{52}*\frac{25}{51}*\frac{24}{50})$
The first term of the addition is understood as follows: drawing a first red card $\frac{26}{52}$ then a second red card: $\frac{25}{51}$ and so on. Likewise, the second term is about drawing 3 black cards.
However, it seems the above is wrong. Can someone please explain what is wrong in my reasoning?
Comment: There's nothing wrong with your thinking nor with your formula.  As a (possibly simpler) approach:  Note that the first card might be of either color, then the second is a match with probability $\frac {25}{51}$ and the third then also matches with probability $\frac {24}{50}$. so the answer is $\frac {25}{51}\times \frac {24}{50}=\frac 4{17}$.
Comment: Thanks for your comment.
Answer: The result of your formula is right.
If you want to use combinations, you can think the formula as:
\begin{align*}
& P = {26 \choose 3}{2 \choose 1}
\end{align*}
There are 52 cards, 26 of each color. You want to have 3 cards of the 26 of the same color. You then must multiply that with the number of combinations that can be formed with two colors choosing only one, which is the same as 2 combinations (black or red).
Using combinations may make it easier if you want to then calculate the probability of drawing three cards of the same suit. Simply replace the 26 with a 13 (because each suit has 13 cards for a total of 52 cards) and replace the 2 with a 4 (because there are 4 suits instead of only 2 colors).


Migrating server with separate registrar - update dns in registar or hosting/cpanel account?

Question: I am moving a website to a new server, the domain is registered with a separate registrar
Another guy owns the registrar account and has added the new nameservers for me, I have updated DNS settings on the hosting account
Am I right in thinking I need to ask the owner of the domain to update the dns settings in the registrar account too?
Email is hosted by the server, and the domain is registered not on the hosting site (so a separate registrar and hosting account)
Apologies for the stupid question
Thanks for any suggestions
Comment: You should give us the domain in question. Your description is fairly confusing, and I'd hesitate to say what you should do given that I'm having trouble deciphering what you *have* done.
Comment: The only records that matter are the ones on the nameservers configured for the domain.
Comment: Thanks, so I do not need to get the domain owner to update any dns settings such as cname, a record etc? They are all fine in the dns zone on the hosting, but I noticed when I do a crawl on SEO software Semrush, it says there was an error crawling due to DNS configuration. I thought I must have missed something when moving to our new hosting site. Have used DNS checkers which don't seem to show any major issues
Comment: the other owner doesn't want it posted sorry! A standard wordpress website, old host got rid of his hosting, so I set them up a hosting account. Simply asking if I need to update any dns settings other than the new nameservers (which we did a week or so ago so have taken effect now) in the registrar account. Is it that confusing? I am a web designer but don't deal much in hosting, sorry!
Comment: Figure out which nameservers are defined at the registrar. Make sure those nameservers have the correct records. To help debug beyond that, we'll need the domain.
Comment: I think you are confused, sorry, nameservers are fine, our new ones have been added. Do I need to edit any DNS settings at registrar other than the nameservers? I am guessing from your response the nameservers are all that matter
Comment: Again, only the nameservers you've defined as the primary nameservers for that domain at the registrar level matter. It doesn't matter if the registrar offers their own nameservers *if you're not using them*.
Comment: We have had *quite* a few people who *thought* they'd configured things correctly, but hadn't. That's why we ask for the domain, and why we hesitate to say "yep you're good" when it's not provided.
Comment: thanks but no issue with nameservers, we successfuly added the new ones, I am asking about records, cnames, mx records etc
Comment: I'm not sure where the disconnect is here. Records exist on a nameserver. You only need records on the nameservers responsible for serving your domain. I can put your records on *my* nameservers but it won't *do* anything, because no one looks there for them. Figure out *which* nameservers are responsible for your domain, and configure them accordingly with the right records. I'm done here unless you'd like to provide the domain.
Answer:
There's absolutely no technical reason to change the Name Servers simply because the website is moving.
<code>"Another guy owns the registrar account and has added the new nameservers for me, I have updated DNS settings on the hosting account."</code> What does that mean, exactly? Does that mean that the new web hosting provider is also hosting the DNS zone?
Assuming the answer to item 2 is Yes, then the answer is no, you don't need to do anything more at the Registrar.
Telling us the actual domain name would go a long way in helping us give you better answers.
Comment: No, like I said I have the hosting account but the domain is with someone else, simply asking if the DNS settings on the domain account need updating (cname, mx record etc). There is a DNS zone on the cpanel in the hosting account which I have access to. I am asking if anything other than nameservers need updating in the domain registrar account. And yes of coarse the nameservers need updating as they have changed, the old host has closed down, we pointed the site at the new hosting account. i.e changed from ns1.oldnameservers.co.uk, ns2.oldnameservers.co.uk to ns1.newnameservers.co.uk etc


Best builtin way for footer generation

Question: What would be the drupal 7 options to generate the footer links?
Writing the links by hand could lead to 404 errors if links change...
Answer: It's fairly simple:

Create a menu at http://mysite.com/admin/structure/menu/add
Add some links to it
Go to http://mysite.com/admin/structure/block and assign the automatically generated block for your new menu to your theme's footer region.

If your theme doesn't have a footer region you can add one by adding a line like the following to the theme's .info file:
regions[footer] = Footer

And then edit page.tpl.php to add the new region wherever it sits best in your HTML:
<?php echo render($page['footer']); ?>

Answer: Clive's answer is great if you want a footer where admins define the footer links manually. I usually prefer auto-generated footers using various Views blocks (I find Views more flexible than anything specific to footers would be).
If you want a footer that automatically tracks an existing menu but presents it differently, look into one of the many modules that put menu data into views. I've had (so far) good experiences with Menu Node Views despite it not having a stable release yet.

Best builtin way for footer generation


Ajax not output to Ctools modal when using pager

Question: So in my continuing quest to display a form with Solr search results in a Ctools modal window and use a pager, I've managed to get everything to display fine after the initial search, but when I use the pager, I get a white screen with the data in json format instead of everything displayed in the modal again. Here is the relevant code:
function imgsearch_page($ajax, $id, $a, $b) {
  if ($ajax) {
    //Load the modal library and add the modal javascript.
    ctools_include('ajax');
    ctools_include('modal');

    $form_state = array(
      'ajax' => TRUE,
      'title' => t('Image Search Form'),
      'next_field_id' => $id,
    );

    // Use ctools to generate ajax instructions for the browser to create
    // a form in a modal popup.
    $search_form = ctools_modal_form_wrapper('imgsearch_form', $form_state);

    if (($form_state['executed'] && $form_state['ajax']) || isset($_GET['page'])) {
      // If the form has been submitted, there may be additional instructions
      // such as dismissing the modal popup.
      if (!empty($form_state['ajax_commands'])) {
        $output = $form_state['ajax_commands'];
      }

      if ($form_state['values']['search_terms'] != '' || isset($_GET['search_terms'])) {
        $page_num = isset($_GET['page']) ? $_GET['page'] : 0;

        if ($_GET['search_terms']) {
          $form_state['values']['search_terms'] = $_GET['search_terms'];
        }
        $results = nb_image_search_search($form_state['values'], $page_num);

        if (is_array($results['images']) && count($results['images'] > 0)) {
          $next_field_id = $form_state['next_field_id'];
          // Create object to store file and target field info. To be stored in ctools cache.
          $file_info = new stdClass();
          // Generate the field name.  field_images is a multivalue field collection, so we just need the next available option
          // in the $field_images['und'] array. The second number (for field_image) will always be 0 since it
          // is a single value field.
          $file_info->fieldname['url'] = '#edit-field-images-und-' . $next_field_id . '-field-image-und-0-imgsearch-file-url';
          $file_info->fieldname['person'] = '#edit-field-images-und-' . $next_field_id . '-field-person-und-0-value';
          $file_info->fieldname['organization'] = '#edit-field-images-und-' . $next_field_id . '-field-organization-und-0-value';
          $file_info->fieldname['year'] = '#edit-field-images-und-' . $next_field_id . '-field-year-und-0-value';

          $file_info->fids = array();

          // Theme the results as a table.
          $header = array(t('Image'), t('File Name'), t('Add to field'));
          $rows = array();
          foreach ($results['images'] as $image) {
            // Create image style derivative for each image.
            $imagestyle = array(
              'style_name' => 'thumbnail',
              'path' => $image['filepath'] . $image['filename'],
              'width' => '',
              'height' => '',
              'alt' => '',
              'title' => $image['filename'],
            );
            $styled_image = theme('image_style', $imagestyle);
            $fid = $image['fid'];

            $rows[] = array(
              'image' => $styled_image,
              'name' => $image['filename'],
              'add' => ctools_ajax_text_button("select", "imgsearch/nojs/imgadd/" . $fid . '/' . $next_field_id, t('Select')),
            );

            $file_info->fids[$fid] = $image['filename'];

            // Cache values for Person, Organization, and Year if they exist.
            foreach (array('person', 'organization', 'year') as $field) {
              if (isset($image[$field])) {
                $file_info->meta[$fid][$field] = $image[$field];
              }
            }
          }
          //Cache image name in ctools object cache so it can be used later in nb_image_search_image_add()
          ctools_include('object-cache');
          ctools_object_cache_set('imgsearch', 'imgsearch_' . $next_field_id, $file_info);

          // Create a render array ($build) which will be themed as a table with a
          // pager.
          $build['search_form'] = isset($search_form[0]) ? drupal_build_form('imgsearch_form', $form_state) : $search_form;
          $build['imgsearch_table'] = array(
            '#theme' => 'table',
            '#header' => $header,
            '#rows' => $rows,
            '#empty' => t('There were no matching results found'),
          );

          // Attach the pager theme.
          $pager = pager_default_initialize($results['total_found'], $results['rows']);
          $build['imgsearch_pager'] = array(
            '#theme' => 'pager',
            '#parameters' => array(
              'search_terms' => $form_state['values']['search_terms'],
              //TODO: add params for Person, Organization, and Year.
              ),
          );

          $form_state['values']['title'] = t('Search Results');
          $output = ctools_modal_form_render($form_state['values'], $build);

          print ajax_render($output);
          drupal_exit();
        }
        else {
          $build['no_results'] = array(
            'markup' => '
No images found
',
          );
        }
      }
    }
    elseif (!isset($output)) {
      $output = ctools_modal_form_wrapper('imgsearch_form', $form_state);
      // Return the ajax instructions to the browser via ajax_render().
      print ajax_render($output);
      drupal_exit();
    }
  }
  else {
    return drupal_get_form('imgsearch_form', $id);
  }
}

The only real difference is that since form field values are removed somehow when the form is submitted via ajax, I use the #parameters element in the pager array, which puts it in $_GET[], allowing me to access it in the search function (nb_image_search_search()). The page argument is also there, put there by the pager. Since ctools_modal_form_wrapper() returns the form in rendered mode, I have to rebuild it again to pass it to ctools_modal_form_render().
Other than that, I don't see any differences. I've stepped through the code all the way to the call to drupal_json_encode() in ajax_render(), but can't get any farther than that.
What would be causing the json to just be displayed on a white screen instead of displaying in the modal, but only when I'm using the pager link?
Thanks.
Answer: The answer turned out to be a missing class in my pager links. Whenever you use CTools Modal, you have to add the ctools-use-modal class to the link. My first thought was to add the class as a link attribute in the link themeing functions, since the theme_pager_link() function accepts attributes (via $values['attributes']). However, the functions that actually call theme_pager_link() (e.g. theme_pager_previous(), theme_pager_next(), and theme_pager_last()) don't bother to actually pass attributes, so there's no way to get them to the link function. Sigh.
So, to work around this, I just created a quick jQuery behavior that adds the ctools-use-modal class to the pager links:
(function ($) {
    Drupal.behaviors.addCtoolsModalClass = {
        attach: function (context) {
          $('#modalContent .pagination ul li a').addClass('ctools-use-modal');
    }
  }
})(jQuery);

and voila!, it works as it should.
Answer: use array_chunks after pager_default_initialize.
A small example from my code, hope it will help you.
$per_page = 10;
$current_page = pager_default_initialize(count($result['node_ids']), $per_page);
$chunks = array_chunk(node_load_multiple($result['node_ids']),$per_page, TRUE);


Finding significant factors in a Latin Hypercube Design

Question: I'm simulating data where I have four factors and one response that I want to examine. I have two different DoE designs, one 2⁴ factorial design and one Latin Hypercube Design. The ultimate goal with the research is to see if I can see any difference between the two models, i.e. do they give different significant factors for my response depending on the DoE model used.
I understand how to plot and analyze the results from the factorial designs by aggregating the results and then performing ANOVA tests or fitting linear models by:
<code>doe.model <- lm(response ~ A + B + C + A * B + A * C + B * C + A * B * C, data = ss.data.doe1)

summary(doe.model)
</code>
However, I cannot seem to find any way of performing a similar test on my Latin Hypercube design, the 4 factors are assumed continuous and the design is 16 runs, i.e. every simulation has different values for the parameters.  Is there any way of performing a similar analysis for my LHC design? Am I missing something?
Answer: This is how you could create a similar design using a Latin hypercube in R
<code>#  Assume each parameter or factor covers some continuous range with some distribution
#    A ~ unif(0, 20)
#    B ~ norm(5, 1)
#    C ~ lnorm(1, 1)
#    D ~ exp(2)

# 2^4 factorial design
dataf <- expand.grid(A = c(0, 20),
                     B = c(3, 7),
                     C = c(0, 0.13),
                     D = c(0, 1))

# create a response function
dataf$Y <- with(dataf, 2 + 1.5*A + 1*B + 10*C + 3*D + rnorm(nrow(dataf), 2))

# build the model with all one-way, all two-way, and some three-way interactions
doe.model <- lm(Y ~ (A + B + C + D)^2 + A*B*C + A*C*D, data = dataf)

summary(doe.model)
#>
#> Call:
#> lm(formula = Y ~ (A + B + C + D)^2 + A * B * C + A * C * D, data = dataf)
#>
#> Residuals:
#>        1        2        3        4        5        6        7        8
#>  0.24862  0.06856 -0.24862 -0.06856 -0.29269 -0.02448  0.29269  0.02448
#>        9       10       11       12       13       14       15       16
#> -0.24862 -0.06856  0.24862  0.06856  0.29269  0.02448 -0.29269 -0.02448
#>
#> Coefficients:
#>              Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  3.438726   0.706903   4.864  0.01659 *
#> A            1.509233   0.045833  32.929 6.16e-05 ***
#> B            1.087473   0.126152   8.620  0.00328 **
#> C           23.426811   7.051292   3.322  0.04498 *
#> D            3.221236   0.722490   4.459  0.02101 *
#> A:B         -0.004900   0.007979  -0.614  0.58257
#> A:C         -0.236171   0.498602  -0.474  0.66808
#> A:D          0.026490   0.031914   0.830  0.46741
#> B:C         -2.080207   1.227472  -1.695  0.18870
#> B:D         -0.190358   0.112834  -1.687  0.19018
#> C:D          1.867989   4.909889   0.380  0.72895
#> A:B:C        0.104881   0.086795   1.208  0.31347
#> A:C:D       -0.280388   0.347182  -0.808  0.47841
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.4513 on 3 degrees of freedom
#> Multiple R-squared:  0.9998, Adjusted R-squared:  0.9992
#> F-statistic:  1525 on 12 and 3 DF,  p-value: 2.46e-05

# Latin Hypercube Design

require(lhs)
#> Loading required package: lhs
#> Warning: package 'lhs' was built under R version 4.1.3

set.seed(190383)
# select the same number of points in the LHS as are in the factorial design
lhs1 <- lhs::randomLHS(2^4, k = 4)

lhs1 <- as.data.frame(lhs1)
names(lhs1) <- c("A", "B", "C", "D")

lhs1$A <- qunif(lhs1$A, 0, 20)
lhs1$B <- qnorm(lhs1$B, 5, 1)
lhs1$C <- qlnorm(lhs1$C, 1, 1)
lhs1$D <- qexp(lhs1$D, 2)

lhs1$Y <- with(lhs1, 2 + 1.5*A + 1*B + 10*C + 3*D + rnorm(nrow(lhs1), 2))

lhs.model <- lm(Y ~ (A + B + C + D)^2 + A*B*C + A*C*D, data = lhs1)

summary(lhs.model)
#>
#> Call:
#> lm(formula = Y ~ (A + B + C + D)^2 + A * B * C + A * C * D, data = lhs1)
#>
#> Residuals:
#>        1        2        3        4        5        6        7        8
#>  0.28229  0.08898 -0.82246  0.08808  0.15481 -0.21934 -0.10925  0.38288
#>        9       10       11       12       13       14       15       16
#>  0.33827 -0.70871  0.14906 -0.42464 -0.27482  0.38189  0.44519  0.24776
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  6.92866   13.29290   0.521   0.6382
#> A            0.92121    1.17079   0.787   0.4888
#> B            0.49425    2.69352   0.183   0.8661
#> C           10.13896    3.47114   2.921   0.0614 .
#> D            2.03010    7.25859   0.280   0.7979
#> A:B          0.07305    0.19649   0.372   0.7348
#> A:C          0.12173    0.29728   0.409   0.7097
#> A:D         -0.30140    0.60378  -0.499   0.6520
#> B:C          0.02521    0.78468   0.032   0.9764
#> B:D          1.32527    2.55502   0.519   0.6398
#> C:D         -3.05593    3.79029  -0.806   0.4791
#> A:B:C       -0.02192    0.05839  -0.375   0.7324
#> A:C:D        0.19428    0.24916   0.780   0.4924
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.8761 on 3 degrees of freedom
#> Multiple R-squared:  0.9999, Adjusted R-squared:  0.9996
#> F-statistic:  2877 on 12 and 3 DF,  p-value: 9.499e-06
</code>
Created on 2024-01-09 with reprex v2.0.2


Can I build an abstract visitor using a typelist?

Question: So I have an abstract class that is the base of any visitor that using a collection of objects:
class visitor
{
  virtual void visit(foo) = 0;
  virtual void visit(bar) = 0;
  virtual void visit(baz) = 0;
  virtual void visit(quux) = 0;
};

But I also have all of these types saved in a boost::mpl::list:
using type_list = boost::mpl::list<
  foo,
  bar,
  baz,
  quux
>;

It seems like a shame to have to update the visitor class whenever I add a new type... is there any way I can generate the visitor class automatically by passing it the type_list?
std::enable_if and boost::mpl::find seems to be the way to go, but I can't template virtual methods can I?
Answer: You can't variadic template virtual methods, but you can template base classes:
template<class T> class singleVisitorBase { virtual void visit(T) = 0; };

template<class list> class visitor;
template<class... Ts> class visitor<boost::mpl::list<Ts...>>
    : singleVisitorBase<Ts>... {};

Another approach is to use recursive inheritance:
template<class list> class visitor;
template<> class visitor<boost::mpl::list<>> {};
template<class T, class... Ts> class visitor<boost::mpl::list<T, Ts...>>
    : singleVisitorBase<T>, visitor<boost::mpl::list<Ts...>> {};

However, I prefer single-level multiple inheritance as it produces cleaner type hierarchies that are easier to inspect in a debugger.
In either of these cases there is some space overhead as visitor will contain n vtable pointers into its vtable, to allow for cast-to-base-type. An alternative with less space overhead but significant compile time overhead is to use Boost.Preprocessor to generate an n-length class body via BOOST_PP_REPEAT:
template<class list, std::size_t N> class visitorImpl;
#define VISITOR_MAX 32
#define VISITOR_VISIT(z,I,_) \
    virtual void visit(typename boost::mpl::at<list, I>::type) = 0;
#define VISITOR_CLASS(z,N,_) \
template<class list>         \
class visitorImpl<list, N>   \
{                            \
    BOOST_PP_REPEAT(N, VISITOR_VISIT, ) \
};
BOOST_PP_REPEAT(VISITOR_MAX, VISITOR_CLASS, )
template<class list> using visitor
    = visitorImpl<list, typename boost::mpl::size<list>::type::value>;

Comment: @SamKellett in typical implementations it should be constant, although there will be some space overhead (20+ vtable pointers into the same vtable); see http://tinydrblog.appspot.com/?p=89001
Comment: is the performance penalty of inheritance a linear or constant thing? like the idea of a class having 20+ parents is a bit scary


Translating array pointer access from C++ to Delphi

Question: I'd like to know if I translated a piece of code correctly from C++ to Delphi.
It looks like it is working, but I have a feeling that I'm reading and writing into memory that I'm not supposed to using Delphi.
Given C++ code:
struct tile_map
{
    int32 CountX;
    int32 CountY;

    uint32 *Tiles;
};

inline uint32
GetTileValueUnchecked(tile_map *TileMap, int32 TileX, int32 TileY)
{
    uint32 TileMapValue = TileMap->Tiles[TileY*TileMap->CountX + TileX];
    return(TileMapValue);
}

uint32 Tiles00[9][17] =
    {
        {1, 1, 1, 1,  1, 1, 1, 1,  0, 1, 1, 1,  1, 1, 1, 1, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1},
        {1, 1, 1, 1,  1, 1, 1, 1,  1, 1, 1, 1,  1, 1, 1, 1, 1},
    };
// More tile map declarations ...
// uint32 Tiles01[9][17] = ...
// uint32 Tiles10[9][17] = ...
// uint32 Tiles11[9][17] = ...

    tile_map TileMaps[2][2];
    TileMaps[0][0].CountX = 17;
    TileMaps[0][0].CountY = 9;
    TileMaps[0][0].Tiles = (uint32 *)Tiles00;

    TileMaps[0][1] = TileMaps[0][0];
    TileMaps[0][1].Tiles = (uint32 *)Tiles01;

    TileMaps[1][0] = TileMaps[0][0];
    TileMaps[1][0].Tiles = (uint32 *)Tiles10;

    TileMaps[1][1] = TileMaps[0][0];
    TileMaps[1][1].Tiles = (uint32 *)Tiles11;

// Usage
    int32 PlayerTileX = 2;
    int32 PlayerTileY = 2;
    uint32 TileMapValue = GetTileValueUnchecked(&TileMap[1][1], PlayerTileX, PlayerTileY);

Delphi translation:
program Project1;

{$APPTYPE CONSOLE}

type
    Puint32 = ^uint32;

    tile_map = record
        CountX : int32;
        CountY : int32;

        Tiles : Puint32;
    end;
    Ptile_map = ^tile_map;

{$POINTERMATH ON}
function GetTileValueUnchecked(TileMap : Ptile_map; TileX, TileY : int32) : uint32; inline;
begin
    result := TileMap^.Tiles[TileY * TileMap^.CountX + TileX];
end;

const //in the future these will be read from file, so const for now
    Tiles00:  array [0..8, 0..16] of uint32 =
    (
        (1, 1, 1, 1,  1, 1, 1, 1,  0, 1, 1, 1,  1, 1, 1, 1, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0,  0, 0, 0, 0, 1),
        (1, 1, 1, 1,  1, 1, 1, 1,  1, 1, 1, 1,  1, 1, 1, 1, 1)
    );
    // More tile map declarations ...
    //Tiles01:  array [0..8, 0..16] of uint32 = ...
    //Tiles10:  array [0..8, 0..16] of uint32 = ...
    //Tiles11:  array [0..8, 0..16] of uint32 = ...
var
    TileMaps : array [0..1, 0..1] of  tile_map;
    PlayerTileX, PlayerTileY : int32;
    TileMapValue : uint32;
begin

    TileMaps[0][0].CountX := 17;
    TileMaps[0][0].CountY := 9;
    TileMaps[0][0].Tiles := Addr(Tiles00);

    TileMaps[0][1] := TileMaps[0][0];
    TileMaps[0][1].Tiles := Addr(Tiles01);

    TileMaps[1][0] := TileMaps[0][0];
    TileMaps[1][0].Tiles := Addr(Tiles10);

    TileMaps[1][1] := TileMaps[0][0];
    TileMaps[1][1].Tiles := Addr(Tiles11);

    // Usage
    PlayerTileX := 2;
    PlayerTileY := 2;
    TileMapValue = GetTileValueUnchecked(@TileMaps[1][1], PlayerTileX, PlayerTileY);
end.

Comment: This really belongs on the code review SE site. I suggest you re-post there. Using `@` is more idiomatic that `Addr`. The code looks good to me though. You've got the concepts just right. I haven't checked that you got all the indices correct. You can do that.
Comment: It did not occur to me to use the @ on a constant, thanks.
I was not aware that there was a code review SE site. I think I have to wait 90 min before I can post there.
Comment: If you are in a hurry you could write some unit tests for your code.  You should be able to satisfy yourself you have it right fairly easily, I would have thought.
Comment: As I said, it seems to work, I was just curious to know if what I did was considered to be correct or not.
Comment: Looks fine to me too. But I prefer Addr, sometimes, if it is not very certain, or only doable with lots of parentheses, to which part of an expression the @ applies.  The Addr() function-like syntax makes this much clearer, IMO.
Comment: @Rudy `@Tiles00` is looking pretty unambiguous from here.  ;-)  And `@` is just a unary operator with a defined precedence and associativity. If you can't remember off the top of your head what those are, there is nothing stopping you adding parens to make it clear. You can perfectly well write `@(arr[i])[j]` just as you write `addr(arr[i])[j]`.
Comment: @David: I didn't mean here, just in general. Sometimes Addr() can make expressions more readable, IMO I find `Addr(a[I])[13]` more readable than `(@a[i])[13]`, which look a lot like `@a[i][13]`.
Comment: @RudyVelthuis It's the parens that do that, which you can use anyway. Once you have the parens then it's 50/50 in my view.
Answer: David Heffernan's comments has been helpful and others seem to agree that the code is correct, so I will mark this as answered.


How to use official documentation to figure out what namespace to use

Question: I'm attempting to create a taxonomy term inside a Form plugin like so
<code>$term = Term::create([
    'name' => 'my taxonomy term',
    'vid'  => 'vocabulary_to_put_it_in',
]);
$term->save();
</code>
When I do so I receive an error

Fatal error: Class 'Drupal\my_module\Form\Term' not found

My gut is telling me that I'm not including the namespace like I should but I'm not sure how to actually figure out how to discover that. I'm creating nodes by using
<code>\Drupal\node\Entity\Node::create
</code>
And I receive no errors for that. I'm assuming that it's because I'm using \Drupal\node\Entity\ before it, but I was only able to figure that out by looking at examples. I looked at this page https://api.drupal.org/api/drupal/core%21modules%21taxonomy%21src%21Controller%21TaxonomyController.php/class/TaxonomyController/8.1.x and saw the class hierarchy class \Drupal\taxonomy\Controller\TaxonomyController but when using
<code>\Drupal\taxonomy\Controller\TaxonomyController\Term::create
</code>
I still get an error saying

Fatal error: Class
  'Drupal\taxonomy\Controller\TaxonomyController\Term' not found

If it is true, that the reason why it is failing, is because I'm not including the right namespace, then how do I use the official documentation to figure out what I should be including? Or is there something else I'm missing entirely?
Answer: I was able to discover the namespace by finding the documentation for the Term class here.
https://api.drupal.org/api/drupal/core%21modules%21taxonomy%21src%21Entity%21Term.php/class/Term/8.1.x
And on that page there's a namespace section that said Drupal\taxonomy\Entity so I used
<code>\Drupal\taxonomy\Entity\Term::create
</code>
and that worked.
So I believe in order to find the right namespace to use, you have to find the page for the class that is using the method. In this case Term. And then find the namespace section and put that before the class when you are using a method of it.
(This might be obvious to everyone else, but it took me a bit to figure out)
Comment: Or get a decent IDE, type `Term`, and look at the auto completed results to decide which is the correct class to use. It will then even add the `use` statement for you. That's much quicker and more intuitive than trying to find pages in online documentation
Comment: Yes, IDEs like PHPStorm will autocomplete. D8 dev with no IDE will take a lot longer.


$100$ numbers are written around the circle

Question:
$100$ positive numbers are written in a circle. The sum of any two
neighbors is equal to the square of the number following them
clockwise. Find all such sets of numbers.

Let us introduce the notation $n=100$, not forgetting that $n$ is even. Let the numbers start in clockwise order $a_{1}, a_{2}, \ldots, a_{n}$. The sum of the conditions of the problem gives $(a_{1}-1)^{2}+(a_{2}-1)^{2}+\ldots+(a_{n}-1)^{2}=n$.
Suppose that two adjacent numbers can simultaneously be at least $2$, and at least one of them is strictly greater than $2$. Then all subsequent numbers in the circle are strictly greater than $2$, which contradicts the obtained equality. The case is considered similarly when two adjacent numbers can simultaneously be no more than $2$, and at least one of them is strictly less than $2$.
Thus, the numbers "alternate": no less than two and no more than two. Suppose WLOG that $a_{1}+a_{2} \geq 4$. Then
$$ \begin{gathered} a_{1}+a_{2} \geq 4, a_{3}+a_{4} \geq 4, \ldots, a_{n-1}+a_{n} \geq 4, \\ 4 \geq a_{2}+a_{3}, 4 \geq a_{4}+a_{5}, \ldots, 4 \geq a_{n}+a_{1}. \end{gathered} $$
Adding all the inequalities, we get the inequality $0≥0$. Therefore,
$$ a_{1}+a_{2}=a_{3}+a_{4}=\ldots, a_{n-1}+a_{n}=a_{2}+a_{3}=a_{4}+a_{5}=\ldots=a_{n}+a_{1}=4. $$
So all numbers are equal to two.
QUESTION: If we add up all the inequalities and get the identity, then we cannot draw any conclusions beyond the fact that this situation is possible. How was the consequence obtained that the sums of all pairs are equal to $4$?
Comment: If you added the inequalities a different way, you would get $a_1+a_2+4 \ge 4+a_2+a_3$ etc leading to $a_1\ge a_3\ge \ldots \ge a_1$ and $a_2\ge a_4\ge \ldots \ge a_2$ so the alternation involves equality.  This means  $a_1+a_2=4$ and you just need to check $(1,3), (2,2), (3,1)$ of which only one works
Comment: @Henry I do not mind. One can deduce that even/odd subsequences consist of equal numbers and then work with this. But you must admit, this is a slightly different reasoning.
Comment: @Henry Note that the numbers are just positive, and need not be integers. $\quad$ As an aside, once we get $ a_i + a_{i+1} = 4$ recall that this is also $a_{i+2} ^2 $, so $a_{i+2} = 2 $ directly (without having to check any cases).
Answer: Here is my explanation of the solution.
To me, it's a case where "minor details are skipped", as opposed to Paul's "big jump / stopped trying to explain / logic leap".
However, the exposition could be significantly improved on by stating where the condition $a_i+a_{i+1} = a_{i+2}^2$ was repeatedly used.

Let us introduce the notation $n=100$, not forgetting that $n$ is even. Let the numbers start in clockwise order $a_{1}, a_{2}, \ldots, a_{n}$. The sum of the conditions of the problem gives $(a_{1}-1)^{2}+(a_{2}-1)^{2}+\ldots+(a_{n}-1)^{2}=n$.

The conditions are $ a_i + a_{i+1} = a_{i+2}^2$. Summing all of them and completing the squares gives us the equation.

Suppose that two adjacent numbers can simultaneously be at least $2$, and at least one of them is strictly greater than $2$. Then all subsequent numbers in the circle are strictly greater than $2$, which contradicts the obtained equality.
The case is considered similarly when two adjacent numbers can simultaneously be no more than $2$, and at least one of them is strictly less than $2$.

With the assumptions, $ a_{i+2} = \sqrt{ a_i + a_{i+1} } > 2$.
Proceed via induction to conclude that $ a_j > 2 \forall j$.
Likewise for the other case.

Thus, the numbers "alternate": no less than two and no more than two. Suppose WLOG that $a_{1}+a_{2} \geq 4$. Then

Hence if $ a_i \geq 2$, then $ a_{i+1} \not > 2 $ so $ a_{i+1} \leq 2$.
Likewise, $ a_{i+2} \not < 2$ so $ a_{i+2} \geq 2$.
Suppose WLOG that $a_3 \geq 2$, then $a_1 + a_ 2 = a_3 ^2 \geq 4$.
From the previous statement, $a_{2i+1 } \geq 2, a_{2i} \leq 2$.

$$ \begin{gathered} a_{1}+a_{2} \geq 4, a_{3}+a_{4} \geq 4, \ldots, a_{n-1}+a_{n} \geq 4, \\ 4 \geq a_{2}+a_{3}, 4 \geq a_{4}+a_{5}, \ldots, 4 \geq a_{n}+a_{1}. \end{gathered} $$

This follows because $ a_{2i+1} + a_{2i+2} = a_{2i+3} ^2 \geq 2^2 = 4$ and $ a_{2i} + a_{2i+1}  = a_{2i+2} ^2 \leq 2^2 = 4$.

Adding all the inequalities, we get the inequality $0≥0$. Therefore,
$$ a_{1}+a_{2}=a_{3}+a_{4}=\ldots, a_{n-1}+a_{n}=a_{2}+a_{3}=a_{4}+a_{5}=\ldots=a_{n}+a_{1}=4. $$

Adding all of the inequalities, we get that $ 2n + \sum a_j \geq 2n + \sum a_j$, which means that equality holds throughout. (If equality doesn't hold throughout, then at least one of the inequalities is strict, which means that the summed inequality is strict, which is a contradiction.)
Hence, $ a_i + a_{i+1} = 4$.
Note: The solution writer might have intended to sum $a_{2i+1} + a_{2i+2} - 4 \geq 0$ in order to get the $ 0 \geq 0$.

So all numbers are equal to two.

Hence $ a_{i+2} = \sqrt{ a_i + a_{i+1} } = \sqrt{4} = 2$.
Answer: There are a number of not-obvious large steps in this deduction. And they certainly messed up the part you are asking about.

Let us introduce the notation $n=100$, not forgetting that $n$ is even. Let the numbers start in clockwise order $a_1, a_2, \ldots, a_n$. The sum of the conditions of the problem gives $(a_1-1)^2+(a_2-1)^2+\ldots+(a_n-1)^2=n$.

This is an easy enough calculation. I think it would have wise to add that the indexing is modulo $n$, so $a_{n+1}$ is $a_1$.

Suppose that two adjacent numbers can simultaneously be at least $2$, and at least one of them is strictly greater than $2$. Then all subsequent numbers in the circle are strictly greater than $2$, which contradicts the obtained equality. The case is considered similarly when two adjacent numbers can simultaneously be no more than $2$, and at least one of them is strictly less than $2$.
Thus, the numbers "alternate": no less than two and no more than two.

I think this could have been more clearly stated. I had to read that last line several times before I could parse what they were saying. But it is just that if $a_k \le 2$, then $a_{k+1} \ge 2$, and if $a_k \ge 2$, then $a_{k+1} \le 2$.

Suppose WLOG that $a_1+a_2 \geq 4$. Then
$$ \begin{gathered} a_1+a_2 \geq 4, a_3+a_4 \geq 4, \ldots, a_{n-1}+a_n \geq 4,\\
4 \geq a_2+a_3, 4 \geq a_4+a_5, \ldots, 4 \geq a_n+a_{1}. \end{gathered} $$

This is a big jump in my opinion that should have more explanation. If $a_1 + a_2 \ge 4$, then $a_3^2 \ge 4$, so $a_3 \ge 2$. This requires $a_4 \le 2$, and so $a_2 + a_3 = a_4^2 \le 4$. A similar argument then shows $a_3 + a_4 \ge 4$, etc.

Adding all the inequalities, we get the inequality $0\ge 0$.

I guess they are summing $a_{2k} + a_{2k+1} - 4 \le 0$ and $a_{2k-1} + a_{2k} -4 \ge 0$ to get $$0 \le \sum_i a_i - 2n \le 0$$
But wow, it is like they stopped even trying to explain anything.
So $\sum_i a_i = 2n$. If even one of the $a_{2k} + a_{2k+1} < 4$, then $\sum_{k=1}^{n/2} (a_{2k} + a_{2k+1}) < 4$, a contradiction. Thus every $a_{2k} + a_{2k+1} = 4$. Similarly every $a_{2k-1} + a_{2k} = 4$.

Therefore,
$$a_1+a_2=a_3+a_4=\ldots=a_{n-1}+a_n =a_2+a_3=a_4+a_5=\ldots=a_n+a_1=4.$$
So all numbers are equal to two.

Once again, another logic leap. There are several ways you can deduce all numbers are $2$ from this, but it is not immediately obvious and takes a bit of chasing to pin down.

I suggest diverging from their argument after they show that the terms alternate being $\ge 2$ and $\le 2$.
Note that if any $a_k = 2$, then if $a_{k + 1} < 2$ or if $a_{k+1} > 2$, then you have a forbidden arrangement per their argument. Thus $a_k =2 \implies a_{k+1} = 2$, which then propagates all around the circle.
So we have only two possible cases: either for all $k, a_k = 2$ or for all $k, a_k \ne 2$.
All $a_k = 2$ is easily verified as a solution.
So now assume all $a_k \ne 2$. WLOG, $a_1 > 2$. It follows that $a_k > 2$ when $k$ is odd and $a_k < 2$ when $k$ is even. But then $$a_{2k} + a_{2k+1} = a_{2k+2}^2 < 4$$ and $$a_{2k-1} + a_{2k} = a_{2k+1}^2 > 4.$$
Summing the first inequality over all $k \le\frac n2$ gives
$$\sum_i a_i < 2n$$
Summing the second inequality over all $k \le \frac n2$ gives
$$\sum_i a_i > 2n$$
which cannot be. Thus there are no solutions with $a_k \ne 2$.
Comment: I disagree with "another logic leap" at the end. It follows immediately from $ a_3 ^2 = a_1 + a_2 = 4 \Rightarrow a_3 = 2$.   $\quad$ Such solution write-ups where the reader is expected to fill in steps (with some thinking / playing around) is not that uncommon in quick solutions to olympiad questions. Finding the "right steps" can be crucial. $\quad$ Once we get that the terms alternate, we could condense your steps to just $2n \leq \sum a_{2i} ^2 =  \sum a_j = \sum a_{2i+1} ^2 \leq 2n$, hence must have equality throughout, so $a_i = 2$.
Comment: Likewise for the "big jump in my opinion", it is actually just a small step: Since the numbers alternate, WLOG $a_{2i+1} \geq 2, a_{2i} \leq 2$. This means that $a_{2i} + a_{2i+1} = a_{2i+2} ^2  \leq 4$ and $ a_{2i+1} + a_{2i+2} = a_{2i+3} ^2 \geq 4$.
Comment: @CalvinLin - I don't see where it says this is a contest solution submitted to judges. How one describes mathematics is of course subject to the audience. When you are speaking to those who already know the solution, and are only trying to convince them that you have figured it out too, you don't need to explain more than a minimum. On the other hand, when you are explaining something to someone not already familiar with it, or even more, trying to convince someone that what you have is correct, then these omissions are not wise.
Comment: To clarify, I'm making the assumption that this is "a quickly stated solution where minor details are skipped".    $\quad$ I'm not saying that this is a "contest solution submitted to judges". If that is the case, then this is horrendously explained, and might not get full credit.  $\quad$ I was pointing out that what you called a leap/big jump (viewed via how you competed the solution) could be completed via an alternative one-line approach.
Comment: @CalvinLin - A one-line approach to cover the logic leap does not stop it from being a logic leap unless that one line is something that should be immediately obvious to the target audience. That every $4k+1$ prime is the sum of two squares can be proved in one line (Zagier), but that line is a doozy that requires a fair bit of interpretation. I do not agree that your one line is sufficiently obvious as to allow its exclusion for the apparent target audience (though I admit to guessing here as who that is).
Comment: I agree with you that " I do not agree that your one line is sufficiently obvious as to allow its exclusion for the apparent target audience", which is why I said that if this solution was submitted to the judges, it might not get full credit (esp if I was grading it). One possible interpretation of this writeup is that "The solution has gaps of wishful thinking, though it lays out a potential path."


Test if one Poisson process's rate is smaller than a number

Question: Suppose $N(t)$ is a Poisson process with rates $\lambda$. Suppose I've been observing it for $t \in [0, T]$ and recorded events. How can I test the null hypothesis $\lambda < \lambda_0$, where $\lambda_0$ is a number?
An obvious solution is to derive MLE of rates $\hat\lambda$ and compare with $\lambda_0$. But I don't know how to calculate the statistical significance then. Any idea?
Answer: $N(t)$ follows a $Poi(\lambda T)$ distribution.  The generalized likelihood ratio test is $\frac{\max_{\lambda \geq \lambda_0} p(n;\lambda T)}{\max_{\lambda < \lambda_0} p(n;\lambda T)}$ where $p(n;\lambda)$ denotes a $Poi(\lambda T)$ PMF and $n$ is the number of counts you get, and you declare $H_0$ if this is below some threshold and $H_1$ if its above the threshold.
Now, the MLE for a poisson distribution with one sample is simply $\hat{\lambda} = n$. So, depending on the value of $n$ relative to $\lambda_0 T$, the numerator or denominator will be $p(n;n)$. Then, note that $(\log p(n;\lambda))' = -1 + n/\lambda$ where the derivative is wrt $\lambda$, so the one of the numerator/denominator which isn't $p(n;n)$ will be $p(n;\lambda_0 T)$.
So, your test statistic will be either $p(n;n)/p(n;\lambda_0 T)$ its reciprocal depending on if $n> \lambda_0 T$ or not. So, you can calculate everything based on this.
Comment: Thanks. Can you elaborate on the first two sentences of the second paragraph here? Sorry kind of a newbie.
Comment: Assume the number of events $n$ satisfies $n \geq \lambda_0 T$. Then, if you solve the maximization in the numerator, you will get $p(n;n)$. This is because maximizing $p(n;\lambda T)$ over $\lambda$ without any constraints is going to give you $p(n;n)$. Similarly, if $n < \lambda_0 T$, you'll have this in the denominator.
Comment: But then the test statistics will be a value, not a random variable? How to proceed to calculate p-value there?
Comment: $n$ is what you observe -- and your observed values of $n$ will follow a $Poi(\lambda T)$ distribution.


Variables Needed for Cox Regression (Survival Analysis)

Question: I'm preparing a longitudinal dataset (with up to 5 observations per participant) for Cox regression in R. I have data for the follow-up period and date (FUPeriod and FU, respectively), the date of hospital discharge (HospDis) and the date of death (Death; if applicable).

ID
FUPeriod
HospDis
FU
Death

1
0
2017-09-26
NA
NA

1
1
2017-09-26
2017-11-16
NA

1
2
2017-09-26
2019-02-12
NA

1
5
2017-09-26
2021-09-10
NA

1
10
2017-09-26
NA
2022-02-20

I'm a little stuck on the variables that I need to create from the available temporal data to start my analyses... I know that at the very least I need a censoring/event indicator variable and a survival time variable. My question is whether the survival time and censoring variables need to have values at all time points (i.e., FUPeriod = 0, 1, 2, 5, 10), or whether they need values only for the last available time point (i.e., FUPeriod = 10)? Are these the variables and values I should ultimately have (where SurvTime is survival time since hospital discharge (HospDis) in months and Event = 1 if Death is a valid date and 0 otherwise?

ID
FUPeriod
HospDis
FU
Death
SurvTime
Event

1
0
2017-09-26
NA
NA
0
0

1
1
2017-09-26
2017-11-16
NA
1.675565
0

1
2
2017-09-26
2019-02-12
NA
16.55852
0

1
5
2017-09-26
2021-09-10
NA
47.47433
0

1
10
2017-09-26
NA
2022-02-20
52.82957
1
Comment: You only need one row for each ID. If the person is censored, time will be the time of their censoring, and if they died, time will be the time of their death
Answer: The answer depends on whether you are modeling covariates whose values change over time, and whether you are building a continuous-time or a discrete-time survival model.
If your model only uses covariate values in place at study entry (SurvTime = 0), then the comment by @user352188 is correct. The data format in that case just needs a single row per individual, which includes the covariate values at SurvTime = 0, the last follow-up time, and whether the last follow-up was an event time (as opposed to right-censoring).
If your model includes covariates whose values change over time, you need to have multiple data rows per individual. Each data row then includes for that individual a start time, a stop time, the covariate values in place during that time interval, and an indicator of whether or not the stop time is for an event. That's called the "counting process" data format.
The above assumes that you have a continuous-time model of survival. The vignettes of the R survival package go into more detail, with examples.
Sometimes there are only data at a small number of fixed times, say every year for 5 years (panel data). Such data are analyzed with discrete-time survival methods, a set of binomial regressions at each fixed time. In that case it's simplest to have a separate data row for each time point at which the individual is still at risk.


Number of expected pairs in a random shuffle

Question: Suppose there are 34 poeople standing in a row in random order, among them 18 are male and 16 are female. If two people adjacent to each other belong to different genders, we consider them to be a couple, how many couples are we expected to see on average?
Comment: Good. That's probably easier, because now your problem is equivalent to the number of *changes* in a random sequence of 18 Ms and 16 Fs.
Comment: Is MFM one couple and one person left over, or two couples (one F having two Ms)?
Comment: Employ indicator random variables.  One candidate would be the indicator of the gender of the person behind a female, for instance.
Comment: They are considered two couples.
Answer: First, we find the probability that two adjacent individuals are of different genders.  Those two people are equally likely to be   any of the ${34 \choose 2}$ pairs of people, of which $18 \times 16$ are male-female pairs, so this probability is $(18 \times 16)/{34 \choose 2}$.
The total number of pairs is $X_1 + X_2 + \cdots + X_{33}$ where $X_i$ is an indicator random variable that is 1 if person $i$ and person $i+1$ are of opposite genders and 0 otherwise.  Its expectation is $E(X_1) + \cdots + E(X_{33})$, but all these variables have the same expectation, the probability we found above. So the answer is
$$ 33 \times {18 \times 16 \over {34 \choose 2}} = {33 \times 18 \times 16 \over (34 \times 33)/2} = {18 \times 16 \over 17} = {17^2 - 1 \over 17}  = 17 - {1 \over 17} \approx 16.94.$$
This agrees with Bernhard's simulation.
More generally, if you have $m$ males and $f$ females and the same problem you get
$$ (m+f-1) {mf \over {m+f \choose 2}} = {(m+f-1) mf \over (m+f)(m+f-1)/2} = {2mf \over m+f}$$
which can  also be checked by simulation.
Comment: Lots of practice.  Seriously, though, if you're only looking for the expected value then using indicator random variables is usually the way to go.  (You can derive the variance this way as well but it's more work - you end up having to consider two indicators at a time.)
Comment: This is a really nice solution. I **did not** expect the answer to be the *harmonic mean* of $m$ and $f$, but there it is!
Comment: Even more generally, one should look for any excuse to use linearity of expectation—which means that writing a random variable as a complicated sum of simple random variables is usually a productive way to find the expectation.
Comment: @Adrian - go ahead! Sadly I can't actually see the app as an iOS user, but from what I  can see at the link it looks like a nice bunch of puzzles!
Comment: Amazing derivation. How did you avoid getting trapped into thinking about drawing random samples without replacement?
Comment: Would you mind if I include a link to this answer in the hint for https://github.com/atorch/probability_puzzles/blob/master/app/src/main/res/values/strings.xml#L651 ?
Answer: Someone much wiser then me will post a theoretical and exact solution. Meanwhile my attempt at a simulation:
<code>once <- function(){
    row <- sample(c(rep("m", 18), rep("f",16)))
    count <- 0
    for(i in 1:33){
        if(row[i]!=row[i+1])
        count <- count + 1
    }
    return(count)
}

run <- replicate(1e5, once())
plot(table(run))
</code>

<code>> table(run)
run
    6     7     8     9    10    11    12    13    14    15    16    17    18    19    20    21    22    23    24    25    26    27    28    29
    5    25    79   280   663  1643  3070  5555  8057 11169 12883 14309 12818 11020  7682  5279  2931  1535   635   261    78    17     4     2

> summary(run)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
   6.00   15.00   17.00   16.96   19.00   29.00
</code>
Comment: @XiaowenLi: If you have a job interview definitely outline a simulation approach too. (And I hope it goes well of course!)
Comment: Thanks for the simulation. Unfortunately this is a job interview question. If I could run simulations many problems could be in solved very easily.
Answer: Alternative solution :
Imagine the 34 people form a circle instead of a row.
Then for any given female A :
E( # couples A is in) =
E( # righthand couples A is in) + E( # lefthand couples A is in) =
P(to A's right is a man) + P(to A's left is a man) =
m/(m+f-1) + m/(m+f-1) =  2m/(m+f-1)
So E( total # of couples) = f * E( # couples A is in) = 2mf/(m+f-1)
This is the expectation when the people form a circle. To form a row the circle is cut in a random place. The  number of potential couples decreases from (m+f) to (m+f-1), i.e. with a factor (m+f-1)/(m+f) and the expected number of couples decreases proportionally:
(2mf/(m+f-1)) * (m+f-1)/(m+f) = 2mf/(m+f)


Creating a Generic Array

Question: Im having a big problem with this code: I need to create my own Merge Method without using java´s Merge Method.
public static <T extends Comparable> T[] merge(T[] a, T[] b){
    T[] c = (T[]) new Object[a.length + b.length];

    /*
     *
     * More code
     *
     *
    */
    return c;
}

The problem is that in runtime I have this error:
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [Ljava.lang.Comparable;
I have already looked for this problem, and the solution is using Reflection.
The problem i have it in this line: T[ ] c = (T[ ]) new Object[a.length + b.length];
The problem is that I dont understand how to use it, have already try using reflection in many different ways but i can´t solve this.
I really appreciate any answer and tell me where should I change my code.
Comment: (rawtypes! Should be `>`, or better `>`.)
Comment: How about `Arrays.copyOf`? http://stackoverflow.com/a/8220245/303939
Answer: You could create an array of the same type as the input arrays, using reflection - something like this:
if(a.getClass() != b.getClass()) // or .equals if you prefer. It doesn't matter for Class objects
    throw new IllegalArgumentException("Arrays don't have the same type");

T[] c = Array.newInstance(a.getClass().getComponentType(), a.length + b.length);

Note that it is impossible to use something like T.class. Note also that this won't work as expected for cases like:
Comparable[] result = <Comparable>merge(new Integer[] {1, 2, 3}, new Integer[] {4, 5, 6});

as the result will be an Integer[] instead of a Comparable[]. However, it is the closest you can do without passing Comparable.class to the function, or pre-allocating the result array before calling it.
Or you could use Arrays.copyOf as suggested by Ming-Tang, and then fill the array with null (otherwise it starts with a copy of a):
if(a.getClass() != b.getClass()) // or .equals
    throw new IllegalArgumentException("Arrays don't have the same type");

T[] c = Arrays.copyOf(a, a.length + b.length);
Arrays.fill(c, null);

This has the same issues.
Comment: you need to cast the result of `Array.newInstance()` to `T[]`
Comment: Because of the bizarre and unnatural behaviour of arrays, you should choose the "greatest common superclass". (Code for which is left as an exercise for the interested reader.) When I say "should", you should follow @immibis' advice and use `List`.
Comment: @TomHawtin-tackline Note that I wrote both answers, because either one could be best for the OP's application. (Although one would certainly hope, for his/her sanity, that the List one works)
Comment: Heh, didn't realise it was the same person. Reference arrays were badly messed up (not that `System.arraycopy` is a nice way of dealing with arrays).
Answer: You could just use List instead.
public static <T extends Comparable> List<T> merge(List<T> a, List<T> b){
    List<T> c = new ArrayList<T>(a.size() + b.size());

    /*
     *
     * More code
     *
     */
    return c;
}

This might not be an option if some other part of your code forces you to use arrays in merge.

Creating a Generic Array