Personalizar nome de campo nas mensagens de erro de validação

Question: Estou utilizando o Controlller padrão de autenticação do Laravel 5.2:
php artisan make:auth

Como os nomes estão em inglês, preciso alterar para o português. Já configurei o locale para português, em /config/app.php:
'locale' => 'pt-BR',

Alterei o AuthController.php conforme segue:
protected function validator(array $data)
{
    $validator = Validator::make($data, [
        'name' => 'required|max:255',
        'email' => 'required|email|max:255|unique:users',
        'password' => 'required|confirmed|min:6',
    ]);
    $correct_names = [
        'name'      => 'Nome',
        'email'     => 'E-mail',
        'password'  => 'Senha',
    ];

    return $validator->setAttributeNames($correct_names);
}

Ao criar um usuário, utilizando http://localhost:8000/register, o nome dos campos ficam corretos, contudo na página de login, http://localhost:8000/login, o nome dos campos continuam em inglês (email and password).
Alguém pode dar uma dica como resolver isso?
Answer: Outra forma de customizar o nome do campo é diretamente no seu arquivo do language, alterando a chave attributes
/*
|--------------------------------------------------------------------------
| Custom Validation Attributes
|--------------------------------------------------------------------------
|
| The following language lines are used to swap attribute place-holders
| with something more reader friendly such as E-Mail Address instead
| of "email". This simply helps us make messages a little cleaner.
|
*/

'attributes' => [
    'email'    => 'E-mail',
    'password' => 'Senha',
],

Dessa forma todas os formulários de erro que conterem esses atributos serão renomeados.
Answer: O código que implementa o método (função) de validação de usuários, tanto no login quando na criação de novos usuários, pode ser substituído no arquivo.
O método de validação da tela de login é validateLogin(), localizado em:

/vendor/laravel/framework/src/Illuminate/Foundation/Auth/AuthenticatesUsers.php

Para substituir esse método, insira essa mesma função no arquivo:

/app/Http/Controllers/Auth/AuthController.php

Com as alterações necessárias, no caso em questão:
protected function validateLogin(Request $data)
{
    $this->validate($data, [
            $this->loginUsername()  => 'required',
            'password'              => 'required'
        ],[],[
            $this->loginUsername()  => 'E-mail',
            'password'              => 'Senha',
        ]
    );
}

Espero ter ajudado!
Answer: Essa personalização é bastante simples, você precisará somente implementar mais um array para introduzir as mensagens de erro personalizadas, por exemplo:
protected function validator(array $data)
{
    $validator = Validator::make($data, [
        'name' => 'required|max:255',
        'email' => 'required|email|max:255|unique:users',
        'password' => 'required|confirmed|min:6',
    ], [
        'required' => 'O campo :attribute é obrigatório',
    ], [
        'name'      => 'Nome',
        'email'     => 'E-mail',
        'password'  => 'Senha',
    ]);

    return $validator;
}

Comment: A forma descrita acima é idêntica a que tenho na pergunta, porém funciona apenas no formulário *register* e não no formulário *login*, quando, por exemplo, deixa-se o campo E-mail e Senha em branco e aperta 'Login', aparecerá o nome do campo como o original do sistema.
Comment: Sei que se alterar a função *validateLogin* no arquivo *AutheticatesUsers.php*, em *Illuminate\Foundation\Auth* e colocar da forma como postou: *$this->validate($request, [$this->loginUsername() =>'required','password' => 'required'], ['required'    => 'O campo :attribute é obrigatório!'], [$this->loginUsername()  => 'Usuário','password'=> 'Senha',]);* vai funcionar.. mas acho que essa não é a melhor saída, alterando o código interno do laravel.... =(


Can a 24V 1Gbps Ethernet port start operating in 100Mbps mode if it was exposed to 48V PoE?

Question: I have heard that a situation where a 24V Gbit Ethernet port was not completely fried after being exposed to 48V current, but downgraded to fast Ethernet - 100Mbps.
Question: I that possible and how?
Answer: POE over gigabit and POE over fast ethernet have the potential to be different.
Ethernet over twisted copper of any kind utilizes "differential signaling" - meaning that the waveform that provides data for each twisted pair is actually juxtaposed against another reference "carrier wave" rather than relying on the difference between the data signal and ground. This allows both of those waveforms to be "pushed up" by X amount of DC voltage without affecting the data signal - and consequently allows every copper conductor in a gigabit line to be used for power as well as data. This is referred to as "Alternative A POE", and is required in gigabit implementations (gigabit requires the use of all eight conducts for data, so we can't dedicate half of them to only power).
Fast Ethernet doesn't use as many conductors (half, for a total of four), so POE implementations can be run on the "other half" of the cable without employing a DC bias in the data lines. This is referred to as "Alternative B".
As far as why that situation you heard of was possible, there are many devices that can autonegotiate POE as well as accept 24V or 48V. Sometimes, that autonegotiation involves using a different 48 volt circuit that is only accessible using fast Ethernet standards, but that wouldn't be terribly common - however it's a thing I've run into in the past with some very old (circa ~2000) devices. In general, this autonegotiation should just work on modern devices regardless of speed. However, you should ALWAYS refer to the device documentation to make sure that all of your equipment will correctly auto-negotiate.
Comment: Thanks for your answer. To clarify what I meant, I should mention that the discussion was about a Switch port that does not support 48V exposed to it - effectively, what the person was saying is that some devices operate in Fast-Ethernet MAX, after being burned by 48V. Which from the information you provided would mean, that some of the power supply copper conductors got burned and there is no way for all 8 conductors to be used, so it effectively downgrades to FastEthernet? Let me know if that is correct.
Comment: The IEEE PoE standards provide for signaling between the power sourcing equipment (PSE) and powered device (PD). This signaling allows the presence of a conformant device to be detected by the power source, and allows the device and source to negotiate the amount of power required or available. Unless the switch in question is not compliant with this fairly old standard, that should never happen.
Comment: However, if you were using "alternative B" and splitting a 48 volt signal that could damage a non-compliant device, though the vast majority of those integrated over-voltage protection (and still do) even before negotiation was a standard.
Comment: @Spooler IEEE PoE works exactly the same way with 100BASE-TX or 1000BASE-T. It does't matter whether data is transmitted on the same pairs or not due to the center-tap 'phantom' power technique.
Comment: Right, I don't know why I thought fast Ethernet wasn't differentially signaled. I'll edit.
Answer: IEEE PoE ("DTE power via MDI")  only uses 48/54 V voltage and is carefully designed to not damage any legacy equipment. However, proprietary, non-standard PoE may not interoperate well with IEEE PoE and may also cause damage to incompatible equipment. You should always prominently mark these ports and keep them well apart from the rest of the network.
With proprietary variants, it's theoretically possible to fry part of the magnetic coupling - the pulse transformers - reducing the working pairs in a link. Some gigabit ports fall back to 100BASE-TX ("Ethernet@Wirespeed") when they detect non-working pairs, so yes, it's possible that a partially fried 1000BASE-T port continues to work at 100 Mbit/s. The fallback may even happen on the non-fried side, so the port may only work with certain combinations. Typically, falling back to 100BASE-TX takes a couple of seconds and the link will take more than the usual time to come up.
Using a simple multimeter on an unpowered port, you should be able to measure 2-5 Ω across each of the pairs (1-2, 3-6, 4-5, 7-8 for 1000BASE-T, with 10BASE-T and 100BASE-TX only using 1-2 and 3-6). An open circuit or 0 Ω means the transformer is fried. (The AC impedance is 100 Ω but a multimeter uses DC.)
Comment: Thanks for your answer @Zac67 . One can test whether the cable has been fried with a simple multimeter continuity test. How should I check the ports on the device to see if some of them are fried or not? Preferably no special equipment.
Comment: I wasn't referring to the cable but the port - I've added some details to my answer.


Monics and monomorphisms are the same as kernels in the additive category of R-modules.

Question: How can we show that in an additive category monics and monomorphisms are the same as kernels?
Actually, I can show that a kernel is a monic and a monomorphism but I could not show that "a monic is a kernel" and "a monomorhism is a kernel in the additive category of R- modules".
Thank you for your help.
Claudia M.
Answer: It's false in arbitrary additive categories (consider torsion-free abelian groups, here $2 : \mathbb{Z} \to \mathbb{Z}$ is monic and epic, but not iso, hence not a kernel), but it holds in the category of $R$-modules. Namely, if $U \subseteq V$ is a submodule, then it is the kernel of $V \twoheadrightarrow V/U$.


Cache field / entity / node

Question: I have a custom field/widget that has a format view function that is highly resource intensive.  I currently am caching the result using cache_set()/cache_get() inside the hook_field_formatter_view function for my field.
Is there a better way I could be doing this?  I tried to use drupal_render()'s #cache property and it cached the output but still ran my hook_field_formatter_view function.
Basically, I know how to do full page cache (for anonymous users) but how do you cache a field/node/entity (for everyone)?
Comment: This is a great question - in Drupal 6 you couldn't really (though Panels allows some granular page caching) but I don't know enough about the new Cache API to comment. Very curious to see an answer though. From a general performance note, have you see AuthCache? http://drupal.org/project/authcache
Comment: That authcache module is interesting.  It is just full page cache though (and drupal 6 only currently).
Comment: What exactly is resource intensive? Is it the loading of additional data that you need to display or the actual rendering? There is http://drupal.org/project/entitycache, but that only caches the loading of entities, not rendering..
Comment: The code inside my ``hook_field_formatter_view`` is resource intensive (not sure what that is considered).  entitycache module seems promising.
Comment: @Berdir check out my answer. There is a new module called https://drupal.org/project/render_cache which fills the void.
Answer:
I currently am caching the result using cache_set()/cache_get() inside the hook_field_formatter_view function for my field

Honestly this is not a bad way to go about doing it. When you have custom logic which is resource intensive, taking responsibility for your own caching decisions is usually the right idea. Honestly, I really wish more developers would familiarize themselves with cache_get/set().
Entitycache appears to be an effort to do this generally, and over time I am sure that some best practices will emerge as more people work in this space. However as of today, for anyone looking at a similar problem-set: "Figure out how to cache it yourself" is a good answer.
Answer: I think you should consider using Render Cache module. This when used in conjunction with Entity Cache it should complement each other well as mentioned on the project page.

Cache field / entity / node


DateTime.ParseExact() raising exception

Question: DateTime.ParseExact() raising and the error message is
The DateTime represented by the string is not supported in calendar System.Globalization.GregorianCalendar.
this way i tried to parse the date
 string strDate ="03/17/2014"
 DateTime dt= DateTime.ParseExact(strDate, "dd/MM/yyyy", CultureInfo.InvariantCulture)

so the above date is correct date then why i am getting error. what would be the best way to parse date when application run in any country and system pc date time setting can be anything. please guide me.
the moment i change the datetime parseexact to only parse like
 string strDate ="03/17/2014"
 DateTime dt= DateTime.Parse(strDate)

the above code works fine. so just tell me what was the problem in ParseExact()
thanks
Answer: You have month before day in the value "03/17/2014" i.e "MM/dd/yyyy", You get the exception because month should not be have 17 it could have values from 1 to 12. You can learn more about custom date formats here.
DateTime dt= DateTime.ParseExact(strDate, "MM/dd/yyyy", CultureInfo.InvariantCulture)

Comment: @Habib: You'd need to define "better" - looser parsing isn't always better. If you're expecting valid content to *always* have two digits, then it makes sense to throw an exception if that expectation is violated.
Comment: +1 would be even better if you use `M/d/yyyy`, since that will parse both single/double digit day / month.
Comment: @SonerGönül, it does :)
Comment: But the day suggest he might have month like 01
Comment: Yes Jon, you are absolutely right. It depends on the requirement.
Answer: There are no 17 months, use "MM/dd/yyyy" instead of "dd/MM/yyyy":
DateTime dt= DateTime.ParseExact("03/17/2014", "MM/dd/yyyy", CultureInfo.InvariantCulture)

Update:

is it mandatory that we need to specify the format. i do not want to
  specify the format then how to use it. basically the pc where
  application will run there datetime setting can be anything like
  dd/mm/yy or mm/dd/yy etc

You have a problem if it can be anything. What month is 02/03/2012, february or march?  You have to specify a string[] of allowed formats. You can give it a priority by the order in the array. Then you can use DateTime.ParseExact with overload that takes a string[].
Another approach would be to accept that you don't know the current culture but you know the string is a valid format of the current culture with  / as date-separator. Then you could clone the current DateTimeFormat and change the DateSeparator
Then you can even use DateTime.Parse:
var customDateTimeFormat = (DateTimeFormatInfo)DateTimeFormatInfo.CurrentInfo.Clone();
customDateTimeFormat.DateSeparator = "/";
DateTime dt = DateTime.Parse("17/03/2014", customDateTimeFormat);

This works for me in germany even if we use . as date-separator, i needed to use "17/03/2014" because the day comes before the month here.
Comment: is it mandatory that we need to specify the format. i do not want to specify the format then how to use it. basically the pc where application will run there datetime setting can be anything like dd/mm/yy or mm/dd/yy etc.
Comment: @Thomas: you have a problem if it can be anything. What month is `02/03/2012`, february or march?  You have to specify a `string[]` of allowed formats. You can give it a priority by the order in the array. Then you can use `DateTime.ParseExact` with overload that takes a `string[]`. http://msdn.microsoft.com/en-us/library/332de853(v=vs.110).aspx
Comment: @Thomas: i have edited my answer to provide another approach.
Answer: From The "MM" Custom Format Specifier

The "MM" custom format specifier represents the month as a number from
  01 through 12 (or from 1 through 13 for calendars that have 13
  months).

You should probably change your day specifier with your month specifier like MM/dd/yyyy

DateTime.ParseExact() raising exception


É possível existir múltiplos form-login com o Spring Security na mesma aplicação?

Question: Bem a questão de inicio pode parecer um pouco confusa, mas basicamente gostaria de saber se é possível ter mais de um formulário de validação do spring security:form-login, pois tenho em meu sistema dois pontos no qual a validação irá ser diferente, ou seja eu teria que ter mais de um authentication-provider para realizar as consultas em tabelas diferentes.
Answer: Boa tarde João
É possível sim ter mais de um authentication provider, em fato, quantos quiser na mesma aplicação.
De acordo com a documentação, é possivel utilizar providers existentes no Spring Security e também criar seus customizados. Após a criação, é preciso registrar em um authentication-manager.
Ex:
<authentication-manager>
   <authentication-provider ref="restAuthProvider/>
   <authentication-provider ref="basicAuthProvider"/>
</authentication-manager>

<bean id="restAuthProvider" class="br.com.auths.RestAuthProvider">
</bean>

<bean id="basicAuthProvider" class="br.com.auths.BasicAuthProvider">
</bean>

Ref: http://docs.spring.io/spring-security/site/docs/4.0.3.RELEASE/reference/htmlsingle/#ns-auth-manager


Can you tag individual searches per search in order to apply different filters to each?

Question: I've got a site where we use solr to generate some menu items and other custom lists as well as running the default search. The issue I'm running in to is all the different hook_apachesolr_query_alter calls across several different modules seem to be getting applied on top of each other stopping everything from working as expected.
Is there an identifier I can look for, or that I can add, in order to identify which query is currently being altered so I can filter them appropriately?
This is what I want to be able to do:
<?php
if ($query->name == 'special_menu'){
  $query->replaceParam('qf', 'sm_vid_Colour');
}
?>

(This is a dupe of my post on Drupal.org at http://drupal.org/node/1976224 but I wasn't getting any love there and thought I'd try my luck here!)
Answer: I found a solution that worked for me. Here it is in case anyone else wants to do something similar, or in case someone wants to suggest a better way of doing it!
So the situation is that for a single page, I need to run a handful of SOLR queries to determine what should show up and what should not. Each of those queries works on different fields, so we need to be able to add a different 'qf' parameter to each of them. This gets harder however, when all hook_apachesolr_query_alter calls trigger on all searches! (See the related issue about enabling 'qf' #1659184: Allow apachesolr_search_search_results() to accept 'qf' (query fields) parameter)
After a lot of experimentation I found that the only field I could manipulate when starting a query and fetch and act on in hook_apachesolr_query_alter was using the search path value in a custom $search_page. This is how I did it:
<?php
$search_page = array(
        'env_id'   => 'solr_server',
        'settings' => array(
          'apachesolr_search_per_page' => 25,
          'spellcheck' => FALSE,
        ),
        'search_path' => 'ducks-BiGhE3Nxv34lyTKVoka74WI3r9yjJyc5ohiuJkeCiqp8gJunyWk3gXMDK8W',
      );

      $search_terms = array('"duck"', '"cheese sauce"');

      // Only run if the solr is enabled server environment is detected.
      if (module_exists('apachesolr') && apachesolr_environment_load($search_page['env_id'])) {
        $results = apachesolr_search_search_results('"' . implode(' ', $search_terms). '"', NULL, $search_page);
?>

Then, in hook_apachesolr_query_alter for this particular module/query:
<?php
function pet_or_dinner_apachesolr_query_alter($query) {
  /* test that this is a 'ducks' query */
  if (stristr($query->getPath(), 'ducks')){
    /* now apply different settings depending on the current page */
    $node = menu_get_object();
    if (isset($node->type) && $node->type == 'pets') {
      //Search only the Pet field
      $query->replaceParam('qf', 'sm_vid_pet');
    } elseif (isset($node->type) && $node->type == 'meals') {
      //Search only the Meal field
      $query->replaceParam('qf', 'sm_vid_meal');
    }
  }
}
?>

I don't know if it's necessary, but I put the extra long random string in the 'search_path' so that I didn't have any site visitors accidentally tripping over it. I don't think it is publicly accessible though. Someone else can confirm or debunk that concept!
But there we go, a hook to be able to perform custom modifications on concurrent queries. Definitely not clean but it gets us over the line!
I'll leave this question open in case someone has a better answer.


How can I add 'repel = TRUE' in below code to make PCA more readable?

Question: I have used below code to create this plot, but it has too many overlaps, tough to read, I have also tried with repel = TRUE, but it also shows some error. My questions are (1) how can I add ggrepel in below code to make this graph more readable. (2), How can I make PCA vector rows bold.
ggplot() +
      geom_hline(yintercept = 0, colour = "gray65") +
      geom_vline(xintercept = 0, colour = "gray65") +
      #geom_point(data = Root.Traits.pca12, aes(x = PC1, y = PC2), colour= "gray40", alpha = 0.5, size = 5) +
      geom_point(data = Root.Traits.pca12, aes(x = PC1, y = PC2, colour=Si), alpha = 0.8, size = 5, shape = 16)+

      #geom_point(data = arrows12, aes(x = PC1, y = PC2, fill=Nlevel), pch=21, colour=NA, alpha = .9, size = 3) +
      geom_segment(data = arrows12, aes(xend = 5*x2, yend = 5*y2), x=0, y=0, colour = "black") +
      geom_text(data = arrows12, aes(x = 5*x2, 5*y2, label = corlabels),colour = "black", alpha = 1, size = 5) +
      coord_fixed(ratio = 1, xlim = c(-8, 6.6), ylim = c(-5, 7.5)) +
      scale_x_continuous(breaks=c(-7.5,-5,-2.5,0, 2.5, 5, 7.5))+
      scale_y_continuous(breaks=c(-5,-2.5,0, 2.5, 5))

Answer: Regarding your Q1, replace geom_text with repulsive textual annotations geom_text_repel(). Your edited code will likely be geom_text_repel(data = arrows12, aes(x = 5*x2, 5*y2, label = corlabels),colour = "black", alpha = 1, size = 5) in library(ggrepel).
For Q2, add size=1in geom_segment(). I hope it will work.


prevent matplotlib ginput registering click on widget button

Question: I've written a code to register the x values clicked in a plot window using ginput, it works fine and only registers when you click within the window.
I then wanted to add some control buttons using matplotlib widgets which I did and they work fine too with the methods, all good so far...
But, my problem is that when I click on the button, the coordinates of the button are also registered by ginput, that I don't want.  Is there any way to prevent that happening?  Making the button area non-active for ginput or somehow detecting that and rejecting those clicks?
I'm also happy to use an alternative to ginput, I'm not really wedded to any particular method (I'm very new to GUI in python and just want to set up a simple example).
Here is my reproducible example, clicking on the test button also adds lines to the list :-(
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.widgets import Button

class Index:
    def test(self, event):
        print ("test")

# fake data
x=np.arange(30)
y=x**2
times=[]

fig,ax=plt.subplots()

ax.plot(x,y)
callback = Index()
buttonname=['test']
colors=['white']
idx=[0.2]
bax,buttons={},{}

# set up list of buttons.
for i,col,button in zip(idx,colors,buttonname):
  bax[button] = plt.axes([0.92, i, 0.07, 0.07])
  buttons[button] = Button(bax[button],button,color=col,hovercolor='green')
  buttons[button].on_clicked(getattr(callback,button))

# register click on plot
while True:
     pts=plt.ginput(1)
     print ("pts is",pts)
     timebegin=pts[0][0]
     times.append(timebegin)
     ax.axvline(x=timebegin)
     print ("adding",timebegin)
     print ("all ",times)
     plt.pause(0.05)

Screenshot of the example:

EDIT: I've thought of a fudge for the moment, where in the button method I add a pop to remove the entry from the times list and also undraw the lines (I also declare times and lines as a global), it works but it is clunky and not very elegant, as the program draw the mistaken line and then deletes it again, ugh.
# in the main code I now append the ax.axvline to a list "lines"
def test(self,event):
    times.pop()
        times.pop()
        lines.pop().remove()

Answer: A way to avoid that problem would be not to use buttons to terminate the input, but instead use the mouse_stop parameter of the ginput function (described here) which works with a mouse button or the enter key.
Then it's possible to add a text to the plot to explicit this method for the user... less interactive than a button, but it works.
Answer: You can do times.append(nan) in the callback, so then if times[-1] is nan, pop() it and ignore the click. There is some delay in registering the button event, so pause() briefly before checking times[-1].
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.widgets import Button

times = []

# on click event, append nan
class Index:
    def test(self, event):
        global times
        times.append(np.nan)
        print('test')

fig, ax = plt.subplots()
x = np.arange(30)
y = x**2
ax.plot(x, y)

callback = Index()
bax = fig.add_axes([0.92, 0.2, 0.07, 0.07])
button = Button(bax, 'Test', color='white', hovercolor='green')
button.on_clicked(callback.test)

while True:
    pts = plt.ginput(1)

    # allow callback to finish (duration may need to be adjusted)
    plt.pause(0.15)

    # ignore if newest element is nan
    if times and np.isnan(times[-1]):
        print(f'ignoring {times[-1]}')
        times.pop()
    else: # otherwise store and plot
        timebegin = pts[0][0]
        print(f'adding {timebegin}')
        times.append(timebegin)
        ax.axvline(x=timebegin)

    print(f'times = {times}')
    plt.pause(0.05)


TF model on Google AI Platform Models Error: "Expected serialized to be a scalar or vector, got shape:[1,12]"

Question: As part of outputs from tfx trainer and pusher, the saved model/pre-trained model has been executed in Google AI Platform Jobs and Models through the following libraries.
from tfx.extensions.google_cloud_ai_platform.pusher import executor as ai_platform_pusher_executor
from tfx.extensions.google_cloud_ai_platform.trainer import executor as ai_platform_trainer_executor

However, when I test the served model in Google AI Platform Models, I got the following error message, can you please help me to understand what does it means? How should I transform my input data to be accepted by the model?
Error:
{
  "error": "Prediction failed: Error during model execution: <_MultiThreadedRendezvous of RPC that terminated with:\n\tstatus = StatusCode.INVALID_ARGUMENT\n\tdetails = \"Expected serialized to be a scalar or vector, got shape: [1,12]\n\t [[{{node ParseExample/ParseExampleV2}}]]\"\n\tdebug_error_string = \"{\"created\":\"@1590723484.491366234\",\"description\":\"Error received from peer ipv4:127.0.0.1:8081\",\"file\":\"src/core/lib/surface/call.cc\",\"file_line\":1056,\"grpc_message\":\"Expected serialized to be a scalar or vector, got shape: [1,12]\\n\\t [[{{node ParseExample/ParseExampleV2}}]]\",\"grpc_status\":3}\"\n>"
}

Sample Input in Google AI Platform Model:
{
  "instances": [
["A", "B", "C", "D","E", "F", "G", "H", "I", "J", "K", "L"]
  ]
}

Here is the code for the trainers.
def get_model(show_summary=True):

#one-hot categorical features
num_A = 4
num_B = 3
num_C = 2
num_D = 8
num_E = 12
num_F = 4
num_G = 16
num_H = 26
num_I = 11
num_J = 2
num_K = 11
num_L = 2

input_A = tf.keras.Input(shape=(num_A,), name="A_xf")
input_B = tf.keras.Input(shape=(num_B,), name="B_xf")
input_C = tf.keras.Input(shape=(num_C,), name="C_xf")
input_D = tf.keras.Input(shape=(num_D,), name="D_xf")
input_E = tf.keras.Input(shape=(num_E,), name="E_xf")
input_F = tf.keras.Input(shape=(num_F,), name="F_xf")
input_G = tf.keras.Input(shape=(num_G,), name="G_xf")
input_H = tf.keras.Input(shape=(num_H,), name="H_xf")
input_I = tf.keras.Input(shape=(num_I,), name="I_xf")
input_J = tf.keras.Input(shape=(num_J,), name="J_xf")
input_K = tf.keras.Input(shape=(num_K,), name="K_xf")
input_L = tf.keras.Input(shape=(num_L,), name="L_xf")

inputs_con = tf.keras.layers.concatenate([
input_A,
input_B,
input_C,
input_D,
input_E,
input_F,
input_G,
input_H
input_I,
input_J,
input_K,
input_L])

dense_1 = tf.keras.layers.Dense(50, activation = 'relu')(inputs_con)
dense_2 = tf keras.layers.Dense(25, activation = "rely") (dense_1)
output = tf.keras.laters.Dense(1, activation = "sigmoid") (dense_2)
model = keras.Model(inputs=inputs, outputs=outputs)

_inputs = [
input_A,
input_B,
input_C,
input_D,
input_E,
input_F,
input_G,
input_H,
input_I,
input_J,
input_K,
input_L]

model = tf.keras.models.Model(_inputs, output)

model.compile(optimizer='rmsprop',
          loss='binary_crossentropy',
          metrics=['accuracy'])

if show_summary:
    model.summary()

return model

Comment: Any update on this issue?
Answer: LLTeng, not sure what your intent for the model is, but based on the instances, this would work:
inputs = tf.keras.Input((12,))
dense_1 = tf.keras.layers.Dense(50, activation = 'relu')(inputs)
dense_2 = tf keras.layers.Dense(25, activation = "rely") (dense_1)
output = tf.keras.laters.Dense(1, activation = "sigmoid") (dense_2)
model = tf.keras.Model(inputs=inputs, outputs=outputs)


Difference between subroutine , co-routine , function and thread?

Question: I have been reading the Art of Programming, vol 1, for the past 2 days.
There is a topic about subroutine and coroutine.
I am confused. I can't understand when the book says that coroutines are initialised later after main(), unlike subroutines. Furthermore, the invoked coroutine becomes a subroutine for a calling coroutine.
I tried researching over previously asked question on here, where I came to know notions of functions and thread too, in the same context.
The book gives an example about a coroutine using the MIX computer model.
Can I get any simple pseudocode, using an example about coroutines and other terminologies in the context of today's high level languages?
So, a related question: How to differ a subroutine, coroutine, function and thread?
Answer: Until we get a post from someone who really knows, here's my understanding to the question, FWIW.
A subroutine and a function are essentially the same thing, with one difference: A function returns some sort of value (usually via the stack or CPU register), while a subroutine does not. Whether subroutine or function, it is a block of executable code, having exactly one point of entry. A co-routine is also a block of executable code, and, just like a subroutine, it has one point of entry. However, it also has one or more points of re-entry. More on this later.
Before getting to threads, let's review: A computer program, also known as a process, will generally have its allocation of memory organized into a code space, a heap, and a stack. The code space stores the one or more blocks of its executable code. The stack stores the parameters, automatic variables, and return addresses of subroutines, functions, and co-routines (and other things too). The heap is the wide-open memory space available to the process for whatever its purposes. In addition to these memory spaces are the CPU registers, each of which stores a set of bits. These bits could be an integer value, a memory address, a bunch of status flags, or whatever. Most programmers don't need to know much about them, but they're there and essential to the operation of the CPU. Probably the ones worth knowing about are the Program Counter, Stack Pointer, and Status Register, but we're not going to get into them here.
A thread is a single logical flow of execution. In a primitive computing system, there is only one thread available to a process. In modern computing systems, a process is composed of one or more threads. Each thread gets its own stack and set of CPU registers (which is usually physically impossible, but made virtual logically  - a detail we'll skip on here). However, while each thread of a process has its own stack and registers, they will all share the same heap and code space. They are also (presumably) running simultaneously; something that can truly happen in a multi-core CPU. So two or more parts of your program can run at the same time.
Back to the co-routine: As mentioned before, it has one or more points of re-entry. A point of re-entry means that the co-routine can allow for some other block of code outside of itself to have some execution time, and then at some future time have execution time resume back within its own block of code. This implies that the parameters and automatic variables of the co-routine are preserved (and restored if need be) whenever execution is yielded to an external block of code and then returns to that of the co-routine. A co-routine is something that is not directly implemented in every programming language, although it is common to many assembly languages. In any case, it is possible to implement a co-routine in a conceptual way. There is a good article on co-routines at http://en.wikipedia.org/wiki/Coroutine.
It seems to me there are two principal motivations in implementing a co-routine design pattern: (1) overcoming the limitations of a single-threaded process; and (2) hoping to achieve better computational performance. Motivation (1) is clear to understand when the process must address many things at once where a single thread is a must. Motivation (2) may not be as clear to understand, since that is tied to a lot of particulars about the system hardware, compiler design, and language design. I can only imagine that computational effort might be reduced by cutting-back on stack manipulations, avoidance of redoing intializations in a subroutine, or relieving some of the overhead of maintaining a multi-threaded process.
HTH
Comment: co-routines are also the basis for [generators](https://wiki.python.org/moin/Generators) in Python for example. Effectively building an iterable of length N with space complexity O(1).
Answer: Focusing on coroutine vs subroutine:
Coroutines can yield and this is interesting.
Yield 'remembers' where the co-routine is so when it is called again it will continue where it left off.
For example:
<code>  coroutine foo {
    yield 1;
    yield 2;
    yield 3;
  }
  print foo();
  print foo();
  print foo();
</code>
Prints:
1 2 3
Note: Coroutines may use a return, and behave just like a subroutine
<code>  coroutine foo {
    return 1;
    return 2; //Dead code
    return 3;
  }
  print foo();
  print foo();
  print foo();
</code>
Prints:
1 1 1
Comment: As always, examples are worth more than thousand words :)
Answer: I'd like to expand on existing answers, adding following:
There exist 4 main concepts in an invocation of a piece of code:

creation of a context  (aka "frame", a local environment for this code to operate)
resuming/invocation of a context (transferring control to that frame, aka jumping)
detaching (resuming another context, similar to 2.)
and destruction of a context

In subroutines creation and resuming happen simultaneously with a "call" instruction - a stack frame gets allocated, arguments and return address are pushed, the execution jumps to the piece of called code. Also, detaching (resuming a caller) and destruction are done simultaneously with a "return" instruction - stack frame is de-allocated, control is transferred to a caller (via previously supplied return address) and the caller is left to scavage the junk on a stack to pick out the return value (depending on your calling conventions).
In coroutines, these main concepts exist independently, decoupled from each other. You create a coroutine at some time, then you can transfer control to it later (so it can yield you some results, possibly multiple times), and then you can destroy it at some later point in time.
Comment: @Softxide which statements of which call? :) You mean execute a different code after a coroutine was re-invoked? Yes. They keep state inbetween invocations, because context is not destroyed upon detaching.
Comment: In co routines , can we skip statements in next call?
Answer: Differences:
(1) Co-routine doesn't have a master-slave relationship while subroutine has.
(2) A subroutine has a single entry point, while co-routine may have multiple entry points.
(3) The calling program is suspended during execution of the called subroutine. Co-routines provide quasi-concurrent execution.
Similarities:
(1) Both co-routines and subroutines are able to use return.
(2) The execution of both co-routines and subroutines are not overlapped.
Answer: Coroutines are simply concurrent subroutines(functions, methods,closures) that are non preemptive.That is they cannot be interrupted,instead coroutines have multiple points thoughout which allows multiple points of suspension or scope reentry.

Coroutines, are  implicitly  concurrent  constructs,  but  concurrency is not a property of a coroutine: something must host several coroutines simultaneously  and  give  each  an  opportunity  to  execute otherwise,  they  wouldn't  be concurrent! Note that this does not imply that coroutines are implicitly parallel.
Comment: Goroutines are coroutines
Answer: from python docs,

Coroutines are a more generalized form of subroutines. Subroutines are entered at one point and exited at another point. Coroutines can be entered, exited, and resumed at many different points. They can be implemented with the async def statement.
Answer: The most straightforward explanation that I found about the difference between subroutine and coroutine is that a coroutine keeps "state" between calls.
In other words a coroutine can store values between calls, and retrieve them when you call it again, while a subroutine always starts its values from the "beginning".


Java Lock explanation

Question: I can't understand the following code:
   public class Counter {
      private long value;
      private Lock lock;
      public long getAndIncrement() {
       lock.lock();
       try {
        int temp = value;
        value = value + 1;
       } finally {
         lock.unlock();
       }
       return temp;
      }
    }

What I can't understand is how Lock is instantiated while it's an interface?
And if it's an anonymous class that implements Lock interface, why I can't see any override of Lock functions (e.g. lock() and unlock() ) ?
In short, the following line really confuses me.
  private Lock lock;

What is lock here? What is its type?
Edit:
Lock is an interface and can't be instantiated. After looking at the constructor:
public Counter(){
   lock = new ReentrantLock();
}

Now, everything is clear. (Thanks to Bhushan Uniyal )
Comment: Lock is a interface and you need to provide it implementation of Lock, there are already some classes which provide the implementation of  Lock  , ReentrantReadWriteLock.ReadLock, ReentrantLockReentrant, ReadWriteLock.WriteLock,
e.g;
`Lock lock = new , ReentrantLockReentrant();`
also you can provide your own implemention
Comment: Where did you get this example? This code doesn't compile...
Answer: Q  how Lock is instantiated while it's an interface?
Lock in an interface, implemented by     ReentrantLock, ReentrantReadWriteLock.ReadLock, ReentrantReadWriteLock.WriteLock
From Java Doc

A reentrant mutual exclusion Lock with the same basic behavior and
  semantics as the implicit monitor lock accessed using synchronized
  methods and statements, but with extended capabilities.
A ReentrantLock is owned by the thread last successfully locking, but
  not yet unlocking it. A thread invoking lock will return, successfully
  acquiring the lock, when the lock is not owned by another thread. The
  method will return immediately if the current thread already owns the
  lock. This can be checked using methods isHeldByCurrentThread(), and
  getHoldCount().
The constructor for this class accepts an optional fairness parameter.
  When set true, under contention, locks favor granting access to the
  longest-waiting thread. Otherwise this lock does not guarantee any
  particular access order. Programs using fair locks accessed by many
  threads may display lower overall throughput (i.e., are slower; often
  much slower) than those using the default setting, but have smaller
  variances in times to obtain locks and guarantee lack of starvation.
  Note however, that fairness of locks does not guarantee fairness of
  thread scheduling. Thus, one of many threads using a fair lock may
  obtain it multiple times in succession while other active threads are
  not progressing and not currently holding the lock. Also note that the
  untimed tryLock method does not honor the fairness setting. It will
  succeed if the lock is available even if other threads are waiting.

lock()

Acquires the lock.
Acquires the lock if it is not held by another thread and returns
  immediately, setting the lock hold count to one.
If the current thread already holds the lock then the hold count is
  incremented by one and the method returns immediately.
If the lock is held by another thread then the current thread becomes
  disabled for thread scheduling purposes and lies dormant until the
  lock has been acquired, at which time the lock hold count is set to
  one.
Answer: Lock is a interface and you need to provide it implementation of Lock, there are already some classes which provide the implementation of  Lock  , ReentrantReadWriteLock.ReadLock, ReentrantLockReentrant, ReadWriteLock.WriteLock,
e.g;
Lock lock = new , ReentrantLockReentrant();
also you can provide your own implementation
public class MyLock implements Lock {

    public void lock() {

    }

    public void lockInterruptibly() throws InterruptedException {

    }

    public boolean tryLock() {
        return false;
    }

    public boolean tryLock(long time, TimeUnit unit) throws InterruptedException {
        return false;
    }

    public void unlock() {

    }

    public Condition newCondition() {
        return null;
    }
}

A lock is a thread synchronization mechanism like synchronized blocks except
locks can be more sophisticated than Java's synchronized blocks. Locks (and other more advanced synchronization mechanisms) are created using synchronized blocks, so it is not like we can get totally rid of the synchronized keyword.
From Java 5 the package java.util.concurrent.locks contains several lock implementations, so you may not have to implement your own locks. But you will still need to know how to use them, and it can still be useful to know the theory behind their implementation.

Java Lock explanation


Calculate confidence intervals on accuracy metrics

Question: I've trained a logistic regression model on sklearn and then used it on some sample data which outputted accuracy, sensitivity and specificity values.
Is there a way I can use the number of samples tested to estimate some form of confidence intervals for these metrics?
i.e. how confident am I that the accuracy is the calculate 87%, and can I calculate a range like 84%-90%?
Many thanks
Comment: Hint: if your sample data is a random subsample of overall population data, then the success or failure of the model on each event is an independent trial with some unknown probability p -- what distribution describes the random variable of the number of successes?
Comment: Accuracy is a proportion, is it not?
Answer: Let me start by saying that accuracy is not an amazing metric.  Not only can it be gamed very easily by guessing the most prominent class, but it also is not a proper scoring rule.  Meaning that choosing the model with the largest accuracy does not mean choosing the model which best models the data.  Additionally, sensitivity/specificity are poor measures of performance when we intend to use the model prospectively since they get the order of conditioning wrong (sens/spec condition on the outcome.  We can't condition on the outcome prospectively, that's the whole point of making a model).  With that aside, there may be ways to none the less obtain a confidence interval for accuracy.
Accuracy is just a binomial outcome (number correct over number predicted), so you could apply any number of binomial confidence intervals.  Ideally, you would use the test set to do this, but there are other ways to doing this which benefit from larger samples.
In particular, you could bootstrap the optimism corrected bootstrap as shown here.  That example uses the <code>{rms}</code> library, but I've created a blog post showing how to do the optimism corrected bootstrap with python and sklearn here.
Comment: +1, and here is the relevant thread: [Why is accuracy not the best measure for assessing classification models?](https://stats.stackexchange.com/q/312780/1352)


How to define custom messages in Drupal 7

Question: I want to show a custom message to user after he/she adds a new content of a specific type in Drupal 7. If I change the '@type %title has been created' to what I want the message will change for all content types.
How can I do it? Is there any module to define messages for different actions?
Comment: I briefly used the Custom Submit Message Module ( https://drupal.org/project/csm ) for a Drupal 6 project and saw that it has a Drupal 7 beta version that might be of interest.
Comment: possible duplicate of [Change form validation error messages in Drupal 7](http://drupal.stackexchange.com/questions/75455/change-form-validation-error-messages-in-drupal-7)
Comment: @NikhilM, I do not think, its duplicate question.
Comment: you can do it with rules
Answer: I just did this the other day, it's a bit of a hack, but I couldn't find another way.  The basic premise is to get and clear all current messages, loop through and call drupal_set_message() on all of them over again, changing the one you want:
<?php
/**
 * Submit function for the message node form.  Check for new message notification and change it.
 */
function mymodule_message_node_form_submit($form, &$form_state) {
  $node = $form_state['node'];
  $t_args = array('@type' => node_type_get_name($node), '%title' => $node->title);
  $replace_message = t('@type %title has been created.', $t_args);

  // This call will get all messages and clear them from $_SESSION
  $messages = drupal_get_messages('status');
  foreach ($messages['status'] as $message) {

    if ($message == $replace_message) {
      drupal_set_message(t('Thank you your message has been sent successfully.  We will be in touch shortly.'));
    }
    else {
      drupal_set_message($message);
    }
  }
}
?>

Answer: I am using this to replace an existing drupal message to my custom message.
function _exclude_message($mymessage, $mytype="status"){
$flag = false if ($messageArr = drupal_set_message()) {  foreach($messageArr as $type=>$messages) {
  if ($mytype==$type) {
    foreach($messages as $key=>$message) {
      if ($mymessage==$message) {
    unset($_SESSION['messages'][$type][intval($key)]);
       $flag = true;
      }
   }
  }
  if (count($_SESSION['messages'][$type])==0) unset($_SESSION['messages'][$type]);}} return $flag;}

function my_theme_status_messages($display = NULL) {
  if (_exclude_message('Sorry, unrecognized username or password. Have you forgotten your password?',"error")) drupal_set_message("Sorry, unrecognized username or password.","error");
 return theme_status_messages($display);
}
under template.php
this code will replace existing "Sorry, unrecognized username or password. " with my custom message "Have you forgotten your password?".
Hope this will help you.
Answer: I had to do that too. My solution is for error messages in validation hook. But should be quite the same for a confirmation message. You just have to find the id of the message (in my example the name of the field).
First I used a function to prevent the message from beeing sent.
function form_unset_error($name) {
   $errors = &drupal_static('form_set_error', array());
   $removed_messages = array();
   if (isset($errors[$name])) {
     $removed_messages[] = $errors[$name];
     unset($errors[$name]);
   }
   $_SESSION['messages']['error'] = array_diff($_SESSION['messages']['error'], $removed_messages);
   if (empty($_SESSION['messages']['error'])) {
     unset ($_SESSION['messages']['error']);
   }
}

So in my validation hook I unset the message and set a custom error message instead :
form_unset_error('field_name][und][0][value');
form_set_error('field_name', t('Custom error message translated'), $limit_validation_errors = NULL);

Hope it helps.


How to compare all the buttons? Or check if all are disabled?

Question: I'm trying to do a memory game using "cards" all works fine but I need to know when the user win the game to display a message but I can't figure out how to do that? How can I check using a condition if all the buttons are .setEnabled(false)? Or should I compare all the buttons using a loop?
I hope you can help me.
Answer: Since you have cartas as an array. You should just iterate through a loop to see if all cards are set to false.
for(int x =0; x< cartas.length;x++)
{
      if(cartas[x].isEnabled()) //enabled
      {
         break; //a button is still enabled so cancel loop
      }
      else if(x == cartas.length-1&& !cartas[x].isEnabled())
       {
            //All cards have been disabled. Do Something.
       }
}

Therefore, the method
isEnabled();

allows you to check if all buttons are
.setEnabled(false);

Comment: @Jos Fernando Valenzuela Hope this helps


Why use the term 'models' to interpret the double turnstile symbol?

Question: A $\vDash $B can be read in words as:

A entails B
B is a semantic consequence of A
A models B

The first two are fine. But the third one seems a bit counter-intuitive to me. Somehow, I can't reconcile the term 'model' (used colloquially) with the notion of entailment or consequence.
Why do mathematicians and logicians use the term 'model' in such contexts? How is the term associated with the idea of entailment? And even more bizarre to me is the use of the $\vDash$ symbol in different contexts. For example, let A be an assignment, and F is an atomic formula. So that A $\vDash$ F (again, read as A 'models' F) means that A assigns a truth value of 1 to F. Again, this is counter-intuitive, and often had me wondering: "what is the underlying concept or idea behind it?"
Comment: Even the terms "fashion model" and "model airplane" seem reasonably consistent with the way the term "model" is used in math and science, to me at least.  What colloquial use of "model" are you thinking of?  Just curious.
Comment: @saulspatz
By colloquial use, I was thinking along the lines of 'represent' or 'imitate'. To be fair, yes, 'model' airplane and 'fashion model' does jibe with common usage. But still, in the case of entailment, or semantic consequence? When we say A models (represents? imitates?) B, how do we relate to the idea of A entails B?
Comment: I have to admit that I don't remember anything about semantics; I had to look up the [double turnstile](https://en.wikipedia.org/wiki/Double_turnstile).  Now that I have, I tend to agree with you, but I may misunderstand the Wikipedia article.  I would say, "The natural numbers with addition are a model for group theory," whereas it seems the usage in the article is, "Group theory models the natural numbers with addition."   I hope someone knowledgeable answers your question. +1
Answer: The corresponding usage of "model" is e.g. "this program models the behaviour of the bees". $A$ is treated as an environment wherein $B$ is true.
Comment: I apologize, but i really do feel like i'm missing a key picture here. Your example jibes precisely with what goes through my mind when i think of the colloquial use of 'models', in this case, it means the computer program being a virtual 'representation' of the behavior of bees. Still, I can't quite relate that analogy with the notion of semantic consequence? What is the abstraction that takes one from the formal definition of 'A models B' to a computer program which models the behavior of bees? Further clarifications would be much appreciated. Thank you.
Comment: Actually I'm not so sure now that you ask.
Answer: The 'reading' #3 is a different thingy from reading #2 and #1.
As far as I know, the double turnstile ($\vDash$) has two uses.
In your reading #3, $A$ is some 'structure', which can be called a 'model', an interpretation, or an 'assignment', while $B$ is a propositional formula. The notation is more like $\mathcal{A}\vDash B$ or $\mathcal{M}\vDash B$, and it should be read/understood as "The structure $\mathcal{A}$ (or $\mathcal{M}$) models B" or "B is true in $\mathcal{A}$ (or $\mathcal{M}$)."
For example, in classical propositional logic, an assignment $\mathcal{A}$ is a two-valued function $A:B\rightarrow \{\top,\bot\}$ where $B$ is a propositional formula.
In readings #1 and #2, $A$ and $B$ are both propositional formulas. That is the 'semantic consequence' that you understand. The notation is now $A\vDash B$. Moreover, this means that for all assignments $\mathcal{A}$, if $\mathcal{A}\vDash A$, then $\mathcal{A}\vDash B$.
Comment: @Anthony: The meanings are closely related. If $\Gamma$ is a set of formulas, then $\Gamma\vDash\varphi$ means that $\mathcal M\vDash\varphi$ for all $\mathcal M$ that satisify $\Gamma$. So when we write $\Gamma\vDash\varphi$ we're essentially using $\Gamma$ as a stand-in for an arbitrary structure that satisfies it -- and that's the point of putting a $\Gamma$ to the left of a $\vDash$ at all,
Comment: Indeed. I very much agree with you that they are two different things. But i can't help but wonder if there is some kind of underlying abstraction which connects the two "thingies" as you put it. I mean for mathematicians, and logicians, two class of people who are hilariously meticulous, and who outright despise ambiguity and imprecision; for them to use the same symbol to denote two things? Surely there have to be a reason? If there isnt any underlying abstraction (an overlap in concept) between the two, then why use two symbols which could potentially lead to ambiguities?
Comment: It's just like how in programming we do operator overloading for user defined types. Or like how the '+' symbol originally used for the addition of natural numbers is being used for other purposes like the addition of matrices, vectors, etc. etc, because there is some underlying similarity in the concepts
Answer: First, as another user has said, it is important to note that there is an important distinction between $\Gamma \models \phi$, where $\Gamma$ is a set of formulas, and $\mathcal{A} \models \phi$, where $\mathcal{A}$ is a structure. As far as I know, the word "models" is used only in the latter case. Since your question was originally about the relation between the "common" use of the word "model" and the logician's use, here I'll focus on this relationship. For more details about the history of the word "model" and its (scientific) uses, I strongly recommend Roland Müller's essay "The Notion of a Model: A Historical Overview", from which I gathered most of my information about this.
Anyway, according to Müller, one of the common uses of the word "model" is in the sense of "prototype" or "original", as when a person serves as a model for a painting, or a small-scale sculpture serves as the model for a larger construction (say, a cathedral). Notice that the salient feature here is less the idea that the "model" is the original, and more the idea that we use it to read off important properties of the object being modeled. This is still in use as when we talk in logic about using a "toy model" for studying a theory; in fact, it seems that nineteen centuries geometers, such as Plücker, literally used such toy models (three-dimensional objects) to aid in the study of geometrical theories. So, at first, the word "model" was linked to this idea of a small-scale prototype which we use to "control" for the desired properties.
Later, however, as Müller emphasizes, geometers began to encounter objects which were difficult to model in this way. In particular, both projective and hyperbolic geometries posed problems for those who wanted to build small-scale objects to serve as models for their study. Indeed, it's not entirely clear how to create a small-scale Klein bottle, or more complicated geometric objects. So, according to Müller, geometers came up with two solutions for this problem: pseudo-models and abstract models.
Pseudo-models are models that introduce systematic distortions in order to aid visualization. That is, you create an object which distorts the original one, but whose distortion is controlled, so that you know exactly which features are being distorted. In other words, we know exactly how to pass from the distorted feature to the undistorted one and vice-versa. Müller example is Poincaré's models of the hyperbolic plane using the circle or the half-plane.
The other device is an abstract model, in which we don't attempt to create a physical model or prototype, but instead attempt to give an abstract description of a mathematical structure which has the desired properties. Although Müller doesn't make the link very explicitly, it seems to me that this latter device is connected with the rise of set-theory and structural descriptions in the work of Riemann and Dedekind. The latter, especially, seemed to think of his "Systems" in this way.
Müller also deals with how the word was eventually introduced by Tarski and Robinson in order to give birth to model theory, but the important point for this question is this: when we say that $\mathcal{A} \models \phi$, we are not saying that $\mathcal{A}$ is a model of $\phi$ in the sense of a representation of $\phi$. Rather, we are saying that $\mathcal{A}$ is a model, or prototype, from which we read off the property that $\phi$. So $\mathcal{A}$ is a model of $\phi$ in the sense that some people speculate that Da Vinci himself is the model of the Mona Lisa: it is the original, or prototype, which Da Vinci studied in order to read the relevant properties which allowed him to paint the Mona Lisa. Alternatively, and more liberally, you can also say that $\mathcal{A}$ is a model of $\phi$ in the sense that a given three-dimensional object (say, a small-scale pyramid) is an object (a model) which allows us to study properties of a geometrical concept (say, the concept of tetrahedron).
So much for the word "model". Now, why use the symbol $\models$ for different things? Well, there is a clear relation between them: $\Gamma \models \phi$ iff for every $\mathcal{A}$, $\mathcal{A} \models \Gamma$ implies $\mathcal{A} \models \phi$. So the overload is harmless (or so it seems to me).
Comment: @Anthony - I'm glad you found this answer helpful. If you are interested in further details about the history of model theory, I suggest checking out Wilfrid Hodge's "The History of Model Theory", which you can download from his website or read in Button & Walsh's *Philosophy and Model Theory* (it's printed there as an appendix).
Comment: You have answered it perfectly, and in a way that satisfied my curiosity. Thank you. I will read up on your suggested literature right away. And will come back here if i have any further questions.
Comment: Might i enquire where one could access Mueller's article?


Splitting Grunt-File into several parts with grunt-load-config: Task not found

Question: I am trying to split my fairly large Grunt File into several smaller parts using load-grunt-config. I have a Gruntfile.js in the root of my project and the child tasks in a directory called grunt.
This is my Gruntfile.js:
module.exports = function(grunt) {
    require('load-grunt-config')(grunt, {
      loadGruntTasks: {
        pattern: '*',
        config: require('./bower.json'),
        scope: 'devDependencies'
      }
    });
};

And this for example is my grunt/copy.js file:
module.exports = function (grunt) {
  return {
      main: {
        files: [{
          cwd: 'init/php/templates',
          src: '<%= init.php.templates %>',
          dest: 'src/php/templates',
          expand: true
        }],
        options: {
          process: function (content, srcpath) {
            return grunt.template.process(content);
          }
        }
      }
  }
};

However no matter what I try (including renaming the copy.js file to grunt-copy.js), Grunt always complains about how it is not able to find my tasks. I am getting this error message:
Warning: Task "copy:main" not found. Use --force to continue.

I am biting on Granite here. Something must be wrong, however I don't really know what, so any hint is appreciated.
Comment: Have you tried to get a basic version working? Maybe only have one task with no additional config: `require('load-grunt-config')(grunt);` ? And maybe simplify you're copy task config just to see whether the issue is with your config or the load-grunt-config plugin.
Comment: Glad you figured at least one part out!
Comment: Silly me, that worked fine. Turns out the problem is the loadGruntTasks part. I use this, to point my install task to the right bower.json, because right now it needs one bower.json in the grunt-folder and one in the root. With the config I solved this problem, but apparently this causes the tasks to crash. Now I need to find a way to just use one bower.json ;)
Answer: For those who are interested. Here is how I solved the Bower.json issue. I am now using the package grunt-bower-task, which is really nice.
Here is my Gruntfile.js
module.exports = function(grunt) {
    require('load-grunt-config')(grunt);
};

The install.js at grunt/bower.js
module.exports = {
  install: {
    options: {
      targetDir: "./assets/dev/",
      layout: "byType",
      install: true,
      verbose: false,
      cleanBowerDir: false,
      cleanTargetDir: true,
      bowerOptions: {}
    }
  }
}

And my bower.json
{
  "name": "myProject",
  "version": "1.0.0",
  "dependencies": {
    "bootstrap": "latest",
    "fontawesome": "latest"
  },
  "exportsOverride": {
    "bootstrap": {
      "vendor/js": "**/*.min.js",
      "vendor/css": "**/bootstrap.min.css"
    },
    "jquery": {
      "vendor/js": "**/jQuery.min.js"
    },
    "fontawesome": {
      "vendor/css": "**/*.min.css",
      "vendor/css/fonts": "**/fontawesome-webfont.*"
    }
  }
}

This automatically sorts all files into vendor/css and vendor/js folders in my dev/assets/ folder and only takes those files from the bower.packages, that I have specified. This really helps to keep the whole setup nice and clean!
Answer: I think you need to remove the grunt.initConfig() in the Gruntfile.js.
and delivery the data which you want to supply to each task with require('load-grunt-config')(grunt, {data: {pkg: grunt.file.readJSON(./package.json), .....}});


SSL Configuration Problem

Question: I've generated an SSL certificate with keytool:
keytool -keystore keystore -alias mySite -genkey -keyalg RSA -keysize 2048

I got this configured and working fine, but obviously it wasn't signed by a trusted CA, so I generated a CSR:
keytool -certreq -alias mySite -keystore keystore -file mySite.csr

GoDaddy gave me two certificates back, which I imported into my keystore:
keytool -keystore keystore -import -alias mySite.com -file mySite.com.crt
keytool -keystore keystore -import -alias gd_bundle -file gd_bundle.crt

I thought this would be sufficient, but I am still getting an untrusted certificate warning when I hit my app through the browser.  Am I missing a step?  My app is running on Jetty, and the only configuration I've provided is the keystore and the key-password.
I am getting the following exception:
javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate

Answer: After messing around with this for a couple hours, I was able to find a solution.  I ended up using openssl instead of keytool:
openssl genrsa -des3 -out mysite.key 2048
openssl req -new -key mysite.key -out mysite.csr

Submit the CSR, and once you get issued a cert:
openssl pkcs12 -export -chain -CAfile gd_bundle.crt -in mysite.com.crt -inkey mysite.key -out mysite.pkcs12
java -classpath jetty-util-6.1.26.jar:jetty-6.1.26.jar org.mortbay.jetty.security.PKCS12Import mysite.pkcs12 keystore

Hope others find this useful!

SSL Configuration Problem


How to sum vector arrays in a list of lists?

Question: I want to sum a list of vectors in a list while keeping their dimension and  list structure. How can this be done?
Given a vector
vecs = [[[1, 1], [2, 2], [3, 3], [4, 4]],[[5, 5], [6, 6], [7, 7],[8, 8]]]
and the expected output is
vecs_sum = [[10,10],[26,26]]
I tried the following, but it just returns [].
vec_sum = []
for inner_list in vec_sum:
  temp = []
  for arr in inner_list:
    temp = [sum(x) for x in zip(arr)]
  vec_sum.append(temp)

Comment: Don't you want to be iterating over `vecs`, not `vec_sum`? `vec_sum` is an empty list.
Answer: This is one approach using a list comprehension
Ex:
vecs = [[[1, 1], [2, 2], [3, 3], [4, 4]],[[5, 5], [6, 6], [7, 7],[8, 8]]]
vecs_sum = [list(map(sum, zip(*i))) for i in vecs]
print(vecs_sum)

Output:
[[10, 10], [26, 26]]

Comment: @SurajSubramanian..looks like you have used `list` as a variable name...
Comment: Your code seems to give an error : `TypeError: 'list' object is not callable`
Comment: Solution works ok, was not able to find better :-) Upvoted.


O que é um algoritmo determinístico e não determinístico?

Question: O que é um Algoritmo determinístico e não determinístico?

Quais as características de ambos?
É possível implementar ambos em qualquer linguagem?

OBS: se possível exemplifique com alguma implementação
Answer: Algoritmo determinístico é o que sempre produz o mesmo resultado dadas determinadas entradas de dados.
A maioria dos algoritmos são determinísticos. Ainda bem :)
Por isso nem sempre os algoritmos reproduzem bem os problemas do mundo, os problemas reais costumam ser indeterminísticos, qualquer tentativa de reproduzir o mundo real beira a insanidade. É possível fazer uma representação simplificada do problema real. Daí a definição de OOP como sendo uma ferramenta para reproduzir o mundo real é fundamentalmente errada.
Algoritmo não determinístico é aquele que pode produzir resultados diferentes mesmo com os mesmos dados entrados.
Isto é comum porque todo algoritmo que depende de dados externos, como o tempo, concorrência ou falha de hardware por exemplo, possivelmente ou certamente produzirá um resultado diferente.
Os exemplos mais óbvios são os que pegam o horário do computador no momento da execução, os que usam geradores de números aleatórios, os que possuem características inerentemente probabilísticas.
Mas há casos menos óbvios, como os que dependem de dados que não podem ser bem classificados. Como o próprio nome diz, quando não é possível determinar o que fazer o resultado pode ser diferente. Um algoritmo de classificação que executa em um roll de dados que possuem duplicidades exatas devem classificá-los como? Existe uma forma de determinar isto? Existem algoritmos que não conseguem.
Um algoritmo que lê arquivos, redes ou outras formas (em baixo nível) são indeterminísticos, não se sabe o que virá desta leitura, já o que será feito com os dados lidos (nível mais alto) possivelmente serão determinísticos, já que eles passam ser uma entrada de novos algoritmos. Estes algoritmos não sabem o que vão encontrar lá, são dados externos, eles podem ter sido modificados por outras fontes. Então olhando em um nível mais alto o algoritmo de leitura será determinístico.
O tracing garbage collector é um dos mais famosos indeterminismos da computação. Por mais que as regras sejam bem estabelecidas, não se sabe quando ele será disparado, não se sabe quando o algoritmo produzirá um resultado que o dispare. Depende de muitos fatores, alguns deles que não estão sob controle da aplicação.
Note que indeterminismo não é o mesmo que aleatório. Não quer dizer que necessariamente acontecerá qualquer coisa, apenas que ela não pode ser determinada antecipadamente.
Sim, é possível implementá-los em qualquer linguagens. Algumas poucas evitam, ou pelo menos tentam isolar, o uso dos indeterminísticos. Mas não tem como fugir deles na maioria dos problemas.
Coloquei no GitHub para referência futura.
Comment: @Caffé concordo que o algoritmo de mais alto nível é determinístico mas o de baixo nível não. Ele manda ler uma posição do disco ou manda ler o que está no *buffer* da rede, a entrada de dados é o pedido de leitura de algo, não são os dados. Pediu para ser **algo**, veio uma coisa, depois pediu de novo, esse algo é o mesmo (mesma posição do disco ou *buffer*, por exemplo), vem outra coisa diferente, porque o que está lá não é de controle daquele algoritmo. Isto é indeterminístico. É dado externo que não pode ser garantido. Acho que o problema aí é a interpretação do que é a entrada de dados.
Comment: @bigown Também achei essa definição um pouco ampla demais. E um *gerador* de números pseudo-aleatórios, é determinístico ou não? Cada vez que você chama, dá uma saída diferente, e sem entrada nenhuma... Mas eu chamaria sim um PRNG de determinístico, pois sua saída é determinada pelo seu **estado** interno. Da mesma forma, a função que lê de um arquivo depende sim do seu estado (ou, se recebe um stream, do estado do stream, pra qual byte do arquivo ele "aponta"), mas isso não torna ele indeterminístico. Isso tudo é só minha opinião, contudo, traçar uma linha que separa os dois é complicado...
Comment: @mgibsonbr Tentei melhorar. O gerador pode ser determinístico, mas o seu uso pode não ser. O *stream* já é algo intermediário, já é algo que tem como entrada os dados que estavam presentes em algum lugar, aí é determinístico mesmo. Tudo depende do que é a entrada de dados. A confusão toda está aí. Esse é o meu entendimento. Concordo que traçar a linha não é tão simples.
Comment: Não entendi bem a parte de leitura de arquivo ou rede. Se o mesmo algoritmo lê a mesma cadeia de bytes mais de uma vez (o mesmo arquivo, ou dados iguais trafegados na rede), por que ele seria não-determinístico?
Comment: Com o seu penúltimo comentário, entendi o que você quis dizer. O que está no comentário (o que é a entrada) está mais claro que na resposta.
Comment: Melhorou agora?
Comment: Haskell trata isso de uma maneira excelente.
Comment: É, [Monads](https://en.wikipedia.org/wiki/Monad_(functional_programming)).
Comment: Não melhorou muito... O exemplo da leitura de dados externos continua trazendo alguma confusão. Se o algoritmo processa o conteúdo do arquivo e o arquivo for o mesmo, as saídas serão as mesmas - o algoritmo é determinístico. Se o arquivo mudar, então as entradas do algoritmo mudaram - mas o algoritmo continua determinístico.
Answer: Um algoritmo determinístico é aquele que se comporta da mesma maneira em diferentes execuções, dadas as mesmas entradas e o mesmo estado interno da máquina (se relevante). Um não determinístico seria aquele que pode se comportar de forma diferente na mesma situação.
Os computadores são determinísticos. Salvo alguma falha no hardware (que introduzisse um bug no programa), qualquer implementação de qualquer algoritmo num computador clássico (não quântico) será determinística, ainda que sua saída ou seu comportamento pareça "caótico" aos nossos olhos. É até possível provar que uma Máquina de Turing não determinística é equivalente a uma determinística (ou seja, todo algoritmo não determinístico poderia ser expresso - e portanto implementado - como um algoritmo determinístico).
Por essa razão, é difícil traçar uma linha que separe o determinístico do não determinístico. Um critério comum para diferenciá-los é se eles dependem de algum fator aleatório na sua execução - ainda que na prática esse fator seja dado por meio de um gerador de números pseudo-aleatórios (portanto determinístico) e/ou alguma entrada externa. Pois mesmo que o algoritmo use uma abordagem diferente (ex.: simular todos os caminhos de execução possíveis) na hora de escolher um ele vai precisar de algum critério para tal.

Quais as características de ambos ?

A principal característica do não-determinístico é que não se pode prever seu tempo de execução, consumo de memória, etc, mesmo conhecidas as entradas [que não o fator aleatório]. Muitas vezes é difícil fazer isso analiticamente também para os determinísticos, mas ao menos duas execuções com as mesmas entradas resultarão sempre no mesmo comportamento.

É possível implementar ambos em qualquer linguagem ?

Sim, pois como já mencionado, é possível simular uma Máquina de Turing não determinística em uma determinística. Se uma linguagem for Turing completa, então ela pode implementar qualquer algoritmo não determinístico.

se possível exemplifique com alguma implementação

Um exemplo clássico é o quicksort com pivô aleatório: escolhe-se um elemento qualquer da lista, passa-se todos os elementos menores que ele pra trás, e todos os maiores pra frente. Então chama-se o algoritmo recursivamente na primeira e na segunda parte, até que uma delas contenha somente zero ou um elemento.
(Note que, na ausência de elementos duplicados, a saída do algoritmo é sempre a mesma, fator aleatório ou não; só o comportamento dele é que muda - quantos passos foram necessários para se chegar à mesma saída)
Contraste o quicksort com pivô arbitrário, que comumente é o elemento central da lista, ou o primeiro, ou um 1/3 ou 2/3 do caminho.
Há ainda casos que não se conhece um algoritmo determinístico eficiente para executar determinada função, mas um não determinístico sim. Como comentei na pergunta sobre computação quântica, um caso famoso é o "teste de primalidade", que recebe um número e afirma com X% de certeza se ele é ou não primo (i.e. há 1-X% de chance dele classificar um número composto como "primo"). Hoje já se conhece um algoritmo polinomial determinístico para determinar se um número é primo, mas ainda existem outros problemas sem solução.
A classe de problemas que podem ser solucionados por um algoritmo probabilístico é a BPP ("probabilístico de tempo polinomial comprometido à erros"), e conjectura-se que BPP = P. E, naturalmente, como para se implementar qualquer algoritmo probabilístico num computador determinístico é necessário um fator aleatório externo, essas implementações poderiam ser chamadas de não determinísticas.


Varnish 4.1 - How to serve cached copy on backend fetch failed instead of 503

Question: I have a site served by apache+tomcat and a cache served by Varnish 4.1
When apache is down, varnish always returns a 503 error.
I would like varnish to return the copy of the pages it has in its cache but my attempts with ttl and grace have been unsuccessful.
I think I've read all the documentation on varnish 4.1 that I could find, any help is really appreciated.
Thanks in advance
EDIT: varnishlog -g request -q "ReqUrl eq '/'"
curl
*   << Request  >> 1410492
-   Begin          req 1410491 rxreq
-   Timestamp      Start: 1646995409.603391 0.000000 0.000000
-   Timestamp      Req: 1646995409.603391 0.000000 0.000000
-   ReqStart       10.xxx.xxx.xxx 57472
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.1
-   ReqHeader      host: akamai5.rsi.ch
-   ReqHeader      user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:98.0) Gecko/20100101 Firefox/98.0
-   ReqHeader      accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
-   ReqHeader      accept-language: it,en-US;q=0.7,en;q=0.3
-   ReqHeader      accept-encoding: gzip, deflate, br
-   ReqHeader      cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; __utma=46365988.836346559.1644414441.16
-   ReqHeader      upgrade-insecure-requests: 1
-   ReqHeader      sec-fetch-dest: document
-   ReqHeader      sec-fetch-mode: navigate
-   ReqHeader      sec-fetch-site: none
-   ReqHeader      sec-fetch-user: ?1
-   ReqHeader      cache-control: max-age=0
-   ReqHeader      x-forwarded-proto: https
-   ReqHeader      x-forwarded-ssl: on
-   ReqHeader      x-forwarded-port: 443
-   ReqHeader      x-forwarded-for: 1178.xxx.xxx.xxx
-   ReqHeader      connection: close
-   ReqUnset       x-forwarded-for: 1178.xxx.xxx.xxx
-   ReqHeader      X-Forwarded-For: 1178.xxx.xxx.xxx, 10.xxx.xxx.xxx
-   VCL_call       RECV
-   ReqUnset       cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; __utma=46365988.836346559.1644414441.16
-   ReqHeader      Cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; cs_fpid=1645804349272_22538249; wt_geid
-   ReqUnset       Cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; cs_fpid=1645804349272_22538249; wt_geid
-   ReqHeader      Cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; cs_fpid=1645804349272_22538249; wt_geid
-   ReqUnset       Cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; cs_fpid=1645804349272_22538249; wt_geid
-   ReqHeader      Cookie: wt_rla=292330999892453%2C2%2C1646731482026; _pipe_c=do_not_track; _ga=GA1.2.836346559.1644414441; __gads=ID=453223b8518b57e5-22f191e137cd00c3:T=1644414444:RT=1644414444:S=ALNI_MbCUi8liJ5sbhjlTe68z1BhLhZJCQ; cs_fpid=1645804349272_22538249; wt_geid
-   VCL_return     hash
-   ReqUnset       accept-encoding: gzip, deflate, br
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   Hit            1410469
-   VCL_call       HIT
-   VCL_return     deliver
-   RespProtocol   HTTP/1.1
-   RespStatus     200
-   RespReason     OK
-   RespHeader     Date: Fri, 11 Mar 2022 10:42:05 GMT
-   RespHeader     Server: Apache-Coyote/1.1
-   RespHeader     Content-Type: text/html;charset=UTF-8
-   RespHeader     Set-Cookie: JSESSIONID=F8D07853DF7D90A3F381B316F64FA285; Path=/; HttpOnly
-   RespHeader     X-Varnish: 1410492 1410469
-   RespHeader     Age: 84
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   RespHeader     X-Cache-Host: rsi-prod-varnish45
-   RespHeader     X-Frame-Options: SAMEORIGIN
-   RespHeader     X-XSS-Protection: 1; mode=block
-   RespHeader     X-Content-Type-Options: nosniff
-   RespHeader     Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' *
-   RespHeader     X-Cache: HIT
-   RespHeader     X-Cache-Hits: 3
-   RespHeader     X-Grace-Hit: yes
-   VCL_return     deliver
-   Timestamp      Process: 1646995409.603614 0.000222 0.000222
-   RespHeader     Accept-Ranges: bytes
-   RespHeader     Content-Length: 191244
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: close
-   Timestamp      Resp: 1646995409.608024 0.004632 0.004410
-   ReqAcct        1130 0 1130 574 191244 191818
-   End

browser
*   << Request  >> 1410496
-   Begin          req 1410495 rxreq
-   Timestamp      Start: 1646995426.730217 0.000000 0.000000
-   Timestamp      Req: 1646995426.730217 0.000000 0.000000
-   ReqStart       10.xxx.xxx.xxx 60908
-   ReqMethod      GET
-   ReqURL         /
-   ReqProtocol    HTTP/1.1
-   ReqHeader      host: www.example.com
-   ReqHeader      user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0
-   ReqHeader      accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
-   ReqHeader      accept-language: it,it-IT;q=0.8,en-US;q=0.5,en;q=0.3
-   ReqHeader      accept-encoding: gzip, deflate, br
-   ReqHeader      cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; __utma=46365988.1217589648.1620392041.1638545296.1642173864.10; __utmz=46365988.16
-   ReqHeader      upgrade-insecure-requests: 1
-   ReqHeader      cache-control: max-age=0
-   ReqHeader      x-forwarded-proto: https
-   ReqHeader      x-forwarded-ssl: on
-   ReqHeader      x-forwarded-port: 443
-   ReqHeader      x-forwarded-for: 178.xxx.xxx.xxx
-   ReqHeader      connection: close
-   ReqUnset       x-forwarded-for: 178.xxx.xxx.xxx
-   ReqHeader      X-Forwarded-For: 178.xxx.xxx.xxx, 10.xxx.xxx.xxx
-   VCL_call       RECV
-   ReqUnset       cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; __utma=46365988.1217589648.1620392041.1638545296.1642173864.10; __utmz=46365988.16
-   ReqHeader      Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
-   ReqUnset       Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
-   ReqHeader      Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
-   ReqUnset       Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
-   ReqHeader      Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
-   VCL_return     hash
-   ReqUnset       accept-encoding: gzip, deflate, br
-   ReqHeader      Accept-Encoding: gzip
-   VCL_call       HASH
-   VCL_return     lookup
-   VCL_call       MISS
-   VCL_return     fetch
-   Link           bereq 1410497 fetch
-   Timestamp      Fetch: 1646995426.730455 0.000238 0.000238
-   RespProtocol   HTTP/1.1
-   RespStatus     503
-   RespReason     Backend fetch failed
-   RespHeader     Date: Fri, 11 Mar 2022 10:43:46 GMT
-   RespHeader     Server: Varnish
-   RespHeader     Content-Type: text/html; charset=utf-8
-   RespHeader     Retry-After: 5
-   RespHeader     X-Varnish: 1410496
-   RespHeader     Age: 0
-   RespHeader     Via: 1.1 varnish-v4
-   VCL_call       DELIVER
-   RespHeader     X-Cache-Host: rsi-prod-varnish45
-   RespHeader     X-Frame-Options: SAMEORIGIN
-   RespHeader     X-XSS-Protection: 1; mode=block
-   RespHeader     X-Content-Type-Options: nosniff
-   RespHeader     Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' *
-   RespHeader     X-Cache: HIT
-   RespHeader     X-Cache-Hits: 0
-   RespHeader     X-Grace-Hit: yes
-   VCL_return     deliver
-   Timestamp      Process: 1646995426.730495 0.000278 0.000040
-   RespHeader     Content-Length: 284
-   Debug          "RES_MODE 2"
-   RespHeader     Connection: close
-   Timestamp      Resp: 1646995426.730527 0.000310 0.000032
-   ReqAcct        929 0 929 490 284 774
-   End
**  << BeReq    >> 1410497
--  Begin          bereq 1410496 fetch
--  Timestamp      Start: 1646995426.730367 0.000000 0.000000
--  BereqMethod    GET
--  BereqURL       /
--  BereqProtocol  HTTP/1.1
--  BereqHeader    host: www.example.com
--  BereqHeader    user-agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0
--  BereqHeader    accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
--  BereqHeader    accept-language: it,it-IT;q=0.8,en-US;q=0.5,en;q=0.3
--  BereqHeader    upgrade-insecure-requests: 1
--  BereqHeader    x-forwarded-proto: https
--  BereqHeader    x-forwarded-ssl: on
--  BereqHeader    x-forwarded-port: 443
--  BereqHeader    X-Forwarded-For: 178.xxx.xxx.xxx, 10.xxx.xxx.xxx
--  BereqHeader    Cookie: _pipe_c=do_not_track; _ga=GA1.2.1217589648.1620392041; __gads=ID=1a22b33c44d55e6f-1a22b33c44d55e6f:T=1620392042:RT=1620392042:S=ALNI_MYR9nyXrxcQ8QV1Y2pNVDp67Gn9-w; wt_rla=292330999892453%2C2%2C1646995242972; cs_fpid=1645855325408_55998969; JSESSI
--  BereqHeader    Accept-Encoding: gzip
--  BereqHeader    X-Varnish: 1410497
--  VCL_call       BACKEND_FETCH
--  VCL_Log        Backend fetch: v_ssl_ece
--  VCL_return     fetch
--  FetchError     Director v_ssl_ece returned no backend
--  FetchError     No backend
--  Timestamp      Beresp: 1646995426.730387 0.000020 0.000020
--  Timestamp      Error: 1646995426.730390 0.000023 0.000003
--  BerespProtocol HTTP/1.1
--  BerespStatus   503
--  BerespReason   Service Unavailable
--  BerespReason   Backend fetch failed
--  BerespHeader   Date: Fri, 11 Mar 2022 10:43:46 GMT
--  BerespHeader   Server: Varnish
--  VCL_call       BACKEND_ERROR
--  BerespHeader   Content-Type: text/html; charset=utf-8
--  BerespHeader   Retry-After: 5
--  VCL_return     deliver
--  Storage        malloc Transient
--  ObjProtocol    HTTP/1.1
--  ObjStatus      503
--  ObjReason      Backend fetch failed
--  ObjHeader      Date: Fri, 11 Mar 2022 10:43:46 GMT
--  ObjHeader      Server: Varnish
--  ObjHeader      Content-Type: text/html; charset=utf-8
--  ObjHeader      Retry-After: 5
--  Length         284
--  BereqAcct      0 0 0 0 0 0
--  End

EDIT 2: hash and cookies
After much investigation we discovered that the problem lies in this configuration:
sub vcl_hash {
  hash_data( req.url );
  if( req.http.host ) {
    hash_data( req.http.host );
  } else {
    hash_data( server.ip );
  }
  # hash cookies for object with auth
  if( req.http.Cookie ) {
    hash_data( req.http.Cookie );
  }
  return( lookup );
}

If cookies are removed from the varnish hash it returns the cache correctly.
Is it useful for cookies to be included in the hash? At first glance I think so, if cookies are used to keep track of user authentication. How can we include only certain cookies in the hash (e.g. those from an authenticated session) and not those that are not useful for caching (e.g. those related to analytics)?
Answer: As long as the sum of the TTL and grace values are greater than zero, no backend fetches are made.
If your TTL expires, grace mode will ensure that stale content is served while a fetch attempt is made.
Grace, which is a stale-while-revalidate mechanism, can actually be abused and turned into a stale-if-error mechanism. By setting the grace high enough, users will never experience the downtime of the backend.
Here's some VCL:
vcl 4.1;

import std;

sub vcl_recv {
    if (std.healthy(req.backend_hint)) {
        set req.grace = 10s;
    }
}

sub vcl_backend_response {
    set beresp.grace = 24h;
}

This VCL code will enforce a 24-hour grace. This means your backend can be down for a day without anyone noticing. However, as long as the backend is healthy, Varnish will only use 10 seconds of grace.
It's important that your backend definition has a .probe property that refers to a health checking probe, otherwise std.healthy(req.backend_hint) will never be able to produce the right results.
I used set beresp.grace = 24h which might be excessive. By setting the grace to such a high duration, these objects remain in the cache while being expired. If you have a large catalog of content that needs to be cached, this may effect your hit rate and may cause forced cache eviction when the cache is full.

IMPORTANT: Varnish Cache 4.1 is end-of-life and should not be used. Please install Varnish 6.0 LTS or Varnish 7 instead.
My VCL example contains a vcl 4.1; version marker which will only work in Varnish 6 or Varnish 7. I do this on purpose to encourage users to use recent versions of Varnish. By switching this to vcl 4.0; the code will probably work on older versions of Varnish. Please consider upgrading.

503 error
Based on the VSL output that you added to your question, I can conclude that you have defined and used a director object in your VCL that doesn't have any backends.
This is the error I found in your VSL output:
Director v_ssl_ece returned no backend

Please have a look at the director named v_ssl_ece.

Check whether or not you need this director
Check if it is assigned correctly through req.backend_hint in vcl_recv or through bereq.backend in vcl_backend_fetch
Check why v_ssl_ece doesn't have any backends

If you cannot find the solution, please share your full VCL file so I can assist.
Comment: Thank you very much! Unfortunately I can't update varnish to the new version :-/
Your solution works very well if I use curl, but if I use a browser (chrome or firefox) varnish returns a 503 error.
I tried simulating a firefox request with curl using all the http headers firefox sends to the server but varnish returns 200 OK if I use curl and 503 if I use firefox.
Do you have any clues about this?
Thanks again
Comment: Please add the output of `varnishlog -g request -q "ReqUrl eq '/'"` to the question. This command will print the logs for the homepage. Ensure you're using the browser to call the homepage. If you get the error, the logs might tell us why.
Comment: I added the output of `varnishlog -g request -q "ReqUrl eq '/'"` to the question, both for a request from curl and a request from firefox. Do see any something that could help me us? Thanks again
Comment: @Erich I updated the answer with information regarding the VSL output
Comment: Sorry for my late reply, we struggled a lot to find the problem. I updated my question again and I hope you have some useful suggestions. Thank you very much!
Comment: @Erich Please don't add the cookies to the hash, because it will cause too many cache variations and too many misses. If you want to create cache variations based on cookie values, please extract select cookies and validate their content to avoid filling up your cache with meaningless cache variations.
Comment: @Erich Please also add your full VCL code, not just the `hash_data` extract. I want to understand what's really going on and tell you what to change to enforce *stale if error* behavior when the backend is down.

My initial feedback is that the `Set-Cookie: JSESSIONID=F8D07853DF7D90A3F381B316F64FA285; Path=/; HttpOnly` could be the reason why the content is not stored in cache.

See https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/#dont-cache-responses-with-set-cookie-headers for more information.


ANOVA for 2 groups with subgroups within the second group?

Question: Say I'm interested in running an experiment. I want to see if student using collaborative techniques (say we pick one random technique) affects test scores. So, obviously the null hypothesis is that there is no change in test scores and the alternative hypothesis is that there is some significant change from employing this treatment. I'll give my experimental design idea and I want to know what would be the test to use if this experiment were to actually take place.
So say there are two courses and the students have roughly the same level in math (or whatever as long as it's the same subject matter). Class A is the control group and class B is the treatment group. In the treatment group, the students are split into many smaller groups since normal classes have 20 + students. There are, say, 4 exams (and they're the same for both groups) given during the year.
How do I test if the treatment made a difference? I'm thinking a One-Way ANOVA. Or a two sample t-test? Is the fact that there are smaller groups in Class B a problem or could just average all those exams and compare it to Class A's? What about the fact that there are 4 exams throughout the year? I feel like that's problematic. Would it be best to pick one exam, say the final exam to test whether one or not the treatment made a difference? Are there other things I should be concerned about? Is One-Way ANOVA the right way to go here?
Comment: Hi, can you clarify what does "split into many smaller groups" mean? Will these smaller groups have somehow different teaching/learning/treatment experience with regards to each other and with regards to the control group? Why is the control group not divided into these smaller groups?
Comment: You need to clarify your post. Sample size? 4 exams during the year indicates this is [tag:repeated-measures]. Maybe add that tag?
Comment: @Sointu it's because in the control group, there is no collaborative learning being administered. For sample size, we can assume there are 30 students in both the control and the experimental group.
Answer: (I'll put this as an answer because of comment space limits, but it's not a comprehensive answer)
So the smaller groups within the treatment group all get different treatment/teaching? In that case, perhaps you can treat each of these groups + control as levels of your "treatment" factor. E.g. if you have 4 of these smaller groups within Class B, then you'd have a 5-level treatment factor. Of course, this would make a very unbalanced design so you couldn't use ANOVA. You could probably use a multilevel regression with the 5-level factor as predictor and (if there is a time component and participants are tested several times across time) participant as random effect, and I guess you'd need to add a "class" random effect too.
I think that as you haven't run the study yet, you might want to consider the design more closely before collecting data. Perhaps you could try recruiting equal number of participants for each of the "subgroups" of your treatment, including the control group? Perhaps even so that you'd sample several classes (more than 2) and assign some number of students from each class in each of the subgroups and also in control group? This way, you could account for possible differences between classes and their possible effects on treatment -> outcome relation, and you'd get more data and maybe also a balanced design.
Comment: I see. But if you want to study the effect of treatment on some learning outcome, you should also divide the control group to smaller teaching groups, otherwise how can you know whether differences between treatment and control group are due to the treatment or to the size of teaching group? As for your second question, that's probably something you're the best expert of, but I personally would use objective grades if they were available (of course you can also test the effects of treatment on other variables, such as self-reported learning experience or some such).
Comment: The smaller groups within the treatment group get the same teaching. I'm just looking to see whether working in groups of say, 5 helps with students' grades. Would it be best to simply calculate and compare the final exam grades since those exams are comprehensive and would be an indicator of how students learned the material overall
Comment: I have multiple groups because it would be somewhat extreme to use the whole class a group. I just want to see if working in groups is useful. Not too worried about the size of the group right now. So in theory it would be best if I just have set groups for the entire semester right? So that's there's no variability in scores attributed to working with different people
Comment: It's difficult to comment on the design of the study from the outside, you're probably the best judge of that as you know how the teaching and all the details will go. I just think it's important to make sure other relevant characteristics (such as teaching group size) are not confounded with your experimental treatment.