TypeScript require generic parameter to be provided

Question: I have the following function:
async function get(url: string): Promise {
    return getUrl(url);
}

However, it is possible to call it like this (U is set to any by TS):
get('/user-url');

Is there a way to define this function such that it requires U to be provided explicitly, as in
get<User>('/user-url');

Comment: What problem are you trying to solve? After all, `U` would default to `{}`, and you'd get an error when you try to access any properties on it, which would alert you to the problem.
Comment: What is your TypeScript definition of *getUrl*? And what are you going to achieve by adding that requirement?
Comment: @jaboja I think the use case is pretty clear, I have seen numerous questions here on SO as to why is `get` (of whatever API) returning `{}` or `any` and not what I expect it to, or how do I tell get what type my API returns. Always the `get` has a type parameter that people do not see at first. With such a restriction it becomes mandatory and obvious
Answer: There is no built-in support for this, we can however engineer a scenario where not passing in a type parameter will generate an error, using default generic type arguments and conditional types. Namely we will give U a default value of void. If the default value is the actual value of U, then we will type the parameter to the function as something that should not really be passed in so as to get an error:
async function get(url: string & (U extends void ? "You must provide a type parameter" : string)): Promise {
    return null as any;
}

get('/user-url'); // Error Argument of type '"/user-url"' is not assignable to parameter of type '"You must provide a type parameter"'.

class User {}
get<User>('/user-url');

The error message is not ideal, but I think it will get the message across.
Edit: For a solution where the type parameter is used in parameter types see here
Comment: doing so will not allow you to actually use `extends` to set up generic type constraint
Comment: Thanks for the link to your other answer.  `get('/user-url')` won't typecheck but `const n: number = await get('/user-url');` will.  You need the `NoInfer` boxing trick from your that other answer to work around that.
Answer: You could use multiple type parameters:
function contractType<T = void, U extends T = T>(value: unknown): U {
    return value as U
}

const example1: string = contractType(17) // error
const example2: string = contractType("value") // error
const example3: string = contractType<string>("value") // ok

https://github.com/Microsoft/TypeScript/issues/14829#issuecomment-288902999
Answer: function createCommand
(a: P): P {
    a.whatEverProperty; // a is always instanceof WhatEver. Never is overlooked here.
}

The easiest solution for most of cases is specifying never as default type argument value. never is always assignable to the type argument, even if you extend it with WhatEver, so the body of function is not affected, but the return value here will be never affecting the consumers ability to use this function in any meaningful way.
(Consumers/callers will get an error at the point they try to use the return value, but if your function is called purely for side-effects, you may want to consider the approach outlined in @Titian's answer)
You can use conditional types to make the return type never when you normally wouldn't return P:
function createCommand
(a: P): P['whatEverProperty'] {
    // a is always instanceof WhatEver. Never is overlooked here.
    return a.whatEverProperty as P extends WhatEver ?
      ? WhatEver
      : never
}

Comment: This does not address the problem of contextual inference.  `const s: string = get("./abc")` will infer the generic type `U = string`.  The actual problem is that the user is not using the generic parameter to type a function argument.  They should actually be returning `unknown` and explicitly type-checking at runtime, or casting.
Answer: The other answers here provide some decent workarounds to get your code to work the way you want but none of them address the fundamental problem, which is that you're trying to do unsafe casts to make the typechecker happy.
I don't have the source of your getUrl<T> function but I strongly suspect that it looks a lot like
function getUrl<T>(url: string): Promise<T> {
  return fetch(url).then(resp => resp.json());
}

In this scenario, Response#json() returns Promise<any> which you're quietly casting to Promise<T> -- which is unsafe.  When you write const user: User = await get("./user");, the typechecker is happy to infer that you want a Promise<User> back and fills it in for you.  It does this even if you provide a default value.
A safer way to do this would be to have getUrl return unknown.  This will force you to either check your return value at runtime with a type guard, or at least explicitly cast it ((await get("./user")) as User etc), in which case your linter can chastise you for unsafe casts.
Answer: Yet another approach! Require a parameter that is used just for the type inference:
/**
 * getFoo gives you a well-typed Foo component
 * ```tsx
 * const foo = getFoo(config)
 * ```
 */
const getFoo = <Keys extends string>(
  _config: Record<Keys, unknown>
) => {
  return FooUntyped as FooType<Keys>
}

If you have objects readily available to pass in for this type inference, and your functions wouldn't otherwise accept a parameter, this may be a good approach for you *as of TS v4.1.2
If you don't have some config object readily available, calling it is a little bit "wtf" but you get an error directly where you call it, instead of somewhere further on:
const foo = getFoo({} as Record<'my' | 'keys', unknown>)

This design/approach happened to actually fix a little kink in my design in another location. I actually added an extra param too my getFoo called defaultProps which are some default props for a given Foo component (Foo is a Form Field component, and config keys are valid field names)
Comment: A bit of a tangent, but this is one circumstance where 1 hook returning 1 component actually seems like a fair pattern. (A hook returning multiple components, however, gets in the way of tree-shaking)
Answer: I stumbled on your question because of the following Ramda pathOr type definition:
export function pathOr<T>(defaultValue: T, path: Path, obj: any): T;

The function can find a value deep inside an object, following a path.
The problem is that if you don't provide the generic type parameter T, Typescript will use the type from the defaultValue. This makes no sense when the object's content doesn't have the same type at that path.
The solution is complex and only works because there are two arguments. I don't think there's an easy answer to your question that works for all cases.
import {toPath} from 'lodash-es'
import {pathOr} from 'ramda'

// Aternative to lodash `get` where you have to provide a default value and a return type that includes the default value.
// Useful to avoid cases where the return type is inferred from the default value's type but does not fit with the object's content at the path.
export const getOr = <T = never, D extends T = T>(object: unknown, pathStr: string, defaultValue: D): T =>
  pathOr(defaultValue, toPath(pathStr), object)

To be fair, ramda's path function and lodash's get function both try really hard to find the correct return type based on the object and the path, but I don't see how using a T type for both the default value and return value could be a good idea.
Answer: As long as you always use the return value, you can add two overrides to your example:
declare function getUrl (url: string): U
type U = Record<string, unknown>

function get (_: U): void
async function get (url: string): Promise
async function get (url: string): Promise {
  return getUrl(url)
}

get('/user-url') // No error, unfortunately
await get('/user-url') // Error
const promise = get('/user-url') // Error

await get('https://...') // Okay

Note that this makes the return type void if  isn't provided, so although you'll get a TypeScript error if you try to use the return value, you can still call it and ignore the return type.


Use of two-sample Kolmogorov-Smirnov test to evaluate similarities between two different distributions

Question: $X_1, X_2, \dots X_n$ and $Y_1, Y_2, \dots Y_n; n = 1000$ are two samples of physical quantities coming from the application of two different mathematical models to some independent and identically distributed (iid) data.
The mathematical model used to generate $Y_i%$ is a simplified version (tuned by the parameter $r$) of the mathematical model used to generate $X_i$.
My goal is to find the value of $r$ that makes the empirical distribution $F_Y(y)$ to be as similar as possible to the distribution $F_X(x)$.
I decided to use the Two-Sample Kolmogorov-Smirnov test (in R) for different value of $r$ ranging in a specific interval. Is this choice correct?
I know that the null hypothesis for the K-S test is that the two distributions are the same. However, I know for sure that the two distributions are different because the two mathematical models are different. Is it correct to evaluate the best value of $r$ by looking at the p-value and the D statistic coming from the K-S test?
Comment: Welcome to the site, @RiccardoCavallari. Please don't sign your posts. Notice that your username (with a link to your user page) & your flair are automatically added to your posts. Regarding your question, do you care if the mean or SD is similar, or only if the shape & ranking is similar? Could you just correlate the 2?
Comment: Thank you @gung, actually, since the two distributions are surely different, it could be enough to find the value of the parameter r that makes the mean and the standard deviation similar. If this is the case, what are the best tools to use?
Answer: You say you're trying to make the two distributions close; the $D$ statistic measures the discrepancy between the two, and (as long as it's not changing the dimension of the fit) choosing $r$ to minimize that discrepancy makes complete sense.
I don't think there's any need to deal with the $p$-value; $D$ is a sensible thing to optimize.
If you're changing the dimension of the fit (adding or removing parameters), just minimizing $D$ isn't going to be sufficient, since more parameters will always tend to improve the fit.
Comment: Thank you! Can you please go into more details about changing the dimension of the fit?
Comment: It's not clear to me what information you might need. If changing $r$ does something like change the amount of smoothing, you may change the dimension of the model with $r$. Are you able to describe what $r$ does?
Comment: But if the $p$-value is large, does it matter if $D$ is small w.r.t. establishing similarity?
Comment: That's not necessarily a question I can answer for you, since the relevance will depend on your personal tradeoff between those things. If D is small and p is large, you have no evidence that they're different; if D is large and p is large, it may simply be lack of power. Lack of rejection doesn't imply they're similar; if you want to do that you might want to look into some form of confidence interval.


How to serialize a DateTimeFormatter in scala?

Question: I am using DateTimeFormatter in scala and now it is giving me trouble with a serialization.
Comment: Did you try initializing the DateTimeFormatter within your last map method as was also suggested as an option in the URL you provided?
Comment: Yes, the same error happens.
Comment: Can you include that code in your original question?
Comment: Yes, of course. I edited the question.
Comment: On which line is the error occurring? Is it when you initialize formatter, or when you initialize parsedDates?
Comment: I posted the whole error, but I think the problem is formatter.
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/195290/discussion-between-jonathan-myers-and-lazar-gugleta).
Answer: The error is occurring because the function is still being defined outside the scope of your map. Re-format how your map is operating to include the DateTimeFormatter being defined inside of map as follows:
<code>val evega = concat.map(_.split(",")).keyBy(_(0)).groupByKey().map{case (k, v) => {
    val formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd")
    val parsedDates = v.map(sa => LocalDate.parse(sa(1), formatter))
    parsedDates.max.getDayOfYear - parsedDates.min.getDayOfYear
}}
</code>


What is the difference between a chamber choir and a church choir?

Question: What is the difference between a chamber choir and a church choir?
Do both sing hymns at a church service?
Answer: Chamber choirs play concert performances, church services aren't typically considered "concerts."
https://en.wikipedia.org/wiki/Chamber_choir

A chamber choir is a small or medium-sized choir of roughly 8 to 40 singers (occasionally called 'chamber singers'), typically singing classical or religious music in a concert setting. (This is distinct from e.g. a church choir, which sings in religious services, or choirs specializing in popular music such as a barbershop chorus).
Comment: This is basically true, but it is also possible that there are churches with a choral program that includes a group called "chamber choir."
Comment: A local church have a chamber choir and I was wondering if such a choir would sing at a church service.
Answer: My deleted answer was: "Basically, one fits in a chamber and the other one in a church ;-)" To clarify: That's exactly the difference, as both can perform any kind of music, because the number of voices in any choir part rarely exceeds the number of people in a chamber choir and almost never exceeds the number of people in a church choir. Rare examples are contemporary music parts like those from Ligety or Stockhausen. If administrator "Tim, Dom" (sic!) lacks the knowledge of music that is asked for here, he better go and troll contributors elsewhere. Thank you.
Answer: The currently accepted answer is essentially correct, but it doesn't consider the context you've mentioned in a comment:

A local church have a chamber choir and I was wondering if such a choir would sing at a church service.

The answer is probably yes.  While it is true that "chamber choir" is typically used for small choral ensembles that present concerts rather than singing in church services, the more important element of the definition is the size of the ensemble.  In a church with a choral program that comprises more than one choir, it would not be exceptional in the least to designate the smaller of two (otherwise similar) groups as the "chamber choir".  Such a group would likely participate in services.


Condition number inequality

Question: Let A be an invertible n x n matrix,
How to show that:
$K(A) \ge \dfrac{\|A\|}{\| B - A \|}$
where $K(A)$ is the condition number of the matrix $A$ and
for any $B$ being an $n\times n$ singular matrix.
Thank you!
Answer: $K(A) = \|A\| \|A^{-1}\|$, so the inequality in question is $\|A - B\| \ge \frac 1{\|A^{-1}\|}$.
Pick a non-zero vector $x \in \ker(B)$. Then $(A - B)x = Ax$, so $\|A - B\| \ge \frac{\|Ax\|}{\|x\|}$.
Next, let $y = Ax$. Then $\frac{\|Ax\|}{\|x\|} = \frac{\|y\|}{\|A^{-1}y\|} \ge \frac{1}{\|A^{-1}\|}.$
Answer: Let $x$ be a nonzero vector in $\ker B$ and let $X$ be the square matrix in which every column is equal to $x$. By a suitable scaling of $X$, we may assume that $\|X\|=1$. Therefore
\begin{align*}
\kappa(A) &= \frac{\kappa(A)\|B-A\|}{\|B-A\|}
= \frac{\|A\|\|A^{-1}\|\|B-A\|\|X\|}{\|B-A\|}\\
&\ge \frac{\|A\|\|A^{-1}(B-A)X\|}{\|B-A\|}
= \frac{\|A\|\|0-X\|}{\|B-A\|}
= \frac{\|A\|}{\|B-A\|}.
\end{align*}
Note: The above proof only makes use of the submultiplicativity of a matrix norm. We deliberately avoid using the inequality $\|M\|\ge\|Mx\|/\|x\|$. While it is true that for any submultiplicative matrix norm $\|M\|$ for square matrices, there exists a compatible vector norm $\|x\|$ such that $\|M\|\ge\|Mx\|/\|x\|$ for any $M$ and any $x\not=0$, this fact is usually not taught in classes.

Condition number inequality


Why does the output contain a blank line at the end?

Question: My file is just one line:
$ cat employee-one-line.txt
101,John Doe:102,Jason Smith:103,Raj Reddy:104,Anand Ram:105,Jane Miller
$

But executing the following awk script, it prints a redundant empty line at end:
$ awk -F, 'BEGIN{ RS=":" } {print $2}' employee-one-line.txt
John Doe
Jason Smith
Raj Reddy
Anand Ram
Jane Miller

$

Why is there a blank line at end?
Comment: To debug output formatting issues, it's a good idea to place characters around the output so you can see what's coming from where. In this case change `{print $2}` to `{print "<" $2 ">"}` and it should be much clearer what's happening.
Answer: Since you have set the record separator to a colon, the last record is 105,Jane Miller, followed by a newline character. print adds an ORS (output record separator) to the end of things that it prints, so you end up with two.
The default behaviour would be that awk consumes the newline as the record separator, then adds it back in whenever you print.
In order to make awk swallow the newline in this case, one option would be to add it to your field separator:
awk -F'[,\n]' 'BEGIN{ RS=":" } {print $2}' file

Answer: Once you defined RS=':', a newline is no longer a record separator, so the newline terminating your file got appended to "Jane Miller".


Need to override single Drupal 8 plugin function

Question: What I want to do is edit what is displayed in the autocomplete/drop selects for an entity reference. Basically instead of just [title] I want to show [title] - [other_distinguishing_field]. I know that I can do this by editing getReferenceableEntities in Drupal\Core\Entity\Plugin\EntityReferenceSelection. All I want to do is override that function. I read that I need to make a plugin for this, but I find all of the tutorials a bit confusing and I don't understand how to implement what I want using a plugin.  Do I need a plugin manager? Do I need to use annotations? Where do I reference Drupal\Core\Entity\Plugin\EntityReferenceSelection? Where do I override the function? Do I need to configure anything on the UI the use my plugin or will it automatically work?
**EDIT
Here is what my Plugin manager looks like using the solution provided by @ya.teck
    <?php
/**
 * @file
 * Contains ExampleEntityReferenceManager
 */

namespace Drupal\example;

use Drupal\Core\Plugin\DefaultPluginManager;
use Drupal\Core\Cache\CacheBackendInterface;
use Drupal\Core\Extension\ModuleHandlerInterface;

/**
 * ExampleEntityReference plugin manager.
 */
class ExampleEntityReferenceManager extends DefaultPluginManager {

  /**
   * Constructs an ExampleEntityReferenceManager object.
   *
   * @param \Traversable $namespaces
   *   An object that implements \Traversable which contains the root paths
   *   keyed by the corresponding namespace to look for plugin implementations,
   * @param \Drupal\Core\Cache\CacheBackendInterface $cache_backend
   *   Cache backend instance to use.
   * @param \Drupal\Core\Extension\ModuleHandlerInterface $module_handler
   *   The module handler to invoke the alter hook with.
   */
  public function __construct(\Traversable $namespaces, CacheBackendInterface $cache_backend, ModuleHandlerInterface $module_handler) {
    parent::__construct('Plugin/EntityReferenceSelection', $namespaces, $module_handler, 'Drupal\Core\Entity\EntityManagerInterface', 'Drupal\Core\Entity\Annotation\EntityReferenceSelection');

    $this->alterInfo('example_entity_reference_manager_info');
    $this->setCacheBackend($cache_backend, 'example_entity_reference');
  }
}

Comment: Please don't implement your answer in your question. Instead it's much better to add another answer, so others can find it where they expect it to be. It's totally OK to answer you own questions.
Answer: Method #1
You can simply create new plugin which extends class of target Core plugin.
File: modules/example/src/Plugin/EntityReferenceSelection/ExampleSelection.php
namespace Drupal\example\Plugin\EntityReferenceSelection;

use Drupal\Component\Utility\Html;
use Drupal\Core\Entity\Plugin\EntityReferenceSelection\DefaultSelection;

/**
 * Entity Reference Selection with distinguishing field.
 *
 * @EntityReferenceSelection(
 *   id = "example",
 *   label = @Translation("Example"),
 *   group = "example",
 * )
 */
class ExampleSelection extends DefaultSelection {

  /**
   * {@inheritdoc}
   */
  public function getReferenceableEntities($match = NULL, $match_operator = 'CONTAINS', $limit = 0) {
    $target_type = $this->configuration['target_type'];

    $query = $this->buildEntityQuery($match, $match_operator);
    if ($limit > 0) {
      $query->range(0, $limit);
    }

    $result = $query->execute();

    if (empty($result)) {
      return [];
    }

    $options = array();
    $entities = $this->entityManager->getStorage($target_type)->loadMultiple($result);
    foreach ($entities as $entity_id => $entity) {
      $bundle = $entity->bundle();

      $translated_entity = $this->entityManager->getTranslationFromContext($entity);
      $label = $translated_entity->label();
      $distinguishing_field = $translated_entity->get('field_example')->get(0);
      if ($distinguishing_field) {
        $label .= ' [' . $distinguishing_field->getValue()['value'] . ']';
      }
      $options[$bundle][$entity_id] = Html::escape($label);
    }

    return $options;
  }

}

Having this done navigate to field settings page and enable your reference method.
Method #2
For select list widget you can install Entity Reference Views Select which allows to build selection options using Views fields.
Method #3
Content Browser module (not stable yet) offers more advanced way to select entities.
Method #4
You may globally override Entity::label() method for required entity type. So that the [title] - [other_distinguishing_field] pattern will be used everywhere including default entity reference selection plugin.
Comment: For method #1, what happens when 2 modules extend the same plugin class? Which "wins"?
Comment: The one selected on field settings form wins. So it doesn't matter how many selection plugins exist until they have unique IDs (configured in annotation). I've just updated the answer.
Answer: You should implement hook_entity_reference_selection_alter().
function mymodule_entity_reference_selection_alter(&$definitions) {
  // Change your class here.
}


java serial read() is waiting indefinitely - how to break it?

Question: I have a serial device connected to my PI that I read data from it...
everything is good , but sometime the cable move or the serial device is unplug.
then the
line = r.readLine();

get stuck
I have try to overcome this problem by:
BufferedReader r = new BufferedReader (newnputStreamReader(p.getInputStream()));
         try
            {
                line = r.readLine();
                if (line.length() == 0)
                {
                    logfile.append("problem");
                    logfile.close();
                    System.out.println("Problem");
                    TimeUnit.SECONDS.sleep(5);
                    break;
                }

            }
            catch (IOException e)
            {
                logfile.append(line);
                logfile.close();
            }

but it doens't do nothing (because I geuss he is still waiting for data)
not even throw an exception
how can I make him say I have a problem ?
maybe to use a timer or something like this ?
if no data for 5 seconds ?
Thanks ,
Comment: Is `r` a `BufferedReader` by any means?
Comment: sorry      BufferedReader r = new BufferedReader (new InputStreamReader(p.getInputStream()));
Answer: Your assumption in this case is right. The readLine() method of the BufferedReader has an internal while-loop which will retrieve all bytes from the underlying input stream and will only break if the arriving character is either \n or \r.
Think of it like:
while(lastChar != '\n' || lastChar != '\r')
{
     //read from stream
}

However once entered the method will not return. The only exceptions are the occurence of those two special characters or if the InputStream is closed (in which case null ist returned).
The trick is to no enter until there is something to read from the InputStream:
public static void main( String[] args ) throws IOException
{

  boolean awaitInput = true;

  while(awaitInput)
  {
    if(System.in.available() != 0)
    {
      awaitInput = false;
      // read logic
    }
  }
}

This is only one of many possible solutions and I used System.in as an example since it is a InputStream like any other as well. But there is also a method called BufferedReader#ready which returns true if there is something to read:
public static void main( String[] args ) throws IOException
{

  BufferedReader br = new BufferedReader( new InputStreamReader(System.in) );

  boolean awaitInput = true;

  while(awaitInput)
  {
    if(br.ready())
    {
      awaitInput = false;
      String line = br.readLine();
      // read logic
    }
  }
}

At last if you want a timeout you can easily do it yourself like this:
public static void main( String[] args ) throws IOException
{
  BufferedReader br = new BufferedReader( new InputStreamReader(System.in) );
  boolean awaitInput = true;
  long timeout = System.currentTimeMillis() + 5_000;
  //                                          ^^^^^ 5_000ms = 5 sec

  while(awaitInput && System.currentTimeMillis() < timeout)
  {
    if(br.ready())
    {
      awaitInput = false;
      String line = br.readLine();
      // read logic
    }
  }
}

Comment: great I have done what you expalin using the System.in.available - and now I think it's OK . I will continue work on it and see if I have more problem... Thanks
Comment: I have try for 1 day to run the r.ready() ,2nd option . and it's gibe me fault once every ~ 3 seconds. while whn I read the data with simple serial program (putty) - I don't get any disconnection.... so it's strange , maybe I can give the read "more" time to read?
Answer: You can use CompletableFuture to read concurrently and be able to use a timeout.
// wrap the readLine into a concurrent call
CompletableFuture<String> lineFuture = CompletableFuture.supplyAsync(() -> r.readLine());
try {
    // do the call, but with a timeout
    String readLine = lineFuture.get(5, TimeUnit.SECONDS);
    // do stuff with the line you read
} catch (TimeoutException e) {
    // plug pulled?
}


How to Link MVC Model data with Angular JS Model

Question: I am using MVC Razor View with AngularJS.
I am creating the UserMaster object in controller and passing to the view
<code>Function AddNewUser() As ActionResult
       Dim objUser As New UserMaster()
        Return View("UserMaster", objUserMaster)
End Function
</code>
In View, By using HTML helper classes the text box and validation controls will be created
<code>@<div ng-app ng-controller="UserController">
    @Html.EditorFor(Function(model) model.UserName)
    @Html.ValidationMessageFor(Function(model) model.UserName)
</code>
In the client side using the following statement the AngularJS model is created
<code>    <script type="text/javascript">
           function UserController($scope, $http) {
           $scope.UserData = @Html.Raw(Json.Encode(Model));
           }
    </script>
</code>
The server side validations(which I have written in the UserMaster Class) are working fine in client side, The razor engine is automatically doing that by generating client side validation scripts.
I am automatically getting flourished model in server side after submit.
But I am not able to manipulate/get the model data in client side using AngularJS (by using $scope.UserData.UserName). I want to access the UserName value inside the text box when the user is modifying it by using angular JS.. How?
Comment: did you ever get this figured out?
Comment: yes... I will update the answer...
Answer: It can be possible by using htmlAttributes in MVC - 5
<code>@Html.EditorFor(Function(model) model.UserName, New With {Key .htmlAttributes = New With {Key .ng_model = "model.UserName"}})
</code>
we have to use ng_model insted of ng-model and MVC will render as ng-model in runtime


How to show that the image of the morphism $X \mapsto X^{p^n}$ equals the separable closure.

Question: Let $K$ be a field with $\operatorname{char}(K) = p > 0$ and let $L = K[X]/(f)$ for some irreducible polynomial $f \in K[X]$. We know that there is a separable polynomial $q_{sep}\in K[X]$ such that $q_{sep}(X^{p^n}) = f(X)$ for some $n\in \mathbb{N}$. Lastly let $\phi:K[X]/(q_{sep}) \to L$ be the $K$-homomorphism with $\phi(X) = X^{p^n}$. I have to show that $\text{im}(\phi) \cong K_s$ where $K_s$ is the separable closure of the extension $L/K$.
Note that $\phi$ was made using the isomorphism theorem and so I know that $K[X]/(q_{sep}) \cong \text{im}(\phi)$
To show that $\text{im}(\phi)\subseteq K_s$ I showed that $K\subseteq K[X]/(q_{sep})$ is a separable algebraic extension. Then it follows that $K\subseteq \text{im}(\phi)$ is a separable algebraic extension and therefore $\text{im}(\phi) \subseteq K_s$.
But now I'm stuck trying to prove the other inclusion.
I know that if I can show that $[K_s:\text{im}(\phi)] =1$, $[K_s:K] = \deg(q_{sep})$ or $[L:K_s] = p^n$ then I have proven that they have the same dimension over K and thus are equal. But I don't know how to continue from here. I could show that $[K_s:\text{im}(\phi)] =1$ by showing that there is only 1 homomorphism from $K_s$ to an algebraic closure that is the identiy on $\text{im}(\phi)$. But I also don't know how to continue from here.
Because $\text{im}(\phi) \subseteq K_s$ is a finite separable extension I know that there is a primitive element $\alpha \in K_s$ such that $K_s = \text{im}(\phi)(\alpha)$. Now I would have to show that $\alpha \in \text{im}(\phi)$. But I'm again stuck here.
All of these seem promising. But it always leads back to showing that a separable polynomial of $K[X]/(f)$ is also in the image of $\phi$.
Could you please give me a hint on how to prove this?
Answer: As you said, we start by noting that $\operatorname{im}(\phi)/K$ is a separable subextension of $L/K$. The next thing we have to show is that $L/\operatorname{im}(\phi)$ is purely inseparable.
But notice that $L = \operatorname{im}(\phi)(X)$ is produced by adjoining a $p^n$th root of $X^{p^n}$ to $\operatorname{im}(\phi)$. This sort of extension is always purely inseparable (see here for a proof).


JSONObject to ArrayList square brackets missing

Question: I am trying to get the rates from https://api.ratesapi.io/api/latest into an ArrayList<Currency> of a custom Currency class:
public class Currency {
    private String shortName;
    private double rate;
    ...
}

The JSON looks like:
{"base":"EUR","rates":{"GBP":0.90033,"HKD":9.1786,"IDR":17304.0,
 "ILS":4.0309,"DKK":7.45,"INR":88.765,"CHF":1.0759,"MXN":26.615,
 "CZK":26.202,"SGD":1.6236,"THB":36.832,"HRK":7.468,"MYR":4.9604,
 "NOK":10.6538,"CNY":8.2325,"BGN":1.9558,"PHP":58.136,"SEK":10.3165,
 "PLN":4.4073,"ZAR":20.7655,"CAD":1.5748,"ISK":160.2,"BRL":6.334,
 "RON":4.836,"NZD":1.7828,"TRY":8.5853,"JPY":124.96,"RUB":86.9321,
 "KRW":1404.99,"USD":1.1843,"HUF":346.23,"AUD":1.6492},"date":"2020-08-06"}

Using org.json I managed to get the data into a JSONObject:
JSONObject obj = new JSONObject(getJSON("https://api.ratesapi.io/api/latest"));

As far as I understand, the normal procedure is now to convert the JSONObject into a JSONArray. However trying:
JSONArray jsonArray = obj.getJSONArray("rates");

fails with the error message:
Exception in thread "main" org.json.JSONException: JSONObject["rates"]
is not a JSONArray.

How do I fix this error or is there another way to make an ArrayList out of the JSON?
I suspect that the problem are missing square brackets in the JSON string.
Comment: Does this answer your question? [simplest way to read json from a URL in java](https://stackoverflow.com/questions/4308554/simplest-way-to-read-json-from-a-url-in-java)
Comment: @KaustubhKhare while related, that dupe isn't going to help OP because the class they want to parse into does not match the structure of the JSON content.
Comment: This is closer https://stackoverflow.com/q/22687771/1690217 the issue is that `rates` is an object with the _properties_ that match your `Currency` structure, you need to convert the `rates` object into an array of it's properties.
Answer: If you take a look at the JSON returned by the API, you get a JSON object:
{"base":"EUR","rates":{"GBP":0.90033,"HKD":9.1786, ... },"date":"2020-08-06"}

You probably want to do something like this:
JSONObject obj = new JSONObject(getJSON("https://api.ratesapi.io/api/latest"));
JSONObject rates = obj.getJSONObject("rates");
final Iterator<String> keys = rates.keys();
while (keys.hasNext()) {
  final String key = keys.next();
  final Currency currency = new Currency(key, rates.getDouble(key));
  // do something with the Currency
}

Comment: @user1583209 as you're working with currency data I'd advice not to use double for the computation. Instead use `BigDecimal` to avoid rounding issues. Something like `new BigDecimal(rates.getString(key))` will probably do
Answer: A working solution using Jackson library and Lombok may be as follows:
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import lombok.*;
import java.util.*;
import java.util.stream.Collectors;

public class CcyApiParser {
    @Getter
    @Setter
    @AllArgsConstructor
    @NoArgsConstructor
    @ToString
    public static class Currency {
        private String shortName;
        private double rate;
    }

    @Getter
    @Setter
    public static class RatesApiResponse {
        private String base;
        private Map<String, Double> rates;
        @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
        private LocalDate date;
    }

    public static void main(String[] args) throws IOException {

        ObjectMapper mapper = new ObjectMapper()
                .registerModule(new JavaTimeModule()); // to parse date

        URL apiUrl = new URL("https://api.ratesapi.io/api/latest");

        // read proper api response
        RatesApiResponse rates = mapper.readValue(apiUrl, RatesApiResponse.class);

        // convert inner rates into list of Currency objects
        List<Currency> ccys = rates.getRates().entrySet().stream()
                .map(e -> new Currency(e.getKey(), e.getValue()))
                .collect(Collectors.toList());

        ccys.forEach(ccy -> System.out.printf("%s=%s%n", ccy.getShortName(), ccy.getRate()));
    }
}

Output
GBP=0.90033
HKD=9.1786
IDR=17304.0
ILS=4.0309
... etc.

Update
It is also possible to customize deserialization of RatesApiResponse and move mapping of "rates" into this class to convert immediately into list of currencies.
    @Getter
    @Setter
    public static class RatesApiResponse {
        private String base;
        @JsonProperty(access = JsonProperty.Access.READ_ONLY)
        private List<Currency> ccys;

        @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
        private LocalDate date;

        // no getter for rates
        // this customized setter for the map of rates converts into a list
        @JsonProperty("rates")
        public void setRates(Map<String, Double> rates) {
            ccys = rates.entrySet().stream()
                    .map(e -> new Currency(e.getKey(), e.getValue()))
                    .collect(Collectors.toList());
        }
    }

// Updates in the test method
RatesApiResponse rates = mapper.readValue(src, RatesApiResponse.class);

rates.getCcys().forEach(ccy -> System.out.printf("%s=%s%n", ccy.getShortName(), ccy.getRate()));

Answer: You can use ObjectMapper class to convert json from some URL to some kind of object. In this case (if json structure is always the same) it can be Map<String, Object>.
ObjectMapper mapper = new ObjectMapper();
URL url = new URL("https://api.ratesapi.io/api/latest");
Map<String, Object> map = mapper.readValue(url, Map.class);

System.out.println(map);
// {base=EUR, rates={GBP=0.90373, HKD=9.1585, ... , AUD=1.6403}, date=2020-08-07}

Then you can get inner rates map, and (if it is needed) convert it to list using java stream api:
Map<String, Double> rates = (Map<String, Double>) map.get("rates");

System.out.println(rates); // {GBP=0.90373, HKD=9.1585, ... , AUD=1.6403}

Convert Map<String, Object> to ArrayList<Currency>:
ArrayList<Currency> list = rates.entrySet().stream()
    .map(entry -> new Currency(entry.getKey(), entry.getValue()))
    .collect(ArrayList::new, ArrayList::add, ArrayList::addAll);

System.out.println(list); // [GBP=0.90373, HKD=9.1585, ... , AUD=1.6403]

Note: add a constructor with two fields shortName and rate;

Note: override the toString method as follows: shortName + "=" + rate;

Maven dependency:
<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-xml</artifactId>
    <version>2.11.2</version>
</dependency>

See also: «Formatting Json Response into an Array Java».
Answer: The object "rates" is not a JSONArray, is a JSONObject.
So you have to do obj.getJSONObject(rates");then iterate on the fields of the JSONObject using map methods (for examply using keySet() )
Answer:
Exception in thread "main" org.json.JSONException: JSONObject["rates"]
is not a JSONArray.

You got this error because rates is not in the form of an array. It is simply an element like base and date but looks like an array. Get it from the JSON string like you get  base and date from it and then process it to create the required List<Currency>.
Given below is the working code with the explanation added as comments in the code:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.net.URLConnection;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.List;

import org.json.JSONException;
import org.json.JSONObject;

class Currency {
    private String shortName;
    private double rate;

    public Currency(String shortName, double rate) {
        this.shortName = shortName;
        this.rate = rate;
    }

    @Override
    public String toString() {
        return shortName + ":" + rate;
    }
}

public class Main {

    public static JSONObject getJSON(String url) throws IOException, JSONException {
        // Create a URLConnection for the given URL
        URLConnection connection = new URL(url).openConnection();

        // Add header to avoid 403 Forbidden HTTP status code
        connection.addRequestProperty("User-Agent",
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:79.0) Gecko/20100101 Firefox/79.0" + "");

        StringBuilder jsonStr = new StringBuilder();

        // Get InputStream from connection and read the response
        try (InputStream is = connection.getInputStream();) {
            Reader reader = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));

            int ch;
            while ((ch = reader.read()) != -1) {
                jsonStr.append((char) ch);
            }
        }
        return new JSONObject(jsonStr.toString());
    }

    public static void main(String[] args) throws IOException, JSONException {
        JSONObject jsonObj = getJSON("https://api.ratesapi.io/api/latest");

        // Get rates from jsonObj
        String rates = jsonObj.get("rates").toString();

        // Remove {, }, and " from the string
        String[] keyValArr = rates.replaceAll("[\\{\\\"}]", "").split(",");

        // List object to hold Currency objects
        List<Currency> list = new ArrayList<>();

        for (String keyVal : keyValArr) {
            // Split each key:value string on ':'
            String[] curRate = keyVal.split(":");

            // Add Currency object to List
            list.add(new Currency(curRate[0], Double.parseDouble(curRate[1])));
        }

        // Display list
        list.forEach(System.out::println);
    }
}

Output:
CHF:1.0804
HRK:7.4595
MXN:26.5127
...
...
...
NZD:1.7786
BRL:6.3274

Comment: **Note:** I used the Firefox plugin, [HTTP Header Live](https://addons.mozilla.org/en-US/firefox/addon/http-header-live/) in order to get the value for the header, `User-Agent`.


Bayesian Analysis in the Absence of Prior Information?

Question: I have always wondered - how confident do researchers tend to be in their "prior" information when deciding to create statistical models using a Bayesian Approach vs. a Frequentist Approach?
As an example : Suppose a team of researchers has some data relating to the healthcare industry - let's say that they are interested in fitting regression models to this data. As such, they are familiar with the scientific background of the data they are working with, but they are unsure what kinds of priors they can place on the parameters of these regression models.
I saw this post over here (What is an "uninformative prior"? Can we ever have one with truly no information?) in which the idea of an "Uninformative Prior" is provided - but apart from the idea of the Uninformative Prior, I can not seem to find more information on this topic.
As such, is the following idea true? When we don't have a lot of confidence on information that can be used to construct Bayesian Priors - is it generally better to stick to Frequentist Approaches instead of "guessing" which Priors will better fit the data and the choice of model?
Comment: The prior distribution is not a "truth" we should aim for. It constitutes a reference measure to which the posterior distribution is compared to assess the information contained in the data. This is why several priors can coexist, why credible intervals are not confidence interval, and the probabilistic meaning of the posterior statements remain mostly epistemic. And definitely not seeking which prior fit the data best.
Comment: Whether you paint on a blank, beige, or vanilla canvas, the painting will look very similar. Professional painters might argue about different qualities of canvas and how pigments can have different adhesion. But that doesn't destroy the analogy and even allows to expand it, we could say that the quality of the analysis is more important than the bias/colour of the naked prior information that is used.
Comment: Is the question about "confidence in prior information" only relevant when using Bayesian approaches? For example, how confident are researchers in the assumptions necessary to do power analysis & sample size calculations for study design?
Answer: If you have no information, you can use "uninformative" priors.  Those priors aim to bring as little information as possible, but as you already learned from the What is an "uninformative prior"? Can we ever have one with truly no information? thread, the name is a little bit misleading because even such priors bring some information to the model. That is why more modern recommendation would be to pick a weakly informative prior (centered on something, but very uncertain).
On another hand, using a frequentist model does not mean making no assumptions: you may still assume things like Gaussian likelihood, a linear relationship between the variables, or using regularized regression you are using implicit priors on the parameters, etc. Even if you wanted to make as few assumptions as possible and used a nonparametric model, you would be making some assumptions. For example, say that you would pick $k$NN regression that just averages among "similar" observations. Still, you need to decide on a similarity metric to define what "similar" means and you need to somehow pick the hyperparameter $k$. With Bayesian models, you make additional assumptions by choosing priors, but other approaches also make assumptions.
But, can you have truly no information about something? Say a meteor hits the planet Earth and brings us a new virus from space. A Bayesian statistician needs to build a model on it. They know nothing about space viruses. Hopefully, they know a lot about viruses from Earth, many scientists also did a lot of educated guesses on what extraterrestrial life forms could be and how they could be similar or different to the life on Earth, etc. The scientist in fact has a lot of prior knowledge and assumptions that they could use to come up with priors.
Comment: *using a frequentist model does not mean making no assumptions: you may still assume things like Gaussian likelihood, a linear relationship between the variables* is correct, but it may also be interesting to note that the same assumptions are made in a Bayesian model. So roughly speaking, the Bayesian model has one *extra* assumption: the prior. Not saying this is bad or anything, though.
Comment: Thanks everyone! I just can't seem to find a specific reference which shows how a group of researchers decided what prior to assume for their models while working on a specific problem. Logically, I always thought this was done through some form of meta-analysis (e.g. read previous research papers on similar topics, decide which papers are suitable to your problem and performed similar analysis under similar conditions ... and then average the findings to create a prior for your problem) .... but I am no sure if this is correct?
Comment: @RichardHardy sure, I assumed its clear but edited to clarify.
Comment: @stats_noob check https://stats.stackexchange.com/questions/1/eliciting-priors-from-experts
Comment: @stats_noob.  here is a paper on how to choose priors in ecological Bayesian analyses that I found very useful for these types of discussions:  https://onlinelibrary.wiley.com/doi/full/10.1111/oik.05985
Answer: I think that the posterior inherits meaning from the prior, which implies that if the prior is meaningless, so is the posterior.
Now there are different varieties of Bayesians when it comes to interpreting probabilities, i.e., assigning meaning to priors (and posteriors). Most (but not all) Bayesian interpretation of probabilities is epistemic, meaning that probabilities formalise a state of belief or knowledge rather than a data generating mechanism that exists in reality (as frequentists do).
Subjectivist Bayesians traditionally state that the prior should formalise your personal prior belief. This particularly means that probabilities do not model data, and data cannot contradict the prior (as observing data later cannot change your belief before data). Note that this applies to both the parameter prior and the likelihood in standard Bayesian setups in which there is a parametric likelihood and a prior on the parameters. The original literature (de Finetti and Ramsey) postulates that you basically have prior beliefs about everything that can be formalised as prior distribution. One way to operationlise that is to ask you to bet on how the data will turn out before you see it, implying that your are forced to offer bets on all kinds of possible outcomes. Existing prior information will make you expect certain outcomes more than others, so that you should put higher prior probability on these. If you don't have much information, you need to spread out probability so that everything that is possible has enough share in the overall distribution that data ultimately can push the posterior there if there is a clear message from the data.
The thing is that in real data analysis hardly anyone is forced to bet in advance, and for sure not on enough events to determine the prior completely. In fact there is some literature about imprecise probabilities in which you are not forced to bet but where the few bets (or specifications, if proper bets are not involved) you are willing to make determine "upper" and "lower" (prior, but later also posterior) probabilities. A similar thing happens in Bayesian sensitivity analysis - you may not be willing to specify a precise prior, but rather a range of different priors may look appropriate to you, and you can run Bayesian analyses with all of these and see how much results differ.
In reality, you may have some information, but this information may not translate readily into prior probabilities such as "the probability for $\theta$ to be between -2 and 2 is 85%"; you may think it's likely, but whether that translates into 70, 85, or 92% may not be so clear to you, and is certainly not enforced by the information itself. The subjectivist approach is that you should basically then decide how to place your bets (i.e., if you are ready to pay 85 for a possible win of 100 in case this happens, or rather not). What I think is misleading about this idea is that subjectivists seem to think that there exists some "true" personal prior that can be "found out" in this way, whereas I think that you are rather forced to make it up out of more or less thin air if you want to do a Bayesian analysis. Obviously sensitivity analysis can show you, when things run well (which often means that you need to have a lot of data), that your final conclusions may not be all that different given different choices that look equally plausible to you, and although you make some choices that don't seem that well founded, this doesn't affect the conclusions much. Unfortunately, in much published real Bayesian analysis, disappointingly little care is invested into motivating the prior and into exploring sensitivity in case of alternative choices, as these are hard tasks and everyone (not only the frequentists!) wants statistics to be easy. If indeed the posterior inherits meaning from the prior, there isn't much to expect from the resulting posteriors.
Objectivist Bayesians would not like to think of probabilities as made up by individuals forced to bet, but the objectivist idea rather is that the prior should only formalise "objective" external information that can be verified and agreed upon. Otherwise non-informative priors should be used. Unfortunately this is of little help in the situation that there is prior information that doesn't come in form of objectively verifiable probabilities. Almost all prior information is of this kind. In practice objectivist analysis therefore either becomes subjective at least to some extent (J. Berger says in some papers that so-called objective Bayes is not really objective but rather points to the attitude that we should try as hard as we can to reach an ultimately unattainable ideal), or people use non-informative priors despite in fact having some information, which deprives the Bayesian approach of one of its major benefits. (On top of that, even what is called non-informative priors will often carry some subtle information that may harm analyses if not consciously acknowledged and used.)
There is also the possibility to actually interpret probabilities in an aleatory way (i.e., non-epistemic, referring to data generating processes in the world; some say "frequentist). A. Gelman occasionally advertises Bayesian statistics in this way. The data is then in fact informative about the prior, and if the data contradict it, it may be changed, although this violates the Bayesian concept of coherence. For setting up a convincing informative model this may make sense, but posterior probabilities should not be interpreted in a naive fashion such as "after data the probability that the true $\theta$ is between -2 and -1.5 is 98.5%", because the model within which that "true $\theta$" is defined is an idealisation and may be changed with more information coming in (which by the way should also be kept in mind for frequentist analyses); also the potential of changing the prior with the data may lead to overprecision, i.e., credible intervals may be too small because the prior that in Bayesian analysis is meant to be fixed before data has been adapted to the data (the same issue occurs in many frequentist analyses).
Personally I think there are big issues with any kind of statistical approach; Bayesians often tell frequentists off for many issues that in some way or another they actually face themselves (for example there is the accusation of "adhockery", frequentists are accused of adapting their approach in non-principled and irregular ways to the situation at hand, but if Bayesians want to learn and adapt the prior rather than being constrained by whatever their first prior choice implies, they violate one of their own major principles).
Setting up a prior basically means to put weights on the parameter (and observation) space, which will then influence analyses. There are situations in which there are good reasons to do this, particularly if there is some information that is meant to influence the analysis (but then of course this needs to be translated into prior probabilities and this is hard). I'm not keen though on doing Bayesian analysis for the sake of it, so from my personal view, if there is no prior information that you want to use to do this kind of reweighting, I'd be happy to do a frequentist analysis. (Note that there are still various ways how prior information can influence your analysis; the idea that relevant information should always and only be used to determine prior probabilities for outcomes and parameters is nonsense.)
I will also acknowledge that there may be reasons to want epistemic probabilities rather than frequentist ones (particularly if the frequentist ideal of unlimited identical repetition of experiments doesn't seem convincing), in which case a Bayesian approach even with supposedly non-informative priors can be preferred. However, the postulate that there are unique true epistemic probabilities looks just as questionable to me as the postulate of infinitely repeatable experiments, resulting in the existence of true frequentist probabilities. Also, be careful to prevent your prior from bringing in some implicit informations/implications that you don't want.
Answer:
When we don't have a lot of confidence on information that can be used
to construct Bayesian Priors - is it generally better to stick to
Frequentist Approaches instead of "guessing" which Priors will better
fit the data and the choice of model?

In my opinion, the full power of a Bayesian analysis is unleashed when a priori information is available, although how to translate this prior information into a distribution is far from trivial and there may be several different ways to do it.
Indeed, building a prior from extra experimental (or expert opinions) data is, to some extent, akin to choosing the statistical model. Thus the inferential task for a subjective Bayesian is heavier than that of a frequentist statistician. The power, however, is that we can, at least in principle, include expert opinion or extra experimental information in our inferential conclusions, information that in a frequentist approach would have been otherwise discarded.
In absence of information, however, we also can find several compelling reasons why a Bayesian approach is still useful. In the linked post you find many of them, and I summarise them as

lead to inferential procedures with good frequentist properties (probability coverage, unbiasedness of the MAP)
default priors tailored to the parameter of the model at hand that are guaranteed to impact the posterior as less as possible.

Notice the bold default, which seems to be the most suitable adjective. Indeed, we cannot call them non-informative since "informativeness" is relative to the measure of information used and may depend on the scale of the parameter.
Among these compelling reasons, I find two of them that are particularly relevant. Firstly, some of these default priors have been discovered and used by frequentists in order to adjust their inference that otherwise would have been inaccurate or useless.
For instance, Firth(1993) showed that the Jeffreys prior in exponential families expressed in the canonical form leads to a maximum a posteriori (MAP) with reduced bias. Furthermore, and more importantly for frequentists, he proposed a weight function for frequentists which solves the problem of perfect separation in logistic regression.
However, the most compelling argument in the defence of default priors, such as the Jeffreys, is invariance under one-to-one reparametrizations. (personal communication by J. Berger in an O'Bayes meeting some years ago). And I totally agree with this. Indeed, if we think invariance to parametrization is a good thing, as most frequentists do, then that's the ultimate defence for the use of non-informative or default prior.
Comment: In some special cases, as you state, a Bayesian procedure can have better frequentist properties than a standard frequentist procedure, and of course then a frequentist should be fine with using the Bayesian procedure, which then actually is a good frequentist procedure. Other than that, your listed "compelling reasons" are not really reasons to prefer a Bayesian approach, as many frequentist procedures have the invariance property as well, and because the frequentist doesn't bother about the posterior, they may not be too interested in your second bullet point.
Comment: Invariance as I define it can be applied to *any* inference; there is nothing specifically frequentist about it, and neither is it exclusive to estimation.
Comment: Hi @ChristianHennig, thanks for your comment! Bayesian invariance has nothing to do with frequentist invariance, apart from sharing the same name! Indeed, a Bayesian estimator can never be invariant in a frequentist sense because of the Jacobian of the transformation, unless the transformation is linear. Am I misinterpreting your point?
Comment: My definition of parameterisation invariance would be that the "relevant outcome" of an analysis is invariant to reparameterisation. This is somewhat imprecise as it depends on what about an outcome is "relevant" (which is to be defined in each case), and applies to some frequentist methodology (ML) but not other (mean squared error), but in general it is a mathematical property and it doesn't matter whether the procedure it applies to is frequentist or Bayesian.
Comment: As far as I can see, yours is close to the definition of invariance (sometimes even called equivariance) in the frequentist sense, i.e. one that has to do with estimation. In a Bayesian context, invariance is related to the behaviour of the prior during a one-to-one change of variable and has nothing to do with the estimation of the parameter per se; see Bernardo and Smith (2000) _Bayesian Theory_, Sect. 5.6.2..
Comment: agreed! but the point is: it has nothing to do with the invariance of the prior, which is the point of my second bullet.


Supremum of set of products of elements of two sets

Question:
Let $A, B \subseteq \mathbb R$ and $C = \{ab\mid a\in A, b \in B\}$. Show $\sup C \geq \sup A \sup B$.

I don't really get the point here. I understand it intuitively - if $A$ and $B$ contain big negative numbers, the supremum of $C$ gets big since the negative signs disappear but the suprema of $A$ and $B$ may be small if $A$ and $B$ only contain small positive numbers.
If $A = \{-3,-2,0,1\}, B = \{-5,0,1\}$, then $\sup A = 1, \sup B = 1$ and $\sup C = (-3)(-5) = 15$. So in this case, the supremum would be bigger. But I don't know how to prove this formally...
Answer: HINT:
$$
\forall a\in A\,\forall b\in B\,a\leq\sup A\text{ and }\,b\leq\sup B\text{ and }ab\leq \sup C.
$$
Comment: Oh yes, then it implies $\sup A \sup B \leq \sup C$. That's all?
Comment: @Thomas Yes. :-)
Comment: Wait: $a \leq \sup A$ and $ab \leq \sup C$ does only imply $ab \leq \sup A \cdot b$ but not $\sup A \cdot b \leq \sup C$, does it?
Comment: But if all terms are positive doesn't that imply this exact *opposite* that $ab \le \sup A\sup B$ so $\sup A\sup B$ is an upper bound of $C$ and so $\sup A \sup B \ge \sup C$ which is the *least* upper bound.
Comment: Yeah, that should be true @fleablood, if $A$ and $B$ are bounded from below by $0$, the inequality is an equality. But I still do not get the argument from before....
Answer: Case 1: $\sup A = 0$ or $\sup B = 0$ (wolog $\sup A = 0$).
If $\sup C < \sup A \sup B = 0=\sup A$ then for all $a\in A, b\in B$ then $ab < 0$.  So $a \ne 0$ but $a < 0$ so $b > 0$.  So for any $b\in B; b> 0$ we can find an $a \in A$ so that $\frac {\sup C}b< a < 0$ so $\sup C < ab < 0$ .  But $ab \in C$ so that's a contradiction.
==========
Case 2: $\sup A < 0; \sup B < 0$.
Trivial:  If $a \le \sup A < 0$ and $b \le \sup B < 0$ then $ab \in C$ so $\sup C \ge ab \ge \sup A\sup B$.
========
Case 3:   $\sup A > 0$ and $\sup B > 0$
If $0 < k < \sup A\sup B$ then $\frac k{\sup B} < \sup A$.  So there is a $k' \in A$ so that $\frac k{\sup B} < k' \le \sup A$.
And $\frac k{k'}< \sup B$ so there is $\overline k \in B$ so that $\frac k{k'} <\overline k\le \sup B$.  So $k <k\overline k\in C$.  So $k$ can't be an upperbound of $C$.  So either $\sup A\sup B$ is a least upper bound or not an upper bound at all[*].
So $\sup A \sup B \le \sup C$.
[*] $\sup A\sup B$ need not be an upper bound if we can have negative elements in both sets with absolute values that exceep the suprememums of the set.  If $a<0$ and $|a|>\sup A$ and $b< 0$ and $|b| > \sup B$ then $ab >\sup A\sup B$.
=======
Case 4:  one supremum is positive and the other negative (wolog $\sup A < 0$ and $\sup B  >0$ )
If $\sup C < \sup A\sup B < 0$ then $\frac {\sup C}{\sup B} < \sup A < 0$ and we can find an $a\in A$ so that $\frac {\sup C}{\sup B} < a$.  As $\sup B > 0$ there is a $b \in B$ so that $0 < b \le \sup B$ so $\frac 1b \ge \frac 1 {\sup B}$ and $\frac {\sup C}b \le \frac {\sup C}{\sup B} < a < 0$.
So $\sup C < ab < 0$.  But $ab \in C$ and that is a contradiction.
Answer: We assume that $\sup A=:\sigma_A$ and $\sup B=:\sigma_B$ are finite. Let an $\epsilon>0$ be given. Since multiplication is continuous at the point $(\sigma_A,\sigma_B)\in{\mathbb R}^2$ there is a $\delta>0$ such that
$$|a-\sigma_A|<\delta \ \ \wedge\ \ |b-\sigma_B|<\delta\qquad\Rightarrow \qquad ab>\sigma_A\sigma_B-\epsilon\ .$$
By definition of the supremum we can choose here $a\in A$ and $b\in B$. This implies
$$\sigma_C\geq ab>\sigma_A\sigma_B-\epsilon\ ,$$ and as $\epsilon>0$ was arbitrary we can conclude that $\sigma_C\geq\sigma_A\sigma_B$.
Comment: I don't really get your continuity-argument. Can you explain it a little bit further?
Comment: @Thomas: See my edit.
Comment: Sorry - what do you mean with continuity of multiplication - what continuous function does that multiplication associate?
Comment: The function $(x,y)\mapsto p(x,y):=xy$ is continuous at all points $(x_0,y_0)\in{\mathbb R}^2$. This is a basic fact of real analysis.


StripeModule failing for android React Native project

Question: We have a project using Tipsi Stripe React module, when trying to build for Android I get the following error message: Native Module StripeModule tried to override StripeModule for module name StripeModule.
The error message does not make sense to me and I don't really know where to begin looking since i'm fairly new to mobile and React Native development.
Building for iOS works fine.
Answer: I have the same problem, i've tried adding
<code>@Override
public boolean canOverrideExistingModule() {
    return true;
}
</code>
to: /tipsi-stripe/android/src/main/java/com/gettipsi/stripe/StripeModule.java based on what i found here How to set canOverrideExistingModule=true.
It compiles, but I haven't tested if it keeps working, if it works i'll fork tipsi-stripe and create a PR later
Comment: I figured it out yesterday, in MainApplication.java, the method List getPackages() {} had the stripeModule listed twice, removing one of them worked. In that light, the error message makes sense as well. I will update my answer when I have some time.
Comment: Here you can look https://stackoverflow.com/questions/41846452/how-to-set-canoverrideexistingmodule-true-in-react-native-for-android-apps/63438974#63438974


Highest Goal Difference to be knocked out of the World Cup at the Group Stage

Question: During the final games of the group stages at the Qatar 2022 World Cup, for 3 brief minutes in Group E, both Spain and Germany were in line to be knocked out of the World Cup.
During those 3 minutes, with the scores being Costa Rica 2-1 Germany; Japan 2-1 Spain - the "As it stands" Group E table looked like this:

Group E
P
W
D
L
GD
Pts

Japan
3
2
0
1
+1
6

Costa Rica
3
2
0
1
-5
6

Spain
3
1
1
1
+6
4

Germany
3
0
1
2
-2
1

This would have been remarkable in of itself, but even more so that Spain had a +6 goal difference, mostly coming from their 7-0 demolition of Costa Rica in the opening round.
What is the highest Goal Difference a team has had and also been knocked out of the World Cup at the group stages.
Answer: That would be Hungary at the 1982 World Cup. They beat El Salvador 10-1 in their first match - the largest ever margin of victory at a World Cup - but then lost 4-1 to Argentina and drew 1-1 with Belgium, which meant they finished 3rd in the group with a goal difference of +6.


Socket.io client socket taking long to respond

Question: I am using socket.io to send/receive messages between client/server. The server has a Redis instance that stores data and responds with the data within milliseconds.
Some of the sockets take very long to return the data (stringified large JSON object) although it is sent from the server side almost immediately. I am therefore looking for suggestion that address the following concerns:

Is it normal for socket.io to take this long to emit a long string ?
How can I know which method or socket is doing the emit that takes long ?
Any further suggestions on how to improve performance?

Help really appreciated
UPDATE:
I tried using Webdis to provide the response to the client without having to go to the server to get the Redis results. However although the response appears in a console.log in about 1 second (which is the same as the DOMLoadedContent below), the websocket still takes about 20s and shows 0 bytes as shown below:
Comment: How long is the string? You refer to it as a "long string". Is it like megabytes or something?
Comment: @Nate The longest it about 19.5 MegaBytes
Comment: Hey @AfshinGhazi, any luck so far on the issue?
Comment: @ForgetfulFellow no haven't resovled yet. I used Webdis to provide the Redis data quickly to the client instead of the server (which would then provide to the client). However I can't understand why the websocket takes about 20s to finish wheras a console.log of the data takes about 1s.
Answer: It seems like you've hit issues that others have seen as well with large file uploads using socket.io:
Node.JS, Socket.IO & large XML files: extreme performance loss?
One possible course of action is to try file streaming in socket.io:
https://gist.github.com/companje/eea17988257a10dcbf04
As for

How can I know which method or socket is doing the emit that takes
  long ?

You can always pass in dateTime's into the client's data received from socket.io and then compute the time difference, printing out the methods that it called.
Comment: I'd also be curious of what is being sent that large. If it's media (image, audio, video) then it would make more sense to send a URL over the socket, and have the browser download the file like normal not through a socket. This approach would reduce load on the socket server and also allow the browser to cache the large data, which is better for everyone.
Comment: @Nate The large data comes from an api call response
Comment: @AfshinGhazi What's the data type? Because if that's all JSON data for building a UI, that sounds crazy!


Yii multiple relations between different models in different modules

Question: How would i add a 3rd relation to my relations() function to join the CommentsPosts model with the store model. Where CommentsPosts.store_id = Store.id and CommentsPosts.store_zip = Store.zip?
in my comment.php
  public function relations()
        {
            return array(
                'user' => array(self::BELONGS_TO, $this->module->userModelClass, 'userId'),
                'posts' => array(self::HAS_MANY, "CommentsPosts", array("commentId" => "id")),
                'store'=>'',

            );
        }

CommentsPost.php is in modules/comments/model and Store.php is in modules/store/model
FYI, i'm using Yii 1.1.15
Comment: I think you can add that relation in the store model
Answer: I'm not sure about your relations but try something like bellow
In Comment post Model add the relation
 public function relations()
        {
            return array(

                'store' => array(self::HAS_MANY, "Store", array("store_id" => "id")),

            );
        }

When retrieving
   $enity->post // give all post
   $enity->post[i]->store // gives the each post data of store

Comment: store is in a different `modules/model` than commentPost. is that ok?
Comment: Try it once with correct relations . I think it's not a problem
Comment: hmm, in my view. i added `print_r($data->store);` and i get this.. `Property "Comment.store" is not defined.`
Comment: Try first print_r ($data->post) if there was data try print_r ($data->post[0]->store)


Graphql: How can get field arguments of a type in resolver?

Question: I use graphql-tools library and makeExecutableSchema function to make my schema by passing schema and resolver to it
here is my schema:
type Trip {
  code: String!
  driver: User!
  vehicle: Vehicle!
  destination: Location!
  passengers(page: Int, count: Int): [User!]!
}
type Query {
  trip(id: String!): Trip
}

and here is my resolver:
// some imports...
export default {
  Query: {
    async trip(_, { id }, ctx, info) {
      const trip = await Trip.findById(id);
      // const page = ???, count = ???
      // work on fetch data...
      return result;
    },
};

how can I get page and count  which are defined as nested argument for passengers?
Comment: Possible duplicate of [How to pass root parameters in the resolver function of a nested query?](https://stackoverflow.com/questions/48094695/how-to-pass-root-parameters-in-the-resolver-function-of-a-nested-query). Check out [my answer](https://stackoverflow.com/questions/48094695/how-to-pass-root-parameters-in-the-resolver-function-of-a-nested-query/48096791#48096791) for that question for a detailed workaround.
Comment: @DanielRearden I need to access a nested argument when using`graphql-tools`, I know it exists in `info` parameter of resolver, but I have no idea how could I parse and use it
Answer: You should define a resolver for the type Trip, such as:
export default {
  Query: {
    async trip(_, { id }, ctx, info) {
      const trip = await Trip.findById(id);
      // const page = ???, count = ???
      // work on fetch data...
      return result;
    },
  Trip: {
    async passengers(trip, { page, count }, ctx, info) {
      ...
    },
  }
};

In GraphQL, it's not the concept of "nested fields of a type", but just combinations of "the type of a field". The trip field of type Query has the Trip type, so when you want to work with the passengers field, it should be considered as a field under the Trip type, not a nested field of the Query type.
Comment: thanks for your replication, it was so helpful.
do you know any way or tool to access the fields(`count` and `page`) on top level? because I want to fetch all data by joining query
I know all data are presented in `info` argument but how can it be parsed
Comment: I think things like Facebook's [dataloader](https://github.com/facebook/dataloader) are tools you might consider. Dataloader can [resolve every individual data request within a batch](https://github.com/facebook/dataloader#batching), and is a different approach of generating a `JOIN` query at top level (which isn't a flexible solution) to load data efficiently.


How to resolve conflict between zen theme breadcrumb function and Path Breadcrumbs?

Question: Apparently there Path Breadcrumbs module and the Zen theme's breadcrumb append title option have a conflict where the Path Breadcrumbs module overrides the theme's implementation of the breadcrumbs. Such that when the PB module is enabled, the theme's breadcrumb trail doesn't append the title to the breadcrumb trail. This leaves me with three options: 1) hack the theme breadcrumb implementation to re-add the titles, 2) implement views page breadcrumbs in code; 3) implement path bread crumbs for all of my pages. None of these options seems particularly pleasant, but I think approaching the theme breadcrumbs will be the least work. Any ideas? or simple workarounds?
Answer: So, more obvious than I'd like to admit: there is a Path Breadcrumbs settings tab where you can configure the theme settings as well as per theme breadcrumb settings. Enabling the theme breadcrumbs for my custom theme solved the problem!.


Can I use bufferedreader after using scanner?

Question: I am testing Scanner and Bufferedreader. I tried to use BufferedReader after using Scanner.
However, if I close the scanner, an error occurs. Why is there an error?
I tried googling, but I don't understand.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.util.Scanner;

public class ScannerAndBufferedReaderTest2 {

    public static void main(String[] args) throws IOException {

        // Scanner
        Scanner sc = new Scanner(System.in);

        System.out.print("Enter number : ");
        int in1 = sc.nextInt();
        sc.close(); // <-- If I erase this part, there will be no error.

        System.out.println("Your number : " + in1);

        // BufferedReader
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(System.out));

        bw.write("Enter number: ");
        bw.flush();
        int in2 = Integer.parseInt(br.readLine());
        br.close();

        bw.write("Your number : " + String.valueOf(in2));
        bw.flush();
        bw.close();

    }

}

/* Error

Exception in thread "main" java.io.IOException: Stream closed
    at java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:176)
    at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:342)
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:181)
    at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
    at test.Test2.main(Test2.java:27)

*/

Comment: Closing Scanner causes closing source of its data, here `System.in` (representing standard input) which prevents using it father.
Comment: [What does scanner.close() do?](https://stackoverflow.com/q/26245468)
Comment: These question might be a little bit rlated: 1.[Close Scanner without closing System.in](https://stackoverflow.com/questions/14962082/close-scanner-without-closing-system-in) 2. [Close a Scanner linked to System.in](https://stackoverflow.com/questions/14142853/close-a-scanner-linked-to-system-in)
Comment: In short: don't close what you didn't open. *Standard in* is 'wired' by the JVM, so let the JVM close it.
Answer: Scanner sc = new Scanner(System.in);  //" 1st time "

    System.out.print("Enter number : ");
    int in1 = sc.nextInt();
    sc.close(); // <-- If I erase this part, there will be no error.

    System.out.println("Your number : " + in1);

    // BufferedReader
    BufferedReader br = new BufferedReader(new InputStreamReader(System.in));  //"2nd time"
    BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(System.out));

    bw.write("Enter number: ");
    bw.flush();
    int in2 = Integer.parseInt(br.readLine());
    br.close();

    bw.write("Your number : " + String.valueOf(in2));
    bw.flush();
    bw.close();
    sc.close(); // **"close your scanner over here"**

you are taking two time input from user using keyboard that is provided by(System.in) while you are closing Scanner, This means that you are closing whole System.in. if you still wanted to close your scanner write at the end of the program.
Good Luck ;)


Best Practice for Server Roles Seperation

Question: I have a Windows Server 2012 DC. I have the host running just Hyper-V and the RAT feature set. I then have DHCP in its own VM, along wit AD DS/DNS/WINS in their own. SQL 2014 ENT in it's own, WSUS in it's own, WDS/MDT and deployment in it's own VM as well.
Is it good practice to keep services in their own virtualized machines for easier maintenance and updating? I know some Windows Servers can't be together anyway and it seems like if you seperate it keeps the risk of complications out and if anything goes very wrong, a reinstall and reconfigure is less of a hassle.
Does anyone have any real world experience that could help me with how to properly setup and maintain all those windows servers?
Answer: There's no "one answer" here, it depends on your goals.
For things like Active Directory Domain Services, you should only run AD DS, DNS, and (if you really have to - yuck) WINS on a Domain Controller.
For other servers, barring a compatibility issue or other vendor-specific recommendation, you can mix and match based on performance and hardware constraints. For example, an MDT server has the potential to consume a lot of network bandwidth, so I would not want to put File Services or SQL Server on the same box, as there's the potential to negatively impact performance. Also, having single-purpose servers makes upgrading easier, as you only have to remedy one application or role at a time, so you have a more modular environment.
On the other side, the more servers you have, the more resources you are dedicating to running the underlying operating system. You also have more servers to patch, maintain, backup, and otherwise secure.
So for your most critical servers, you should probably keep them modular. For tier 2 and 3 servers, it really depends.
Answer: You seem to be pretty much on the right track.
The host OS should be used for nothing but hyperv.  Having nothing important on it means its easy to migrate everything to a new host, in many cases with 0 downtime.
I usually put DHCP and DNS on the domain controllers.  Unless its a huge network DHCP doesnt need its own server.  Make sure you have at least 2 domain controllers.
Some things, like exchange and sharepoint, should always have a separate VM (or several, depending on scale).  Putting them on the same VM as other services makes upgrading them difficult to impossible.
Others can share, depending on the size of the network.  WSUS can live on your file server in a small environment for example.
The size of the VMs disks makes a difference too.  Keeping things that use a lot of storage on their own VM makes backing up and restoring a lot faster than one massive multi TB machine.
Another consideration is who uses them, and how critical they are.  For services only the IT department uses, and that can tollerate some downtime, I set the servers to automatically do windows updates.  This has two advantages - less servers I have to stay late and manually patch during maintenance windows, and I usually learn about broken updates that kill my servers BEFORE I patch critical systems.  I couldnt do that if more important things lived on the same VMs.


toeplitz Matrix for a given N

Question: For a given N, I need to create the following matrix:

I have understood that :
 from scipy.linalg import toeplitz
 y=toeplitz(range(1,N))

will create a Toeplitz matrix. But it is not the exact matrix given above.
Appreciate your help
Answer: From the docs:

scipy.linalg.toeplitz(c, r=None)
Construct a Toeplitz matrix.
The Toeplitz matrix has constant diagonals, with c as its first column and r as its first row. If r is not given, r == conjugate(c) is assumed.

Thus you need to pass the first column and the first row like this:
toeplitz(c=[1, *np.arange(N,1,-1)], r=np.arange(1,N+1))

toeplitz Matrix for a given N


How to reverse arrows (tails) in Diagrammer, R?

Question: Let's create a very simple diagram:
niv <- c("A","B","C","D","E")
from <- c("A","A","A","E","B","C","D")
to <-   c("B","C","D","B","C","A","E")
arr <- c(rep("normal",6), "inv")
temp <- data.table(from=factor(from, levels=niv),
to=factor(to,levels=niv))
nodes <- create_node_df(n=length(niv), label=niv,
width=0.3)
edges <- create_edge_df(from = temp$from,
to = temp$to, rel = "leading_to", label=temp$from,
arrowhead=arr, penwidth=3, color="blue")
graph <- create_graph(nodes_df=nodes,edges_df=edges)
render_graph(graph)

The diagram shows edges from one node to another.
Usually people paint that edges with an arrow head, but I would like to paint some with the opposite symbol, a tail head.
I've found that by using the parameter arrowhead="inv" I can change the arrow to be a tail instead. (In my example the edge from D to E has a tail).
But there is a problem, the tail is painted on the wrong side.  I would like to paint it at the beginning of the edge (arrow).
How can I change it?
There is a parameter rel = "leading_to"  but I haven't been able to find what other values it can get.
Also there is a parameter "dir" but I don't know how to use it either.
Thanks.
A bonus question would be: How to prevent Diagrammer from writing the labels just above the edges?  I would like they don't overlap.
Comment: Does switching the "from" and "to vectors get you the result you want?
Comment: That was a possible solution I was thinking but I would need to switch only the edges I want reversed. Anyway I was looking for some more elegant solutions, without modifying the original input.
Answer: You can switch the dir to "back":
library(DiagrammeR)
library(data.table)
niv <- c("A","B","C","D","E")
from <- c("A","A","A","E","B","C","D")
to <-   c("B","C","D","B","C","A","E")
temp <- data.table(from=factor(from, levels=niv),
                   to=factor(to,levels=niv))
nodes <- create_node_df(n=length(niv), label=niv,
                        width=0.3)
edges <- create_edge_df(from = temp$from,
                        to = temp$to, dir = "back", label=temp$from,
                        arrowtail="inv", penwidth=3, color="blue")
graph <- create_graph(nodes_df=nodes,edges_df=edges)
render_graph(graph)


How to write an array of strings into a single, formatted, string

Question: I have an array of items:
array(
    'apples',
    'oranges',
    'pineapples'
)

that I would like to format like so: "Some of my favorite fruits are apples, oranges and pineapples"
So, using PHP I would like to be able to transform that array into a reader friendly apples, oranges and pineapples.
At first I thought I might be able to use array_map but i'm not sure how I would go about telling what the last and 2nd to last items are (it's easy to add a , after each item with array_map, however you wouldn't want a comma before the last item)
Thoughts?
Answer: Divide the problem to conquer it: Take out the last element, create the comma separated list and then add the last element.
$array = array(
    'apples',
    'oranges',
    'pineapples'
);

$last = array_pop($array);
echo implode(', ', $array), ' and ', $last; # apples, oranges and pineapples

Naturally this only makes sense when there are at least two values in the array. Functions used: implodeDocs (probably more suitable than array_map in this case) and array_popDocs.

On the other hand, if the values do not contain any , you can first create a comma separated string with the implode function and then use a regular expression to replace the last comma with and.
That done will also work on arrays that have less than 2 elements. I do this in a loop that removes one element from the array in each step so that it shows how it behaves:
$array = array(
    'apples',
    'oranges',
    'pineapples'
);

foreach($array as $v)
{
    echo preg_replace('(,([^,]*)$)', ' and$1', implode(', ', $array)), "\n";
    array_pop($array);
}

Output:
apples, oranges and pineapples
apples and oranges
apples

Answer: Ok, so as others have commented implode is a solution.
I would probably end up using a combination of implode, strripos and substr_replace:
$string = implode(', ', $fruit);
$pos = strripos($string, ', ');
echo substr_replace($string, ' and ', $pos, 2);

which results in apples, oranges and pineapples.
I'd gladly post a more elegant solution if I could think of one, but this is a quick and easy way to get at the change you need.
Comment: I concede that the one @hakre posted using "array_pop" is a cleaner way of doing it!
Answer: You can know which one is the last element by counting them:
$array = array(
    'apples',
    'oranges',
    'pineapples'
);

$count= count($array);
// because the key start from 0 , we must subtract 1
$lastKey= $count-1;

/// now u can do that
echo "
1st :".$array[0];
echo "
2nd :".$array[1];
echo "
3rd :".$array[2];
echo "
last:".$array[$lastKey];

// or same as u like

foreach($array as $key=>$value){
    echo $value;
    if($key == $lastKey){
        echo ".";
    }elseif($key == ($lastKey-1)){
        echo " and ";
    }else{
        echo " , ";
    }
}

Answer: Perhaps your looking for implode, which will essentially turn an array to string


Are Measurements made on the same patient independent?

Question: I am currently working on the classification of pulmonary diseases using spirometry. This is a procedure in which the patient blows air in a tube and we collect air volume, pressure, etc, in order to obtain  the spirometric parameters.
My question is : If I perform the spirometry in the same patient three times, can I consider these three exams, at the same day,  to be three different data points in my training of testing set or it is better to average the results and consider only one data point? If the patient comes in a different day, can I considered this exams to be independent?
I think it is okay to consider the exams to be different data points, but I would like to hear other opinions.
Comment: FWIW, for spirometry the industry standard is to run each patient 3x. If the measures vary by <10%, it is considered a good test. Most advocate averaging the measurements, but a minority advocate using the best under the theory it's a more accurate measurement of what their lungs can do. From a stats point of view, either is a good measurement as long as the same method is used for everybody, & using the average yields a 58% reduction in measurement error relative to best. There isn't an appreciable trend except for patients w/ COPD b/c the huffing & puffing leads them to clear their airways.
Comment: @gung: Wow. This is some very impressive and very unexpected knowledge.
Answer: They are definitely three different data points, but they are also definitely not independent (whether they are same day or different day). What you should do about this depends on the goals of your analysis, but it is likely that a multi-level model is a good choice. Averaging the points is also possible, but it reduces variability and eliminates the ability to look at trends over time.
Answer: I mostly concur with @PeterFlom's answer. In my opinion, you should not average your data (you are basically throwing away 2/3 of your information, why would you want to do that?), but you should definitely account for the fact that measurements on the same patient will tend to be closer together than measurements on different patients. In such a situation, I usually recommend mixed linear models, which are a simple instance of the multi-level models @PeterFlom recommends.
Specifically, you would use a generalized linear mixed model. The link function would be logistic, as in "ordinary" logistic regression. However, the functional form would include multiple observations on each participant, modeled by a random effect, just as in "ordinary" linear mixed models, $y∼F(Xβ+Zγ)$. In R, you can fit this by glmer() in the lme4 package, using the binomial family. For prediction, you could use a single measurement.
Whether or not a mixed model predicts better than a non-mixed model in a particular setting is hard to say, of course. What the mixed model does is account for intra-person variability. If you just average the three original data points, you lose all the variability between measurements, so you will be too optimistic about your ability to predict from a single new observation.
If, on the other hand, you simply throw in all observations without taking the grouping into account, you will again be too optimistic, as all standard errors will shrink. Think of what would happen if you started with a single observation per participant, say 100 data points... and then simply copied each observation 100 times. You would end up with 10,000 "observations" and far smaller standard errors than with the original data, although you didn't enter any new information.
In addition, mixed models allow modeling other grouping factors, like the location, its specific demographics, its staff, diagnostician characteristics etc. So they are a lot more general than averaging.
Comment: @amoeba: I think you may have misunderstood my answer (and possibly the others you downvoted as well). I did not talk about modeling spirometry results as a DV, but as an IV - with the challenge that the measurements are correlated for each participant. My suggestion was that the OP should not average the three measurements and use the average as an IV, but to use all three measurements as an IV and account for the dependence between each participant's multiple measurements using mixed models. Please reconsider your downvote on my and the other answers.
Comment: I will be happy to revert my downvote, but let me see if I understand. Let's say that we want to predict single categorical $y$ (healthy/ill) by a set of spirometry observations $\mathbf{X}$ (air volume, pressure, etc). I guess you are talking about a logistic regression model (do you?), $y\sim F(\boldsymbol{\beta} \mathbf{X})$. Now for each participant we have three sets of spirometry measures, $\mathbf{X}^{(i)}$ with $i$ from 1 to 3. How do you include this into the glm? I am not familiar with mixed models for classification, that's why I (probably) got confused. If so, I apologize.
Comment: You would use a generalized linear mixed model. The link function would be logistic, as in "ordinary" logistic regression. However, the functional form would include multiple observations on each participant, modeled by a random effect, just as in "ordinary" linear mixed models, $$y\sim F(X\beta + Z\gamma)$$. In R, you can fit this by glmer() in the lme4 package, using the binomial family: http://cran.rstudio.com/web/packages/lme4/lme4.pdf And yes, for prediction, you could use a single measurement.
Comment: (-1) The OP wrote that his or her goal is "classification of pulmonary diseases using spirometry". Your answer is about modeling spirometry results as DV, but what OP wanted is using spirometry results as IV to classify diseases...
Comment: (cont.) I forgot to add that at test time we want to predict $y$ for a new patient by having only one instance of $\mathbf{X}$, not three.
Comment: So now you include all three measurements per subject as rows + a random effect of subject: $Z$ codes the subject id, and $\gamma$ is a random effect coefficient -- did I understand this correctly? But when a new subject comes, what do we use as $Z\gamma$? I guess without any information about this subject we simply put there $0$, right? So at test time, we end up calculating $F(X\beta)$? If so, are you saying that this model will classify better than a simple $y \sim F(X\tilde \beta)$ model without a random effect? Is there an intuition why? Thanks a lot.
Comment: Yes, exactly. Whether or not a mixed model predicts better than a non-mixed model in a particular setting is hard to say, of course. What the mixed model does is account for intra-person variability. If you just average the three original data points, you lose all the variability between measurements, so you will be too optimistic about your ability to predict from a single new observation. In addition, mixed models allow modeling other grouping factors, like the location, its specific demographics, its staff, diagnostician characteristics etc. So they are a lot more general than averaging.
Comment: I don't think I would suggest averaging, I would just pool the observations without grouping. Then the issue with being overly optimistic should not arise. It would be very interesting to see examples about when such a model would perform noticeably different from what you suggested! But in any case, I see now that my original criticism was wrong, so I remove the downvote(s). Thanks again.
Comment: I am sorry, but it turns out that votes can be changed only during the first 5 minutes, and after that they get locked unless the answer is updated. I was not aware of that. So I would be happy to remove by downvote (and actually upvote), but you would need to update your answer first. If you want, you could e.g. add what we were discussing in the comments...
Comment: I put most of my comments in the original article and also discussed your suggestion of simply putting all observations into the model.
Answer: The three exams are different data points. Though they are clearly not independent (nor random) observations of all possible exams in your population of interest, at least for any analysis I can imagine.
Others have emphasized that you may do well to include those data points in your analysis (since you already have them), as simple replicates within patient [a nested design] or including "time/visit" as an absolute (e.g. date) or relative (number of visit) variable of interest [some form of repeated-measures design], if interesting. I agree that this is the most interesting (and probable) scenario.
However, it may not be necessary, pay for increased complexity, or improve your conclusions if you are only interested in between-subjects variables. Let's say that you only care for differences between males and females, or you want to explain air volume by patient age. Since you know that you can not properly characterize a patient in one blow (cause measurements result variable even for the same patient at the same moment), then you take several measures and average them. You don't care about that variation, it's just inevitable; you just want to get as close as possible to the "true" (mean) value for that patient (at/in that time). This may be the most reasonable analysis.
[Check this paper for a good read about simplicity vs. complexity in statistical analyses.]
Comment: Thanks for the link to a nice paper, it makes a very good point!
Answer: In accordance with the other answers (no, these observations are certainly not independent, so what do you do about it)....
But do you want to use this information to predict other variables? Many of the suggestions so far seem to be assuming you want to use spirometry as a dependent variable, and thus modelling the error is more straightforward (using a multilevel model). If you instead want to use the spirometry measures as an independent variable, you would be well served by using a confirmatory factor analysis model with the 3 repeat measures modeled as indicators of a single underlying latent variable. The variance of the underlying latent variable is that shared by all three measures, and thus a better reflection of what you are really after (compared to taking the mean, for example).
Comment: I am not sure factor analysis would be useful here: I guess OP wanted to use the classifier to classify the disease given a single spirometry measurement of a new patient, without waiting to collect three measurements first (so you would not be able to apply FA on the real test data).
Answer: the measurements can be independent or not. if you describe the measured value as $y_t=x_t+\varepsilon_t$, where $x_t$ - true value, and $\varepsilon_t$ - measurement error, then independence means that $cov(\varepsilon_t,\varepsilon_{t-i})=0$ for all times. this may or may not be true. if you have two measurements one immediately after another then it's most likely not true. if two measurements were time separated but conducted buy the same technician again this may not be true. etc.
on the other hand it must be possible to setup the measurement in a way that $\varepsilon_t$ would be independent of each other and the $x_t$.
$y_t$'s are most definitely not independent through $x_t$ correlations, but that's not what is meant by independence