Python: Create a new column based on different conditions

Question: I have a dataset with different columns. It looks like that:
ID = [97,97,97,19,19,33,33,33,33,33,11]
DATE = [2018-09,2018-09,2020-02,2021-01,2021-01,2017-04,2017-04,2017-04,2019-09,2019-09,2022-05]
Destination_1 = [BRAZIL,BRAZIL,BRAZIL,ARGENTINA,MOROCCO,INDONESIA,USA,BRAZIL,EGYPT,LEBANON,USA]
Destination_2 = [BRAZIL,URUGUAY,SINGAPORE,VENEZUELA,THAILAND,PERU,ECUADOR,USA,ALGERIA, EGYPT,CANADA]

I would like to create a new column OUTPUT based on the following condition:
for each ID if at the same DATE at least one of the DESTINATION_1 equals DESTINATION_2 then the output sould be TRUE otherwise FALSE.
The desired result for the OUTPUT column should then be:
OUTPUT=[TRUE, TRUE, FALSE , FALSE, FALSE, FALSE, FALSE, TRUE, TRUE, TRUE, FALSE]

meaning that if at the same DATE at least one destination is common to each ID then the condition is satisfied and the output is TRUE.
How could I do so? So far I tried:
   for sublist in df["ID"]:
  if (df["Destination_1"] == df["Destination_2"]).any():
    print("True")
  else :
    print("False")

but this doesn't work for me and I do not get how to add the second condition concerning the DATE without getting an error.
Could someone help me please?
Answer: You can try groupby and use isin to check value existence then assign
out = (df.groupby(['ID', 'DATE'])
       .apply(lambda g: g.assign(OUTPUT=[g['Destination_1'].isin(g['Destination_2']).any()]*len(g))))

print(out)

    ID     DATE Destination_1 Destination_2  OUTPUT
0   97  2018-09        BRAZIL        BRAZIL    True
1   97  2018-09        BRAZIL       URUGUAY    True
2   97  2020-02        BRAZIL     SINGAPORE   False
3   19  2021-01     ARGENTINA     VENEZUELA   False
4   19  2021-01       MOROCCO      THAILAND   False
5   33  2017-04     INDONESIA          PERU    True
6   33  2017-04           USA       ECUADOR    True
7   33  2017-04        BRAZIL           USA    True
8   33  2019-09         EGYPT       ALGERIA    True
9   33  2019-09       LEBANON         EGYPT    True
10  11  2022-05           USA        CANADA   False

Comment: thanks for your help! It works perfectly, can I ask you where to find good documentation concerning the lambda function please?
Comment: @Cai I would refer to [How to Use Python Lambda Functions – Real Python](https://realpython.com/python-lambda/), but I don't why it needs to be registered to read now. Anyway, `lambda` function isn't really a much complicated concept any tutorial you find on the web I think is enough.


DbExpressionBinding requires an input expression with a collection ResultType

Question: I want to group the table by a column and get the counts, then create dictionary using the result. The last statement returns the error of

DbExpressionBinding requires an input expression with a collection ResultType.

What the error means?
var a = context.Table;
var b = a.GroupBy(x => x.RecordType, (k, cnt) => new { RecType = k, cnt = k.Count() });
var c = b.Select(x =>
    new KeyValuePair<string, Tuple<Type, int>>(
        x.RecType, Tuple.Create(ObjTypes[x.RecType], x.cnt)))
    .ToDictionary(x => x.Key, x => x.Value);

Answer: Your GroupBy call is slightly wrong - it's ignoring the cnt parameter. It should be written like this:
var b = a.GroupBy(x => x.RecordType, (k, cnt) => new { RecType = k, cnt = cnt.Count() });

That exception message is one of the more unhelpful I've seen. The problem is that when you call Count on RecordType (aliased as k) in the C# expression it compiles fine, because string implements IEnumerable<char>. Unfortunately, when EF tries to convert that expression to SQL it needs to be able to use the SQL count operator, which can't work on a string - it needs to operate over a sequence of rows. This results in the somewhat cryptic exception you received.
Incidentally, the query assigned to c won't work either - Tuple.Create can't be translated into SQL, nor can ObjTypes[x.RecType]. You'll need to call ToList on variable b before you can do all that additional shaping of your results.
Comment: @dc7a9163d9 The construction of the `KeyValuePair` isn't in a supported manor.  Just remove the `Select` entirely and do the projection in the `ToDictionary` call.
Comment: And this is why we use meaningful variable names for parameters, even for lambdas.
Comment: @Servy, good suggestion, that's more efficient than calling `ToList`.
Comment: You should basically never use `ToList` if you're just going to call another LINQ method on that list.  It's eagerly evaluating the query, creating a list, and populating it all entirely needlessly.  The only reason to use `ToList` would be when a query is entirely finished and eager materialization is important.


Google App Script Search UI Tool with Previous Function ERROR

Question: Previous Function is working but when i'm going to click previous for the last sesrch of it. im getting an error 'CANNOT CALL METHOD 'toString of undefined.
Please help with my code. Im having a trial and error with the highlighted code below in the previous function. please help thank you.
function click(eventInfo){
 var ss = SpreadsheetApp.openById('1qpx2qZvBriBZ3jkOfHjMpsRbL9w04JZEJMeor11OJC81').getSheetByName("Teaching");
 var app = UiApp.getActiveApplication();
 var row=ss.getActiveRange().getRowIndex();
 var lastrow = ss.getLastRow();
        app.getElementById("txbid").setEnabled(true).setText("Search");

  var item = eventInfo.parameter.itemname;
   var next = app.getElementById("nextid").setEnabled(false);
   var pre = app.getElementById("previd").setEnabled(false);
   var lbl = app.getElementById("lblid").setText("No match");
   var f1 = eventInfo.parameter.thirdname;
  var f2 = eventInfo.parameter.fourthname;
  var f3 = eventInfo.parameter.fifthname;
  var f4 = eventInfo.parameter.emailname;
  var f1box = app.getElementById("thirdid");
  var f2box = app.getElementById("fourthid");
  var f3box = app.getElementById("fifthid");
  var f4box = app.getElementById("emailid");
    var hidden = app.getElementById("hidden");
   var data = ss.getRange(2,1,lastrow,6).getValues();// get the 3 columns of data
       for(nn=0;nn<data.length;++nn){ ;// iterate trough
         if(data[nn].toString().toLowerCase().match(item.toString())==item.toString()&&item!='') {;// if a match is found in one of the 3 fields, break the loop and show results
  f1box.setText(data[nn][2]);
  f2box.setText(data[nn][3]);
  f3box.setText(data[nn][4]);
  f4box.setText(data[nn][5]);
      app.getElementById("nextid").setEnabled(true).setText("Next");
      app.getElementById("txbid").setEnabled(true).setText("Search");
           ss.getRange(nn+2,2).activate();
            lbl.setText("found from"+data[nn][0]+"-"+data[nn][1]+", click Next").setEnabled(true);
          hidden.setValue(nn.toString())
          break;}}
  return app;
}
function next(eventInfo){
   var ss = SpreadsheetApp.openById('1qpx2qZvBriBZ3jkOfHjMpsRbL9w04JZEJMeor11OJC81').getSheetByName("Teaching");
 var app = UiApp.getActiveApplication();
 var row=ss.getActiveRange().getRowIndex();
 var lastrow = ss.getLastRow();
        app.getElementById("txbid").setEnabled(true).setText("Search");

  var item = eventInfo.parameter.itemname;
   var next = app.getElementById("nextid").setText("No match").setEnabled(false);
   var pre = app.getElementById("previd").setEnabled(true);
   var lbl = app.getElementById("lblid").setText("0 results to display");
   var f1 = eventInfo.parameter.thirdname;
  var f2 = eventInfo.parameter.fourthname;
  var f3 = eventInfo.parameter.fifthname;
  var f4 = eventInfo.parameter.emailname;
  var f1box = app.getElementById("thirdid").setText("");
  var f2box = app.getElementById("fourthid").setText("");
  var f3box = app.getElementById("fifthid").setText("");
  var f4box = app.getElementById("emailid").setText("");
    var start = Number(eventInfo.parameter.hidden)+1;
    var hidden = app.getElementById("hidden");
   var data = ss.getRange(2,1,lastrow,6).getValues();// get the 3 columns of data
       for(nn=start;nn<data.length;++nn){ ;// iterate trough
         if(data[nn].toString().toLowerCase().match(item.toString())==item.toString()&&item!='') {;// if a match is found in one of the 3 fields, break the loop and show results
  f1box.setText(data[nn][2]);
  f2box.setText(data[nn][3]);
  f3box.setText(data[nn][4]);
  f4box.setText(data[nn][5]);
         app.getElementById("previd").setEnabled(true).setText("Previous");
      app.getElementById("nextid").setEnabled(true).setText("Next");
      app.getElementById("txbid").setEnabled(true).setText("Search");
           ss.getRange(nn+2,2).activate();
            lbl.setText("found from"+data[nn][0]+"-"+data[nn][1]+", click Next").setEnabled(true);
          hidden.setValue(nn.toString())
          break;}}
  return app;
}
function previous(eventInfo){
   var ss = SpreadsheetApp.openById('1qpx2qZvBriBZ3jkOfHjMpsRbL9w04JZEJMeor11OJC81').getSheetByName("Teaching");
 var app = UiApp.getActiveApplication();
 var row=ss.getActiveRange().getRowIndex();
 var lastrow = ss.getLastRow();
        app.getElementById("txbid").setEnabled(true).setText("Search");

  var item = eventInfo.parameter.itemname;
   var next = app.getElementById("nextid").setEnabled(true);
   var pre = app.getElementById("previd").setText("No match").setEnabled(false);
   var lbl = app.getElementById("lblid").setText("No match");
   var f1 = eventInfo.parameter.thirdname;
  var f2 = eventInfo.parameter.fourthname;
  var f3 = eventInfo.parameter.fifthname;
  var f4 = eventInfo.parameter.emailname;
  var f1box = app.getElementById("thirdid").setText("");
  var f2box = app.getElementById("fourthid").setText("");
  var f3box = app.getElementById("fifthid").setText("");
  var f4box = app.getElementById("emailid").setText("");

var start = Number(eventInfo.parameter.hidden)-1;
        var hidden = app.getElementById("hidden");
         var data = ss.getRange(2,1,lastrow,6).getValues();// get the 3 columns of data

       for(nn=start;nn<data.length;--nn){ ;// iterate trough***

if(data[nn].toString().toLowerCase().match(item.toString())==item.toString()&&item!='')

{;// if a match is found in one of the 3 fields, break the loop andshoresults
        f1box.setText(data[nn][2]);
        f2box.setText(data[nn][3]);
        f3box.setText(data[nn][4]);
        f4box.setText(data[nn][5]);
            app.getElementById("previd").setEnabled(true).setText("Previous");
      app.getElementById("nextid").setEnabled(true).setText("Next");

      app.getElementById("txbid").setEnabled(true).setText("Search");

             ss.getRange(nn+2,2).activate();
            lbl.setText("found from"+data[nn][0]+"-"+data[nn][1]+", click Next").setEnabled(true);
          hidden.setValue(nn.toString())
          break;}}

return app;
    }
Answer: You are iterating the data array with index nn but the last item of this array is empty because of the way the array is built.
When you declare it using
var data = ss.getRange(2,1,lastrow,6).getValues();// get the 3 columns of data

you start from row 2 and get lastRow value for length which means data contains the empty row below the last one (lastRow returns the total height of filled data range).
try using var data = ss.getRange(2,1,lastrow-1,6) and it should solve the issue
note : this is a (modified) code I originally wrote a few years ago, not sure I made that mistake in the original script ;-)
Comment: im still getting the error cannot call method toSting of undefined. Should i also edit other code? I should get "no match" when it will be the last result for the Previous button.. please help thanks
Comment: can i get a copy of your previous function?or can you paste it here so i can see whats my error.  its a big help.. :(  please check my highlighted code from the above... thanks Sir Serge. var start = Number(eventInfo.parameter.hidden)-1;      or   for(nn=start;nn<data.length;--nn)   ??
Comment: I can't find my original script in the right version but if you want you can share a copy of your spreadsheet (in read only and without private data) and show the link here.
Comment: this is my sample spreadsheet = https://docs.google.com/spreadsheets/d/1bAcfcbn8Sqm2RhqgxPzOZO-7VE2e_RbFR7LK5MY0RR0/edit?usp=sharing                                                my Script link = https://script.google.com/macros/s/AKfycbxOFbRts8jo2NVn6U8LSW16RBtsfeE2pdhN-M8WtqnpjiomSXQ6/exec    (it's also in the spreadsheet.) thanks for the help and time. Can i get your email? so i can easily contact you with this prob?thankyou @Serge insas


Adding annotations to MapKit in UIRepresentable

Question: I'm working in SwiftUI and using MKMapView to show a map, annotations, and overlays on a view.
I was using the new Map() but the lack of overlay support has pulled me back to UIKit Representable.
When I was using Map it was easy to add annotations, but when using UIKitRepresentable I'm a bit confused on where to put the data, and how to make annotations from an array pulled from a network call.
Everything I've read has been either in Obj-C or adding a single annotation point. I'm trying to add (at present ~800) which is why I wanted to take advantage of the MKMapView in its reusability and clustering.
This is what I have at the moment:
struct UIMapView: UIViewRepresentable {

  @EnvironmentObject var dataModel: DataModel
  @EnvironmentObject var mapViewModel: MapViewModel

  func makeCoordinator() -> Coordinator { Coordinator() }

  func makeUIView(context: Context) -> MKMapView {
    let view = mapViewModel.mapView
    drawOverlayRing(view: view)
    view.delegate = context.coordinator
    return view
  }

  func updateUIView(_ uiView: MKMapView, context: Context) {}

  class Coordinator: NSObject, MKMapViewDelegate {
    func mapView(_ map: MKMapView, viewFor annotation: MKAnnotation) -> MKAnnotationView? {
      if annotation is MKUserLocation { return nil }

      let identifier = "pinAnnotation"
      var annotationView = map.dequeueReusableAnnotationView(
        withIdentifier: identifier
      ) as? MKPinAnnotationView

      if annotationView == nil {
        annotationView = MKPinAnnotationView(
          annotation: annotation,
          reuseIdentifier: identifier
        )
        annotationView?.canShowCallout = true
      } else {
        annotationView?.annotation = annotation
      }
      return annotationView
    }
  }

  func getAnnotations(view: MKMapView) {
    for location in dataModel.locations {
      let annotation = MKPointAnnotation()
      annotation.title = location.title
      annotation.coordinate = CLLocationCoordinate2D(
        latitude: location.latitude,
        longitude: location.longitude
      )
      view.addAnnotation(annotation)
    }
  }
}

Answer: I sometimes put my annotations in updateUIView, something like this:
func updateUIView() {
    // remove the old ones
    uiView.removeAnnotations(uiView.annotations)
    uiView.addAnnotations(toMapAnnotations(locations: dataModel.locations))
}

func toMapAnnotations(locations: [CLLocationCoordinate2D]) -> [MapAnnotation] {
    return locations.map { MapAnnotation(location: $0) }
}

final class MapAnnotation: NSObject, MKAnnotation {...}

Comment: Hey, with this, where do you define `uiView` as?
Comment: In my case, I typically declare, `let mapView = MKMapView()` in the `struct UIMapView: ...`.
 I use that in `func makeUIView(context: Context)` to customise `mapView`, and use `func updateUIView(_ uiView: MKMapView, context: Context) {...}`. In `markb` case, it seems a `MapViewModel` is used.
Answer: The code will be provided later, since updateUIView is called multiple times.
I think it is best to avoid calling addAnnotations.
Comment: Please add further details to expand on your answer, such as working code or documentation citations.


Compact set in the order topology

Question: Let $(X, \leq)$ be a linearly ordered set and let $\mathcal{T}$ denote the order topology on $X$. Prove that $(X, \mathcal{T})$ is compact if and only if every nonempty set of $X$ has a greatest lower bound and a least upper bound.
I don't know how to apply the definition of compactness in this topology. Can anyone give an idea?
Answer: For one direction, suppose that $A\subseteq X$ is a non-empty set with no least upper bound. Let $U$ be the set of upper bounds for $A$; show that $U$ is open. ($U$ may of course be empty.) Then show that
$$\{U\}\cup\{(\leftarrow,a):a\in A\}$$
is an open cover of $X$ with no finite subcover. You can use much the same idea if you have a non-empty set with no greatest lower bound.
For the other direction suppose that every non-empty subset of $X$ has a greatest lower bound and a least upper bound. In particular, $X$ has a least element $a$ and a greatest element $b$. Now imitate the proof that $[0,1]$ is compact to show that $X$ is compact.

Compact set in the order topology


How to delay the display of some HTML until after javascript has loaded

Question: When a page loads on my site, the HTML appears before the javascript, which leads to a flicker when the javascript loads. The answer to this stackoverflow post gave a great solution. But I would like to load at least some of the HTML before the Javascript so that the user is not faced with a blank page during a slow connection. For example, I would like to load the header immediately, but wait to load the HTML for the javascript enhanced accordion until after the javascript loads. Any suggestions?
Here's the code that I borrowed from the answer linked above:
CSS:
#hideAll
 {
   position: fixed;
   left: 0px;
   right: 0px;
   top: 0px;
   bottom: 0px;
   background-color: white;
   z-index: 99; /* Higher than anything else in the document */

 }

HTML:

Javascript
 window.onload = function()
     { document.getElementById("hideAll").style.display = "none"; }

Comment: so what's wrong with using onload? it will run your script when the page is loaded
Comment: @Ibu: `onload` doesn't fire until *everything* --including images--have loaded.  This could be a very long delay on a slow connection.  The appropriate event to use for this kind of document manipulation is [DOM ready](http://www.javascriptkit.com/dhtmltutors/domready.shtml).
Answer: I'd suggest that you define the base/JavaScript-enabled styles of elements you want to display with CSS in the regular style block:

And then use the noscript tags (in the head) to amend this in the absence of JavaScript:

This ensures that the content is hidden on document load, preventing the flash, but visible to those users that have JavaScript disabled.
The above has been amended to prevent the 'flash of no content' (as described by @Josh3736 in his answer), and now uses opacity to hide the content:

Live demo.
I'm not, unfortunately, entirely sure that I understand your question. Which leaves me proposing a solution for the question I think you asked (all I can offer, in excuse, is that it's early in the UK. And I'm not awake by choice...sigh); if there is anything further that I'm missing (or I'm answering the wrong question entirely) please leave a comment, and I'll try to be more useful.
Comment: @David Thomas Thank you. It sounds like you understood my question. But, in short, I'd like to prevent the javascript-affected HTML from loading until after the javascript has loaded, while allowing the HTML not affected by javascript to load asap (e.g., the header, etc.). I'm just trying to see if I can implement your solution before asking for more help...
Comment: @Ken: oh...for some reason I assumed that by 'loading' you meant 'displaying.' HTML doesn't, natively, allow for selective content loading; it's either *there*, or it *isn't*.
Comment: @David Thomas I'm essentially just trying to avoid the flicker that occurs when the HTML displays before the Javascript loads. I apologize for using the wrong terms. It's late here, and I'm a novice - bad combination, I suppose.
Comment: @Ken, it's early here and I'm tired. So...a particularly unfortunate combination all ways around, really... =) But no worries. I've amended the demonstration to use `opacity` rather than `display` to hide the content to show. Which prevents the "flash of no content" [described by @josh3736](http://stackoverflow.com/questions/6119171/how-to-delay-the-display-of-some-html-until-after-javascript-has-loaded/6119569#6119569).
Comment: @David: It is more appropriate to use `visibility:hidden;`, which does not collapse the content block like `display:none;` and does not bump in to IE's lack of support for `opacity`.  However, (regardless of whether you use `opacity` or `visibility`) this approach *still* leads to a flash of content.  For example, if each accordion pane is 200px, with 5 panes, that's 1000px of empty space taken up by the accordion.  When the script runs and hides 4 of those 5 panes, the accordion `` will now be only 200px, making the rest of the page jump up 800px, which is a nasty flash.
Comment: @Josh, I agree with you as regards the `visibility:hidden;` option, but I couldn't think of a way to animate from `hidden` to `visible`. As to the reduction in size, from 1000px to 200px, that can be addessed by using the appropriate css selectors (in most cases), but without seeing the actual mark-up (or a live demo) it's difficult to advise specifically on that aspect.
Comment: I really appreciate your help with this. Am I understanding correctly that I should place the initial style for the element to show with javascript in the head, and then place the opacity:1 style within the body?
Comment: No, the `style` blocks have to be in the `head`, even if it's wrapped in a `noscript` tag. In the linked demo I posted look at the source.
Comment: @David: `$('#accordion').css({ opacity: 0, visibility: 'visible' }).show('fast');`  As far as the selectors, you're right, assuming you can know at 'compile time' which pane will be shown when the page is loaded.  If, for example, a query string/hash parameter is used to select the opening pane (which is something [you should do](http://benalman.com/code/projects/jquery-bbq/examples/fragment-jquery-ui-tabs/)), pure CSS won't cut it.
Answer: The hack in the linked question is—in my opinion—very poor advice.  In this case, it is a better idea to include some script directly following your accordion elements.

...

However, inline script intermingled with your HTML markup is a Bad Idea and should be avoided as much as possible.  For that reason, it is ideal to include inline only a function call to a function declared in your external script file.  (When you reference an external script (

...

...

...

Of course, the correct way to do this is to just attach to the DOM ready event from script.js and not use any inline script at all.  This does, however, open up the possibility of a content flash on extremely slow connections and/or very large documents where downloading all of the HTML itself takes several seconds.  It is, however, a much cleaner approach – your script is guaranteed to be loaded before anything is rendered; the only question is how long it takes for DOM ready.  Using jQuery, in script.js:
$(document).ready(function() {
    // Do whatever with your accordion here -- this is guaranteed to execute
    // after the DOM is completely loaded, so the fact that this script is
    // referenced from your document's  does not matter.
});

Clever use of


How does a hadoop map operation manage with data redundancy on the HDFS cluster?

Question: Since hadoop runs over HDFS, and data is duplicated across the HDFS cluster for redundancy, does a hadoop map operation actually waste a lot of processor cycles by running mappers over  same data points, on different nodes in the cluster? (as the nodes, by design, have some data overlap between them, as per the replication level).
Or does it first, depending on some job management strategy of some sort, only address part of the nodes, to avoid that kind of duplicate calculation, in some very clever way?
Answer: Each mapper gets an individual InputSplit to process. So if you have 100 InputSplits 100 mappers will be spawned by framework. Then every mapper is going to check if it has all data it needs - if not it will download all required  data and begin the computation. One InputSplit will never be assigned twice.


page--front.tpl.php being ignore in favour of page--node--page.tpl.php

Question: I've got an issue which I'm not sure how to address. I've got a specific page that I've set as front page. I've created a custom template for it and named it page--front.tpl.php and it was as expected until I've added page--node--page.tpl.php which seems to be overriding page--front.tpl.php.
Is this the correct and expected behaviour? How can I force it the page--front.tpl.php to be used?
I'm using Drupal 7 with and a child of the Bootstrap's theme.
Comment: Don't have time to test for sure, but try implementing `hook_preprocess_page()` and re-ordering the `$vars['theme_hook_suggestions']` array into something that suits you better
Answer: If you need to have a custom theme for nodes, try to do this using THEME_preprocess_node as follow
function THEME_preprocess_node(&$variables){
     $node = $variables['node'];
     if($node->type == "SOME_CONTENT_TYPE")
          $variables['theme_hook_suggestions'][]='page__node__SOMETHING';
}

Then create page--node--SOMETHING.tpl.php in your theme folder. To get the values of the fields you can use the entity module and do something like the following code (I supposed field_something is machine name of one of your fields):
function THEME_preprocess_node(&$variables){
     $node = $variables['node'];
     if($node->type == "ANY_CONTENT_TYPE"){
        $variables['theme_hook_suggestions'][]='page__node__SOMETHING';
        $wrapper = entity_metadata_wrapper('node',$variables['node']);
        $variables['title'] = $wrapper->title->value();
        $variables['something'] = $wrapper->field_something->value();
     }
}

By this code $title and $something variables are available in  page--node--SOMETHING.tpl.php file.
Comment: That didn't quite work. It put `page--front.tpl.php` inside my `page--node--page.tpl.php`
Comment: Did you clear the cache? Did you remove page--node--page.tpl.php? Put the code of front page in the question
Comment: Yep, I did, using it `drush`
Comment: Like I said put your `page--front.tpl.php` and `template.php` code in here


What Really is the  Class in Flex 4?

Question: I am learning the networking portion of Flex 4.5 right now, and the course I am reviewing recommends that I use the <s:CallResponder> class whenever I make a call to a server, either with the <s:HTTPService> component or with my own custom service call to a database.
Both in the course itself and in the Adobe documentation, I cannot find a really good description as to why I should take the approach. Could someone please describe why this is a good idea, and perhaps provide a case where it is highly recommended?
See this example below for a simple case where I am using it. I declare an instance of the class inside of the <fx:Declarations> tagset, and I use it inside of the fetchData() method:
<?xml version="1.0" encoding="utf-8"?>
<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009"
               xmlns:s="library://ns.adobe.com/flex/spark"
               xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600"
               skinClass="skins.CustomAppSkin">
    <fx:Script>
        <![CDATA[
            import mx.collections.ArrayCollection;
            import mx.controls.Alert;
            import mx.rpc.events.FaultEvent;
            import mx.rpc.events.ResultEvent;

            [Bindable]
            private var booksCollection:ArrayCollection;

            private function formatPrice(data:Object, columns:GridColumn):String {
                return priceFormatter.format(data.price);
            }

            protected function fetchData(event:MouseEvent):void {
                booksResponder.token = books.send();
            }

            protected function processXML(event:ResultEvent):void {
                this.booksCollection = event.result.catalog.book;
            }

            protected function loadHandler(event:FaultEvent):void {
                Alert.show(event.fault.faultString, event.fault.faultCode);
            }
        ]]>
    </fx:Script>

    <fx:Declarations>
        <s:HTTPService id="books" url="data/books.xml"/>
        <s:CurrencyFormatter id="priceFormatter" currencySymbol="$" fractionalDigits="2" trailingZeros="true" useCurrencySymbol="true"/>
        <s:CallResponder id="booksResponder" result="processXML(event)" fault="loadHandler(event)"/>
    </fx:Declarations>

    <s:Panel id="panel" title="Products" horizontalCenter="0">
        <s:DataGrid dataProvider="{booksCollection}" height="400">
            <s:columns>
                <s:ArrayList>
                    <s:GridColumn headerText="Title" width="250" dataField="title"/>
                    <s:GridColumn headerText="Author" dataField="author"/>
                    <s:GridColumn headerText="Genre" width="100" dataField="genre"/>
                    <s:GridColumn headerText="Publish Date" width="100" dataField="publish_date"/>
                    <s:GridColumn headerText="Description" width="400" dataField="description"/>
                    <s:GridColumn headerText="Price (USD)" width="100" dataField="price" labelFunction="formatPrice"/>
                </s:ArrayList>
            </s:columns>
        </s:DataGrid>

        <s:controlBarContent>
            <s:Button id="getData" label="Get Data" click="fetchData(event)"/>
        </s:controlBarContent>
    </s:Panel>
</s:Application>

Answer: A responder is an object that has a variable that contains a reference to a function to call on successful result and can optionally have a reference to a function to call if a fault occurs.  If you didn't have a responder of some type set up, it couldn't call your result and fault handlers.
Comment: Personally, I think that the auto-generation encourages you to put service logic in the View, which is REALLY bad architecture, but maybe the next version of Flash Builder will do a better job, now that people who have a firmer understanding of clean architecture are more involved in Flex as an open source project.
Answer: Maybe some "historical" background would help. The name "Responder" refers to the way NetConnection class, which is the Flash player built-in API handles the communication. When you make a call to NetConnection (which is commonly used for either plain HTTP connection, where messages are encoded in AMF, or for video download / streaming, which is usually an RTMP-variant connection). Responder is used with NetConnection to communicate using AMF encoding over HTTP.
There is a built-in Responder class, which is used with NetConnection to store references to two callbacks which will be called in case of successful communication and in the case of communication failure.
The above scheme is different from other networking API that exist in Flash player. It's hard to think about the reason, but it's been like that since AS2. In AS2, however, there was LoadVars object, which also used callbacks for fault and success messages. XML class behaved in a similar way - you would also create two callback functions and assign them to default XML handlers. It's not impossible that responder is a rudiment of the pre-AS3 design.
When Adobe (for the reasons known only to them alone) decided to pile up a bunch of indirection layers on top of NetConnection (known in Flex as RemoteObject) they retained the old design choices. They also made more recent API behave in a more archaic way. Thus, for example, HTTPService, which is basically the URLLoader in disguise, behaves the same way as RemoteObject. This probably helped somewhat in MXML templates, but as soon as you realize that MXML language is extremely limiting, the relative merits of features tailored specifically for MXML layout will rather become a plaque made up of multiple questionable design decisions.
Amy: afaik DCD feature (the code generation for services used in Flash Builder) has not been released to Apache - it's a part of Flash Builder. So, it's unlikely to change, unless Adobe will change that themselves.

What Really is the  Class in Flex 4?


Can not receive message with Masstransit in .NetCore

Question: I am working with Masstransit, RabbitMq and .NET core. I have problem when I want to publish message and receive. My UserEventHandler and UserCommandHandler class never calls. Can anybody help me? Bus is started. I also have a situation when I remove UserEventHandler class from registration my breakpoint stops on UserCommandHandler class.
Here is my UserEventHandler class:
public class UserEventHandler : IConsumer<IUserEvent>
{
    private ConcertDbContext _context;
    public UserEventHandler(ConcertDbContext context)
    {
        _context = context;
    }
    public Task Consume(ConsumeContext<IUserEvent> context)
    {
        var concerts = _context.Concerts.Where(c => c.Name == context.Message.Name);
        foreach (var concert in concerts)
        {
            concert.SoldTickets = context.Message.TicketBuyed;
            _context.Concerts.Update(concert);
            _context.SaveChanges();
        }
        throw new NotImplementedException();
    }
}

Here is my UserCommandHandler class:
public class UserCommandHandler : IConsumer<IUserCommand>
{
    public async Task Consume(ConsumeContext<IUserCommand> context)
    {
        await context.Publish<IUserEvent>(new UserEvent(context.Message.Name, context.Message.TicketBuyed));
    }
}

I registered UserCommandHandler here in Startup.cs class:
        services.AddMassTransit(x =>
        {
            x.AddConsumer<ConcertEventHandler>();
            x.AddConsumer<UserCommandHandler>();
            x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
            {
                var host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri),
                    hst =>
                    {
                        hst.Username(RabbitMqConstants.Username);
                        hst.Password(RabbitMqConstants.Password);
                    });

                cfg.ReceiveEndpoint(host, RabbitMqConstants.CreateConcertService, e =>
                {
                    e.PrefetchCount = 16;

                    e.ConfigureConsumer<ConcertEventHandler>(provider);
                });

                cfg.ReceiveEndpoint(host, RabbitMqConstants.BuyCncertTicketService, e =>
                {
                    e.PrefetchCount = 16;

                    e.ConfigureConsumer<UserCommandHandler>(provider);
                });

                // or, configure the endpoints by convention
                cfg.ConfigureEndpoints(provider);
            }));

Here is I registered UserEventHandler class
                services.AddMassTransit(x =>
        {
            x.AddConsumer<ConcertCommandHandler>();
            x.AddConsumer<UserEventHandler >();
            x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
            {
                var host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri),
                    hst =>
                    {
                        hst.Username(RabbitMqConstants.Username);
                        hst.Password(RabbitMqConstants.Password);
                    });

                cfg.ReceiveEndpoint(host, RabbitMqConstants.CreateConcertService, e =>
                {
                    e.PrefetchCount = 16;

                    e.ConfigureConsumer<ConcertCommandHandler>(provider);
                });
                cfg.ReceiveEndpoint(host, RabbitMqConstants.BuyCncertTicketService, e =>
                {
                    e.PrefetchCount = 16;

                    e.ConfigureConsumer<UserEventHandler>(provider);
                });

                // or, configure the endpoints by convention
                cfg.ConfigureEndpoints(provider);
            }));

Answer: Each service should be using separate queues, currently you are configuring each service with separate consumers on the same receive endpoint queue names. That will cause the messages to be moved to the _skipped queue since there is not a consumer on that service instance for the published message.
Also, remove the cfg.ConfigureEndpoints at the end of each service configuration. You are manually configuring your receive endpoints, so it will create duplicate queues and endpoints for your consumers.
Comment: Well, don't use `RabbitMqConstants.CreateConcertService` on two separate endpoints with different consumers.
Comment: How to do that? How to separate queues?
Comment: Thank you! It helped me


C# find text in a list that is in a tsv file

Question: I am trying to solve a task in a project on codeacademy, the task is to find a word in the tsv file which is made to a list with the following code:
List<Language> languages = File.ReadAllLines("./languages.tsv")
        .Skip(1)
        .Select(line => Language.FromTsv(line))
        .ToList();

I have tried the following and got error CS1929:
var ChiefDeveloperMS = languages.Where(chief => $"{chief.Contains("Microsoft")}");

foreach (var d in ChiefDeveloperMS)
        {
          Console.WriteLine($"{d.Prettify()}");
        }

The word i am looking for is Microsoft.
Prettify is a method in the language class with the following code:
public string Prettify()
    {
      return $"{Year}, {Name}, {ChiefDeveloper}, {Predecessors}";
    }

sample from the tsv file:
Year    Name    Chief developer, company    Predecessor(s)
1991    Visual Basic    Alan Cooper (sold to Microsoft) QuickBASIC
2001    GDScript(GDS)   OKAM Studio Godot
2001    Shakespeare Programming Language    Jon Åslund, Karl Hasselström    none
2002    Io  Steve Dekorte   Self,NewtonScript,Lua
2002    Gosu    Guidewire Software  GScript
2005    F#  Don Syme (Microsoft Research)   OCaml,C#,Haskell

Output that i got in the console:
1996, VBScript, Microsoft, Visual Basic
2001, Visual Basic .NET, Microsoft, Visual Basic
2006, Windows PowerShell, Microsoft, C#,ksh,Perl,CL,DCL,SQL

expected output:
1991    Visual Basic    Alan Cooper (sold to Microsoft) QuickBASIC
1996    VBScript    Microsoft   Visual Basic
2000    C#  Anders Hejlsberg (Microsoft, ECMA)  C,C++,Java,Delphi,Modula-2
2001    Visual Basic .NET   Microsoft   Visual Basic
2005    F#  Don Syme (Microsoft Research)   OCaml,C#,Haskell
2006    Windows PowerShell  Microsoft   C#,ksh,Perl,CL,DCL,SQL

I have tried many variations, searched all over and asked on the codeacademy facebook group and still cant find the answer, the correct namespaces are on top in progam.cs.
Yesterday i got help on stackoverflow with a workaround, it worked in a way but the problem is that it does not print the words that is in parentheses in the tsv file on the console.
Here is the workaround code:
var ChiefDeveloperMS = languages.Where(chief => chief.ChiefDeveloper == "Microsoft");

Constructor for the language class:
public Language(int year, string name, string chiefDeveloper, string predecessors)
    {
      Year = year;
      Name = name;
      ChiefDeveloper = chiefDeveloper;
      Predecessors = predecessors;
    }

What i dont understand is why the .Contain() method is not working when i have been using it successfully in a similar manner many times.
Should i just give this up and come back to this later? Is this to complicated for a beginner?
I am new here, if more information is required i will be happy to edit my post.
Comment: Not enough info to answer; give sample content of TSV, implementation of `Language.FromTsv` and be more specific about what word you're looking for, in what field, and what output result you want
Comment: `languages.Where(chief => chief.ChiefDeveloper == "Microsoft");` - don't name your linq lambdas like this. languages is a list of Language, the liqn operates on each element (which is a language) - write like `languages.Where(lang => lang.ChiefDeveloper == "Microsoft");` because it makes it more clear that the list item is a language, not a chief
Comment: I will try to provide more information, the tsv file is quite big, contains 120 lines @CaiusJard thanks for the tips in regards to how to name my linq
Comment: "sample content" like, 5 lines, one of which mentions "Microsoft"
Comment: @CaiusJard sample added
Comment: *does not print the words that is in parentheses in the tsv file* - only one of the words in the TSV file is wrapped in parentheses. Is that what you want? "GDS" ?
Comment: There word Microsoft is in 6 places in the file with chief developer 3 of them in parentheses and 3 that gets printed with the correct information. I need all 6 to be printed. Also codeacademy told me to use .Contains() method @CaiusJard yes you are correct that is what i want
Comment: Post a sample of the TSV that is RELEVANT to your question. You've posted 5 random lines, none of which contains "microsoft" and only one of which contains parentheses
Comment: @CaiusJard sample updated
Comment: Now give the expected output
Comment: Output and expected ouput updated @CaiusJard
Comment: Why does some of your expected output lines have a comma after the year and other lines do not?
Comment: @CaiusJard I made a and forgot to remove the commas when i copied it from the console, the ones without commas are copied from the tsv file istelf i will edit it to make it correct
Answer: So, the reason why this:
var ChiefDeveloperMS = languages.Where(chief => $"{chief.Contains("Microsoft")}");

doesn't work is because it's all messed up
Where is a LINQ command that expects you to pass a function to it that returns a boolean (or an expression that evaluates to be a Boolean and
chief => $"{chief.Contains("Microsoft")}"

is not an expression that results in a Boolean. I'll pick it apart so you can know what went wrong

chief => gives a name to the list element that will be passed to the expression/function. You're calling Where on a list of Language objects so calling this lang would be better than chief. chief is (/lang would be) a Language object. If you want to get the ChiefDeveloper property of it you'd have to say chief.ChiefDeveloper -> can you see why I recommend to look at the collection of objects and pick a name that makes a good singular reference for an item in the collection? lang.ChiefDeveloper makes more sense than chief.ChiefDeveloper and helps you keep straight in your head what you're working with

$" starts an interpolated string

{ within an interpolated string marks the start of a field that is evaluated

chief.Contains("Microsoft") is a problem. Contains asks whether a string contains another string, but remember that your chief is a Language object, not a string. Language doesn't have a Contains method. If it did, it would return true or false, this would be incorporated into the string by the interpolation process and you'd end up with one of the following strings:
"true"
"false"

These are not booleans. They are strings that contain words that we commonly use when talking about booleans but they aren't booleans, "1234" is not a number or a time, and "04-09-2000" is not a date. These are all strings, regardless of what your brain might recognize and interpret when it sees them (oh, and 9th April or 4th Sep?)
Put all this together, name our collection properly, name our expression input variable well, make sure it returns a Boolean...
var msLangs = allLangs.Where(lang => lang.ChiefDeveloper.Contains("Microsoft"));

You can now iterate and print them:
    foreach (var msLang in msLangs)
    {
      Console.WriteLine(msLang.Prettify());
    }

Note I'm keeping the sensible naming - collections have plural names

msLangs: a collection of Microsoft languages
msLang: one of the above named collection items

You don't need to put msLang.Prettify inside an interpolated string. Prettify already returns a string. Wrapping it in an interpolated string that contains nothing else is a wasted operation
I also want to point out that prettify does not return a string in the format you say you want output. Prettify adds commas
Because it's probably fair to say your specification of the problem hasn't been as accurate as it could have been, I'm not sure whether to fix this last point or not. Either modify your prettify so it emits tabs rather than commas or bypass prettify and interpolate a string yourself.
Another (and final) point I want to raise is that software engineers work to specifications and the specs have to absolutely be accurate, complete, comprehensive etc. Hopefully this whole experience has been a worthwhile one for you because at the start your spec was awful; it couldn't at all be implemented - it was the half baked sort of thing your boss would drop on your desk and expect you to do, and if you did it and assumed etc you'd have ended up giving him something he didn't want. Always, when talking to other software devs, be meticulous about the accuracy of your specification and it's completeness. Always, when receiving a spec off someone, demand the same.. otherwise it will be "garbage in, garbage out"
Comment: Don't worry, we all need more practice! :)
Comment: Thank you for a really good and in depth answer. Clearly i need a lot more practice. it worked with 1 modification, i had to change the allLangs to languages. I cannot thank you enough, i will take your advice and try my best to implement it.
Comment: yes; i renamed languages to allLangs so that I could later draw a distinction with `msLangs` "microsoft languages" vs `allLangs`. Should have pointed that out, but you cracked it! ;) In a few years (or months even) read this question back; I'm sure you'll say "whoa, I don't even recognize the guy that wrote that Q". It's a nice way to reflect on the developer you were vs are!
Answer: A tsv file is a TAB delimited file.  The data you posted does not have tabs.  The code below will work with a file that contains tabs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.IO;

namespace ConsoleApplication1
{
    class Program
    {
        const string FILENAME = @"c:\temp\test.txt";
        static void Main(string[] args)
        {
            DataTable dt = new DataTable();
            int row = 0;

            StreamReader reader = new StreamReader(FILENAME);
            string line = "";
            while ((line = reader.ReadLine()) != null)
            {

                string[] data = line.Split(new char[] { '\t' });
                if (++row == 1)
                {
                    foreach (string name in data)
                    {
                        dt.Columns.Add(name, typeof(string));
                    }
                }
                else
                {
                    dt.Rows.Add(data);
                }
            }

        }
    }
}

Comment: I will try this, it is cool to learn a new way to do things. The only thing is that they ask to use .Contain() method specifically which means there is a way to do it with the mentioned method.


Jmeter MQ GUI is working properly but cli summary showing error. Aggregate report shows different result

Question: Hi I am new to Jmeter and I am trying to use Jmeter to do some performance testing on IBM MQ docker.
I have setup IBM MQ docker following this:
https://developer.ibm.com/tutorials/mq-connect-app-queue-manager-containers/
My apps can consume and publish to the broker so its working fine.
I am trying out Jmeter for performance testing and I have setup the Jmeter(5.6.2) with the jmx file provided following the pdf guide.
https://github.com/ibm-messaging/mqperf/tree/gh-pages/jmeter
After running the GUI, it seems working fine.

But when I run it in cli, it seem to be giving me errors. And the result seems weird by having more received msg than send. It could be due to retry but I am not sure. The following is the command I used:
jmeter.bat -n -t "C:\Users\alex\Desktop\all jmeter stuff\jmeter test plan jmx files\MQ-Sender-Receiver-Plan.jmx" -l "C:\Users\alex\Desktop\all jmeter stuff\jmeter test plan jmx files\results.jtl" -j "C:\Users\alex\Desktop\all jmeter stuff\jmeter test plan jmx files\log.jtl"

I have the cli logs here. Thanks in advance for any helps!
2023-07-12 18:28:32,787 INFO o.a.j.u.JMeterUtils: Setting Locale to en_EN
2023-07-12 18:28:32,799 INFO o.a.j.JMeter: Loading user properties from: user.properties
2023-07-12 18:28:32,799 INFO o.a.j.JMeter: Loading system properties from: system.properties
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: Copyright (c) 1998-2023 The Apache Software Foundation
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: Version 5.6.2
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: java.version=11.0.19
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: java.vm.name=Java HotSpot(TM) 64-Bit Server VM
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: os.name=Windows 10
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: os.arch=amd64
2023-07-12 18:28:32,804 INFO o.a.j.JMeter: os.version=10.0
2023-07-12 18:28:32,813 INFO o.a.j.JMeter: file.encoding=Cp1252
2023-07-12 18:28:32,813 INFO o.a.j.JMeter: java.awt.headless=true
2023-07-12 18:28:32,813 INFO o.a.j.JMeter: Max memory     =4294967296
2023-07-12 18:28:32,813 INFO o.a.j.JMeter: Available Processors =12
2023-07-12 18:28:32,818 INFO o.a.j.JMeter: Default Locale=English (EN)
2023-07-12 18:28:32,818 INFO o.a.j.JMeter: JMeter  Locale=English (EN)
2023-07-12 18:28:32,818 INFO o.a.j.JMeter: JMeterHome=C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2
2023-07-12 18:28:32,818 INFO o.a.j.JMeter: user.dir  =C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\bin
2023-07-12 18:28:32,818 INFO o.a.j.JMeter: PWD       =C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\bin
2023-07-12 18:28:32,819 INFO o.a.j.JMeter: IP: 22.214.171.124 Name: P_IFSLT2650 FullName: parkview-100-169.tm.net.my
2023-07-12 18:28:32,828 INFO o.a.j.s.FileServer: Default base='C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\bin'
2023-07-12 18:28:32,829 INFO o.a.j.s.FileServer: Set new base='C:\Users\alex\Desktop\all jmeter stuff\jmeter test plan jmx files'
2023-07-12 18:28:32,926 INFO o.a.j.s.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2023-07-12 18:28:32,941 INFO o.a.j.s.SaveService: Using SaveService properties version 5.0
2023-07-12 18:28:32,944 INFO o.a.j.s.SaveService: Using SaveService properties file encoding UTF-8
2023-07-12 18:28:32,946 INFO o.a.j.s.SaveService: Loading file: C:\Users\alex\Desktop\all jmeter stuff\jmeter test plan jmx files\MQ-Sender-Receiver-Plan_no_diode - Copy.jmx
2023-07-12 18:28:33,010 INFO o.a.j.JMeter: Creating summariser
2023-07-12 18:28:33,020 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2023-07-12 18:28:33,021 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2023-07-12 18:28:33,021 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2023-07-12 18:28:33,028 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must contain the string: '.functions.'
2023-07-12 18:28:33,028 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must not contain the string: '.gui.'
2023-07-12 18:28:33,225 INFO o.a.j.t.TestPlan: added C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\lib\mqjars\com.ibm.mq.allclient.jar to classpath
2023-07-12 18:28:33,225 INFO o.a.j.t.TestPlan: added C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\lib\mqjars\jms.jar to classpath
2023-07-12 18:28:33,226 INFO o.a.j.t.TestPlan: added C:\Users\alex\Desktop\all jmeter stuff\apache-jmeter-5.6.2\lib\mqjars\org.json.jar to classpath
2023-07-12 18:28:33,434 INFO o.a.j.JMeter: Running test (1689157713434)
2023-07-12 18:28:33,435 INFO o.a.j.e.StandardJMeterEngine: Starting setUp thread groups
2023-07-12 18:28:33,435 INFO o.a.j.e.StandardJMeterEngine: Starting setUp ThreadGroup: 1 : setUp  Thread Group (Sender Connections)
2023-07-12 18:28:33,435 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group setUp  Thread Group (Sender Connections).
2023-07-12 18:28:33,435 INFO o.a.j.e.StandardJMeterEngine: Test will stop abruptly on error
2023-07-12 18:28:33,435 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:28:33,438 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2023-07-12 18:28:33,438 INFO o.a.j.e.StandardJMeterEngine: Starting setUp ThreadGroup: 2 : setUp Thread Group (Receiver Connections)
2023-07-12 18:28:33,438 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group setUp Thread Group (Receiver Connections).
2023-07-12 18:28:33,438 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2023-07-12 18:28:33,438 INFO o.a.j.t.ThreadGroup: Starting thread group... number=2 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:28:33,440 INFO o.a.j.t.ThreadGroup: Started thread group number 2
2023-07-12 18:28:33,441 INFO o.a.j.e.StandardJMeterEngine: Waiting for all setup thread groups to exit
2023-07-12 18:28:33,444 INFO o.a.j.t.JMeterThread: Thread started: setUp Thread Group (Receiver Connections) 2-1
2023-07-12 18:28:33,444 INFO o.a.j.t.JMeterThread: Thread started: setUp  Thread Group (Sender Connections) 1-1
2023-07-12 18:28:33,446 INFO o.a.j.s.SampleResult: Note: Sample TimeStamps are START times
2023-07-12 18:28:33,446 INFO o.a.j.s.SampleResult: sampleresult.default.encoding is set to UTF-8
2023-07-12 18:28:33,446 INFO o.a.j.s.SampleResult: sampleresult.useNanoTime=true
2023-07-12 18:28:33,446 INFO o.a.j.s.SampleResult: sampleresult.nanoThreadSleep=5000
2023-07-12 18:28:34,740 INFO o.a.j.p.j.s.J.Create Sender Connections to Queue Manager: ####### Sender connection #0 to queue manager QM1 established.
2023-07-12 18:28:34,740 INFO o.a.j.p.j.s.J.Create Receiver Connections to Queue Manager: ####### Receiver connection #0 to queue manager QM1 established.
2023-07-12 18:28:34,752 INFO o.a.j.t.JMeterThread: Thread is done: setUp  Thread Group (Sender Connections) 1-1
2023-07-12 18:28:34,752 INFO o.a.j.t.JMeterThread: Thread finished: setUp  Thread Group (Sender Connections) 1-1
2023-07-12 18:28:34,753 INFO o.a.j.r.Summariser: summary +      1 in 00:00:01 =    0.8/s Avg:  1303 Min:  1303 Max:  1303 Err:     0 (0.00%) Active: 1 Started: 2 Finished: 1
2023-07-12 18:28:34,753 INFO o.a.j.t.JMeterThread: Thread is done: setUp Thread Group (Receiver Connections) 2-1
2023-07-12 18:28:34,753 INFO o.a.j.t.JMeterThread: Thread finished: setUp Thread Group (Receiver Connections) 2-1
2023-07-12 18:28:34,754 INFO o.a.j.e.StandardJMeterEngine: All Setup Threads have ended
2023-07-12 18:28:34,770 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Senders
2023-07-12 18:28:34,770 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Senders.
2023-07-12 18:28:34,770 INFO o.a.j.e.StandardJMeterEngine: Test will stop abruptly on error
2023-07-12 18:28:34,770 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:28:34,771 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2023-07-12 18:28:34,771 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 2 : Receivers
2023-07-12 18:28:34,771 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Receivers.
2023-07-12 18:28:34,771 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2023-07-12 18:28:34,771 INFO o.a.j.t.ThreadGroup: Starting thread group... number=2 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:28:34,772 INFO o.a.j.t.JMeterThread: Thread started: Senders 1-1
2023-07-12 18:28:34,772 INFO o.a.j.t.ThreadGroup: Started thread group number 2
2023-07-12 18:28:34,772 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2023-07-12 18:28:34,773 INFO o.a.j.t.JMeterThread: Thread started: Receivers 2-1
2023-07-12 18:29:00,002 INFO o.a.j.r.Summariser: summary +   4986 in 00:00:25 =  197.5/s Avg:     2 Min:     1 Max:  1303 Err:     0 (0.00%) Active: 2 Started: 2 Finished: 0
2023-07-12 18:29:00,003 INFO o.a.j.r.Summariser: summary =   4987 in 00:00:27 =  187.7/s Avg:     3 Min:     1 Max:  1303 Err:     0 (0.00%)
2023-07-12 18:29:30,002 INFO o.a.j.r.Summariser: summary +   6000 in 00:00:30 =  200.0/s Avg:     2 Min:     1 Max:     9 Err:     0 (0.00%) Active: 2 Started: 2 Finished: 0
2023-07-12 18:29:30,003 INFO o.a.j.r.Summariser: summary =  10987 in 00:00:57 =  194.2/s Avg:     2 Min:     1 Max:  1303 Err:     0 (0.00%)
2023-07-12 18:29:34,771 INFO o.a.j.t.JMeterThread: Stopping because end time detected by thread: Senders 1-1
2023-07-12 18:29:34,772 INFO o.a.j.t.JMeterThread: Thread finished: Senders 1-1
2023-07-12 18:29:39,809 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:29:44,823 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:29:49,837 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:29:54,852 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:29:59,870 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:04,886 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:04,886 INFO o.a.j.r.Summariser: summary +    961 in 00:00:35 =   27.5/s Avg:    33 Min:     1 Max:  5031 Err:     6 (0.62%) Active: 1 Started: 2 Finished: 1
2023-07-12 18:30:04,888 INFO o.a.j.r.Summariser: summary =  11948 in 00:01:31 =  130.6/s Avg:     5 Min:     1 Max:  5031 Err:     6 (0.05%)
2023-07-12 18:30:09,903 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:14,915 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:19,929 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:24,946 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:29,961 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:34,982 INFO o.a.j.p.j.s.J.Receive Messages: ############ consumer.receive timed out (timeout value: 5000)
2023-07-12 18:30:34,983 INFO o.a.j.r.Summariser: summary +      6 in 00:00:30 =    0.2/s Avg:  5015 Min:  5012 Max:  5020 Err:     6 (100.00%) Active: 1 Started: 2 Finished: 1
2023-07-12 18:30:34,984 INFO o.a.j.r.Summariser: summary =  11954 in 00:02:02 =   98.3/s Avg:     7 Min:     1 Max:  5031 Err:    12 (0.10%)
2023-07-12 18:30:34,985 INFO o.a.j.t.JMeterThread: Stopping because end time detected by thread: Receivers 2-1
2023-07-12 18:30:34,986 INFO o.a.j.t.JMeterThread: Thread finished: Receivers 2-1
2023-07-12 18:30:34,986 INFO o.a.j.e.StandardJMeterEngine: Starting tearDown thread groups
2023-07-12 18:30:34,986 INFO o.a.j.e.StandardJMeterEngine: Starting tearDown ThreadGroup: 1 : tearDown Thread Group (Sender Connections)
2023-07-12 18:30:34,986 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group tearDown Thread Group (Sender Connections).
2023-07-12 18:30:34,986 INFO o.a.j.e.StandardJMeterEngine: Test will stop abruptly on error
2023-07-12 18:30:34,987 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:30:34,989 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2023-07-12 18:30:34,989 INFO o.a.j.e.StandardJMeterEngine: Starting tearDown ThreadGroup: 2 : tearDown Thread Group (Receiver Connections)
2023-07-12 18:30:34,989 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group tearDown Thread Group (Receiver Connections).
2023-07-12 18:30:34,990 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2023-07-12 18:30:34,990 INFO o.a.j.t.ThreadGroup: Starting thread group... number=2 threads=1 ramp-up=1 delayedStart=false
2023-07-12 18:30:34,990 INFO o.a.j.t.JMeterThread: Thread started: tearDown Thread Group (Sender Connections) 1-1
2023-07-12 18:30:34,991 INFO o.a.j.t.ThreadGroup: Started thread group number 2
2023-07-12 18:30:34,991 INFO o.a.j.t.JMeterThread: Thread started: tearDown Thread Group (Receiver Connections) 2-1
2023-07-12 18:30:35,042 INFO o.a.j.p.j.s.J.Close Receiver Connections: ###### Receiver Connection Closed
2023-07-12 18:30:35,042 INFO o.a.j.p.j.s.J.Close Sender Connections: ###### Sender Connection closed
2023-07-12 18:30:35,043 INFO o.a.j.t.JMeterThread: Thread is done: tearDown Thread Group (Sender Connections) 1-1
2023-07-12 18:30:35,043 INFO o.a.j.t.JMeterThread: Thread is done: tearDown Thread Group (Receiver Connections) 2-1
2023-07-12 18:30:35,043 INFO o.a.j.t.JMeterThread: Thread finished: tearDown Thread Group (Sender Connections) 1-1
2023-07-12 18:30:35,043 INFO o.a.j.t.JMeterThread: Thread finished: tearDown Thread Group (Receiver Connections) 2-1
2023-07-12 18:30:35,043 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2023-07-12 18:30:35,048 INFO o.a.j.r.Summariser: summary +      2 in 00:00:00 =   30.8/s Avg:    50 Min:    49 Max:    51 Err:     0 (0.00%) Active: 0 Started: 2 Finished: 2
2023-07-12 18:30:35,049 INFO o.a.j.r.Summariser: summary =  11956 in 00:02:02 =   98.3/s Avg:     7 Min:     1 Max:  5031 Err:    12 (0.10%)

Answer: Looking into the code:
def consumer=vars.getObject("Consumer");
def session=vars.getObject("ReceiverSession")

def msg = consumer.receive("${receive_timeout}" as Integer)

if(msg!=null) {
    if("${transacted}".equals("true")) session.commit()

    log.debug("########### Message Received"+msg)

    SampleResult.setResponseData(msg.toString())
    SampleResult.setDataType(org.apache.jmeter.samplers.SampleResult.TEXT)
} else {
    log.info("############ consumer.receive timed out (timeout value: " + "${receive_timeout}" + ")")
    //Mark timeout as 'error' to enable filtering out in listeners (to just analyse successful receive rate etc)
    SampleResult.setSuccessful(false)
}

it appears that you will get the following error:

############ consumer.receive timed out (timeout value: 5000)

when the receiver thread fails to retrieve the message withing 5 seconds timeout.
So first thing I would check is your MQ broker container logs
In general JMeter in non-GUI mode has much better performance than in GUI mode so it might be the case it just overloads your MQ broker instance and it cannot properly respond on message receive requests.
Apart from checking the logs you can additionally ensure that the broker has enough headroom to operate in terms of CPU, RAM, Network, Disk, etc. It can be done using container monitoring solution like docker stats or JMeter PerfMon Plugin. If this is the case you might need to increase container limits or Docker engine
Comment: I found out that it is because I increased the duration for running Receivers thread longer than sender thus this timeout error kicks in. Why I added longer duration for the Receivers thread is because I found that the test just ended without consuming all message that were published by the Senders to my broker thus I extended the duration for Receivers. There's is still some behavior that I do not understand for now but the timeout at least explained the error I am seeing using cli. Thanks.


Retornar o valor a esquerda

Question: bom dia.
Uma dúvida, como posso retornar os resultados de uma coluna que contenha apenas os valores a esquerda que iniciam com a palavra 'carro' por exemplo?
Estou usando o sql server.
Obrigado.
Comment: Não entendi a parte `"..apenas os valores a esquerda..."`
Answer: Pode usar o operador like:
SELECT * FROM EXEMPLO WHERE VALOR LIKE 'CARRO%'

O operador % é o coringa, você pode usar ele em qualquer posição. Por exemplo, se você quer ver se a palavra está no final do resultado você usaria %carro, se no meio do resultado usaria %carro% e assim por diante.
Comment: Show Diego, funcionou perfeito. Muito obrigado, abs.

Retornar o valor a esquerda


IPython: How to show the same plot in different cells?

Question: I'm still new to IPython Notebooks, Jupyter, and Python in general.
I'm creating a scatter plot in a Jupyter notebook using the following code:
import numpy as np
import matplotlib.pyplot as plt

n = 1024
X = np.random.normal(0, 1, n)
Y = np.random.normal(0, 1, n)
plt.axes([0.025, 0.025, 0.95, 0.95])
plt.scatter(X, Y, s=50)

plt.show()

My question is, how can I get a reference to the plot object so I can use it in a different cell later on in the notebook? Additionally, I may need to modify the plot before showing it again.
Also, I have %matplotlib inline at the top of my notebook.
Here are some info about my environment:

Python: 3.5.2 64bit [MSC v.1900 64 bit (AMD64)]
IPython: 4.2.0
numpy: 1.11.1
scipy: 0.17.1
matplotlib: 1.5.1
sympy: 1.0
OS: Windows 7 6.1.7601 SP1
Comment: Sorry for digging out this question. My workaround is to call `fig.savefig` in the new cell to save to an image file and open the image in a viewer (e.g., EyeOfGnome) that refreshes automatically to show the current copy. I still don't know how to show the figure in a new notebook cell. I think it might have something to do with Jupyter internals, particularly the `%matplotlib inline` directive. Maybe it will work in some backend.
Comment: Another workaround is to use the interactive backend: `%matplotlib notebook`. A new plotting command will udpate the original canvas. But I believe this is still not quite what is intended.
Answer: I have found a solution! Basically you create a figure and the axis with fig, ax = plt.subplots() and then use the ax variable to draw (potentially in multiple cells). In any of the cells you want to replot the figure, just write fig as the last line of the cell, resulting in the cell using the updated figure as output.
See my answer here for more details.
Answer: Try the object-oriented interface of matplotlib  - matplotlib.figure; you can use the reference of the Figure object created, as required.

Create a figure object - fig = plt.figure()
Add axes to it - ax = fig.add_axes([0.025, 0.025, 0.95, 0.95])
Plot on the axis created - ax.plot(X, Y)
Comment: If I go this rout, I'd be creating `scatter` plot every time I call `ax.scatter()`. Still that did not work as expected in my notebook.


$X$ compact convex subset of $\mathbb{R^n}$. $f:X\to X,f(Fr(X))\subset Fr(X)$. $f|_{Fr(X)}$ is homotopic with identity on $Fr(X)$. Show that $f(X)=X$

Question: Let $X$ be compact convex subset of $\mathbb{R}^n$ with non-empty interior. $f:X\to X$ is map such that $f(Fr(X))\subset Fr(X)$. Moreover $f|_{Fr(X)}$ is homotopic in $Fr(X)$ with identity on $Fr(X)$. Show that $f(X)=X$.
I would like to use the fact that if $g:[0,1]^n\to[0,1]^n$ and $g(F_k)\subset F_k$ where $F_k$ is face of $[0,1]^n$, then $g([0,1]^n)=[0,1]^n$.
Assuming that $f(Fr(X))=Fr(X)$ this should be straightforward because $X$ is homeomorphic with $[0,1]^n$. However I do not know how to show that this assumption holds.
[edit]
Assuming that $f(Fr(X))\neq Fr(X)$, we get with other assumptions that $Fr(X)$ is homotopic with it's proper subset. It does not look right.
Comment: What is $Fr(X)$? The boundary of $X$ (closure minus interior)?
Comment: @NajibIdrissi, yes.
Answer: All closed, bounded, convex subsets with non-empty interior are homeomorphic. One way is to show that each set is homeomorphic to the unit ball.
Suppose $X$ is closed, bounded and convex, with non-emtpy interior. Without loss of generality (by applying translation and scaling as necessary; all bicontinuous operations), we may assume that the open unit ball is contained in $X$.
Consider the gauge function of $X$:
$$\gamma(x) = \inf \lbrace \lambda \ge 0 : x \in \lambda X \rbrace.$$
This function is continuous, positive-definite, and takes values between $0$ and $1$ when mapping points inside $X$. In fact, $\gamma(x) = 1$ if and only if $x \in \operatorname{Fr}(X)$. Let $B$ be the closed unit ball, and define,
$$f : X \to B : x \mapsto \begin{cases}\frac{\gamma(x)}{\|x\|}x & \text{if } x \neq 0 \\ 0 & \text{if } x = 0\end{cases}.$$
It takes a little work, but one can show $f$ is a homeomorphism. I don't have a proof to hand, so I'll leave that to you to figure out the details.
Answer: You know that $X$ is homeomorphic to $[0,1]^n \approx D^n$. Let $h : X \to D^n$ be a homeomorphism. By invariance of domain we see that $h(int(X) = int(D^n)$, hence also $h(Fr(X)) = Fr(D^n) = S^{n-1}$. Let $f' = F\mid_{Fr(X)} : Fr(X) \to Fr(X)$ and $h' = h_{S^{n-1}} : S^{n-1} \to S^{n-1}$. Then $\phi = h' f' h^{-1}$ is homotopic to the identity.
If $f'$ would not be surjective, then we can pick $x \in Fr(X)$ with $x \notin f'(Fr(X))$. But then $\phi(S^{n-1}) \subset S^{n-1} \setminus \{ h'(x) \} \approx \mathbb R^{n-1}$. Hence $\phi$ is null-homotopic. This contradicts the fact that $\phi$ is homotopic to the identity.


When reverse item loaded on one factor when they are not supposed to

Question: I'm conducting a survey online and one of my measurement is 'Credibility' variable, which uses total of 9 variable (see image). These variables are supposed to fall into three dimensions, which are credibility, expertise, and attractiveness. However, the three questions which are not related to each other loaded on one factor (2) and rest on the other side.
Can this mean that people were not paying attention?
While my initial expectation is that all of these 9 variable (or at least most) to be significant (in anova), however, those three that were reverse coded were only ones that are significant. What could this mean?
Could someone help me to find problem in this, so I can conduct successful manipulation?
Thank you very much
Answer: For the first question: No, it is not likely the result of people not paying attention -- that would likely result in very low correlations and low factor loadings.  It is more likely that something is wrong either with your theory or your questions.
For your second question: Being reverse coded shouldn't have anything to do with the results of an ANOVA. That part is likely a coincidence. But have you checked the assumptions of the model? Colinearity seems likely, given the names of your variables.


strcmp only works at the start of a loop

Question: I've been trying to create a simple program that loops through an array's members and scan the characters looking for a set specific character. I have ran into an issue where strcmp() only works at the start of the loop. I'm struggling to understand why this happens and any help would be appreciated.
char *file[3] = {"!x", "!!x", "x!"};

for (int i = 0; i < sizeof(file) / sizeof(file[0]); i++) {
  char *line = file[i];
  printf("\n");
  for (int i = 0; i < strlen(line); i = i + 1) {
    char character = line[i];
    if (strcmp("!", &character) == 0) {
      printf("[YES] %c\n", character);
    } else {
      printf("[NO] %c\n", character);
    }
  }
}

Output

[YES] !
[NO] x

[YES] !
[NO] !
[NO] x

[NO] x
[NO] !

Comment: Why do you use a **string** compare function to compare single **characters**?
Comment: you may have undefined behaviour: nothing guarantees that the pointer on a char has a zero somewhere near since it's not a string.
Comment: The comments here about a null-terminate string being required are, of course, correct. The behavior still seems strange since, in this case, we know what comes after `&character` since we know the rest of the characters in the string.  It's not clear to me why it misbehaves in this specific way.
Answer: The strcmp function expects the address of a null terminated string.  Instead you're passing it the address of a char.  strcmp then attempts to read memory locations past character, resulting in undefined behavior.
The real issue however is that you don't want to compare strings.  You want to compare characters.
if (character == '!') {

Answer: The problem here is, you're supplying wrong argument to strcmp(), the &character is not a pointer to a string.
Quoting C11, chapter int strcmp(const char *s1, const char *s2);

int strcmp(const char *s1, const char *s2);
The strcmp function compares the string pointed to by s1 to the string pointed to by
s2.

So, it expects both the arguments to be of string type, which in your case is not.
You can simply use the comparison operator == to compare chars, like
 if (line[i] == '!')  //notice the '' s, they are not ""s

and so on.
Answer: strcmp() compares null terminated strings.  In the code:
char character = line[i];
if (strcmp("!", &character) == 0)

the character is not a null-terminated string.  It is accidental that it works at all.
You need something more like this to compare strings:
char character[2] = { line[i], '\0' };
if (strcmp("!", character) == 0)

Or like this to compare characters:
char character = line[i];
if (character == '!')


Does the problem of multiple testing also apply to the testing of assumptions?

Question: When dealing with statistical tests, sometimes we can run into cases where many assumptions would apply and consequently would need to be tested. In complex models, testing many assumptions at 5% may in itself produce at least one significant test which may violate assumptions that would in turn prevent further testing.
Is there any rationale why we correct when performing post-hoc tests but not when testing assumptions even though the same problem applies in both situations in theory? Is it ever done and are there any recommendations?
Comment: Ordinarily, one might do one or two tests (e.g, normality, homoscedasticity) to check assumptions. If these tests are used to 'branch' to t vs. Wilcoxon, or to pooled vs. Welch t, there may be a question about the true P-value of the 'hybrid' (2-step) test. That is one reason some statisticians try to avoid using tests of assumptions in this way. (For example, by using Welch t tests when there is any question of heteroscedasticity.) // Formally, one usually seeks to avoid false discovery doing (up to 10) _post hoc_ tests on pairs of levels of, say, a one-way ANOVA with 5 levels of the factor.
Comment: Good points by @BruceET. Also, tests of assumptions tend to be a poor choice for checking model assumptions: They have poor power in the scenario when assumptions matter the most (small sample size), and are more likely to detect small (probably true) deviations that do not really matter (e.g. slightly non-normal residuals are known to be pretty harmless for e.g. randomized controlled trials with large samples sizes) for large sample sizes. Even "better", testing assumptions & changing analysis strategy based on it is known to inflate the type I error rate of the analysis you do in the end.
Answer:
Aris Spanos argues here and elsewhere that tests of assumptions (misspecification tests) should not be taken as affecting the later model-based inference, because what they are testing is essentially different from the later inference. This can be made mathematically precise in cases in which misspecification tests are run in a way that makes them statistically independent of the later inference, as in linear regression when doing misspecification testing on the residuals. In these cases running tests without correction is fine. In many other cases, misspecification tests may be approximately independent of the later inference, in which case misspecification testing can be treated as "mostly harmless". (This however does not automatically imply that it's a good thing to do, see below.)

Standard correction, e.g., Bonferroni, of test levels is generally not a good idea in connection with misspecification testing, because rejection of a misspecification test is meant to indicate problems with the model assumptions rather than being treated as another "scientific discovery". Correction for multiple testing generally protects the level but will affect the power, however in this case affecting the power means that problems with the model may remain undetected. On the other hand, as we don't aim for rejecting the model assumption, there is little danger that a rejection of it will be overinterpreted other than by applying more caution when running the final inference. This is arguably tolerable even if ultimately the probabilty of a random rejection of any null hypothesis in the two-stage procedure involving the assumptions test may be larger than the nominal level. This is not a big problem if only the final hypothesis is of real scientific interest, so one can consider the level of that final test alone.

To what extent misspecification testing is helpful is controversial. Unfortunately it depends strongly on what exactly is done, i.e., what misspecification test(s) you use, what model-based inference you want to run, what you do alternatively in case you find problems with the assumptions, and unfortunately also on how exactly the assumptions are violated in case they are violated (which of course the analyst cannot know). There are a number of studies that investigated procedures involving model assumption testing, and results are quite diverse. A survey is here. The authors also point out that although much work comes out critical about misspecification testing, and it sometimes is not a good idea indeed, the existing work overall may be too pessimistic, particularly because unless we can use subject matter knowledge, we don't have very good alternatives: Pretty much all problems with formal misspecification testing exist for informal alternatives like visual checking as well, and the dangers of using methods without checking assumptions are well known and documented.

As a side remark, some work investigating specific procedures involving misspecification tests recommends to run the misspecification test at a higher level than the usual 5%, which is the opposite of standard corrections.


UnicodeDecodeError on WTForms

Question: This bug has caused me many hours of research without any solution in sight.
This is the traceback on GAE:
Traceback (most recent call last):
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/site/decorators.py", line 36, in decorated_view
    return func(*args, **kwargs)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/site/views.py", line 134, in publish_news
    return render_template('news.html', post=news, form=form)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/templating.py", line 128, in render_template
    context, ctx.app)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/flask/templating.py", line 110, in _render
    rv = template.render(context)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/jinja2-2.6/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/site/templates/news.html", line 1, in top-level template code
    {% extends "base.html" %}
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/site/templates/base.html", line 93, in top-level template code
    {% block body %}
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/site/templates/news.html", line 55, in block "body"
    {{ form.tweet }}
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/wtforms/fields/core.py", line 133, in __html__
    return self()
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/wtforms/fields/core.py", line 149, in __call__
    return self.meta.render_field(self, kwargs)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/wtforms/meta.py", line 53, in render_field
    return field.widget(field, **render_kw)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/wtforms/widgets/core.py", line 257, in __call__
    escape(text_type(field._value()), quote=False)
  File "/base/data/home/apps/s~xxx/1-0-38.386228028384623065/lib/wtforms/fields/core.py", line 529, in _value
    return text_type(self.data) if self.data is not None else ''
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 60: ordinal not in range(128)

It seems {{ form.tweet }} is causing this.
class PublishForm(Form):
    tweet = TextAreaField('tweet', [validators.DataRequired(), validators.Length(-1, 123, 'Max 140 chars allowed')])

The view is like this:
# coding: utf8

def publish_news(news_id):
    news = ndb.Key(urlsafe=news_id).get()
    form = PublishForm()
    if form.validate_on_submit():
        ...
        news.put()
        return redirect(url_for('list_news'))
    if not form.tweet.data:
        tweet = u''
        for s in news.slug_list:
            if '-' in s:
                continue
            tweet = tweet + u'#' + s + u', '
        tweet = tweet[:-2].encode('utf-8')
        head_line = news.head_line.encode('utf-8')
        max_tweet_length = 140 - (len(tweet) + 4 + 23)  # three dots + space + url
        tweet = head_line[:max_tweet_length] + '... [URL] ' + tweet
        form.tweet.data = tweet
    return render_template('news.html', post=news, form=form)

It crashes during GET, never gets to POST. What could I be missing?
Answer: I had to change this line
form.tweet.data = tweet

to
form.tweet.data = tweet.decode('utf-8')

And remember once back at POST, you have to do .encode('utf-8') again.
Comment: have you find other solution for this>I find it's rather annoy, because I need to write this code again and again.
Comment: No other solutions I'm afraid. Its very annoying, I agree.

UnicodeDecodeError on WTForms


Adding Pushover integration in Django

Question: I've recently started using Pushover.net, I've done some searching and can't find any examples of it being integrated with a django project.
Since i can't find any examples I've decided it would be fun to try myself. What I'm interested in is how you would suggest I do it. I want the actual pushover part as decoupled a possible, hence doing it asas an app.
What I'm not entirely sure on how to approach is the user authorization. The idea being a user enters their pushover user key and its saved in a user profile model using django's AUTH_PROFILE_MODULE with some functions such as has_pushover  but obviously I'd like some security so the user keys aren't stored in plaintext. What do people suggest for this?
Is there some inbuilt django security I can use?
Answer: In the past when I've needed to encrypt Django fields I used the encrypted fields available in django-fields. You could use one of these on your UserProfile model and define a has_pushover() method on the model which basically returns whether the pushover token field is None or not.
I'm guessing because you're talking about storing each user's Pushover token you are wanting to build an app for pushing arbitrary notifications to your website's users? This is in contrast to having the website just push notifications to yourself for site events.

Adding Pushover integration in Django


Need help w/ JavaScript and creating an html table from array

Question: I've tried everything I can find via google and nothing has worked correctly. Output is just a single row with all the contents of the array listed. What I need is a way to write the contents of an array but after 3 cells, automatically start a new line. I'll post the code I've made below as well as the question. (yes this is from an assignment. :( )
//***(8) place the words in the string "tx_val" in a table with a one pixel border,
//***    with a gray backgound. Use only three cells per row. Empty cells should contain
//***    the word "null". Show the table in the span block with id="ans8"

var count = i % 3;
var nrow = "";
var out = "

"
for (var i=0; i<txArr.length; i++)
{
    out += ("" + txArr[i] + "	");
    count++;
    if (count % 3 == 0)
    {
        nrow += "

";
    }
}

document.getElementById('ans8').innerHTML = out + nrow;

Comment: No problem with having assignments posted here, so long as you state that this is the case. However, you could tell us what exactly you are having difficulties with...
Comment: Can you post a sample output for the array?
Comment: You don't need the `nrow` var, just change that line to `out += ""` and you're half-way there.
Comment: Your code doesn't work? Please provide any error messages.
Answer: you need to print the tr's inside the table (annd add a
!):
var count = i % 3; // btw. what's this??
var nrow = "";
var out = "

"
for (var i=0; i<txArr.length; i++)
{
    out += "" + txArr[i] + "	";
    count++;
    if (count % 3 == 0)
        out += "

";
}
out += "
";

document.getElementById('ans8').innerHTML = out;

Comment: don't need the `nrow` variable anymore though; and isn't out reserved (I don't remember)
Answer: Rather than try to write out the html, try manipulating the dom.  It seems much more straightforward to me.  Take a look at the following:
var row = table.insertRow();
msdn
mdc
var cell = row.insertCell();
msdn
mdc
var cellContent = document.createTextNode(txArr[i]);
msdn
mdc
cell.appendChild(cellContent);
msdn
mdc
For deciding when to start a new row, just use the modulus operator (%
msdn
mdc
) against i:
if (i % 3 == 0)
{
    row = table.insertRow()
}

You'd end up with something like this:
var container = document.getElementById("ans8");
var t = container.appendChild(document.createElement("table"));
var row;
txArr.forEach(function (item, i)
{
    if (i % 3 == 0)
    {
        row = t.insertRow()
    }
    row.insertCell().appendChild(document.createTextNode(item));
});

I'll leave a little for you to figure out - border, background color, getting the word "null" in there.  It is your homework after all.  :-)
Also, for older browsers you'll need to add Array.forEach in yourself.
Answer: It might be a tad easier to accomplish with something like
buffer = "
";
for(var r = 0; r < 10; r++){
   buffer += "
";
   for(var c = 0; c < 3 ; c++){
      buffer += "Cell: " + r + ":" + c + "	";
   }
   buffer += "
";
}
buffer += "
";
 document.getElementById("ans8").innerHTML = buffer;

That would create a table 30 rows long by 3 columns for each row.
Answer: you might be assigning values to "count" too early as you don't know what i is yet. and you are not spitting out the value of nrow anywhere... change it to out.
var count;
var nrow = "";
var out = "

"
for (var i=0; i<txArr.length; i++)

{
out += ("" + txArr[i] + "	");
count++;
if (count % 3 == 0)
    {
    out += "

";
    }
}

 document.getElementById('ans8').innerHTML = out + nrow;

Answer: Basically I would split it up into 3 functions, for readability and maintenance.  These functions would consist of creating a cell, a row, and a table.   This definitely simplifies reading the code.  As I have not tested it, below is an example of what I would do.
function createTableCell(value) {
  return value == null? "NULL	":"" + value + "	";
}

function createTableRow(array) {
  var returnValue = "";
  for (var i = 0; i < array.length; i++) {
    returnValue = returnValue + createTableCell(array[i]);
  }
  return "
" + returnValue + "
";
}

function arrayToTable(array, newRowAfterNArrayElements) {
  var returnValue = "
";
  for (var i = 0; i < array.length; i = i + newRowAfterNArrayElements) {
     returnValue = returnValue + createTableRow(array.split(i, (i + newRowAfterNArrayElements) - 1));
  }
  return returnValue + "
";
}

document.getElementById("ans8").innerHTML = arrayToTable(txArr, 3);

In addition this makes your code much more dynamic and reusable.  Suppose you have an array you want to split at every 4 element.  Instead of hardcoding that you can simply pass a different argument.
Answer: I prefer using an array over concatination
var html = [];
html.push("

");
var i = 0;

for (var k in txArr)
{
    if(i>=3){
      i=0;
      html.push("

");
    }
    html.push("" + txArr[k] + "	");
    i++;
}
html.push("

");

document.getElementById('ans8').innerHTML = html.join('');

// wrapped in function
function arrayToTable(a,cols){
 var html = [];
    html.push("

");
    var i = 0;

    for (var k in a)
    {
        if(i>=cols){
          i=0;
          html.push("

");
        }
        html.push("" + a[k] + "	");
        i++;
    }
    html.push("

");
    return html.join('')
}
document.getElementById('ans8').innerHTML = arrayToTable(txArr, 3);

Answer: Yes, you can build from scratch...but there's a faster way.  Throw a grid at it.  The grid will take data in a string, array, json output, etc and turn it into a proper HTML outputted table and will allow you to extend it with sorting, paging, filtering, etc.
My personal favorite is DataTables, but there are numerous others out there.
Once you get proficient, setting one of these up literally takes 5 minutes.  Save your brain power to cure world hunger, code the next facebook, etc....
Answer: Here's a live example of doing this with DOMBuilder, and of using the same code to generate DOM Elements and an HTML String.

http://jsfiddle.net/insin/hntxW/

Code:
var dom = DOMBuilder.elementFunctions;

function arrayToTable(a, cols) {
  var rows = [];
  for (var i = 0, l = a.length; i < l; i += cols) {
      rows.push(a.slice(i, i + cols));
  }
  return dom.TABLE({border: 1, bgcolor: 'gray'},
    dom.TBODY(
      dom.TR.map(rows, function(cells) {
        return dom.TD.map(cells);
      })
    )
  );
}

var data = [1, 2, null, 3, null, 4, null, 5, 6];

document.body.appendChild(arrayToTable(data, 3));
document.body.appendChild(
  dom.TEXTAREA({cols: 60, rows: 6},
    DOMBuilder.withMode("HTML", function() {
      return ""+arrayToTable(data, 3);
    })
  )
);


Convert one JSON structure to another using Json.Net

Question: Input JSON:
{
    "name" : "objname",
    "abc" : 1,
    "def" : 2
}

Desired output JSON:
{
    "objname" :
    {
        "abc" : 1,
        "def" : 2
    }
}

I tried as shown below, but I feel it's not the correct way.
// This is the class object
public class Obj
{
    public string Name { get;set;}
    public string abc { get; set; }
    public string def { get; set; }
}

var obj = JsonConvert.DeserializeObject<Obj>(json);

StringBuilder sb = new StringBuilder();
sb.Append(" { ");
sb.AppendLine(obj.Name);
sb.AppendLine(" :  {");
sb.AppendLine(GetMemberName(() => obj.abc) + ":" + obj.abc + ",");
sb.AppendLine(GetMemberName(() => obj.def) + ":" + obj.abc);
sb.AppendLine(" :  }");

Comment: 1) Your desired output `"objname" : { "abc" : 1, "def" : 2 }` isn't even JSON, it's missing the outer `{` and `}`.  2) What have you tried so far?
Comment: @GarrGodfrey: No, JSON doesn't have to start with `{` or `[`. It's perfectly valid to have a JSON document which is *just* a string, null token or number for example. But yes, for an object, you do need the braces.
Comment: So please show what you have tried, and what happened.
Comment: input is invalid json and so is the output
Comment: Look at the homepage for this package. http://www.newtonsoft.com/json. You will see a Serialize and DeserializeObject. That's what you need. And please try before asking here.
Comment: @GarrGodfrey  input isn't, output is
Comment: input missing a comma, so JSON.NET won't parse it. JSON must start with { or [, so it will not generate that output.
Comment: then the answer is "yes"
Comment: sorry, syntax in question was incorrect. yeah, i do have tried to get solution, but failed. so tried asking here quickly. now i do have edited, it would be grateful, if i get any solution. thanks in advance.
Comment: Thanks, @JonSkeet. You are correct, since 2014 those are allowed. I'm just used to maintaining backward compatibility.
Comment: @JonSkeet :  I know its poor code, but this is what i tried.    var obj = JsonConvert.DeserializeObject(input);

            StringBuilder sb = new StringBuilder();
            sb.Append(" { ");
            sb.AppendLine(obj.Name);
            sb.AppendLine(" :  {");
            sb.AppendLine(GetMemberName(() =>obj.abc)+":" + obj.abc+ ",");
            sb.AppendLine(GetMemberName(() => obj.def) + ":" + obj.abc);
            sb.AppendLine(" :  }");
Comment: Please include that *in the question*, along with what `Obj` looks like, and details of what happened.
Answer: The first step is deserializing. You can just go to a dictionary:
Dictionary<string, string> dict =
    JsonConvert.DeserializeObject<Dictionary<string, object>>(json);

Then, to get it back, you probably need to use a dictionary as well. You cannot use a predefined (even compiler generated) class, since you don't know the property name.
String name = (string)dict["name"];
// remove name from dictionary
dict.Remove(name);
var output =
   new Dictionary<string,object>() {{name, dict}};

Then, save it back out:
string json = JsonConvert.SerializeObject(output);

Comment: You will have to deserialize to a `Dictionary` instead => [.net fiddle](https://dotnetfiddle.net/Ml1FRX)
Comment: That will **not** produce the requested output => [.net fiddle](https://dotnetfiddle.net/vlCaqA)
Comment: yep, typo. fixing
Comment: yeah, saw your fiddle. I was intended as a guide, not ready to go
Comment: @GarrGodfrey Thanks a lot. Its working. I was not knowing how to leverage dictionary.


$L^p_x$ norms of $f(x, x+y)$

Question:
Let $f \in \mathcal{S}(\mathbb{R}^2)$. Then do there exists inequalities of the form $$\lVert f(x, A(x,y))\rVert_{L^p_x(\mathbb{R})} \lesssim_{A, p,q} \lVert f(x,y) \rVert_{L^q(\mathbb{R}^2)} $$ where $A : \mathbb{R}^2 \to \mathbb{R} $ is linear?

An often studied case is when $ f$ is separable: That is if $ f(x, y) = K(x) g(-y)$ then we have
$$  f(x , x -  y)  = K(x) g( y - x)\quad \text{ implying} $$ $$ \lVert  f (x, x - y)\rVert_{L^1_x} = \lVert  K(x) g(y - x ) \rVert_{L^1_x} =| K \star g(y)| \leq    \lVert  K \star g \rVert_{L^\infty} \leq \lVert  K \rVert_{L^2} \lVert  g \rVert_{L^2} = \lVert  f \rVert_{L^2}.
$$
So are there similar results for some more general assumptions on $f?$
Comment: If $A(x, y)=ax+by$ then from the change of variable formula for integrals you see that what is relevant is $|b|$.
Comment: @GiuseppeNegro Apologies, I had phrased the question a little differently than what I wanted. I have edited it correctly now.
Answer: Let $A(x, y)=ax+by$. Up to a rotation and a dilation of $\mathbb R^2$ we can assume $a=1, b=0$. Thus, you are asking whether
$$\tag{1}
\left(\int_{-\infty}^\infty |f(x, 0)|^p\, dx\right)^\frac1p\lesssim \left(\int_{\mathbb R^2} |f(x, y)|^q\, dxdy\right)^\frac1q.$$
As you mention in comments to this answer, this is not possible. Take $f\in L^q(\mathbb R^2)$. You can always rescale it as $f_\lambda(x, y)=f(x, \lambda y)$. The left-hand side of (1) is left unchanged, but the right-hand side is not, and you can make it arbitrarily small. Thus, (1) cannot hold with a constant that is independent on $f$.

In technical jargon you are asking whether an $L^q(\mathbb R^2)$ function can have an $L^p(\mathbb R)$ trace. This is not possible, as we just saw. You need to require more regularity on $f$, which will rule out the scaling counterexample. For more information on this subject, check https://en.wikipedia.org/wiki/Trace_operator, or look up "Trace theorems" in your favorite book in PDE.

PREVIOUS VERSION OF THIS ANSWER. Here I erroneously refer to a conjectured inequality
$$\tag{2} \lVert f(x, A(x, y))\rVert_{L^p_{x, y}(\mathbb R^2)}\lesssim \lVert f \rVert_{L^q(\mathbb R^2)}.$$
Provided $b\ne 0$, the left-hand side of (2) is actually equivalent to $\lVert f \rVert_p$. Indeed, letting $(x', y')=(x, A(x, y))$ we see that $dx'dy'=|b|dxdy$, so
$$
\int_{\mathbb R^2} |f(x, A(x, y))|^p\, dxdy=\frac{1}{|b|} \int_{\mathbb R^2} |f(x', y')|\, dx'dy'.$$
Therefore, you are essentially asking whether $\lVert f\rVert_p\lesssim \lVert f\rVert_q$, which is only possible for arbitrary $f$ in the trivial case $p=q$.
Comment: Sorry, if it wasn't clear in the question but the norm is only on $x$. As such, I am more asking (for ex: $A(x,y) = x+ y$)
$$\forall y \in \mathbb{R}:  \left( \int_{\mathbb{R} }   \lvert f(x, x + y) \rvert^p \mathrm{\,d} x \right)^{\frac{1}{p}} \lesssim C \left( \int_{\mathbb{R}^2}  \lvert f \rvert^q \mathrm{\,d} x \mathrm{\,d} y \right)^{\frac{1}{q}}
$$
But I now realise that this is fruitless as I can just mollify $\mathbf{1}_{(x, x + y)} $ given a fixed $y;$ then the right hand side can be made arbitrarily small for $ q = p$.
Comment: Oh sorry now I see. I had totally overlooked the $L^p_x$

$L^p_x$ norms of $f(x, x+y)$


Circular Buffer Comparison / Lookup Algorithm

Question: I want to detect if a circular buffer is a mirror, or rotation, of a list of other buffers, of the same length.
For example given the following 3 buffers:
AAABCCCA
AABCCBAA
AAAACCAA

Then:
CCBAAAAC would match, as it is a mirrored rotation of the first buffer.
Currently I simply compare every buffer in every rotation, then reverse the buffer and do it all again.
This requires:
2*n*i buffer comparisons.
(Where n is number of buffers to compare two, and i is the length of the buffer).
Does anyone know a faster algorithm?
The list to compare against grows (as I find one not in the list I had it). Is there a method where I can somehow 'order' buffers, so that comparisons can be done faster?
One idea I had was to also store a sorted version of each buffer, so that you can quickly compare if the same number of items exist.
But is there are more general solution to somehow order by 'circular sequence'?
Comment: double up search pattern, then look for exact match in it. ??
Comment: That is essentially the same thing as I have said. The pattern matching would look in i places across n strings, and you would have to do it backwards, for 2*n*i. However what you have said is the easiest way to code it using standard functions like strstr etc.
Answer: For each buffer, you could rotate it until it has the minimum lexicographical value.
For example, your example buffers could be rotated like this:
AAABCCCA -> AAAABCCC
AABCCBAA -> AAAABCCB
AAAACCAA -> AAAAAACC

This basically means putting the lowest-valued item first.  If there is a tie, compare the next items, etc.  This gives a unique ordering for any pattern.
Comment: Wow. I don't know why I didn't realise this. To many buffers spinning around in my head. I can do that, then order order them lexicographically, which means i could do a binary search. The number of buffer comparisons would then be down from 2*i*n to log(n).
Comment: You could store/lookup the minimum lexicographical value between the rotated original and the rotated mirror, rather than storing both. @Myforwik It would be `i log n` not just `log n`. I think you could use a [trie](http://en.wikipedia.org/wiki/Trie)-like structure to get down to `O(i)`.
Comment: @Myforwik Oh, "buffer comparisons". Then `log n` is correct. The Trie is i.t.o character comparisons, not buffer comparisons. A more common way would be to count character comparisons instead.