Rails functional tests, routing error

Question: I'm new to testing, and I'm having some difficulties trying to run a functional test.
I've a messages_controller, and a user_controller here.
in the routes, I've defined that the users resources has_many message resources.
Now I'm trying to run a simple test in the messages controller:
def test_index
  get :index, { :user_id => 1 }
  assert_template 'index'
end

But get a routing error from rails, that he cant find a route to messages. I don't want to include a route to messages only because of the tests. How can I tell the test that he must access from the /users/messages url?
the full routes.rb:
ActionController::Routing::Routes.draw do |map|

  map.login  'login',   :controller => :user_sessions, :action => :new
  map.logout 'logout',  :controller => :user_sessions, :action => :destroy
  map.signin 'signin',  :controller => :users,         :action => :new

  map.connect 'search/:action/:word', :controller => :search
  map.connect 'search/:word',         :controller => :search, :action => :index

  map.resources :forums do |forums|
    forums.resources :forum_posts, :collection => {:preview => :post }, :as => :posts do |post|
      post.resources :forum_posts, :as => :reply
      post.resources :reports
    end
  end

  map.resources :newsitems, :as => :news do |news|
    news.resources :comments do |comment|
      comment.resources :reports
    end
  end

  map.resource :user_sessions
  map.resources :users,
                :as => :profiles,
                :controller => :profiles,
                :has_many   => [ :messages ]
  map.resource :profiles
  map.resource :me,
               :controller => :me,
               :has_many   => [ :messages ]

  map.resources :comments, :has_many => [ :reports ]
  map.resources :forum_posts, :has_many => [ :reports ]
  map.resources :reports

  map.home  '/', :controller => :home
  map.root  :controller => :home

  map.namespace :admin do |admin|
    admin.namespace :forum do |forum|
      forum.resources :categories
      forum.resources :posts
      forum.resources :forums
      forum.root      :controller => :home
    end
    admin.resources :notices
    admin.resources :users
    admin.workflow  'workflow/:action', :controller => :workflow
    admin.resources :newsitems
    admin.resources :reports
    admin.resources :comments
    admin.root :controller => :home
  end

  map.connect ':controller/:action/:id'
  map.connect ':controller/:action/:id.:format'
end

UPDATE
I've noticed that every functional test get a routing error. Even the simpliests ones like newsitem. I've no idea why.
Answer: I recreated your scenario in a blank rails app with the routing code and and test you specified, and it worked without a problem, as it should.  I'll paste my controller code here since that's the only part you left out:
class MessagesController < ApplicationController
  def index
    @messages = User.find(params[:user_id]).messages
  end
end

If yours is doing basically the same thing, then a routing issue could be caused by a conflict in your routing file, which is what I suspect might be the case.  Can you post it?  FYI, I wrote an article on testing your routes, and that would be a very good idea because it would catch routing errors early, before they interfere with controllers.
Anyway, if you can post your routes I can take a look.
UPDATE: After looking at your routes, there are a couple conflicts.  You can have messages as a sub-resource of more than one other resource, but in your messages controller you're going to have to account for the possibility of either a params[:me_id] or params[:profile_id].  It looks like they're both really the user model underneath, so it can be as simple as:
@user = User.find(params[:me_id] || params[:profile_id])

and you'll probably want to abstract that out into a method you call with before_filter.
The other issue is that you have two overlapping profiles routes, and I'm not sure why.  I don't think it's a routing error in the test, because tests bypass the routing engine anyway.  I think it's an error in the index view, because it probably contains links to messages with improperly formatted urls.  If you have a link to a message, for instance, and you have a @profile object, then you'll need to call them like this:
<%= link_to message.name, profile_message_path(@profile, @message) %>

However, if you're using non-nested paths like message_path(@message), it will fail because there are no non-nested message routes.
Comment: this is my rule for users/messages:

  map.resources :users,
                :as => :profiles,
                :controller => :profiles,
                :has_many   => [ :messages ]

should it still work? because it isn't
Comment: thanks for the help! I added the full routes.rb. Is it possible that there is a route conflict with messages? because users and the me controller have it?
Comment: it still not working here. But i've noticed that every functional test get a routing error. I've no idea why.
Answer: It was a problem in my "journey" gem. They made routes more stricter in journey 1.0.4 which only show up on "test" environment. It is good for "developement" and "production".
** Ensure you are using exactly the same parameters as declared in routes **
Either add:
get :index, :locale => "en"

or in your Gemfile update:
gem 'journey', '1.0.3'

The second solution is a workaround for time being. Ideally you should be testing your routes will all exact params. Journey 1.0.4 is a lot stricter

Rails functional tests, routing error


Sudoku mathematically, MILP?

Question: My homework contains a word (freely-translated) "target-function" that I should generate somehow for 9x9 sudoku solver with some MILP problem. But I am bit lost what they mean. I have sofar described the problem as below. Is this the target function for MILP problem (apparently meaning mixed integer linear problem or something like that)?
\begin{cases}
\forall r\in {1,2,...,9}\sum_{c=1}^{9}V_{rc}=45 \\
\forall c\in{1,2,...,9}\sum_{r=1}^{9}V_{rc}=45 \\
\forall r\in{1,2,...,9}\forall n\in {1,2,...,9} \sum_{c=1}^{9}x_{rcn} = 1 \\
\forall c\in{1,2,...,9}\forall n\in{1,2,...,9}\sum_{r=1}^{9}x_{rcn} = 1 \
\end{cases}
where $x_{rcn}$ is a binary function with $r$ for row,  $c$ for column and $n$ for number between 1 and 9. $V_{rc}$ means the number $n$ in the position $(r,c)$. When I look at wikipedia about the mathematics of sudoku here, I am worried that my way of formulating the problem is not right at all. When I google for MILP, I do not get descriptive hits -- what does this MILP function thing mean?
I have not yet understood how I can formulate the above messy formulates more elegantly, perhaps with the group tabels...sorry I am bit puzzled with the tables in the wikipedia article.
Comment: FYI - this is not about *Moms I'd like to Punch*...
Comment: @hhh: if you don't know «what the purpose of the target function» is while doing this kind of homework, you should *really* get your notes/textbook and take the time read them carefully and at leisure, and/or ask your instructor... You are doing homework about MILPs and you do not even know what the letters mean: even asking here infinitely many questions is nt the way to learn the material!
Answer: The linear program can be formulated just using the binary 0/1-variables
$x_{rcn}$ with the meaning $x_{rcn} = 1$ if and only if there is the number $n$ in the
cell $rc$, i.e in the row $r$ and the column $c$.
There are $9*9*9 = 729$ variables $x_{rcn}$.
The constaints are:
(1) There is exactly one number in a cell:
$$x_{rc1} + x_{rc2} + ... + x_{rc9} = 1; r, c = 1, .., 9$$
(2) The number $n$ appears exactly once in each column and each row:
$$x_{r1n} + x_{r2n} + ... + x_{r9n} = 1; r, n = 1, .., 9$$
$$x_{1cn} + x_{2cn} + ... + x_{9cn} = 1; c, n = 1, .., 9$$
(3) The number $n$ appears exactly once in each of the nine 3x3 boxes:
$$x_{11n} + x_{12n} + ... + x_{33n} = 1; n = 1, .., 9$$
$$...$$
$$x_{77n} + x_{78n} + ... + x_{99n} = 1; n = 1, .., 9$$
The number of constrains is:
(1) 81,
(2) 81 + 81 = 162,
(3) 81,
totally 324.
Additional constrains are given by predefined positions. For example if
there is the number 3 in the position (2, 5), then $x_{253} = 1$.
There is no explicit target function. If a target function is needed,
then any linear function of the free binary variables can be taken, e.g. $x_{111} = min/max$.
Comment: what is the purpose of the target function?
Comment: Target (goal) function is needed when your solution method requires a function to be minimized or maximized.

Sudoku mathematically, MILP?


How to play like Nirvana

Question: As you may have understood by my username I'm a fan of Nirvana, I've been trying to sound like Nirvana with my amp but still haven't got the sound I want with the settings that I've tried... Any ideas on what i should do?
Comment: https://www.youtube.com/watch?v=3B8N2E3W5v8
Answer: Bear in mind that the albums are often double tracked and or have things like stereo separation effects that alter the sound from the way the band actually plays a song live. Also, microphones play a bigger role in recorded tone than average people credit.
Cobain was not known for a ton of effects: clearly he used tremolo and phaser/flanger effects on albums but in the following photo, it looks like he is only using boss "DS-2 turbo distortion" and a "Sansamp classic" amp simulator pedal. Pretty simple. It is possible some effects are in a different loop off camera.
Many guitarists simply use a chain of gain. I myself use a compressor, a special wah never off, and then a tube screamer fed into an amp that is up just enough to barely break when the pedals are off.
Comment: Tremolo? Where?
Comment: I also doubt he used a phaser.
Comment: Tremolo - Fender-style vibrato effect on amp. Chorus, phaser, flange and tremolo can all sound very similar especially when used together. Not really important, as many photos only show a very small number of effects. I think in a photo from a well-known live show (easily seen on youtube), there he has a boss bs-2 and a Rat both taped down to the floor, but not sure there was anything else in use. Note I also suggest that the sound engineers may have been responsible for some of the effects in the studio.
Comment: The point about recordings sounding different than live is very true. My point was I doubt he ever used tremolo or phaser. Chorus, yes.

How to play like Nirvana


How to iterate over fields (names + types) of a tuple/object in a macro?

Question: I want to write macro which has to perform some logic based on the fields of a named tuple or object. I assume this is best accomplished by passing the tuple/object as a typed parameter to the macro.
The question is, how can I iterate over the fields of a typed parameter in general? I'm basically looking for the equivalent of fieldPairs for macros, i.e., instead of taking a concrete tuple/object it would have to operate on NimNode and also return the field names/types as such (for further AST generation).
Answer: I have found a solution to the problem, which seems to work fine, but I'm not sure if there are better alternatives. The solution is based on using getTypeImpl on the typed parameter. To see how it works it helps to look at the output of t.getTypeImpl.treeRepr for a simple tuple and object.

Tuple: The type impl AST of for instance (x: 0, y: 1, name: "") would look something like this:
TupleTy
  IdentDefs
    Sym "x"
    Sym "int"
    Empty
  IdentDefs
    Sym "y"
    Sym "int"
    Empty
  IdentDefs
    Sym "name"
    Sym "string"
    Empty

Note: the typeKind of getTypeImpl is ntyTuple
Object: The type impl AST of an object with identical structure would be:
ObjectTy
  Empty
  Empty
  RecList
    IdentDefs
      Sym "x"
      Sym "int"
      Empty
    IdentDefs
      Sym "y"
      Sym "int"
      Empty
    IdentDefs
      Sym "name"
      Sym "string"
      Empty

Note: the typeKind of getTypeImpl is ntyObject

This shows that the information we are looking for is available in the IdentDefs. We only have to make sure to handle tuples and objects appropriately: For a tuple the IdentDefs are the direct children of the NimNode, whereas for an object the IdentDefs are stored in the child on index 2 (the child on index 0 contains pragma information, child on index 1 is information on the parent).
Overall a macro could look like (with some debug output added for illustration):
macro iterateFields*(t: typed): untyped =
  echo "--------------------------------"

  # check type of t
  var tTypeImpl = t.getTypeImpl
  echo tTypeImpl.len
  echo tTypeImpl.kind
  echo tTypeImpl.typeKind
  echo tTypeImpl.treeRepr

  case tTypeImpl.typeKind:
  of ntyTuple:
    # For a tuple the IdentDefs are top level, no need to descent
    discard
  of ntyObject:
    # For an object we have to descent to the nnkRecList
    tTypeImpl = tTypeImpl[2]
  else:
    error "Not a tuple or object"

  # iterate over fields
  for child in tTypeImpl.children:
    if child.kind == nnkIdentDefs:
      let field = child[0] # first child of IdentDef is a Sym corresponding to field name
      let ftype = child[1] # second child is type
      echo "Iterating field: " & $field & " -> " & $ftype
    else:
      echo "Unexpected kind: " & child.kind.repr
      # Note that this can happen for an object with a case
      # fields, which would give a child of type nnkRecCase.
      # How to handle them depends on the use case.

# small test
type
  TestObj = object
    x: int
    y: int
    name: string

let t = (x: 0, y: 1, name: "")
let o = TestObj(x: 0, y: 1, name: "")

iterateFields(t)
iterateFields(o)


How to fix AJAX HTTP request which was terminated abnormally?

Question: I keep getting this AJAX error message every time when I add taxonomy terms or open view pages with AJAX, to name a few.
<code>An AJAX HTTP request terminated abnormally.
Debugging information follows.
Path: /system/ajax
StatusText: error
ResponseText:
ReadyState: 4
</code>
Any suggestions to how I can resolve the problem? I'm using the dev desktop.
Comment: For Drupal 7 you may try to apply the following core patch: [D7-fix_autocomplete_terminated_error-1232416-179-do-not-test.patch](https://www.drupal.org/node/1232416#comment-8748879).
Comment: Do you frequently update your modules? Its a relative common problem, which will get fixed soon, if its a popular module you are using. However, you should probably try to locate the problem a bit further!
Answer: This issue sounds like yours: Drupal alerts "An AJAX HTTP request terminated abnormally" during normal site operation, confusing site visitors/editors
Answer: For Drupal 7 you may try to apply the following core patch: D7-fix_autocomplete_terminated_error-1232416-179-do-not-test.patch which goes like:
<code>--- a/misc/ajax.js
+++ b/misc/ajax.js
@@ -448,7 +448,10 @@ Drupal.ajax.prototype.getEffect = function (response) {
  * Handler for the form redirection error.
  */
 Drupal.ajax.prototype.error = function (response, uri) {
-  alert(Drupal.ajaxError(response, uri));
+  // Fix for autocomplete terminated error.
+  if (response.status != 0) {
+    alert(Drupal.ajaxError(response, uri));
+  }
   // Remove the progress element.
   if (this.progress.element) {
     $(this.progress.element).remove();
diff --git a/misc/autocomplete.js b/misc/autocomplete.js
index 8f7ac60..980c1ca 100644
--- a/misc/autocomplete.js
+++ b/misc/autocomplete.js
@@ -306,7 +306,10 @@ Drupal.ACDB.prototype.search = function (searchString) {
         }
       },
       error: function (xmlhttp) {
-        alert(Drupal.ajaxError(xmlhttp, db.uri));
+        // Fix for autocomplete terminated error.
+        if (xmlhttp.status != 0) {
+          alert(Drupal.ajaxError(xmlhttp, db.uri));
+        }
       }
     });
   }, this.delay);
</code>
Answer: I have also been seeing this lately. Mine popped up immediately and did not have the turning indicator at all.
I have come to a post at Required autocomplete reference field gives "ReadyState: 0" ajax error #5 @jenlampton gave ma a Clue as to my problem. Although I did use http and not https, I used a link without the www as referenced in my settings.php configuration.
Adding www in my browser made the error disappear.
In conclusion, check your settings.php configuration where
$base_url = 'http://www.example.com"
if you have https use https, if you have www.example.com do not go to example.com.


After re-stringing acoustic guitar my action feels way off.

Question: I recently bought an acoustic guitar, and it had this beautiful action on it. I've restrung it recently (same gauge strings) and now there's a very noticeable difference in playability. Is this normal, am I going to have to get the action adjusted already?
Comment: Did you use the same gauge? If not then it can change the tension on the neck. You should try and use the same type of strings every time to avoid having to get a setup. (I tend to buy in bulk for a year or so of the exact same kinda and I use an electric too)
Answer: Usually when you restring an acoustic guitar, it will feel, and play, slightly different for a while. New strings that are fresh on the guitar tend to feel a little firmer and heavier to the fingers, but as time goes by they will start to feel a little less tense as they begin to stretch.
Are you absolutely sure that they are the same gauge strings? The reason I ask is because if they were heavier than the strings the guitar came with, the increase in tension (heavier strings cause more tension on the neck) could cause the neck to bend slightly inwards, causing an 'up bow,' and therefore increasing the gap between the fretboard and the strings, giving a higher action. It would be a very small change, but still noticeable to you as you play it.

There are other things that could have caused a change in feel. If you started to store the guitar somewhere different to where you were before, a change in temperature or air moisture could cause the guitar to expand or contract slightly, potentially affecting the action. I do however suspect that this is not the case as this happens over a longer period of time.
The bridge piece may have fallen out of it's correct seating when you changed the strings. It may have risen up or down between the old strings being taken off and the new ones put on. This would have a noticeable effect on the action.

If the guitar however sounds different after a restring, that is, sounds out of tune (even when tuned according to a tuner) you probably have intonation problems. See this previous question about acoustic guitar intonation adjustments.
I recommend to most people that after they purchase a guitar they ask the seller to set it up for them. Many guitar shops offer a free set-up for each guitar purchase, but sadly if you are unlucky then yours may not. The question and answer I linked to outlines roughly what the technician will do. A professional set up will ensure that the instrument is in the best condition possible for playing.
Hope this helps.


Error message when using between() function with variable names

Question: When I run the following code section, I am getting the error message copied below.
all_data_db2 <- all_data_db2 %>%
  filter (!is.na(result)) %>%
  mutate(OOS=
    ifelse(!is.na(lsl) & !is.na(usl) & between(result,lsl,usl), "In Spec",
    ifelse (is.na(lsl) & !is.na(usl) & result <usl, "In Spec",
    ifelse (!is.na(lsl) & is.na(usl) & result > lsl, "In Spec",
    ifelse(is.na(lsl) & is.na(usl), "No Spec Limits",
    "OOS")))))

produces error message:
Show in New WindowClear OutputExpand/Collapse Output
Error in UseMethod("rename") : no applicable method for 'rename' applied to an object of class "character"
Show in New WindowClear OutputExpand/Collapse Output
Error: Problem with mutate() column OOS. i OOS = ifelse(...). x left must be length 1 i The error occurred in group 1: code_date = "2021-12-29". Run rlang::last_error() to see where the error occurred.
However, when I enter values for lsl and usl in the between function only as shown below, the code runs just fine:
all_data_db2 <- all_data_db2 %>%
  filter (!is.na(result)) %>%
  mutate(OOS=
    ifelse(!is.na(lsl) & !is.na(usl) & between(result,200,2000), "In Spec",
    ifelse (is.na(lsl) & !is.na(usl) & result <usl, "In Spec",
    ifelse (!is.na(lsl) & is.na(usl) & result > lsl, "In Spec",
    ifelse(is.na(lsl) & is.na(usl), "No Spec Limits",
    "OOS")))))

As shown above, I am using lsl and usl variable names in other parts of this code section without an issue, so very confused why they aren't working with the between() function. I also tried "as.numeric" in the between function (such as: between (result, as.numeric(lsl), as.numeric(usl)) and still got the same error message as above.
Any ideas or suggestions?
Comment: `between` is not vectorized for boundary values (left and right) i.e. it takes only a single value.  You may need to use comparison operators (`>`, `<`) if we want to check for each value of the column
Comment: Sorry, i copied multiple error messages by mistake - the error message I am getting is: Problem with mutate() column OOS. i OOS = ifelse(...). x left must be length 1 i The error occurred in group 1: code_date = "2021-12-29". Run rlang::last_error() to see where the error occurred.
Answer: The reason is that according to ?between, the boundary values (left, right) are not vectorized

left, right - Boundary values (must be scalars).

The second piece of code works only because it is a single value - scalar and not the whole column.  An option is to change the between to comparison operators (>, <)
library(dplyr)
all_data_db2 <- all_data_db2 %>%
  filter (!is.na(result)) %>%
  mutate(OOS=
    ifelse(!is.na(lsl) & !is.na(usl) & result > lsl & result <usl, "In Spec",
    ifelse (is.na(lsl) & !is.na(usl) & result <usl, "In Spec",
    ifelse (!is.na(lsl) & is.na(usl) & result > lsl, "In Spec",
    ifelse(is.na(lsl) & is.na(usl), "No Spec Limits",
    "OOS")))))

Instead of a nested ifelse, may use case_when
all_data_db2 <- all_data_db2 %>%
  filter (!is.na(result)) %>%
  mutate(OOS=
    case_when(!is.na(lsl) & !is.na(usl) & result > lsl &
         result < usl ~  "In Spec",
    is.na(lsl) & !is.na(usl) & result <usl ~  "In Spec",
    !is.na(lsl) & is.na(usl) & result > lsl ~  "In Spec",
    is.na(lsl) & is.na(usl) ~  "No Spec Limits",
    TRUE ~ "OOS"))

Comment: great, thanks. That solved my problem.


Contrapositive proof of prime ideal of semirings

Question: I'm starting to study about the Ternary A-semirings and studying the paper of Daddi and Pawar entitle: Ideal Theory in Commutative Ternary A-semirings, which was published in Int. Math. Forum vol. 7 (2012) no. 42, pp. 2085-2091. In Theorem 3.3, they show that if $P$ is a maximal ideal with respect to $T$, then $P$ is prime. I understand that they will be prove by contrapositive proof of the definition of prime, which is likely to assume that $A\nsubseteq P, B\nsubseteq P$ and $C\nsubseteq P$. I do not understand why you assume that $P\subset A, P\subset B$ and $P\subset C$.
Please explain the reason for me to understand. Thank you very much.
Comment: @WimC: Thanks for the correction.
Answer: They are using a stronger (unstated, as far as I can see) result: $P$ is not prime if, and only if, there exists three ideals $A$, $B$ and $C$ of which $P$ is a strict subset, such that $[ABC]\subseteq P$.
So to prove that $P$ is prime, they show that for all ideals $A$, $B$ and $C$ that strictly contain $P$, $[ABC]\not\subseteq P$. I haven't proven that the stronger result holds, but it's true in usual (not ternary) rings, and it probably holds as well in the case of ternary rings.
Comment: Thank you for your answer.


What are some common jazz chord progressions?

Question: What are some common progressions for Jazz improvisation?
I have been playing with ii-V-I and variations -- what are some others? Is the answer here just to really dig through standards until I've 'internalised' this stuff a little better?
Comment: This question shouldn't have been closed.  Voted to reopen.  Perhaps could do with an edit to remove the "Is the answer.." bit, but the op is clearly asking for common jazz chord progressions which are not variations of a ii-v-i.
Comment: I think this is an acceptable question; if we can have questions about a composer's or era's style, why can't we have questions about specific chord progressions that are common in a genre. An answer could talk about 12-bar blues, rhythm changes (and variants), other contrafacts, alternative turnarounds, etc. Seems like a decent question to me (at least as it currently stands in its edited form).
Comment: @Ben - "improvising" - The question has some substance, the rationale would be that the op knows ii-v-i progressions and variations thereof, and would like to progress from that and add something more to his playable repertoire.  The goal obviously isn't that apparent and could be worded much better, but its there nonetheless.   This question doesn't qualify as a "blatantly off-topic question", therefore it shouldn't have been closed, at least not without some flags or general community disagreement.
Comment: @Ben one of the reasons for closing is a word you misread?  Nice job.
Comment: @Ben - example of a similar and inherently answerable question.  http://music.stackexchange.com/q/1024/149
Comment: @Matthew - I never said it was a good question, however, I do think its an acceptable one.  The op is not asking for a list of chords, he is asking for Jazz chord progressions which are not variations of ii-v-i.  This is an answerable question, that some people would find useful. Also is there not supposed to be at least some community consensus before 'borderline' questions are just closed, IE: a flag or two on the question?
Comment: @DRL Why is asking for a list of chords a good question?
Comment: @DRL: The question has no real substance and doesn't provide any rationale for asking – a better question would have a specific goal in mind instead of asking for a list of progressions. (What does "improving jazz" even mean?)
Comment: @jae: Without my (admittedly idiotic) misread, the question is merely a bad unanswerable question instead of a confusing one.
Answer: There are, of course, an enormous variety of chord progressions used in jazz.  That said, here are three you should know:
12-bar Blues
The basic 12-bar blues as played in jazz (not as played in blues) usually goes something like:

I-IV-I-I-IV-IV-I-vi-ii-V-I-turnaround

In blues, all these chords would be dominant sevenths.  Jazz players, however, frequently play their diatonic versions.
Common variations include replacing the chords in a given bar with a ii-V cadence that resolves to the chord in the next bar.  For example, you could replace the I in bar 4 with a ii-V that resolves to the IV in bar 5.
Rhythm Changes
Rhythm Changes is a progression based around the chords of the popular Gershwin tune "I Got Rhythm".  It's in the usual 32-bar AABA format, and the traditional key is Bb:

A section: BbM7 Gm7 | Cm7 F7 | BbM7 Gm7 | Cm7 F7 | Fm7 Bb7 | EbM7 Edim | Cm7 F7 | Bb7 turnaround |
B section: D7 D7 | D7 D7 | G7 G7 | G7 G7 | C7 C7 | C7 C7 | F7 F7 | F7 F7 |

These changes are so popular among jazz musicians in large part because they offer myriad opportunities for variation.  You can substitute to your heart's content here and find different ways to navigate the same basic structure.  Here are some common substitution ideas.
Coltrane Changes
Like a challenge?  Try this on for size.  I recommend reading this Wikipedia article, which does a better job of explaining the ideas behind the so-called "Coltrane Changes" (as most famously used in "Giant Steps") than I could do.
Answer: If it is not yet part of your "II-V-I" variations, you want to learn about the tritonic substitution concept, which consists in changing a V7 chord in a progression with a bII7 chord. The interval between V and bII is 3 tones == 1 tritone == diminished 5th == augmented 4th. The progressing IIm7 -> bII7 -> IM7 adds some nice chromaticity opportunities.
Try practising the minor II-V-I (II-7b5, V7 with b9, #9 #11, b13, Im (with nice extensions such as m69, mM7).
Bebop sound with IVm7 bVII7 IM7.
Answer: I doubt you're talking about Traditional Jazz (before Swing, Big Band, and other more modern styles), and I know it's kinda late for answering this, but I thought I should add a very typical occurrence in that style.
I - III7 - VI7 - II7 - V7 - I

One example is the Basin Street Blues which for Bb instruments goes similarly to this:
C - E7 - A7 * - D7 - G7 - C

(* Yes, the A7 briefly goes to Bb7 and then back, but I'm ignoring this for simplicity's sake).
Basically, starting with the second chord, every chord is the V7 of the next chord until you're back at I (e.g. E7 is V7 for A, etc.).
It can make your life pretty interesting when you're playing in C and all of a sudden encounter a chord with 4 #'s (one of which is ignored for  the 7).
This exists in a number of variations where you go I - III7 or I - VI7 or I - II7 and then continues from there, and this type of sequence may be inserted in various locations in the chord structure, but once you are in one of these sequences, they nearly always follow that same principle back to the I.
Comment: interesting - I hadn't seen this progression structure
Comment: I don't know where the guitar tabs suddenly came from (I didn't enter those)... Anyway, I just spent a week down in New Orleans for a Traditional Jazz Camp and must have played about 50 different songs, and at least 15-20 of them probably have one variation or another of this progression with at least the last three chords present (II7 - V7 - C). For this version, there's also the variation of getting to the II from the IV, usually later in the chord structure. Also a lot of diminished chords in Traditional Jazz.
Comment: Found a link with more examples for this structure: http://playing-traditional-jazz.blogspot.com/2013/01/the-georgia-chord-progression.html - the song All Of Me (some of you may actually know this one) has a minor chord in there, but the principle is the same.


Calculating a Fisher expected information

Question: Let $X_1,..., X_n$ be a random sample from a distribution with probability density function $f(x;\theta) = (1/2\theta) exp(-|x|/\theta)$ for $-\infty<x<\infty$.
(a) Find the maximum likelihood estimator of $\theta$ and calculate the Fisher (expected) information in the sample.
I've calculated the MLE to be $\sum |X_i|/n$ and I know the definition of Fisher expectation, but I'm getting really stuck with calculating it. I think the $|X_i|$ terms are throwing me off.
Any help in doing this problem would be much appreciated!
Thanks
Answer: Recall that $$I(\theta)=-\mathbb{E}\left[\frac{\partial^2}{\partial \theta^2}l(X\,| \,\theta)\right]\,$$
under certain regularity conditions (that apply here), where $I$ is the Fisher information and $l$ is the log-likelihood function of $X$. The log-likelihood function in this case is given by $$\begin{align} l(X\,|\,\theta) &=\text{log}f(X\,|\,\theta) \\&=\log\left(\frac{1}{2\theta}\text{exp}\left(-\frac{|X|}{\theta}\right)\right) \\ &= -\frac{|X|}{\theta} - \text{log}(2\theta)\,\,. \end{align}$$
It follows that $$\frac{\partial}{\partial \theta}l(X \,|\,\theta) = \frac{|X|}{\theta^2}-\frac{1}{\theta} \implies \frac{\partial^2}{\partial \theta^2}l(X \,|\,\theta) = -\frac{2|X|}{\theta^3}+\frac{1}{\theta^2}\,.$$
So, we have
$$I(\theta)=-\mathbb{E}\left[-\frac{2|X|}{\theta^3}+\frac{1}{\theta^2}\right]=\mathbb{E}\left[\frac{2|X|}{\theta^3}-\frac{1}{\theta^2}\right]=\frac{2}{\theta^3}\mathbb{E}(\,|X|\,)-\frac{1}{\theta^2}\,.$$
It remains to compute the expectation of $|X|$. To this end, I will set up the integral. By definition of expected value for transformations of continuous random variables, we have
$$\mathbb{E}(\,|X|\,)=\int_{-\infty}^{\infty}|x|\,f(x \,|\, \theta)\,\text{d}x=\int_{-\infty}^{\infty}\frac{|x|}{2\theta}\text{exp}\left(-\frac{|x|}{\theta}\right)\,\text{d}x = \theta\,.$$
Note: to compute the integral, alter its form by taking advantage of the fact that $|x|$ is symmetric (and, you can also decompose the integral based on cases for $|x|$).
Thus, the Fisher information is $$I(\theta)= \frac{2}{\theta^3}\mathbb{E}(\,|X|\,)-\frac{1}{\theta^2} = \frac{2}{\theta^2}-\frac{1}{\theta^2}=\frac{1}{\theta^2}$$
For a sample $X_1,X_2,...,X_n$ of size $n$, the Fisher information is then
$$I(\theta \,|\,n)=nI(\theta)=\frac{n}{\theta^2}\,.$$
Therefore, by the Cramér–Rao inequality, the variance of any unbiased estimator $\hat{\theta}$ of $\theta$ is bounded by the reciprocal of the Fisher information (this includes the MLE that you have computed, which achieves this lower bound, and is said to be an efficient estimator). In other words, $$\text{Var}(\hat{\theta}) \geq \frac{1}{nI(\theta)} = \frac{\theta^2}{n}\,\,.$$
Comment: but how do I differentiate $|x|$?
Comment: Simplify the integral using a symmetric argument, so that you won't have to...but for future reference, look up the signum function $\text{sgn}(x)$.
Comment: The substitution I mentioned is one way to compute the integral, but there are other ways...the symmetry of $|x|$ yields the best approach. @Brian
Comment: @Brian, in regards to making the integral manageable: what is $|x|$ for $x \geq 0$ and for $x < 0$? You can split the integral into these two cases and then apply the substitution and integration by parts.
Comment: thanks, I understand now. Looked up $sgn(x)$ and see why it works now!
Comment: No problem whatsoever! Happy to help! @Brian
Comment: @Brian note that I computed the information for a single sample of this distribution with parameter $\theta$, but you can generalize to a sample of size $n$ by inserting a sum $\sum_{i=1}^{n}$ before each $|X|$ and replacing each of these with $|X_i|$...however the calculation still reduces to finding the expectation of the $|X_i|$, which is what I have outlined.
Comment: yes, thanks, my problem was for a sample size of n but did generalise it
Answer: To find MLE of $\theta$ we take likelihood function as $$L(x,\theta) = \frac{1}{(2\theta)^n} \exp \left( \frac{-\sum |x_i|}{\theta} \right).$$  Next we take its $\log$ as $$l(x,\theta) = - n \log 2 \theta - \frac{\sum_{i=1}^n |x_i|}{\theta}.$$  Differentiate w.r.t $\theta$ and equate to zero.  $$\frac{\partial l (x,\theta)}{\partial \theta} = \frac{-n}{\theta} + \frac{\sum |x_i|}{\theta^2} = 0$$ yields $$-n \theta + \sum |x_i| = 0$$ hence MLE is $$\theta = \frac{\sum_{i=1}^n |x_i|}{n}.$$
The Fisher information is given as $$I(\theta) = -E \left[ \frac{\partial^2 l(\theta)}{\partial \theta^2} \right]$$ i.e., expected value of the second derivative of the log likelihood $l(\theta)$.
$$\frac{\partial^2 l(\theta)}{\partial \theta^2} = \frac{n}{\theta^2} - \frac{ 2 \sum_{i=1}^n | x_i|}{\theta^3}$$ Taking expectation we have $$I(\theta) = - E \left[ \frac{\partial^2 l(\theta)}{\partial \theta^2} \right] = - \left[ \frac{n}{\theta^2} - \frac{2n\theta}{\theta^3} \right] = \frac{n}{\theta^2}.$$

Original images: one, two.
Comment: No I am not claiming E(sigma[x])=n theta. I have just edited my answer to make my point clear and that is sigma x(i)=n theta.
Comment: @afedder I had not taken expectation of the first partial derivative of theta w.r.t. "x". I had equated it to zero.
Comment: @ afedder Do check my answer.
Comment: You are correct, but there is no justification of the equalities after "taking the expectation we have", and hence, it is unhelpful to the OP.
Comment: I have attempted to typeset your images.  Please check to ensure that no errors have been introduced.
Comment: @ArthurFischer, thanks for typesetting, no error have been introduced.


Como remover popup ao abrir detalhes do produto?

Question: Estou utilizando um tema, e este tema ao clicar no produto na pagina principal abre um popup com os detalhes na página.
Gostaria de que ao clica em "visualizar" fosse direcionado para a página do produto ou então remover o sinal de "+" do preço do combobox (na página do produto não aparece o sinal de "+" pois consegui tirar, porém no popup aparece).
Magento versão. 184.108.40.206
Site para visualização
Comment: Problema com o sinal de "+" foi resolvido. Alterado arquivo configurable.js comentado as as linhas no código a seguir: [link] (https://bitbucket.org/snippets/aniix_store/5y8jd)
Comment: para o popup acredito que seja este código [link] (https://bitbucket.org/snippets/aniix_store/eky4n) mas não sei como alterar para apenas direcionar para a página do produto.
Comment: Diogo  se resolveu seu problema, marque a resposta como a solução. Tenha um bom dia! :)
Answer: Nunca trabalhei com Magento, mas analisando seu código da pra simplesmente fazer um "truque" com CSS.
.item a.product-image {
   display: block; /* adicionei esse */
   z-index: 1; /* adicionei esse */
   position: relative; /* adicionei esse */
   outline: medium none;
}

Basicamente ele faz o link sobrepor a div de visualizar popup.
Pra remover o botão vá na linha 4040 do style.css, na classe .item .product-hover-box e adicione:
display: none;

Faça o mesmo na linha 7943 do style.css, na classe .mt-actions e adicione novamente:
display: none;

Comment: Funcionou. Muito obrigado, passei um tempo procurando como fazer estas alterações. Não fazia ideia de como remover aquele pop-up, nem imaginava que com css resolveria meu problema.
Comment: Fiz a alteração no style.css mas nao adiantou, talvez eu tenha alterado o arquivo css errado.
Comment: @DiogoGumeiro As vezes está em cache o arquivo, dê um CTRL + F5 e verifique se essa `class` está corretamente no componente.
Comment: Notei que agora consigo clicar na foto e redirecionar para a página do produto. Só preciso ver de remover estes botões de visualizar e comprar.
Comment: @DiogoGumeiro Atualizei a resposta com outras 2 alterações pra vc fazer, da uma olhada :)


It's possible to develop for .net 2.x in Visual Studio 2008?

Question: It's possible to develop only for .net 2.0 running Visual Studio 2008?
Thanks!
Answer: Check out this article - http://weblogs.asp.net/scottgu/archive/2007/06/20/vs-2008-multi-targeting-support.aspx
You might be able to ditch 2005 now...
I still use 2.0 projects in VS 2008 on a weekly basis and it works perfectly.
Comment: Does this mean that I could work seamlessly with other devs on a 2.0/2005 project, while running VS2008? I really don't have to upgrade the project file? And VS won't encourage me to use language features not available in 2.0sp1? Am I understanding this correctly?
Comment: They would have to use 2008 too - because it will change for example, the web.config...
Comment: @Michael - no, it upgrades the project file when you open it, so 05/08 aren't compatible in that sense.
Comment: Everyone could use 2008 couldn't they?
Comment: It'd be an easy sell if we could upgrade one at a time vs. 25-50 people all at once. Alas.
Comment: You can have two solutions for the same project, one for 2008 and one for 2005. The only difference as far as I can remember is the version of Visual Studio which goes from 9 to 10 I think.
Answer: Yes, by setting the project property - but be aware that you're really targeting .NET 2.0 SP1 (which is required for .NET 3.5, effectively).
That means if you start using DateTimeOffset (introduced in .NET 2.0SP1) the compiler will think everything is fine and dandy - but if you deploy the app on a vanilla .NET 2.0 box, it won't work.
Comment: It's possible to use FxCop to help prevent this problem.  But it's by no means a guarantee.  http://davesbox.com/archive/2008/08/25/new-for-visual-studio-2008-sp1-and-fxcop-1-36-multi-targeting-rule.aspx
Answer: Yes, VS2008 supports multi targeting. Check the project properties where you can select the target framweork.
Answer: Go to the Properties page of your project and click the "Application" tab.  Under "Target Framework", you should be able to set the version you want to build for.
Note that the cool thing about VS 2008 is that even if you target to 2.0, you can still use some new language features in C# like "var", etc, even though you aren't using 3.5!
Comment: For more about which features can be used and which can't, see http://csharpindepth.com/Articles/Chapter1/Versions.aspx


Who is the Third Umpire?

Question: For an international cricket match, there should be a TV umpire, a Match referee, and a Reserve umpire along with two umpires on the field,
Who is the third umpire of the above?
What are the responsibilities for each of these umpires?
Answer: The third umpire is none other than the TV umpire. His duties include:

Checking replays and different camera angles to determine if the batsman has made his ground in close run-out decisions that the on-field umpires cannot determine with their naked eye in real-time.
The same procedure is followed for stumping decisions.
When a batsman is dismissed, the third umpire checks if the delivery is legal or a no-ball due to overstepping.
To check a catch is cleanly taken when catches are taken inches off the ground.
To check if the fielder has touched the boundary line when he slides/catches the ball centimeters away from it.

The decision of the third umpire is either relayed over the big-screen or is communicated via a walkie-talkie to the on-field umpires.
The Match Referee has no active role to play during the game. His duties include:

Observe the game from the sidelines and ensure that the ICC code is followed.
Make decisions and fine players for improper conduct after going through the umpires matc report.

The Fourth Umpire/Reserve Umpire is on stand-by in case the on-field umpires or the third umpire are not in a position to perform their duties due to various reasons. His duties include:

Take over as third umpire in case the on-field or the third umpire is not able to do so. The fourth umpire takes over as the third umpire and the third umpire replaces an on-field, if an on-field umpire cannot continue.
Bringing a set of new balls in case of a replacement is required to the ball in play.

Who is the Third Umpire?


BeautifulSoup Iterating through pages but data scraped is duplicating every loop

Question: import requests
from bs4 import BeautifulSoup
import pandas as pd
from pandas import DataFrame
import numpy as np
import time

headers = {"Accept-Language": "en-US,en;q=0.5"}

symbol = []
name = []
asset_class = []

pages = np.arange(1, 5)
list=[]

for page in pages:

    print("https://etfdb.com/screener/#page=" + str(page))  # to watch progress
    page = requests.get("https://etfdb.com/screener/#page=" + str(page),
                        headers=headers)
    time.sleep(3) #in seconds
    soup = BeautifulSoup(page.text, 'html.parser')  # Parse the HTML as a string

    table = soup.find('table', class_ = "table table-bordered table-hover table-striped mm-mobile-table")

    for etf in table.find_all('tbody'):
        rows = etf.find_all('tr')
        for row in rows:
            Cell = row.get_text().rstrip()
            Cell = Cell.replace("\n\n\n", "\n").replace("\n\n", "\n")
            list.append(Cell.splitlines())

df = DataFrame(list, columns=['Empty','Symbol', 'ETF Name',
                        "Previous Closing Price","Total Assets ($MM)",
                        "Avg. Daily Share Volume (3mo)","YTD Price Change","Asset Class"])
del df['Empty']
print(df)
df.to_csv('etfs.csv')'''

I added a sleep timer between pages, but that didn't help. Data is flowing in from all the pages but it is just repeating on every page. I made sure that the page numbers are working correctly in the browser.
Answer: The data is loaded from external source via Javascript. You can use this example how to load the data:
import json
import requests

params = {"page": 1, "only": ["meta", "data"]}
url = "https://etfdb.com/api/screener/"
page = 1

while True:
    print("Getting page {}...".format(page))
    params["page"] = page
    data = requests.post(url, json=params).json()
    # uncomment to see all data:
    # print(json.dumps(data, indent=4))

    if not data["data"]:
        break

    # print some data:
    for d in data["data"]:
        print("{:<60} {}".format(d["mobile_title"], d["price"]))

    page += 1

Prints:
Getting page 1...
SPY - SPDR S&P 500 ETF                                       $408.52
IVV - iShares Core S&P 500 ETF                               $410.01
VTI - Vanguard Total Stock Market ETF                        $212.85
VOO - Vanguard S&P 500 ETF                                   $375.55
QQQ - Invesco QQQ                                            $335.08
VEA - Vanguard FTSE Developed Markets ETF                    $50.44
IEFA - iShares Core MSCI EAFE ETF                            $74.02
AGG - iShares Core U.S. Aggregate Bond ETF                   $114.31
IEMG - iShares Core MSCI Emerging Markets ETF                $65.30
VWO - Vanguard FTSE Emerging Markets ETF                     $52.65
VTV - Vanguard Value ETF                                     $132.85
VUG - Vanguard Growth ETF                                    $270.14
BND - Vanguard Total Bond Market ETF                         $85.06
IJR - iShares Core S&P Small-Cap ETF                         $109.39
IWM - iShares Russell 2000 ETF                               $222.56
IWF - iShares Russell 1000 Growth ETF                        $254.60
IJH - iShares Core S&P Mid-Cap ETF                           $265.11
GLD - SPDR Gold Trust                                        $164.51
VIG - Vanguard Dividend Appreciation ETF                     $149.85
EFA - iShares MSCI EAFE ETF                                  $77.75
IWD - iShares Russell 1000 Value ETF                         $154.08
VO - Vanguard Mid-Cap Index ETF                              $226.92
VB - Vanguard Small Cap ETF                                  $217.44
VXUS - Vanguard Total International Stock ETF                $64.13
VGT - Vanguard Information Technology ETF                    $378.80
Getting page 2...
VCIT - Vanguard Intermediate-Term Corporate Bond ETF         $93.55
BNDX - Vanguard Total International Bond ETF                 $57.28
XLK - Technology Select Sector SPDR Fund                     $140.42
LQD - iShares iBoxx $ Investment Grade Corporate Bond ETF    $131.06
ONEQ - Fidelity NASDAQ Composite Index Track                 $53.72
XLF - Financial Select Sector SPDR Fund                      $34.85

...and so on.


lsmeans vs. differences between lsmeans

Question: I calculated the least-squares means and standard errors for a linear mixed model. I am attempting to plot the lsmeans and standard errors for the combinations of the two factors, but I notice a discrepancy in what is allegedly significant.
> library(lmerTest)

> print(summary(data_dist))
   ID        interface    direction    error_dist
12     : 18   fs     :108   depth :72   Min.   :-0.34375
13     : 18   none   :108   height:72   1st Qu.:-0.13037
14     : 18   RW_none:  0   width :72   Median :-0.04048
15     : 18                             Mean   :-0.03464
16     : 18                             3rd Qu.: 0.06022
17     : 18                             Max.   : 0.36864
(Other):108

> print(summary(model_error_dist))
Linear mixed model fit by REML ['merModLmerTest']
Formula: error_dist ~ direction * interface + (1 | ID) + (1 | ID:direction)
Data: data_master

REML criterion at convergence: -320.9162

Random effects:
Groups       Name        Variance Std.Dev.
ID:direction (Intercept) 0.005864 0.07658
ID           (Intercept) 0.003395 0.05827
Residual                 0.008463 0.09199
Number of obs: 216, groups: ID:direction, 36; ID, 12

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)
(Intercept)                     0.019411   0.031728  35.080000   0.612   0.5446
directionheight                -0.074568   0.038046  31.210000  -1.960   0.0590 .
directionwidth                 -0.058316   0.038046  31.210000  -1.533   0.1354
interfacenone                  -0.037312   0.021683 176.810000  -1.721   0.0870 .
directionheight:interfacenone  -0.008412   0.030664 176.810000  -0.274   0.7841
directionwidth:interfacenone    0.061797   0.030664 176.810000   2.015   0.0454 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr) drctnh drctnw intrfc drctnh:
directnhght -0.600
directnwdth -0.600  0.500
interfacenn -0.342  0.285  0.285
drctnhght:n  0.242 -0.403 -0.201 -0.707
drctnwdth:n  0.242 -0.201 -0.403 -0.707  0.500

> st <- step(model_error_dist)

> print(st)

Random effects:
                   Chi.sq Chi.DF elim.num p.value
(1 | ID)             3.28      1     kept    0.07
(1 | ID:direction)  37.83      1     kept  <1e-07

Fixed effects:
                    Sum Sq Mean Sq NumDF  DenDF F.value elim.num Pr(>F)
direction           0.0446  0.0223     2  22.00  2.6379     kept 0.0940
interface           0.0206  0.0206     1 176.81  2.4308     kept 0.1208
direction:interface 0.0529  0.0265     2 176.81  3.1263     kept 0.0463

Least squares means:
                                 direction interface Estimate Standard Error   DF t-value Lower CI Upper CI p-value
direction  depth                       1.0        NA   0.0008         0.0298 27.4    0.03  -0.0604   0.0619  0.9800
direction  height                      2.0        NA  -0.0780         0.0298 27.4   -2.62  -0.1392  -0.0169  0.0143 *
direction  width                       3.0        NA  -0.0267         0.0298 27.4   -0.89  -0.0878   0.0345  0.3790
interface  fs                           NA       1.0  -0.0249         0.0229 12.8   -1.09  -0.0744   0.0246  0.2971
interface  none                         NA       2.0  -0.0444         0.0229 12.8   -1.94  -0.0939   0.0051  0.0747 .
direction:interface  depth fs          1.0       1.0   0.0194         0.0317 35.1    0.61  -0.0450   0.0838  0.5446
direction:interface  height fs         2.0       1.0  -0.0552         0.0317 35.1   -1.74  -0.1196   0.0092  0.0909 .
direction:interface  width fs          3.0       1.0  -0.0389         0.0317 35.1   -1.23  -0.1033   0.0255  0.2283
direction:interface  depth none        1.0       2.0  -0.0179         0.0317 35.1   -0.56  -0.0823   0.0465  0.5762
direction:interface  height none       2.0       2.0  -0.1009         0.0317 35.1   -3.18  -0.1653  -0.0365  0.0031 **
direction:interface  width none        3.0       2.0  -0.0144         0.0317 35.1   -0.45  -0.0788   0.0500  0.6523
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

 Differences of LSMEANS:
                                             Estimate Standard Error    DF t-value Lower CI Upper CI p-value
direction depth-height                            0.1         0.0348  22.0    2.26   0.0066   0.1510   0.034 *
direction depth-width                             0.0         0.0348  22.0    0.79  -0.0448   0.0996   0.439
direction height-width                           -0.1         0.0348  22.0   -1.47  -0.1236   0.0209   0.154
interface fs-none                                 0.0         0.0125 176.8    1.56  -0.0052   0.0442   0.121
direction:interface  depth fs- height fs          0.1         0.0380  31.2    1.96  -0.0030   0.1521   0.059 .
direction:interface  depth fs- width fs           0.1         0.0380  31.2    1.53  -0.0193   0.1359   0.135
direction:interface  depth fs- depth none         0.0         0.0217 176.8    1.72  -0.0055   0.0801   0.087 .
direction:interface  depth fs- height none        0.1         0.0380  31.2    3.16   0.0427   0.1979   0.004 **
direction:interface  depth fs- width none         0.0         0.0380  31.2    0.89  -0.0437   0.1114   0.381
direction:interface  height fs- width fs          0.0         0.0380  31.2   -0.43  -0.0938   0.0613   0.672
direction:interface  height fs- depth none        0.0         0.0380  31.2   -0.98  -0.1148   0.0403   0.335
direction:interface  height fs- height none       0.0         0.0217 176.8    2.11   0.0029   0.0885   0.036 *
direction:interface  height fs- width none        0.0         0.0380  31.2   -1.07  -0.1183   0.0368   0.292
direction:interface  width fs- depth none         0.0         0.0380  31.2   -0.55  -0.0986   0.0566   0.585
direction:interface  width fs- height none        0.1         0.0380  31.2    1.63  -0.0156   0.1396   0.113
direction:interface  width fs- width none         0.0         0.0217 176.8   -1.13  -0.0673   0.0183   0.260
direction:interface  depth none- height none      0.1         0.0380  31.2    2.18   0.0054   0.1606   0.037 *
direction:interface  depth none- width none       0.0         0.0380  31.2   -0.09  -0.0811   0.0741   0.928
direction:interface  height none- width none     -0.1         0.0380  31.2   -2.27  -0.1640  -0.0089   0.030 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Final model:
lme4::lmer(formula = error_dist ~ direction + interface + (1 |
    ID) + (1 | ID:direction) + direction:interface, data = data_master,
    REML = reml, contrasts = l)

Plotting the lsmeans and standard errors gives me this graph:

From this plot, I can see that my standard error is huge and most comparisons between means will likely be insignificant. However, in the "Differences of LSMEANS" section of the output above, I see four significant differences.
This one I expect, based on the plot above:
direction:interface  depth fs- height none        0.1         0.0380  31.2    3.16   0.0427   0.1979   0.004 **

This one, for example, surprises me:
direction:interface  height fs- height none       0.0         0.0217 176.8    2.11   0.0029   0.0885   0.036 *

It seems I don't actually understand what these outputs are telling me. Can somebody please explain the reason for this discrepancy? Thanks!
Answer: I think that this is due to familywise error rate (FWER).
In multiple comparison tests you need to control FWER, for example by using conservative Bonferroni correction. In lmerTest this is not included (probably will be in the near future), but you may use the p.adjust function in order to create adjusted p-values.
If I am not wrong, in your case you have 15 comparisons of direction:interface. So by using the Bonferroni correction, the adjusted p-value for direction:interface  height fs- height none becomes 0.036*15=0.54
Comment: I had wondered about this. I guess I didn't figure the problem would be this large. I think you might be right though.

lsmeans vs. differences between lsmeans


apt-get trying to connect to unknown IP

Question: Cofession: I am just a noobie with shell and networking so sorry if this is some sort of blunder!
apt-get doesnt seem to work in my shell. It seems to be trying to connect to an IP (10.6.1.57) which doesnt even respond to a ping.
This is what happens when i try apt-update:
0% [Connecting to 10.6.1.57 (10.6.1.57)] [Connecting to 10.6.1.57 (10.6.1.57)]
{This seems to go on for a long time}
Here is my /etc/apt/sources.list file:

# deb cdrom:[Ubuntu 14.04 LTS _Trusty Tahr_ - Release amd64 (20140417)]/ trusty main restricted

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu trusty main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu trusty-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu trusty universe
deb-src http://archive.ubuntu.com/ubuntu trusty universe
deb http://archive.ubuntu.com/ubuntu trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu trusty-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://archive.ubuntu.com/ubuntu trusty multiverse
deb-src http://archive.ubuntu.com/ubuntu trusty multiverse
deb http://archive.ubuntu.com/ubuntu trusty-updates multiverse
deb-src http://archive.ubuntu.com/ubuntu trusty-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse

deb http://archive.ubuntu.com/ubuntu trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu trusty-security universe
deb http://archive.ubuntu.com/ubuntu trusty-security multiverse
deb-src http://archive.ubuntu.com/ubuntu trusty-security multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
deb http://archive.canonical.com/ubuntu trusty partner
deb-src http://archive.canonical.com/ubuntu trusty partner

## Uncomment the following two lines to add software from Ubuntu's
## 'extras' repository.
## This software is not part of Ubuntu, but is offered by third-party
## developers who want to ship their latest software.
deb http://extras.ubuntu.com/ubuntu trusty main
deb-src http://extras.ubuntu.com/ubuntu trusty main
Comment: Do you use a proxy server?
Comment: Your question is better suited to [Super User](http://superuser.com/tour). [Stack Overflow is a question and answer site for professional and enthusiast programmers](http://stackoverflow.com/tour).
Comment: What's the output to `grep -R "10.6.1.57" /etc/apt` ?
Comment: Check if any files in `/etc/apt/sources.list.d` have this IP hard-coded. Or maybe your DNS is giving this IP while resolving some host names.
Comment: Thanks!!! The IP was there hardcoded
Answer: Check this file too: /etc/apt/apt.conf.d/00apt-cacher-proxy maybe there is some proxy as it was in my case.
Answer: Check in /etc/apt/sources.list.d/ for list files containing that ip.
Comment: Nope, grep didnt find any files with the given IP in sources.list.d
Comment: Sorry, got it!! thnkz
Answer: I only changed one thing to get it working. Added "nameserver 18.104.22.168" to /etc/resolve.conf and it worked good
Answer: That can be both custom repository address and APT proxy override configuration. They can be set in several locations under /etc/apt/.
Resolve address 10.6.1.57 into host name (e.g. with host 10.6.1.57) and then search for both IP and host name in the whole /etc/apt/ directory.
grep -r 10.6.1.57 /etc/apt/

In my case it was custom Acquire::http::Proxy setting in /etc/apt/apt.conf.

apt-get trying to connect to unknown IP


How to run a script using SSH

Question: Complete beginner question...
Let's say I have SSH access to my website. And, I have a file:
mywebsite.com/foo/my_script.php
How can I use SSH to run my_script.php?
Answer: <code>ssh firstname.lastname@example.com "/usr/bin/php /path/to/my_script.php"
</code>
Just make sure that the user you use to login is the user you want to run the script as.
Also make sure that you actually have /usr/bin/php installed. Running a PHP script from the command line isn't the same as starting it from say Apache.
Comment: I wasn't able to get it right in one line (like your example). I made it work by entering separate commands. First, login to ssh. Second "ls" and "cd" a few times to get to the file. Finally, run the file with "php my_script.php"
Comment: Perhaps you don't have PHP cli installed as /usr/bin/php. What does "which php", on the server, tell you?
Answer: You can also make the PHP file like this:
<code>#!/usr/bin/php
<?php
    echo 'Hello world';
?>
</code>
and then just type
<code>/path/to/my_script.php
</code>
in SSH and it will work also

How to run a script using SSH


AzureADB2C Client Credentials Grant with Client Assertion as Opposed to Static Secret

Question: I have a requirement to provide API to our consumers. The intention is to secure the API using AzureAD B2C - Client Credential Grant flow.
I have created a custom policy on B2C tenant that provides the access token. Things work fine with the clientId and Secret authentication method.
I now want to secure the OAuth2 conversation further by allowing the client to use the signed client_assertion as opposed to static client secret using their protected key. I have uploaded the public portion of the key into the relevant app registration.
Unfortunately, consuming the /token endpoint with the signed client_assertion results in an error.
REQUEST
https://tenant.b2clogin.com/tenant.onmicrosoft.com/b2c_1a_demo_clientcredentialsflow/oauth2/v2.0/token
grant_type=client_credentials&scope=https%3A%2F%2Fapi%2F.default&client_id=d5339984-e6c7-457a-9ef9-21fb6e3e6c59&client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3Aclient-assertion-type%3Ajwt-bearer&client_assertion=eyJhbGciOiJo
RESPONSE
HTTP/1.1 400 Bad Request
{"error":"invalid_request","error_description":"AADB2C99027: Policy 'B2C_1A_Demo_ClientCredentialsFlow' does not contain a AuthorizationTechnicalProfile with a corresponding ClientAssertionType.\r\nCorrelation ID: 5eb76fa5-c919-4877-a722-0d38408e18c6\r\nTimestamp: 2023-01-19 07:38:25Z\r\n"}
Can someone please tell me if B2C is intended to support client assertions? Metadata JSON on the policy returns only the following two authentication methods:
"token_endpoint_auth_methods_supported": [ "client_secret_post", "client_secret_basic" ]
Is it possible to include private_key_jwt as a supported authentication method using custom policy configuration? Is it possible to configure the AuthorizationTechnicalProfile for the policy with a corresponding ClientAssertionType?
I hope that I have explained the problem well enough.
I have tried various strategies, incluling the use of AAD token endpoint, login.microsoftonline.com with the B2C tenant Id. Using that endpoint, the custom policy on B2C is completely ignored, therefore generating a vanilla token with none of my curated claims.
Comment: Also, https://learn.microsoft.com/en-us/azure/active-directory-b2c/custom-policy-developer-notes#oauth-20-application-authorization-flows says that the Azure AD B2C client credentials flow is currently in preview. Does anyone know when this feature will be available in General Availability mode?
Answer: TLDR: As of June 2022, Azure AD B2C does not support client assertions.
This issue on Github asks for documentation for error number AADB2C99027. In the course of the discussion, a member of the team states

Unfortunately, we decommissioned client_assertion flow because it didn't  follow OIDC spec – So we shouldn't be documenting the error.

From that, I take that there are no plans to support client_assertion flow.


Available Tensions vs. (Non-)Diatonic Non Chord Tones

Question: In, for example, the C major key, the available tensions for the C major seventh chord are 9, 13, because:
Harmony 3 - Berklee, Barrie Nettles, p17:

For diatonic chord progressions, available tensions and other non-chord tones will be diatonic.

Where:
Harmony 3 - Berklee, Barrie Nettles, p15:

Available tensions are non-chord tones which are a whole-step above a chord tone (a major ninth reduced by an octave).

Which means there are other available tensions, just non-diatonic. i.e., for a C major seventh: 9, #11, 13. But the #11 is flat out denied "available tension" status by some sources:
Secondary dominants - aucklandguitarschool.co.nz:

"Available tensions" are non-chord tones that are diatonic and a major 9th above a chord tone.

Anyhow, if I would assume that available tensions do include the non-diatonics, what would I call them as a whole to minimize ambiguity? All available tensions? Chromatic tensions? NCTs that are a major 9th above a chord tone?
And so, just to clarify, would one use a non-diatonic available tension in a non-diatonic chord progression?
Comment: I wouldn't even call some of those *extensions* 'tensions'.
Answer: Definitions are not always that strictly adhered to, and different authors/sources may use slightly different definitions.
It's pretty common usage to define "available tension" just for a certain chord type, without considering a related scale. E.g., a Cmaj7 chord could be used as a I chord in major (where the #11 wouldn't available diatonically), but it could also be used as a IV chord in the key of G major, where the #11 is available. So you could say that in principle, the #11 is an available tension for a major seventh chord. If you want to use it depends on the context. Note that even if the major seventh chord is a I chord, you could add the #11, which would give the chord a lydian flavor.
And concerning the definition mentioning "a major 9th above a chord tone", it's not correct, at least not for dominant seventh chords. E.g., the tension b9 is very common for a dominant seventh chord resolving to a chord a perfect fifth below.


Submitted Revit 2022 model to Forge. It accepted SVF format but returned PDF's

Question: Submitted Revit 2022 model to Forge. Accepted SVF format but returned PDF's
acceptedJobs":{"output":{"formats":{"0":{"type":"svf","views":{"0":"2d","1":"3d"}}}}}}
snippet from manifest:
{"urn":"urn:adsk.viewing:fs.file:dXJuOmFkc2sub2JqZWN0czpvcy5vYmplY3Q6aW5lcnRpYXNhbmRib3hfdGVzdC9pbmVydGlhc2FuZGJveDByMjIuemlw/output/Resource/Sheet/A101 - LEVEL 1 FLOOR PLAN 1420741/pdf/A101 - LEVEL 1 FLOOR PLAN.pdf"
Model works as expected in 2021 format, I used the same model upgraded from 2021 to 2022 and it changes from producing the expected svf to pdf.
Comment: what do you want us to help you with?
please edit your question with what you want to achieve and what you already tried
Answer: UPDATE Feb 2023
Now we have an advanced option to control if the resulting 2D views will be converted as PDFs.
Please, refer to https://aps.autodesk.com/blog/advanced-option-rvtdwg-2d-views-svf2-post-job
...
When you trigger the conversion, 2D sheets for Revit file with 2022 and higher versions are exported as PDF.
2D properties are supported by GET properties.


TSQL function which will raiserror if passed null?

Question: I'm looking to add some code to my TSQL arsenal to defend against performing aggregations in SQL when the data in a column is null. Ideally there would be a SUM_NN (for sum no null), in sql server which would raiserror if any of the values were null.
Since you can't raiserror from a UDF, the only way I could think of doing it looked like this, though I don't like this solution:
CREATE FUNCTION dbo.NULL_TEST_F(@arg FLOAT)
RETURNS FLOAT
AS
BEGIN
    IF(@arg IS NULL)
        SELECT 1/0
    RETURN @arg
END

Note: I think this is stupid solution, but I've gotten burned way too many times when I'm missing data. Also, we're using SQL Server 2005, but I'm open to 2008 and 2012 solutions. Additionally, I'd like to know how other databases deal with this type of issue.
Comment: Why do you need to throw an exception in case of NULL? How does it help aggregates?
Answer: This was my final solution:
CREATE FUNCTION [dbo].[NullTest_F]
(
    @input FLOAT,
    @message VARCHAR(100)
)
RETURNS FLOAT
AS
BEGIN
    DECLARE @test INT
    IF(@input IS NULL)
        SELECT @test = CAST(@message AS INT)
    RETURN @input
END

I can then embed this with a useful error message when running aggregate functions. Example:
CREATE TABLE Data (
    Date DATETIME,
    DataPoint FLOAT
)

INSERT INTO Data (Date, DataPoint) VALUES ('2012-03-01', 4)
INSERT INTO Data (Date, DataPoint) VALUES ('2012-03-02', 6)

SELECT SUM(NullTest_F(DataPoint, 'Missing data at' + CONVERT(VARCHAR(10), Data))
FROM Data

Answer: Maybe this one will help:
https://stackoverflow.com/a/4681815/1371070
You could create a function like suggested in the answer linked above and call it from your aggregate in case @arg is null.
It's still the same strategy overall but It's a better error to throw than divide-by-zero, I guess.


TypeError: unsupported format string passed to list.__format__ | Dicionarios e listas

Question: Ola, estou fazendo meu código aqui até uma parte que acontece esse erro:
  File "path/main.py", line 45, in <module>
    print(f'{j[0]["id"]:<30} {j[1]:^30} {j[0]["GolsTotais"]:>30}')
TypeError: unsupported format string passed to list.__format__

Meu código:
jogadores = list()
gols = list()
listaAux = list()

dic = dict()

aux = 0
somaGols = 0

while True:
    dic['id'] = aux
    dic['nome'] = str(input('Nome do jogador > ')).strip().capitalize()
    dic['Qpartidas'] = int(input(f'Quantas partidas {dic["nome"]} jogou > '))

    for c in range(0, dic['Qpartidas']):

        golsAux = (int(input(f'Quantos gols na {c+1}º partida > ')))

        gols.append(golsAux)
        somaGols += golsAux

    dic['GolsTotais'] = somaGols
    listaAux.append(dic.copy())
    listaAux.append(gols[:])
    jogadores.append(listaAux[:])
    listaAux.clear()
    gols.clear()
    dic.clear()

    continuar = int(input('Deseja continuar? [1] Sim [2] Não > '))

    if continuar == 2:
        break

    aux += 1

print('-='*50)
cont = 0
print(jogadores)
while True:
    print(f'{"id":<30} {"gols":^30} {"total":>30}')
    for j in jogadores:
        print(j)
        print(f'{j[0]["id"]:<30} {j[1]:^30} {j[0]["GolsTotais"]:>30}')

    input()

Answer: Consegui resolver a minha questão, o problema era apenas a formatação:
Antes:
print(f'{j[0]["id"]:<30} {j[1]:^30} {j[0]["GolsTotais"]:>30}')

Depois:
print(f'{str(j[0]["id"]):<15} {str(j[1]):^15} {str(j[0]["GolsTotais"]):>15}')

Não podemos fazer alinhamento em listas, transformando-as em strings conseguimos!
Comment: "*Não podemos fazer alinhamento em listas, transformando-as em strings conseguimos*" - Complementando, esse comportamento é explicado aqui: https://pt.stackoverflow.com/q/448955/112052 - Mas na verdade eu sugiro rever esta estrutura, pois está bem confusa: vc tem uma lista onde cada elemento é outra lista, que por sua vez contém um dicionário e uma outra lista. Uma sugestão para simplificar seria cada dicionário que representa um jogador ter todas as informações nele (inclusive os gols): https://ideone.com/WfbgoI
Comment: Aaah, sim. Eu sei que está bem confusa ela, é que estou fazendo o curso de python e o professor pediu para mim fazer desta maneira como treino.


update with script pre-installed in elasticsearch

Question: I want to use scripts in the way[ https://www.elastic.co/blog/running-groovy-scripts-without-dynamic-scripting ] for security purpose.
I tried this in sense(chrome plugin) and it works well. Wondering how to achieve this in elastic4s.
For example, I want to remove a field from doc, and code like this:
def replaceWithId(alarmId: String, fieldName: String, fieldValue: Map[String, Any]) = {
client.execute {
  update id alarmId in IndexType script """{"file":"removeOperationField"}"""

}}

but it failed.
Comment: Is your file actually called "removeOperationField" or "removeOperationField.groovy" or something ?
Comment: file name is removeOperationField, since its type is groovy we don't have to declare lang.
Comment: java client can do this by `client.prepareUpdate("index", "type", "id")
      .setScript("removeIpField", ScriptService.ScriptType.FILE)
      .get `
Comment: Yep I know you don't need to declare the language, but just wanted to make sure you hadn't given the file a file extension when you saved it.
Comment: full file name is "removeOperationField.groovy", file extension doesn't matter(I tried this).
Answer: You need to set the script type, rather than including it in the json explicitly, eg:
def replaceWithId(alarmId: String) = {
  client.execute {
    update id alarmId in IndexType script("scriptname", ScriptType.File)
  }
}

Comment: After upgraded to elastic4s 1.6.0, I finally saw this function. I was using 1.3.3 where script(string) cannot specify script type.


Mysql 5.1 my.cnf not in /etc/mysql

Question: I installed mysql-server-5.1 on my Ubuntu server using apt-get install mysql-server, but i cant find my.cnf file.
It is not in /etc/mysql.
In /etc/mysql i have only:
conf.d
\-mysqld_safe_syslog.cnf
debian-start
sebian.cnf

I have tried uninstalling/installing mysql multiple times, but still nothing.
Also, when installing mysql, i get messages like this: (i have installed dialog)
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline

Comment: Try to find it: find / -name my.cnf
Answer: EDITED : Sometimes the my.cnf is located in just the /etc directory.
/etc/my.cnf
Answer: If you installed mysql-server, it should have installed mysql-common which includes that file.  I'm guessing that something failed in that installation, disk out of space, interrupted for whatever reason.
apt-get --reinstall install mysql-common

which is a predependency for mysql-server-5.1
If it comes up with errors, try:
dpkg --reconfigure -a

or
apt-get -f install

Answer: You can copy one from /usr/share/doc/mysql-server-5.1/examples/ . Rename it to my.cnf and place in /etc/mysql/my.cnf
and about dialog , set your TERM environment variable.

Mysql 5.1 my.cnf not in /etc/mysql


How can I learn about the values of these cmake variables?

Question: I would like to learn more about cmake and decided to use the cmake -P together with going over its wiki, for instance, this page "CMake Useful Variables".  So, I cut-and-pasted the page content into a file /tmp/t1.cmake on a notebook running Ubuntu 12.04LTS 64bit.
But, upon executing cmake -P /tmp/t1.cmake on this box, I got
cmake -P t1.cmake
-- CMAKE_BINARY_DIR:         /tmp
-- CMAKE_CURRENT_BINARY_DIR: /tmp
-- CMAKE_SOURCE_DIR:         /tmp
-- CMAKE_CURRENT_SOURCE_DIR: /tmp
-- PROJECT_BINARY_DIR:
-- PROJECT_SOURCE_DIR:
-- EXECUTABLE_OUTPUT_PATH:
-- LIBRARY_OUTPUT_PATH:
-- CMAKE_MODULE_PATH:
-- CMAKE_COMMAND: /usr/bin/cmake
-- CMAKE_ROOT: /usr/share/cmake-2.8
-- CMAKE_CURRENT_LIST_FILE: /tmp/t1.cmake
-- CMAKE_CURRENT_LIST_LINE: 49
-- CMAKE_INCLUDE_PATH:
-- CMAKE_LIBRARY_PATH:
-- CMAKE_SYSTEM:
-- CMAKE_SYSTEM_NAME:
-- CMAKE_SYSTEM_VERSION:
-- CMAKE_SYSTEM_PROCESSOR:
-- UNIX: 1
-- WIN32:
-- APPLE:
-- MINGW:
-- CYGWIN:
-- BORLAND:
-- MSVC:
-- MSVC_IDE:
-- MSVC60:
-- MSVC70:
-- MSVC71:
-- MSVC80:
-- CMAKE_COMPILER_2005:
-- CMAKE_SKIP_RULE_DEPENDENCY:
-- CMAKE_SKIP_INSTALL_ALL_DEPENDENCY:
-- CMAKE_SKIP_RPATH:
-- CMAKE_VERBOSE_MAKEFILE:
-- CMAKE_SUPPRESS_REGENERATION:
-- CMAKE_C_FLAGS:
-- CMAKE_CXX_FLAGS:
-- CMAKE_BUILD_TYPE:
-- BUILD_SHARED_LIBS:
-- CMAKE_C_COMPILER:
-- CMAKE_CXX_COMPILER:
-- CMAKE_COMPILER_IS_GNUCC:
-- CMAKE_COMPILER_IS_GNUCXX :
-- CMAKE_AR:
-- CMAKE_RANLIB:

I can understand that some of the variables are not applicable to my system and thus their values should be empty. But how about CMAKE_SYSTEM, CMAKE_SYSTEM_NAME, CMAKE_SYSTEM_VERSION, CMAKE_SYSTEM_PROCESSOR etc?  They should be available, right?  I also tried  some system variables listed in the man page, e.g. CMAKE_HOST_SYSTEM_NAME etc, no go either.  If I don't know of such variables' values, how can I use them properly?
I would like to know what I am missing here.
Comment: For example, the man page of project command only states "Additionally this sets the variables _BINARY_DIR and _SOURCE_DIR to the respective values." and nothing else. I would be greatful if you could share how you learned the IMHO "undocumented" behavior of the project command?
Comment: Yes. Ruslo. What I missed from cmake documentation that I have studied so far is how you seasoned pros learned about such "insights"?  A hint or pointer would be great. I will do the digging.
Answer: Many of these are set by the project command.  Unfortunately, if you add this command, you can't use CMake in scripting mode (i.e. you can't do cmake -P)
So, probably your best bet is to add a CMakeLists.txt alongside your t1.cmake with something like:
cmake_minimum_required(VERSION 2.8)
project(ShowVariables)
include(t1.cmake)

then do:
mkdir /tmp/build
cd /tmp/build
cmake ..

Comment: Unfortunately a lot of what I have learned about CMake has been trial and error.  The mailing list is great.  Even if you don't have questions, watching the list was an excellent source of info for me.
Comment: Hi Fraser, many thanks for this handy and working tip. Much appreciated. I will accept this as answer. As the saying goes "it's better to teach a man how to fish rather than to give him a fish". I read the cmake man page carefully, but never picked out what you said above. Did you learn what the `project` command does from reading `cmake` source or from any other doc that I missed?  If the later, a pointer would be much appreciated!


Como pegar um output parameter do MySql via C#

Question: Pesquisei e apliquei o que vi, porém está dando erro:
MySqlConnection con = new MySqlConnection(banco.Conexao);
MySqlCommand comando = new MySqlCommand("sp_venda", con);
comando.CommandType = System.Data.CommandType.StoredProcedure;

comando.Parameters.Add("usuario", MySqlDbType.Int32).Value = banco.Usuario;
comando.Parameters.Add("dinheiro", MySqlDbType.Double).Value = banco.formataDinheiro(Venda.Dinheiro);
comando.Parameters.Add("debito", MySqlDbType.Double).Value = banco.formataDinheiro(Venda.Dinheiro);
comando.Parameters.Add("credito", MySqlDbType.Double).Value = banco.formataDinheiro(Venda.Dinheiro);
comando.Parameters.Add("voucher", MySqlDbType.Double).Value = banco.formataDinheiro(Venda.Dinheiro);
comando.Parameters.Add("desconto", MySqlDbType.Double).Value = banco.formataDinheiro(VendaBLL.somaDescontos().ToString());
comando.Parameters.Add("troco", MySqlDbType.Double).Value = banco.formataDinheiro(Venda.Troco);

MySqlParameter paramHorario = new MySqlParameter("horario", MySqlDbType.Time);
paramHorario.Direction = System.Data.ParameterDirection.Output;

comando.Parameters.Add(paramHorario);

con.Open();
comando.ExecuteNonQuery();
Venda.Horario = comando.Parameters["horario"].Value.ToString();
con.Close();

O erro ocorre na seguinte linha:
comando.ExecuteNonQuery();

A conexão abre normalmente. O erro retornado é: "FormatException: Cadeia de entrada não estava em um formato incorreto."
O que estou fazendo de errado?
Edit
 public static string formataDinheiro(string grana)
 {
   if (DetectaTipo(grana) == "null")
      return "null";

 return grana.Replace("R$ ", "").Replace(",", ".");
 }

Comment: Poderia por favor colocar aqui sua função `banco.formataDinheiro()`?
Comment: @CiganoMorrisonMendez pronto (:
Comment: Atualizei minha resposta.
Answer: Quer dizer que algum dos seus parâmetros não está formatado como se deve. Sua função banco.formataDinheiro() não está devolvendo um Double, e sim um String. Por isso o erro.
Para se trabalhar com dinheiro, o ideal é o tipo Decimal. O Double pode trazer complicadores, como problemas nos arredondamentos.
Comment: É porque os parâmetros são resolvidos em tempo de execução do comando. Por isso o erro é "tardio".
Comment: Funcionou! Não imaginava que poderia ser isso por que enquanto debugava o compilador passava por essa parte tranquilamente. Seguirei seu conselho e mudarei o tipo pra decimal. Assim que der, marco a resposta como correta


How to selectively enable or disable SSL for some URLs in Tomcat 7

Question: I am using SSL in Tomcat 7, create a keystore and configure server.xml, and also config in web.xml as below
<security-constraint>

        <web-resource-collection>
            <web-resource-name>SSL</web-resource-name>
            <url-pattern>/inspectionGp/*</url-pattern>
            <url-pattern>/inspection/*</url-pattern>
            <url-pattern>/inspGpDt/*</url-pattern>
            <url-pattern>/inspDt/*</url-pattern>
        </web-resource-collection>
        <user-data-constraint>
            <transport-guarantee>CONFIDENTIAL</transport-guarantee>
        </user-data-constraint>
</security-constraint>

That works for SSL, these restricted URLs are forced to use only SSL, but other URLs can also use both HTTP and HTTPS, it's strange, I don't want use HTTPS for the URLs which are not configured in <url-pattern>. I use  <transport-guarantee>NONE</transport-guarantee>, that will result in all URLs support both HTTP and HTPPS. So can anybody help explain how I can resolve it?
Comment: Why? You're going to the trouble to enable SSL - why not just use it for everything?
Answer: Definitely consider using SSL for everything.  Many of our clients are going this direction and I recommend it fully.
If you mix HTTP/HTTPS and want to maintain session state, you'll have to circumvent some of the mechanisms in Tomcat that try to prevent this (secure-only cookies for JSESSIONID).  Once you successfully do that, then somebody could possibly hijack a session from the HTTP and take actions on the HTTPS side.
All that said, if you want to have this level of fine-grained control, you could consider using a ServletFilter.  The filter could be applied first in the web.xml and be applied to the "REQUEST" dispatcher on the "/*" URL pattern (or just patterns where you need to make this determination).
The filter would then look at the request.  If it is a path that must be HTTPS it would check the request.isSecure().  If not secure it would issue a 301 redirect to the HTTPS side of things, making sure to maintain the full request path and any URL parameters.  Likewise to redirect from HTTPS over to HTTP.
I am passionately in love with servlet filters.  They are the Swiss Army Knife of request handling.  There is nothing you cannot do.  Most importantly, the behavior is part of the app and not a customization to the Tomcat setup.  You can deploy the app anywhere and it'll "just work".
Comment: but I just want to know whether tomcat has native configuration way to disable ssl for some url, such as static resources cause our static resource is also in tomcat.
Comment: Can you still hijack a session with the filter?
Comment: The biggest risk to hijacking a session is creating session cookies with secure=false.  However, the default behavior is to create with secure=true on an HTTPS request.  Thus, with the defaults, the HTTP side of things will not be in the same session as the HTTPS.  If this is acceptable you're set.  If you want to share session then you expose yourself to session hijacking.  One might be able to craft more complicated schemes (like cookie accessible on HTTP side that only provides enough access to know if logged-in and invalidate the session), but sounds like more trouble than it's worth.


How to horizontally move a div as you scroll down in javascript and or css

Question: Does anybody know how can I get the effect of a carousel like the one in the bottom of this site https://brand.twitch.tv/  ??
I was using this example I found in codepen for the structure of this section.
but when trying to add it to my site with a vertical scroll, instead of being one continuous scroll like the one in the twitch site, its just a scroll inside another scroll. It feels weird for the user and you can miss it if you scroll down too fast.
Thanks in advance!
https://codepen.io/lemmin/pen/xRyXMZ

    body {
      margin:0;
      font-family:Georgia;
    }
    #container .box {
      width:100vw;
      height:100vh;
      display:inline-block;
      position:relative;
    }
    #container .box > div {
      width:100px;
      height:100px;
      font-size:96px;
      color:#FFF;
      position:absolute;
      top:50%;
      left:50%;
      margin:-50px 0 0 -50px;
      line-height:.7;
      font-weight:bold;
    }
    #container {
      overflow-y:scroll;
      overflow-x:hidden;
      transform: rotate(270deg) translateX(-100%);
      transform-origin: top left;
      background-color:#999;
      position:absolute;
      width:100vh;
      height:100vw;
    }
    #container2 {
      transform: rotate(90deg) translateY(-100vh);
      transform-origin: top left;
      white-space:nowrap;
      font-size:0;
    }

    .one {
      background-color: #45CCFF;
    }
    .two {
      background-color: #49E83E;
    }
    .three {
      background-color: #EDDE05;
    }
    .four {
      background-color: #E84B30;
    }

1

2

3

Last

Comment: https://css-tricks.com/aos-css-driven-scroll-animation-library/
Answer: One way to achieve this is by using Scrollmagic. The learning curve is a little steep but the rewards are worth it.
https://codepen.io/sean_pwc/pen/wvaaYWE
I've forked your pen and amended slightly. Here's how it works.
1) We add a bunch of divs that we want to be part of the normal page scroll. Nothing new here.

1

2

3

Last

I've set a height of 300px and the boxes take up full width of the screen. Then we add a section that we'd like to scroll horizontally. Notice the change in styles on the boxes - flex-direction is set to row, so that the boxes sit next to each other, and we set a width on them.

1

2

3

4

The comes the magic.
var controller = new ScrollMagic.Controller();

var scrollHorizontal = new TimelineLite()
  scrollHorizontal.to("#scrollHorizontal", 1, {x:'-100%'})

var horizontalScroll = new ScrollMagic.Scene({
      triggerElement: "#scrollHorizontal",
      triggerHook: 'onLeave',
      duration: 3000
    }).setPin("#scrollHorizontal").setTween(scrollHorizontal).addTo(controller);

I recommend you read the docs and play around with the demos to figure out whats going on. But essentially you set a controller which contains your animations.
var controller = new ScrollMagic.Controller();

Then we target the element we would like to move. Here, we target #scrollHorizontal, which is the wrapper for the h-boxes, and then we tell it move itself all the way over to the left until it is off the screen. Like you would position a side nav off the screen.
var scrollHorizontal = new TimelineLite()
  scrollHorizontal.to("#scrollHorizontal", 1, {x:'-100%'})

Now this will work, but the vertical scroll will still take effect, and it won't feel great. So we pin the element to be scrolled to the top of the screen - essentially scrollmagic adds a bunch of whitespace (set by the duration key, which is a height in pixels), which the user scrolls through, but it is hidden behind the pin, and we just see whatever you set to happen in scrollHorizontal (which was to move completely to the left).
var horizontalScroll = new ScrollMagic.Scene({
      triggerElement: "#scrollHorizontal",
      triggerHook: 'onLeave',
      duration: 3000
    }).setPin("#scrollHorizontal").setTween(scrollHorizontal).addTo(controller);

So we set a target element, and when the browser detects it, the element is pinned, the animation you declared in newTimelineLite() is actioned, for the duration you set, and then when the duration is over, we un-pin and go back to a natural vertical scroll. Play around with duration to change the speed of the horizontal scroll.
References:
https://scrollmagic.io/
https://scrollmagic.io/docs/index.html
EDIT
I should add, the codepen is using 4 scripts:
https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js
https://cdnjs.cloudflare.com/ajax/libs/gsap/2.1.3/TweenMax.min.js
https://cdnjs.cloudflare.com/ajax/libs/ScrollMagic/2.0.7/ScrollMagic.min.js
https://cdnjs.cloudflare.com/ajax/libs/ScrollMagic/2.0.7/plugins/animation.gsap.min.js


Model building with 10-fold validation

Question: I have yet to find a sufficient and succinct answer regarding model building with 10-fold cross validation (in this case, using Caret). I've found responses here, for instance: https://stackoverflow.com/questions/33470373/applying-k-fold-cross-validation-model-using-caret-package, and here: How to choose a predictive model after k-fold cross-validation?.
Would one build a model with all of the data, using R^2 (in this case, I'm just doing simple multiple regression) and likelihood ratio tests to determine the best (parsimonious) model, and then run a 10-fold validation? I've done train/test cross-validation, where one would build a model with, say, 70% of data, and then obtain an RMSE by comparing model predictions (from test dataset) w/ actual observations, but it isn't immediately clear to me how this translates to something like a 10-fold--and whether one, again, in the case of 10-fold, would just build the model with the entire dataset, vs a subsection.
After determining the best model, one would implement a package like Caret
train_control<- trainControl(method="cv", number=10)

model<- train(resp~ (all variables included in final model), data=mydat, trControl=train_control, method="rpart")

and then one would just obtain an RMSE as usual by model$pred and comparing this with actual values?
In this particular instance, I have a final sample of 350 participants. These are from the Camp Fire wild disaster in California in 2018, looking at factors like emotional support, family support, services, etc. (roughly 12 variables of interest, ~6 of which remain if I were to use all data) as a function of well-being. Things like ethnicity and gender and group (5 levels and 2 levels and 2 levels) are included in preliminary models, but none explain much variance. I've also looked at some interactions (marriage on SES) but those also don't show much significance. Only one time point.
With regard to a parsimonious model being "the best," it is merely my understanding that one attempts (through determining model significance via LRT) to achieve a balance between bias/variance among additional variables, thus, being less likely to over/under-fit the model. In this way, I believe that one can best also achieve a balance between an explanatory and predictive model. I'm certainly not steadfast in that; it is only what I have culled from experience.
Thanks much!
Comment: Think of it this way. Choosing between a regression with three variables and a regression with four variables is exactly the same as choosing between a regression with three variables and a neural network. You choose the best performing model either way. How do you decide what the best performing model is? K fold cross validation.
Comment: Please say more about why you consider the most parsimonious model to be the best. If you were interested in prediction that probably would not be the case. Also, how large is your data sample and how many predictors are you evaluating (including extra levels of categorical predictors, and interaction terms)? The answer to your question might depend on your reply. Please add that information by editing the question itself, as comments are often overlooked and sometimes get lost.
Comment: You definitely don't want to do k fold validation after selecting the model. The purpose (well one of them) of k fold cross validation is to select the model
Comment: Updated my question to reflect EdM's comment. @astel, so what you say is one of my points of confusion. From what I've read, it seems that one might want to compare two models using different approaches (say linear vs neural network) and thus, one would use k-fold to determine the best approach but model selection of the variables themselves would be separate from this and not involve k-fold.
Comment: @James Certainly the variable selection could be part of that. What makes you think otherwise?
Comment: @Dave, "what would make me think otherwise?"...just what I've read and thinking how one would engage in regular cross-validation (with 70% train dataset); where one would build a model with that dataset...one wouldn't really build a model (selecting variables) by obtaining an RMSE and then deciding to add another variable. The RMSE would just clarify whether or not the model is sufficient. With that in mind, I could see one using the RMSE (if poor) to determine whether or not one should take a different approach (i.e. neural network).
Comment: @astel, I could only include one "@" above, but I see your response as similar to Dave's. I may be wrong (or not completely right), but the above is my understanding. I think all of this highlights my confusion in moving from regular cross-validation to a k-folds approach.
Comment: "Regular" cross validation and k fold cross validation are the same thing. K fold is just regular done many times and averaged to reduce the variance in your estimate
Comment: Thanks, Astel, yes, I understand that they serve the same purpose; while it is totally clear to me how one would build a model with 70% data, it isn't clear to me how one would build a model with k-fold (maybe my confusion here is specifically with the Caret package and if I were to look at a function for k-fold vs using the Caret package, it would be clearer?).
Answer: It makes a difference whether you want to maximize the predictive accuracy of future data or infer parameters from at-hand data.
Predicting future data
If your goal is out-of-sample prediction, then it makes sense to evaluate the model on out-of-sample prediction (e.g., cross-validation). Simple as that. The R^2 on a "regular" regression model is in-sample.
Cross-validation is a nice out-of-sample method because you make the most of the data: every data point is used both for training and validation. You would use cross-validation, both for parameter selection and for hyperparameter tuning, e.g., if you chose a more flexible multiple regression model like elastic net.
Once you find a good model/hyperparameters, you can train it on all your available data and use that model for prediction. The new data would then be just like your hold-out data in the CV, so your CV above helped you learn about the performance of your model in this scenario.
Making inferences from existing data
While frequentist methods are popular (p-values, confidence intervals, likelihood-ratio tests), I'd recommend going Bayesian for better interpretability of the resulting parameter values (posterior distributions). Model selection for Bayesian regression models is a field in heavy development, but take a look at the rstanarm or brms packages for regression. They both support k-fold and leave-one-out cross-validation for model comparison (via the loo package), which makes a lot more sense for inference in a Bayesian setting, because the posteriors represent possible worlds rather than just the single most likely world as frequentist does.
Notes on cross-validation

CV need not be 10-fold. It can be 11-fold or 5-fold or something else. At the other end of the spectrum, you have leave-one-out (LOO) CV.
While RMSE is a common index for fit, you could easily have situations where another metric better represents your goal. I often work with problems where the cost of error is linear, not quadratic, and so I use mean absolute error - both as the cost function in the algorithm (regression etc.) and at the model selection level (aggregate of out-of-sample performance).
CV is only valid if you assume all folds to have essentially the same properties (you can help this along with stratification), and that your training set is representative of the yet-to-be-seen data you want to predict. This is an issue with time series where a leave-future-out (e.g., "rolling window" CV or "walk-forward" CV) method is often more appropriate.
Comment: @James I've updated my reply to answer both of your questions. If your goal is prediction, then yes, training on all data is OK because you will predict on out-of-sample data anyway, just as you did in CV. But I got the suspicion that you may want to just learn about relationships in your data, so I added a section about that too.
Comment: Predictive accuracy and inferential unambiguity should be closely related. Broad posteriors will often correspond to weak predictive accuracy. So I'd say that for inference, you just learn from your posteriors no matter their shape or width, as long as the model seems appropriate. You just learn very little (change credence in parameter values very little) if the data is uninformative and the prior-posterior relationship quantifies how much you've learned.
Comment: Thanks, Jonas. I'm getting more confused, but maybe closer to understanding. So, yes, it would make sense to evaluate the model using out-of-sample, but you'd still build the model using in-sample, correct? Re 4th paragraph: wouldn't finding a "good" model be the training of the model? And wouldn't you already have your predictions from the k-fold validation?  Re the final paragraph; yep, I'm (somewhat) familiar with different approaches: i.e. LOO and different metrics; thank you for your example of when you use MAE.
Comment: Thanks. I appreciate the clarifications. I've used STAN (with rStan) a bit and have explored BRMS, but have never used rstanarm, though I'd still consider myself a comparative beginner with Bayesian techniques. Re whether the focus is on predictive accuracy vs inference, it's good to clarify--it exacerbates confusion, because it isn't always clear whether one considers these in answering a question. I've found this to be a good paper: https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf for anyone looking through this post.
Comment: Still, in better trying to understand the nuances of the predictive/explanatory relationship--would you agree that one should only bother looking at inferences within the context of a model that meets some threshold predictive capacity (RMSE, MAE, etc value)? Or is there an argument to wave aside predictive capacity when looking at inference?

Model building with 10-fold validation


How to create car start/stop button in flutter?

Question: I want to create car selling app. I want use car start/stop button on onboarding screen & on tap play car engine start sound? Could you help for creating this widget?
Car start/stop button widget

Creating car start/stop button
Comment: I would use a Column and have one of the widgets depend on a boolean, if the button is clicked return a green widget , if the button is false(not clicked) return a grey button.
Answer: You could do something like:
<code>import 'package:flutter/material.dart';

class CarButton extends StatelessWidget {
  const CarButton({super.key});

  @override
  Widget build(BuildContext context) {
    bool isPressed = false;
    return GestureDetector(
      onTap: () {
        isPressed = !isPressed;
      },
      child: Column(
        children: [
          isPressed
              ? Container(
                  height: 10,
                  width: 30,
                  color: Colors.green,
                )
              : Container(
                  height: 10,
                  width: 30,
                  color: Colors.grey,
                ),
          Text('Start'),
          Text('Stop')
        ],
      ),
    );
  }
}
</code>
Comment: you may have to make it a stateful widget and use setState in onTap