DFA vs NFA engines: What is the difference in their capabilities and limitations?

Question: I am looking for a non-technical explanation of the difference between DFA vs NFA engines, based on their capabilities and limitations.
Comment: @SilentGhost I know its not heavy on the math, but that article depends on one knowing all all the mathmatical symbolism they have not explained is. Like many wikipedia articles it was written by someone who knows the subject so well, the can't see it from a beginners perspective, and lets face it that's who's going to read the article the most.
Comment: [This is by far the best explanation I have seen of DFAs vs NFAs](https://swtch.com/~rsc/regexp/regexp1.html). TL;DR, DFAs and NFAs have the same capabilities and limitations but some implementations using NFAs have bad performance.
Comment: http://en.wikipedia.org/wiki/Deterministic_finite-state_machine
Answer: Deterministic Finite Automatons (DFAs) and Nondeterministic Finite Automatons (NFAs) have exactly the same capabilities and limitations.  The only difference is notational convenience.
A finite automaton is a processor that has states and reads input, each input character potentially setting it into another state.  For example, a state might be "just read two Cs in a row" or "am starting a word".  These are usually used for quick scans of text to find patterns, such as lexical scanning of source code to turn it into tokens.
A deterministic finite automaton is in one state at a time, which is implementable.  A nondeterministic finite automaton can be in more than one state at a time:  for example, in a language where identifiers can begin with a digit, there might be a state "reading a number" and another state "reading an identifier", and an NFA could be in both at the same time when reading something starting "123".  Which state actually applies would depend on whether it encountered something not numeric before the end of the word.
Now, we can express "reading a number or identifier" as a state itself, and suddenly we don't need the NFA.  If we express combinations of states in an NFA as states themselves, we've got a DFA with a lot more states than the NFA, but which does the same thing.
It's a matter of which is easier to read or write or deal with.  DFAs are easier to understand per se, but NFAs are generally smaller.
Comment: Ok, a now-deleted answer confused NFA's with DFA's.  I've seen people do that before, and apparently that's down to an otherwise useful book, or so this claims anyhow: http://fanf.livejournal.com/37166.html
Answer: Here's a non-technical answer from Microsoft:
DFA engines run in linear time because they do not require backtracking (and thus they never test the same character twice). They can also guarantee matching the longest possible string. However, since a DFA engine contains only finite state, it cannot match a pattern with backreferences, and because it does not construct an explicit expansion, it cannot capture subexpressions.
Traditional NFA engines run so-called "greedy" match backtracking algorithms, testing all possible expansions of a regular expression in a specific order and accepting the first match. Because a traditional NFA constructs a specific expansion of the regular expression for a successful match, it can capture subexpression matches and matching backreferences. However, because a traditional NFA backtracks, it can visit exactly the same state multiple times if the state is arrived at over different paths. As a result, it can run exponentially slowly in the worst case. Because a traditional NFA accepts the first match it finds, it can also leave other (possibly longer) matches undiscovered.
POSIX NFA engines are like traditional NFA engines, except that they continue to backtrack until they can guarantee that they have found the longest match possible. As a result, a POSIX NFA engine is slower than a traditional NFA engine, and when using a POSIX NFA you cannot favor a shorter match over a longer one by changing the order of the backtracking search.
Traditional NFA engines are favored by programmers because they are more expressive than either DFA or POSIX NFA engines. Although in the worst case they can run slowly, you can steer them to find matches in linear or polynomial time using patterns that reduce ambiguities and limit backtracking.
[http://msdn.microsoft.com/en-us/library/0yzc2yb0.aspx]
Comment: The MSDN article is quite misleading; NFAs and DFAs are equally powerful.  NFA algorithms do not require backtracking (which has worst case exponential behavior).  The reason backtracking is needed is because "regular expressions" are much more powerful (e.g. backreferences) than regular languages and so they can't be modeled by the canonical NFA/DFAs.  Example of well-implemented NFA algorithm that doesn't use backtracking: http://swtch.com/~rsc/regexp/regexp1.html
Comment: On the topic of catastrophic backtracking: http://stackstatus.net/post/147710624694/outage-postmortem-july-20-2016
Comment: The MS article is in fact wrong if you stick with the standard definitions: An NFA does not backtrack at all. Certain backtracking automata are easier to implement if you start with an NFA, and maybe the article is referring to such an implementation.
Comment: Here's an reference implementation of a non-backtracking NFA algorithm: https://github.com/iter-tools/regex.

It's simplest to think of it as evolving the pattern. Consuming an input character eliminates some branches of the pattern that can no longer match, and alters (maybe) others that still can.

If you started with the pattern `b|.a|abc`, applying input `a` would get you pattern `a|bc`. Applying input `b` would get you pattern `c`. At this point the pattern matches if the next character is `c`.
Answer: A simple, nontechnical explanation, paraphrased from Jeffrey Friedl's book Mastering Regular Expressions.
CAVEAT:
While this book is generally considered the "regex bible", there appears some controversy as to whether the distinction made here between DFA and NFA is actually correct. I'm not a computer scientist, and I don't understand most of the theory behind what really is a "regular" expression, deterministic or not. After the controversy started, I deleted this answer because of this, but since then it has been referenced in comments to other answers. I would be very interested in discussing this further - can it be that Friedl really is wrong? Or did I get Friedl wrong (but I reread that chapter yesterday evening, and it's just like I remembered...)?
Edit: It appears that Friedl and I are indeed wrong. Please check out Eamon's excellent comments below.

Original answer:
A DFA engine steps through the input string character by character and tries (and remembers) all possible ways the regex could match at this point. If it reaches the end of the string, it declares success.
Imagine the string AAB and the regex A*AB. We now step through our string letter by letter.

A:

First branch: Can be matched by A*.
Second branch: Can be matched by ignoring the A* (zero repetitions are allowed) and using the second A in the regex.

A:

First branch: Can be matched by expanding A*.
Second branch: Can't be matched by B. Second branch fails. But:
Third branch: Can be matched by not expanding A* and using the second A instead.

B:

First branch: Can't be matched by expanding A* or by moving on in the regex to the next token A. First branch fails.
Third branch: Can be matched. Hooray!

A DFA engine never backtracks in the string.

An NFA engine steps through the regex token by token and tries all possible permutations on the string, backtracking if necessary. If it reaches the end of the regex, it declares success.
Imagine the same string and the same regex as before. We now step through our regex token by token:

A*: Match AA. Remember the backtracking positions 0 (start of string) and 1.
A: Doesn't match. But we have a backtracking position we can return to and try again. The regex engine steps back one character. Now A matches.
B: Matches. End of regex reached (with one backtracking position to spare). Hooray!
Comment: In some sense, it's all just terminology.  Having said that, DFA's are deterministic (and it's in the name) and NFA's are non-deterministic (again, as the name says).  That has a pretty-straightforward reason: DFA are always in exactly one state, and when presented with a character, there's always one unique (deterministic) next state that character always corresponds to.  So, you first explanation is a fine regex algorithm, but it's not a DFA - obviously, and as you describe, there can be multiple options and you never know which one is "best" until the string ends.
Comment: Your second algorithm labeled **NFA engine** is indeed one possible NFA implementation.  It resolves the same ambiguity are your first (also NFA) algorithm differently: namely by just picking one option and backtracking as needed.  So, it is indeed an NFA, but it's not the *only possible* NFA, as your first method demonstrates: that deals with the same nondeterminism differently.  I suppose you could call this a backtracking NFA engine, to distinguish the two.
Comment: This answer is inaccurate - catastrophic backtracking is orthogonal to the whole NFA/DFA distinction.  And what you describe as a DFA is actually an NFA (using the typical superposition of states) - DFA's are only ever in one state, hence "deterministic", and NFA's may be in multiple states, hence non-deterministic.
Comment: Finally, it's worth nothing that in *any* finite state automaton, as the name implies, the states are *finite* - more specifically, after embedding any relevant info into the state, that tuple still needs to have a finite number of options.  And *that* means that strictly speaking, perl-compatible engines aren't really *any* type of FSA, neither DFA nor NFA: after all, you can include arbitrary-length backreferences, and there's an **infinite** number of arbitrary length strings.
Comment: The distinction is critically important to performance because the infinite state space mean that you can't precompile the NFA nor efficiently execute it using the first algorithm.  In the general case backtracking breaks when faces with regexes (and you come across these *in practice*) which cause catastrophic backtracking.
Comment: So, there might be some non-NFA means of doing backreferences efficiently (I suspect that actually there is), but NFA can't be used to deal with backreferences.  Strictly speaking they can't do it at all, and loosely speaking and permitting unbounded annotation they can't do it reliably.
Comment: Yep, that's pretty much it.  Wikipedia has [an example NFA](http://en.wikipedia.org/wiki/Nondeterministic_finite_automaton#Example) and [an example DFA](http://en.wikipedia.org/wiki/Deterministic_finite_automaton#Example).
Comment: **@Tim_Pietzcker:** Thanks for posting the NFA engine steps too. Deleted the other question... :-)
Comment: @Eamon: Thank you very much for this insightful information. It's a shame that upvoting your comments won't give you rep.
Comment: @Eamon Nerbonne: Sorry, but before I came to your explanation, everything was clear. You mean that both examples are NFAs. I sort of agree. I see why DFA can't handle all the advances (actually non-regular) regex stuff. What I'm missing is an example DFA for simple regexes. Or would you agree that the first example could be transformed to DFA by defining the DFA states as sets of the NFA states and just rephrasing the algorithm?
Answer: Both NFAs and DFAs are finite automata, as their names say.
Both can be represented as a starting state, a success (or "accept") state (or set of success states), and a state table listing transitions.
In the state table of a DFA, each <state₀, input> key will transit to one and only one state₁.
In the state table of an NFA, each <state₀, input> will transit to a set of states.
When you take a DFA, reset it to it's start state, give it a sequence of input symbols, and you will know exactly what end state it's in and whether it's a success state or not.
When you take an NFA, however, it will, for each input symbol, look up the set of possible result states, and (in theory) randomly, "nondeterministically," select one of them.  If there exists a sequence of random selections which leads to one of the success states for that input string, then the NFA is said to succeed for that string.  In other words, you are expected to pretend that it magically always selects the right one.
One early question in computing was whether NFAs were more powerful than DFAs, due to that magic, and the answer turned out to be no since any NFA could be translated into an equivalent DFA.  Their capabilities and limitations are exactly precisely the same as one another.
For those wondering how real, non-magical, NFA engine can "magically" select the right successor state for a given symbol, this page describes the two common approaches.
Comment: Is the "then the DFA is said to succeed for that string" meant to be a "NFA" instead?
Answer: I find the explanation given in Regular Expressions, The Complete Tutorial by Jan Goyvaerts to be the most usable. See page 7 of this PDF:
https://www.princeton.edu/~mlovett/reference/Regular-Expressions.pdf
Among other points made on page 7, There are two kinds of regular expression engines: text-directed engines, and regex-directed engines. Jeffrey Friedl calls them DFA and NFA engines, respectively. ...certain very useful features, such as lazy quantifiers and backreferences, can only be implemented in regex-directed engines.
Comment: I believe that reference was the basis of what is now [Regular-Expressions.info](https://www.regular-expressions.info) It's the same author and I recognize some of the phrasing.


Building tables using Latex

Question: I am very new to LaTex
I met it a week ago while trying to build tables of my results output from Stats
I have read a lot of documentation but I have not been able to implement many.
I discovered I can go from .csv or excel, I downloaded excel2LaTex add in and yet I not been able to use it after a lot of efforts.
Can I get clear implementable guidance, probably how to build tables from excel or csv, dummy-level guidance.
Thank you
Dr Obum
Comment: It might be easier to read the csv directly using csvsimple or pgfplotstable.  The latter is better for numerical data.
Answer: Welcome to LaTeX, the in-side of the typography magic.
What you are using seems to me more like a black-box, where you would be on the out-side, and everything is somewhat opaquely done by some "Wizard"
I would suggest you, rather to

open the csv with a notepad or any plain text editor,
replace each comma (,) with an ampersand (&)-- if possible separated by two spaces on both sides to improve readability, and then
include the whole data within this code:
\begin{table}
    \begin{tabular}{l l l ...}
         ... ... ...
         ... ... ...
    \end{tabular}
\end{table}

with so many l, c or rs as the number of columns in your Excel data (for left, center or right-justifying the contents).
More about this here: https://www.sharelatex.com/learn/Tables
Answer: At first, when I must create table I used outer tools which generate LaTeX code.This is some of them

Table generator
LaTeX table editor
LaTable - WYSIWYG editor for LATEX tables
LaTeXTab: CSV/Excel to LaTeX Table - package for Sublime Text 3
Answer: My suggestion in three words: R, xtable, knitr.  Save the example with .Rnw extension. You can compile it easily with Rstudio or see How to build Knitr document from the command line.

\documentclass{article}
\renewcommand\belowcaptionskip{1ex}
\usepackage{booktabs,verbatim,parskip}
\begin{document}
\subsection*{The original CSV file:}
% Just to make the table.csv
<<echo=F,comment="">>=
library(xtable)
table <- data.frame(A=c(1.234,2.123,3.1416,4.44,5.001),
B=c(21,12,23,14,45))
row.names(table) <- c("One","Two","Tree","Four","Five") # Ahem!
write.csv(table, file = "table.csv",row.names=T)
@
\verbatiminput{table.csv}

\subsection*{The \LaTeX\ table:}
<<results="asis",echo=F>>=
tablecopy <- read.csv(file = "table.csv",  row.names = 1, check.names = F)
print(xtable(tablecopy,caption="The CSV data"),
caption.placement="top",include.rownames=T,booktabs = TRUE)
@
\end{document}

Building tables using Latex


How to use EOF in Xtext

Question: I'm trying to make an Xtext grammar recognize the end of a file as a valid statement, but all I've tried so far does not work.
Trying to use EOF directly will show me a " no viable alternative at input 'EOF' " message in the editor:
Object :
    typ=('char'| 'any' | 'number' | 'space' | 'symbol') ( ' ' | ';' | EOF )
;

So I tried to introduce a terminal that links to EOF:
terminal ETX: EOF;
...
Object :
    typ=('char'| 'any' | 'number' | 'space' | 'symbol') ( ' ' | ';' | ETX )
;

The xtext is compilable that way, but will give me the same error as above in the
"language editor window" (the one which opens when running the xtext as eclipse application)
The error occurs when I do not type a space or semicolon behind the last element (char):
symbol number char(!)

The same thing happens when I put all three allowed elements into a terminal:
terminal END: ( ' ' | ';' | EOF);
...
Object :
    typ=('char'| 'any' | 'number' | 'space' | 'symbol') END
;

The grammar is used to describe a kind of regular-expression-language,
which will be used to parse Strings of a defined format. For example:
(I will replace spaces with underscores to make this clear)
char_symbol_number_
char_symbol_number;

are accepted by the code correction (no errors are displayed).
Whereas
char_symbol_number

will give me the above mentioned error in the second eclipse editor window.
(The one with code completion etc. for my defined language.)
To point this out, what I'm trying to accomplish is a pure cosmetical adjustment,
so that I can leave the last statement as is without having to add a space or semicolon.

So the question is: How do I make Xtext recognize EOF (nothing) as a valid ending statement?
Answer: I can only guess from the grammar snippets how your final syntax should look like, but wouldn't
Object: typ=('char'|'any') ';'? ;

work for you?
Comment: Thank you for your quick response. No it does not work, I get the same " no viable alternative at input '' " error.
Comment: Please post your complete grammar and a complete sample file that you'd expect to be valid.

How to use EOF in Xtext


Javascript in HTML vs in a .js file

Question: I heard that putting the javascript in the html code isn't good for SEO and that it's better to put it in a .js file. It's clear that it makes the code more understandable but is it the only reason?
Comment: Refer this site for  Javascript in HTML vs in a .js file:https://softwareengineering.stackexchange.com/questions/223918/is-it-better-to-put-the-js-code-on-the-html-file-or-in-an-external-file#:~:text=It%20is%20better%20to%20create,you%20serve%20to%20the%20client.&text=Yes%2C%20obviously%20it%20is%20faster,JS%20code%20as%20an%20external.
How javascript impact SEO refer this site:https://medium.com/@benjburkholder/javascript-seo-server-side-rendering-vs-client-side-rendering-bc06b8ca2383
Answer: Other reasons to keep JavaScript in external files:

Pages usually load faster because JavaScript and stylesheets are cached by the browser.
Checking for scripting errors with JSLint becomes easier.
Code reuse between pages becomes simple.
Minifying JavaScript to reduce file size while maintaining non-minified versions for editing is easier.
Centralised hosting of common files becomes possible. (e.g. Google Libraries API )

Note that using JavaScript should make little difference to SEO (externally or internally) unless:

It's being used to generate links or other dynamic content on the page.
Your scripts are so big that they drastically increase page load time.

In the case of 1, it's best not to create links with client-side JavaScript when you can. In the case of 2, it's good practice to minify stylesheets and JavaScript, then combine files to serve it with as few HTTP requests as possible.
Comment: You javscript should be no-way near 1mb, let alone several! A few hundred kbs is considered a good page size.
Comment: From what size do you consider that the js file is big ? Several megabytes ?
Answer: If you include the JavaScript in the HTML, then the user has to download the JavaScript for every page they view that needs it.
If you make it an external file, then they have to download it once, which is slightly slower than including it in the page, because of the extra HTTP request. However, after that, it will be cached and they don't need to download it again.
In terms of the impact on SEO, that's the benefit - your site is quicker, which Google rewards greatly.
This means that if you have a piece of JavaScript that a user is going to need repeatedly, it should be external. If it's a one-off, you should include it on the page to reduce the number of HTTP requests.
Examples of where on-page JavaScript could be beneficial:

Arbitrage advertising sites, which typically pull a user in and bounce them out again. The user is typically not going to browse around. However, if you expect the user to come back again, you probably should make the JavaScript external so that it's cached when they return.
One-page sites. The same caveat applies as to arbitrage sites.
Pages which use a completely different set of JavaScript to every other page on your site. If, for example, you have a JavaScript slider on a particular page on your website and most of your visitors only look at that page once, then including the JavaScript in the header will probably be worth it for the SEO.

For code management, you are still best to develop with the code in an external file and then use a deployment script to grab the code and insert it for the production version of your code.
Finally, shared libraries like jQuery should always be external, because they will often already be cached in a visitor's browser from another site they visited.
Answer: There are two additional reasons for external JS files.

Caching. Once an external JS file is downloaded by the browser for the first time, it stays in browser's cache and the same code does not have to be downloaded every time a visitor reaches a page that she has not opened
Code management. If you want to change your script, you do it once, by modifying the JS file. The same effect may be obtained by server-side inclusion directly into an HTML document but the advantage of caching (point (1)) is not there anymore.

Javascript in HTML vs in a .js file


RESTORE DATABASE - The instance of the SQL Server Database Engine cannot obtain a LOCK resource at this time

Question: While doing a restore of a big database 1.8 TB into a different server
I came across this error message when the restore was about 90% completed:

90 percent processed.
Msg 3013, Level 16, State 1, Line 52
RESTORE DATABASE is terminating abnormally.

Msg 1204, Level 19, State 4, Line 52
The instance of the SQL Server Database Engine cannot obtain a LOCK resource at this time.
Rerun your statement when there are fewer active users.
Ask the database administrator to check the lock and memory configuration
for this instance, or to check for long-running transactions.

SELECT [instance]=@@servername,[setting name]=name, value, value_in_use, [description]
FROM sys.configurations
where name in ('min server memory (MB)',
               'min memory per query (KB)',
               'max server memory (MB)',
               'optimize for ad hoc workloads')

                SELECT [instance]=@@servername,
                       cpu_count AS [Logical CPU Count],
                       hyperthread_ratio AS [Hyperthread Ratio],
                cpu_count/hyperthread_ratio AS [Physical CPU Count],
                CONVERT(decimal(12,2),physical_memory_kb/1024.00/1024.00) AS [Physical Memory (GB)],
                affinity_type_desc,
                virtual_machine_type_desc,
                sqlserver_start_time
                FROM sys.dm_os_sys_info WITH (NOLOCK)

using this to check the sql error log, I could not find any clear indication of the cause of the above.
EXEC xp_readerrorlog 1

this is what I found:
Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.

what to do to allow my restores to work thoroughly and not stop at the middle?
Answer: I remember I had similar problem during the database restore, in my case it were reporting queries that run every 15-20 min and took tens of gigabytes of RAM for memory grants, as well as some extra memory for locks. That seemed to create memory pressure and disrupt the database restore.
The solution in my case was to temporarily disable SQL account that reporting queries used, until database restore was completed.
For your case I would suggest either increasing max server memory to a bigger number, or prevent users or apps or reports from running heavy queries for the duration of restore.


How to set inner object Data property for a datagridview

Question: I have created Entity Framework for a model.
I would like to access inner object and bind that inner object to a datagridview.
I can access inner object but i cant set data property of datagridview to show this.
Can any one please help me out
Here the code
I have two classes in EF.
ItemMaster And Category.ItemMaster Contains Category Id.Category Contains Category Name.
now in EF i can access Category Object like
<code> public static List<ItemMaster> GetAllItemMasters()
 {
 using (var itemContext = new EntitiesContext())
 {
        var items = itemContext.ItemMasters.Include("Category").Where(p=>  (p.IsDelete==   null || p.IsDeleted.Value == false)).ToList();
        return items;
    }
</code>
now while binding to a datagridview i have only properties like categoryid
in code behind i can access inner object like itemmaster.category.categoryname
but i cant set data property of datagridview as category.categoryname.
So can you please help me out
Answer: Add a reference to your namespace (if needed), then unbox the item into the parent class.
<code>((MyClass)Container.DataItem).MyInnerClassObject.SomePropertyValue
</code>
Comment: @RoopeshPm - Post your code.  I need to see the class you're binding to and the markup where the binding occurs.  Thanks.


Windows Server 2012 Essentials - Safe to delete default accounts?

Question: I'm using Windows Server 2012 Essentials with the trial version. It has a 25 limit and currently I'm a bit over that.
I'm wondering if is safe to delete some of the default users that got automatically added in way back when we setup our last server, some of these are: SBSMonAcct, Discovery Search Mailbox, Host Master, Microsoft Exchange, Microsoft Exchange Approval Assistant, Standard User, Standard User with admin links..
Most of these are marked as "inactive", whatever that means. Basically I think these are never used, but I don't know enough to know if it is safe to delete them.
Answer: I think you may delete the "inactive" accounts... no the others.
But run a "baremetal" backup first.


Are the concept of time and space apriori to natural language or are they just references within natural language?

Question: Are the concept of time and space apriori to natural language or are they just references within natural language? Time and space are fundamental concepts to existence and ontology. Natural languages seems to rely on referents so a person cannot understand time and space just from understanding language without experiencing time and space for himself, or is it really the case? Is time and space apriori components to language, that are deeply embedded into language so that a person can understand time and space from natural language alone (meaning that language has an apriori component as well as time and space has an apriori component inside that apriori component), or are they not? If not, then could there be a reality where there's more than time and space that are fundamental concepts to existence?
Comment: A person cannot understand anything from natural language alone, without some sort of external reference/experience, be it ostension or practical use, it is just a collection of sounds and symbols. What are references "within" natural language? Being "deeply embedded" is very different from being "*a priori*", and it is unclear what "*a priori* to natural language" even means. Does it just mean external? The usual meaning of "*a priori*" is prior to empirical experience, which is the opposite of how you seem to use it.
Comment: I believe you mean "prior" to natural language,  not "apriori" to natural language.
Answer: Kant thought so.
But Wittgenstein argues:

"If I say of myself that it is only from my own case that I know what
the word "pain" means - must I not say the same of other people too?
And how can I generalize the one case so irresponsibly?
Now someone tells me that he knows what pain is only from his own
case! --Suppose everyone had a box with something in it: we call it a
"beetle". No one can look into anyone else's box, and everyone says he
knows what a beetle is only by looking at his beetle. --Here it would
be quite possible for everyone to have something different in his box.
One might even imagine such a thing constantly changing. --But suppose
the word "beetle" had a use in these people's language? --If so it
would not be used as the name of a thing. The thing in the box has no
place in the language-game at all; not even as a something: for the
box might even be empty. --No, one can 'divide through' by the thing
in the box; it cancels out, whatever it is.
That is to say: if we construe the grammar of the expression of
sensation on the model of 'object and designation' the object drops
out of consideration as irrelevant."
-Philosophical Investigations 293

This is the core statement of what has been developed into The Private Language Argument. He also says:

" "I can only believe that someone else is in pain, but I know it if
I am."—Yes: one can make the decision to say "I believe he is in
pain" instead of "He is in pain". But that is all. —— What looks
like an explanation here, or like a statement about a mental process,
is in truth an exchange of one expression for another which, while we
are doing philosophy, seems the more appropriate one.
Just try—in a real case—to doubt someone else's fear or pain."
-Philosophical Investigations 303

I suggest time is like this. On the one hand, it is a linguistic token. But on the other hand, we use that token to point to a 'sensory primative', something less doubtable than anything else in each of our own experience.
Time and space are experienced as, ways of ordering experiences. That is: sequences, and relationships like 'locality'. But, they also become questions for science, about curved spacetime, the Thermodynamic Arrow of Time (inc entropy and irreversability), holographic correspondences of surfaces and bulk in Anti de Sitter space, etc - but updates there in physics, don't affect how we experience organising our subjective sensory information, into sequences and salience landscapes.
So we need to be careful to distinguish time and space in our direct experiences, and time and space as ways to organise and relate our experiences, as abstractions in the realm of linguistic tokens.
This comes into focus in discussions of time being emergent, like in Loop Quantum Gravity. There multiple 'competing' outcomes cluster around events (measurements), and then specific timelines get chosen from among the jostling possibilities. Time becomes only one way to understand organisation, in addition to the quantum spin lattice network. Another example is Hilbert Space, the imaginary 'space' of all states of a quantum system, which offers an alternative way to organise expectations about measurements, to ordinary spacetime.
So I would say: we have experiences, which have a deeply inter-subjective quality rooted in our similar bodies and similar experiences. But: we have language and culture, which abstract and organise experiences in sometimes profoundly different ways, in relation to different salience landscapes, which relate to different forms of life.
I would look to meditation, and especially Zen, to understand our direct experiences without the process of abstraction, that we associate with language using humans but not so much other minds. This begins with awareness itself as an intersubjective universal, and assembles a structure of mental phenomena from the five ayatana or sense-gates, to include three additional principles or organisational domains, up to Eighth Consciousness (compare to the Five Wits relating sensations and mental experiences in Western philosophy). The eighth domain can be directly compared to the noosphere, or memesphere.
Noether's Theorem shows us how we can relate dimensions to laws of conserved quantities under transformation, which can help us understand the Buddhist perspective that organising experiences is not subsequent to experiencing space and time, but that space and time are patterns or symmetries which emerge consequent to experiencing things, which is founded in what awareness is itself - for which I look to Hofstadter's idea of Strange Loops, as being defined by types of self-reference that we call subjectivity, especially as expanded by the tools of language, which rely on intersubjectivity (see Indra's Net and According to the major theories of concepts, where do meanings come from?).
Because subjectivity is a precursor to experience, we may reasonably expect to find that time and space are not fundamental, but only necessary to organising our decisions and experiences. Eternal Recurrence, karmic rebirth, and intersubjectivity itself, can be understood as ways of organising or experiences and decisions that point beyond linear timelines and local subjectivity, towards other ways of organising ourselves, potentially with higher emergent dimensionality capable of moving in a larger space of counterfactual timelines and variations in initial conditions.
Answer: If nothing else, it seems like the questions, "Where is X?" and, "When is X?" require irreducibly spatial or temporal inputs to resolve. I.e., I can't adequately answer, "Where is X?" just by referring to when X is, or to other temporal facts, e.g. how fast X is moving. Without knowing where X was at some relative starting point, and in what directions X has been heading, then merely knowing the rate of X's movement over time will not allow me to determine where X is (or has been), unless perhaps there were a rate of movement so fast that whichever direction you traveled at that rate, from whichever starting point, would somehow result in any X ending up wherever any other would, but that's a convoluted "possible description" that I don't want to put much stock in.
Or suppose you ask, "Why does X exist?" Let's say the answer was, "In order to exist on Planet Y." Would this teach us that X is, in fact, on Planet Y? I don't think it would have to mean this. Or, "How does X exist?" would not lead us so much, again, to where X is (right now), unless the way that X exists confines it to some position; but identifying that possible position in the first place, seems to prerequire a notion of places as such.
Comment: Don't you think here, you are giving a coordinate system, as a special case of *narrative structuring*?
Comment: To say nothing of Z.
Comment: @CriglCragl, IIRC Alessio Moretti or some Beziau(?) guy said something about a logical "flower" for narratology(?), along the lines of the other Moretti structures. I think Moretti's been at pains to showcase his flower theory as more than a variant of graph theory, but I don't recall what he's said about set/category theory besides, just that he seems to be empathetic to intuitionism, too (he places a very high premium on the geometry of the flowers). An evolutionary sequence of Moretti structures, so to speak, with a narratological base, might yield spacetime as a subnarrative?
Answer: According to the empirical studies of the Natural Semantic Metalanguage (NSM) project, both time and space are core concepts, represented by several semantic primes each - core concepts of human thinking that cannot be reduced to others, and are shared by all natural human languages. The list of NSM primes has been empirically developed over more than 40 years as a growing number of languages from all major language families have been studied, such that it is reasonable to conclude that these primes are found in all natural languages.

WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR SOME TIME, MOMENT
WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE

You can look at the other primes yourself, all of which are equally fundamental to human language. Whether or not they're fundamental concepts to existence is debatable, but without them we wouldn't have human language like we do.


How to protect PGP key details against be snooped?

Question: I haven't found an answer to the following.
Hypothesis: If someone gets physical access to my computer,
how I would make sure that they (the ones snooping around) do not find
my PGP details; like keys or email addresses that are linked to keys?
Is the only and best solution to encrypt your whole hard drive and hope that your password is strong enough? Or can you just kept your PGP key details hidden on an encrypted volume, and take them out when you want to encrypt or decrypt some message?
I haven't been able to prove the later.
Comment: You might want to ask this on [Security.SE] rather than here.
Answer: As for all infosec related questions, the first thing to consider is who is the attacker you want to mitigate. Then, elaborate possible attacks he might perform, and how much effort your data is worth to him.
You will never be able to prove safety. You'd need to prove safety of not only the whole software in use (including operating system and application software), but also the hardware and cryptographic algorithms in use. Right now, science still fights with proving very minor snippets of software, far away from even proving GnuPG or similar applications.
All you can do is consider possible (known, expected) attack vectors, select by the value of your information to the attacker and apply appropriate counter measures. To get a maximum extend of security, consider using an offline machine not connected to any computer networks inside a protected vault. Does not seem appropriate? Very likely it isn't, but you'll have to choose somewhere in-between convenience (and cost effectiveness) and safety.
Encrypted Hard Drive
All together, not even encrypting your hard drive will be safe, if you put your machine into sleep mode and the attacker is able to read the memory (with a large amount of effort), or even modifies/backdoors your hardware (or simply gets hold of the running machine, while it's unlocked). Malwared software will obviously also expose your private keys.
Encrypting your hard drive, including swap space, is probably enough against most attackers.
Encrypted GnuPG Home Directory
If you only want to be sure a "casual" attacker (no special hardware support, no much deeper than "off-the-shelve" IT knowledge) or thief of your computer does not get access to the information stored in your GnuPG keyring, an encrypted volume should be fine. Possible attack vectors I can imagine:

temporary files left over
your mail store, which contains encrypted/signed mails
malware that accesses your keyring while it's unlocked/sniffing your passphrase
admin users accessing your keyring while it's unlocked/sniffing your passphrase
...


$\mathbb{Z}$-module $\prod\limits_{\text{p prime} } \mathbb{Z}/p\mathbb{Z}$

Question: I am trying to establish if the $\mathbb{Z}$-module $\displaystyle\prod_{\text{p prime} } \mathbb{Z}/p\mathbb{Z}$ is torsion-free. So I think that the elements of  $\displaystyle\prod_{\text{p prime} } \mathbb{Z}/p\mathbb{Z}$ are all infinite tuples where on the first position in any of such tuple we'd have an element from $\mathbb{Z}/2\mathbb{Z}$, on the second position an element from $\mathbb{Z}/3\mathbb{Z}$, etc. Is that correct? So my guess would be that this module is not torsion-free. For example, we have $2\{[0],[1],[0],\dots,[0]\}=\{[0],[0],[0],\dots,[0]\}$. Is that right? In general, assuming that the above is true, what would be that module's complete torsion part?
Answer: Your example is almost right; the nonzero second coordinate $[1]$ is an element of $\Bbb{Z}/3\Bbb{Z}$, not of $\Bbb{Z}/2\Bbb{Z}$ so
$$2([0],[1],[0],\ldots)=([0],[2],[0],\ldots).$$
You are on the right track though. Note that elements of a product are usually denoted with round braces, not curly ones. Also, since the product is infinite it is customary not to write a last coordinate, but in stead to end with some dots...
The torsion part is the submodule $\bigoplus_{p\text{ prime}}\Bbb{Z}/p\Bbb{Z}$ of tuples with finite support. I'll leave it for you to prove.
Comment: Of course, I should've written $2([1],[0],[0],\dots)=([0],[0],[0],\dots)$. Thank you for pointing that out. OK, thank you! I'll try to prove that.


Error rebuilding ATMHud.DLL for Monotouch

Question: After updating to the latest versions of Xamarin Studio this morning the ZBar and ATMHud references in our project started throwing a bunch of errors, all of which were:
The type 'MonoTouch.UIKit.UIViewController' is defined in an assembly that is not referenced.  Consider adding a reference to assembly 'monotouch, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' (CS0012)'
I did some reading and found out that I needed to rebuild both of the libraries because of something to do with "assembly strong names" in Xamarin.  I was able to muddle my way through rebuilding the ZBar library, but the ATMHud rebuild has given me some trouble and I'm not familiar enough with what I'm doing to fix the problem.
I downloaded the ATMHud project from github and followed the instructions in the readme which state "Run 'make' in the binding directory to build ATMHud.dll", but when I do that I get errors.
Teds-Mac:binding ted$ make
/Developer/MonoTouch/usr/bin/btouch -e atmhud.cs enums.cs AssemblyInfo.cs --out=ATMHud.dll --link-with=libATMHudSDK.a,libATMHudSDK.a
/var/folders/75/vl_6zkyx03sc4tymnp41lsgw0000gn/T/09x9yx7m.nz9/AtmHud/AtmHud.g.cs(751,86): error CS0103: The name class_ptr' does not exist in the current context
/var/folders/75/vl_6zkyx03sc4tymnp41lsgw0000gn/T/09x9yx7m.nz9/AtmHud/AtmSoundFX.g.cs(75,104): error CS0103: The nameclass_ptr' does not exist in the current context
Compilation failed: 2 error(s), 0 warnings
btouch: API binding contains errors.
make: * [ATMHud.dll] Error 1
I've scanned through the files in the folders but can't seem to track down any references to "class_ptr".
Can anyone help with what might be the cause of the problem I'm seeing, and can you confirm that this will get me past the original "assembly strong names" problem I'm seeing?
Answer: As a temporary measure you can fix your old assemblies with this tool: https://github.com/rolfbjarne/iOSRefUpdater
Update
I've fixed the ATMHud bindings, now they should compile with newer versions of Xamarin.iOS installed as well.
Comment: Worked like a charm, I appreciate the quick reply


Refunding of payments made

Question: I was scammed November 2017 by a woman called, Adele Johnson. I wanted to trade on bitcoin, but didn't had the experience to do it myself. She offered her help to trade on behalf of me. I paid R8000 into my Luno account where she requested all my registration details as well my wallet address. She made use of Tradeway Option Platform to trade for me. According to her my money growth to $50 000 and for me to withdraw this money I need to pay $7000. I told her I don't have such money, she got angry and blocked me.
A man, called Artem Vladys told me he can help me to trace this money through interpol. He requested a payment of $300 and he assured me that will be the only payment I had to make; which I made via Luno, I bought him bitcoin worth $300 and sent it to the wallet adress: 154oDjBXc1wSzYncP6YwrU7a9duVGd3J3. He requested another amound of $200, I unfortunately refused to make another payment before I haven't seen what he did to assist me tracing my lost money. He got mad and he blocked me.
Both people said they work for bitcoin.
All the transactions shows in my luno account. Is there any possibility that I can trace this money? Is there not honarable people/organisation who can get my money back?
Answer:
Both people said they work for bitcoin

Some people lie.

Is there any possibility that I can trace this money?

Criminals often try to make that difficult for you. It is unlikely that anyone can trace exactly where your money went.

Is there not honarable people/organisation ...

There are but

... who can get my money back?

Part of the design of Bitcoin and other cryptocurrencies is to make it hard to track transactions to real people.
What you have done is the equivalent of putting banknotes in an envelope and posting it to a post-office box address in a foreign country where such boxes can be rented without presenting any ID.
You should report the crime to your local police. You would be wise to assume your money is lost.

Refunding of payments made


Is the product of two invertible symmetric matrices always diagonalizable?

Question: I have two symmetric matrices $A$ and $B$, which are both invertible. Their eigenvalues are obviously real, but not necessarily positive. I know that if one matrix were positive definite, we could use (Product of two symmetric matrices is similar to a symmetric matrix) to show that $AB$ is similar to a symmetric matrix (with real eigenvalues). In my cases, the eigenvalues of $AB$ are generally complex, but can one adjust the proof to show that $AB$ is always diagonalizable?
Answer: No. Here is a counterexample that works not only over $\mathbb R$ but also over any field:
$$
\pmatrix{1&1\\ 0&1}=\pmatrix{1&1\\ 1&0}\pmatrix{0&1\\ 1&0}.
$$
In fact, it is known that every square matrix in a field $F$ is the product of two symmetric matrices over $F$. See Olga Taussky, The Role of Symmetric Matrices in the Study of General Matrices, Linear Algebra and Its Applications, 5:147-154, 1972 and also Positive-Definite Matrices and Their Role in the Study of the Characteristic Roots of General Matrices, Advances in Mathematics, 2(2):175-186, 1968.
Comment: I'm glad to see that the OP decided at last to mark your answer as the accepted one, since it is mucher better than mine.
Answer: In general, no. Take$$A=\begin{bmatrix}1 & 1 \\ 1 & \frac{15}{64}\end{bmatrix}\quad\text{and}\quad B=\begin{bmatrix}1&-4\\-4&0\end{bmatrix}.$$Then both $A$ and $B$ are symmetric and invertible. But$$AB=\begin{bmatrix}-3&-4\\\frac1{16}&-4\end{bmatrix},$$which is not diagonalizable: its only eigenvalue is $-\frac72$, and the corresponding eigenspace is $1$-dimensional.


DevIL error 1290. How can I fix it?

Question: I would like to load image to my application, but I have an error:
http://img510.imageshack.us/img510/5814/blad07864.png
This is a code of this application:
<code>#include <stdio.h>
#include <stdlib.h>

#undef _UNICODE
#include "il.h"

#pragma comment( lib, "DevIL.lib" )

// Wow. DevIL is amazing.

// From http://gpwiki.org/index.php/DevIL:Tutorials:Basics

// The library consists of three sub-libraries:
//  * IL - main DevIL library. It allows you to load and save images to files. Every function in this library have 'il' prefixed to their name.
//  * ILU - this library contains functions for altering images. Every function in this library have 'ilu' prefixed to their name.
//  * ILUT - this library connects DevIL with OpenGL. Every function in this library have 'ilut' prefixed to their name.

int main()
{
  ilInit();
  printf("DevIL has been initialized\n");

  // Loading an image
  ILboolean result = ilLoadImage( "tex1.png" ) ;

  if( result == true )
  {
    printf("the image loaded successfully\n");
  }
  else
  {
    printf("The image failed to load\n" ) ;

    ILenum err = ilGetError() ;
    printf( "the error %d\n", err );
    printf( "string is %s\n", ilGetString( err ) );
  }

  int size = ilGetInteger( IL_IMAGE_SIZE_OF_DATA ) ;
  printf("Data size:  %d\n", size );
  ILubyte * bytes = ilGetData() ;

  for( int i = 0 ; i < size; i++ )
  {
    // see we should see the byte data of the image now.
    printf( "%d\n", bytes[ i ] );
  }
}
</code>
I found code from this site: http://bobobobo.wordpress.com/2009/03/02/how-to-load-a-png-image-in-c/
Can you help me?
Answer: According to this post, 1290 means the image path wasn't found. Try using an absolute file path and see if it can load then.

DevIL error 1290. How can I fix it?


Nature of Dengue Fever Genetic Material

Question: Once the genetic material for Dengue Fever is inserted into the human DNA, would the Dengue Fever molecular material be in the form of its own isolated DNA fragment or will it be inserted directly into some existing piece of our DNA?
Comment: I'm looking at the life cycle for dengue virus, it looks like a positive-sense ssRNA virus. The viral genetic material doesn't integrate into the host genome. Someone correct me if I'm wrong!
Comment: @Kendall you are correct. ss(+)RNA viral genomes behave like mRNA in the host, and are directly translated by ribosomes. The viral proteins thus produced then direct production of more genomes and the assembly of virus particles to infect more cells and other hosts. Viruses that integrate into the host's genome are called *retroviruses*.
Comment: Ah, thank you for the clarification. If you wouldn't mind posting that as an answer I could accept it.
Answer: In a 2011 Nature review they provide a figure for the life cycle of the Dengue virus,

Source: Immunity to dengue virus: a tale of original antigenic sin and tropical cytokine storms
The Dengue virus is a positive-sense ssRNA virus. This means the ssRNA that enters the host may immediately be translated. The viral ssRNA does not integrate with the host genome!

Nature of Dengue Fever Genetic Material


"This Card is Already in wallet" message when I'm trying to add card to Wallet & Apple Pay which was removed

Question: I removed credit card and I'm trying to add it again in "Wallet & Apple Pay", but I get the message as in title.
At iWatch I removed and added the same card without problem.
Answer: I removed other cards which I had already in Wallet and then one by one added them, also the problematic card.
Answer: I removed all cards from wallet and when I wanted to re-add these got the message "card already in wallet". Online all solutions pointed me to iCloud but the portal has been changes and these solutions were no longer relevant.
The following solved the problem for me:

On iPhone go to Settings app and click on top section where your picture is (Apple Id, iCloud settings).
Select your phone in the list of devices at the bottom of this screen
At the bottom you will see "Wallet & Apple Pay" - from here you can click on "Remove Items"
Re-add your cards, no error will be displayed


Custom Jackson deserializer not able to determine target class

Question: I currently have a Join object that I am trying to write a custom deserializer for.  We store the database key and the Java class name of the object in the JSON, and that looks something like this:
{
    "@class": "com.mysite.data.ExpertFor",
    "id": "expert-for-1",
    "leftId": "user-profile-1",
    "leftType": "com.mysite.data.UserProfile",
    "rightId": "hotel-1",
    "rightType": "com.mysite.data.Hotel"
}

We do this so we don't have to store the actual object in the database; rather, we retrieve it when we deserialize it.  The issue is that my custom serializer cannot see the @class element in the JSON stream - therefore, it cannot properly instantiate the needed object after all the data is read.  Here is my deserializer:
public Join deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException {
    Map<String, Object> values = new HashMap<String, Object>();

    // Loop through the JSON document and pull out all
    // the values and put them into the map
    JsonToken token = jp.nextToken();
    while (token != JsonToken.END_OBJECT) {
        logger.info("Non-data field: [" + token + "], name is [" + jp.getCurrentName() + "], text is [" + jp.getText() + "]");
        if (token == JsonToken.VALUE_STRING) {
            values.put(jp.getCurrentName(), jp.getText());
        }
        token = jp.nextToken();
    }
    // remainder of code omitted for brevity
}

Now, the Log4J lines that are written to the logs show all of the appropriate elements of the JSON I would expect, EXCEPT the @class element
Data field: [VALUE_STRING], name is [id], text is [expert-for-1]
Data field: [VALUE_STRING], name is [leftId], text is [user-profile-1]
Data field: [VALUE_STRING], name is [rightId], text is [hotel-1]
Data field: [VALUE_STRING], name is [leftType], text is [com.mysite.data.UserProfile]
Data field: [VALUE_STRING], name is [rightType], text is [com.mysite.data.Hotel]

I have tried adding the @JsonTypeInfo annotation to my class, like
@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "@class")

but it appears the element is getting 'swallowed' by the code, and I don't know how to get at it.  And I need this, because any descendants of Join will need to be deserialized.  Here is some additional information about the hierarchy this class is in:
Adding the @TypeInfo annotation was only there as an attempt to try and see what was going on.  I am getting the same behavior whether that is there or not.  What's really strange is that we are using this same basic functionality in other classes that need custom deserialization and those see the @class just fine.  Here is the heirarchy of this:
@JsonAutoDetect(value = JsonMethod.NONE)
@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "@class")
public abstract class TaggedObject implements Serializable {
...
}

and then we have DataObject
public abstract class DataObject extends TaggedObject {
...
}

and then we have our Join class:
@JsonSerialize(using = Join.Serializer.class)
@JsonDeserialize(using = Join.Deserializer.class)
public abstract class Join<L, R> extends DataObject {
...
}

I would think the @JsonTypeInfo would be inherited from TaggedObject, but it is not seen here, whereas it is seen by other classes within the same hierarchy.
Anyone have any ideas on how I can grab the class of this object so I can create the appropriate object?  What am I missing here?
Answer: You should not try to manually handle @class AND declare it to be handled by Jackson with @JsonTypeInfo -- either let Jackson fully handle it (which it can and will do), or remove anntoation and handle things manually.
Answer: using Jackson Tree Model, the following code prints the missing @class:
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(jsonString);
Iterator<String> iterator = root.fieldNames();
while (iterator.hasNext()) {
    String fieldName = iterator.next();
    String fieldValue = root.get(fieldName).asText();
    System.out.println("name is [" + fieldName + "], text is [" + fieldValue + "]");
}

Output:
name is [@class], text is [com.mysite.data.ExpertFor]
name is [id], text is [expert-for-1]
name is [leftId], text is [user-profile-1]
name is [leftType], text is [com.mysite.data.UserProfile]
name is [rightId], text is [hotel-1]
name is [rightType], text is [com.mysite.data.Hotel]

Comment: While this code may answer the question, providing additional context regarding how and/or why it solves the problem would improve the answer's long-term value.


Requirements to move an LXC container from host to another

Question: What are the requirements and things to know to move an LXC container from an host to an other ?
Answer: AFAIK, we haven't working and stable solution for this task. Bleeding of edge technology for containers live migration are OpenVZ and P.Haul (part of CRIU project): https://github.com/xemul/p.haul
As main requirement I will mention vanilla kernel from kernel.org.
Answer: I use Duply to backup LXC containers. Unlike backing up a normal machine you DO want to include <code>/dev</code> from the LXC container in the backup.
Exclude the following files from the backup:
<code>- /var/lib/lxc/*/rootfs/lost+found
- /var/lib/lxc/*/rootfs/media/*
- /var/lib/lxc/*/rootfs/mnt/*
- /var/lib/lxc/*/rootfs/proc/*
- /var/lib/lxc/*/rootfs/run/*
- /var/lib/lxc/*/rootfs/sys/*
- /var/lib/lxc/*/rootfs/tmp/*
</code>
& see my notes here for further details.


How do you use block in other hand

Question: How do you use the block in other hand in Minecraft 1.10.2?
Answer: You use it just like normal, by right clicking. But, it will only get used if you have nothing that can be used in your right hand, so you can right click and place blocks with the left hand if you have an item that cannot be used by right clicking in the right hand.
Answer: The item in your left hand can't be used with left-clicking, only right-clicking. When you right-click, the game first tries to use the item in your right hand. If the item doesn't do anything when right-clicked, it "right-clicks" your left hand.
That means that when you have a a stone block in your left hand, and a dirt block in your right, you won't be able to place the stone, only the dirt. But if you have a non-right-clickable item in your right hand, you will be able to place your stone.

How do you use block in other hand


What are dotted quaver beats in 12/8 time?

Question: In grade 3 music theory I learned about compound time and how in 12/8 for example you have 4 dotted crotchet (quarter note) beats per bar. Each beat them gets 3 quavers (8th notes). But what does it mean when a beat is notated as two dotted quavers? Does this mean that in that instance, the dotted crotchet is temporarily halved giving us 2 quavers per beat and thereby suspending compound time temporarily? I say temporarily because it is supposed that a beat should be divided into 3 for compound time so by dividing it in 2 it is like suspending compound time temporarily.
Answer: In 12/8 the beat is normally divided into three quavers.   But, just as in a 'simple' time (2/4, 3/4, 4/4...) we might want a triplet, in a 'compound' time (6/8, 9/8...) we might want a duplet.
I wouldn't say that compound time is 'suspended'.  One cross-rhythm doesn't change the meter any more than one chromatic chord changes the key.  And other instruments may be continuing the compound groove at the same time.
Traditionally, this would be written with a 2-bracket.  But a dotted quaver is  half a dotted crotchet, and there is a growing trend to write a pair of dotted quavers instead.
I prefer the duplet as giving a clearer indication that 'normal' grouping is being overridden.
Gould agrees with me.  Doubtless some people here will not!

(Gould p.207)
Answer: It's two to the beat instead of three to the beat.  It can be notated as a duplet (duole), but this is rarely done since a dotted quaver has the right length without requiring special notation, even if its intuitive relation to the meter is less obvious than duplet notation would exhibit.
Comment: Until recently, the duplet notation was standard.
Answer: 12/16 would be compound quadruple time with dotted quaver beats. 12/8 would be compound quadruple time with dotted crotchet beats and the rarely used 12/4 would be the compound quadruple time signature with dotted minim beats.
All of them have four beats, all four beats having dots, the only difference being what consist of a beat, dotted quaver vs dotted crotchet vs dotted minim.


What is the typical accuracy of masked language models during BERT pretraining?

Question: I was reading the BERT paper but I didn't find any tables concerning the performance of the masked language models during pretraining. Does anyone know the accuracy of BERT's masked language model?
Answer: It is very hard to tell and researchers usually don't measure it because it's not a comparable number such as GLUE score. It depends on size of your vocabulary and also the masking strategy (for example newer BERT successors use span-masking). BERT is a subword language model (uses WordPiece tokenizer) so if you use large vocab then sentences are usually tokenized to "smaller" subword units that are easier to predict. I trained a BERT-like model for Czech language, which is morphologically more complex than English, and got top@1 test accuracy ~0.55, top@3 test accuracy ~0.68 after 1.5M steps with 30K vocab size.


What are the missing pieces that prevents us from deriving the laws of chemistry from physics?

Question: What are the missing pieces that prevents us from deriving the laws of chemistry from physics? People say it's emergent properties, but it's hard to believe that there are emergent properties between physics and chemistry. Simulating physical laws for trillions of atoms should allow us to derive chemical laws. No? Is there something missing to this mystery?
Comment: For your "method" the missing piece is computational power. Simulating physical laws for trillions of atoms by our current means would take longer than the age of the universe, and without actually doing the simulating we can't check if the chemical laws do indeed obtain. Of course, a mathematical shortcut (like the one we have for thermodynamics in statistical mechanics) would be much more preferable, but that piece is missing as well.
Comment: @Conifold: And hence chemistry is one of the most promising areas to apply quantum computing to https://www.scientificamerican.com/article/how-quantum-computing-could-remake-chemistry/
Comment: There are [ab initio methods in computational chemistry](https://en.wikipedia.org/wiki/Computational_chemistry#Ab_initio_methods) which can derive at least some types of chemical behavior starting only with quantum laws (mathematical approximations are used to predict the consequences of those quantum laws, but the approximations don't import any specific empirical facts taken from chemistry). As Conifold said, it's probably just lack of computational power that prevents such methods from being used to derive all of chemistry, or at least there's no good evidence against that explanation.
Comment: https://en.wikipedia.org/wiki/Density_functional_theory (relevant)
Answer: I find this implicit disdain towards emergent properties as 'not really explained' puzzling. I think it relates to a misconception about the ontology of emergent properties. See these discussions:

What's the "opposite" of emergence?
Is the concept of emergence sufficient in blocking reductionism?
What are the alleged reasons for emergence?

In chemistry, chemical buffering is an example of an important emergent property fully understood from first principles.
The Gibbs free energy bridges physics with chemistry and biology, through an understanding of the 'harvestable' entropy. Through an understanding of the physics of entropy, of emergent dynamics, we can understand fundamental physics constraints on life.
There are unsolved problems in physics, regarding specific areas of chemistry:

we don't know the strength of gravity at atomic scales
the thermodynamic arrow of time is emergent, and it's relation to the symmetries of space-time, are not understood
there are many unsolved problems in modelling superconductivity, with new behaviours of material surfaces under tension
Bose-Einstein condensates are not fully understood, & probably require space labs to work on
fluid dynamics & turbulent flow
amorphous solids & glass transitions
fractional Hall-effect
endless problems of biochemistry like magnetoception & the origins of homochirality

Does that mean 'the laws of chemistry have not been derived from physics'? I heard a good metaphor, that saying other sciences are reducible to physics, is like saying literature is reducible to the alphabet. We don't understand Minoan cuneiform script 'linear A', so has literature not been fully reconciled with alphabets? It's a doubly irrelevant idea, it is exactly the emergent ideas of literature that matter, and the existence of unsolved problems does not impact the solved provlems and the successful midels and accounts that go with them.
Increasingly all of the laws of physics are expected to be emergent from a more fundamental strata, and in this answer I make the case mathematics is emergent too, and that in this light lists of valid laws will never be complete, because new ones can continue to emerge: About Wigner's view on the relation between mathematics and physics?
Comment: Nice answer! I particularly like the 'literature reducible to alphabet' analogy. Haven't heard that one before...
Comment: @TheDoctor: Your disagreement seems to entail a few wishful assumptions. What makes you think there isn't a 'whole layer of complexity' between physics and chemistry? Quantum theory and non-linear dynamics alone upset that particular apple cart. That old Newtonian 'clockwork universe' paradigm doesn't hold as much water as it used to...
Comment: Why don't you know the strength of gravity at atomic scales?  The formula of Newton is pretty clear.   You're right, though, they don't know about amorphous solids and glass transition, but neither do they really know how to derive the melting point of steel and metal  transitions.  OTOH, they DO know pretty well about fluid mechanics and turbulent flows, so I don't know why you put this one here.
Comment: @TedWrigley:  I must respectfully disagree with this analogy: one cannot derive literature from the alphabet because there is a whole layer of complexity between the two.  This shouldn't be the case with chemistry -- they simply don't know enough about atoms and *presume* that everything conforms to atomic theory.
Comment: @TheDoctor: "theoretical understanding of the solutions to these equations is incomplete. In particular, solutions of the Navier–Stokes equations often include turbulence, which remains one of the greatest unsolved problems in physics, despite its immense importance in science and engineering" From https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existence_and_smoothness Can you give a reference to melting point of steel not being understood? The 'layer of complexity' can be things like catalysis or emergence generally:  https://link.springer.com/article/10.1023/A:1020672005348
Comment: @TheDoctor: Einstein proved Newton incomplete. No quantum-gravity proves Einstein incomplete (eg blackhole mysteries). We don't know the strength of gravity at small scales because it is so weak no one has figured out how to measure it, even colliding blackholes have only just become detectable. Tidal effects on particle collisions are providing insights on gravity at atomic scales. But it's not even been proven antimatter creates gravity like normal matter: https://en.wikipedia.org/wiki/Gravitational_interaction_of_antimatter
Answer: Short Answer
There is no canonical response on why total reductionism fails between any two bodies of theories, so it is only possible to offer some canonical insights into the challenges of reductionism. I'll try to put some perspective on what some paths of thought are in contemporary philosophy. Perhaps the idea that chemistry can be fully reduced to physics is inherently flawed. From the SEP article 'scientific reduction':

Conflicts between reductionist views and their dualist and pluralist metaphysical rivals have loomed large in the history of philosophy. The classical debate between materialism and mind-body dualism is simply the most familiar of many such disputes. The Cartesian dualist is committed to distinct types of substances—minds and bodies each with its own essence. Different versions of dualism—interactive, epiphenomenal, parallelist – disagree about whether the two causally interact, but they all assert the ontological difference between the mind and body. Neither reduces to the other. They are distinct, and both are real.

Long Answer
The Problem with 'Scientific Theory'
First, let us note that the question of what constitutes adequate scientific theory is a difficult notion because of the very real issues raised by scientific demarcation. What constitutes a scientific theory to be reduced is a somewhat ambiguous topic. Before you can reduce a theory, you have to circumscribe it. A gem of an example of one such metatheoretical proposition is Hempel's DN model. But, clearly, in contemporary times, confirmationism, verificationism, and falsificationism are all incomplete candidates. About all anyone seems to agree on is that there are a collection of propositions formulated in a language appealing to theory-laden observation. So, defining what can be reduced into what is a problem.
Certainly, reductionism might best be understood as an extension of the physicalist program to show that all things are not only natural but reduce to physical properties and relations. But the business of doing so is a messy business. One such contemporary attempt to elucidate such a framework is Daniel Stoljar's Physicalism. In it, he lays out the claims that physicalism is a Weltanschauung and that it is encapsulated by the claim:

(7) Physicalism is true if and only if every instantiated property is either physical or else is necessitated by some instantiated physical property.

The rest of the book is largely devoted to what it means to be necessitated by a physical property.
So, is chemistry a science that needs to be reduced to physics? If so why? Clearly, pchem is some sort of attempt to do that. But can all 'chemical properties' be reduced to purely physical properties? How does one begin to answer that question? Does that mean that philosophy of chemistry reduces to the philosophy of physics?
What is Scientific Theory? Realism and Instrumentalism
What does it even mean to reduce properties to properties? Are properties and relations objective and real as the scientific realist maintains, or are they both just artifices of language whose use is merely a tool of pragmatism, such as the instrumentalists believe?
Consider the notion of mereological nihilism (SEP). Simplified, there are simply no things that exist beyond configurations of elementary particles. Chair? No. Configuration of particles shaped as a chair. What does one make of such radical philosophical positions in regards to physical ontological questions? If there are only physical particles in configuration, has not the entire body of all theories been successfully reduced to physics? Perhaps such a position is best seen as just metaphysical speculation engaged in the elimination of linguistic entities and not actual physical ones.
The Problem with Causality
One of the world's leading experts on mathematical causality, Judea Pearl, in his book Causality has a wonderful epilogue that fleshes out how causality has advanced from Aristotle's intuitive categories to modern-day statistical notions that see causality as an upward bound on correlation. On page 427:

Many scientific discoveries have been delayed over the centuries for the lack of a mathematical language that can amplify ideas and let scientists communicate results. I am convinced that many discoveries have been delayed in our century for lack of a mathematical language that can handle causation.

Consider WP's pchem again:

Some of the relationships that physical chemistry strives to resolve include the effects of:
Intermolecular forces that act upon the physical properties of materials (plasticity, tensile strength, surface tension in liquids).
Reaction kinetics on the rate of a reaction.
The identity of ions and the electrical conductivity of materials.
Surface science and electrochemistry of cell membranes.1
Interaction of one body with another in terms of quantities of heat and work called thermodynamics.
Transfer of heat between a chemical system and its surroundings during change of phase or chemical reaction taking place called thermochemistry
Study of colligative properties of number of species present in solution.
Number of phases, number of components and degree of freedom (or variance) can be correlated with one another with help of phase rule.
Reactions of electrochemical cells.
Behaviour of microscopic systems using quantum mechanics and macroscopic systems using statistical thermodynamics.

Will all causal models in chemistry ultimately reduce to deductively certain causal models in physics? Seems a tad bit optimistic to hope for considering that physics can't even predictively causally model all physical processes. Consider the n-body problem. The more objects in the system, the more complex the system becomes. And stochastic systems can at best be approximated, never modeled with deductive certainty. Perhaps, Charles S. Pierce is right in saying that there is just inherent indeterminism in the universe that theory can never fully describe or explain. And if physical theory is essentially incomplete with regards to explanation, what does that say for theories that supervene on it or concepts that emerge from it?
Summary
Theories are linguistic structures and they are ultimately grounded in experience. And even when we build them, there's some thinking a la Quine that scientific theory is underdetermined, that is a theory can't even be decided as true absolutely. What does it even mean, then, to suggest that one theory fully reduces from another? Again from SEP on scientific reduction:

As we have seen, reduction has been conceived of as a relation that primarily holds between theories, properties, substances, concepts, or events. It has been described as an explanatory relation that can be cashed out in terms of derivation, and it could be tied to mechanistic or ontological dependence, supervenience and identity. It has been argued that theory-reduction is prior to ontological reduction; and it has been argued that ontological reduction is more fundamental than theory reduction. Reduction has been couched in structuralist, empiricist and functionalist frameworks, and it has been one of the core notions of theories defending one form or another of scientific unification. Most philosophers have abandoned this latter view. But even if the outlook for strong general reductionism, a form of reductionism that amounts to scientific as well as ontological unity might be dim, exploring the question of what it takes for a property or theory to reduce to another property or theory will help not only understand what the search for unity consists in, a search that dominated large parts of the philosophy of science in the last century; it will also help judging and understanding its rivals, such as non-reductive physicalism, versions of dualism, or even the recently defended rainforest pluralism (Wimsatt 1994). If we know the conditions imposed by the reduction relation on pairs of theories or properties, we will also know what these anti-reductionist theories are bound to deny.

Scientific theories are not legos. So it might be presumptive to think that chemical theory can be assembled out of physical theory in the same way one set of legos comprises another. Perhaps the best answer to what pieces are missing is simply to respond that the extended metaphor is fatally flawed; expecting a list of problems to be resolved to ensure total reduction might be the wrong way of thinking about the nature of reduction itself. Whatever the case, the philosophical discussion of grounding chemistry in physics continues particularly as pchem moves forward, and as long as pchem has problems, total reduction (whatever that may be) has problems.
Answer: Good question.
The reason you can't derive chemistry from physics is because the atomic model is inadequate.  Physics cannot explain why atoms exist.  What forces bring an atom together, for example?  No adequate explanation exists and these forces are responsible for what happens in many of the interactions of chemistry.  What color will emerge from an interaction between two chemicals?  They have no model.  If they had a complete model, we'd have a Grand Unified Theory, and we (they) don't.
These results need a quantum chromo-dynamics (QCD) explanation and such a complete model does not exist yet within the orthodoxy.
[Edited]
Comment: Physics exactly explains why atoms formed, in the epoch of recombination, & then in stars. Chemical reactions & bonds can be predicted with high precision, to the point the limit is only computing power: https://en.wikipedia.org/wiki/Protein_folding#Computational_studies_of_protein_folding You seem to imply QCD isn't physics.. And you basically just make strident, wrong claims, with no references of philosophical context.
Comment: "epoch of recombination" is essentially a meaningless term.  And if you use it merely to try to add something "emergent" into the question, you're wrong. Why did you choose to argue rather than inquire?  You SE-ers always attempt to play down any dissent I make about the orthodoxy.  If physics had it all explained, they wouldn't need multi-billion dollar budgets to test out their theories.
Comment: What do you find incomplete about it? The Cosmic Microwave Background is powerful evidence for it: https://en.wikipedia.org/wiki/Recombination_(cosmology) You've given a very poor attempt to address the question, that seems to amount to chemistry & physics denial, without even make a case what's wrong with them. In my answer I've listed some of the open questions requiring these budgets, & pointed out why they don't affect the validity of models at large.
Comment: Stop complaining about my answer, when you don't have one.  GOD, that's what is so annoying about SE -- all of the self-righteous moderators.  Firstly,,AH,  you know damn well that science can't even explain the origin of the Big Bang or any other model to account for exajoules of energy in this universe, so why should I bother explaining FURTHER the origin of atoms?
Comment: My answer is above.
Comment: @CriglCragl:  If science DID have the explanation of how atoms formed, why would they need multi-million dollar particle accelerators?
Comment: Higgs-Boson mainly. Also searching for dark matter candidates. And just plain curiosity, like discovering a very briefly existing tetraquark pentaquark states which may impact supernovae. There are certainly mysteries remaining. But why atoms are structured the way they are, & when & why they appeared, is not one of them.


Why does my computer freeze when I unplug my headphones?

Question: Whenever I plug in or unplug my headphones into my computer, it freezes for about 10 seconds before it is able to do anything which is rather annoying.  This didn't use to happen though, would anyone know the cause of this or how to fix it?
Comment: It might be good to give some more context: What kind of computer? What operating system (version)?
Comment: As already said by Jonik, provide more information about your system, your headphones, etc. The question is extremely generic.
Comment: Macbook pro
OS: 10.6.4
Processor: 2.53 GHz
Memory: 4GB
Headphones: V•moda
Answer: I faced a similar problem. I had a bluetooth headset and a wired one. Whenever I disconnected my wired one, sometimes the mac tried to connect to the bluetooth headset, and when it was switched off/out of range, the system would hang for sometime.
could be something similar. Can you give the list of speakers/headphones configured on your mac and their order (a screenshot would be convinient)
Comment: Is there a spot were it lists that?  Also, I dont have any bluetooth headphones, but I guess my mom does and she sits on the couch by me when we're in the living room.  Don't know if that would cause some blue tooth annoying-ness.
Comment: In "System Preferences" -> "Sound" -> "Output" tab.
See the "Input" tab also.
Answer: It is certainly not normal and it is indeed a software problem. Although low level drivers are normally for all users, it wouldn't harm if you try this either under a new fresh user (just create one, reboot your computer, log in with the new one and do the plug-unplug thing).
Also, you can try rebooting and plugging-unplugging in the login screen. If it also hangs there, then there's certainly something "odd" about your sound drivers and further investigation (and/or system logs analysis) will be required.
If you have a spare drive, and to make sure there isn't anything hardware related, you might want to try to install a fresh copy of your OS into that drive and see if it happens there, on the fresh install.
There might be a problem with the headphone's internal hard/soft as well, so testing on another machine wouldn't hurt to rule out an external problem.
Answer: Happens the same in my MacBook, but I think it is because it is switching the software (OS) to send the sound signal to the speakers or the headphones plug. In my MacBooks takes 3 seconds, maybe in yours is more (10 seconds) because you have a lot of software running in the background.
Comment: On my previous MacBook Pro it was instantaneous, now on this one it takes a few seconds to switch the sound output from the internal speakers to the headphones, but it doesn't freeze the computer in the while.


Get Characteristic Function from MGF?

Question: I've been calculating characteristic functions and MGF's and was wondering whether we can always get the characteristic function simply by substituting $it$ instead of $t$ in the resulting equation. For example the normal MGF is
$
\exp{(\mu t + \frac{\sigma^2 t^2}{2})}
$
but if we substitute $it$ into that we get $
\exp{(\mu it - \frac{\sigma^2 t^2}{2})}
$ which is the characteristic function.
Can we always do this if the MGF exists in a neighborhood of 0?
Answer: You are correct about the link between these two functions, provided the MGF exists. This was discussed in another question in this site: Link between moment-generating function and characteristic function

Get Characteristic Function from MGF?


How to see your 'old' professions?

Question: Is there a way to know which professions a character previously had on which level?
Comment: Are you referring to seeing the "base" professions as opposed to specializations, e.g. Guardian instead of Dragonhunter?
Answer: Professions are your class in GW2. If you mean your crafting disciplines (referred to as "professions" in other games), the answer is that it shows on the login screen, when you mouseover your character.

On my first character in this image, my active crafting disciplines are shown as icons on my character (Weaponsmith, Tailor, and Artificer), but in the mouseover tooltip, it also shows Jeweler, which is an inactive crafting discipline on this character.
Comment: @zozo It's a Tempest, the Elite Specialization for Elementalists.
Comment: What is your first character there? I don't recognize the symbol.
Comment: This doesn't seem to be the case for me. I started playing again this weekend after years of inactivity, and my old characters have no crafting disciplines listed whatsoever (but definite. **However** I haven't logged in with them yet, maybe that gets refreshed after logging in. As a software developer, it makes some sense that the new features of showing crafting disciplines on the character select screen will only be triggered by logging in (or even more logically, logging OUT) with the character.

How to see your 'old' professions?


Is the discoloration on my bed frame rails mold?

Question: So the wood panels lay on top of the metal beams, it's a bed frame but I noticed these spots on the metal and rubbed them off and smelt like rotten eggs please let me know
Comment: Probably the other question should be closed since this one has more detail.
Answer: This was already answered hours ago by another one of our more experienced users, when you asked this under a different username:
Is this mold? Black? White?
Honestly, it appears to be simple iron staining. Appears black on wood, white on zinc coatings. Simple reaction of carbons in the metal and tannins in the wood. Although it is activated by moisture, it is not a mold, nor harmful.
Comment: This seems a more plausible answer to me than the other one...
Comment: The steel marks (iron oxide stains) on wood can be [removed with acids](http://www.instructables.com/id/Removing-Black-Stains-in-Wood-Furniture-With-Oxali/) (or sanding) although it's hardly worth the hassle in this case.
Answer: Yes, the wood (first picture) is iron-stained as NPM says, although the chemists say that simple mechanical action (rubbing) is what caused it.
The 2nd picture to me shows corroded galvanized steel (steel elecotplated with zinc). The outer white circles are white rust (corroded zinc), while the inner disks are exposed (and possibly slightly corroded) steel.
The two process are probably related by the wood slats rubbing on the galvanized steel frame when you move in bed etc.


Ratio of Water to Isopropyl Alcohol?

Question: I am looking for a very simple formula to calculate the ratio of water to 99% isopropyl alcohol that I need to prepare a solution with a density of 0.925 g/cm3. I am assuming there is one.
Comment: You'll also need a specific temperature.
Answer: If performing an interpolation still passes as «simple» for you, you may find experimentally determined values about iPrOH in water tabulated for example in journals about (chemical) engineering.  The previews of this and this give you about 10 entries for free (which may be complemented by the entries in the CRC Handbook of Chemistry and Physics like here).  If more data are required, a more recent here suggests going to a library and having a look into Ullmann's Encyclopedia, too.
While keeping an eye on the temperature of the two, determine the volume of water and iPrOH to mix in separate containers in advance (1L of water and 1L of iPrOH don't add up to 2L of blend).
Answer: There are no formula for calculating the density of a mixture water + any alcohol as a function of the proportion of water.
Comment: No theoretical formula, but an empirical one fit to experimental data might exist.

Ratio of Water to Isopropyl Alcohol?


How does one reliably test core Linux software (like package managers, UNIX utils, desktop environments, kernel modules, etc.)?

Question: If I'm not mistaken, a major part of testing software is deploying and using it on your platform of choice. For critical system software such as UNIX utils, package managers, desktop environments/window managers, kernel modules, etc., how would a developer reliably test the software without bricking their development environment?
Comment: If you're worried about bricking something (like a windowing system), you can run software in places other than your development environment.  Like a machine dedicated for that.  Or in a VM, or partitioned into a sandbox.  Things like package managers are typically configurable to use alternate paths for installation, caches, etc, so they can live reasonably isolated.
Comment: @coteyr: some community members on this site close vote and downvote almost every question which shows not much research effort, or is very broad. Said that, this question clearly fulfills both criteria (zero research effort, and extremely broad). However, since it is one of those broad questions which can be answered in an equally broad and comprehensive way, I consider not to vote at all.
Comment: Two words: virtual machine.
Comment: Not sure why it's downvoted...
Comment: I always found that odd about this SE. The questions are supposed to be broad enough that they help everyone. Yet if it's too broad, the question gets a downvote. Ask to narrow a question, and it gets a vote to close.  This is a valid question if a bit "high level".
Comment: @coteyr: don't overthink this, these are only the votes of a very small group, where I am convinced at least one or two of those persons have some kind of obsessive-compulsive disorder (but I am not a psychologists). They see some buzzword or some minor indication of violation of the site rules, and then they feel they must downvote and / or close-vote. And you cannot argue with such persons.
Answer: You typically use a test environment that can easily be reset or rebuilt to a good state. There are different possibilities: Dedicated real machines (normally required for testing drivers), virtual machines, containers, emulators, separate filesystem paths, etc.
If you're adventurous and fully trust your backup process, you may of course test your developed software in your development environment, and restore it from backup if it breaks beyond repair. However, this is most often not the most efficient approach, as you would need to perform backups frequently so you don't lose too much.
Comment: Just changing to a tempdir is good enough for most software. But the Unix ecosystem has had ultra-basic containerization in the form of chroot() for the last 45 years. For example, the Debian build system uses chroot-environments to isolate dependencies.
Answer: The answer to the question is not really OS-dependent, but at the same time, it is. For example, on Linux, you can test the window manager, and if it breaks, you don't have to reboot, whereas on Windows, there is much less separation.
That said, you test the tools much the same way as any software. Let's say you need to test a cp command line tool. You don't need to test that you actually copy a file. If you can mock things so that you are testing what fopen is called etc., that is generally "good enough" for general testing.
For lower-level things like drivers, there is nothing for it but to have a real machine with real hardware and test against that.
If you are looking at something much higher level, like "packaging tools," you still do the same thing. General testing can be done by mocking things (one way or another). For example, if you are writing apt, You might create a repo with only one package and a small manifest. Then run your tests against that.
In the end, while the toolchain will change, the concept is the same.

Test the most atomic thing you can.
Trust the libraries that are not yours (or gain trust) (no need to test fopen that's not part of your project)
Don't test 1s and 0s written to disk, test that the input produced the desired output (can save and read a file, not 101100110100 was written to sector 3)
Fake the rest. (VM, different directory, mock objects, etc)


Is it possible to brute force the missing part of my Ethereum private key?

Question: I'm missing 22 characters out of 64 characters of my private key. I'm pretty sure I also know the first four, leaving out only 18 characters to find. Are there any ways to brute force this against my address?
Answer: 18 hex characters is 72 bits, which gives 4722366482869645213696 possible combinations.
A powerful bitcoin miner that you could buy is capable of calculating a hashrate of 100 TH/s or 1000000000000 hashes per second, which would take up to 150 years to find the key, and a tremendous amount of energy, not to mention having to check each key against the blockchain.
Although if you had the entire bitcoin network hash rate at your disposal, it would only take about 15 seconds. So sorry but I'm afraid it's not possible at this time.
Answer: Well, each character could be one of 16 valid hex numbers (0-9/a-f), if you're missing 18 characters, you potentially have 16^18 combinations to try, that's about 5 quintillions. assuming you can test a billion a second (which is already extremely optimistic), that'd take about 150 000 years. So unfortunately, your answer is most likely no
Answer: Yes, but not in a reasonable amount of time.
You are not brute-forcing anything technically, and you don't need to check the blockchain, but you need to compute a public key from the private one and comparing the result with the public key you already know. If you don't have the public key and only have the address, the process takes much longer (you have to hash the public key to obtain the address and then compare your address).
Suppose we assume the best case (you have the public key), taking a fast lib (i.e., https://github.com/paritytech/libsecp256k1), a fast consumer general purpose processor (i.e., i7-12700k, 400k key per second), and your key in the middle of the random set of key and not the last one you check (i.e., 2722366482869645213696 keys). In that case, finding the key takes around 215 million years.
Dedicated hardware can increase substantially the computation power for specific tasks, but even with, let's say, an x1000 performance increase, you still need to wait 215.000 years — or 215 years if you use 1000 devices in parallel.


Derailleur Hanger Replacement

Question: I have just replaced my derailleur hanger but it wont tighten properly the back keeps spinning so it never gets fully tight?
Is there a special tool I need or is it meant to be like this?
Comment: It's impossible to say without some details.
Comment: A couple of photos of your hanger in place might help here.  I guess its a claw hanger that goes in the same slot as the axle ?  Or is it a proprietary hanger that only suits your brand of bike frame ?
Comment: Are you asking about the bolt? The recessed "nut" part usually has a slot. You should be able to use a flat screw driver to prevent it from rotating.
Comment: I'll note that I've had good luck buying hangers through [Wheels Manufacturing](https://wheelsmfg.com/).
Answer: Remove the bolt and nut. Make sure the nut and the area where it seats are clean and degreased. Then ideally using grease (oil is also okay), lubricate just the threads of the nut without getting any lubricant on the contact are between the nut and the frame. Usually that is all it takes to prevent the nut from spinning as you tighten the bolt, or you may have to press on it from the back with your finger as you tighten. You can use a stubby screwdriver or chainring nut tool, but it's not usually necessary on hanger bolts that have been lubricated, and you want the lube anyway so you can remove it later.
If you're still having trouble, a little "wrench English" can help: with your wrench, tilt the bolt off-axis as you're turning it in. You're trying to get the sleeve nut to "catch" and hold still as you turn the bolt.
You do want it tight so that it stays put as best as possible. That said, usually and for most designs the quick release will do that all by itself, so in practice it doesn't really matter most of the time.
Answer: Additionally, you can purchase chainring nuts that have serrations/teeth on the nut, so they won't spin hopelessly as you try to tighten them down.

Derailleur Hanger Replacement


Run a vim macro from inside a bash script

Question: In a bash script I want to edit a file. Is there a way to write a macro in vim, save it and then call it in bash script?
Answer: No problem:
Create the macro file with vi commands. For example:
:1,$s/a/b/g
:wq

Then run vi -s macro_file data_file
One advice: Try to use sed/awk and instead of vi for this purpose
Comment: And there is the equivalent but more obscure `+` syntax as in: `vim +PluginInstall +qall` for [installing Vundle](https://github.com/gmarik/Vundle.vim).
Answer: This is an example of the use of vim to create HTML versions of files with vim :
for f in *.c; do
    vim -f +"syn on" +"run! syntax/2html.vim" +"wq" +"q" "$f"
done

Answer: You could use ex, which comes with vim and is the command line tool for vim.
Eran's answer is definitely correct, but I'd personally use ex -c (vim -c is also possible):
> echo asdf > blub
> ex blub -c "normal a_" -c "normal l.l.l." -c wq
> cat blub
a_s_d_f_

This is my preferred way, because there are not multiple files.
If you remap escape to jj (for example), you can easily make more complicated normal statements:
ex file -c "normal a_jjl.l.l." -c wq

For your information:

normal sends letters in normal mode to
wq quits the program
to send multiple commands in one string (no additional -c option), use the pipe character |. But: This does not work in the case of normal.
This works only for 10 -c's. But for more commands, I would recommend to create a file.


Latent Space of VAE

Question: Say I have a dataset $D = \{a_1,a_2,a_3,...a_n\}$ on which I train a basic variational autoencoder (VAE) on (a couple of fully connected layers separated by nonlinearities). Does the latent space of the VAE have a feature coordinate for $a_1 + a_2$ and $a_1 - a_2$ (which are not part of dataset $D$)?
Answer: It depends. Let $X$ be the domain of the data, i.e. $a_i\in X$. $D$ is then a sample from $X$, following some distribution $P$. The point of the VAE is to model this distribution of the data such that we can sample from it. So, parts of the dataspace $X$ will be modelled well (where there are many training examples) and others poorly.
So the question is whether $a_i+a_j$ or $a_k - a_\ell$ actually form "reasonable" (i.e., not too low density from $P$) values. Of course,
since it seems $X=\mathbb{R}^d$ in your case,
the VAE will have no problem encoding or decoding these values; the issue comes in whether the latent encodings will be useful or sensible. They need not necessarily be in the dataset, but they can't be too far away from it.
(This problem area is called domain adaptation in general). In other words, it will have "feature coordinates" for $X$ being an unbounded vector space, but whether they are useful or sensible depends on the situation.
For example, suppose $X$ is the natural images and all your $a_i$'s are images of sunflowers. You can perform $a_\alpha + a_\beta = c$ in a pixel-wise manner, but the encoding of $c$ is unlikely to have a reasonable latent representation. However, if $a_i$ is something like an embedding from a word model then it may be just fine.
Note that it's often a good idea to evaluate reconstruction performance of a VAE on a held-out test set (outside the training data, but still from the same or similar $P$).
So, I'd ask myself this: could $a_i + a_j$ be reasonably considered part of a test set? If so, then yes, the latent embeddings will probably be ok too.
Answer: I think you are interested in knowing whether the model learns distributions of linear combinations of the input data.
In the special case of Gaussian latent space, the encoder estimates the mean $\mu_1$, $\mu_2$ and variance $\sigma_1^2$, $\sigma_2^2$ from $a_1$ and $a_2$. Then we sample from the two distributions $N(\mu_1, \sigma_1^2)$ and $N(\mu_2, \sigma_2^2)$.
If $a_1$ and $a_2$ are independent, their sum (difference) is also normally distributed,  with mean equal to the sum (difference) of their means, and variance equal to the sum of the variances. So, in this case, you can easily extrapolate points that represent linear combinations of the observations from the parameters of their distributions.
If they are not independent, their linear combination is still normally distributed, but you need to take into account of their covariance.
Does VAE learn these directly from the observations? Not true in general.

Latent Space of VAE