Delete performance for LOB data in SQL Server

Question: This question is related to this forum thread.
Running SQL Server 2008 Developer Edition on my workstation and an Enterprise Edition two-node virtual machine cluster where I refer to "alpha cluster".
The time it takes to delete rows with a varbinary(max) column is directly related to the length of the data in that column.  That may sound intuitive at first, but after investigation, it clashes with my understanding of how SQL Server actually deletes rows in general and deals with this kind of data.
The problem stems from a delete timeout (> 30 seconds) issue we are seeing in our .NET web application, but I have simplified it for the sake of this discussion.
When a record is deleted, SQL Server marks it as a ghost to be cleaned up by a Ghost Cleanup Task at a later time after the transaction commits (see Paul Randal's blog).  In a test deleting three rows with 16 KB, 4 MB, and 50 MB data in a varbinary(max) column, respectively, I see this happening on the page with the in-row portion of the data, as well as in the transaction log.
What seems odd to me is that X locks are placed on all of the LOB data pages during the delete, and the pages are deallocated in the PFS.  I see this in the transaction log, as well as with sp_lock and the results of the dm_db_index_operational_stats DMV (page_lock_count).
This creates an I/O bottleneck on my workstation and our alpha cluster if those pages are not already in the buffer cache.  In fact, the page_io_latch_wait_in_ms from the same DMV is practically the entire duration of the delete, and the page_io_latch_wait_count corresponds with the number of locked pages.  For the 50 MB file on my workstation, this translates to over 3 seconds when starting with an empty buffer cache (checkpoint / dbcc dropcleanbuffers), and I have no doubt it would be longer for heavy fragmentation and under load.
I tried to make sure that it wasn't just allocating space in the cache taking up that time.  I read in 2 GB of data from other rows before executing the delete instead of the checkpoint method, which is more than is allotted to the SQL Server process.  Not sure if that is a valid test or not, since I don't know how SQL Server shuffles the data around.  I assumed it would always push out the old in favor of the new.
Further, it doesn't even modify the pages.  This I can see with dm_os_buffer_descriptors.  The pages are clean after the delete, while the number of modified pages is less than 20 for all three small, medium, and large deletes.  I also compared output of DBCC PAGE for a sampling of the looked up pages, and there were no changes (only the ALLOCATED bit was removed from PFS).  It just deallocates them.
To further prove that the page lookups/deallocations are causing the issue, I tried the same test using a filestream column instead of vanilla varbinary(max).  The deletes were constant time, regardless of LOB size.
So, first my academic questions:

Why does SQL Server need to lookup all of the LOB data pages in order to X lock them?  Is that just a detail of how locks are represented in memory (stored with the page somehow)?  This makes the I/O impact depend strongly on data size if not completely cached.
Why the X locks at all, just to deallocate them?  Isn't it sufficient to lock just the index leaf with the in-row portion, since the deallocation does not need to modify the pages themselves?  Is there some other way to get at the LOB data that the lock protects against?
Why deallocate the pages up front at all, given that there is already a background task dedicated to this type of work?

And maybe more important, my practical question:

Is there any way to make deletes operate differently?  My goal is constant time deletes regardless of size, similar to filestream, where any cleanup happens in the background after the fact.  Is it a configuration thing?  Am I storing things strangely?

Here is how to reproduce the described test (executed through SSMS query window):
CREATE TABLE [T] (
    [ID] [uniqueidentifier] NOT NULL PRIMARY KEY,
    [Data] [varbinary](max) NULL
)

DECLARE @SmallID uniqueidentifier
DECLARE @MediumID uniqueidentifier
DECLARE @LargeID uniqueidentifier

SELECT @SmallID = NEWID(), @MediumID = NEWID(), @LargeID = NEWID()
-- May want to keep these IDs somewhere so you can use them in the deletes without var declaration

INSERT INTO [T] VALUES (@SmallID, CAST(REPLICATE(CAST('a' AS varchar(max)), 16 * 1024) AS varbinary(max)))
INSERT INTO [T] VALUES (@MediumID, CAST(REPLICATE(CAST('a' AS varchar(max)), 4 * 1024 * 1024) AS varbinary(max)))
INSERT INTO [T] VALUES (@LargeID, CAST(REPLICATE(CAST('a' AS varchar(max)), 50 * 1024 * 1024) AS varbinary(max)))

-- Do this before test
CHECKPOINT
DBCC DROPCLEANBUFFERS
BEGIN TRAN

-- Do one of these deletes to measure results or profile
DELETE FROM [T] WHERE ID = @SmallID
DELETE FROM [T] WHERE ID = @MediumID
DELETE FROM [T] WHERE ID = @LargeID

-- Do this after test
ROLLBACK

Here are some results from profiling the deletes on my workstation:

| Column Type | Delete Size | Duration (ms) | Reads | Writes | CPU |
--------------------------------------------------------------------
| VarBinary   | 16 KB       | 40            | 13    | 2      | 0   |
| VarBinary   | 4 MB        | 952           | 2318  | 2      | 0   |
| VarBinary   | 50 MB       | 2976          | 28594 | 1      | 62  |
--------------------------------------------------------------------
| FileStream  | 16 KB       | 1             | 12    | 1      | 0   |
| FileStream  | 4 MB        | 0             | 9     | 0      | 0   |
| FileStream  | 50 MB       | 1             | 9     | 0      | 0   |

We can't necessarily just use filestream instead because:

Our data size distribution doesn't warrant it.
In practice, we add data in many chunks, and filestream doesn't support partial updates.  We would need to design around this.

Update 1
Tested a theory that the data is being written to the transaction log as part of the delete, and this does not seem to be the case.  Am I testing for this incorrectly?  See below.
SELECT MAX([Current LSN]) FROM fn_dblog(NULL, NULL)
--0000002f:000001d9:0001

BEGIN TRAN
DELETE FROM [T] WHERE ID = @ID

SELECT
    SUM(
        DATALENGTH([RowLog Contents 0]) +
        DATALENGTH([RowLog Contents 1]) +
        DATALENGTH([RowLog Contents 3]) +
        DATALENGTH([RowLog Contents 4])
    ) [RowLog Contents Total],
    SUM(
        DATALENGTH([Log Record])
    ) [Log Record Total]
FROM fn_dblog(NULL, NULL)
WHERE [Current LSN] > '0000002f:000001d9:0001'

For a file over 5 MB in size, this returned 1651 | 171860.
Further, I would expect the pages themselves to be dirty if data were written to the log.  Only the deallocations seem to be logged, which matches what is dirty after the delete.
Update 2
I did get a response back from Paul Randal.  He affirmed the fact that it has to read all of the pages in order to traverse the tree and find which pages to deallocate, and stated that there is no other way to look up which pages.  This is a half answer to 1 & 2 (though doesn't explain the need for locks on out-of-row data, but that is small potatoes).
Question 3 is still open: Why deallocate the pages up front if there is already a background task to do cleanup for deletes?
And of course, the all important question: Is there a way to directly mitigate (i.e. not work around) this size-dependent delete behavior?  I would think this would be a more common issue, unless we're really the only ones storing and deleting 50 MB rows in SQL Server?  Does everyone else out there work around this with some form of a garbage collection job?
Comment: I wish there were a better solution, but haven't found one. I have a situation of logging large volumes of varying size rows, up to 1MB+, and I have a "purge" process to delete old records. Because the deletes were so slow, I had to split it into two steps - first remove the references between tables (which is very fast), then delete orphaned rows. The delete job averaged ~2.2 seconds / MB to delete data. So of course I had to reduce contention, so I have a stored procedure with "DELETE TOP (250)" inside a loop until no rows are deleted anymore.
Answer: I can't say why exactly it would be so much more inefficient to delete a VARBINARY(MAX) compared to file stream but one idea you could consider if you're just trying to avoid time outs from your web application when deleting these LOBS. You could store the VARBINARY(MAX) values in a separate table (lets call it tblLOB) that is referenced by the original table (lets call this tblParent).
From here when you delete a record you can just delete it from the parent record and then have an occasional garbage collection process to go in and clean up the records in the LOB table. There may be additional hard drive activity during this garbage collection process but it will at least be separate from the front end web and can be performed during non-peak times.
Comment: I would be cautious of having just a time stamp to indicate it is deleted. That will work but then you will eventually have a lot of used space occupied by in active rows. You will need to have some kind of gc process at some point, depending on how much is deleted, and it will be less impacting to delete less on a regular basis rather then lots on an occasional basis.
Comment: Thanks.  That is exactly one of our options on the board.  The table is a file system, and we are currently in the process of separating the binary data out to a completely separate database from the hierarchy meta.  We could either do as you said and delete the hierarchy row, and have a GC process clean up orphaned LOB rows.  Or have a delete timestamp with the data to accomplish the same goal.  This is the route we may take if there is no satisfying answer to the problem.


Chamar método ao fechar dialog

Question: Este código é o do showcase do primefaces :
<h:panelGrid columns="1" cellpadding="5">

</h:panelGrid>

    <h:outputText value="Resistance to PrimeFaces is futile!" />

    <h:outputText value="This is a Modal Dialog." />

    <h:outputText value="This dialog has nice effects." />

Coloquei pra ajudar no entendimento. Eu gostaria de saber como posso invocar um método caso eu feche o dialog pelo 'x' no canto superior direito como está na imagem.
Answer: Na documentação do Primefaces Dialog (primefaces_user_guide.pdf) tem um exemplo:

  //Content

.
public class DialogBean {
  public void handleClose(CloseEvent event) {
    //Add facesmessage
  }
}

Comment: Show, muito obrigado.

Chamar método ao fechar dialog


Understanding predicativity

Question: In understanding the differences between impredicative and predicative definitions, I was able to understand impredicative as the following
A definition is said to be impredicative if it invokes over the set being defined or another set which contains the thing being defined. A prime example of this definition is the Russell's paradox
Now comparing to predicative definition, reading wiki it says that it entails constructing theories where quantification over lower levels results in variables of some new type, distinguished from the lower types that the variable ranges over.
The definition for predicative seems to be on a whole new level in terms of the description. An example of an predicative definition I sort of tried to connect the description with was Frege's first and second order calculus.
Could anyone perhaps offer a more simpler definition of a predicative definition, along with an example?
Thanks!
Answer: The set of even numbers has a predicative definition: a number $n$ is even if and only if there is a number $k$ with $k = 2n$. That definition does not quantify over sets at all, just over numbers.   More generally, a definition of a set of natural numbers which only quantifies over numbers, and not over sets, is usually seen as predicative.
The term "predicative", however, does not have a completely clear and formal definition.  There are some mathematical systems that are often thought of as "predicative" and others that are usually thought of as "impredicative", but the dividing line is not clear. This is similar to the definition of "constructive" - there is no firm definition of what makes a mathematical system constructive.
So, rather than learning about predicativity/impredicativity from definitions, you have learn about it from examples.  Like "beauty", "constructivism", and "elegance", the idea of "predicativity" is more suited for philosophy of mathematics (or "metamathematics" in the broader sense) than for formal mathematics.   However, as is common in the current approach, we can use formal results of mathematical logic to help clarify what is going on with the concept of predicativity, just as we can for constructivism. (Beauty and elegance are harder for us to explain formally with the current state of knowledge.)
Answer: As Professor Mummert has noted, the notion of a "predicative definition" is vague, although I would disagree that the same holds for "predicative mathematics".  There are many complicated issues involved.
With respect to "definition", is it "obvious" that mathematics ought to be based upon "undefined primitives"?  Russell and Whitehead made such a claim.  You will find a detailed analysis with criticism of "Principia Mathematica" in the book $\underline{Definition}$ by Richard Robinson.  Among the kinds of definitions one finds in non-foundational mathematics is "implicit definition".  And, you will find that Professor Robinson does discuss them as legitimate forms of mathematical definition.  When you think about the matter closely, you will realize that the "intensional definition" -- upon which Church introduced the lambda calculus -- is, in fact, a variation of implicit definition.  The functions which Church introduced may be applied to themselves.  Such functions are not representable in Zermelo-Fraenkel set theory because the axiom of foundation restricts that notion of set to being well-founded.  Thus, the extension of a function in the sense of what Church did (that is, its representation as a set of ordered pairs) would have to appear as a domain element of the function.  The axiom of foundation restricts against this infinite descending chains of membership relations.
Now, consider the definition,
$$\forall x \forall y ( x \subset y \leftrightarrow
( \forall z ( y \subset z \rightarrow x \subset z )
\wedge
\exists z ( x \subset z \wedge \neg y \subset z ) ) )$$
I use this form of sentence for both the set theory and the arithmetic (interpreted as proper divisor) in which I am interested.  The syntax is clearly circular.  Is it an impredicative definition?
According to a monograph by Moschovakis, a sentence of this nature appears to be impredicative if one naively attributes it to be a second-order sentence but is, in fact, recursively constructive.  And, indeed, you will find a sentence of this form used in $\underline{Set Theory}$ by Kunen in his discussion of forcing.  By contrast, the full-blown transfinite recursion is presented by Jech in the first edition of his book $\underline{Set Theory}$.
When I say that "predicative mathematics" does not suffer from the same problem as "predicative definition", it is because it originates with Russell and Whitehead with the express purpose of avoiding the circularity which they believed responsible for the many early paradoxes in set theory.  So, one understands sets, first and foremost, as collections of individuals which are not, themselves, a collection.  Then, one may form additional sets from those individuals and those initial sets of individuals.  The next "type" will be sets formed of "objects" previously obtained through "set formation".  I apologize for finishing with all of these quotes.  But, in natural language it gets complicated.  In combination with the axioms of union and power set, the axiom of foundation provides for this structure.
This kind of distinction may be found in Aristotle.  For Aristotle, individuals are primary substance.  Notions such as "species" and "genus" are substances in the sense that what they categorize are individuals.  But, Aristotle refers to them as secondary substances.
One of the interesting things one discovers when reading Aristotle is that his only admonition against circularity is that against trying to simultaneously attribute truth to deductive reasoning and inductive reasoning at the same time.  In modern mathematics, this seems to be related to the Lyndon interpolation theorem.  The proof of that theorem uses negation normal forms.  The significance of this is the restricted second-order language presented by Flum and Ziegler in the early 1980's.  Its formation rules are governed by negation normal forms while its semantics coincide with first-order semantics on trivial topologies and discrete topologies.  It is clear that predicative mathematics will avoid invoking both the universal quantifier and the existential quantifier simultaneously.  It emphasizes the existential quantifier as being semantically prior to the universal quantifier.  But, without some accommodation to the logic, the syntactic definition of individuals (as opposed to relations) merely on the basis of "properties" puts one at risk of attributing truth to both the universal quantifier and the existential quantifier simultaneously.
This is what the distinction between "predicative definition" and "impredicative definition" is trying to restrict.  But, it is not at all clear that a classification of definitions is the appropriate vehicle.  What is at stake is the claim that "mathematics is extensional" and the interpretation of quantifiers as collections which are objects.  The circularity of intensional definitions and recursive definitions does not seem to always lead to paradox.
Comment: A lot of reference to unfamiliar topics makes it difficult to comprehend this comment. I do wish that it lead to more discussion.

Understanding predicativity


moving a mysql slave to a new host : Failed to open the relay log

Question: I have a master-master setup which I want to move to a new pair of servers (from linux to opensolaris).
linux boxes names are shard1-1 and shard1-2, and osol boxes names are shard1-1-osol and shard1-2-osol.
this is what I did:
 1. took down mysql server on both linux boxes, copied the data to the osol boxes.
 2. edited master.info on each of the osol boxes, changed master from shard1-1 to shard1-1-osol and vice versa on both machines.
when I started mysql on the first osol machine, I see this in the error log:
looks like mysql does not like the host name change, and it's bitching about it.
the mysqld-relay-bin.000707 file exists in the datadir, so it's not clear what mysql is whining about.
<code>090902  2:26:03  InnoDB: Started; log sequence number 246 3528553673
090902  2:26:03 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--relay-log=shard1-1-osol-relay-bin' to avoid this problem.
090902  2:26:03 [ERROR] Failed to open the relay log './mysqld-relay-bin.000707' (relay_log_pos 19653396)
090902  2:26:03 [ERROR] Could not find target log during relay log initialization
090902  2:26:03 [ERROR] Failed to initialize the master info structure
</code>
Answer: I have to disagree to an extent. You can rename the log files and update the relay-log.info,  relevant .index file and/or set 'log-bin' set in my.cnf to make it work.
Answer: The server can't find the relay logs because they are named based in part upon the host name.  You are going to have ALL KINDS of problems trying to bring the slave up using that technique.  I suggest reinitializing the slave from the new master.  Just drop all the databases on the slave, take a dump from the master using mysqldump, import it onto the slave and then use CHANGE MASTER TO in order to set up replication from the new master.  It is WAY easier then trying to troubleshoot all the problems you are going to run into.
Comment: that's exactly what I ended up doing already.
Answer: A useful guide to a problem which has similar effects to yours is Watch out for hostname changes when using replication!, by Trent from Sun/MySQL.
Comment: thanks, also pretty useful.
I took a more brute force approach for restarting the replication - I simply nuked the relay logs and index and started clean.
Answer: If it is just complaining about the relay logs, in most cases, they are disposable if the master still has the binary logs around.  You can just run CHANGE MASTER TO on the slave  and it will flush the existing relay logs and start anew.  You don't need to make a new fresh copy.


SQL Server Snapshot on live DB

Question: I am in need of some help.I want to reinitialize one of my subscribers with a new snapshot.my previous snapshot i generated was when activity was low on the production database.It took under 2 minutes.
My question is,can i generate a new snapshot during the day when the applications are using the database live? would it lock my tables badly to where transaction wont be followed to the database?
Answer: Yes, you can create initial snapshot during the day.
With transactional replication Snapshot Agent takes locks only during the initial phase of snapshot generation.
Performance impact of this operation depends on current workload of your system. So, if previous snapshot generation took 2 minutes, I think I will take not much more time.

SQL Server Snapshot on live DB


DataContractJsonSerializer deserializing List throwing error

Question: I have a custom Exception :
[Serializable]
public class MyCustomException : Exception
{
    public List<ErrorInfo> ErrorInfoList { get; set; }

    protected MyCustomException (SerializationInfo info, StreamingContext context)
        : base(info, context)
    {
        this.ErrorInfoList = (List<ErrorInfo>)info.GetValue("ErrorInfoList", typeof(List<ErrorInfo>));
    }

    [SecurityPermissionAttribute(SecurityAction.Demand, SerializationFormatter = true)]
    public override void GetObjectData(SerializationInfo info, StreamingContext context)
    {
        if (info == null)
        {
            throw new ArgumentNullException("info");
        }

        info.AddValue("ErrorInfoList ", this.ErrorInfoList, typeof(List<ErrorInfo>));

        base.GetObjectData(info, context);
    }
}

Whenever it tries to deserialize, this line throws an "Object must implement IConvertible" exception: (List<ErrorInfo>)info.GetValue("ErrorInfoList", typeof(List<ErrorInfo>))
Here's the bit of code that does the serialization:
using(MemoryStream memStm = new MemoryStream())
{
    XmlObjectSerializer ser = new DataContractJsonSerializer(
        typeof(MyCustomException),
        new Type[] {
            typeof(List<ErrorInfo>),
            typeof(ErrorInfo)
        }
    );

    ser.WriteObject(memStm, (MyCustomException)context.Exception);
    memStm.Seek(0, SeekOrigin.Begin);
    using (StreamReader streamReader = new StreamReader(memStm))
    {
        response.Content = new StringContent(streamReader.ReadToEnd());
    }
}

Here's the bit of code that does the deserialization:
using(MemoryStream memStm = new MemoryStream(response.Content.ReadAsByteArrayAsync().Result))
{
    DataContractJsonSerializer deserializer = new DataContractJsonSerializer(
        typeof(MyCustomException),
        new Type[] {
            typeof(List<ErrorInfo>),
            typeof(ErrorInfo)
        }
    );
    UserPortalException upEx = (UserPortalException)deserializer.ReadObject(memStm);
    throw upEx;
}

Here's the code for the ErrorInfo class:
[Serializable]
public class ErrorInfo : ISerializable
{
    public enum ErrorCode {
        [.....]
    }

    public ErrorCode Code { get; set; }

    public void GetObjectData(SerializationInfo info, StreamingContext context)
    {
        info.AddValue("Code", this.Code , typeof(ErrorCode ));
    }

    public Error(SerializationInfo info, StreamingContext context)
    {
        this.Code = (ErrorCode)Enum.Parse(typeof(ErrorCode), info.GetInt32("Code").ToString());
    }
}

Comment: The error being thrown is pretty self-explanatory or is there another question here?
Comment: I don't understand how to fix the exception... Which class needs to implement IConvertible?
Comment: Where's the code for the ErrorInfo class?
Comment: I just added it to the question.
Answer: The basic problem here is is that the ISerializable interface was originally designed (in .Net 1) to work with BinaryFormatter.  And, while BinaryFormatter serialization streams contain complete type information, JSON is weakly typed.  This causes problems described in Stand-Alone JSON Serialization:

Supported and Unsupported ISerializable Types
In general, types that implement the ISerializable interface are fully supported when serializing/deserializing JSON. However, some of these types (including some .NET Framework types) are implemented in such a way that the JSON-specific serialization aspects cause them to not deserialize correctly:

With ISerializable, the type of individual data members is never known in advance. This leads to a polymorphic situation similar to deserializing types into an object. As mentioned before, this may lead to loss of type information in JSON. For example, a type that serializes an enum in its ISerializable implementation and attempts to deserialize back directly into an enum (without proper casts) fails, because an enum is serialized using numbers in JSON and JSON numbers deserialize into built-in .NET numeric types (Int32, Decimal or Double). So the fact that the number used to be an enum value is lost.

What you are experiencing is just such a loss of type information.  If you look at the JSON generated for your custom exception, you will see:
{"ErrorInfoList":[{"__type":"ErrorInfo:#Question40048102","Code":0}],"ClassName":"Question40048102.MyCustomException","Message":null,"Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":null,"RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":null}

There is a "__type" type hint for each ErrorInfo, but no type hint for the ErrorInfoList, because DataContractJsonSerializer does not support type hints for collections.  Thus, the ErrorInfoList gets deserialized as an object [] array containing ErrorInfo objects rather than a List<ErrorInfo>, leading to the error you see.
So, in principle, you could change your initialization of ErrorInfoList as follows:
this.ErrorInfoList = ((IEnumerable<object>)info.GetValue("ErrorInfoList", typeof(object []))).Cast<ErrorInfo>().ToList();

However, this would break binary and XML data contract deserialization where the entry value is already typed correctly.  It would also break Json.NET deserialization which uses a completely different mechanism, namely storing JToken values inside the SerializationInfo and deserializing on demand using a custom IFormatterConverter.
Thus a bit of code smell is required to support all of the above serializers:
[Serializable]
[KnownType(typeof(List<ErrorInfo>))]
[KnownType(typeof(ErrorInfo))]
public class MyCustomException : Exception
{
    public List<ErrorInfo> ErrorInfoList { get; set; }

    public MyCustomException()
        : base()
    {
        this.ErrorInfoList = new List<ErrorInfo>();
    }

    protected MyCustomException(SerializationInfo info, StreamingContext context)
        : base(info, context)
    {
        foreach (SerializationEntry entry in info)
        {
            if (entry.Name == "ErrorInfoList")
            {
                if (entry.Value == null)
                    this.ErrorInfoList = null;
                else
                {
                    if (entry.Value is List<ErrorInfo>)
                    {
                        // Already fully typed (BinaryFormatter and DataContractSerializer)
                        this.ErrorInfoList = (List<ErrorInfo>)entry.Value;
                    }
                    else if (entry.Value is IEnumerable && !(entry.Value is string))
                    {
                        var enumerable = (IEnumerable)entry.Value;

                        if (!enumerable.OfType<object>().Any())
                        {
                            // Empty collection
                            this.ErrorInfoList = new List<ErrorInfo>();
                        }
                        else if (enumerable.OfType<ErrorInfo>().Any())
                        {
                            // Collection is untyped but entries are typed (DataContractJsonSerializer)
                            this.ErrorInfoList = enumerable.OfType<ErrorInfo>().ToList();
                        }
                    }

                    if (this.ErrorInfoList == null)
                    {
                        // Entry value not already deserialized into a collection (typed or untyped) of ErrorInfo instances (json.net).
                        // Let the supplied formatter converter do the conversion.
                        this.ErrorInfoList = (List<ErrorInfo>)info.GetValue("ErrorInfoList", typeof(List<ErrorInfo>));
                    }
                }
            }
        }
    }

    [SecurityPermissionAttribute(SecurityAction.Demand, SerializationFormatter = true)]
    public override void GetObjectData(SerializationInfo info, StreamingContext context)
    {
        if (info == null)
        {
            throw new ArgumentNullException("info");
        }

        info.AddValue("ErrorInfoList", this.ErrorInfoList, typeof(List<ErrorInfo>));

        base.GetObjectData(info, context);
    }
}

[Serializable]
[KnownType(typeof(ErrorInfo.ErrorCode))]
public class ErrorInfo : ISerializable
{
    public enum ErrorCode
    {
        One,
        Two
    }

    public ErrorCode Code { get; set; }

    public void GetObjectData(SerializationInfo info, StreamingContext context)
    {
        info.AddValue("Code", this.Code, typeof(ErrorCode));
    }

    public ErrorInfo() { }

    protected ErrorInfo(SerializationInfo info, StreamingContext context)
    {
        this.Code = (ErrorCode)Enum.Parse(typeof(ErrorCode), info.GetInt32("Code").ToString());
    }
}

Answer: Try implementing IConvertible on your ErrorInfo class.
My guess is that it can't go from 'what ever' is in value named 'ErrorInfoList' inside the SerializationInfo context to a List. So I'd implement IConvertible on ErrorInfo.
Comment: I implemented `IConvertible` on `ErrorInfo` class, but I'm getting the same exception.

Not only that, but I'm not even hitting any breakpoints in the `IConvertible` methods of `ErrorInfo`...


Javascript calculator returns string instead of value

Question: I am a beginner in javascript and I am currently trying to make a simple calculator. The problem I am having is that the calculator returns the actual equation(as a string) instead of the result value. Here is a jsFIDDLE document
$(document).ready(function () {
var score = "";

$("#nOne").click(function () {
    score += 1;
});
$("#nTwo").click(function () {
    score += 2;
});

$("#add").click(function () {
    score += "+";
});

$("#nEqual").click(function () {
    alert(score);
});

});
Also is there a preference in using < input type="button"> or a
 as a button or they are all the same in making a simple calculator?
Comment: @raina77ow No worries, looks like nine other users have missed it too : ). Using `eval()` would probably be the simplest solution here, the question is, who dares to propose it as an answer?
Comment: good hint but still add button will mess things up. This is wrong logic to write calculator in my opinion
Comment: @raina77ow How would setting `score=0` solve the problem? There's `score += '+'` which converts `score` to a string again...
Comment: Web is full of "simple javascript calculator example"s, please just do some search to find them all (30 200 000 results), not my downvote though.
Comment: @Teemu Indeed, missed that `+` case. Now we're looking into `eval` direction, I suppose.
Answer: You can use eval() JavaScript function.
In the example, the buttons are not adding values, instead you are concatenating string values. After pressing #nOne, #add, #nTwo, you get "1+2" (string) in score, not 3 (number). eval() execute an expression (string) "converting" it to code. w3schools.com/jsref/jsref_eval.asp
$("#nEqual").click(function () {
    alert(eval(score));
});

although it would be ugly code!
Comment: If you can also explain, why `eval()` is needed, I'll upvote your answer ; ).
Comment: In the example, the buttons are not adding values, instead you are concatenating string values. After pressing `#nOne`, `#add`, `#nTwo`, you get "1+2" (string) in `score`, not 3 (number).
eval() execute an expression (string) "converting" it to code.
http://www.w3schools.com/jsref/jsref_eval.asp
Comment: Actually you should add the explanation to your answer, it'd serve future visitors better (and OP ofcourse)... +1 despite of referring w3schools : ).


Como converter coordenadas de tela para coordenadas cartesianas?

Question: Em Python:
Como converter coordenadas da tela para coordenadas cartesianas, onde podem haver pontos positivos e negativos e o centro da tela é (0,0)?
Comment: deu certinho cara, valeu!
Comment: gostaria de saber se você testou minha solução.
Answer: Este é um problema de conversão de sistema de coordenadas e considerando que a sua linguagem de programação tem como origem (0, 0) o canto superior esquerdo da tela temos (em JavaScript - em Python é análogo) :

var larguraTela = 600;
var alturaTela = 300;
var getXY = function(x, y) {
    var novoX =  x - (larguraTela / 2);
    var novoY = (y - (alturaTela / 2)) * -1;
    return [ novoX, novoY ];
}

document.addEventListener('click', function(evt) {
    console.log(evt.x + ", " + evt.y);
    console.log(getXY(evt.x, evt.y));
}, false);

Fiz em javaScript / HTML para testar no browser dando cliques no retangulo e observando a console do browser, mas o que importa é a função getXY() que espera como parâmetro a coordenada X e Y originais e retorna um array de dois elementos: x e y convertidos.
Observe que é necessário multiplicar por (-1) em Y devido a direção inversa em relação ao sistema cartesiano tradicional.


Transfer learning

Question: I have several regarding transfer learning. I'm.very new to this community. So please correct me if there any mistake in question itself.

How to select desired pre trained model for transfer learning?

If in case I need to predict the image whose class is not in the pre trained model's class what should I do?

Should i need to limit the layers of TL model till conv and maxpool(no fully connected layer) and train it again?

4.If again training is required means what is the significance of transfer leaning?
Answer: From my understanding, people are trying different things on different tasks. And there are no universal guidelines to pick the "right" pre-trained model.

It is better to choose the pretained model, that training data is close to your data. For example, if your task is working on images with cat and dogs, it is better to use a pretrained model that is also trained with cats and dogs instead of medical images.

It will always to be the case that pre-trained model's predicted class is different on your task. And this does not matter at all, because we just use pre-trained model to generate features, i.e., we will strip out the final layers to predict classes, but use the middle layer as "embedding" on new task.

To answer your questions in the comment

You can search for pre-trained network to get the data used and the problem it was trying to solve. For example, if I search VGG16, I got

VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman from the University of Oxford in the paper "Very Deep Convolutional Networks for Large-Scale Image Recognition". The model achieves 92.7% top-5 test accuracy in ImageNet, which is a dataset of over 14 million images belonging to 1000 classes.

So, I know the training data is ImageNet and it is a multi class classification setting. You can further search ImageNet to see how the images and classes look.

The key point of the transfer learning is using the pre-trained model to generate features. Let me use another example to illustrate this idea:
think about word embedding in NLP, you may view the word embedding vector is a pre-trained model. And we will build "second half" of the neural network to use them for the new task.

the weights of pre trained model will be only for that pre-trained model's class.

Yes, but we are adding additional layers, and these layer can "adopt" and use the "weights for pre-trained class" to perform the new tasks.
Comment: Thank you sir for your response. I have got some dobuts again. In 1st From where I get to know about the pre trained model's class? In 2nd What the middle layer actually do? Because if we use different class we have to again train the model for that class right, if model is trained on someother class? Because the weights of pre trained model will be only for that pre-trained model's class right?
Comment: @BEingMECHon see my edited answer.
Comment: Thank you so much sir.

Transfer learning


python peewee insert, with table name in variable

Question: Most of time we'll use following way to create items with peewee:
User.create(name='aa', age=20, ...)
But in some circumstances the Table name is stored in a variable. I notice I can use database.execute_sql() to execute raw SQL statement. But I think it's a little messy.
Is there a way to do something like this?
#{TableName}.create(name='aa', age=20, ...)
Thanks for your time!
Answer: Using the approach outlined here:
class Table(Model):
    text = TextField()

    class Meta:
        database = DB

table_name = 'Table'
table = globals()[table_name].create(text='lorem ipsum')


Piping resized image to express response setting response header to 'Content-Length:0' and no data in response

Question: I am building a dummy image generator to improve my understanding of Node and Express. I get the dimensions from the URL and use GM package to resize it. The resulting stream is piped into the response. But I don't get the data on the front end and when I check the response tab in the Network panel of the dev tools in Chrome I see 'this response has no data available'. Can someone please give me some pointers as I am not sure where I can begin debugging this.
const IMAGE_DIR_PATH = path.join(__dirname, './../../data/images');

const getRandomImageName = () => {
  return new Promise((resolve, reject) => {
    fs.readdir(IMAGE_DIR_PATH, (err, images) => {
      err ? reject(err) : resolve(
        images[Math.floor(Math.random() * images.length)]
      );
    });
  });
};

const pickDimensions = (dimensionsStr) => {
  const dimensions = dimensionsStr.toLowerCase().split('x');
  return {
    width: dimensions[0],
    height: dimensions[1] ? dimensions[1] : dimensions[0]
  };
};

exports.getRandomImageByDimension = (req, res, next) => {
  const dimensions = pickDimensions(req.params.dimensions);

  //getRandomImageName returns a Promise
  //resolves with the name of the file example: abc.jpg
  getRandomImageName()
  .then((img) => {
    res.set('Content-Type', 'image/jpeg');

    //I am getting the right image and the path below is valid
    gm(path.resolve(`${IMAGE_DIR_PATH}/${img}`))
    .resize(dimensions.width, dimensions.height)
    .stream(function (err, stdout, stderr) {
      if (err) { throw new Error(err); }
      stdout.pipe(res);
    });
  })
  .catch(e => {
    res.status(500);
    res.send(e);
  });
};

The response headers I receive is shown below:
Comment: I'd start debugging this using a much simpler example. First create a Node script that doesn't use Express or GM at all, just open an image file and pipe it into another file, check that works, using hard-coded filenames so there's no doubt what it's doing. Then add in the resizing stuff, using hard-coded sizes and check that works. Then gradually introduce the Express bits and remove the hard-coding. Keep checking it still works at each stage until you isolate exactly where it goes wrong.
Answer: Turns out that I had forgotten to add ImageMagick as a dependency for gm and this was causing an error. Strangely the error was not shown in the console and I found it only after breaking down the app into simpler pieces and debugging as skirtle mentioned in a comment above.
To fix I did the following:
npm install --save imagemagick

Then in the file where I was using the gm package, I set the imageMagick property:
const gm = require('gm').subClass({imageMagick: true});

Note: I was running this on a Windows 7.


Custom forms inside another form?

Question: Is there any way to create a custom form in which one of its elements is another custom form using the YAML Form module?
Answer: No, please see Provide mechanism to build composite and/or base forms
Comment: Yes! It's exactly what I'm looking for! Thank you so much! Awesome module!

Custom forms inside another form?


Selecionar opção do select de acordo com parâmetro da url

Question: eu tenho o script abaixo que pega um parâmetro da URL(se tiver), procura em todos os selects da página e seleciona a opção com o mesmo valor(mesmo valor do parâmetro). Se não tiver um parâmetro na URL, ele seleciona a primeira opção do select. Até aqui, blz!
Eu preciso que se tiver um parâmetro na URL ele procure em todos os selects da página e se encontrar ele seleciona e se caso não exista no PRÓXIMO select, ele marque a primeira opção.
if (hash !== undefined) {
$('.kit-list').each(function (i) {
    $('.kit-list:eq(' + i + ') select option').filter(function () {
        return $(this).text() == hash.size.replace('%20', ' ');
    }).prop("selected", true);
    $('.kit-list:eq(' + i + ') select').change();
});
} else {
$('.kit-list').each(function (i) {
    $('.kit-list:eq(' + i + ') select').find('option').eq(1).attr('selected', 'selected');
    $('.kit-list:eq(' + i + ') select').change();
});
}

Comment: Exemplo do HTML? Exemplo de valores que podem estar no hash?
Answer: Resolvi meu problema da seguinte forma(não sei se é a melhor mas funcionou):
if (hash !== undefined) {
    $('.kit-list').each(function (i) {
        if ($('.kit-list:eq(' + i + ') #espec_0_opcao_0 option[value=' + hash.size.replace('%20', ' ') + ']').length > 0) {
            $('.kit-list:eq(' + i + ') #espec_0_opcao_0 option').filter(function () {
                return $(this).text() == hash.size.replace('%20', ' ');
            }).prop("selected", true);
            $('.kit-list:eq(' + i + ') #espec_0_opcao_0').change();
            console.log('sim')
        } else {
            console.log('nao')
            $('.kit-list:eq(' + i + ') #espec_0_opcao_0').find('option').eq(1).attr('selected', 'selected');
            $('.kit-list:eq(' + i + ') #espec_0_opcao_0').change();
        }

    });
} else {
    $('.kit-list').each(function (i) {
        $('.kit-list:eq(' + i + ') select').find('option').eq(1).attr('selected', 'selected');
        $('.kit-list:eq(' + i + ') select').change();
    });
}


Divergence of the infinite sum of $\frac1n$

Question: I'm asked to show that the harmonic series $$\sum_{n=1}^{\infty}\cfrac1n=1+\cfrac12+\cfrac13+\cfrac14+...$$
is divergent.
I figured it would be satisfactory enough to note that $n>1$ for all $n>1$, and so the limit of $\cfrac1n$ will approach $0$ as $n\rightarrow\infty$, then so too should its sum converge. Instead, my textbook lists out the terms $s_8, s_{16}, s_{32}$, and so on. But I have no clue what they're doing beyond $s_2$; \begin{align*} s_4&=1+\frac12+\biggr(\frac13+\frac14\biggr)>\frac12+\biggr(\frac14+\frac14\biggr)=1+\frac22 \\ s_8 &=1+\frac12+\biggr(\frac13+\frac14\biggr)+\biggr(\frac15+\frac16+\frac17+\frac18\biggr)>1+\frac12+\biggr(\frac14+\frac14\biggr)+\biggr(\frac18+\frac18+\frac18+\frac18\biggr)=1+\frac12+\frac12+\frac12=1+\frac32\end{align*}
And this continues infinitely. I don't understand exactly what's going on, especially the portions that repeat $1/n$ at the same value $n$ inside the parentheses.
Can someone explain what's happening?
Comment: Just because $x_n$ go to $0$ as $n$ goes to $\infty$ doesn't mean that the sum converges. If $x_n$ didn't go to $0$ as $n$ went to $\infty$ we would know the sum diverges.
Comment: Series converges implies the tail term goes to $0$. It is a necessary condition for convergence, but not sufficient. In the argument taking $s_8, s_{16}, \cdots$, it says that $s_{2^n} > 1 + \frac{n}{2} \to \infty$ as $n \to \infty$. Can you see why this implies the divergence?
Comment: "then so too should its sum converge"  If you believe this (and it's reasonable [somewhat] although incorrect assumption) the that would mean the sum *con*verges, not diverges.  But no, $a_n\to 0$ does *not* mean $\sum a_n$ converges and $\sum \frac 1n$ *is* a classic counterexample. Reread the argument it should make sense.
Comment: Intuitively what's going on is that the partial sums are growing logarithmically (compare the sum with an integral). Now if you compute $log(2^{n+1})-log(2^n)$ you notice that it's a constant positive value for every n. But if you add a constant infinitely many times then of course it diverges.
Answer:
"then so too should its sum converge"

If you believe this (and it's reasonable [somewhat] although incorrect assumption) the that would mean the sum converges, not diverges.  But no, $a_n\to 0$ does not mean $\sum a_n$ converges and $\sum \frac 1n$ is a classic counterexample. Reread the argument it should make sense.
Thing is we can group the terms $\frac 11, \frac 12, \frac 13,.....$ into groups.
$(\frac 11)$ by itself.
$(\frac 12, \frac 13)$ a group of $2$
$(\frac 14, \frac 15, \frac 16,\frac 17)$ a group of $4$.
Each group is twice as big as the group before.
$(\frac 18,...., \frac 1{15})$ as a group of $8$ and
$(\frac 1{16},....., \frac 1{31})$ as a group of $16$..... and so on.
Each group goes from $\frac 1{2^k}.... $ to $\frac 1{2^{k+1} -1}$ and has $2^k$ terms for some integer $k$.
So what is the sum of each group?
$\frac 1{2^k} + \frac 1{2^k + 1} + \frac 1{2^k+ 1} + ..... + \frac 1{2^k + (2^k-1)} = $ what?
Well, I don't know.  But notice each term $\frac 1{2^k + i} > \frac 1{2^{k+1}}$ so
$\frac 1{2^k} + \frac 1{2^k + 1} + \frac 1{2^k+ 1} + ..... + \frac 1{2^k + (2^k-1)} > $
$\frac 1{2^{k+1}} +\frac 1{2^{k+1}}+\frac 1{2^k+ 1}....+\frac 1{2^{k+1}} =$
$\underbrace{\frac 1{2^{k+1}} +\frac 1{2^{k+1}}+\frac 1{2^k+ 1}....+\frac 1{2^{k+1}}}_{\text{there are }2^k\text{ of them}}=$
$ 2^k\times \frac 1{2^{k+1}} = \frac 12$.
So the sum of each group is more than $\frac 12$.
So the sum of $\sum_{j=1}^{\infty} \frac 1n =$
$(\frac 11) + $
$(\frac 12 + \frac 13) + $
$(\frac 14 + \frac 15 + \frac 16 + \frac 17) + $
$(\frac 18+ ..... + \frac 1{15}) + $
$(\frac 1{16} + ..... + \frac 1{31}) + $
$......$
$(\frac 1{2^k} + \frac 1{2^k + 1} + ....... + \frac 1{2^{k+1} -1} ) + $
$..... $
which is the sum of all the sums of the groups.  But the sums of each of the groups is more than $\frac 12$.
so....
$\sum_{j=1}^{\infty} \frac 1n =$
$(\frac 11) + $
$(\frac 12 + \frac 13) + $
$(\frac 14 + \frac 15 + \frac 16 + \frac 17) + $
$(\frac 18+ ..... + \frac 1{15}) + $
$(\frac 1{16} + ..... + \frac 1{31}) + $
$......$
$> $
$\frac 12 + \frac 12 + \frac 12 + \frac 12 + ..... \frac 12 + .....$
which is the sum of an infinite number of $\frac12$s.
Which clearly diverges DESPITE then fact that $\frac 1n \to 0$.
$\sum_{j=1}^{\infty} \frac 1n =$
$(\frac 11) + $
$(\frac 12 + \frac 13) + $
$(\frac 14 + \frac 15 + \frac 16 + \frac 17) + $
$(\frac 18+ ..... + \frac 1{15}) + $
$(\frac 1{16} + ..... + \frac 1{31}) + $
$......$
$> $
$(\frac 12) + $
$(\frac 14 + \frac 14) + $
$(\frac 18 + \frac 18 + \frac 18 + \frac 18) + $
$(\frac 1{16}+ ..... + \frac 1{16}) + $
$(\frac 1{32} + ..... + \frac 1{32}) + $
$......$
$=$
$1\times \frac 12+$
$2\times \frac 14 +$
$4\times \frac 18 +$
$8\times \frac 1{16}+$
$16\times \frac 1{32}+$
$...$
$2^k*\frac 1{2^{k+1}} +$
$.....$
$=$
$\frac 12 + \frac 12 + \frac 12 + \frac 12 + ..... \frac 12 + .....$
$\to \infty$.
Answer: For a full explanation:
Just because $x_n → 0$ as $n→\infty$ doesn't mean that the sum converges. The inverse is true however: if $x_n$ didn't go to $0$ as $n → \infty $ we would know the sum diverges.
Now to explain what is going on, the proof groups the sum into groups with a lower bound of $\frac{1}{2}$. If does so by making an $n$-th grouping as follows:
\begin{equation*}
\begin{split}
\sum_{n=1}^{\infty}\frac{1}{n} & = 1+\frac{1}{2}+\frac{1}{3}+\frac{1}{4}+...+ \frac{1}{2^{k-1}+1}+...+\frac{1}{2^k}+... \\ & = 1+\frac{1}{2}+\frac{1}{3}+\frac{1}{4}+...+ \left(\frac{1}{2^{k-1}+1}+...+\frac{1}{2^k}\right)+...\\ & \geq 1+\frac{1}{2}+\frac{1}{4}+\frac{1}{4}+...+ \left(\frac{1}{2^{k}}+...+\frac{1}{2^k}\right)+... \\& = 1+\frac{1}{2}+\frac{1}{2}+...+\frac{1}{2}+... \\& = 1+\sum_{n=0}^{\infty}\frac{1}{2}
\end{split}
\end{equation*}
and you can do this grouping infinitely many times which gives you infinitely many $\left(\frac{1}{2}\right)$s in the sum, which implies our lower bound diverges which implies our original sum diverges.
Answer: So a different approach is using Cauchy's criteria for series convergence. It states that if for every $\varepsilon>0$ there exists $n_{0}\in\mathbb{N}$ so that for every $p,m>n_{0}$, $|\sum_{n=m}^{p}an|<\varepsilon$, the series converges.
Now as was proved above, you can always find enough elements the series so that their sum is greater than $\frac{1}{2}$.
So, for $\varepsilon=\frac{1}{2}$ we have that for every $n_{0}$ we can find $p,m>n_{0}$ such that $|\sum_{n=m}^{p}\frac{1}{n}|>\frac{1}{2}$. This contradicts Cauchy's criteria, and therefore the series diverges.
Answer: You can show divergence of a series by finding another series with smaller terms but that is known to diverge (test by comparison).
Here the auxiliary series is found by forming groups of terms and replacing every element in the group by the smallest of them, e.g.
$$\frac15+\frac16+\frac17+\frac18\ \to\ \frac18+\frac18+\frac18+\frac18.$$
Hence,
$$1+\frac12+\color{blue}{\frac13+\frac14}+\color{green}{\frac15+\frac16+\frac17+\frac18}+\cdots>1+\frac12+\color{blue}{\frac24}+\color{green}{\frac48}+\cdots$$
and the auxiliary series obviously diverges.

Addendum:
It is known that the partial sums grow like the logarithm of $n$. For this reason, when you group the terms and double the group size every time, you can expect that the groups have approximately a constant sum.


How can I use kernel functions in SVM root(execute) mode?

Question: I ran into the following problem: When I initialized the kernel hypervisor, for me it is SVM and I exit from vmrun and get into my SvmExitHandler (this is the dispatcher that manages exit codes), then when I try to call any IRQL Passive level api (given that I am on it), I get a BSOD KERNEL_SECURIY_CHECK_FAILURE.
At the same time, when I call the same API with the same arguments, but before starting the SVM, then everything works for me. How can I organize communication with kernel functions when I am in SVM? For example, I need to process each instruction during tracing
<code>extern "C" VMM_STATUS SvmVmexitHandler(PRIVATE_VM_DATA* Private, GUEST_CONTEXT* Context, GUEST_SSE_CONTEXT* SSeContext, GUEST_AVX_CONTEXT* AvxContext)
    {
        // Load the host state:
        __svm_vmload(reinterpret_cast<size_t>(Private->VmmStack.Layout.InitialStack.HostVmcbPa));

        // Restore the guest's RAX that was overwritten by host's RAX on #VMEXIT:
        Context->Rax = Private->Guest.StateSaveArea.Rax;

        VMM_STATUS Status = VMM_STATUS::VMM_CONTINUE;

        switch (Private->Guest.ControlArea.ExitCode)
        {
        case VMEXIT_CPUID:
        {
            CPUID_REGS Regs = {};
            int Function = static_cast<int>(Context->Rax);
            int SubLeaf = static_cast<int>(Context->Rcx);
            __cpuidex(Regs.Raw, Function, SubLeaf);

            switch (Function) {
            case CPUID_VMM_SHUTDOWN:
            {
                // Shutdown was triggered:
                Status = VMM_STATUS::VMM_SHUTDOWN;
                break;
            }
            case CPUID::Generic::CPUID_MAXIMUM_FUNCTION_NUMBER_AND_VENDOR_ID:
            {
                // Vendor = 'Hyper-Bridge' as RBX + RDX + RCX:
                Context->Rax = Regs.Regs.Eax;
                GetHvCpuName(Context->Rbx, Context->Rcx, Context->Rdx);
                break;
            }
            default:
            {
                Context->Rax = Regs.Regs.Eax;
                Context->Rbx = Regs.Regs.Ebx;
                Context->Rcx = Regs.Regs.Ecx;
                Context->Rdx = Regs.Regs.Edx;
                break;
            }
            }
            break;
        }
        case VMEXIT_MSR:
        {
            if ((Context->Rcx & MAXUINT32) == static_cast<unsigned int>(AMD_MSR::MSR_EFER) && Private->Guest.ControlArea.ExitInfo1)
            {
                EFER Efer = {};
                Efer.Value = ((Context->Rdx & MAXUINT32) << 32) | (Context->Rax & MAXUINT32);
                if (!Efer.Bitmap.SecureVirtualMachineEnable)
                {
                    InjectEvent(&Private->Guest, INTERRUPT_VECTOR::GeneralProtection, EXCEPTION_VECTOR::FaultTrapException, 0); // #GP (Vector = 13, Type = Exception)
                    break;
                }
                Private->Guest.StateSaveArea.Efer = Efer.Value;
            }
            break;
        }
        case VMEXIT_VMRUN:
        {
            InjectEvent(&Private->Guest, INTERRUPT_VECTOR::GeneralProtection, EXCEPTION_VECTOR::FaultTrapException, 0); // #GP (Vector = 13, Type = Exception)
            break;
        }
        case VMEXIT_EXCP_DB://TRACE EXCEPTION BY TF FLAG
        {
            KdPrint(("RIP %p\n", Private->Guest.StateSaveArea.Rip));

            Private->Guest.StateSaveArea.Rax = Context->Rax;
            return Status;
        }
        }

        if (Status == VMM_STATUS::VMM_SHUTDOWN)
        {
            // We should to devirtualize this processor:
            Context->Rax = reinterpret_cast<UINT64>(Private) & MAXUINT32; // Low part
            Context->Rbx = Private->Guest.ControlArea.NextRip;
            Context->Rcx = Private->Guest.StateSaveArea.Rsp;
            Context->Rdx = reinterpret_cast<UINT64>(Private) >> 32; // High part

            // Load the guest's state:
            __svm_vmload(reinterpret_cast<size_t>(Private->VmmStack.Layout.InitialStack.GuestVmcbPa));

            // Store the GIF - Global Interrupt Flag:
            _disable();
            __svm_stgi();

            // Disable the SVM by resetting the EFER.SVME bit:
            EFER Efer = {};
            Efer.Value = __readmsr(static_cast<unsigned long>(AMD_MSR::MSR_EFER));
            Efer.Bitmap.SecureVirtualMachineEnable = FALSE;
            __writemsr(static_cast<unsigned long>(AMD_MSR::MSR_EFER), Efer.Value);

            // Restoring the EFlags:
            __writeeflags(Private->Guest.StateSaveArea.Rflags);
        }

        Private->Guest.StateSaveArea.Rax = Context->Rax;

        // Go to the next instruction:
        Private->Guest.StateSaveArea.Rip = Private->Guest.ControlArea.NextRip;

        return Status;
    }
</code>
So if I access the ZwWriteFIle API directly in order to write the RIP value to VMEXIT_EXCP_DB, then I will crash to the BSOD, since in fact all interrupts and other things are disabled when I am in SVM. How can I handle this?! I will be glad of any help!
Answer: The full answer to this question will be: Nohow, because in order to get into the interrupt, my IRQL level rises, and then decreases after I got into my SvmVmexitHandler and for this reason I can't use any IRQL dependent native API in it
Comment: As it's currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).


Time complexity of a function

Question: Can we categorize $f(n)=10+(-1)^n$ as exponential or polynomial ?
And how can I compare this function with functions like $g(n)=2^n$ and $h(n)=n^2$ so as to understand if it is greater or not? I know that $f(n)$ equals $11$ or $9$ depending on n's value but how about the time complexity?
Comment: That qualifies as proportional to $o(1)$.
Answer: Let $g(n)=1$, then as $$9g(n)\leqslant f(n)\leqslant 11 g(n)$$ for all $n$, it follows that $f\in\Omega(1)$. That is, the time complexity of $f$ is constant.

Time complexity of a function


Reference request : Study of Differential topology post Milnor's book

Question: I am just about to finish my study of Milnor's book 'Topology from the Differentiable Viewpoint' and I really love the subject. I would like to continue my study of Differential Topology and am looking for some good references. What are the canonical choices of textbooks for this subject ? I am aware of the book by Morris Hirsch but am not sure what are the prerequisites or whether it is a good book.
I have almost no knowledge of Algebraic Topology and my knowledge of analysis in limited to undergrad level real and functional analysis. I have a background knowledge of smooth manifolds and differential geometry (Riemannian metrics, curvature, connections etc.)
Thanks !
Comment: You may find Bott-Tu useful in bridging the gap between differential topology and algebraic topology.
Answer: If you're going to continue in topology, you should learn algebraic topology. Hatcher is a good place to start, and with your background, I would suggest Bott-Tu to bridge the gap between differential and algebraic topoogy.
For other differential topology books, Hirsch is good, as is Guillemin-Pollack. (The latter is standard introductory material; as I have not read Milnor's book, I do not know how strong the overlap is.)
Comment: Hatcher is a trivial book, real men read spanier!!!!
Comment: Hatcher is elementary and is an appropriate textbook for a very first algebraic topology course. More importantly, though, don't get hung up on order -- you should  never read things "in order"! Learning is absolutely not a linear process. Read them concurrently and use one to get you unstuck in the other.
Comment: Guillemin and Pollack was inspired by Milnor's book, but there's a good deal of stuff that's quite expanded and somewhat different (including differential forms, Stokes's Theorem, and the Degree Theorem). But you sound like you have a solid analysis (single and multivariable) background, so Hirsch's book is definitely appropriate for you.
Comment: Thanks! About Bott & Tu , I wish to ask : is it a good idea to learn basic homological algbera and spectral sequences (say from Weibel ch 1-5) before reading Bott-Tu or is it more logical to read Bott & Tu first and then read homological algbera ?
Comment: And which order would you suggest : learning Algberaic Topology from Hatcher first and then reading Bott-Tu or the other way around ? Thanks !


Proteger a string de conexão em uma aplicação .NET Winforms?

Question: Tenho uma aplicação .NET Windows Forms que se conecta a um servidor SQL Server diretamente, sem o uso de uma camada intermediária como um WebService ou WebAPI, através de SqlConnection. Essa aplicação é instalada no computador dos clientes enquanto o servidor fica disponível através da internet.
Qual a melhor maneira, sem alterar toda a aplicação, de evitar que a string de conexão (connection string) seja capturada através de um sniffer tipo o WireShark rodando na mesma rede?
Answer: Segundo este artigo em inglês, o componente de conexão do SQL Server já vai criptografar as partes importantes da string de conexão durante a conexão, como a senha e nome de usuário... a não ser que use ODBC.
Seguindo alguns links desse artigo, encontrei esse artigo da MSDN: Criptografando conexões com o SQL Server que mostra como forçar o uso de SSL para criptografar TODO o tráfego de dados entre o cliente e o servidor SQL Server.
Comment: Mesmo que o usuário e senha fiquem ocultos, todo o retorno que o banco enviar para a aplicação poderá ser capturado pelo sniffer. Se você fizer um "select * from usuario" vai ser mostrado todos os dados de todos usuários no tráfego recebido mesmo que a aplicação não exiba nenhum desses dados.
Comment: Basta usar SSL, e problema resolvido.
Comment: Fora o trabalho para implementar depois basta usar `SSLstrip` para capturar.
Comment: Isso que o Maicon disse, de que "todos os dados de todos usuários no tráfego" é natural, mesmo com webservices isso acontece. A ideia é evitar que alguém consiga autenticação para manipular os dados fora de sua aplicação. Vou tentar implementar a opção encrypt=yes da connectionString, depois se tiver sucesso volto aqui pra marcar a resposta.
Comment: O SQL Server não usa HTTPS e sim [TSL/SSL](http://technet.microsoft.com/pt-br/library/cc784450(v=ws.10).aspx), por isso o SSLStrip não deve conseguir ler as transmissões do SQL Server, pois o segundo [funciona sobre HTTPS](http://security.stackexchange.com/questions/41988/how-does-sslstrip-work).
Comment: @MiguelAngelo Erro meu quanto ao capturar a conexão criptografada do SQL Server, tinha levado em conta apenas você ter mencionado o uso de SSL. Pelo menos até o momento né huaehaeuhea e ganhou +1 :)
Answer: O correto mesmo seria você não ter essa conexão do cliente -> banco e sim cliente -> webService (com autenticação) -> banco. Por mais que você tente fazer algum tipo de encode/encriptação na string, todo o tráfego feito entre o cliente e o banco poderá ser capturado pelo sniffer. Qualquer select poderá ser capturado mesmo que sua aplicação não mostre esses dados eles virão no tráfego e podem ser capturados sem dificuldade.
Veja:

Mesmo que tente implementar uma conexão segura, isso poderá ser burlado com o sslstrip. É mais fácil criar um WebService e com isso deixar a comunicação com o banco isolada do cliente.
Comment: Se conseguir evitar que o acesso direto ao banco seja barrado, ou seja, evitar que as informações de autenticação sejam acessíveis, já resolve. Pois mesmo com webservices ou webapis, informações sobre tipos e campos dos dados poderão ser interceptados.
Comment: Também não posso considerar que seja mais fácil implementar com webservice uma aplicação considerável que já está em produção.
Comment: @iuristona Realmente, levando em conta que já está em produção fica difícil de ter todo esse retrabalho. Quanto ao WebServices serem interceptados a diferença é que no WebService você consegue ter um controle de quem requisitou, como enviar, consegue tratar os dados antes de enviar e obstrui qualquer possibilidade do usuário saber a localização do banco.
Answer: Verifiquei que a versões do SQL antigas, anteriores a 2005 (especificamente a 2000) não funcionam com o parâmetro encrypt=yes na string de conexão (connection string). Tentei em outro server mais recente e recebi o erro:

provider: Provedor SSL, error: 0 - A cadeia de certificação foi
  emitida por uma autoridade que não é de confiança.

Então encontrei outro parâmetro trustservercertificate=true que adicionei à minha string de conexão e funcionou. Esse parâmetro permite que meu client aceite um certificado emitido pelo server.
Em testes que fiz com WireShark não consegui interceptar nem a string de conexão, nem as instruções SQL, nem os dados de retorno. Os dados agora trafegam criptografados.
Então, a solução que encontrei foi incluir esse trecho ao final da string de conexão:
encrypt=yes;trustservercertificate=true;


Erro ao validar pagina inicial após fazer Login/Senha

Question: Eu tenho uma validação de acesso (Login/Senha) que caso for verdadeira, deveria retornar para página inicial, porém me retornar o seguinte erro:

InvalidOperationException: RenderBody invocation in '/Views/Shared/_Layout.cshtml' is invalid. RenderBody can only be called from a layout page.

Segue Controller que faz a verificação e deveria retornar para view:
if (cli.ValidarAcesso(cli.CliCodigo, cli.CliSenhaWeb, _config) > 0)
            {

                return View("_Layout", new Clientes());
            }

Answer: Esse erro não é do Controller, é da sua View. Tente colocar @RenderBody() no "centro" da sua view. Se não funcionar, posta o código da sua view aqui.


Co-transitivity of the constructive order relation

Question: One of the first exercises in J.L. Bell's A Primer of Infinitesimal Analysis asks the reader to show that, for arbitrary real numbers $a$, $b$, and $x$, if $a < b$, then either $x > a$ or $x < b$.
The reason this is not entirely trivial is that we're working in a constructive setting, one in which the law of the excluded middle does not hold. The goal is to prove the statement above from the axioms governing the strict ordering relation:

$a < b$ and $b <c$ implies $a < c$
$(a < a)$ is false
$a < b$ implies $a + c < b + c$
$a < b$ and $c > 0$ implies $ac < bc$
either $0 < a$ or $a < 1$
$a \neq b$ implies $a < b$ or $b < a$

Note that without the law of the excluded middle, we cannot establish the classical trichotomy that, for any two real numbers $x$ and $y$, one of $x < y$, $x = y$, or $x > y$ is true: it may still be the case that we cannot distinguish $x$ from $y$ sufficiently to establish any of these relations.
In Bell's presentation, we also know at this point that the real numbers with the operations of addition and multiplication form a field in the usual way, though with the caveat that in order to assume $x$ has a multiplicitive inverse we must know $x \neq 0$, a condition that is not automatic for numbers other than zero since we can't appeal to the law of the excluded middle.
Some Googling tells me that the statement to be proved is sometimes presented as a axiom (called co-transitivity) of constructive order relations. How can it be proved from Bell's axioms?
Comment: The conclusion in Axiom 3 should be $a+c<b+c$.
Comment: "a condition that is not automatic for numbers other than zero since we can't appeal to the law of the excluded middle" - this isn't quite right. Looking at page 16, Bell *defines* $x \not = y$ to be $\lnot (x = y)$ so as soon as you know a number is other than zero you know it is $\not = 0$. What the lack of excluded middle says is that you can't assert $x = 0 \lor x \not = 0$; you can still assert $\lnot (x = 0) \to x \not = 0$.
Comment: @pash: You cannot prove an arbitrary number is nonzero (of course) but in this system, if you know that $0 < x$ then you can prove $0 \not = x$ and so $1/x$ exists. So you can prove $x \not = 0$ when you have various other assumptions on $x$. The issue is that a number cannot be "other than zero" without being $\not = 0$. A number that is neither $= 0$ nor $\not = 0$ cannot be described as "other than zero" because, for all we know, it might turn out to be equal to zero once we get more information about it.
Comment: @Carl Mummert, yes, "as soon as you know a number is other than zero" you know $x \neq 0$, but you cannot reason by cases about an arbitrary number using the law of trichotomy, as you often would in a classical setting. So when you're considering an arbitrary quantity $x$, there's no way to get to $x \neq 0$. Right?
Comment: @Carl Mummert, yes, that is my understanding, too. The caveat I gave was a poorly articulated attempt to clarify that, unlike in a classical setting, it is not the case in this field that only zero lacks a multiplicative inverse. ... And now I understand why constructivists had to come up with the notion of an [apartness relation](http://en.wikipedia.org/wiki/Apartness_relation).
Answer: As a rule of thumb, in constructive mathematics often the way you prove a formula involving a disjunction is to start with an axiom which already has a disjunction. In this case notice that the only axioms mentioning disjunctions are 5 and 6. Also note that axiom 5 is kind of similar to what we have to prove, so that's the one we're going to use. It just needs a bit of adjusting.
First of all see that since $a < b$ we can apply axiom 3 to show that $0 < b - a$, so in particular we can divide by $b - a$.
Let $x' = \frac{x - a}{b - a}$. Now applying axiom 5 we know that either $0 < x'$ or $x' < 1$.
Suppose that $0 < x'$. Then by axiom 4 we can multiply through by $b - a$ (recall $0 < b - a$) to get $0 < x - a$, and then get $a < x$ by axiom 3.
Now suppose that $x' < 1$. Then multiplying through by $b - a$ again this time gives $x - a < b - a$. Then apply axiom 3 to get $x < b$.
But now we've shown that either $a < x$ or $x < b$ as required.
Comment: For the sale of dotting all the i's: Having that $0<b-a$ and wanting to divide by $b-a$, we invoke Axiom 2 to say $b-a\neq0$.


Is the hypothesis of $G$ being a finite group necessary in this exercise?

Question: I've been trying to solve an exercise for my Algebra class and even found this question asked already:
Suppose $H$ is the only subgroup of order $o(H)$ in the finite group $G$. Prove that $H$ is a normal subgroup of $G$.
Although, is the finiteness hypothesis a must for this exercise?
Comment: Welcome to [math.se].
Can you please [edit] your post and write your attempts at solving the problem?
If your question is clear and focused on your specific difficulty and you show your effort in solving the problem, it's more likely to get good and helping answers.
By the way, take the opportunity to take the [Tour], if you haven't done it already. See also some tips on
[ask], on
[formatting help](https://math.stackexchange.com/editing-help)
and on writing down equations using
[LaTeX / MathJax](https://math.meta.stackexchange.com/q/5020).
Comment: I don't know if I agree with the downvotes on this question. He's asking whether or not a hypothesis can be omitted, the answer to which could be as simple as a yes or a no with a citation.
Comment: Is $H$ required to be finite?  Or could $o(H)$ denote an infinite cardinal?
Comment: If $o(G)$ is finite, then so is $o(H)$, @GEdger.
Comment: Yes, but if not, I believe @GEdgar is wright... the new question I should solve is: is it possible to exist a group $G$ with infinite order but with one subgroup such as the $H$ above?
Answer: Suppose $H$ is the only subgroup of $G$ of order $|H|$. Fix an element $g \in G$ and let $K = gHg^{-1}$. Take $a,b \in K$. There exist $a' , b' \in H$ such that $a = ga'g^{-1}$ and $b = gb'g^{-1}$. Thus
$$
ab = ga'g^{-1} gb' g^{-1} = ga'b'g^{-1}
$$
which is an element of $K$ since $a'b' \in H$. Moreover, $e \in H$ since $H$ is a subgroup, so
$$
e = geg^{-1} \in K
$$
Thus, we have shown $K \leq G$. Now, consider the maps
\begin{align}
f &: H \to K \\
&h \mapsto ghg^{-1} \\
g &: K \to H\\
&k \mapsto g^{-1}k g
\end{align}
I leave it to you to check that $g = f^{-1}$, which shows that $|H| = |K|$. By assumption, this means $H = K = gHg^{-1}$. Since $g \in G$ was arbitrary, this proves that $H$ is a normal subgroup of $G$.
Notice that nowhere did we need to assume that $G$ was finite.
Comment: In fact, this proof works even if $H$ is an infinite cardinality! So we don't even need to assume that $H$ is finite.


Binding of the control created by xaml works fine, but binding of the control created by code does not work

Question: // MainWindow.xaml.cs

namespace TestApp2
{
    public sealed partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
        }
    }
}

<Window x:Class="TestApp2.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:ex="using:TestApp2">

</Window>

// App.xaml.cs

namespace TestApp2
{
    public partial class App : Application
    {
        public App()
        {
            InitializeComponent();
        }

        protected override void OnLaunched(LaunchActivatedEventArgs args)
        {
            var mainWindow = new MainWindow();
            mainWindow.Content = new CustomButton();
            mainWindow.Activate();
        }
    }
}

// CustomButton.cs

namespace TestApp2
{
    internal class CustomButton : Button
    {
        public object MyProperty
        {
            get => GetValue(MyPropertyProperty);
            set => SetValue(MyPropertyProperty, value);
        }
        public static readonly DependencyProperty MyPropertyProperty =
            DependencyProperty.Register(nameof(MyProperty), typeof(object), typeof(CustomButton), new(null));

        public CustomButton()
        {
            SetBinding(ContentProperty, new Binding
            {
                Source = this,
                Path = new(nameof(MyProperty)),
            });

            // This code should always update the binding,
            // but it does not work if you create a control in the code.
            MyProperty = "Something";
        }
    }
}

When you run the above code, only the null value, which is the value at the first binding, is loaded and not updated after that.
However, if you comment mainWindow.Content = new CustomButton(); out and uncomment  to create it in xaml instead of in the code, the binding will work fine.
I tried setting the binding in the Loaded event because I thought it might be related to initialization, but the result was the same.
Why does the binding fail?
I'm trying to create inheritable pages or custom controls, so I'm trying to create controls and bindings from code without using xaml.
Answer: It seems that the binding breaks when you create the control programmatically.
Instead you can use the property changed event:
internal class CustomButton : Button
{
    public static readonly DependencyProperty MyPropertyProperty =
        DependencyProperty.Register(
            nameof(MyProperty),
            typeof(object),
            typeof(CustomButton),
            new PropertyMetadata(null, OnMyPropertyPropertyChanged));

    private static void OnMyPropertyPropertyChanged(DependencyObject d, DependencyPropertyChangedEventArgs e)
    {
        if (d is CustomButton customButton)
        {
            customButton.Content = e.NewValue;
        }
    }

    public object MyProperty
    {
        get => GetValue(MyPropertyProperty);
        set => SetValue(MyPropertyProperty, value);
    }
}

Comment: I'm sorry. I misunderstood your question. I updated my answer.
Comment: Did it work for you? I got the same result that I wrote in the question. Loads only the value when calling SetBinding and is no longer updated. I called SetBinding when I clicked because I thought the timing of calling SetBinding might be a problem, but the result was the same no matter when I called it.


Adding Page break to multi step form

Question: I am having somewhat of a problem, I am trying to use ajax commands to go to the next case on my form. I dont really understand how to complete this and ive tried numerous ways without any success. Basically what im trying to do in my ajax callback load my preloader function into the #neato_form div's with a 10 second delay and then make it go to case 2 which is step 2 on my form.
Ive included my ajax callback since this is the main problem. Along with my form and submit function in case ive done something wrong there.
Any Suggestions is greatly appreciated.
Ajax commands
function neato_form_ajax_callback($form, &$form_state)
{
    $commands = array();

    $commands[] = array(
        'command' => 'mortgage_response',
        'leadstatus' => 'success',
    );

    $commands[] = ajax_command_replace('#neato_form', preloader());
    $commands[] = ajax_command_invoke(NULL, "preloader");

    if($commands[0]['leadstatus'] == 'success'){

        $commands[] = ajax_command_replace('#neato_form', $form_state['step2']);
    }

    return array('#type' => 'ajax', '#commands' => $commands);
}

Form and Submit function
function neato_form($form, &$form_state)
{
    $form_state['step'] = isset($form_state['step']) ? $form_state['step'] : 1;

    $form['#prefix'] = '
';
    $form['#suffix'] = '
';

    switch($form_state['step'])
    {
        // Step 1
        case 1:
            $default_value = '';
            if(isset($form_state['values']['step_1']))
            {
                $default_value = $form_state['values']['step_1'];
            }
            elseif(isset($form_state['storage']['step_1']))
            {
                $default_value = $form_state['storage']['step_1'];
            }

            $form['step_1'] = array
            (
                '#type' => 'textfield',
                '#title' => t('Step 1'),
                '#required' => TRUE,
                '#default_value' => $default_value,
            );
            break;

        // Step 2
        case 2:
            $default_value = '';
            if(isset($form_state['values']['step_2']))
            {
                $default_value = $form_state['values']['step_2'];
            }
            elseif(isset($form_state['storage']['step_2']))
            {
                $default_value = $form_state['storage']['step_2'];
            }

            $form['step_2'] = array
            (
                '#type' => 'textfield',
                '#title' => t('Step 2'),
                '#required' => TRUE,
                '#default_value' => $default_value,
            );

            break;

        // Step 3
        case 3:
            $default_value = '';
            if(isset($form_state['values']['step_3']))
            {
                $default_value = $form_state['values']['step_3'];
            }
            elseif(isset($form_state['storage']['step_3']))
            {
                $default_value = $form_state['storage']['step_3'];
            }

            $form['step_3'] = array
            (
                '#type' => 'textfield',
                '#title' => t('Step 3'),
                '#required' => TRUE,
                '#default_value' => $default_value,
            );

            break;
    }

    $form['buttons'] = array
    (
        '#type' => 'container',
    );

        $form['buttons']['submit'] = array
        (
            '#type' => 'submit',
            '#value' => t('Submit'),
            '#ajax' => array
            (
                'wrapper' => 'neato_form',
                'callback' => 'neato_form_ajax_callback',
            ),
        );

    return $form;
}

function neato_form_back_submit($form, &$form_state)
{
    $form_state['step']--;
    $form_state['rebuild'] = TRUE;
}

function neato_form_submit($form, &$form_state)
{
    $step = $form_state['step'];
    $form_state['storage']['step_' . $step] = $form_state['values']['step_' . $step];

    if(isset($form_state['values']['forward']) && $form_state['values']['op'] == $form_state['values']['forward'])
    {
        $form_state['step']++;
    }
    elseif(isset($form_state['values']['submit']) && $form_state['values']['op'] == $form_state['values']['submit'])
    {

        $items = array($form_state['storage']['step_1'], $form_state['storage']['step_2'], $form_state['storage']['step_3']);
        drupal_set_message(t('My !values', array('!values' => theme('item_list', array('items' => $items)))));

        $form_state['step'] = 1;
        $form_state['storage'] = array();
    }
    $form_state['rebuild'] = TRUE;
}

Answer: I think you should attach your custom ajax preloader function to the form as follows:
$form['#attached']['js'] = array(
  array(
    'type'  => 'file',
    'data' => drupal_get_path('module', 'MY_MODULE') . '/js/ajax_preloader.js',
  ),
);

ajax_preloader.js
(function($) {
  Drupal.behaviors.events = {
    attach: function(context, settings) {
      $('#steps-form', context).ajaxStart(function() {
        // ...
      });
      $('#steps-form', context).ajaxSuccess(function() {
        // ...
      });
    }
  };
})(jQuery);

alternatively, you can create your custom command (myPreloaderExample)
(function($, Drupal) {
  Drupal.ajax.prototype.commands.myPreloaderExample = function(ajax, response, status) {
    $('#steps-form').ajaxStart(function() {
      //...
    });
    $('#steps-form').ajaxSuccess(function() {
      // ...
    });
  };
}(jQuery, Drupal));

I'm not sure what your aim is, but if you're going to create a multistep ajax form with the following features:

Show only the current form item
Store all previously provided values (allow to preview all the values without loosing them)

You can create this kind of form as follows:
/**
 * An example of steps form.
 */
function steps_form($form, &$form_state) {

  // Because we have many fields with the same values, we have to set
  // #tree to be able to access them.
  $form['#tree'] = TRUE;

  $form['steps_fieldset'] = array(
    '#type' => 'fieldset',
    '#title' => t('Steps form'),
    // Set up the wrapper so that AJAX will be able to replace the fieldset.
    '#prefix' => '
',
    '#suffix' => '
',
  );

  $form['steps_fieldset']['actions'] = array(
    '#type' => 'actions'
  );

  // Build the fieldset with the proper number of steps.
  // Use $form_state['step'] to determine the number of textfields to build.
  if (empty($form_state['step'])) {
    $form_state['step'] = 1;
  }

  // An array with form values.
  if (empty($form_state['storage'])) {
    $form_state['storage'] = array();
  }

  // Define the number of steps.
  $form_state['steps_number'] = 3;

  // build all textfields with steps
  for ($i = 0; $i < $form_state['step']; $i++) {

    // set the type of each previous form item as a hidden. Only last step will be shown.
    $type = ($i == ($form_state['step']-1)) ? 'textfield' : 'hidden';

    // get the default value of step.
    $default_value = isset($form_state['storage'][$i]) ?  $form_state['storage'][$i] : '';

    $form['steps_fieldset']['step'][$i] = array(
      '#type' => $type,
      '#title' => t('Step') . ' ' . ($i+1),
      '#required' => FALSE,
      '#default_value' => $default_value,
    );
  }

  // The next step button.
  if ($form_state['step'] != $form_state['steps_number']) {
    $form['steps_fieldset']['actions']['add_step'] = array(
      '#type' => 'submit',
      '#weight' => '1',
      '#value' => t('Next step'),
      '#submit' => array('steps_form_add_one'),
      '#ajax' => array(
        'callback' => 'steps_form_callback',
        'wrapper' => 'steps-fieldset-wrapper',
      ),
    );
  }

  // The previous step button.
  if ($form_state['step'] > 1) {
    $form['steps_fieldset']['actions']['remove_step'] = array(
      '#type' => 'submit',
      '#weight' => '0',
      '#value' => t('Previous step'),
      '#submit' => array('steps_form_remove_one'),
      '#ajax' => array(
        'callback' => 'steps_form_callback',
        'wrapper' => 'steps-fieldset-wrapper',
      ),
    );
  }

  $form['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Submit'),
  );

  $form['#attached']['js'] = array(
    array(
      'type'  => 'file',
      'data' => drupal_get_path('module', 'MY_MODULE') . '/js/ajax_preloader.js',
    ),
  );
  return $form;
}

/**
 * Callback for both ajax-enabled buttons.
 *
 * Selects and returns the fieldset with all values in it.
 */
function steps_form_callback($form, $form_state) {

  $commands = array();
  // return the form
  $commands[] = ajax_command_html('#steps-fieldset-wrapper', render($form['steps_fieldset']));
  // add your custom ajax command
  $commands[] = array(
    'command' => 'myPreloaderExample',
    // define some custom variables if you need to
    'someValue' => 'some value',
  );

  return array('#type' => 'ajax', '#commands' => $commands);
}

/**
 * Submit handler for the "add-one-more" button.
 *
 * - save the form value, so the user will be able to preview all previously provided values without loosing them.
 * - increments the step counter.
 * - rebuild the form.
 */
function steps_form_add_one($form, &$form_state) {

  $form_state['storage'][$form_state['step']-1] = $form_state['input']['steps_fieldset']['step'][$form_state['step']-1];
  if ($form_state['step'] < $form_state['steps_number']) {
    $form_state['step']++;
  }
  $form_state['rebuild'] = TRUE;
}

/**
 * Submit handler for the "remove one" button.
 *
 * Decrements the step counter and causes a form rebuild.
 */
function steps_form_remove_one($form, &$form_state) {
  if ($form_state['step'] > 1) {
    $form_state['step']--;
  }
  $form_state['rebuild'] = TRUE;
}

/**
 * Final submit handler.
 *
 * Reports what values were finally set.
 */
function steps_form_submit($form, &$form_state) {
  $output = t('All my values: @steps',
    array('@steps' => implode(', ', $form_state['values']['steps_fieldset']['step'])) );
  drupal_set_message($output);
}

Results

Adding Page break to multi step form


Background Images In UITableView Sections Hides Section Titles

Question: I am using the following method to set titles in the section's header of UITableView, works fine.
- (NSString *)tableView:(UITableView *)tableView titleForHeaderInSection:(NSInteger)section
{
    NSString *sectionHeader = nil;

    if (section == 0)
    {
        sectionHeader = nil;
    }
    if(section == 1)
    {
        sectionHeader = @"Categories";
    }
    if (section == 2) {
        sectionHeader = @"General";
    }
    return sectionHeader;
}

Using the following method, i am setting the height for header in section, works fine.
- (CGFloat)tableView:(UITableView *)tableView heightForHeaderInSection:(NSInteger)section
{
    return 20;
}

BUT, as soon as i set the background images in the section header using the following method, the titles in section's header disappears and become empty but of course with the image place.
- (UIView *) tableView:(UITableView *)tableView viewForHeaderInSection:(NSInteger)section
{

    UIView *headerView = [[UIView alloc] initWithFrame:CGRectMake(0, 0, tableView.bounds.size.width, 20)];
    UIImageView *headerImage = [[UIImageView alloc] initWithImage:[UIImage imageNamed:@"section_background.png"]];

    headerImage.frame = CGRectMake(0, 0, tableView.bounds.size.width, 20);

    [headerView addSubview:headerImage];

    return headerView;
}

How can i set the titles in section headers without removing the background images in it. Can somebody be kind enough to point it out. Thanks in advance.
Comment: Check this
http://stackoverflow.com/a/10168961

this will help you...
Answer: It happening because your headerView overlap on you title of Header. so Best way is...also add UILabel To your headerView. and give text of label as your title os header in section.
I try with following code .. use it as your way, may be helpful in your case.
- (UIView *) tableView:(UITableView *)tableView viewForHeaderInSection:(NSInteger)section
{

    UIView *headerView = [[UIView alloc] initWithFrame:CGRectMake(0, 0, tableView.bounds.size.width, 20)];
    UIImageView *headerImage = [[UIImageView alloc] initWithImage:[UIImage imageNamed:@"1.png"]]; //set your image/

    UILabel *headerLbl = [[UILabel alloc] initWithFrame:CGRectMake(15, 0, 205, 15)];//set as you need
    headerLbl.backgroundColor = [UIColor clearColor];
    headerLbl.text = [self.listOfHeader objectAtIndex:section];
    [headerImage addSubview:headerLbl];

    headerImage.frame = CGRectMake(0, 0, tableView.bounds.size.width, 20);

    [headerView addSubview:headerImage];

    return headerView;
}

Answer: Another approach by setting dimensions of CGRect directly from image H, W,
 - (UIView *)tableView:(UITableView *)tableView viewForHeaderInSection:(NSInteger)section {

 UIImage *myImage = [UIImage imageNamed:@"header.png"];
 UIView *tempView = [[UIView alloc] initWithFrame:CGRectMake(0, 0, myImage.size.width, myImage.size.height)];

 UIImageView *imageView = [[UIImageView alloc] initWithImage:myImage];
 imageView.frame = CGRectMake(0,0,myImage.size.width, myImage.size.height);

  headerLabel = [[UILabel alloc] initWithFrame:CGRectMake(15, 5, 300, 25)];
  headerLabel.font = [UIFont fontWithName:@"TrebuchetMS-Bold" size:15.0];
  headerLabel.backgroundColor = [UIColor clearColor];

  NSString *myString = [NSString stringWithFormat:@"This is Header ;)"];
  NSLog(@" headerLabel.text: %@", myString);
  headerLabel.text = myString;
 [imageView addSubview:headerLabel];

 [tempView addSubview:imageView];

 return tempView ;

 }

- (CGFloat)tableView:(UITableView *)tableView heightForHeaderInSection:(NSInteger)section {
 return 50;
 }


R: matrix of string of integers to array of integer counts

Question: I have a character matrix of strings of comma-separated integers:
> mat<-matrix(c(NA,"1",NA,"2,1","3","1,3,3"),nrow=2)
> mat
     [,1] [,2]  [,3]
[1,] NA   NA    "3"
[2,] "1"  "2,1" "1,3,3"

I desire to have an output which is a numeric array where the z indices represent the counts of the integers in the matrix:
, , 1

     [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]   1    1    1

, , 2

     [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]   NA   1    NA

, , 3

     [,1] [,2] [,3]
[1,]   NA   NA   1
[2,]   NA   NA   2

How may I accomplish this?
To have an idea of the scale of the data, the final array will be of dimensions ~20,000 x 2,000 x 200, and the matrix will be the first two dimensions of the array (20,000 x 2,000).
Answer: This uses a loop and probably won't be the most efficient solution:
mat<-matrix(c(NA,"1",NA,"2,1","3","1,3,3"),nrow=2)

#split the strings
temp <- strsplit(mat, ",", fixed=TRUE)

#unique values
levels <- na.omit(unique(do.call(c, temp)))

#convert to factors and use table
temp <- t(sapply(temp, function(x) table(factor(x, levels=levels))))

#make it an array
array(temp, c(nrow(mat), ncol(mat), length(levels)))
# , , 1
#
#      [,1] [,2] [,3]
# [1,]    0    0    0
# [2,]    1    1    1
#
# , , 2
#
#      [,1] [,2] [,3]
# [1,]    0    0    0
# [2,]    0    1    0
#
# , , 3
#
#      [,1] [,2] [,3]
# [1,]    0    0    1
# [2,]    0    0    2

Edit:
This avoids applying table and factor in a loop and should be faster:
temp <- strsplit(mat, ",", fixed=TRUE)

id <- rep(seq_along(temp), sapply(temp, length))
temp <- factor(do.call(c, temp))
array(t(table(temp, id)), c(nrow(mat), ncol(mat), length(levels(temp))))

Comment: Is there a away to replace sapply with mclapply here?
Comment: Of course you can do that. You'll just need `do.call(rbind, temp)` afterwards.


How do I stop a node submission programmatically?

Question: i'm using Drupal 8. I would like to stop the user to submit a node when he tap save, and show some message. How can I do it programmatically?
Thanks!
Answer: You can hide the form submit button using hook_form_alter() and show a link Save and Publish on form. And include a jQuery snippet to show message, which will be triggered when user clicks on a link Save and Publish.
Answer: If you want to prevent form submission and show message, you can add form validation to your form (using hook_form_FORM_ID_alter):
function MYMODULE_form_node_MYNODETYPE_form_alter(&$form, &$form_state, $form_id) {
  $form['#validate'][] = '_node_MYNODETYPE_form_validate';
}

function _node_MYNODETYPE_form_validate($form, $form_state) {
  // add some logic to show or not show message here
  $form_state->setErrorByName('field_MYFIELD', 'Message to show by my field');
}

Answer: To avoid saving the results, you must remove (or override) the save function from the form.
Try this:
function custom_module_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id)
{
        $form['actions']['submit']['#submit'][0] = "_custom_submit";
        unset($form['actions']['submit']['#submit'][1]); //Avoid save results
}


javascript : to fetch iframe element ,but it can not get the value

Question: i try to fetch the frame head value ,but it doesn't work well,look at the following code
1.php

/admin/index.php

Look the result
when i open http://127.0.0.1/1.php in firefox or chrome it become that followings
open in browser
but when i execute it in the console it come into that
enter image description here
so My question is Why my code can't get the value of Tag iframe
Comment: Please do not post images. Edit your question and post the code properly
Answer: You need to wrap your code under window load function or may be iframe load because iframe takes some time to load and you have to wait for it to be fully loadded.

Comment: Thanks ,I get it. And solve it successfully, By following code.
setTimeout("a = window.top.frames[0].a;alert(a)", 3000 )


Como guardar e recuperar vários retornos de uma requisição AJAX em loop?

Question: Senhores, boa noite. Estou tentando realizar a seguinte tarefa, estou enviando via AJAX a uma outra página 3 cep's, ao chegar lá eu dou apenas um echo do cep enviado pra saber que chegou, esse echo entra na verificação de sucesso aonde eu testo a validade, ou seja, diferente de vazio e entra a próxima ação. Quando dou apenas um alert(data) ele mostra os ceps um de cada vez conforme loop, porém se eu digo pra realizar um incremento a cada vez e já fora do loop eu dou um alert de n então o alert resulta em 0 (zero).
Me desculpem a ignorância caso a pergunta seja boba ou eu que esteja cometendo algum erro trivial.
O intuito desse teste é que a ideia é chegar na seguinte solução, um db que contem dados precisa receber dados do csv desde que esses dados já não existam no banco, logo pensei na seguinte solução, a a pagina um carrega o csv, chama o ajax, o ajax chama a pagina php responsável pela verificação e insert e sempre que esta retornar 1 (um) o ajax faz um incremento, por que isso, pra poder informar ao usuário que de X registros importados N foram inseridos.
function testeAjax() {

var cep = ['cep 01', 'cep 02','cep 03'];
var n = 0;
for(var c=0 ; c < cep.length ; c++){

    jQuery.ajax({
        type: "POST",
        url: "./_inc/controler/uploadArquivosControle.php",
        data: { acao:'insertUpdate', cep: cep[c] },
        success: function( data ){

            if(data!==''){
                n++;
            }

        }

    });

}
alert(n);
}

<?php
$acao = $_REQUEST['acao'];

$dao = new UploadArquivosDAO();

switch ($acao) :
    case 'insertUpdate':

        $cep      = addslashes($_POST['cep']);
        echo $cep;
    break;
endswitch;

Comment: O problema é que quando chega na linha `alert(n);` o Ajax ainda não foi processado porque ele é assíncrono.
Comment: Imagine a situação comparativa em que os Ajax sejam 3 emails que vc manda para um amigo e espera uma resposta de cada mensagem enviada. As respostas não vêem no mesmo momento e ao chegar na linha do alert vc não teve nenhuma resposta ainda. Ou seja, seu amigo ainda vai ter que abrir o email dele, ler a mensagem e responder, e isso não acontece na mesma hora que vc mandou as mensagens.
Answer: Como havia comentado, o Ajax é assíncrono. O código dentro do for é chamado mas o alert que vem logo depois é executado imediatamente antes do Ajax dar resposta.
Neste caso o correto seria usar a função de forma recursiva e ir chamando cada Ajax após o anterior retornar a requisição no success. Como no exemplo você deseja fazer 3 requisições Ajax (tamanho da array cep), faça um if verificando se a variável que conta o número de requisições é menor que o tamanho da array: se for menor, você faz uma nova requisição; se não, dispara o alert.
Crie a variável que conta as requisições fora da função usando um nome específico para o caso para não conflitar com outras possíveis variáveis que vier a usar (no caso usei num_cep). Essa variável começa com o valor 0. A cada success você irá incrementar a variável e chamar novamente a função. Quando o valor da variável for igual ao número de itens da array cep, o if cairá no else disparando o alert com o número de requisições efetuadas que retornaram !== '' na variável inseriu:
var num_cep = 0;
var inseriu = 0;
function testeAjax() {

   var cep = ['cep 01', 'cep 02','cep 03'];
   var cep_length = cep.length;

   if(num_cep < cep_length){

       jQuery.ajax({
           type: "POST",
           url: "./_inc/controler/uploadArquivosControle.php",
           data: { acao:'insertUpdate', cep: cep[num_cep] },
           success: function( data ){
               if(data!==''){
                   inseriu++;
               }
               num_cep++;
               testeAjax();

           }

       });

   }else{
      alert(inseriu);
      // reseta as variáveis
      num_cep = 0;
      inseriu = 0;

   }
}


Send an email as a google group from python

Question: I want to send email in python and the following code works. However I want to send the email as a google group. Since a google group has no password, I am not able to login to the server. Is there anyway I can go about this?
<code>def sendEmail(self, toEmail, subject, message ):
    msg = MIMEMultipart()
    password = "*****"
    msg['From'] = "email@example.com"
    msg['To'] = toEmail
    msg['Subject'] = subject
    msg.attach(MIMEText(message, 'plain'))
    server = smtplib.SMTP('smtp.gmail.com: 587')
    server.starttls()
    server.login(msg['From'],password)

    server.sendmail(msg['From'], msg['To'].split(","), msg.as_string())
    server.quit()
    logging.debug('sent email to %s', (msg['To']))
</code>
Answer: It sends like you should use a Service Account to send the email on your behalf.
Here is a pretty good guide (not written by me): https://medium.com/lyfepedia/sending-emails-with-gmail-api-and-python-49474e32c81f
Answer: You need to add group address as allowed send-as address in Gmail settings. Then you'll be able to use group address as "From" address. Authentication should still be done with your (non-group) address, of course.