Syntax, Semantics and related foundational questions I'm currently taking a graduate course in Mathematical Logic, and, just to give some context, so far we've seen 1st order languages, some set theory and some model theory (Completeness and Lowenheim Skolem). The fact is that we're not even mentioning some profound questions and issues that naturally arise when seeing this theory for the first time in an advanced perspective. I've come to build, during these weeks, some intuitions, arguments and questions that lie in the feeble boundary between logic and philosophy. 

The two main problematic subjects that have come to my attention are the relationship between syntax and semantics (Tarski's) and the sort of dual nature of ZFC, which can be seen both as an axiomatic theory that precedes the first order languages (and that is used to formalize them), and that moreover can (a posteriori) be seen as a first order theory. 

In particular: 
The choice of semantically interpret theories with structures, which are sets, denotes to me the underlying assumption that sets are the only "true" extra-linguistic entities that exist and that we necessarly refer to when using the mathematical language.  

In this perspective the ZFC theory that precedes first order formalization, can be seen as a metalinguistic attempt to explain how these entities (sets) actually work in "reality" (whatever it means).
Afterwards, when we want to formalize (first order) languages, we cannot but rely on those entities (sets) and on the metalinguistic theory that describes them (ZFC).
So the Completeness theorem can be seen as a bond between the universe of sets and the actual reality, since it guarantees the strict coherence between the semantic proofs (done inside the realm of sets) and syntactical proofs, which consist of a mechanical manipulation of finite strings of symbols, which can be done by an actual machine (computer).

Moreover, starting from this given and chaotic universe of sets, the Von Neumann Hierarchy arises once we search for order. We can in fact find (by defininig them in the ZFC-metalanguage) ordinals, and after that we can construct the Von Neumann Hierarchy. Also Cardinals arise from the search for a "canonical" equivalence relation inside this universe.
When we study $\mathcal{L}_{\in}$-structures, we cannot but rely on this pre-existing universe, which provides us with the actual sets that will make up the model. 

Finally:


Does this assumption of pre-existing sets actually underlie the semantic interpretation of theories?
Does the formalist approach only consider the syntactical dimension? Investigating semantics would necessarly imply the existance of an extra-linguistic reality that the language is referring to.
Is this interpretation somewhat coherent?


Just a curiosity: have someone recently tried to base mathematics on a pre-cognitive level (sortoflike Kant tried to do) in a constructivist (epistemological) fashion?
You could spend a lifetime on these questions. So, be careful about being curious.

One problem related to your questions is that of numerical identity.  Logicians and analytical philosophers seem content with simply stipulating that the sign of equality means numerical identity.  The history of the matter is that numerical identity and numerical difference are often attributed to cognitive experience. Kant invoked spatial intuition against the principle of the identity of indiscernibles, Wittgenstein asserted that names were like geometric points and treated the sign of equality as eliminable, Black wrote a dialogue using spatial symmetry to discredit the Leibnizian principle.  You will probably find the best account of the situation in Strawson's book, "Individuals".  He discusses quantitative identity, qualitative identity, and the skeptical arguments used to deny the cognitive basis for numerical identity.

Look at the appendix of Kreisel's book on mathematical logic to find the passage reasserting what has always been the case for symbolic methods.  The cognitive experience of a (bounded region) linguistic symbol on a page is taken as primitive sense data. Hilbert called this the a prioriness of symbol.  So, when you speak of occurrences of symbols in your syntactic studies, what you understand as a multiplicity is still relying upon what promoters of a strict dichotomy between syntax and semantics want to deny.

If you find the passage from Strawson, you will be told the problem -- locations differentiate parts of space and parts of space separate locations.

One thing that this means is that there is an amibiguity between membership as a set-theoretic relation and geometric incidence.  This is further compounded by the fact that some practitioners interpret sets simply as comprehensions while others interpret sets as collections taken as objects.  Bolzano definitely distinguished between the two, Cantor probably did, and the category-theoretic account of set developed by Lawvere certainly does.

Within the comments of the last paragraph lies a problem that urelements are not really compatible with the geometric account.  In that respect, sets based on comprehensions are fundamentally different, and, if one is talking about sets of symbols in metamathematical descriptions of formal systems, one is using sets composed of urelements. Zermelo-Fraenkel set theory admits the ambiguity associated with the cognition of numerical identity when one discusses it as a theory of pure sets.  Zermelo originally associated the identity of elements with singletons.  So, sets had been determined by elements and elements had been determined by sets. Skolem's criticism of Zermelo's concept of properties led to the study of the system as a first-order theory.

One aspect of first-order logic is its reliance upon the necessary truth of reflexive equality statements. To the extent that logicism and logical atomism have their origin in metaphysical narratives, this is the infamous law of identity. Formalism is more heavily influenced by linguistic analysis. It requires the necessary truth of reflexive equality statements to enforce the uniform interpretation of symbols with the same shape (cognitive basis). But, Tarski's semantic conception of truth does not dictate the necessary truth of reflexive equality statements.

Informal mathematical practice generally treats definition and existence separately.  Hilbert and Bernays tried to develop mathematical logic with this in mind. But associating well formedness with the provability of existence creates difficulties for a semantic theory. In so far as this has been addressed, one would be looking at negative free logic.  This logic has some features comparable with the partial functions of recursion theory.  And, if one understands equality in the sense of Tarski's transitivity axiom from cylindric algebra, one can use reflexive equality statements as the denotation predicate mentioned at the end of Abraham Robinson's paper,  "On Constrained Denotation".  Free logics use a separate operator to convey the existential import of a term. Tarski's axiom for transitivity actually incorporates an existential quantifier into the account of equality. This is why it can be used to develop a free logic without the special operator.

You will find mention of Tarski's work on cylindric algebra in "Sets for Mathematics" by Rosebrugh and Lawvere.

Do not think that mathematical logic as presented in your introductory coursework is not disputed. The expression can have several meanings.  And, there are several distinct paradigms, of which you are now learning just one.
You could spend a lifetime on these questions. So, be careful about being curious.

One problem related to your questions is that of numerical identity.  Logicians and analytical philosophers seem content with simply stipulating that the sign of equality means numerical identity.  The history of the matter is that numerical identity and numerical difference are often attributed to cognitive experience. Kant invoked spatial intuition against the principle of the identity of indiscernibles, Wittgenstein asserted that names were like geometric points and treated the sign of equality as eliminable, Black wrote a dialogue using spatial symmetry to discredit the Leibnizian principle.  You will probably find the best account of the situation in Strawson's book, "Individuals".  He discusses quantitative identity, qualitative identity, and the skeptical arguments used to deny the cognitive basis for numerical identity.

Look at the appendix of Kreisel's book on mathematical logic to find the passage reasserting what has always been the case for symbolic methods.  The cognitive experience of a (bounded region) linguistic symbol on a page is taken as primitive sense data. Hilbert called this the a prioriness of symbol.  So, when you speak of occurrences of symbols in your syntactic studies, what you understand as a multiplicity is still relying upon what promoters of a strict dichotomy between syntax and semantics want to deny.

If you find the passage from Strawson, you will be told the problem -- locations differentiate parts of space and parts of space separate locations.

One thing that this means is that there is an amibiguity between membership as a set-theoretic relation and geometric incidence.  This is further compounded by the fact that some practitioners interpret sets simply as comprehensions while others interpret sets as collections taken as objects.  Bolzano definitely distinguished between the two, Cantor probably did, and the category-theoretic account of set developed by Lawvere certainly does.

Within the comments of the last paragraph lies a problem that urelements are not really compatible with the geometric account.  In that respect, sets based on comprehensions are fundamentally different, and, if one is talking about sets of symbols in metamathematical descriptions of formal systems, one is using sets composed of urelements. Zermelo-Fraenkel set theory admits the ambiguity associated with the cognition of numerical identity when one discusses it as a theory of pure sets.  Zermelo originally associated the identity of elements with singletons.  So, sets had been determined by elements and elements had been determined by sets. Skolem's criticism of Zermelo's concept of properties led to the study of the system as a first-order theory.

One aspect of first-order logic is its reliance upon the necessary truth of reflexive equality statements. To the extent that logicism and logical atomism have their origin in metaphysical narratives, this is the infamous law of identity. Formalism is more heavily influenced by linguistic analysis. It requires the necessary truth of reflexive equality statements to enforce the uniform interpretation of symbols with the same shape (cognitive basis). But, Tarski's semantic conception of truth does not dictate the necessary truth of reflexive equality statements.

Informal mathematical practice generally treats definition and existence separately.  Hilbert and Bernays tried to develop mathematical logic with this in mind. But associating well formedness with the provability of existence creates difficulties for a semantic theory. In so far as this has been addressed, one would be looking at negative free logic.  This logic has some features comparable with the partial functions of recursion theory.  And, if one understands equality in the sense of Tarski's transitivity axiom from cylindric algebra, one can use reflexive equality statements as the denotation predicate mentioned at the end of Abraham Robinson's paper,  "On Constrained Denotation".  Free logics use a separate operator to convey the existential import of a term. Tarski's axiom for transitivity actually incorporates an existential quantifier into the account of equality. This is why it can be used to develop a free logic without the special operator.

You will find mention of Tarski's work on cylindric algebra in "Sets for Mathematics" by Rosebrugh and Lawvere.

Do not think that mathematical logic as presented in your introductory coursework is not disputed. The expression can have several meanings.  And, there are several distinct paradigms, of which you are now learning just one.