Re-representation is a critical ability to (i) understanding human creative problem solving, and (ii) modeling computational cognitive systems able to support or perform creative problem solving tasks on their own. This paper proposes a unified multi-level cognitive approach to investigating re-representation: the study of sensory-based, concept-based and problem template based possible forms of re-representations in an integrated manner. Descriptions and explanations of each level prepare the ground for further computational modeling. A study is deployed in order to explore the relationship between the various tasks proposed to reflect re-representation. A significant correlation between the investigated tasks is discovered. Two previous studies from the literature are replicated. A new strong and significant relationship between the Pattern Meanings Test and the Alternative Uses Test is observed. Imagine that you are given a problem, and try to solve it in vain for half an hour. After a while, you take a break from it and focus on something else. In the middle of this other activity, an altogether different way of seeing your previous problem pops into your mind. It feels like the elements of the problem have shifted, have reorganized in the mean time; that now you understand what is truly important, and how you should have tackled the problem in the first place. You experience a moment of clarity and promise: this new way of seeing the problem may allow you to find the solution, or at least to make further progress. It is quite possible you have, in the case above, experienced a case of re-representation.Re-representation is a human cognitive ability encountered in processes of creativity and creative problem-solving. From a computational perspective, the better understanding and modeling of re-representation will enable us not only to study a cognitive skill to a deeper level, but also to implement the next generation of knowledge discovery and creative problem solving systems of cognitive inspiration. A computational system endowed with re-representation would have the same ability to view problems and datasets in different ways, generating new creative and insightful solution paths and discovering new knowledge. With re-representation being a cognitive process likely to become the next challenge in knowledge based systems, the main research questions addressed here are: (i) how can we better understand, characterize and empirically study re-representational abilities of the human mind, (ii) how can we begin to conceptualize and model them via computational modeling techniques? and (iii) is there promise that existing tasks from the empirical literature can be used to study re-representation in a cohesive manner?Various cognitive abilities related to creativity have been studied in the literature, with computational systems capable of similar feats sometimes being implemented. Example of such abilities are: remote association  is repeatedly re-represented by the participants through various known representations.The number of alternate interpretations that participants come up with is part of the evaluation of results in pattern and line meaning tests. This type of interpretation and re-representation can still be positioned at the level of interpreting features as objects. Unlike in the ambiguous figures task, the objects through which the re-representation is made are not restricted to two, nor present in the figure; they rely on the knowledge of the solver. Because of this, more than two interpretations are also possible .Alternative Uses test by Guilford et al. . Then, the affordances of the objects they were re-represented into can be applied back to the initial objects . This very inference is actually part of the set of creative inferences made by a system computationally implementing this process  can be done is the following: after the object has been given , its properties  and their combinations are re-represented as other possible objects  in various relationships are presented to the solver. For example, the weight problem  that can help them solve the problem. After a while, it becomes clear that one of the core issues of solving the problem is the ability to attach one of the nails on the table to the ceiling, in order to fix the string on the ceiling and attach the pendulum to it. Various ways of achieving this are attempted. A productive path is the endeavor to create a hammer out of existing objects—attempts to realize this involve human solvers proposing to use the top of the staircase as a hammer, wrapping the spool of string or folded plastic glasses in a sock in order to make them “harder,” or, Duncker's correct solution, using the pendulum itself to hammer in the nail before attaching it to a piece of string. This shows re-representation at work in insight problems—re-representing some objects (the pendulum) as another needed object (the hammer)—in a flavor which is similar to OROC's processes when solving the AUT. However, multiple objects and sets of actions might also be represented in multiple ways, using already known ways of solving problems  and their subsets. A deeper examination and example of this level are shown in section 5.the ability to re-represent features as different sets of objects or images in the case of ambiguous figures;the ability to re-represent objects and their properties as different objects; we will generally discuss this in a household domain environment, focusing on the ability to use objects creatively when traditional objects that would be used are missingthe ability to re-represent objects and scenes under different problem templates, as to be able to computationally tackle the solving of human insight problems in the future.In summary, the three levels at which re-representation can be explored proposed here are:An overview of the various levels of re-representation and empirical tasks with which re-representation could be studied at that level is presented in rho = 0.28, p = 0.007). Doherty and Mair .Some preliminary evidence points in the direction of the validity of such an approach: various theories have anticipated the relation between ambiguous figures and creativity, the earliest being expressed in the gestaltist movement . and Mair  have shoIn the following, each level of re-representation will be analyzed in computational terms, using the context of the tasks of which an overview was provided in section 2. The levels will thus be defined in terms of input, pre-existing knowledge in the cognitive system, output, and processes of re-representation.input – defines the initial sensory or conceptual input of the task (in objective terms), be it features, objects, their properties, object groups, scenes or problems, or descriptions thereof;the pre-existing knowledge – estimates what kind of knowledge needs to be assumed to exist in the system, in order for the processes of re-representation and various types of output to be possible;the output – represents the types of images, objects, uses, object groups and plans of action (or representations thereof) that are the results of the problem representation and re-representation process;the re-representation – the process which we are trying to observe at these various levels, takes into account existing knowledge and input, as to produce various forms of output.These are described below:The following sections (3–5) explore how each of the previously sketched levels of re-representation can be described in such computational modeling terms.Level one refers to the ability to re-represent features as different objects and images. The ambiguous figures tasks is tackled in section 3.1. The pattern and line meaning tasks are explored together in section 3.2. Section 3.3 provides a short summary of the analytically observed traits of re-representation at this level.The objective input in the duck-rabbit figure can be taken as a set of visual features. The output is two images (or image interpretations). The pre-existing knowledge is the knowledge of the object represented in the image, or sometimes familiarity with such a type of representation of this objectIn order to interpret such features as being part of and representing an object, a subset of these features has to be selected, grouped and matched to a known object or image. Some such sets of features can be pre-grouped in feature groups which can be mapped to various object parts, as shown in fg1 as a beak or ears, fg5 as a feathers or fur texture and fg6 as the back of the head or the pout might make all the difference to seeing one of the images, or the other. It is unclear in which order the process happens: it might be that (i) cognitive systems directly cast features in terms of one interpretation or the other (a top down view), or that (ii) the various sets of features trigger the body parts, which then trigger the interpretation of the image (a bottom up view) or that (iii) one of the features, on which the system is focusing, triggers the bigger picture, which in turn triggers the interpretation of the body parts (a mixed view)In the case shown in switch between these representations is the ability to (a) switch between a potential mapping of the (objective) features to different feature groups and/or objects or (b) switch between different interpretations of these feature groups as objects. In the case of the duck-rabbit illusion, a switch might very well be influenced by (b) - the ability to re-categorize one of the initial sets of feature groups as a different animal body part. However, in the case of different ambiguous figures, where feature groups cannot be allocated very neatly to one or another interpretation, features groups themselves might need a different re-grouping to be able to map to (or under the mapping of) different body parts. Thus (c) the ability to switch between different objects as possible interpretations of features and thus arrange them in feature groups might come into play. All three switches are different facets of the re-representation process.However, it is clear that (images of) ducks and rabbits must have been encountered previously. The ability to Before moving on to the pattern and line meanings task, it is worth noting that not all ambiguous features are about a re-representation of visual features as different groups or an interpretation of such groups as different categories. For example, in the girl-saxophonist illusion , the relFurthermore, sometimes such relations are not between visual features groups, but rather between interpretations of spatial relations. For example, In the pattern and line meanings tasks, the input is still a set of features. The output consists of different objects, groups of objects and potentially scenes , if the measure of ambiguity is how many mappings can be made from the stimulus to known objects or scenes representations.The figure stimuli are also much more abstract than the ones presented in the ambiguous figures task, and many more features need to be added to construct a full representation. Ambiguous figures can be considered as “abstract” stimuli as well of course, in the sense that they are not complete 2D depictions of the object (like a full image), but rather drawings or sketches. However, the sets of features in the ambiguous figures contain much more information about the figure being represented, thus constituting a much more complete representation.Ambiguous figures rely on the identification of an already known pattern, which pre-exists in the memory of the system (or has been experienced in a similar form), with the main task being one of discriminating which features to take into account and in which role. The pattern and line meanings task rely much heavier on memory, as multiple known similar patterns can be elicited, and, in a sense, the more such patterns a system has acquired, the higher its potential performance in such a test, if re-representational abilities are in place . The same can be said about the line meanings test , the butterfly-shaped kite answer. We observe contour shape modifications: for example, the circle in the initial stimulus has to be compressed to an oval to match (j), the space needle elevator. Other types of pattern completion do exist: for example, it is the contour of the 5 circles that seems to determine answers (b) and (k). Only three such circles are selected for the group required in answer (d) arrow, and the pattern completion is done to the shape of a triangle—the three circles form its vertices. The number might play a role in answers (f) hand and arm and (g) five fingertips. Answer (i) theme park ride contains both a mapping of the initial circles and of the imaginary completion one, whilst possibly (f), but more clearly (g) map the initial circles too.Procedures of interpretation of the various parts of the stimuli as a group are at play from the very beginning. We also see Gestaltic procedures of completion—thus the five circles of the initial stimulus are interpreted as representing a circle in a variety of answers, like in (a),(c),(e), etc. We observe procedures such as tilting of the various objects—like the circle in (h), the These examples speak about a process of parsing and re-parsing initial features, while using them to search through a variety of known objects, in multiple inclinations and poses. Besides being answers, these can also be interpreted as visual (imaginary) comparisons of known objects to existing features of the stimulus, while searching for possible matches.Also, various types of similarity of matches can exist, depending on which visual properties are involved—for example, a set of shape contour features initially taken from a mushroom can be mapped to an icecream shape, however a different set  could trigger the possibility of re-representing the mushroom as an umbrella. Thus parts of the visual pattern might trigger certain possible interpretations, while other features might trigger othersThe first level refers to the interpretation of grouped or ungrouped sets of features as possible known images, objects or scenes. The input in this case is features. The output is images, like in the ambiguous figures task, and objects or scenes, like in the pattern meanings and line meanings task. These objects need to have been previously known, though the particular amount of knowledge required by each task can be different.The process of re-representation is, in this case, the process of mapping a set of input features to different types of known objects and scenes, and switching between such various mappings. This output is interpreted as being represented, in an incomplete form, in the input: as sketches (ambiguous figures), or as allusions to an object or a set of objects/visual scene (in the line meanings and pattern meanings tasks).A newspaper can be used as an umbrella or A cup can be used as a bell) and coming up with creative uses for existing objects (like A wooly shoe can be used to wipe kitchen surfaces with). Some such forms of creative inference are imperfect: some properties and/or some object parts might be missing from the initial object, in order for it to be properly transformed into or re-represented as a different object. An example of properties missing: a magazine, made of glossy paper, and preferably of a large size, might work much better as an umbrella than a newspaper; some of the newspaper's properties (being made of paper) come slightly short of getting the job done as an umbrella. As for missing parts, a cup is not yet a bell, it still requires other parts, perhaps a spoon to hit it with or some other form of clapperLevel two refers to the ability of using existing objects as if they were other objects (The input to level two is objects and their properties. The output is (a) other objects and/or (b) new affordances for existing objects in the specific case of the Alternative Uses test.I can use a shoe if I don't have a hammer, to hammer in the nail. The shoe and the hammer are already known, while the creative inference might be known or made on the spot. If re-representation is indeed at play at this level, the assertion A cup can be used as a bell can be taken as I can re-represent the shape and material features of a cup as the body of a bell. In a more general case, assertions of the form I can use objectAto replace objectB, in the context of taskX can be taken as I can re-represent (a subset of) the features and properties of objectB, or the features and properties required for taskX, as objectA.The output is other objects (case a), when the re-representation is done overtly, referring to objects: A brick can be used to hit a nail can be taken as I can re-represent (a subset of) the features and properties of a hammer, which has the affordance of putting a nail in the wall, as a brick. More generally, answers of the type I can use object A for affordance X, can be taken as I can re-represent (a subset of) the features and properties of object B, or features and properties connected to affordance X, as object A.The output can be an affordance (case b), like in the case of the Alternative Uses test, when the answer given by a participant is that a brick can be used to hit a nail, driving it through the wall. A covert reference to known objects which one has performed this task with  might be made in the process. The initial object, the covert referent object and its affordances must all be known for the creative inference to be made. Answers that What can you use a cup for?' is shown in A small set of answers to the Alternative Uses test question ‘Cup. OROC has previously encoded a Cup with a set of material, shape, size, affordance and other properties. It then uses its type of knowledge organization to search for other objects which it has encoded in properties of the required object—thus finding and proposing for the task objects like a Bowl, Vase and a Bucket, which have similar material and shape.An example of an implementation of these principles is the OROC system  in such a problem is much easier to talk about in concrete terms  than in abstract insight problems (which might involve abstract re-representations).problem templates . This involves the re-representation of elements of the problems like the paper, as shown in rolling shirt of the person in the room into a piece of rope, as shown in shirt object as a form of a rope object;rolling the T-paper, T-shirt), like in connecting strings or items that can be re-represented as a string , attaching the heavy object to one of the strings, and using its new affordance as a pendulum to swing it around, having it move toward you while you are holding on to the first string. This requires recasting or re-representing the string and pliers or the string and jar objects and/or their properties through the template of a pendulum. Thus, an object template is used to re-represent part of the problem objects, leading to a new type of affordance, and a possible solution.or, the actual acknowledged solution to this problem—building a pendulum like the one in While attempting to solve this problem, human participants come up with different possible fragments of the solution  or with constructions which point to a process of re-representation of the problem objects through various problem and action templates, or object templates. For example, some people try to build solutions which involve:Some such problem solving fragments might involve simple grouping of existing elements like  in an exSome object groupings and higher level representations of sets of objects as one functional unit might thus trigger other objects in the scene to be grouped in a certain way, in the same manner in which interpreting a set of visual features as part of a certain group might trigger or be related to already existing possible interpretations of the entire imageFurthermore, the use of various templates allows for others to be deployed, as shown in the previous examples: climbing on the chair is a necessary part of hammering the ropes to the ceiling; rolling the paper or the cloth is necessary in deploying another template which involves lengthening and connecting the ropes, or whatever objects are now re-represented as the ropes, etc. pliers have been re-represented as a hammer and part of a fixing the string to the ceiling with a nail problem template, human solvers might encounter difficulties in re-representing (the same pliers) as a weight, part of a pendulum template. This is similar to what we remarked in the case of some ambiguous figures—when one set of features has been represented as a nose, they or an overlapping subset cannot be represented as a saxophone at the same time. Re-representation thus is required here too, just at a different level.The use of certain representations precludes or at least impedes others being used. For example, if the To explore whether relations between these tasks can be found empirically, and thus re-representation can be studied at multiple levels, a study involving these tasks was deployed and is reported in the following section.n = 8), 20–30 years (n = 30), 30–40 years (n = 49), 40–50 years (n = 23), 50–60 years (n = 24), 60–70 years (n = 3), and over 70 years (n = 1). Most participants had higher education with 58 (42%) having completed undergraduate courses and 16 (11.6%) having completed postgraduate courses. Nine (6.5%) were enrolled in postgraduate courses, 12 (8.7%) in undergraduate courses, 31 (22.5%) had finished their high school diploma, and 12 (8.7%) finished secondary school. Using a 5-point Likert-scale, self-ratings of both creativity  and problem-solving  were gathered.A sample of 175 participants was recruited using Crowdflower, 37 of which had to be excluded due to giving incomplete or nonsensical answers. This resulted in a total sample size of 138 participants  whose answers were used for further analysis. Ages were recorded using an ordinal scale with age brackets of 10 years. The age ranges of the participants were: under 20 years .“Jack and Jill are arguing about who weighs more. What could they do to find out for certain?” (Jack and Jill weight problem).“Each of the problems was accompanied by a picture, both of which can be found in the SD = 21.12), and were able to switch between representations about every 3 seconds.The results for the ambiguous figures task can be seen in Additionally, the number of key presses over time for all participants was plotted using frequency polygons. This was done for pictures 1–5 individually and for all pictures combined. These graphs can be seen in Fluency is the number of answers given; Flexibility measures how many semantic domains these answers cover, based on human ratings. Additionally, response times per pattern , and response times per meaning are shown. The response times per meaning were not recorded directly—they are averaged from the number of responses produced and the total response time per pattern.Results for the pattern meanings test are shown in Fluency and Flexibility constitute the same measures as in the pattern meanings test, only for the number of alternative uses named for each given object. Response times per object and response time per use can also be found in the table.Results can be found in Answers were rated for their correctness. Answers that included parts of the right solution received a score of 0.5. For the candle problem 51 participants (37%) had the right answer, 48 (35%) answered partly right, and 39 (28%) did not give the right answer to the problem. Four participants (3%) came up with the correct solution to the Jack and Jill weight problem, and one (1%) answered partly right.Fluency in the pattern meanings test (α = 0.91), in the alternative uses task (α = 0.93) and for the number of key presses in the ambiguous figures task (α = 0.89). As all values reached satisfying levels, mean values were calculated for each task and used as an indicator for performance.For reliability analysis, Cronbach's alpha was calculated for Fluency in both the pattern meaning test and the alternative uses task, as well as for performance in the ambiguous figures task were correlated with each other. A strong . Correlations of Fluency values in the pattern meanings test  and in the alternative uses task  with the ambiguous figures task had a medium effect size.The so computed indicators of Fluency as indicators for performance. The Kruskal-Wallis test revealed differences in Fluency for the pattern meanings test  and the alternative uses task , but not for performance on the ambiguous figures task . Pairwise comparisons revealed that for the pattern meanings test there was a significant difference for the comparison correct—not correct (p < 0.01). Additionally, there were significant differences for the alternative uses task; here, Fluency differed significantly between the groups correct—partly correct (p < 0.05), and correct—not correct (p < 0.001).A possible connection between performance on the insight problems and the other tasks was explored using the Kruskal-Wallis test by ranks. As only so few participants had the right answer to the Jack and Jill weight problem, only the answers to the candle problem were considered. In this section, all three levels of re-representation are shown in the context of a unified model.Feature Space Level: At the feature space level, the input stimulus I is formally represented as a set of features:n = total number of individual features perceived fgi = ith feature of the input stimulus. The re-representation here involves matching a subset of features of the input to known visual patterns; this type of re-representation involves grouping the features under various known patterns, searching and comparing them to patterns and objects in KB and then interpreting them as one of these known patterns or objects.Ambiguous Figures testA) fg1−fg6 are grouped together and fg1, fg5, fg6 are matched to the visual stimulus of beak, feathers and backofhead, respectively, the figure is represented as a duck. By contrast, when the features fg1, fg5, fg6 are matched to the visual stimulus of ears, fur and pout, the figure is represented as a rabbit. Re-representation in this case is the process of regrouping the initial sets of features in different ways and matching them to a different pattern.In the context of ambiguous figures, re-representation happens as follows. Given the rabbit-duck problem in Pattern and line meaning testIn the context of the patterns meanings test  ngs test , when thlollipop . When thtriangle , the setand palm . The varIt is important to mention that even when the same set of features are focused upon, there can be multiple re-representation outcomes. For example, in the same set of features are being focused upon in both (a) and (b); however in (a) the set of features are re-represented as a lollipop while in (b) as a tree. Re-representation in this case is the ability to regroup the features and re-map them to a different known pattern or object.Concept Level: The object level of re-representation can happen at follows. In the CreaCogs framework ,fs_material refers to the feature space of material in the KB,fs_color refers to the feature space of colors in the KB,fs_size refers to the feature space of height, depth and width of the objects in the KB,fs_affordance refers to the feature space of actions/motions that can be done using the object,fs_name refers to the feature space of object names in the KB .An object/concept X in the CreaCogs framework is described as an activation of its features as follows:According to this scheme of formalization, the object cup and vase can be described as:I can use object B to replace object A, in the context of task X can be taken as I can re-represent (a subset of) the features and properties of object A (which have been connected to task X) as object B.The re-representation at this level is done by comparing the original object to the other object in terms of object properties such as material, shape, size etc. This is shown in pot  are similar to that of a cup , and draws inferences about the affordances of a pot also applying to a cup. This is a form of re-representing a pot as a cup. Similarly, the features of a cup can be re-represented as a vase, by noticing the similarity between the shape and material of vase to that of cup. This object cup , relations (Ri), actions (Hi), goals (Gi) and constraints (Cxi). Where,Concepts are everyday objects with properties such as shape, material, size etc.Relations are an association between two or more concepts.Actions are the action performed on the concepts which may or may not lead to a change in relation.Goals are a set of concepts and relations.Constraints describe the resource or action limits of the task.C1, C2−String1, String2C3−PersonC4−ChairC5−PliersC6−Jar_of_nailsC7−Sheets_of_paperC8−NailsR1, R2−hanging, hangingR3, R4, R5−onFloor(Chair), onFloor(Sheets_of_paper), onFloor(Jar_of_nails), onFloor(nails)R6−holdsGsol−{tie}Cx1−ifRx = holds, then¬Ry = reachFor the two strings problem, the concepts, relations, goal and constraints can be conceptualized as follows:Cproblem = {C1, C2, C3, C4, C5, C6, C7, C8}Rproblem = {R1, R2, R3, R4, R5, R6}Hproblem = {}Gproblem = {Gsol}Cxproblem = {Cx1}I = {Cproblem, Rproblem, Hproblem, Gproblem, Cxproblem}The re-representation here is of objects, objects sets and problem templates as different objects and problem templates. chair, agent) are thus re-represented as elements of this problem. In In I → {C1, C2, C3} → PT:{C = {C1, C2, C3}, R = {holds}, G = {tie}}I → {C2, C3} → PT(elongate_string):{C = {C2, Tshirt}, H = {attach:{C = {C2, C5}, R = {attached{C2, C5}, H = {swing(C2)}, G = {Cnew = {pendulum}}The goal of this study was to investigate if there is an ability of re-representation that underlies performance on the ambiguous figures task, the pattern meaning test, the alternative uses test and insight problems. Indeed, many of the obtained results lead in that direction. The results of Wiseman et al.  and DoheA possible connection to performance on the insight tasks was explored using performance on the candle problem as a grouping variable. This was due to several reasons. First, due to the fact that in this study only two insight problems were given, it was not possible to explore the data at an interval level. Also, the Jack and Jill weight problem turned out to be quite difficult for participants, so that only very few were able to solve it. This limited the analysis to a group-wise comparison, dividing participants by their ability to provide a correct or at least partly correct solution for the candle problem. This was not possible for the Jack and Jill weight problem, as the groups would have been too different in their size. Due to violation of the assumption of normality, it was also not possible to conduct an ANOVA, thus analysis was limited to non-parametric tests. These revealed that on average, people with higher fluency ratings would have a higher chance at answering the candle problem correctly, indicating that these tasks share common features. This notion does not seem to hold true for the ambiguous figures task, though this comparison lacks power and ought to be investigated in future studies.Generally our results support the hypothesis of an underlying ability of re-representation. The correlations between the ambiguous figures task, the pattern meanings task and the alternative uses test provide strong evidence for that notion, and insight task results, though exploratory in their nature, show some clear tendencies. What is however notable in the correlational results obtained is that they do not directly reflect the proposed level structure, as seen in The correlation of the pattern meanings task with the alternative uses task shows a higher effect size than the other two correlations. From our theory, we would expect to see the ambiguous figures task and the pattern meanings task more closely related, as both involve the re-representation of features, in contrast to the alternative uses test, where participants need to re-represent objects. One thing that could explain this is the aspect of motivation. As there was no time limit given for the pattern meanings task and the alternative uses test, motivation might have played an important role in how long participants were trying to come up with new solutions. This was not the case for the ambiguous figures task, where participants had a time limit of 1 min after which the task was over, so that motivation might not have been as important.The fact that in one task the possible representations were given, while in the other two participants had to come up with their own, might also have an influence on motivation and participant performance. Verbal fluency may also have an impact on the results, in that people who generally have a higher verbal fluency may be better at coming up with more answers in the pattern meanings task and alternative uses task. In future studies, the influence of these possible confounds should be investigated and, if necessary, controlled for.Three levels of re-representation have thus been described, in the context of four empirical tasks. These tasks can be used to explore the relation between the different levels and compare computational models and artificial cognitive systems output to human output. A similar type of computational process could thus be used to solve all these types of tasks, with differences in terms of the type of knowledge and amount of knowledge used.grouping various input features into different possible known outputs, which are already known (or in structures similar to those outputs) in order to obtain different functionality of the representation, access different affordances connected to other objects, access different possible actions connected to other problem templates or action plans, or transform the initial representation altogether, creating a new object, problem template etc.At the computational system level, this requires an integrated multi-level modeling approach: a unified framework in which all three levels are modeled, and in which only the input, output, and access to the amount of memorized knowledge brought to the process changes, but the process stays the same or is similarly implemented. This is showcased in The output of the various levels which have been described here defines the interpretation or types of representations which are projected unto the input by the cognitive system, or the secondary properties which might follow from this representation . Sometimes the output of a new representation is clear, and at other times this can only be observed because of its secondary properties.As shown in A computationally similar view on different processes will help simplify the modeling and implementation of each of these processes and tasks in computational cognitive systems. This will allow the further study of these tasks. If indeed the process is the same or similar at the three levels expressed here, then this unified synthetic approach should help understand re-representation in a much clearer manner by comparative exploration and experimentation of the specific levels, assuming initial differences in representation complexity (and in the manipulation of representations of different type and/or complexity). Association, similarity or other processes which might make re-representation possible have been tackled elsewhere .In their paper, Öllinger et al.  examine In Öllinger et al.'s paper, representational change involves at least two possibilities—chunk decomposition and constraint relaxation. Chunk decomposition is analogous to object decomposition in CreaCogs, as an object which is perceived as a whole can be broken down to its constituent parts (or further sub components), and its components can be applied to solving the problem, or regrouped in other object configurations (the same stands for problem templates). The parallel to constraint relaxation in CreaCogs is relaxing the natural and learned biases against breaking objects, using problem templates in an already familiar way, crossing common sense or common practice norms or aesthetic values. This helps ease the re-representation process.Öllinger et al. focus on the influence of various strategies, like chunk decomposition and constraint relaxation (or just standard type) on the participants further ability to solve problems requiring the same or other types of representational change. They however do not model nor describe the representational change process in terms which could serve for a computational implementation. The present paper does not make a statement regarding how representational change of one type influences other types of representational change. Instead, it focuses on finding various tasks with which the process of representational change could be explored and tested. This paper also aims for these empirical tasks to involve different types of stimuli, with different degrees of information complexity, and represent different knowledge levels of a cognitive agent and architecture. An initial re-representation mechanism which can be computationally modeled was put forward for all these information (and stimuli) levels.The ambiguity of the various stimuli is also an interesting issue. In some cases, the ambiguity may allow for easier and more creative re-representations, and perhaps computational measures of ambiguity could be proposed at all the three levels. Our initial guess is that such a measure of ambiguity would require taking into account how many representations a certain scene can be parsed and represented as, which would probably be dependent on the cognitive system doing the re-representation. This would be hard to measure in natural cognitive systems, though perhaps a set of strong dominating possibilities of re-representations could be discerned for a particular set of stimuli, through empirical investigation. However, a computational measure of ambiguity would be interesting and much more feasible to deploy in artificial cognitive systems. The generativity of a particular artificial system might be a function of this measure of ambiguity and the system's capacity for re-representation.This paper has put forward a multi-layered approach to re-representation. This approach aims to meaningfully integrate the performance in four different tasks to the study of re-representation as one cognitive process. If the differences based on knowledge between these different levels prove to influence the process itself in significant ways, re-representation can still be integrated as one class of processes. Such integration will support the unified empirical and computational study of a class of creative problem solving processes.This paper has proposed that re-representation, a key process in creative problem solving and human creativity, can and should be analyzed in a more integrative manner. Toward this, three levels of re-representation were explored. Tasks were used to exemplify and analyse the characteristics the re-representation process takes at each of these levels, in order to clarify the process itself.An empirical study across these tasks confirmed our hypothesis that relations can be established between these tasks, replicated two previous studies with an ample number of participants and uncovered a new strong and significant relation between the Pattern Meanings Test and the Alternative Uses Test.Our proposal was that the re-representation process be modeled all three levels: (i) re-representation of features as different objects or images, (ii) re-representations of objects and their properties as different objects, and (iii) re-representation of objects, objects sets, scenes and problems as different object and problem templates. An initial unified model for the re-representation process has been put forward and showcased at all these levels.Formalize each proposed level of re-representation;Attempt to model the various levels computationally under the same cognitive framework;Explore the underlying factors of empirical performance in these tasks.As future work we will aim to:This study was carried out with written informed consent from all subjects. An approval of the ethics committee was not necessary, as no risks were attached to the study. All subjects gave written informed consent in accordance with the Declaration of Helsinki.A-MO conceptualized and structured the theoretical foundation of the article. A-MO wrote sections 1–5 and the discussion in section 8, as well as the conclusion. A-MO and MS planned and deployed the empirical study. The data were analyzed by MS. MS wrote section 6. Post-revision, A-MO defined new research to be added to respond reviewers, and the unified model. AB wrote a new section 7 and added to the Discussion. MS did the Kruskal-Wallis analysis and added to the Discussion. A-MO supervised the process and revised and refined the new text.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Long working hours and a lack of sleep have been suggested to negatively affect the safety of workers. Here, we examined the association between long working hours/sleep-related problems and near-misses/injuries in industrial settings using a nationally representative sample of workers in Japan. Based on the composition ratio of workers by industry, sex, and age in Japan, data from a web-based cross-sectional survey for 18,682 participant full-time workers  were analyzed. Nearly 30% and 5% of participants reported any types of near-misses during the past six months and injuries during the past year, respectively. For all types of near-misses and some types of injuries, a significant difference in frequency distribution was observed by industry. After adjustment for demographic, job-, and life-related variables, participants who worked long hours  were more likely to report job-related near-misses/injuries than those who worked 35–40 hours per week. The presence of sleep-related problems was also significantly related to near-misses and injuries. However, while sleep-related problems were significantly associated with near-misses/injuries in all industries, the association between long working hours and near-misses/injuries differed by industry. Odds ratios for near-misses/injuries were strongly significant in the “transport/postal services” industry for those who worked more than 51 hours per week compared to those who worked 35–40 hours per week. Comprehensive protective measures for workers, including (1) reducing total hours of service/job-related fatigue, (2) maintaining sufficient sleep hours/good sleep, and (3) increasing awareness about the impact of overwork/long working hours and sleep-related problems on workers’ safety among employers, workers, clients/customers, and the general public might be effective for preventing near-misses and injuries in industrial settings among workers, especially those who work long hours in the “transport/postal services” industry. Despite substantial prevention efforts, occupational injuries are a major problem worldwide, includinPrevious epidemiological studies using a nationally representative sample of workers on the association between long working hours/sleep-related problems and injuries in industrial settings, adjusting for the effect of other factors associated with the risk of injuries , have been conducted in western countries.–10 HowevPrevious studies suggest that the type of near-misses/injuries and the impact of fatigue due to long working hours and sleep-related problems on these events differed by industry, including transport/postal services and medical/health/welfare industries., 15, 16 http://global.cross-m.co.jp/solution/online/index.html). To minimize selection bias, this survey selected a sample of Japanese workers aged 20 to 64 years based on the composition ratio of workers by industry, sex, and age in the Labour Force Survey by the Japanese Ministry of Internal Affairs and Communications.[To investigate working and living conditions and health/safety-related outcomes among Japanese workers, a web-based cross-sectional survey of 30,000 workers was conducted in collaboration with a research company in Tokyo, Japan. This company has one of the largest online research panels in Japan with over 1.8 million voluntarily registered panelists ).The outcome measures of the present study were occupational injuries and near-misses. According to previous reports on near-misses, we definHave you experienced any near-misses in industrial settings during the past six months?”). Participants who answered “yes” were subsequently asked to select all types of near-misses they have experienced from the following five categories: (1) falling, (2) crashing/crashed, (3) getting caught in/between, getting cut, (4) traffic accident, and (5) others . These categories were classified based on previous reports on near-misses.[First, participants were asked whether they had experienced any near-misses in industrial settings during the past six months, under the instruction of the definition of a near-miss . Participants who answered “yes” were subsequently asked to select all types of injuries they have experienced from the same categories as the near-misses.Second, participants were asked whether they had had any occupational injuries during the past year . The PSQITo measure depressive symptoms, we used the Center for Epidemiological Studies Depression Scale (CES-D). The CES-To assess job-related stress, we used the Brief Job Stress Questionnaire (BJSQ),, 25 whicAs potential confounders, other demographic  and job-related variables  were also measured.First, cross-tabulations were conducted between each variable and industry. To examine whether the frequency distribution of the variables, including working hours and outcomes, differed by industry, we performed chi-square tests and analysis of residuals.To examine the association between long working hours/sleep-related problems and near-misses/injuries, multinomial logistic regression analyses were carried out with the presence/absence of injuries/near-misses as the dependent variable. The dependent variable was categorized as follows: “none (reference),” “near-miss (and no injury),” and “injury.” The logistic regression coefficients were transformed to odds ratios (ORs) with 95% confidence intervals (CIs). In the multinomial logistic regression models, we adjusted for sex, age group, education, night/shift work, job-related stress, and depression. To examine whether the association between long working hours/sleep-related problems and near-misses/injuries differed by industry, logistic regression analyses were performed by industry.P < 0.05 was considered statistically significant. For chi-square analysis regarding the frequency of injuries and near-misses, a significance level of P < 0.01 was chosen to consider the number of significance tests undertaken. All analyses were conducted using SAS version 9.4 .Of the 20,052 eligible participants, 1,007 worked less than 35 hours per week during the past six months and 363 worked less than six months of service at the time of survey; they were excluded from the statistical analyses. Data from 18,682 participants  were analyzed.The characteristics of study participants by industry are shown in Regarding injuries, 5.0% of participants reported any type of injury . Except Regarding industry , in the In the present study, we examined the association between long working hours/sleep-related problems and near-misses/injuries in the workplace using a nationally representative sample of workers in Japan. For all types of near-misses and some types of injuries, significant differences in frequency distributions were observed by industry. After adjustment for demographic, job-, and life-related variables, participants who worked long hours  were more likely to report job-related near-misses/injuries than those who worked 35–40 hours per week. The presence of sleep-related problems was also significantly related to near-misses and injuries. However, whereas the ORs for near-misses/injuries were significantly higher for those with sleep-related problems regardless of industry, the association between long working hours and near-misses/injuries differed by industry.Nearly 30% and 5% of participants reported any types of near-misses during the past six months and injuries during the past year, respectively. Although the characteristics of the study sample and design differed from those of previous studies, the proportion of workers who experienced occupational injuries in the present study appeared to be greater than that reported in studies in developed western countries. This mayFor all types of near-misses and some types of injuries, a significant difference in frequency distribution was observed by industry. In particular, more than 20% of participants in the “transport/postal services” and “medical/health/welfare” industries reported “almost caused traffic accident” and “almost suffered other injuries,” respectively. Participants in the “medical/health/welfare” industry reported “almost suffered other injuries” and “got other injuries” more frequently, because this category included job-related events such as needlestick injuries, exposure to harmful substances, and musculoskeletal injuries .After adjustment for demographic, job-, and life-related variables, participants who worked more than 51 hours per week were more likely to report job-related near-misses/injuries than those who worked 35–40 hours per week. These results are consistent with the results of previous studies on occupational injuries, suggesting that long working hours have a substantial detrimental effect on the safety of workers., 9 FurthThe presence of sleep-related problems was also significantly related to near-misses/injuries. Previous systematic reviews have suggested that fatigue due to long working hours, heavy workloads, and insufficient sleep/sleepiness have detrimental effects on job performance, decision-making, and, subsequently, safety.–6 Dependkaroshi  and karojisatsu , constitute major occupational issues in Japan, particularly among workers in the “transport/postal services” industry.[While sleep-related problems were significantly associated with near-misses/injuries in all industries, the association between long working hours and near-misses/injuries differed by industry. In particular, ORs for near-misses/injuries were strongly significant in the “transport/postal services” industry for those who worked more than 51 hours per week compared to those who worked 35–40 hours per week. These findings are consistent with previous studies that reported that fatigue after long periods without sleep has a negative impact on performance and accuracy for safety., 32 Workindustry., 34 NatiPrevious studies suggest that fatigue due to long working hours and poor quality of sleep are associated with medical errors and job-related injuries among workers in “medical/health/welfare” industry. In contrRecent epidemiological studies regarding the safety and health of commercial truck drivers suggest that multiple rest-break periods during a trip can promote recovery from fatigue and reduce fatigue-related risk of crash/injuries., 37 PeriIn addition to sleep-related problems, night/shift work was associated with significantly high ORs for near-misses/injuries in almost all industries. Irregular work patterns and night/shift work against circadian rhythm, especially nighttime driving, are suggTo our knowledge, this is the first study to examine the association between long working hours/sleep-related problems and near-misses/injuries in industrial settings using a nationally representative sample of workers. However, the current study has several limitations to note. First, we relied on self-reported job-related factors, including working hours, for exposures and near-misses/injuries as the outcome measure, which may have led to recall bias. Furthermore, in this study, the term “injury” was used to include minor injuries as well as injuries resulting in sick leave or hospitalization, and the severity of injuries remained unclear. However, previous systematic reviews on the association between long working hours and occupational injuries suggest that, regardless of the severity of injuries, working longer hours increased the risk. Second, In the present study using a nationally representative sample of workers in Japan, after adjustment for sociodemographic and job-related variables, both long working hours and sleep-related problems were positively associated with safety outcomes  among workers. However, while sleep-related problems were significantly associated with near-misses/injuries regardless of industries, the association between long working hours and near-misses/injuries differed by industry. Comprehensive protective measures for workers, including (1) reducing total hours of service/job-related fatigue, (2) maintaining sufficient sleep hours/good sleep, and (3) increasing awareness about the impact of overwork/long working hours and sleep-related problems on workers’ safety among employers, workers, clients/customers, and the general public might be effective for preventing near-misses and injuries in industrial settings among workers, especially those who work long hours in the “transport/postal services” industry.S1 Appendix(DOCX)Click here for additional data file.


Background: Gender inequality and cultures of binge drinking may increase the risk of second-hand harms from alcohol. Methods: Using the 2014–2015 National Alcohol Survey and 2015 National Alcohol’s Harm to Others Survey (N = 7792), we examine associations of state-level gender equality measures  and binge drinking cultures (rates of men’s and women’s binge drinking) with individual-level indicators of second-hand harms by drinking strangers and partners/spouses. Results: In main effects models, only male binge drinking was associated with greater odds of harms from drinking strangers. There were significant interactions of gender equality with male binge drinking: High male binge drinking rates were more strongly associated with stranger-perpetrated harms in states low on contraceptive access or abortion rights compared to states high on these measures. Conversely, male binge drinking was more strongly associated with spouse/partner-perpetrated second-hand harms in states with more economic equality, compared to states lower on this measure. Conclusions: Detrimental effects of high male binge drinking rates may be modified by gender equality. Targeted interventions may reduce alcohol-related harms experienced by women in states with high rates of male binge drinking. Restrictions in access to contraception and abortion may exacerbate harms due to men’s drinking. Gender equality refers to the parity of women and men (an absence of inequity or bias) on key economic, social and political indicators . Gender Greater gender equality may be associated with increases in some health risk behaviors among women . This coSecond-hand harms due to alcohol—which also are called alcohol’s harm to others, externalities and collateral damage from drinking —range inMany studies have examined the relationship between gender equality and violence against women ; and state alcohol policies and regulatory regimes . There aBased on the extant literature showing associations between gender equality, alcohol consumption and violence, we hypothesize that harms due to someone else’s drinking would be more common in states with lower gender equality (Hypothesis 1a) and in states with high rates of binge drinking (Hypothesis 1b), especially binge drinking by men. We also examined whether these contexts had synergistic effects, anticipating that the combination of low gender equality with higher rates of men’s binge drinking would be associated with particularly elevated levels of second-hand harms from alcohol. That is, we hypothesize that the expected relationship between a culture of binge drinking by men in the state and second-hand harms due to drinking would be exacerbated by gender inequality (Hypothesis 2). Given gender differences in the social patterning of second-hand alcohol harms, we also expected the interaction of gender inequality and drinking culture to emerge for alcohol-related harms reported by women rather than for harms reported by men (Hypothesis 3). For men, we instead anticipated that higher rates of binge drinking would be associated with more harms, regardless of the level of gender equality in the state. Because we were not sure how these associations might differ by perpetrator type, we examined an overall indicator of harms, and then used separate indicators for stranger- and spouse/partner-perpetrated harms.We used pooled cross-sectional data from the 2014–2015 National Alcohol Survey (NAS) and 2015 National Alcohol’s Harm to Others Survey (NAHTOS). These surveys incorporated the same dual-frame landline and mobile telephone sampling design that also included Black/African American and Hispanic/Latino oversamples . One eligible respondent from each household was randomly selected, and computer-assisted interviews were administered in either English or Spanish. The overall cooperation rates for the NAS and NAHTOS were 45.4% and 47.3%, respectively (averaged across landline and cell phone surveys). The Institutional Review Boards of the Public Health Institute in Oakland, CA, and of the fieldwork agency  approved all survey protocols. The human subjects’ protections for the 2015 US National Alcohol Survey were approved on 6 September 2013 IRB#I13-019). The protocol was subsequently included in an overarching approval  for the National Alcohol Survey Series (IRB#I16-028). The human subjects protections for the 2015 US National Alcohol’s Harm to Others Survey were approved on 18 December 2014 (IRB#I13-018). For more details on the NAS, see Karriker-Jaffe et al. . The pro. n = 49) were not included in the analyses.For both surveys, respondent addresses were geocoded and linked with the state-level predictors described below. Due to missing data on some state-level indicators, respondents living in Washington, DC (N = 7792) was restricted to respondents who had no missing data on key dependent variables, and who had location identifiers at the state level. In the weighted sample, 52.6% of respondents were female (n = 4636), the average age was 47.1 years old (SD = 17.8) and the majority was non-Hispanic White (66.3%), followed by Hispanics (14.1%), Blacks (11.9%) and respondents of other or missing race/ethnicity (7.7%).The analytic sample (Respondents reported whether they experienced any of eight harms due to someone else’s drinking in the past year. Second-hand harms included: (a) having family problems or marriage difficulties; (b) being pushed, hit or assaulted; (c) having house, car or other property vandalized; (d) having financial trouble; (e) being in a traffic accident; (f) being harassed, bothered, called names or otherwise insulted; (g) feeling threatened or afraid; and (h) being physically harmed. These items have been used on prior US ,20,41, CGender equality measures. In the present study, indicators for state policies related to gender equality included two indicators of reproductive rights  and one indicator of economic equality between men and women.Reproductive rights. Reproductive rights/autonomy measures often incorporate indicators such as birth outcomes or life expectancy or adolescent birth rates [th rates ,11 that th rates .Contraceptive access is a dichotomous measure indicating whether the state offered or required insurance coverage of contraception and/or low-income access to family planning, versus neither [ neither . Of all Abortion rights scores from the year 2015 were derived from NARAL, a national pro-choice organization that tracks abortion and contraception policies in each state [ch state . Each ofch state . Of all Economic equality. We used an existing measure of state-level economic equality between men and women that operationalizes economic equality using median annual earnings for women employed full-time year round, the earnings ratio between women and men employed full-time year round, percent of women in the labor force and percent of all women employed in managerial or professional occupations [upations . States Drinking culture, by gender. To operationalize drinking cultures, we used 2013 data from the Behavioral Risk Factors Surveillance System [r = 0.87, p < 0.01).e System  on the pMedian income. Data on each state’s annual median household income came from the US American Community Survey, which is administered by the US Census Bureau [r = 0.33, p < 0.01) and women’s  binge drinking rates, as well as with the measures of gender equality, with the weakest association being with the indicator of contraceptive access  and stronger associations with the indicators for abortion rights  and economic equality .s Bureau . The weiDrinking status. Each respondent’s past-year drinking status was derived from the National Institute on Alcohol Abuse and Alcoholism’s (NIAAA) stated guidelines for at-risk or heavy drinking. At-risk drinking is defined as consuming more than three or four drinks on a single day, or more than seven or 14 drinks per week, for women and men, respectively. Of the weighted analytic sample, 33.3% were non-drinkers, 38.9% were drinkers who did not exceed the recommended guidelines, and 27.8% were at-risk drinkers in the past year.Demographic covariates. Models also adjusted for respondents’ gender, age, marital status , race/ethnicity , education (less than a college degree versus a 4-year college degree or more), and income in the past year .The data consisted of a two-level hierarchy with individuals nested within states, and therefore we tested associations using multilevel logistic regression. We estimated main effects and interaction models using the state-level gender equality indicators and binge drinking rates by gender using the full sample, followed by gender-stratified analyses. For significant interactions, we also report Wald chi-square tests of the difference between the main effects and interaction models. Interactions were graphed using values of continuous variables at +/− 1 standard deviation from the mean. Data were weighted to adjust for the sampling strategy and nonresponse; all analyses were conducted using Stata version 15 .Descriptive statistics overall and for the gender-stratified samples are shown in In simultaneous main effects models testing the first hypothesis (1a), none of the gender equality measures were significantly associated with the second-hand harms due to alcohol overall, nor for men or women separately. These models also provide tests of the second hypothesis (1b): Adjusting for measures of gender equality and other state-level variables, the state rate of binge drinking by men was associated with increased odds of harms perpetrated by strangers overall (OR (95% CI) = 1.11 ), although associations were not statistically significant in the gender-stratified models (OR (95% CI) = 1.12  for women; 1.09  for men). In the full sample, the association of male binge drinking rates with stranger-perpetrated harms persisted once individual-level covariates were added to the simultaneous model (OR (95% CI) = 1.12 ). The male binge drinking rate was not significantly associated with the indicator of any past-year second-hand harm from alcohol, and it also was not significantly associated with spouse/partner-perpetrated harms. State rates of binge drinking by women and the state-level median household income were not significantly associated with second-hand harms from alcohol in any of the models.As expected (Hypothesis 2), there were some statistically significant interactions between indicators of gender equality and state male binge drinking rates. The findings for any past-year harm and for stranger-perpetrated harms suggested that the association between male binge drinking and second-hand harms was exacerbated by gender inequality.p < 0.001). The marginal predicted probabilities showed a significant association of states’ male binge drinking rates with past-year harm due to someone else’s drinking only in states with restricted contraceptive access (the association was not significant in states with good contraceptive access): Predicted probabilities ranged from 0.161  at the lowest rates of binge drinking by men to 0.251  at the highest rates of binge drinking by men in states with restricted contraceptive access and from 0.178  at the lowest rates of binge drinking by men to 0.195  at the highest rates of binge drinking by men in states with good contraceptive access. The interaction is depicted in As shown in p < 0.001), with predicted probabilities of experiencing one or more harm due to someone else’s drinking ranging from 0.156  at the lowest rates of binge drinking by men to 0.249  at the highest rates of binge drinking by men in states with restricted contraceptive access and from 0.190  at the lowest rates of binge drinking by men to 0.200  at the highest rates of binge drinking by men in states with good contraceptive access. The interaction was not statistically significant for men, so it was dropped from the model. In the main effects model for men  = 327.57, As shown in p < 0.001; The interaction pattern was consistent when using the indicator of abortion rights. As shown in p < 0.001; A different pattern of results emerged for spouse/partner-perpetrated harms. Only economic equality evidenced a significant interaction with the male binge-drinking rate  = 166.53, p < 0.001; The marginal predicted probabilities suggested that states’ male binge drinking rates were not significantly associated with past-year harm due to a partner’s drinking in either group of states, however: Predicted probabilities ranged from 0.029  at the lowest rates of binge drinking by men to 0.016  at the highest rates of binge drinking by men in states without economic equality and from 0.033  at the lowest rates of binge drinking by men to 0.047  at the highest rates of binge drinking by men in states with economic equality. The stratified models revealed that this pattern was unique to harms reported by women  = 79.34, In this US study, with the exception of an association between male binge drinking and harms perpetrated by drinking strangers, we did not see any strong independent associations of either gender equality (Hypothesis 1a) or state-level drinking culture (Hypothesis 1b) with second-hand harms from alcohol. Our findings on Hypothesis 1a thus did not replicate prior studies showing an association between gender equality and reduced violence ,30,31, sWhen we examined alcohol-related harms perpetrated by intimate partners (spouses or boyfriends/ girlfriends), a different interaction pattern emerged. There were somewhat elevated rates of spouse/partner harm in states with economic equality but high levels of male binge drinking, but somewhat lower rates of spouse/partner harm in states without economic equality but high rates of male binge drinking. It may be that there is a backlash against women in these higher-equality, heavy-drinking states . There aWe provide new evidence under Hypothesis 1b that a state-level drinking culture characterized by greater binge drinking by men is associated with second-hand harms from drinking strangers. Our findings are similar to prior research, showing that high levels of binge drinking on college campuses were related to reports of second-hand harms experienced by residents of nearby neighborhoods . HoweverTo our knowledge, this is one of the first US studies on the topic of state-level gender equality, drinking cultures and alcohol’s harm to others. Although we had the benefit of a large, nationally-representative sample of adult respondents, our data were cross-sectional, and causality cannot be implied. Although it is possible that reductions in the second-hand effects of alcohol could precede changes in gender equality, it is less likely that changes in these harms would reduce binge drinking by men or prompt legislators to enact laws and policies protecting women’s reproductive rights. Thus, it seems reasonable to assume the outcomes studied here may have been at least partially a result of the policy context and drinking culture of the respondents’ state of residence. Stronger causal models would enhance the evidence base on this topic.Another study limitation is that there may be other important effect modifiers that we did not take into account, such as a respondent’s level of education  or work Further, the indicators of gender equality are not independent, and isolation of their effects may be impossible. For example, four states scored low on all three measures of gender equality, and fourteen states scored low on two of the three measures. Future work may benefit from the utilization of summary scores that better describe the status of women on multiple dimensions. Finally, additional work is needed to disentangle the effects of gender equality, drinking cultures and attitudes toward violence (and particularly violence to women) on alcohol’s second-hand harms, as these may be related in complex ways . Future Findings from a large, population-representative study of US adults suggest detrimental effects of high rates of male binge drinking may be exacerbated by gender inequality, with this effect being particularly harmful for women. As reproductive rights decrease in U.S. states , policy


Millions of users share images and videos generated by mobile devices with different profiles on social media platforms. When publishing illegal content, they prefer to use anonymous profiles. Multimedia Forensics allows us to determine whether videos or images have been captured with the same device, and thus, possibly, by the same person. Currently, the most promising technology to achieve this task exploits unique traces left by the camera sensor into the visual content. However, image and video source identification are still treated separately from one another. This approach is limited and anachronistic, if we consider that most of the visual media are today acquired using smartphones that capture both images and videos. In this paper we overcome this limitation by exploring a new approach that synergistically exploits images and videos to study the device from which they both come. Indeed, we prove it is possible to identify the source of a digital video by exploiting a reference sensor pattern noise generated from still images taken by the same device. The proposed method provides performance comparable with or even better than the state-of-the-art, where a reference pattern is estimated from video frames. Finally, we show that this strategy is effective even in the case of in-camera digitally stabilized videos, where a non-stabilized reference is not available, thus solving the limitations of the current state-of-the-art. We also show how this approach allows us to link social media profiles containing images and videos captured by the same sensor. Digital videos (DVs) are steadily becoming the preferred means for people to share information in an immediate and convincing way. Recent statistics show a 75% increase in the number of DVs posted on Facebook in one year  and postIn such a scenario, it is not surprising that DVs gained importance also from the forensic and intelligence point of view: videos have recently been used to spread terror over the web, and many critical events have been filmed and shared by thousands of web users. In such cases, investigating the digital history of DVs is of paramount importance in order to recover relevant information, such as acquisition time and place, authenticity, or information about the source device. Recently, Multimedia Forensics tools have been developed for such tasks, based on the observation that each processing step leaves a distinctive trace on the digital content, as a sort of digital fingerprint. By detecting the presence, the absence or the incongruence of such traces it is possible to blindly investigate the digital history of the content .In particular, the source identification problem—that is, univocally linking the digital content to the device that captured it—has received great attention in the last years. Currently, the most promising technology to achieve this task exploits the detection of the Sensor Pattern Noise (SPN) left by the acquisition device . This foIn contrast, research on source device identification for DVs is not as advanced. This is probably due to the higher computational and storage requirements for video analysis, the use of different video coding standards, and the absence of sizeable datasets available to the community for testing. Indeed, DV source identification borrowed both the mathematical background and the methodology used for still images ,9: in a However, if we consider that ribed in , where aThe first contribution of this work is a hybrid source identification approach that exploits still images for estimating a fingerprint that can be used to verify the source of a video. To this aim, the geometrical relation between image and video acquisition processes are studied for 18 modern smartphones, including devices featuring in-camera digital stabilization. As a second contribution we prove that the proposed technique, while preserving the state of the art performance for non-stabilized videos, is able to effectively detect the source of in-camera digitally stabilized videos as well. Furthermore, we show that this hybrid approach allows linking images and videos downloaded from different social media platforms, namely Facebook and YouTube.The rest of the paper is organized as follows: Everywhere in this paper, vectors and matrices are indicated in bold as The task of blind source device identification has gathered great attention in the multimedia forensics community. Several approaches were proposed to characterize the capturing device by analysing traces like sensor dust , defectification , allowinfication . Recentlfication  addressefication  to addrefication , Chen etommended  for the ommended , yieldinommended  are outdommended , consideommended  firstly ommended  considerp stands for progressive). Recently, Taspinar et al. [Most of the research on video forensics neglects the analysis of digitally stabilized videos, where the SPN can hardly be registered. In  an algorIt should be noted that all the mentioned works discuss source identification either for still images or videos (with the only exception of ), and inDVs are commonly captured at a much lower resolution than images: top-level portable devices reach 4K video resolution at most , while the same devices easily capture 20 Megapixels images. During video recording, a central crop is carried out to adapt the sensor size to the desired aspect ratio (commonly 16:9 for videos), then the resulting pixels are scaled to match exactly the desired resolution see . As a diThe Hybrid reference-based Source Identification (HSI) process consists of identifying the source of a DV based on a reference derived from still images. The strategy involves two main steps: (i) the reference fingerprint is derived from still images acquired by the source device; (ii) the query fingerprint is estimated from the investigated video and then compared with the reference to verify the possible match.N still images (or frames) Moreover, the camera fingerprint g filter ,25 is apstimator :(1)K˜=∑iThe fingerprint of the video query is estimated in the same way from the available video frames. Let Note that we are assuming that the video fingerprint is at a smaller resolution than the image fingerprint since it represents the default camera settings in most mobile devices nowadays. The model still holds in the opposite scenario. If we denote such a transformed fingerprint with  problem :(3)H0:KIThen, the test statistic is built as proposed in , where tCE ratio  as:(4)PCL plausible scaling factors k is the number of tested scaling and cropping parameters. This method is expected to be computationally expensive, especially for large dimension images. This problem can be mitigated considering that:the resize and cropping factors are determined by the device model . It is thus possible to build a lookup table that eliminates the need for an exhaustive search when information about the reference model is available;even when no information about the model is available, it is not necessary to repeat the whole search on all frames. Once a sufficiently high correlation is found for a given scale, the search can be restricted around it.In order to consider the possible different scaling factors of the two fingerprints—since videos are usually resized—a brute force search can be conducted considering the tors. In  it is shRecent devices feature digital stabilization as a means to reduce the impact of shaky hands on captured videos. By estimating the impact of user movements, the software adaptively determines which portion of the sensor should be used to obtain a stable image. Image stabilization can usually be turned on and off by the user on devices based on Android, while in iOS devices this option cannot be modified in the native camera application. Source identification of videos captured with active digital stabilization cannot be accomplished using the classical approach of PRNU fingerprint computation, since it would require the fingerprint to remain spatially aligned across all frames. This condition is not met due to the stabilization process . HSI solGiven a query video and a set of images belonging to a reference device, the proposed pipeline is summarized in First, the device fingerprint Equation . Then, sEquation .The proposed technique can be applied to match multimedia contents exchanged through different SMPs. Let us consider a user, “Bob”, that publishes videos with criminal content to a SMP using an anonymous profile. At the same time, Bob is leading his virtual social life on another social network, where he publicly shares his everyday pictures. Unaware of the traces left by the sensor, he captures with the same device the contents shared on both profiles. In such a scenario, the fingerprints derived from the images and videos on the two social platforms may be compared with the proposed method to link Bob to the criminal videos.Analysing multimedia contents shared on SMPs is not a trivial task. Indeed, besides stripping all metadata, SMPs usually re-encode images and videos. For example, Facebook policy is to down-scale and re-compress images so to obtain a target bit-per-pixel value ; YouTubeF: during acquisition, the image F by factors Interestingly, the hybrid approach can be applied to this scenario. In Modern devices allow the user to adjust the zooming factor before or even during video acquisition. This is commonly implemented by using a sub-portion of the device sensor, possibly followed by some form of scaling to match the desired output resolution. Therefore, if a video is captured using digital zoom, an additional geometric transformation step must be considered in the HSI process. Following the same reasoning of On the reference side: 100 flat-field images depicting skies or walls; 150 images of indoor and outdoor scenes; 1 video of the sky captured with slow camera movement, longer than 10 s;still, move and panrot videos respectively. Thus, each device has at least 9 videos, each one lasting more than 60 s.On the query side: videos of flat surfaces, indoor scenes and outdoor scenes. For each of the video categories  at least 3 different videos have been captured considering the three different scenarios available in the Dataset: (i) still camera, (ii) walking operator and (iii) panning and rotating camera. We will refer to them as We tested the proposed technique on a subpart of the VISION dataset , consistWe determine the cropping and scaling parameters applied by each device model in the considered set;We verify that, in the case of non-stabilized video, the performance of the hybrid approach is comparable with the source identification based on a video reference;We show the effectiveness in identifying the source of in-camera digitally stabilized videos;We show the performance in linking Facebook and YouTube profiles;We demonstrate the effectiveness of the method in the presence of digital zoom.The experimental section consists of five parts, each focused on a different contribution of the proposed technique:P statistic  and the scaling factor, maximizing the y axis to match the video fingerprint; C9 is a pretty unique case in which the full frame is applied for video and is left and right cropped by 160 pixels to capture images.The scaling and cropping factors applied by each device were derived by registering the reference video fingerprint Equation ). For eaIn the case of stabilized videos, the cropping and scaling factors change in time, with possible rotation applied too. For these devices we thus determined the registration parameters of the first 10 frames of the available video reference; the main statistics are reported in The information provided in In this section we compare the proposed technique with the state of the art approach , where tEquation ). We refThe plot shows that distributions can be perfectly separated when the reference is estimated from images (100% accuracy), while in the video reference case the accuracy is 99.5%, confirming that performance are comparable.State of the art results in identifying the source of a stabilized video are provided in . The autWe will now show that this performance drop can be solved by exploiting the proposed HSI method.For each device, the reference fingerprints Results clearly show that, using ieved in , but in https://lesc.dinfo.unifi.it/it/node/184). The runtime depends on several factors, such as the reference and the query resolution, the availability of accurate matching parameters  took between ters see  and the ters see  allowed In this section we test the HSI approach in the application scenario of linking Facebook and YouTube accounts containing images and videos captured with the same device. For clarity, we considered the non-stabilized and stabilized cases separately. Furthermore, we conducted two experiments: one estimating the SPN from images uploaded to Facebook using the high-quality option, and another experiment estimating the SPN from images uploaded using the low quality option. A detailed explanation of the differences between the two options is given in , here weWhen Facebook images uploaded in low quality are used as reference, the estimated pattern is expected to be less reliable than for the high quality case. This degradation on the reference side can be mitigated by using more robust estimates on the query side; for this reason, for the low quality case, we also considered using 500, 800 and 1000 frames for extracting the query pattern, achieving ROC curves reported in We now focus on the case of in-camera stabilized videos downloaded from YouTube. As a final experiment, we tested the validity of the proposed approach in a sample case where a video is obtained using digital zoom. First, we concentrated on the case of a static digital zoom followed by uploading and downloading from YouTube. We selected device C17 and captured a flat video at default resolution (1080p), setting a slight amount of digital zoom before initiating the recording ; the obtained video was then uploaded to YouTube and downloaded at two resolutions, namely 1080p and 720p. Then, we applied the HSI framework to match these videos to the device fingerprint obtained from 100 flat images, and used the first 150 frames of the video for query fingerprint computation. We also tested the case of a flat video captured while adjusting the zoom factor during the acquisition : this situation is more challenging, because each frame of the video has to be matched separately. Even though, the hybrid approach successfully matched most of the examined frames : In this paper we proposed an hybrid approach to video source identification using a reference fingerprint derived from still images. We showed that, in the case of non-stabilized videos, the hybrid approach yields performance comparable with or even better than the current state-of-the-art strategy, which uses a video to compute the reference pattern. As a major contribution, our approach allows reliable source identification even for videos produced by devices that enforce digital in-camera stabilization , for which a non-stabilized reference is not available. We reported the geometrical relationships between image and video acquisition process of 18 different devices, even in case of digitally stabilized videos. The proposed method was applied to link image and video contents belonging to different social media platforms: its effectiveness has been proved to link Facebook images to YouTube videos, with promising results even in the case of digitally stabilized videos. Specifically, when low quality Facebook images are involved, we showed that some hundreds of video frames are required to effectively link the two sensor pattern noises. We finally showed that presence of digital zoom can be handled with our approach. The main limitation of the proposed approach is the need for a brute force search for determining scale  when no information on the tested device is available. A possible way to mitigate this problem would be to design SPN descriptors that are simultaneously invariant to crop and scaling. This challenging task is left for future work.