Gilles Hermann for his scientist intuition that made him a source of ideas but also his support and crucial contribution in various ways.

I gratefully thank our European partners and especially the Biozentrum staff that participated in this work by providing biological samples and assistance with the microscope. Special thanks go in particular to Hervé Remigy for his enthusiasm, dynamism and valuable guidance regarding biological and microscope issues. The results and continuity of this work would not have been possible without the great help and coordination of Pr. Andreas Engel and Urs Müller that were abundantly helpful during all this time. I am heartily thankful to Nicolas Coudray for his advice and help. His precious presence and involvement with original ideas were so much valuable to me during these last three years.

I offer my regards to all of those who supported me in any respect during the completion of the project from the TROP group. Among Master and PhD students that were present in the TROP group, I would like to thank Paméla Daum for her efforts and time to work out segmentation evaluations.

Lastly, I would like to thank Patrice Wira for his great encouragement during these last three years. I cannot find adequate words to express my gratitude to him.

Deepest gratitude is also due to the members of the examiners committee, Pr. Frederic Truchetet and Pr.

Stephen McKenna who accepted to be members of the reading committee. I would like to thank them for the precious time they offered to read this thesis and their suggestions to improve the manuscript quality.

Table of Contents

Introduction

The use of digital image processing techniques for the development of tools dedicated to image analysis has gained a growing interest in various domains during the last few years. The need to automatically manipulate and process a large volume of acquired data has rendered to image analysis tools a great importance pushing towards new image processing technologies.

Research in the image processing domain is related to many other domains and applications. Such processing is now commonplace in a number of diverse fields such as medicine, biological research, cancer research, drug testing, metallurgy, etc. This is especially the case with biology, leading to a strong connection where the advances in one domain are narrowly related to the advances in the other domains: for instance, on one hand, recent automation of electron microscopes led the production of many new images that need to be processed automatically, and the diversity of the specimens observed constitute new types of images to be processed opening new problems and perspectives for the image processing domain (in term of contrast, and heterogeneity of the specimens, for instance). On the other hand, the elaboration of automatic algorithms to process these images and to assist the biologists in assessing their experiments aims to accelerate and facilitate the research in the biological domain.

In this thesis, starting from the partition of an image, we developed a chain of algorithms required to obtain, in a fully automatic manner, a description of biological specimens: those algorithms include the validation of the partition used, distinction of foreground objects, followed by a semantic labelling of the regions, and, finally, the exploitation of this data through a classification procedure.

This work has been motivated by the High Throughput 3-Dimensional Electron Microscopy (HT3DEM) specific targeted research project, funded within the 6 th framework Programme of the European Commission over 4 years, from October 2005. The aim is to develop an automated platform to enable highthroughput screening and analysis of specimens issued from 2D-crystallization experiments. The 2Dcrystals are biological membranes reconstituted experimentally, in which the membrane proteins are inserted in an organized manner. This configuration is appropriate to the study of proteins and the elaboration of their 3-dimensional structure. The 2D-crystallization process is however complex as it depends on many parameters (types of proteins, pH, temperature, detergent used, etc.), which can only be experimentally established. To accelerate the research in the domain, the European project managed the development of an automated platform that deals with all the steps, from the automatic production of the protein membranes, to the analysis of the experiment using a Transmission Electron Microscope (TEM).

The research of the MIPS laboratory in image processing contributed to the selection of targets for automatic image acquisition, and to the characterization of the 2D-crystallization experimental results.

This thesis started in October 2006, one year after the start of the HT3DEM project, within the MIPS laboratory, supervised by J.-P. Urban and collaborating with J.-L. Buessler and G. Hermann. The objective was to contribute to the analysis of membrane images acquired at medium magnification (around x3,000).

Those images are particularly difficult to process, as they are very noisy, the gray-levels are heterogeneous, and the contrast of the objects (the membranes) to analyze is low and highly inhomogeneous. Furthermore, the membrane size and shape vary considerably. Because of these specificities, most of the usual techniques, as they will be presented throughout the manuscript, are unsuitable to process such images.

Several studies have been run in parallel in the MIPS laboratory to achieve the ambitious goal of automatic sample screening and automatic specimen characterization. From October 2006 to November 2008, a PhD thesis has been realized by Nicolas Coudray [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF] to elaborate the scenario of the microscope screening, and to develop the primary algorithms allowing region targeting. For medium magnification images, a multi-resolution based algorithm has been developed to split an image into membrane regions.

This work partition was our base for membrane characterization. A Master student also participated in the advancement of the research: Paméla Daum [START_REF] Daum | Evaluation de segmentations d'images : mesures de similarité avec une référence partielle[END_REF]) elaborated a new tool to quantitatively evaluate the performance of the segmentation and labelling algorithms at medium magnification.

Our research proposes the following contributions:

-A validation process based on a new criterion for region merging has been developed. Existing criteria to merge segmented regions to overcome the over-segmentation problem often rely on interregion properties (e.g. contrast). These criteria are however not applicable in our images due to their particular nature. We therefore decided to look for a strategy based on a local validation of each transition. The original principle of the method is a novel criterion based on the gradient continuity along a contour. To get the best sensitivity, we use a validation technique based on statistical hypothesis testing. The results demonstrated on a variety of TEM images the efficiency of the proposed method.

-The second algorithm presented in this report is a labelling algorithm. It aims to attribute a semantic meaning to each region obtained after the image has been partitioned, and after the partition has been validated by our algorithm. The proposed process is decomposed into two steps. The first step aims to achieve a low level semantic labelling to distinguish the foreground from the background.

The background identification is used as a reference for the second semantic labelling, which aims to identify the number of membranes that are superposed within each image object.

-A primary classification procedure is also proposed to extract characteristics helping the validation of 2D-crystallization experiments: ideally, biologists expect the experiments to lead to large sheetlike non-stacked membranes, but, in practice, many experiments are needed to achieve such high quality crystals, giving in-between samples of various and different qualities.

The present manuscript is organised in four chapters:

The first chapter places the study in the general context of membrane proteins structure analysis by biologists. Biological notions are explained, the 2D crystallization process is described, and the strategy for the automatic control of the microscope is detailed. The HT3DEM European project is also presented providing an overall aspect of the global pipeline and emphasizing our contribution: the development of software tools for the analysis of TEM images.

The second chapter includes TEM image characteristics that are widely discussed in order to understand their nature and features. It then presents the first step of the automatic analysis that corresponds to the image segmentation implemented by Coudray. This segmentation step constitutes the staring point of the study presented in this manuscript and for this purpose we integrate a detailed presentation of this technique.

A discussion concerning the results obtained within the 2D crystallization context is provided as it initiates the contribution of this thesis, as the segmentation step leads to an over-segmentation that we need to reduce.

In the third chapter a literature overview shows that almost all techniques for over-segmentation reduction and region fusion are established on region comparisons. Similar regions are merged according to intraregion criteria such as region contrast. In our images, as explained in chapter 2, object gray-levels present fluctuations that do not allow such simple comparisons. On the other hand, the gradient continuity along a contour provides a reliable criterion used in a statistical hypothesis testing.

The fourth chapter presents an approach to recognize image objects from the segmented image as given in the previous chapter. Foreground-background distinction is therefore necessary and elaborated into two steps: the principal background region extraction and then the secondary ones. The original method introduced uses the segmented image, the mean gray-level of each segmented region, and a priori knowledge.

An extended conclusion of our overall method is provided with perspectives that are detailed in the appendix.

The appendix corresponds to preliminary results of the classification process. The foreground objects as extracted before must be characterized precisely in this last step. Their classification is carried out by applying decision rules that have been elaborated by simulating an expert's reasoning. Each foreground object is analyzed and characteristics are extracted and used as input data for their classification providing at the same time complementary statistical information destined to fill in a database.

Chapter 1

General Context of the Study

Introduction

The first chapter places the study in the general context of protein structure analysis by biologists. The three dimensional structure of proteins is generally determined after crystallization of the purified molecule by being inserted into the cellular membrane. These proteins, even if they are very important in pharmaceutical applications, are difficult to study because they do not form crystals easily. 2D crystallization is an alternative technique that uses the chemical affinity of these proteins with membranes. The insertion of a large quantity of purified proteins into lipid membranes produces a periodic structure, which can be characterized by electron crystallography. The resulting data is then analyzed to determine the 3D structure of the protein.

The aim of the European project HT3DEM is to develop an automatic platform to analyze and study crystallization conditions. All steps are automated and ensure the simultaneous preparation of 96 samples, each corresponding to specific crystallization conditions. The evaluation of the crystallization, at present manually assessed by an expert biologist using the electron microscope, must also be automated in order to analyze a large number of samples.

The MIPS laboratory is in charge of software development tools for the analysis of TEM images. The extracted information contributes to the strategy of the microscope control; it is used to determine the zones of interest that trigger acquisitions at different magnifications.

Biological Context

Before going further into our analysis we provide certain definitions of the biological specimen to be studied: membranes and membrane proteins.

Biological membranes are vital components of all living systems, forming the boundaries of cells. They consist of a lipid bilayer and embedded proteins (Figure 1.1). A lipid bilayer is a double sheet structure of lipids that have both hydrophilic and hydrophobic parts. They are arranged with their hydrophilic parts pointing out to the water that surrounds the bilayer and hydrophobic ones pointing into the core of the bilayer. This bilayer prevents molecules such as proteins and ions from entering into cellular areas. Proteins that are associated with a lipid membrane are called membrane proteins.

Generally, a protein is defined as a complex organic macromolecule composed of several amino acid sequences, joined by peptide bonds; its chemical composition relies basically on carbon, hydrogen, nitrogen and oxygen compounds. A protein can be classified into three categories: globular proteins, fibrous proteins and membrane proteins. of the total number of cellular proteins and 60% of drug targets; until today it remains a class of proteins that we know very little about since they are hard to study in comparison with globular proteins that are water-soluble [START_REF] Stahlberg | Two-dimensional crystals: a powerful approach to assess structure, function and dynamics of membrane proteins[END_REF][START_REF] Engel | Membrane protein structure and function[END_REF].

To analyse the proteins, it is necessary to extract them from the membrane and to solubilize them. Their amphipatic (hydrophilic and hydrophobic) nature prevents them from being directly dissolved in an aqueous (water) environment. To achieve this goal, detergents, another class of amphipatic molecules will be used.

Detergents will protect the hydrophobic part of the membrane protein that was embedded in the membrane of the cell (the core of the lipid layer being hydrophobic) with its hydrophobic tail, and present its hydrophilic head to the water environment. Denaturation of the protein will therefore be prevented. After purification of this solubilized protein, natural or synthetic lipids can be added and detergent removed.

Lipids being amphipatic modules as well, a bilayer similar to the cell membrane containing the membrane protein will form. This process is called reconstitution since the membrane protein will be back in its initial native state.

Protein Analysis Methods

For three-dimensional membrane protein structure analysis, the major approaches are: 3D crystallization with X-ray diffraction, Nuclear Magnetic Resonance (NMR), single particle analysis by electron microscopy, and 2D crystallization using electron diffraction by electron microscopy.

3D Crystallization

3D crystallization is a classic and powerful technique for soluble protein analysis. It has allowed the structure analysis of many protein families. It consists of growing 3D protein crystals by protein precipitation. These crystals are then assessed by means of X-ray diffraction analysis [START_REF] Deleu | Aperçu des techniques d'analyse conformationnelle des macromolécules biologiques. Biotechnologie[END_REF].

Membrane proteins however cannot be easily analyzed with this method due to their instability to form good crystals under the specific 3D crystallization conditions because of the presence of detergent [START_REF] Stahlberg | Two-dimensional crystals: a powerful approach to assess structure, function and dynamics of membrane proteins[END_REF][START_REF] Engel | Membrane protein structure and function[END_REF], Pusey et al. 2005).

Nuclear Magnetic Resonance

NMR is a method well suited for the analysis of soluble protein; recently some progress has been made but is limited to membrane protein of small size, or only fragments of membrane proteins.

Single Particle Analysis

Single particle analysis is a method for determining the structure of large proteins and biological macromolecules (e.g. viruses) by electron microscopy without crystallizing them. This method works with both soluble and membrane proteins. It enables membrane protein analysis which has been problematic in structural studies by X-ray crystallography (3D crystallization). This procedure implicates four steps: collection of image data, selection of particles, alignment of particles and classification. During data collection, large amounts of images of proteins are acquired where particles are manually or automatically selected. Acquired images contain many views of the protein with several orientations. After a particle alignment procedure the proteins are classified, according to their orientation. Then clustered raw images are added to enhance signal to noise ratio (class averages). This method does not require a particular organization of proteins or a high number of sample trials. On the other hand, the main drawbacks for the protein 3D reconstruction are:

x The 3D structure is reconstructed by estimating the projection angle of each cluster (e.g. Penczek et al.1992).

x The 3D model resolution is weak (6-15Å) compared to the one achieved by the 2D crystallization as we will see below (Zhang et al. 2001, Stewart and[START_REF] Stewart | Noise bias in the refinement of structures derived from single particles[END_REF]. Nevertheless recent progress in automatic electron micrographs acquisition and increasing computational power enable the calculation of high resolution 3D maps from millions of particles.

2D Crystallization

2D crystallization refers to the association of proteins with a membrane by the reconstitution of membrane proteins into a lipid bi-layer. Structural data of proteins can be obtained by analyzing 2D crystals by electron crystallography. Their 3D structure is obtained from the diffraction pattern (Henderson et al. 1986) that is calculated from different tilt angles [START_REF] Stahlberg | Two-dimensional crystals: a powerful approach to assess structure, function and dynamics of membrane proteins[END_REF]; the amplitude and phase of the spots are used for the determination of the 3D structure of the crystal. A 3D map having a 5 Å resolution can yield information on the orientation of "alpha helices" within the protein (alpha helices are one of the major shapes adopted by amino acids chains) and beyond 2 to 3 Å atomic structure can be seen using a computational approach like molecular replacement within the electron density.

2D crystallization requires lower protein concentrations compared to 3D crystallization [START_REF] Hasler | 2d crystallization of membrane proteins: rationales and examples[END_REF][START_REF] Walz | Electron crystallography of two-dimensional crystals of membrane proteins[END_REF][START_REF] Henderson | Realizing the potential of electron cryo-microscopy[END_REF][START_REF] Fernández | Image processing and three dimensional reconstruction in electron microscopy of biological specimens[END_REF][START_REF] Hovmöller | Solving crystal structures at atomic resolution from hrtem and refining using electron diffraction data[END_REF]. Protein crystallization is only the first part of the 2D crystallization process; 3D reconstruction of the protein structure is the second one. We focused on the 2D crystallization problem; finding the right conditions to produce wellordered crystals is one of the major difficulties encountered in 2D crystallography. More details on the 2D crystallization and major parameters that are involved within this process are described below.

2D Crystallization method

Biologists have tested various 2D crystallization methods that will be presented briefly. A discussion regarding the crystallization operation will follow. Specimen preparation (grid preparation, negative staining, etc.) and electron microscopy assessment for condition determination will be explained.

2D Crystallization Procedure and Parameters

The conditions used during this reconstitution (lipids nature, salts nature and concentration, acidity of the medium) may promote the packing of the membrane proteins into regular arrays. dialysis [START_REF] Jap | 2d crystallization: from art to science[END_REF], absorption to polystyrene beads called "biobeads" [START_REF] Rigaud | An efficient strategy for two-dimensional crystallization of membrane proteins[END_REF]), controlled dilution (Remigy et al. 2003) or by using methyl-cyclodextrin [START_REF] Signorell | Controlled 2d crystallization of membrane proteins using methyl-[beta]-cyclodextrin[END_REF]. Each detergent has its own critical micellar concentration (the concentration above which detergents are not single and separate molecules, but they start to assemble in structures known as micelles). The most successful method until now is dialysis. Nevertheless this method is slow when the detergent used during reconstitution has a low Critical Micelle Concentration (CMC): small micelles in solution occur at a low detergent concentration.

Only single molecules of detergent can pass through the membrane. Hence it takes a long time to remove this class of detergents. This is unfortunate since they are known to be the most efficient detergents to keep sensitive membrane proteins in their native state preventing deadly aggregation. The biobead method is insensitive to the detergent nature, but it is so efficient that it is difficult to regulate reconstitution kinetic.

The dilution approach is also independent from the detergent nature, but dropping protein concentration is frequently lethal to successful crystallization. The cyclodextrin method where a crown soluble sugar molecule having a hydrophobic interior is added to the crystallization solution is an approach similar to the biobeads experiment, but kinetic can be controlled using microfluidic devices. Hence, detergents can be neutralized efficiently without affecting protein concentration.

2D Samples Used in the Current Application

Samples, used within the context of this thesis, were provided from the Biozentrum Maurice E. Muller Institute for Structural Biology. 2D crystallization trials were produced by using several types of proteins (e.g. Ompf, SoPIP2) and the cyclodextrin method as a main technique for detergent removal.

Determination of the Crystallization Parameters

The analysis of the sample after the crystallization step can have two purposes:

-the study of the three-dimensional structure of the protein based on crystallography techniques;

-or the evaluation of the 2D crystallization process.

The sample preparation techniques and the imaging techniques differ radically according to the purpose of the analyses. The study of the structure is briefly presented in sub-section 1.6. The evaluation of the crystallization process is developed more thoroughly, since it is in this context that the automatic analysis has been established.

The crystallization protocol is composed of a large number of parameters that have been mentioned

previously. The study of a new protein starts with the determination of the optimal parameters that will lead to 2D crystals. Multiple combinations must be evaluated by examining the obtained crystals. This evaluation is performed by examining the sample with the TEM after a specimen preparation that is described hereafter.

Grid Preparation

For EM screening, a specimen preparation is essential. Once 2D crystallization trials are completed a 4 µl solution drop is initially placed on a copper grid, covered by a carbon film. Such EM grids are shown in Figure 1.2; presently the grid we use is 3 mm in diameter, a thin foil mesh with square spacing patterns (200 or 400 mesh), just like the grid example on the right hand side of Figure 1.2. It is conductive and stable under an electron beam. The carbon film is the support for the protein macromolecule to be analyzed by EM. It is electron lucent and stable in the beam even at higher magnifications [START_REF] Bradley | The preparation of specimen support films[END_REF]. On the other hand, films are fragile and therefore can be broken or can be dissolved by chemicals (e.g. detergent).

Specimen analysis of such damaged grid squares becomes therefore impossible.

Freshly prepared carbon substrate films tend to be hydrophobic and the hydrophilic specimen will not bind (soluble protein, single particle surrounded by detergent or 2D crystals). To overcome this problem, a technique widely used and known as glow discharge treatment (made with air) transforms the carbon film into a hydrophilic surface, negatively charged, permitting easy dispersion of aqueous elements. prevent electron beam radiation damage, experiments using biological material can also be carried out at lower temperature (cryo-microscopy); this method is however not suited for high throughput. The EM electrons interacting with the stain molecules surrounding the protein will give a contrasted negative image of the sample (negative staining). Negative staining highlights surface features by penetrating into the small surface details of the membranes. In this way, its deposits darken interstices (lipids) between proteins, as shown in Figure 1.3, delineating in this way membranes' structural details. However, a marginal effect of staining may be noticed locally, surrounding membranes, where stain may persist because of the surface tension.

Globally, the negative staining protocol involves the following steps:

-adsorption of the specimen to a glow-discharged carbon-coated EM grid;

-washing with several drops of water;

-staining with one or two drops of heavy metal solution.

However, this basic protocol can be adapted depending on the specifications and needs of the biological application. Biologists use more water droplets to remove completely cyclodextrin that may interfere with negative staining, or when detergents are used (single particle analysis of detergent solubilized membrane protein). Additionally, if we wish to obtain thinner stain layers, "overflow" stain solution can be removed from the grid by aspiration (e.g. with a paper). This is called "blotting". Stain may be deposited randomly over the grid; it can affect the background appearance and even artifacts of different size may be created. Moreover, maximal resolution is between 10 and 20 Å because of the heavy metal micro-crystals' size. This resolution is sufficient for the assessment of a crystal at high magnification but is not enough for the 3D reconstruction of the membrane protein [START_REF] Ohi | Negative staining and image classification: powerful tools in modern electron microscopy[END_REF].

Finally, during staining, proteins may be fully covered, obscuring the structural detail of membranes. Such a crystal cannot provide diffraction peaks at high magnification since its "periodic texture" is hidden by stain and therefore its presence cannot be verified.

Above observations show that having a rigorous staining procedure is mandatory. It may facilitate the observation of membranes by increasing the local contrast between membranes and background, but produces artifacts or contrast unrelated to membranes or even "grain" effects. Such effects degrade the quality of the grid and therefore complicate further tasks such as processing of EM images.

Assessment of 2D Crystals

Electron Microscopy

The objective of the next step is to evaluate the crystallization results. Once the grid is prepared, it is transferred to the TEM for crystal assessment. Checking for the presence and quality of crystals is the most time-consuming step. In a sample treated manually, the microscopist can search for the presence of crystals by means of spectral analysis at high magnification, after a prior scan of the grid to spot the most interesting zones, potentially crystalline.

Properties often used in sample characterization are derived from examination of electron microscope images. In 2D crystallization, this step is even more delicate as the diversity of sample objects in electron microscope images is considerably wide. Reproducing the repetitive tasks that humans accomplish during detection and recognition of membranes turns out to be extremely complex. While interpreting images experts often use analysis criteria that cannot be easily introduced for computer vision. It is important therefore to understand and interpret the images in order to elaborate the appropriate image processing techniques.

Characterization of 2D Crystallization Trials in EM

Experts observe and interpret the content of a grid qualitatively. Biologists consider that a systematic exploration of the entire grid is not necessary since the presence of only a certain number of crystals is sufficient to validate the crystallization protocol. First, the expert scans the grid at a medium magnification (x1000 to x5000) to identify the membrane regions that are more likely to diffract; in this way he creates a global idea of the sample quality and estimates visually membrane features. At this magnification certain conclusions can be empirically extracted (degree of protein reconstitution, etc.). A few regions, the more interesting ones, are then selected to be investigated at high magnification (x50000, approximately 0.5 nm/pixel), where the diffraction peaks confirm the existence of crystals.

Membranes in TEM Images

TEM observation allows determining various indicators of the quality of the crystallization. Crystallization does not always produce fine crystals. For example, when the proteins fail to be inserted within the membranes, proteins solubilized in detergents are isolated and expanded throughout the grid (single particles). Also, when detergent is removed too rapidly, proteins tend to aggregate by attaching their respective hydrophobic parts as hydrophilic ones are in contact with water. These ones appear as very dark regions in TEM images and they can be noticed at medium magnifications (x5000) as illustrated in Figure 1.4. Finally, if crystallization conditions are optimal, membranes are formed correctly. The diffraction pattern obtained at high magnification will be analyzed in order to verify whether proteins were properly inserted into the lipid membrane. Membrane size and geometry, observed at medium magnification, contribute also to the characterization of a good crystallization. vesicles (characterized by a circular shape, usually appear as "squeezed" circular objects), tubes (cylinder shapes) and sheets (characterized by rectilinear contours). In these TEM images, we can distinguish:

-The background, which corresponds to the carbon film and has the brightest gray-level in the image.

-Membranes may be identified within regions whose intensity is slightly darker than the background; such image regions interest us as the most as they are not stacked and are defined in this manuscript as mono-membrane regions. This type of region presents low contrast, corresponding to a single membrane thickness with no superposition or folding. They may appear with different shapes and sizes (as explained in the above paragraph); these regions are those that are likely to correspond to crystals.

-Membranes that are superposed have a higher electron-scattering power than non-stacked membranes. These regions, however, do not interest biologists as they do not allow to test the diffraction [START_REF] Mosser | Two-dimensional crystallogenesis of transmembrane proteins[END_REF]. Stacked membranes are quite dark regions in the image; the more they are stacked, the darker they appear (Figure 1.4).

Finally, one has to note the presence of different artifacts, the most common ones being artifacts of stain or remaining detergent, or holes in the carbon due to over-exposure to the electron beam.

Cryo-Electron Microscopy and 3D Reconstruction

The crystallization trials are assessed by negative staining electron microscopy to determine the optimal crystallization conditions; protein structure analysis may be then studied. Once crystals have been obtained

under optimal conditions, the proteins' 3D structure can be determined by cryo-electron microscopy while the sample is cooled to the temperature of liquid nitrogen and the electron dose is kept very low. Minimal dose mode is available in microscopes at high magnification and a micrograph of the unexposed crystal can be captured with the lowest possible electron dose.

The goal of an electron crystallographic study is a three-dimensional map of the protein. However, low electron dose corresponds to high noise level and a single cryo electron-micrograph of a protein cannot provide precise 3D information. Therefore, different views of the crystal are collected by tilting the sample in the microscope. Because of radiation damage, every crystal can only be imaged at one tilt angle, so a large number of crystals is needed. The periodic information of the crystal can be separated from random noise by Fourier filtering: a Fourier transform of a 2D crystal consists of a 2D lattice of spots. Globally, the amplitudes and phases of the FFT are extracted and kept for further processing. The amplitude and phase data of all images is merged into a three-dimensional data set. After merging the data from all films, the 3D data set can be processed by standard crystallographic methods and a 3D density map of the protein is the final result [START_REF] Vonck | Two-dimensional crystallisation and electron crystallography of membrane proteins, Atelier 147, La Grande Motte Biophysique des protéines membranaires[END_REF].

Objectives of the European Project HT3DEM

Motivations for the Automation of a Protein Analysis Pipeline

Successful crystallization involves many different parameters (protein-lipid ratio, pH, temperature, solubility, etc.). The mechanisms that yield crystals are not yet completely understood; their complexity and the number of parameters to take into account are difficult to handle. Therefore, identifying the conditions for producing well-ordered crystals represents a significant obstacle in 2D crystallography. However, these conditions are determined and established empirically within a large number of crystallization trials; each trial sample must be placed on an EM grid, stained and inserted into an electron microscope in a holder that typically holds just one grid at a time. It is obvious that the production and manual examination of numerous crystallization trials to establish conditions is time consuming and demands a lot of effort. For such a number of trials an automatic and quick production and analysis scheme is therefore necessary. There is therefore a need, underlined by various authors (Ellis and Herbert 2001[START_REF] Schmidt-Krey | Electron crystallography of membrane proteins: Two-dimensional crystallization and screening by electron microscopy[END_REF][START_REF] Cheng | Towards automated screening of two-dimensional crystals[END_REF] to implement an automatic and rapid analysis of 2D crystallization trials for the study of membrane proteins.

Motivations for Automatic Image Analysis

Electron microscopic examination of specimens by an expert depends on the microscopist's experience, attention and criteria. Moreover, a manual assessment of a large number of crystallization "condition" trials demands a lot of effort and time; this difficulty inspires the great interest to introduce automatic procedures that may reduce screening time and improve robustness by automatic processing of electron microscope images.

HT3DEM Project

The European project HT3DEM (High Throughput 3-Dimensional Electron Microscopy, http://www.ht3dem.org) is dedicated to the automatic sample preparation and evaluation of 2D x A crystallization robot is developed to prepare 2D crystals by cyclodextrin-driven detergent removal (label 2, Figure 1 .6). This robot operates on a 96-wellplate (label 1) and can automatically manage the crystallization process and all its parameters. Such a process may take several days.

x A second robot controls the sample grid preparation and negative staining for EM analysis (label 3).

96 copper grids are thereby prepared, each containing the results of a crystallization experiment.

x The 96 grids are then introduced in 8 grid holder cassettes, of 12 grids each (label 4), and the cassettes are loaded in the cassette revolver (label 5). The revolver, attached to the microscope (label 6), is controlled to introduce iteratively in the column each grid of each cassette.

x A software package controls the microscope and the automatic image acquisitions (label 7). This software handles acquisition parameters and therefore controls the image quality, the speed of the system process, etc.

x Image analysis software that processes the acquired images (label 8) aims at providing information concerning the sample but also piloting the microscope to optimize grid membrane region selection.

x Finally, a database (label 9) manages the data produced in the above pipeline (conditions of crystallization, staining procedure and image processing outputs).

The main objective of HT3DEM consists in developing a prototype platform to assess 2D crystallization experiments automatically by EM and prove its feasibility. The automatic preparation of the biological specimen and its image analysis are essential in order to achieve a high throughput pipeline; an initial version of each part has already been developed and a prototype of this pipeline has been installed at the Biozentrum (Basel) where intensive tests are in progress.

Thesis Motivations

Our contribution within the HT3DEM project is related to the image processing algorithms elaborated for the automatic evaluation of samples by EM. More specifically, image analysis techniques are necessary to elaborate an exploration strategy of the sample with the TEM and estimate the quality of the crystallization experiments. A microscope control strategy was therefore adopted [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF]) consisting of such imaging techniques; it needs to extract on-line information to drive the microscope towards the membrane regions to be analyzed at high magnification. These image processing techniques are gathered in a toolbox referred to as Animated TEM (ANalysis of IMages for Automatic Targeting and Extraction of Data in Transmission Electron Microscopy) toolbox for the microscope control strategy.

Image analysis must also extract information that characterizes the specimen and that is intended for the biologist. This thesis thus concerns:

x The automatic identification of mono-membrane regions in images taken at medium magnification and their characterization in terms of size, quantity and density on the grid.

x Specimen characterization by classifying (as presented in section 3.3) the sample elements regarding their size and type of membranes.

Original image processing techniques for detection and recognition are designed and applied on TEM images, which remains a much unexplored field. The need to develop systems for automatic image acquisitions embraces a wide variety of applications. The automatic processing of these images represents therefore a important issue. Several automation projects have been developed recently in the field of electron microscopy, but the interpretation of the images has been left to a human operator. TEM image analysis however represents a whole new research area in the domain of image processing; original image processing techniques for detection and recognition are designed and applied on TEM images.

Towards Automatic Acquisition for Microscope Control -State of the Art

The interpretation of the images is often essential to select the subsequent acquisitions. Examples of automation of this stage are still very scarce and limited to very specific applications.

In microscopy, many algorithms were developed for image acquisition in recent regarding various applications. In cryo microscopy (Carragher et al. 2000[START_REF] Zhang | Automated data collection with a tecnai 12 electron microscope: applications for molecular imaging by cryo microscopy[END_REF], or in electron tomography (Koster et al. 1992), (Suloway et al. 2009) or in electron 2D crystallography (Oostergetel et al. 1998), completely automatic acquisition remains a complex problem to explore. Acquisition systems may also be found in light microscopes that are proven to be very efficient and rapid by acquiring for example 3000

images per hour (Paran et al. 2007). Bonton et al. (2002) developed an automatic system of image acquisition for accurate measurement of pollen concentration for allergy prevention purposes. These grains are automatically detected with light microscope acquired images and then acquisitions at different defocus are digitized in 3D; this 3D representation of pollen structure is automatically recognized.

Presently, the main motivation behind a system that acquires automatically images for sample assessment is to facilitate and accelerate biologists' tasks in membrane protein analysis. The automatic process consists of two parts: the control of the microscope (microscope control and image acquisition) and the acquired image analysis. For each of these two parts we can distinguish different approaches and degree of autonomy (manual, semi-automatic, automatic). The microscope control manages acquisition parameters (defocus, illumination etc) and orders either a predefined scanning or a successive scanning guided by automatic target selection. This target selection can therefore be provided by image analysis which may also offer information to the biologists about the specimen. Techniques that combine image analysis techniques for automatic image acquisition of 2D crystals are rare. To understand the approach employed, we will discuss a manual analysis of a sample grid by a microscopist; then present semi-automatic techniques implemented in the literature.

Manual Analysis

Experts observe and interpret the content of a grid considering that a systematic exploration of the grid is not necessary since the presence of only a certain number of crystals is sufficient to validate the crystallization protocol. The manual analysis of a grid by an expert may take 10-20 minutes, depending on the quality of the grid. Biologists consider that finding a minimum of crystals present on the grid is sufficient to validate the crystallization protocol. As already presented in this chapter, a first scanning of the grid at a medium magnification (x1000 to x5000) is performed to identify the membrane regions of interest;

this observation helps the expert to estimate the quality of the sample and membrane size and type. The more interesting regions are then selected to be investigated at high magnification (x50000, approximately 0.5 nm/pixel), where the diffraction peaks confirm the existence of crystals.

Semi-Automatic Analysis

GRACE Software

Oostergetel et al. (1998) developed software for EM image acquisition of 2D crystals. A semi-automatic control of the microscope enables the user to localize the regions of interest on the specimen at a medium magnification (x2000) in the search mode of the microscope. Their coordinates are stored and subsequently imaged at high magnification (x50000 -x100000) on the slow scan CCD camera, taken at low electron dose. The Fourier Transform (FT) of this preview image is then calculated yielding diffraction peaks, signs of the crystalline quality of the area.

The authors underline that, although they believe that "a full automation is possible in principle, in practice,

given the vast amount of structural information that is present in a good specimen, it will be difficult to effectively replace the intelligent input from an experienced operator completely by automatic procedures".

Leginon Software for 2D Crystal Analysis

Concurrently, [START_REF] Potter | Leginon : a system for fully automated acquisition of 1000 micrographs a day[END_REF] adapted an already existing version of the Leginon software package, developed for single particle analysis. The acquisition procedure, adapted to membranes, starts by scanning the grid at low magnification (x660) where images are acquired. Potential regions of interest (isolated or overlapped catalase crystals) are identified automatically with the use of image processing techniques (cross-correlation with a template, thresholding and Gaussian models) that provide their location (center point of regions as target selection) for acquisition at higher magnification. The quality of crystals is then assessed automatically by calculating the power spectrum, identifying the diffraction peaks and counting their number.

However, this strategy was developed for specimens of negatively stained catalase that form nice, large and rectangular crystals. The technique is therefore adapted to the specific problem and not to a larger variety of membranes for a more global study; image processing techniques need to be robust for different membrane properties such as shape, intensity, etc.

From all these above applications we may see that the idea of even semi-automatic acquisitions guided by image processing techniques turns out to be very attractive, when such an approach is possible. However, the complexity of image processing algorithms seeking a fully automatic process is another issue. Replacing the expert's experienced eye by a computer system is still a huge challenge. Nevertheless, many efforts have been placed in the domain of image processing to interpret images and obtain semantic representations. The criteria used during this evaluation are often chosen by the expert who has knowledge regarding these images.

Presently, the strategy introduced intends to propose an automatic solution based on such criteria. The strategy developed is split into three magnification phases; automatic image acquisitions are therefore guided by image analysis techniques and implemented at three magnification phases (low, medium and high magnification) and will be described in the following sections.

TEM Automation for Specimen Analysis -General Strategy

The success of the crystallization is determined by the peaks in the diffraction pattern at high magnification (around 5 Å/pixel). At this magnification, the field of vision is equal to 500nm and so the screening of a 3 mm diameter sample grid is unrealistic. Thousands of images should be acquired only for a grid square (60 µm x 60 µm) during such an investigation that turns out to be time-energy consuming and inefficient when the specimen is rare. Therefore, the grid has to be screened at lower magnifications to direct the microscope to the regions of interest where proteins are likely to be crystallized. The existing semiautomatic scenarios (e.g. Oostergetel et al. 1998[START_REF] Cheng | Towards automated screening of two-dimensional crystals[END_REF]) assume user interaction, a parameter that extends the time for the analysis of a grid reducing in this way the efficiency of the whole system. They are therefore not compatible with the objectives of our project.

The strategy adopted by the project for the microscope control emulates the decisions of a microscopist during a manual analysis, as described above (paragraph 1.8); it therefore implicates the implementation of image analysis techniques that interact with the acquisition system and are gathered in a toolbox referred to as Animated TEM toolbox. It consists of three gradual magnification phases where each is related to two tasks: an "acquisition task" that manages the collection of image acquisitions series and an "imaging task" that realizes the processing of this collection of images. Image processing algorithms are therefore associated to:

x Low magnification (x50), to provide information concerning the quality of the grid map. At this magnification we visualize the whole grid and our study is restricted only to the grid squares avoiding the copper grid bars. This is the lowest magnification where the microscope can operate, as it allows for image acquisition of the whole grid rapidly. This magnification is sufficient to detect squares where the carbon film is absent or broken and to eliminate them before any ambiguities at higher magnifications are induced.

x Medium magnification (x3000, or 10 nm/pixel approximately for an image of 1024 x 1024 pixels), to determine successively the Regions Of Interest (ROI) within the square view. Squares that were selected at low magnification are now zoomed at medium magnification to detect and characterize membranes. In this strategy, the detection of the mono-membrane regions is pursued as a major objective.

x High magnification, the crystallization quality of the ROI is evaluated by means of spectral analysis. Crystalline structure is verified by the presence of diffraction peaks, a typical pattern related to a crystal.

Figure 1.7 shows a table listing the properties and tasks to be accomplished at each magnification. Starting at low magnification an overall view of the grid is provided and its quality is assessed. Only the grid squares that are not damaged are to be explored in detail. Images are then acquired at medium magnification in order to localize the regions that are potentially crystalline and provide statistical information (size, type of membranes) concerning the sample that will be saved to the database. Finally the regions previously selected are zoomed at high magnification to verify the presence of crystals. This process is automatic, rapid, efficient and sufficiently accurate to replace the microcopist; evidence that supports the previous claim will be given in the following chapters. The next paragraphs will present the image processing algorithms elaborated in the Animated TEM toolbox.

Phase

Analysis of Low Magnification Acquisitions

Grids with completely undamaged carbon film are very rare and an automatic analysis of their quality is essential to avoid losing time analyzing areas where the film is damaged. A process for grid assessment was developed by [START_REF] Zhang | Automated data collection with a tecnai 12 electron microscope: applications for molecular imaging by cryo microscopy[END_REF] in cryo-microscopy that relies on a user defined reference. The challenge in this phase has been to develop an algorithm adapted for every grid in electron crystallography that is completely independent from any user input.

To achieve this goal, Coudray developed an original method mainly based on histogram analysis. It consists of a grid identification step, a first on-line assessment step, and a second optional assessment step based on information gathered during the first two steps.

As a preliminary step, a global histogram analysis is done to identify the grid squares: the relative opacity of the copper to electrons creates a sharp peak at the beginning of the histogram. The threshold is set at the first local minima after this peak.

Then, the analysis of the carbon film at each square relies on the following observations. Four classes of squares can be identified as illustrated in Figure 1.8: (A) the undamaged squares where membranes are big and contrasted enough to be visible; (B) the undamaged squares where the membranes are too small to be visible; (C) the squares where the film is broken; (D) the squares where the film is completely missing.

The first quality assessment is based on local histogram segmentation (Figure 1.8): for each square, the last peak of the histogram is identified and thresholded. This peak represents the background and so squares of type (A) and (C) can easily be discriminated; for squares of type (A), the surface covered by bright pixels has 'holes' due to the darker, large and visible membranes and so the area enclosed in the convex envelop of the background is approximately equal to the size of the square. For squares of type (C), the area covered by the brightest pixels does not cover the whole square. Squares of types (B) and (D) are more difficult to be identified as the only visible difference between them is their mean gray-level. A second quality assessment is therefore defined based on the information gathered during the first quality assessment: statistical information about the mean gray-level of the background for squares of type (A) and (C) has been already gathered during the first assessment and thus the mean gray-level of the background pixels. This information is used in the second quality assessment step to identify, by the gray-level analysis, the squares of type (B) and (D). Such an assessment is shown in Figure 1.9.

Medium Magnification Analysis

The processing of images at medium magnification addresses firstly the problem of selecting membrane regions as targets for high magnification acquisition. Two steps are therefore developed: a pre-selection of TEM images based on histogram analysis and regions of interest selection within selected images for analysis at high magnification.

Histogram-Based Pre-Classification of Images

The image pre-classification Coudray proposed is a fast way to reject images that are not worth being If more than 50% of the image is covered by the mesh grid we consider that the square surface to study is not large enough and therefore is not further studied.

Selection of ROIs

Only the interesting images (with an expected histogram) are then analyzed to identify the membranes and the regions to be analyzed at high magnification. The outputs of this step are the coordinates of the selected ROIs to be visualized and analyzed at high magnification.

A first prototype uses a fast selection of a few interesting contours. The size of the ROIs is chosen to be at least as large as the highly magnified images to avoid redundant selections. These squares are placed around the edges detected with the Prewitt filter and a selection is done among them. The selection assumes the presence of an object (sufficient amount of edges in the ROI squares), a limited amount of overlapping ROIs to avoid redundant acquisition at high magnification, and a rejection of the stacked membranes by checking the mean gray-level.

The more complete characterization of the grid, which is the subject of this thesis, introduces the possibility to select directly the potentially crystalline mono-membrane regions according to criteria established in collaboration with the biologists.

High Magnification Analysis

The images acquired at a high magnification (about 5 Å/pixel) are potentially crystalline areas identified at medium magnification. The automatic analysis of the power spectra of the images aims to detect the diffraction peaks that will attest the presence of crystals.

Specimen Characterization

The automatic TEM control strategy and the image analysis provide information acquired at different magnifications and different positions on the grid. This information is used to characterize the specimen globally.

Biologists are mainly interested in protein analysis and more particular in:

x Information concerning 2D crystal membranes (size, quantity, etc.) to validate the success of the 2D crystallization.

x Information concerning the specimen sample itself must be provided (size, type, etc.) for its analysis and classification.

Conclusion and contribution of the thesis

This important information can be extracted from images acquired at medium magnification. For this purpose and in this thesis context, our efforts are mainly focused on studying medium magnification images.

The image processing techniques implemented for such images are adapted to the general scheme of the microscope control, to the nature of the TEM images and our motivation for specimen element characterization. The image analysis algorithms must therefore allow specimen localization and recognition to provide statistical information such as object counting and categorization according to their size and type.

Moreover, information regarding mono-membrane regions has to be extracted as the existence of these regions validates and characterizes the crystallization process. Their detection and recognition is therefore a task of great importance in our application.

To achieve such complex tasks, we need to elaborate and employ a sophisticated strategy implicating complex image processing tasks. A processing chain is elaborated for this magnification constituting the contribution of this thesis. At first, a segmentation step is necessary to partition the image into regions and isolate the interesting mono-membrane zones. This step is quite complex since an ideal method does not exist and so it is adapted each time to the image specifications. The difficulty in our case lies on the TEM images nature; interesting regions have a low contrast in a particularly noisy context. The best compromise came with a multi resolution segmentation method [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF]) dedicated to the segmentation of TEM images. This method will be described in detail in Chapter 2 where aspects and problems of the method will analyzed. To resolve them a strategy is proposed and applied to facilitate recognition tasks. The processing chain assembles a number of methods such as region fusion for more meaningful image region segmentation, mono-membrane region recognition and characterization, and classification; all these aspects will be presented further (chapters 3, 4) in the thesis.

The biological context of this study was presented in this chapter. The specimen preparation until its insertion into the microscope was presented in detail. At the same time, the project motivations were presented to better understand the goal of our study and TEM images at different magnifications were illustrated. The three magnifications strategy for the automation of the microscope was then explained, presenting the imaging techniques associated with each magnification. Finally, motivations for the current study were briefly discussed; important objectives of this work were mentioned, underlining the need for specimen and membrane characterization.

Chapter 2 Identification and Characterization of Membranes

Introduction

The observation of the sample at medium magnification provides the information that is essential to characterize the success of the 2D crystallization. The resulting objects, and especially the artificial membranes, are identifiable at this scale. Our image analysis techniques perform their localization, recognition and supply a certain number of statistics such as their number, size, type, etc.

The discussion is focused principally on the localization of membrane zones, because of their importance for the crystallization characterization, and also because they are difficult to identify in TEM images. These zones correspond to a unique thin membrane layer and will be called mono-membrane regions. In such regions the membranes are expanded on the grid support as flat layers, presenting no folding or superposition.

Mono-membrane regions are distinguished from the background only by a slight contrast. The continuity of the contours indicates the presence of the object in the image since the surface itself does not present any significant features. The image analysis strategy elaborated during this thesis study relies on a preliminary detection of contours and then on a structure based on local successive comparisons of neighboring regions to build the object map of the image.

The objectives and motivations for membrane identification and characterization are described in this chapter. Characteristics of TEM images will be widely discussed to understand their nature and particular features. The first step of the automatic analysis that corresponds to the image segmentation will then be presented. This segmentation step has already been implemented in a recent thesis of the MIPS laboratory [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF]) and constitutes the starting point of the study presented in this manuscript: the elaboration of an algorithmic strategy to label image regions and identify the objects. We integrate a detailed presentation of the segmentation technique in order to provide a complete overview of the processing chain of TEM images at medium magnification. A discussion concerning the results obtained in the 2D crystallization context will introduce the contribution of this thesis.

Objectives

The microscope control requires on-line information at medium magnification for mono-membrane region detection so that the microscope zooms towards them for further analysis at high magnification. Moreover, the analysis at this magnification allows us to characterize the specimen elements that appear in the biological sample.

Concerning the localization of the regions of interest at medium magnification for the control of the microscope, a method has already been implemented [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF] based on a simple and quick contour detection without providing a precise localization of mono-membrane regions.

This chapter presents the first step of the automatic analysis that corresponds to the image segmentation.

This segmentation step has already been implemented in a recent thesis of the MIPS laboratory [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF]. The segmentation technique extracts the contours of the image at different scales; at each scale a gradient analysis is performed in association with an adaptive threshold algorithm. Multi-resolution gradient information is fused in order to form an image called an RGL (Reconstructed Gradient Like) image. The RGL image looks like a common gradient image; the gray-level is important in contour zones rather than anywhere else in the image. The difference is that each gray-level here represents the scale at which each contour pixel was thresholded. This type of gradient image is well suited to the watershed transformation, which is applied in order to obtain a proper partition of the image into regions.

A detailed presentation of the segmentation technique is integrated in this chapter in order to provide a complete overview of the processing chain of TEM images at medium magnification. A discussion concerning the results obtained within the 2D crystallization context initiates the contribution of this thesis, focused on the steps following the image segmentation. These steps constitute the core of the whole processing chain at this magnification since the results obtained are important for the validation of the crystallization process and the collection of sample information for the biologists.

TEM Image Characteristics

In this section we will introduce a detailed description of the nature of our images and membrane features.

Membranes at medium magnification have particular properties that are presented now to understand the decisions made concerning their detection and identification.

Contrast

TEM images are typically 12-bit or 16-bit gray-level images (depending on the encoding) acquired with 1K (1024x1024 pixels) to 4K CCD cameras. They appear within a wide gray-level range presenting a rich contrast (Figure 2.1).

It is known that the contrast of TEM images [START_REF] Hayat | Principles and Techniques of Electron Microscopy: Biological Applications[END_REF] comes from selective electron scattering. Atoms of the biological membranes interact with electrons by absorbing and scattering them. Image contrast is therefore related to both specimen and electrons. More specifically, the image contrast depends on:

x The atomic density of the specimen; contrast increases as specimen density becomes greater.

x The beam electron quantity; collecting more electrons at the output of the microscope increases image contrast (and resolution).

Therefore, the easiest way to increase contrast [START_REF] Bozzola | Electron Microscopy[END_REF] for a given specimen, depends on the electron scattering. It is possible to achieve it by several adjustments on the TEM. The easiest way is to increase the time of exposure of the camera to the electron beam. Simply put, we allow the beam to hit the specimen and the camera for a longer period of time. Membranes, however, are sensitive to exposure to electrons and a CCD detector can be saturated with an exaggerated electron dose; therefore a compromise is found for proper electron dose utilization. Another important acquisition parameter is the focus. Defocusing improves the image quality as it increases the phase contrast but at the same time an important defocus may rotate the image and cause image distortions. Our standard acquisition conditions were 1 s of exposure time with a defocus of 50-100 µm.

Noise

We will present a short overview of the image noise nature and introduce the hypotheses we took into account during the implementation of our algorithms. For this purpose, a quantification of the noise and the image contrast will be introduced: the noise standard deviation ı n , the signal to noise ratio SNR, and the contrast to noise ratio CNR. The estimation of these three parameters is important to characterize the noise in TEM images.

Computation of Noise Standard Deviation

Standard deviation is a measure that quantifies noise present in images. Many different methods are proposed and compared in the literature for noise evaluation [START_REF] Olsen | Estimation of noise in images: an evaluation[END_REF]. In this manuscript, for the standard deviation estimation we will use a simple method that consists in computing the image standard deviation in blocks of 7x7 pixels; the noise standard deviation estimate then corresponds to the mode of the standard deviation histogram of the image [START_REF] Olsen | Estimation of noise in images: an evaluation[END_REF], [START_REF] Mastin | Adaptive filters for digital noise smoothing: an evaluation[END_REF]. This method is a good compromise for our images and it is used from now on for the noise amplitude estimation.

Mono-membrane regions have a low local contrast (difference of membrane-background average gray level) in a highly noisy context (standard deviation of the region), fact that makes their identification a difficult task. Figure 2.2 shows a typical TEM image with a variety of membrane contrasts, and, for monomembrane regions pointed at by the green arrows, for example, the local contrast is equal to 127, while the standard deviation estimate is equal to 190 (arrow 2). These values constitute typical mono-membrane signal and noise amplitudes in our TEM images.

Signal to Noise Ratio SNR

To compare the relative amounts of signal and noise in the image, we define the signal to noise ratio SNR. V the noise variance.

The method of [START_REF] Sijbers | Quantification and improvement of the signal-to-noise ratio in a magnetic resonance image acquisition procedure[END_REF] was used to evaluate the above signal to noise ratio. Two acquired images are used assuming a perfect geometrical registration of these acquisitions, and an additive independent noise. The signal to noise ratio is finally quantified using the correlation function of two independent image acquisitions.

The contrast of the observed objects is however too heterogeneous. The SNR is a mean measure and cannot be used as a significant indicator. It is therefore easy to understand that for images with many dark regions against a bright background the average SNR stays high; the stronger the signal variations for a given noise the better the average SNRs.

Contrast to Noise Ratio CNR

In order to fix an idea of the poor contrast of membranes and the particular noisy context, we define the contrast to noise ratio CNR AB [START_REF] Sijbers | Quantification and improvement of the signal-to-noise ratio in a magnetic resonance image acquisition procedure[END_REF] as the ratio mathematically expressed as below in equation 2.2:

A B AB n GL GL CNR V 2.2 i
GL represents the average gray-level of region i. CNR AB is a local measure that provides a more reliable interpretation of the region contrast strength. On Figure 2.2, for example, the CNR equals 1.35 between region 1 and the background, and 0.66 between region 2 and the background.

Multiplicative Noise in TEM Images

In electron microscopy, the formation of the image is associated with an important noise. The observation of this noise suggests a strong correlation with the intensity of the signal. It was therefore necessary to evaluate if the use of an additive noise model was sufficient.

To our knowledge no study of this has yet been published in the field of electron microscopy. We based our study on the approach presented by [START_REF] Withagen | CCD characterization for a range of color cameras[END_REF]. Images acquired by CCD cameras are generally corrupted by additive and multiplicative noise. Multiplicative noise is signal dependent and is difficult to be removed without impairing image details. Let's take an image as example where multiplicative noise is present. Bright areas pixels will have multiplied intensity values; on the other hand random variation of dark areas does not change significantly. This observation can be seen in Figure 2.3 especially in the dark areas. [START_REF] Lee | Noise modeling and estimation of remotely-sensed images[END_REF] introduced a method to measure the standard deviation of this kind of noise. An image I decomposed of a signal g corrupted by an additive and multiplicative noise a and m respectively corresponds to a model as follows (equation 2.3):

( , ) ( , ) ( , ) ( , ) I x y g x y m x y a x y

2.3

In a small block of the image (here we used 7x7 pixels) that we consider homogeneous the mean value of signal g is equal to the mean value of the image I. These hypotheses lead to a mathematical expression for the computation of the image variance:

Additive Noise in TEM Images

Even if the noise present in the image contains a minor multiplicative component, we will consider from now an additive model of the noise. This approach is sufficient as such, since our study concerns monomembrane regions. Therefore, the noise present in the image may be supposed additive, Gaussian, and independent from the signal. We assume an image model where image I is composed of a signal g and an additive Gaussian noise n as given in equation 2.5.

( , ) ( , ) ( , ) I x y g x y n x y 2.5

Membrane Regions

Images at this magnification globally appear within a wide gray-level range (Figure 2.4), containing objects of different contrasts. Mono-membrane zones to be detected are low-contrasted and often non-homogeneous regions. An image object, at this point, corresponds to specimen elements as illustrated in the below figure.

In this sense and as already defined in chapter 1, the main objects found in our images are:

x Mono-membrane regions may be found isolated or being part of a more global object in the image, appearing slightly darker than the background. For this type of regions, membranes are not aggregated and are spread over the grid support as flat transparent and flexible layers (Figure 2.7).

x Because of their flexibility they can be found folded or superposed on the grid. Dark objects or dark sub-regions therefore may represent either multiple superpositions of membranes, protein aggregates or stain artifacts.

x The background is the non-homogenous bright zone that presents gray-level fluctuations.

The properties of TEM images provide only a few features to identify membranes. During preliminary studies we evaluated numerous techniques for image recognition and analysis. We summarize our observations here by presenting their main characteristics.

At medium resolution, a membrane region presents no identifiable texture. The periodic structure of crystalline membranes can only be detected at higher magnification. Membrane size and membrane quantity vary considerably in TEM images. Membrane size may begin from a few nanometers and even reach several micrometers. A convenient resolution, which corresponds to the magnification at which our images are acquired, lies between 5 and 20 nm/pixel. In this way, small membranes appear correctly allowing even bigger ones to be satisfactorily visualized. Additionally, their quantity in the specified field of view varies importantly; we may visualize samples that are densely "membrane-populated" (e.g. numerous isolated membranes or piles of aggregated ones) and others that are sparsely populated. Membranes do not exhibit characteristic shapes that would ease their recognition. Most membranes display any sort of shape. Some membranes form objects of specific shapes like tubes or vesicles. Other membranes show very rectilinear portions of contours. Shape analysis of contours, presented as a preliminary result in the appendix, helps classifying membranes in families of objects. However, it cannot be used in the prior step of localizing the membrane regions in the image.

Membrane Borders

The difficulty of implementing gradient detection techniques to determine the contour of membrane regions led us to a detailed analysis of membrane object borders. A systematic collection of profiles was used to extract transitions of different objects; these showed large diversity of profiles on the membrane borders. The transition appears rarely as a step; it is generally smoothed within 5 to 7 pixels and sometimes marked by a zone of pronounced contrast (like a crack, as it will be defined in the next section). Such a profile collection is presented in Figure 2.8 of a mono-membrane region (ı n = 55), where the identified contour pixel is situated in the middle of the extracted profile (11 th pixel). This collection of profiles illustrates well the large diversity of membrane contrasts for a given image and validates their poor contrast (CNR = 0.73).

Background

TEM images are usually corrupted by considerable background fluctuations. The amplitude of these fluctuations exceeds widely the typical contrast of a mono-membrane region. This effect is visible in image background zones that present darker gray-levels than some membranes regions.

Globally, the images are often corrupted by spurious intensity variations. These imperfections are due to various reasons: the image formation process (e.g. illumination heterogeneity), negative staining artifacts, carbon film support. These observations are taken into account in further processing steps later on, as will be described in chapter 4. Figure 2.9 is a typical example where the beam intensity inhomogeneity induces a visible shading effect: the bottom left corner of the image is clearly brighter than the upper right corner.

Region Segmentation

Gray-level image segmentation is concerned with the delimitation of the objects of interest in a digital image. We suggest a segmentation technique based on the use of the watershed transformation to partition the image into homogeneous regions.

The multi-resolution method, implemented by [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF], will be briefly presented in this section; we will firstly refer to the motivations that contributed to the conception of the segmentation strategy and then present the strategy: a contour analysis through several scales that leads to a segmentation of the image into regions. It is mainly based on a noise-and scale-adaptive thresholding method, the creation of an improved gradient image, and the watershed transformation.

Motivations

Edge-based segmentation seems appropriate for our images where the contrast of the membranes is locally enhanced by the staining. However, contour analysis raises other problems, such as how to assure the continuity of contours or process an image with variable amplitude or type of contours. The importance of the noise makes smoothing a necessary task for low contrast edge detection. On the other hand, the size of the filter kernel is a delicate decision. Multi-resolution approaches seem to consider this difficulty by applying to the image a number of different size filters. Together, these ideas confirm that a contour approach is more appropriate than a region based approach. Common edge detection techniques are not sufficient enough to detect different contrasts present in the image. A more detailed description of the method implemented is given in the following section. A more detailed analysis and state of the art can be found in [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF].

Coudray's Algorithm

The multi-resolution approach is described in the diagram of Figure 2.11 illustrating the general structure of the segmentation strategy. The main idea behind this technique is a multi-resolution gradient analysis associated to an adaptive threshold allowing the extraction of contours at different scales.

A multi-resolution edge detection algorithm has been developed to partition the image: edges are firstly extracted at different scales, using a gradient analysis associated to an adaptive thresholding. At each resolution, the threshold allows the selection of the pixels corresponding to a real gradient limiting the false positive rate at 2%. To fix thresholds across scales, a method has been introduced based on a piecewise linear regression to fit the whole descending slope of the gradient histogram. This method gives a reliable estimation of the threshold, and is practically insensitive to the noise distribution and to random histogram fluctuations.

More specifically, a non-dyadic pyramid is used for the multi-resolution implementation where a gradient filter (Prewitt) is applied at each resolution. Then a threshold allows selecting the pixels corresponding to a real gradient limiting the quantity of false positive pixels. Contour pixels that are below the threshold will be detected at a coarser scale. The threshold remains robust through scales and determines with the same percentage of confidence real contours.

A set of binary images result from this processing step containing the contour pixels identified at each scale.

The larger the scale, the larger, in pixels, the contour appears in the binary image due to smoothing and down sampling that are associated to multi-resolution approaches. Contours are therefore localized more precisely at finer scales. A compromise between localization and detection is assured with the RGL image as this last forms the weighted combination of the binary thresholded images. The value of each pixel in the RGL image indicates the scales at which it has been thresholded. Compared to a gradient image, the RGL image is a virtual representation of contours, almost noise-free thanks to the adaptive thresholding. It is therefore well suited for the contour identification as the watershed algorithm segments an image into regions with closed contours. Watershed progressively floods the regions starting with the lower values and marks the merging of two basins with the watershed line [START_REF] Vincent | Watersheds in digital spaces: an efficient algorithm based on immersion simulations[END_REF], [START_REF] Beucher | Use of watersheds in contour detection[END_REF]. The edge pixels provided form 1-pixel-width discrete curves with 4-pixel connectivity and thus a convenient partition of the image. Contours from now on are defined as the borders delimiting the regions determined from the segmentation algorithm. An example of such a segmentation applied to a typical TEM image, containing isolated and aggregates of vesicles, is shown in Figure 2.10.

Over-Segmentation: Introduction to the Problematic

Segmentation techniques are major components of image analysis and hundreds of segmentation algorithms have been proposed in the last few years. Most of them resolve only a defined problem depending on the image specifications but at the same time create an over-segmentation.

The nature of TEM images and the resulting over-segmentation therefore led us to the elaboration of an original algorithm that treats over-segmentation. The tests performed in the thesis by [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF] showed that the multi-resolution techniques allowed the extraction of all the major contours (particularly mono-membrane ones) even if they appear as contrasts with variable amplitude, type or slope. It was important for our application to detect a maximum of contours to ensure low contrasted membrane detection. The combination of the RGL image and the watershed algorithms realized a compromise between a good detection and a precise localization of the contours, which, on one hand ensured the detection of low contrasted membranes, but, on the other hand, generated over-segmentation.

In the following section, we will discuss methods to reduce over-segmentation with the use of preprocessing approaches. It will be then explained why post-processing approaches are preferred.

Introductory Literature

Over-segmentation is a common problem in the domain of image processing while watershed transformation is used for segmentation. Watershed-based segmentation allows however a double framework for over-segmentation reduction, presented in the literature as pre-processing or post-processing techniques.

Watershed may be used in combination with a family of pre-processing techniques well-known as geodesic reconstruction. [START_REF] Najman | Geodesic Saliency of Watershed Contours and Hierarchical Segmentation[END_REF], and [START_REF] Beucher | Segmentation d'images et morphologie mathématique[END_REF] introduced and used different versions of morphological operations (reconstruction by erosion) that simplify the gradient image and reduce oversegmentation. Meyer (1990) tries to filter the image by eliminating details in the image with the help of morphological filters, the levelings, and preserving sharp edges. A different solution [START_REF] Frucci | Using resolution pyramids for watershed image segmentation[END_REF] may be used by means of multi-resolution computation of the image gradient to select region minima for watershed transformation. [START_REF] Nguyen | Watersnakes: energy driven watershed segmentation[END_REF] proposed watersnakes, an integration of energy-based and watershed segmentation, to remove wrongly introduced limbs around the objects of interest. Most of these approaches, however, work well for images with weak noise and well-contrasted edges. Post-processing merging steps are necessary after their application. Presently, the multi-resolution segmentation algorithm (paragraph 2.3.2) contributes already to the noise smoothing of the image, identifying at large scales contour pixels that were not detected at finer ones. No further pre-processing, as suggested above, can be applied. Post-processing methods, on the other hand, were considered as a solution to our over-segmentation problem. A post-processing stage is therefore considered to merge regions that in our case are surrounded by low contrasted contours. Merging criteria can be therefore introduced depending on the nature and characteristics of the segmentation and TEM images (as described in paragraph 2.2). Figure 2.12 shows a simple example of a TEM image and its segmentation into regions. The plot of the figure shows that average region gray-level is not a sufficient criterion for region merging. Regions and mostly the background suffer from gray-level fluctuations and strong noise. Our criterion therefore needs to stay sensitive to low contrast and noise amplitude.

False Contours

The 3D representation of Figure 2.13 shows an important property of membranes: the heterogeneity of intra membrane regions. Membranes may be only rarely found as homogeneous regions within TEM images;

globally, mono-membrane regions are low-contrasted compared to the background, present important intraregion fluctuations in a highly noisy context; all these parameters complicate more their nature and therefore their identification. The membrane border of Figure 2.13 is visible only by comparing local gray-level intensities in a relatively small area around the contour. The human eye detects the contour confirming the existence of a sufficient gradient in the contour neighborhood that marks the membrane border (pixels in yellow) indicating local approaches. An important feature element is therefore raised based on local contrasts significant enough to certify the existence of membrane borders. As a result we noticed that: i) real contours correspond to more regular and coherent gradients, ii) other contours do not correspond to real borders. Presently, regularity and coherence concern the consistency of the gradient amplitude and direction along a contour. The segmentation algorithm searched for gradients in order to detect a high number of contours in the image, even the low contrasted ones. Our objective is to find membranes and validate the backgroundmembrane transition that looks more like a "noisy step" transition (a more extensive study will be provided in chapter 3). We therefore impose another criterion, the one of a local gradient characteristic to our transitions in order to evaluate our detected edges. In this way, we are able to differentiate the real ones from artifact contours induced by the strong noise or the algorithm itself. We define these contours as false contours and we will use this term from now on to refer to them. These contours belong to one of the following groups:

x Ridge-like contours (Figure 2.14) appear as strong gradients like an inverse Gaussian profile. This kind of edge does not present a step-like profile and is not characteristic of a membrane-background transition but more to a crack or a slight folding of a membrane. This type of contour is therefore a frequent and characteristic transition appearing in the interior of objects; an interesting element to be used for future object characterization studies. x Null-gradient contours (Figure 2.15) are characterized by a complete absence of coherent gradient.

They are artifacts of the RGL image [START_REF] Coudray | Techniques de segmentation d'images et stratégie de pilotage pour l'analyse automatique d'échantillons en microscopie électronique Application à la cristallisation 2d[END_REF]) coming from the "widening" of the thresholded contours detected at coarser resolutions. The idea of resolution as utilized in the segmentation method used in the current study is proportional to the width of thresholded contours.

These can create false basins and therefore false watersheds.

x Trigger-gradient contour where (Figure 2.16) gradient exists only partially. These contours are driven by partial gradients and they arise as artifacts when contours are not correctly traced by the watershed transform.

Conclusion

This chapter presented the first step of the automatic analysis used in medium magnification accompanied by an extended discussion regarding the difficulty of correcting the partition of such images. The properties of this technique are emphasized in the context of TEM images. The detection performance of this method is satisfactory even for mono-membrane regions with a localization precision of the contour that is adapted to the nature of our images. However, this technique detects some contours that do not correspond to a real border between objects in the initial image. The segmentation obtained forms thus an interesting base for our additional processing methods. It will be admitted that all the region borders to be analyzed are represented with localized contour segments with sufficient precision. A complementary technique, detailed in the following chapter, is however necessary for the identification and elimination of false contours.

Chapter 3

Statistical Evaluation of Contours

Introduction

The segmentation step, as presented in the previous chapter, provides a list of contour segments that separate the image regions. Over-segmentation reduction is a classical step in the domain of image processing and in our case a preliminary step for region identification.

A literature overview shows that almost all techniques are founded on region comparisons. According to a specific criterion, similar regions are merged. In our images, object gray-levels present fluctuations that do not allow such simple comparisons (e.g. region average gray-level). The gradient continuity along a contour provides a reliable criterion. To get the best sensitivity we use a validation technique based on statistical hypothesis testing.

During the first section of the chapter, the problem is introduced. Principal methods for over-segmentation reduction are presented in section 3.1.1. Then the proposed method is presented and described in detail with a few intermediate results showing its robustness and efficiency.

Problem Statement

The segmentation step uses basically a local gradient measure to detect the pixels associated with a contour, and then attempts to connect the best candidates to draw borders of regions. This method inevitably produces false contours that are not associated with object borders. Most of these false contours are artifacts of spurious gradients that complicate object recognition by unnecessarily over-segmenting image regions.

The efficiency of the region identification step is therefore significantly reduced by this over-segmentation.

A criterion is therefore needed allowing us to verify whether a contour determined by the segmentation algorithm is probably a real object boundary.

State of the Art

Different criteria are presented in the literature in order to deal with over-segmentation. [START_REF] Frucci | Using resolution pyramids for watershed image segmentation[END_REF] and [START_REF] Weickert | Efficient image segmentation using partial differential equations and morphology[END_REF] referred to region merging by simply using the contrast difference between adjacent segmented regions. [START_REF] Pavlidis | Integrating region growing and edge detection[END_REF] dealt with the problem of merging after having applied the split-and-merge algorithm.

The main idea was to check whether all region boundaries qualify as edges. Two features were employed within the merit function introduced: the contrast across a boundary and its shape. Contour gradient information and minimum valid region size were verified as criteria in order to merge regions (Jung et al.

2005).

Criteria regarding region homogeneity as defined above cannot be applied directly to our images, as membranes do not possess sufficient region characteristics (section 2.2) or texture features. On the other hand, membrane borders correspond to real contours that are correlated with a true 'step -like' gradient; a fact that inspired us to employ methods based on the investigation of the local contrast of a contour adapted to the image specifications, as described in the following section.

The current study presents a statistical technique that should preserve mono-membrane regions in spite of their low CNR. The statistical technique helps us to optimize the validation thresholds. To evaluate the edges identified by the segmentation method, we measure a local gradient along the contour segment. A statistical threshold is introduced that aims to classify False Contours FC (according to the definition of section 2.4.2) from True Contours TC with a certain degree of confidence and by taking into account the noise amplitude and the sample size of data used for the test.

Statistical Contour Evaluation: Presentation of the Algorithm

The overall evaluation algorithm allowing the reduction of over-segmentation will be presented below.

During the first part of this section, certain mathematical notations will be given followed by the algorithm presentation: the criterion used and the merging method. Algorithm properties and results will then be discussed.

Algorithm Principle and Mathematical Notations

Contour segments

A brief definition of contour segments is given: the image created by the multi resolution watershed segmentation is formed by regions and edge segments. A region, defined by a group of adjacent pixels, is labeled R i , where 1,..., i K and K is the number of regions extracted in the image. An edge segment is defined by groups of pixels that rise between adjacent regions. We denote as S(u,v) the set of pixels that separate regions labeled as R u and R v .

Null hypothesis test

The contour evaluation is based on a statistical test: a null hypothesis test, and for this purpose a few aspects of such a test will be provided [START_REF] Ning-Zhong | Statistical Hypothesis Testing: Theory and Methods[END_REF]. Hypothesis testing is an essential part of statistical inference. In each problem considered, the question of interest is actually simplified into two hypotheses and there is a choice to be made; the null hypothesis, denoted H 0 , against the alternative hypothesis, denoted H 1 . The test aims at by evaluating the error risk from available observations. Presently, these hypotheses are formulated as: H 0 : "The pixel is not a contour pixel." against H 1 : "The pixel is a contour pixel."

The outcome of a hypothesis test is "Reject H 0 in favor of H 1 " or "Accept H 0 ".

Type I Error

In a hypothesis test, a type I error occurs when the null hypothesis is rejected while it is in fact true; simply meaning hypothesis H 0 is wrongly rejected. A type I error is often considered to be more serious, but it is never equal to 0. The hypothesis test procedure is such that a 'low' probability of rejecting the null hypothesis wrongly is ensured. This probability is symbolized as: P(type I error) =Ơ

3.1

Type II Error

In a hypothesis test, a type II error occurs when the null hypothesis H 0 is not rejected while it is false. A type II error is frequently due to sample sizes being too small. The probability of a type II error is generally unknown and is written:

P(type II error) = ơ 3.2
A summarized presentation of decision results of a hypothesis testing is given in Table 3.1:

Decision

Hypotheses

Reject H0 Accept H0

H0

Type I Error Right decision

H1

Right decision

Type II Error

Test Statistic

A test statistic is the quantity calculated from our data sample; it is used within the test to decide whether or not the null hypothesis should be rejected. In this thesis, it actually constitutes an original local gradient value to be compared with the statistical threshold.

Critical Value

The critical value for a hypothesis test is a threshold T; this is the value to which the test statistic is compared to perform the null hypothesis test. It is dependant of the estimated standard deviation of the image noise and takes into account the pixel number that contributes to the test statistic measure. The critical value for any hypothesis test depends on several parameters; two of them are, the significance level at which the test is carried out, and type of the test (one-sided or two-sided).

Significance Level

The significance level of a statistical hypothesis test is a fixed probability and more specifically, the probability of type error I and is set by the investigator. Often the significance level is chosen as small as possible in order to protect the null hypothesis and to prevent any false conclusions. The significance level is usually denoted by: Significance Level = P(type I error) = Ơ

3.3

Usually, the significance level is chosen to be 0.05 (or equivalently, 5%).

One-Sided Test

A one-sided test is a statistical hypothesis test in which the values for which we can reject the null hypothesis, H 0 are located entirely in one tail of the probability distribution. A one-sided test is also referred to as a one-tailed test of significance. The choice between a one-sided and a two-sided test is determined by the purpose of the investigation or prior reasons for using a one-sided test.

Description of the Algorithm

A contour segment separates two regions; it will be therefore validated or not to merge regions. The technique developed introduces a local measure: for each one of the pixels of the contour, our method evaluates the correlation of neighboring gray-levels with a profile, characteristic of a transition. The hypothesis test eventually uses the average measure of the segment pixels.

Local Correlation with a Reference Profile

The criterion is based on the examination of the gradient perpendicular to the contour and is computed for each contour pixel p of segment S. Three main steps are related to this technique:

x The estimation of the local orientation of the contour;

x The perpendicular profile extraction from the image;

x The correlation of the profile with a reference filter that constitutes the test statistic.

Estimation of the Local Orientation

The estimation of the local orientation is made within a given neighborhood (in our case a 7x7 pixel window was employed and found sufficient for the orientation estimation) around the pixel of interest with x, y, the edge pixels coordinates, cov(x,y), the covariance of x,y, var(x), the variance of x, and x , the mean value of x. For vertical lines, the above equation gives slope equal to zero.

Perpendicular Profile Extraction

This estimation is used to orient a profile extraction mask centered at each pixel p and perpendicular to the contour. Since we consider a mask of a 1-pixel width we extract a 1D profile for each contour pixel, (Figure

Test Statistic: Gradient Correlation

Profile collections (section 2.2) allowed us to establish characteristic profiles. Often optimal filters are obtained by correlating a template (known signal) with the signal under investigation to detect the presence of the template in this signal. In our case, 1D profiles may occur in different transition forms (as already discussed in chapter 2, section 2.2.4). Several filters seemed appropriate for the gradient calculation and therefore different gradient filters were used (ramp, step, sigmoid) and provided approximately similar results.

Presently, the reference profile represents a line f of a slope f D to obtain a normalized (mean zero and variance one) filter:

i i f f D E f
, where parameters f D and f E are determined by:

0 1 2 1 i i n ¦ °® ¦ °f f 3.5 A convolution ( )
C p or test statistic is computed for each pixel, by fitting with the filter coefficients of equation 3.5:

1 ( ) 1 n C p ip i n i ¦ X f 3.6
where ip X represents the image data extracted by the 1D oriented mask for pixel p, and n the number of pixels of the 1D vector. An analytical expression for ip X represents the profile in terms of its underlying signal and the random part:

k b ip ip p ip H X Z 3.7
where k is the amplitude coefficient of the normalized signal ip Z , b p represents the average value of the signal X ip and İ ip the additive noise. The coefficient ( ) C p of equation 3.6 then results in:

1 1 ( ) 1 1 n n C p k ip i ip i n n i i H ¦ ¦ Z f f 3.8
The first term represents the amplitude of the signal weighted by the correlation coefficient with the filter and the second one corresponds to the covariance of the noise with the filter.

Edge Pixel Hypothesis Test

In order to decide whether a contour pixel is part of an edge significant enough to separate two adjacent regions, a null-hypothesis test, as already presented in the previous section, is applied. The test statistic ( ) C C p , as described in the above section, is computed for assessing the truth of hypothesis H 0 . This hypothesis is accepted if: C T d

3.9

Threshold T Computation

To determine the thresholds, we need to establish the distribution of the test statistic for the null hypothesis.

By definition, while hypothesis H 0 is true, k = 0 and we only observe noise within the 1D data of equation 3.7. The convolution or otherwise covariance (3.8) depends only on the noise:

1 ( ) 1 n C p ip i n i H ¦ f 3.10
where f i corresponds to the filter coefficients as defined in equation 3.5. C(p) is therefore a weighted sum of the independent variables of equation 3.10 that have a normal distribution:

(0, )

noise N V

3.11

We therefore deduce that the properties of the variance of C(p) has as well a normal distribution of mean zero and variance:

2 2 2 2 2 1 1 1 n C noise i noise i f n n V V V ¦ 3.12
Threshold T then becomes:

a noise T n V 3.13
where ı noise corresponds to the standard deviation estimation of the noise, n is the number of pixels of the 1D data and a represents the confidence with which we verify our hypothesis or significance level and is associated to the probability expressed as below:

0 ( | ) P C T H D t 3.14
where D is the desired significance level of this test and C=C(p). For D = 0.3%, coefficient a is fixed to 3.

The threshold, as determined in equation 3.13, is unfortunately set very high for the characteristic membrane CNRs. Statistical decision procedures are based on sample information and so the possibility of errors must be considered. Probability of equation 3.14 actually represents how much we risk committing a Type I error. On the other hand, when a false null hypothesis can fail to be rejected another kind of error is made (3.1.2.4): a Type II error with probability E occurs:

0 1 ( ( )) H k noise n P C T C E V or 1 | k k P C T C C E 3.15
where C k1 is the mean of coefficients C distribution for a given signal amplitude k 1 .

The importance of the noise, does not allow the choice of a threshold T that optimizes the criterion in order to take a decision at a pixel level. An empirical verification shows that this approach is not sufficient: as it can be seen in Figure 3.3, Type II error is too important to be ignored for the classification of the red window zone of the image that contains a mono-membrane region. The membrane border that interests us is situated between regions 4 and 5 with its classification into False Contour Pixels (FCP) and True Contour Pixels (TCP) illustrated in the same figure. Its classification shows an important quantity of false negative contour pixels arising within a low contrasted membrane contour with a probability E estimated from equation 3.15 at 73%. The 1D edge pixel test remains valid for classifying significant image contrast contours as noticed in various images. However, it is not sufficient for our problem, as low contrasted edges are those who are important for the application. With the help of equation 3.15, we have evaluated that only for contrast amplitudes with CNR>2.4, probability E becomes minor.

Edge Segments Validation: Large Segments

In order to lower probability E for a proper classification, we considered a bigger neighborhood around the pixel of interest. In this way the number of pixels n that contribute to the computation of the coefficient C is increased, allowing a more significant threshold T to be defined.

An example of a 7x21 pixel size neighborhood has been considered. The signal extraction masks of 7x21 pixels are centered on the contour pixel and oriented perpendicularly on the contour. The normalized filter associated to such a convolution is created by duplication of 1D filter weights parallel to the contours. Its application assured a better recognition of real edge pixels but the rate of false positive pixel is still too high to permit a proper classification. As an example, the null hypothesis test H 0 was applied at each edge pixel of the green window part of the image as shown in Figure 3.3. After all contour pixels are classified; a method to retain or not the whole contour depends on its percentage of validated contour pixels. The selected threshold leads in this case to a good classification of the contours of the membranes. However, false contours are not correctly identified, as it is pointed out in Table 3.2. Segment S(1,2) of Table 3.2 presents approximately 50% FC and 50% TC, prohibiting any reliable decision. A large number of images show that the edge pixel test is insufficient whatever method of combination is used for a given segment (1D or different size of 2D profiles). Therefore, the percentage of FCP within the segment cannot constitute a proper measure.

Since taking a decision at a pixel level is not satisfying, the transition absence hypothesis test is applied within an edge segment. For this reason, (Figure 3.4) the average correlation ( , ) C u v with the slope transition profile is computed along the edge segment S(u,v). For this computation, we take into account the gradient's amplitude and direction along the contour. The notion of direction is simply explained by the fact that we averaged the signed correlations ( ) C p of segment pixels:

( , ) 1 ( ) ( , )
p S u v C C p l u v ¦ 3.16
where l corresponds to the segment length.

We then compare this coefficient with threshold T cs that now depends on the number N of pixels that contribute to the average coefficient calculation. This number expresses the number of pixels within the local neighborhood of a segment, used for the average ( ) C p computation.

a noise T cs N V 3.17

Edge Segments Validation: Small Segments

Most segments are long enough to reduce the error rate considerably. The very small segments (smaller than 15 pixels) are not processed likewise; not only is the test statistically more uncertain, but they are often located at the intersection of several contour segments where the hypothesis of an orthogonal gradient signal is less reliable. For small segments, local contrast is computed by comparing gray-level intensities computed for pixels that are adjacent to the border between two regions. Under the assumption that the two distributions have the same variance, the threshold can be calculated as follows (Ning-Zhong and Jian

2008):

-its value C ij is less than threshold Tc ij.

-its value C ij is less compared to all segments of regions R i and R j .

Results

Several experiments were conducted on a variety of TEM images where the efficiency of the proposed method for region merging has been systematically assessed. Various series of images acquired with different TEMs taken under standard acquisition conditions of illumination and exposure time were used during this evaluation.

To evaluate the relevance of our algorithm, we utilize two principal criteria that derive the objectives of our application: i) the elimination of all the false contours that divide the background into several regions, ii) the preservation of the contours that separate objects from the background. The exhaustive validation of the final segmentation result by an expert is tedious and difficult work to accomplish especially because of the high-level experience of experts. These partial criteria verify the essential information for further processing.

Qualitative Analysis

It is difficult to devise a systematic test to validate the segmentation results. We can though analyze our results qualitatively and quantitatively. Qualitatively, we visually verify the capability of the algorithm to recognize real contours from false ones. Three typical TEM images in Figure 3.7 illustrate recognition results evaluated by an expert.

TEM image of Figure 3.7a was acquired with a Tecnaï T12 and pixel size of around 13 nm/pixel. It represents a typical image containing many different-size membrane sheets of different contrast with well chosen acquisition conditions. Our technique extracts and suitably selects the important contours, by finally segmenting low contrasted membrane regions properly. Moreover, background over-segmentation is eliminated, according to expert's evaluation.

Figure 3.7e shows an image acquired by another microscope, an H7000, and pixel size of 10 nm/pixel. The image formed consists of small isolated vesicles and a stack of large folded ones. Illumination conditions, however, were not optimal as the heterogeneity of either the background or the objects is more accentuated.

The results as figured in Figure 11g and 11h and as evaluated by the expert show an efficient contour segment classification.

The last example of TEM image (Figure 3.7i) acquired with a Tecnai 12 (7 nm/pixel) presents the same background heterogeneity. Membrane regions appear particularly low contrasted (CNR = 0.15) within a dirty background full of artifacts (white circular zones). The approach proposed even under such conditions permits to globally obtain a proper contour validation. Only a few false contours were not correctly identified; their misclassification does not affect the overall result of the algorithm as major false segments that over-segment the background were properly identified.

Quantitative Analysis

A more systematic test of the algorithm results (segmentation and contour evaluation algorithms) was realized by utilizing a simulation framework. A representative synthetic image is established including the ground truth for validation. This synthetic image is manually created according to the characteristics that TEM images present, as referred in section 2.2; it contains two objects with dark and bright zones (with two low contrasted regions), a heterogeneous background and gray-level fluctuations applied to only one of the two low contrasted membrane zones. To complete our image model, an additive Gaussian noise is applied to our synthetic image. A noisy version of our synthetic image is shown in Figure 3.8. For this example the noise standard deviation is fixed to 7, as it corresponds to a typical contrast between two regions with CNR equal to 0.6. This noise level contributes to the generation of false contours during segmentation; only 5.6% of the false contour pixels are not correctly eliminated by the contour evaluation algorithm while all low contrasted region contours are preserved.

a) b) c) d) e) f) g) h) i) j) k) l)

Results Comparison with Classic Merging Method

In parallel, to make a comparison of our proposed method with state-of-the-art methods, we chose one of the most common region merging techniques based on region homogeneity criteria [START_REF] Frucci | Using resolution pyramids for watershed image segmentation[END_REF][START_REF] Weickert | Efficient image segmentation using partial differential equations and morphology[END_REF]; it simply consists of finding neighboring regions that do not differ by more than a specified contrast gray-level value; in this case different values are set as the contrast threshold. Figure 3.10 shows the results obtained for a representative TEM image by applying: our statistical method and a method that uses the global gray-level contrast with thresholds 110, 120, 130, as presented in 3.1.1.

A comparative study of the two algorithms is performed with the help of a segmentation evaluation tool as introduced by [START_REF] Daum | Evaluation de segmentations d'images : mesures de similarité avec une référence partielle[END_REF]; we wish to observe the efficiency of our algorithm while eliminating the background over-segmentation compared to the one given by the global contrast evaluation. The tool used is based on a comparison of reference segmentations and the segmentations to be evaluated. The criterion therefore is based on the surface comparison of such segmented regions. A surface overlap ratio is computed for the regions to be compared with a certain tolerance. In this way, we are able to conclude an over or under-segmentation of regions, of a good or bad segmentation. Presently, we make use of simplified segmentation references; by the term simplified, only a certain number of regions are taken into account as a reference. As in the present study, we are particularly interested by an evaluation of the reduction of background over-segmentation either by our method or the classical one, the background region of the image is provided by an expert (Figure 3.1), to be used for the comparison. The expert therefore made a selection of segmented regions that represent the background of the image and appear in the reference image in orange. The reference of Figure 3.9 is then compared to our segmentation results and the segmentation of Figure 3.10a to obtain an evaluation of the merging method efficiency. Our method seems well-adapted with satisfactory results (Figure 3.11d). Global contrast methods, on the other hand, were proved to be insufficient no matter the threshold value. This method was tested on 10 representative TEM images where regions are wrongly merged. For the present example, evaluation results show that the background still remains over-segmented into several regions (red, blue, cyan and yellow) including even a region that was falsely merged (green) with a membrane one (Figure 3.11c); these regions represent 5% of the total background surface reference for this example.

Introduction

The specimen elements present on the grid are correlated with one or several image regions as provided by the previous processing step, as described in chapter 3. Before attempting to identify the image objects, it is essential to distinguish object regions from background regions in the segmented image. Common methods described in the literature regarding the background extraction problem are based on global or local thresholding techniques. However, for complex images such as TEM images corrupted by strong noise and background fluctuations, such methods are not satisfying.

An original method was therefore developed for TEM background extraction. The background extraction is performed in two steps. The first allows finding the principal background region; the application of some a priori criteria, elaborated for bright field TEM images, provides an estimation of the mean background graylevel. The second step identifies all smaller background regions. Each step uses the segmented image, the mean gray-level of the regions and a priori knowledge regarding the aspect of the specimen elements on the grid.

In this chapter, we will show that the identification of background regions in TEM images is based on a basic assumption: specimen elements are expanded on the grid so that the background present in the images to be analyzed covers a large zone of the image. The background corresponds to the carbon film that serves as a support to the specimen and has the brightest gray-level in the image. These criteria introduce a low error risk that will be discussed. A more complex method will be therefore introduced for the identification of smaller background regions based on local comparison of gray-levels.

Problem Statement

The previous chapter described the processing step that yields a final segmentation of the image into a number of regions to be identified. Before attempting to recognize objects, it is interesting to determine all the regions that belong to the background. This step allows isolating groups of adjacent foreground segmented regions. These groups, that we call foreground entities, will afterwards be analyzed and classified to recognize the objects present in the sample.

The distinction between background and foreground image regions cannot be achieved with a simple graylevel thresholding. The absence of other direct criteria led us to develop an identification strategy of the background regions.

As already detailed in previous chapters 2 and 3, a direct identification of regions is a difficult task to accomplish since they do not "carry" enough features. Segmented regions cannot be directly identified as membrane regions or not (chapter 2, section 2.2.3) since no sufficient characteristics can be found within the regions. At the magnification where our images are acquired, crystal ordered structure is not identifiable and texture features are therefore not useful. Moreover, membranes present a great and diverse variety of shapes prohibiting the application of shape recognition algorithms.

The background, which physically corresponds to the carbon film, represents regions that have the brightest gray-levels of the image. The average gray-level GR i of segmented regions R i seems therefore a representative feature candidate of the region to explore and utilize for characterization.

Motivations

Relevant Literature

Common methods that exist in the literature regarding the background extraction problem are based on global or local threshold techniques. However, in complex images with strong noise or non-homogeneous background, such methods cannot be considered and specific information, related to the application, is utilized.

Examples of background extraction can be found in color natural images where the size, the position and the color of the background [START_REF] Lu | Background Removal in Image indexing and Retrieval[END_REF] are used as hypotheses for its extraction. In low quality text document images, background is discriminated from text based on local statistical properties of predefined regions (connected components) [START_REF] Garain | On foreground -background separation in low quality document images[END_REF] or for page segmentation another idea is to efficiently produce a flexible description of the background space that surrounds the printed regions in the page image; from this description, the contours of printed regions are identified enabling the identification and analysis of segmented text blocks [START_REF] Antonacopoulos | Page Segmentation Using the Description of the Background[END_REF]. [START_REF] Frucci | Oversegmentation reduction in watershed-based grey-level image segmentation[END_REF] followed another scheme for images of neurons of rabbit cerebral cortex based on gray-level changes of adjacent regions.

TEM images contain rich gray-level information implying a multi-modal image histogram, presenting several classes of gray-level distributions articulated in well-defined peaks. Thresholding algorithms of multi-modal histograms [START_REF] Otsu | A threshold selection method from gray-level histograms[END_REF][START_REF] Hou | On minimum variance thresholding[END_REF][START_REF] Tsai | A fast thresholding selection procedure for multimodal and unimodal histograms[END_REF], Sahoo et al. 1998) would be ideal for their identification.

Our Approach

Criteria, based on hypotheses specific to our application, were adapted to our images in order to provide the background region and are validated for a large number of TEM images. The approach developed is based on the general assumption that specimen elements are scattered enough on the grid so that the background represents an important part in every image. Moreover, the background has the brightest gray-level in the image. This information is used as the criterion to select the main background region.

The background does not necessarily form a single and continuous region. The identification of all background elements is more complex because of the image particularities (such as gray-level fluctuations, etc.) presuming local contrast comparisons. This local comparison will be completed systematically for all the regions to assign a stacking-level label to each region using the main background region as initial reference. The regions labeled "0" are assigned to the background. The labels with greater number provide an interesting feature for a classification, as presented in the preliminary study (Appendix).

The concept of stacking-level is an unusual notion in the image processing domain; however it is a feature with a great importance to our problem. It translates the mean gray-level of a region into a number of superposed membrane layers. Membranes are practically transparent and appear slightly darker than the background; they are also flexible and can therefore be found folded or even superposed on the grid. The gray-level of a region may thus be considered as characteristic of the number of superposed membranes, called a stack of membranes.

The background extraction algorithm presentation is therefore decomposed into 2 steps:

x The detection of the principal background region. TEM images are pre-selected so that the background is present in the images. The application of some a priori criteria provides the principal background region. At this stage, only a unique background region, the major one, is detected.

x The detection of secondary background regions with the introduction of a priori knowledge. An evaluation of object stacking-level is effected based on topological and statistical criteria.

Background Extraction: Presentation of the Algorithm

Principal Background Detection

Background Characterization

In Even though TEM images present gray-level fluctuations related to the acquisition conditions (non-uniform illumination, etc.), the background appears globally bright and large. As presently the segmented image regions are the input of our further processing, the mean gray-level of the regions and their size are used as criteria for the detection of the main background region.

Background Extraction Algorithm

We propose a background extraction technique that firstly computes the average gray-level and size for each segmented region. Among large regions the brightest one is chosen as the main region of the background. This detection method provides adequate results while detecting the main background region in TEM images. However, more complex cases may exist: the background is composed by several regions e.g. the large one that is well-detected plus one or more smaller ones that are not yet detected. We therefore need to complete the background extraction process by introducing supplementary information to accomplish a semantic labeling of our segmented regions according to their stacking-level.

Secondary Regions of Background Extraction in TEM Images

The above criterion aimed at detecting the principal background region within TEM images by using simple image features. Improving the background extraction consists of determining all regions that compose the image background; this latter may be split into more than one region not adjacent to the principal one (separated by foreground entities) or adjacent to the principal one (some rare false contours that were not eliminated during the statistical contour evaluation).

We propose a method that allows the distinction of different classes in the image. A collection of a certain number of non-stacked regions is collected, to determine the quantum, a gray-level quantity characteristic of the thickness of a single membrane layer. The principal background region will constitute a reference region for the quantum determination. Secondary background regions have average contrast inferior to the quantum.

Stacking Representation

This work consists of labeling each region according to its stacking-level; regions that do not have any stacking-level (level "0") will be classified as supplementary background ones. The interest of this representation is multiple: it allows tagging mono-membrane regions for their further characterization; these regions will be examined in order to verify the membranes crystal structure and therefore validate the crystallization trials.

We will thus create a classification of the image regions that are homogeneous and therefore have a single label characteristic of their stacking-level. Consequently, many stacking-levels will be present in the image;

label "level 0" is attributed to the background regions and label "level 1" to mono-membrane regions that are those that interest us during the present study. This method [START_REF] Hermann | Evaluation of membrane stacking in electron microscope images[END_REF]) is iteratively applied in order to determine upper-stacked levels ("level 2", "level 3", etc.); this information is useful to characterize the objects present in the images.

The proposed stacking representation is made by defining gray-level ranges representative of different stacking-levels. A gray-level interval is determined from an average gray-level, called quantum, calculated from the contrast between the principal extracted background and certain mono-membrane regions automatically selected according to the following hypotheses.

Mono-Membrane Region Selection

To determine the quantum, the gray-level quantity related to a mono-membrane region, we introduce certain hypotheses. These hypotheses are based on spatial structure criteria and permit to select regions having label "level 1" with strong confidence.

These hypotheses are:

x Region size: large regions are selected for our computation. Very small regions are ignored as they can bias our estimations.

x Neighboring regions: mono-membrane regions are brighter than any other neighboring regions, excluding the principal background region.

x Background proximity: within a collection of objects, mono-membrane regions are mostly found at their periphery. At their center, membrane aggregates can be found. Selected regions are therefore adjacent to the background.

The following section describes the quantum determination steps. The classification will not be correct, if these observations are not taken into account; fluctuations can bias the average gray-level of the principal background region and therefore could be considered darker than it is. Consequently, the contrast of a "level 1" region may be under-estimated, being inferior to the quantum, and wrongly classified. Figure 4.6 shows the labeling of the TEM image of the same figure where we notice that "level 1"membrane regions are identified as background ones represented in yellow. These phenomena therefore cannot be ignored requiring a different approach.

The correction of these problems is not trivial because the general image fluctuation model is too complex.

Methods found in the literature simplify the problem in different ways. [START_REF] Tomazevic | Comparative evaluation of retrospective shading correction methods[END_REF] describe the problem of fluctuations in images from optic devices. They propose a comparative evaluation of retrospective shading correction methods. Shading corresponds to intensity inhomogeneity and the shading model can be simplified by considering two components: an additive and a multiplicative. Among all the proposed methods, the surface fitting technique seemed the most appropriate. Fitting methods were also applied to TEM images (Mendizabal et al. 2004). We tested and evaluated fitting methods but the obtained correction does not permit to classify all membranes correctly. Fitting methods deal with additive or multiplicative issues. This is not sufficient for characterizing our shading drawbacks. Other models (Dorval between membrane levels and gray-level is not proportional. Moreover, the errors introduced during the quantum calculation would be amplified in further layers.

An iterative approach is therefore proposed. For each layer, a quantum is iteratively defined. For the following quanta, the three steps of the quantum algorithm computation are adapted. The local minima are searched in regions that are not yet classified. A few regions are selected for the quantum calculation, eg.

the local minima that are close to the background or the regions already labeled (inferior stacking-level).

The quantum Qi of level i is therefore defined as: with M i the number of considered regions for the layer i.

Once quanta are calculated, the gray-level limits of different layers can be defined. Upper gray-level is always calculated as 150% of the quantum of the considered layer and lower gray-level limit is defined by the upper-level of the previous layer. The results of the algorithm are illustrated in Figure 4.8.

Results

The proposed algorithm was tested on various TEM images. These images were chosen in collaboration with the biologist experts in order to validate our work so that our results correspond to their expectations.

These images were carefully chosen as they represent a large panel of different cases. They differ by their aspect and attributes:

x Utilized microscopes (Tecnai 12, Hitachi, etc.)

x Acquisition conditions (illumination intensity, exposure time, defocus, etc.)

x Sample nature (different proteins to be crystallized, detergent removal, staining, etc.)

x Type of the objects present in the image (sheets, vesicles, aggregates, stain artifacts, water drops, etc.)

Consequently, 45 images were chosen and segmented by our pre-processing algorithms. The expert checked the regions corresponding to the image background, that constitutes our reference. In order to validate our algorithm, three classes are defined:

x well detected background when more than 90% of the background is detected,

x partially detected background when 80-90% of the background is detected,

x and background misclassification if less than 80% of the background is detected.

Results presented in the principal background extraction column of The third column of the table then provides the analysis of the background including the secondary regions.

The well-detected background class now represents 95% of the images; 50% of the images of the second and third class are now part of the well-detected class. We note as well that within a majority of our examples, the size of undetected background regions was reduced. Qualitative results of our algorithm behavior are illustrated in Figure 4.9 where the main background region in black is extracted satisfactorily.

In Figure 4.10 the following color code is used: in black the principal background region; in yellow the secondary regions that are found with the help of the stacking algorithm. Colors red, green, magenta correspond to stacking layers "level 1", "level 2"... The first image shows a case where the background was over-segmented in several "pieces". The larger background region is extracted by the first background detection technique. The rest of the background regions are then detected by the stacking algorithm. The second example illustrates a unique background region that was over-segmented. False contours were not detected in their integrity; because of that the background extraction has only partially been detected by the first step. This extraction is however well corrected by the second step of the stacking representation.

In some rare cases where the background is only partially detected, those misclassifications can be explained by the following characteristics of the images:

x The first step of the background detection requires a significant presence of the background regions in our images. The automatic pre-selection of images is based on statistical criteria (histogram analysis); this background extraction condition may not be valid.

x We may notice in a few examples that after the improvement step a few ambiguous regions were identified as background. These regions are extremely bright and even for experts their identification is a difficult task. Various artifacts in TEM images bias the classification of certain segmented regions, e.g. water drops present on the grid appear brighter than the background, stain artifacts may also disturb segmentation and thus our identification. In such cases, foreground regions may be classified as background ones without affecting the overall classification results.

On-Line Integration

Chapter 3 and 4 detailed a group of techniques that refined the automatic mono-membrane region selection and characterization in terms of their size, quantity and density on the grid. Thanks to the multi-resolution analysis improved by the contour evaluation for region merging, membranes are also well delimited and described in terms of stacking-level. In the first chapter, we have introduced that, at medium magnification, the two goals (specimen characterization and ROI selection) could be achieved independently, but it implies a reduced control of the targeted ROIs. To improve the relevance of the selection, the characterization can be used to assist the process of ROI selection: in the tested implementation, large "level 1" membrane regions were chosen as selections for high magnification analysis.

A final prototype has been installed at Biozentrum Maurice E. Muller Institute for Structural Biology in collaboration with our partners from Basel. An on-line integration of the above algorithms, as they are described in chapters 3 and 4, has been realized on a Tecnai 12. An interface between the image processing algorithms and the microscope control has been developed, based on the microscope interface developed by FEI Company while our image analysis software has been implemented in Matlab 7.6.0 (R2008a) version with a computer processor Intel(R) Xeon(R) CPU; W3520 et 2,67 GHz and 3.48 GB of RAM memory and a 1kx1k CCD camera.

Conclusion

We developed original techniques, as presented in this chapter, in order to extract the background of TEM images.

After having segmented the image, the background extraction is effected in two steps. The first allows finding a single background region or the principal region when the image background is split into several parts. The second step concerns the identification of secondary background regions. An original method was therefore developed by using prior knowledge. A quantum was determined based on topological and statistical criteria. This processing step permits to create a representation not in gray-level but in stacking-level. It is now possible to localize supplementary background regions, non-stacked membranes, aggregates, etc. The advantage of this method is to be able to overcome heterogeneous illumination artifacts since images are not globally considered but only locally. The developed method allows the background extraction of a wide TEM image collection in a quasi-exhaustive manner; for a single background piece or divided ones. The foreground and the background of the image are distinguished allowing executing other processing tasks like regions of interest to verify the sample crystalline structure or apply shape recognition algorithms to precisely characterize the nature of objects present in the image.

along the continuity of a contour provides a reliable criterion: this criterion is used within statistical testing for optimum decision performances. To detect false contours, a convolution measure is carried out between neighboring gray-level pixels to the contour and a reference "slope-like" filter. The filter is oriented and applied perpendicularly to the contour. Our validation threshold is relative to the image noise amplitude, and to the number of pixels that contribute to the correlation measure with a certain confidence. It is robust and suitable for all contour segment length.

Contour localization may be a limitation factor that plays a significant role while evaluating contours, as the precision in their localization may bias the criterion reliability.

Our method is capable of detecting and eliminating approximately 95% of false contours. These results are satisfactory as they correspond to a sufficient detection quantity required for our application. These convenient results do not prohibit further adaptations or improvements such as introduction of other filters, to optimize performances. We should however not forget that with such operations we may gain in precision but certainly not in computation time and complexity.

This method finally, could be also extended, with certain adaptations, to other image cases, such as medical imaging, where images present similar characteristics (e.g. heterogeneous contrast) with the ones coming from microscope devices.

The image background can then be properly distinguished from the foreground: an original method dedicated to TEM images is established and detailed in chapter 4. The application of the above criteria (size and intensity) as presented at the beginning of this paragraph provides an estimation of the principal background region. The background does not necessarily form a single and continuous region. The identification of all the background elements is more complex as local contrasts had to be employed. This local processing is effectuated for all regions to assign a label, referred to as stacking level, a notion that links gray-levels and number of stacked membranes. Minor background regions are assigned level 0 and identified as secondary background regions. Furthermore, the application of topological criteria allowed mono-membrane region localization and determine this way the range of contrasts characteristic to stacking level 1 and so on for higher levels.

This method relies on the segmentation results provided by the region merging, and on statistical criteria such as the mean gray-level of each segmented region, their size, and a priori knowledge. Our method is segmentation dependent, and in case of a large background region absence, the identification procedure will be disturbed. In a few examples there may be a few ambiguous regions that were identified as background.

These regions, even for a biologist expert, are extremely difficult to identify. Such artifacts are rare and in this case they are not properly identified. The information present in the image does not allow solving such uncertainties. Further acquisitions of the same region at intermediary magnifications would help. few of the most recent studies - [START_REF] Nasser Esgiar | Fractal analysis in the detection of colonic cancer images[END_REF], Zhang et al. 2003[START_REF] Amaral | Classification of breast-tissue microarray spots using colour and local invariants[END_REF], Bilgin et al. 2007, Sakim et al. 2009 ). Different classifiers are found in the literature and may be categorized according to the model used for the classification: Classification Rules, Decision Trees, Instance Based Learning and Bayesian Classifiers [START_REF] Sierra | On Applying Supervised Classification Techniques in Medicine[END_REF].

In a preliminary study, detailed in the appendix, we show that classic classification techniques were employed as they seemed appropriate to our problematic. We take advantage of expert's knowledge and build a decision scheme. Three principal classes are defined: isolated objects, composed objects and other.

Each one of these classes is sub-divided to a certain number of sub-classes that permit to refine the classification. A primary stage of feature extraction is essential so that rules can be given. With our partner experts in the domain of biology, we realized an extensive study of objects found in representative TEM images. Their shape and stacking level were clearly extracted as main characteristics to be used for their classification. Objects can be found isolated, aggregated or unclassified; for the category of membranes, the distinction is effectuated between isolated or aggregates or piles of vesicles or sheets and tubes.

Supplementary characteristics (elongation, form factor) are derived for artifact classification.

Three categories of features are necessary to perform our object classification:

x A region feature, the stacking-level, allows the distinction between isolated and composed objects.

x Contour features, characterize the nature of contours by analyzing portion of rectilinear (tube or sheet-like) or curvilinear (vesicle-like) portions. In order to detect these portions, we can split the object contour into simple elements by placing "breakpoints" where curvature highly varies (more details in appendix).

x Shape descriptors, like elongation and form factor provide aspect ratios of objects for artifact classification.

The rules of classification are then elaborated with the biologists in a tree decision-like form, and thanks to these rules a classification for each object is supplied. The appendix provides a detailed preliminary study of this problematic with the associated preliminary results. An extended bibliography concerning classification methods is given. Literature regarding contour characterization and "breakpoint" detection is also detailed in order to better understand all aspects of the problem and have a complete overview of the classification method.

Perspectives related to the European project

Extensive tests for on-line integration of the classification algorithm and evaluation of results by the experts are in process. The algorithms have been successfully implemented and tested on the Tecnai T12, at the Biozentrum, in Basel, thanks to the efficient integration and development of an appropriate controlling process. The developed tools allow automatic acquisition of images, and it outputs some hints concerning

A. Motivation

Background-foreground discrimination provides a number of foreground regions that are not characterized in detail yet. Each one of them may be however part of a more global entity in the image: the foreground image entities or simply entities that we want to classify. Foreground segmented regions can correspond in terms of specimen elements either to isolated or stacked clusters.

Biologists while scanning a grid search to interpret its content and extract knowledge regarding elements to evaluate the crystallization results and revise conditions.

Biologists' objectives and needs

In particular, a microscopist while observing a sample in the microscope performs a quantitative and qualitative evaluation synthesis of the sample.

-Qualitative analysis: As already mentioned in chapter 1, optimal crystallization conditions are globally accomplished when crystals diffract. On the other hand, large 2D crystals with as few aggregates as possible present on the grid indicate however a correct and proper crystallization as the right parameters (lipid to protein ratio, temperature, etc.) were met to produce large crystal lattice surfaces. The microscopist therefore searches the existence of mono-membrane regions and their size within the specimen elements spread all over the grid. Moreover, aggregation phenomena are related to the detergent nature and removal method while their presence often implicates proteins that are not stable and start to aggregate before crystallizing. All these observations are very important for the biologist as empirical conclusions may be established in order to adjust crystallization parameters.

Different morphologies may be found as well in the form of vesicles, sheets, narrow tubes, aggregates or piles of sheets and vesicles. Their morphology is related to the type of proteins and lipids or other parameters that are involved into the crystallization procedure. Such examination may help the biologist to better comprehend the nature of proteins themselves and their crystallization "preferences".

-Quantitative evaluation: A numerical evaluation of the above elements is therefore essential to create a general appreciation representative for each grid. The microscopist counts roughly probable crystalline elements and groups of different categories, as referred in the above paragraph, arriving to a final appreciation of the grid quality.

Our Objectives

Our objectives are in accord with biologists' needs for membrane and specimen element characterization. For this purpose, we elaborate techniques in order to:

realized an extensive study of objects found in representative TEM images. Their shape and stacking level were clearly extracted as main characteristics to be used for their classification. Objects can be found isolated, aggregated or unclassified; for the category of membranes, the distinction is effectuated between isolated or aggregates or piles of vesicles or sheets and tubes. Supplementary characteristics (shape descriptors, etc.) are derived for artifact classification.

In the following sections, each one of the two columns of

C. Presentation of Entities Characteristics

Identifying the features and setting a limit between groups in the feature plan can separate them into respective classes. Finding a single parameter that can be successfully used to separate classes is not always possible, and even when it is, selecting the best one may be a fastidious process. In most cases, a combination of parameters is needed for successful classification. The goals in deciding upon the measurement parameters to use are:

-To be able to discriminate the various classes, preferably with minimum overlap -To have the smallest practical number of parameters Sometimes these goals conflict with one another. For example, adding more parameters often provides less overlap of classes, but it may complicate the whole process. Therefore, reliability is difficult to predict when a characteristic extraction scheme is initially established.

Presentation of the features used in our classification

Feature extraction is an important topic in pattern recognition and in general data classification and can be viewed as finding a vector or a set of vectors that represent an observation. They are the criteria to be used in order to discriminate classes of similar properties. Features searched within our images are based on geometrical analysis of image entities' contours and stacking levels of regions. The selection of these features is based on an analysis procedure of different series of membrane objects and they are presented in Table C as intra-region features, contour features and shape descriptors: i)

the notion of stacking level, as introduced in chapter 4, relates the gray level of a region to the number of layers of superposed membranes, ii) the shape of an object (rectilinear or curvilinear), as already mentioned in chapter 1, is a descriptive element of the morphology of an object, iii) elongation as defined in (Russ 2007) is a shape descriptor equal to length to width ratio of an entity, iv) form factor as presented (Russ 2007) is also a shape descriptor computed as the ratio of object area to square object perimeter.

Region Features

The stacking level feature, as presented in chapter 4, consists in attributing a label to each segmented region according to its stacking level. Each region has a single label characteristic to its degree of stacking level. Consequently, many stacking levels may be present within an entity: label "level 0" is attributed to the background regions that may be included in the object, label "level 1" to monomembrane regions until upper-stacked levels ("level 2", "level 3", "level 4"). The dispersion of stacking levels within an entity is represented in a vector containing the percentage of each one of the stacking levels (in pixels) present within the object. This information is used to characterize the entities present in the images and is considered as a feature of a great importance.

For a limited number of cases, where isolated aggregates may be found in an image the gray-level intensity is used as a supplementary feature to validate the stacking level estimation.

Shape Descriptors

Shape is an important parameter to consider in the domain of image processing but often defined in an approximate way (round and linear, smooth and coarse etc.). General "axiomatic" notions are however admitted about the important aspects of shape. On the other hand, finding descriptors of shape is a difficult task because the correspondence between them and our everyday experience is not significant.

Shape analysis is a major and large field that aims at analyzing and processing geometric shapes.

Shape analysis is related to various tasks as statistical analysis of geometric shapes, shape matching and shape recognition.

For shape recognition many classic shape descriptors exist as combinations of size parameters such as ratios of length, width etc. In (Russ 2007) an extended list of such descriptors is presented, where the elongation and form factor are also included and presented below: The form factor is given as a surface to perimeter ratio and the elongation as the length to width ratio.

These parameters are measured as the major and minor diameter of the equivalent rectangle. These descriptors only provide aspect ratio of objects since there are hundreds of ways that size parameters can be combined to provide the same descriptor results.

Extraction of Contour Features

Introductory Literature Other methods consider a different representation of the object, the boundary traced as the exterior contour of the object. Within this representation features may be extracted by searching directly specific features such as straight lines, circular arcs, etc. The most conventional method to be applied was the Hough transformation [START_REF] Duda | Use of Hough transform to detect lines and curves in picture[END_REF]. However, such approaches may be applied only on images where hypotheses of linearity or non-linearity are valid. Otherwise, a number of peaks is accumulated in the transform space where a threshold is difficult to be defined to distinguish not welldefined peaks. Moreover, such approaches suffer from inaccuracy due to computational complexity and storage limitations. Model-based approaches [START_REF] Belongie | Shape Matching and Object Recognition Using Shape Contexts[END_REF][START_REF] Fougerolle | Radial Supershapes for Solid Modeling[END_REF]) cannot be applied to our image cases due to the diversity of membrane shapes and therefore the complexity of models.

Contour Portions Characterization

Once the exterior contour of an entity is processed, as described above, to detect the break points a representation of the contour is obtained where the contour is split into two kinds of pixels:

x Breakpoints

x Contour portions Contour portions are contiguous pixel groups separated by breakpoints; these portions of the exterior contour are those who will be analyzed in terms of linearity or circularity in order to attribute a shape characteristic that will serve to classify our entity. Among these parts, only the longest ones are selected (a threshold length value is introduced as 20 pixels) and by means of a linear regression a fitting to our data is applied. An evaluation error (we took into account a square quadratic error) is then calculated to estimate the fitting degree of the line to the data points. If this estimation error exceeds the value of 3, it is considered as not linear and therefore to simplify the process and according to our hypothesis, it is labelled as curvilinear. Two percentages of rectilinear and curvilinear pixels within the whole exterior contour are then provided. The proportion of rectilinear contours is therefore equal to the percentage of rectilinear pixels in the contour that were detected within the contour and the proportion of curvilinear contours is equal to the percentage of curvilinear pixels.

D. Classification Rules

A set of rules is created and applied to each new set of feature measurements as these are described above. When they are used, the cut off values of the parameters for each class are determined by the expert. The complexity of decision paths increases rapidly with a large number of rules. In the present study, there are 6 initial observations (measurements), 14 rules and 3 possible global conclusions.

Some of the rules shown have more than two possible outcomes.

Decision Rules

The rules used throughout the process for each class are presented below:

x If 'level 1' is dominant within the object, more than 95% of the object surface we search the elongation and proportion of rectilinear and curvilinear contours to decide the class: 'Sheet', 'Vesicle', 'Membrane', 'Tubes'.

x If 'level 1' is less important within the object, less than 95% of the object surface

AEIf 'level 4' is more than 95% of the object surface, then class 'Aggregate' is attributed.

AEif 'level 4' is between 5% and 95% of the object surface, then class '… -Aggregates' is chosen.

-The proportion of rectilinear and curvilinear contours is searched to decide the class: 'Sheets -…', 'Vesicles -…', 'Membranes -…'.

AEif 'level 4' is less than 5% of the object surface and other stacking levels are present then class '… -Stacks' is chosen.

-The proportion of rectilinear and curvilinear contours is searched to decide the class: 'Sheets -…', 'Vesicles -…', 'Membranes -…'.

x Else label 'artifact' is attributed.

AEIf 'level 0' is not present and level 4 is dominant, 'type I' is attributed.

AEIf 'level 0' is dominant, more than 95% of the object surface and curvilinear proportion is dominant, label 'type II' is supplied.

  INTRODUCTION................................................................................................................................................................................................................................................................................................... 1.2 BIOLOGICAL CONTEXT ................................................................................................................................. 1.3 PROTEIN ANALYSIS METHODS...................................................................................................................... 1.3.1 3D Crystallization ................................................................................................................................... 1.3.2 Nuclear Magnetic Resonance................................................................................................................

  3 HT3DEM Project .................................................................................................................................... 1.7.4 Thesis Motivations .................................................................................................................................. 1.8 TOWARDS AUTOMATIC ACQUISITION FOR MICROSCOPE CONTROL -STATE OF THE ART ............................ 1.9 TEM AUTOMATION FOR SPECIMEN ANALYSIS -GENERAL STRATEGY ......................................................... 1.9.1 Analysis of Low Magnification Acquisitions.......................................................................................... 1.9.2 Medium Magnification Analysis............................................................................................................. 1.9.3 High Magnification Analysis .................................................................................................................. 1.10 SPECIMEN CHARACTERIZATION.................................................................................................................... 1.11 CONCLUSION AND CONTRIBUTION OF THE THESIS......................................................................................... ........................................................................................................................................... 2.2 OBJECTIVES .................................................................................................................................................. 2.3 TEM IMAGE CHARACTERISTICS ................................................................................................................... 2.3.1 Contrast ................................................................................................................................................... 2.3.2 Noise ....................................................................................................................................................... 2.3.3 Membrane Regions ................................................................................................................................. 2.3.4 Membrane Borders.................................................................................................................................. 2.3.5 Background ............................................................................................................................................. 2.4 REGION SEGMENTATION............................................................................................................................... 2.4.1 Motivations ............................................................................................................................................. 2.4.2 Coudray's Algorithm............................................................................................................................... 2.5 OVER-SEGMENTATION: INTRODUCTION TO THE PROBLEMATIC.................................................................... 2.5.1 Introductory Literature ............................................................................................................................ 2.5.2 False Contours......................................................................................................................................... 2.6 CONCLUSION ................................................................................................................................................ ........................................................................................................................................... 3.2 PROBLEM STATEMENT.................................................................................................................................. 3.2.1 State of the Art ........................................................................................................................................ 3.3 STATISTICAL CONTOUR EVALUATION: PRESENTATION OF THE ALGORITHM ................................................ 3.3.1 Algorithm Principle and Mathematical Notations................................................................................... 3.3.2 Description of the Algorithm .................................................................................................................. 3.4 RESULTS .......................................................................................................................................................

Figure 1 . 1 A

 11 Figure 1.1 A simple scheme of a protein membrane: proteins are embedded into the lipid membrane and may transport an ion (left) from the exterior to the interior of the cell, convert energy (centre), or transport a molecule from the interior to the exterior of the cell (right) Proteins referred to as membrane proteins are inserted into a lipid membrane. They realize key functions of cells as they fulfill a number of different vital tasks in cellular signaling and metabolism -receptors, structural proteins, etc. They control the movement of substances into and out of cells, accelerate chemical reactions related to the function and energy conversion and create links between cells. When a cell is attacked by a virus, for example, membrane proteins are the "entrance gates" of the cell. Many diseaselinked mutations occur in membrane proteins, a fact that relates them to numerous diseases like hypertension, heart diseases, lung and brain edemas, addictions, depression, and cystic fibrosis. Their study

Figure 1 . 2

 12 Figure 1.2 Different types of grid lattice typically used for TEM screening (copper grid)

Figure 1 . 3

 13 Figure 1.3 Negative stain TEM specimen preparation scheme; a side view of a stained protein crystal that is placed on a carbon film mounted on a copper grid Negative staining may increase membrane contrast but it can present many drawbacks. It is important to note that staining may have different characteristics that can lead to different staining results; specimens may show severe deformations such as flattening or even incomplete stain embedding.

Figure 1 . 4

 14 Figure 1.4 Typical examples of TEM images at medium magnification containing 3 examples of membrane gray-level intensity: protein aggregates, stacked membranes and mono-membrane regions Figure 1.5 shows that membranes can be categorized into three basic families at medium magnification:

Figure 1 . 5

 15 Figure 1.5 Typical examples of TEM images at medium magnification containing 3 types of membranes, from left to right respectively: vesicles, sheets and tubes

Figure 1 . 6

 16 Figure 1.6 The 3D-EM High Throughput Chain designed in the HT3DEM project Figure 1.6 gives an overview of the 3D-EM High Throughput Chain developed in the project.

Figure 1 . 8 Figure 1 . 9

 1819 Figure 1.8 The different classes of squares: column 1: class (A) for undamaged film and visible membranes; column 2: class (C) for damaged film; column 3: class (B) or (D): carbon and small membranes or no carbon at all; line 1: graylevel images; line 2: aspect of the local histogram and segmentation of the last peak; line 3: brightest pixels segmented

  treated. The classes of histograms typical to image classes are: expected histogram, aggregate-like histogram, grid-like histogram and noise-like histogram, as Figure 1.10 illustrates. The images with a histogram of type a) are the only that are worth being analyzed. The histogram is characterized by a main bright peak located at the right-side of the histogram. Category 1 (figure 10.b) consists of images where membranes are stacked and the background is a minority part of the image. Histograms of aggregate-like images show a distribution with a dominant peak on the left side of the histogram. Once this type of histogram is encountered, the image is rejected. Images of category 2 (figure 10.c) are acquired at the borders of the grid squares where the dense mesh grid is dark (grid-like images) and the yielding histogram is characterized by a peak visible at the very left-side of the histogram.

Figure 1 .

 1 Figure 1.10 Examples of type of images and corresponding histogram : a) Interesting image (scale bar: 1 µm), b) Aggregate-like image (scale bar: 1 µm), c) Grid-like image (scale bar: 1 µm), d) Noise-like image (scale bar: 1 µm) Noise-like images of category 3 (figure 10.d) have histograms that present several peaks in the zone of bright gray-levels of the histogram. Many reasons may be responsible for the formation of such a category of images: an important detergent quantity that is not correctly removed during the reconstitution of proteins, numerous artifacts related to grid contamination, carbon film defects that are not detected at low magnification. Such an image histogram denotes an image that is set aside from additional processing. Images of category 4 (figure 10.a), expected images, are those who are searched for processing and study.They contain relatively bright, non-stacked membranes with a large background that covers the major part of the image. The histogram of such images consists of a principal peak placed towards the bright graylevels (on the right side) of the distribution. The usefulness of this pre-classification process is important as only interesting images are processed, accelerating in this way the microscope control, facilitating decisions concerning the image processing algorithms to be introduced later on by reducing the diversity of images to be processed and provide at the same time the information necessary to the biologists for the sample quality.

  Peak identification is made difficult by the ring-shaped noise visible in the reciprocal space. These rings, called the Thon rings[START_REF] Thon | Phase contrast electron microscopy[END_REF], are due to the Contrast Transfer Function (CTF) of the microscope. A simple threshold of the peaks is therefore not applicable. The process proposed to automatically detect the peaks is based on the background removal of the power spectrum image and image thresholding for the peak identification. The resulting image is shown in Figure1.11.

Figure 1 .

 1 Figure 1.11 Image of a crystalline membrane (left; scale bar: 100 nm) with its power spectra (center) and its mean radial profile (inset). Segmented diffraction peaks (right) after background correction of the power spectra

Figure 2 . 1

 21 Figure 2.1 Illustration of different image contrasts: gray-level intensity for different regions that correspond to a single membrane thickness (left), superposed membranes (centre) and aggregates (right)

  For images containing more mono-membrane regions the average SNR can reach values inferior to 2 where such values are considered as bad SNR; for most image processing techniques used the SNR must be greater than 5. This measure does not reveal the real nature of our images and furthermore of our regions of interest as only mono-membrane regions interest us; local SNRs can be inferior to 1 as for the mono-membrane region 2 of Figure2.2.

Figure 2 . 2

 22 Figure 2.2 A densely 'membrane-populated' TEM image with a variety of membrane contrasts. Mono-membrane regions, like the membranes pointed at by green arrows

V 2 mVFigure 2 . 3

 223 Figure 2.3 Electron microscope images with variable gray-level intensity and dark regions on the left-hand side; and corresponding scatter plots on the right-hand side

Figure 2 . 4

 24 Figure 2.4 Electron microscope images; red encircled zones represent specimen elements and blue encircled regions are mono-membrane regions that correspond to a single membrane thickness

Figure 2 . 5 Figure 2 . 6

 2526 Figure 2.5 Electron microscope image with green squares representing the membrane and background blocks used for the CNR estimation

Figure 2 . 7

 27 Figure 2.7 Example of low contrasted membranes in a TEM medium magnification image, and histograms of monomembrane regions and of the whole image

Figure 2 .

 2 Figure 2.8 A collection of 1D profiles of the membrane surrounded by the red dotted circle

Figure 2 . 9

 29 Figure 2.9 Shading effect (normalized image) of a typical TEM image

Figure 2 .

 2 Figure 2.10 Example of a TEM image on the left and its segmentation in black on the right hand side

Figure 2 .

 2 Figure 2.12 Upper left: Initial TEM image; Upper right: Segmentation (contours in magenta) of the image, Lowcenter: Plot of mean gray-level values of segmented regions, blue lines: membrane regions, red horizontal line: background region

Figure 2 .

 2 Figure 2.13 Left: Initial TEM image, Center: Zoom of the white window of the initial image, Right: 3D representation of the zoomed window (colors correspond to gray-level amplitudes)

Figure 2 .Figure 2 .

 22 Figure 2.14 Left: Initial TEM image, Right: Corresponding segmentation with ridge-like contour indicated by black arrow

Figure 2 .

 2 Figure 2.16 Left: Initial TEM image, Right: Corresponding segmentation with Trigger gradient contour indicated by black arrow

  [START_REF] Haris | Hybrid image segmentation using watersheds and fast region merging[END_REF] utilized a more sophisticated version of the previous criterion by measuring the weighted square contrast of two neighboring regions.[START_REF] Deka | Watershed Segmentation for Medical Ultrasound Images[END_REF] proposed a similarity criterion based on the area of segmented regions.[START_REF] Davis | Background-substraction using contour-based fusion of thermal and visible imagery[END_REF] used statistical approaches such as the similarity test of the Student's t-test to determine whether the pixels of two adjacent regions are similar or not.[START_REF] Hamarneh | Watershed segmentation using prior shape and appearance knowledge[END_REF] utilized prior shape and appearance knowledge in order to bring together the contour segments that correspond to a specific organ in medical images. A joint region merging criterion of region homogeneity (based on the contrast measure of[START_REF] Haris | Hybrid image segmentation using watersheds and fast region merging[END_REF]) and edge integrity was proposed by[START_REF] Hernandez | Joint region merging criteria for watershed-based image segmentation[END_REF]. Edge integrity was measured as the ratio of the number of strong adjacent pixels (pixels with gradient value larger than a threshold) within the boundary pixels. By combining region homogeneity (based on the contrast measure of[START_REF] Haris | Hybrid image segmentation using watersheds and fast region merging[END_REF]) and edge integrity,[START_REF] Tan | A novel merging criterion incorporating boundary smoothness and region homogeneity for image segmentation[END_REF] accomplished region fusion for a more visually appropriate segmentation. The measure of edge integrity was inspired from the prior knowledge of smoothed object contours.

(

  Figure 3.1) in the binary image of contours. More specifically, we employ a linear regression in the least square sense to obtain the slope that provides the local orientation of the contour:

Figure 3 . 1

 31 Figure 3.1 Estimation of local (red window) orientation of pixel of interest (in green) within the binary image of contours

3. 2 )

 2 . The best profile length was established after a systematic study of the profiles in several electron microscope images. Different profile lengths were evaluated yielding a meaningful length for this 1D profile equal to 21 pixels. This value seemed a good compromise against various constraints such as region size (e.g. profile length cannot exceed region/regions surface).

Figure 3 . 2

 32 Figure 3.2 Mask centered on the pixel of interest and applied perpendicularly to the contour segment in order to extract the 1D profile

Figure 3 . 3

 33 Figure 3.3 First row: Two characteristic regions are chosen in TEM image, red window contains a membrane Second row-Left: Zoom of the red window with a mono-membrane region, Right: Zoom of the green window of with background between two membrane parts, Third row-Left: Pixel hypothesis test (1x21) -white plain dots: FCP, black squares: TCP, Right: Pixel hypothesis test (7x21) -white plain dots: FCP, black squares: TCP, with black arrows indicating false contour pixels wrongly classified

Figure 3 . 5

 35 Figure 3.5 Left: Contour segment classification of a TEM imagecontour segments that are TCS are represented in black and contour segments that are FCS in white, Right: Iterative elimination of FCS giving the final configuration Figure 3.5 illustrates an example of such a configuration. Only the 'strongest' FCSs of all were kept in order to maintain acceptable region topologies. Figure 3.6 shows 4 images that represent the iterations while segments are examined and eliminated: in particular only the red segments are eliminated, beginning with the weakest one. Thus, the red segment in the 'first iteration' image of Figure 3.6 has the weakest / C N value; it is eliminated without introducing biased topologies. While elimination occurs, new segments are raised in the segmentation configuration. These ones are attributed a new ratio value as the average of the sub-segments that belong to the new final one. Iterations are continued until no further need for segment elimination exists.

Figure 3 . 6

 36 Figure 3.6 Iterations illustrating the false contour elimination of the example used in the above figure, ratio values from the left to the right: 0.64, 1.06, 1.65, and 2.68.

Figure 3 . 7

 37 Figure 3.7 First left column: Initial images, Second column: Multi-resolution segmentation, Third column: Detection of false contours (white segments), Last column: Final image segmentation after having them eliminated

Figure 3 . 8

 38 Figure 3.8 From left to right: Initial synthetic image, Multi-resolution segmentation, Detection of false contours (white segments), Final image segmentation after having them eliminated

Figure 3 . 9 Figure 3 .

 393 Figure 3.9 Reference region of the background Final segmentation results obtained by the global contrast method with different thresholds 110, 120 and 130 respectively are presented in Figure 3.10. This visualization illustrates the difficulty to set a threshold and how the different thresholds affect the merging results. According to the expert, the background is clearly over-segmented and even membrane zones are not properly delimitated.

Figure 3 .

 3 Figure 3.11 a) Region merging by means of global gray-level contrast, b) Region merging using the statistical approach, c) Evaluation of background over-segmentation of a, over-segmented regions are represented in yellow, cyan, green and blue, d) Evaluation of background over-segmentation of b, where no over-segmentation is present

Figure 4

 4 segmentation into regions of a typical example of a TEM image (coded in 16 bit and with gray-level range 1500-6000). The diagram below shows a histogram of average gray-level values of segmented regions of this TEM image. The red vertical dotted line indicates the average intensity of the background as chosen by the expert and as shown in Figure4.1; the expert therefore selected segmented regions that represent the background of the image (in white). It is clearly seen and demonstrated in the same figure that a threshold cannot be reliably set. More elaborated criteria are needed to be combined for a clear distinction between background and foreground. Inevitably, the recognition problem is considered differently: gradual steps for foreground-background distinction will be introduced beginning from extracting the background.

Figure 4 . 1

 41 Figure 4.1 First Row: The original TEM image, Second Row: its segmentation into regions (contours in blue) and the background (in white) provided by the expert, Third row: histogram of average gray-level intensities of regions and the average gray-level of the background region (red doted line)

Figure 4 . 2 ,

 42 however, shows a representative gray-level histogram of a TEM image, where our classes cannot be visualized. We cannot therefore determine the relevant thresholds of gray-level intervals that are characteristic of the background or foreground objects.

Figure 4 . 2 A

 42 Figure 4.2 A typical histogram of a TEM image; this histogram is unimodal where no classes are identifiable TEM image histograms have a unimodal distribution disturbed by the gray-level fluctuations of the background and the objects (as already introduced in chapter 2, section 2.2.5) due to heterogeneous

  chapter 1, the histogram-based pre-classification of images section presented an image pre-classification technique that rejects medium magnification TEM images that are not worth being treated. Only images with an "expected histogram" are therefore analyzed; Figure 4.3 shows an expected histogram characterized by a main bright peak and the equivalent TEM image.

Figure 4 . 3

 43 Figure 4.3 An eligible TEM image with its histogram and red pixels corresponding to peak histogram thresholding The nature of the specimen and the acquisition conditions of our images ensure a bright background; this property is shown by the position of the main peak on the histogram of Figure 4.3. Moreover, the histogram's main peak amplitude reflects the existence of a high number of pixels that are part of the background of the image. Pixels in red in the above figure are derived by thresholding the histogram peak and therefore demonstrate the validity of our intuitive assumption for a large background.

Figure 4 .

 4 Figure 4.4 shows such an extraction for the initial image shown in Figure 4.4a. Figure 4.4b shows the segmentation after having merged the regions by iteratively eliminating the false contour segments.Figure

Figure 4 .Figure 4 .

 44 Figure 4.4 a) Initial image, b) Final segmentation after elimination of false segments, c) Background extraction (in white), d) Background (in white) with segmented membrane regions (in black) The efficiency of the proposed method for the background extraction has been systematically assessed on various series of images acquired with different TEM taken under standard acquisition conditions of illumination and exposure time. These images contain membranes of different types (such as sheet-like membranes or vesicles) and sizes where our algorithm was able to extract the image background satisfactorily.

Figure 4 . 5

 45 Figure 4.5 Upper left: Initial image, Upper right: Background extraction in black, Lower left: Regions in white represent the local minima for quantum determination, Lower right: Selected regions in white

Figure 4 .

 4 Figure 4.6 is a typical example where graduated shading is visible. The bottom left corner of the image is clearly brighter than the upper right corner. Moreover, membrane regions suffer from intensity inhomogeneity. Globally, the images are often corrupted by spurious intensity variations. These imperfections are due to various reasons: the image formation process, negative staining artifacts, carbon film support.

Figure 4 . 6

 46 Figure 4.6 Left: Initial Image, Right: Membrane stacking where shading drawbacks affect the stacking representation

Figure 4 . 8

 48 Figure 4.8 From left to right: Initial Image, Multi-resolution segmentation, Detection of false contours (red segments),Final image segmentation after having them eliminated, Labeled image; black: background, red: non-stacked membrane regions, green: two-layer membrane regions, magenta: three-layer membrane regions, blue: more than three layers

Figure 4 . 9 Figure 4 .

 494 Figure 4.9 Examples of TEM images and their main background region extraction (in black)

  Figure P will be presented in detail. Object feature extraction will be extensively detailed in the sections below. Then the classification rules will be given in combination with the classes defined by the expert accompanied by results detailed in the last section of the appendix.

Figure P .

 P Figure P. General scheme used for foreground entities classification

Figure R .

 R Figure R. Evolution of scales to the contour smoothing effect for Ʊ 8, 12, 16, 20, 24, 28 respectively

Figure T .

 T Figure T. Examples of occluding contour of entities break points in red and contour portions to be analyzed in yellow

Figure V .

 V Figure V.According to the colour code given above objects are identified as sheet (100% level "1"), membranes-stacks (86% level "1") and sheets-stacks (10% level "1")

  .. 1.3.3 Single Particle Analysis .......................................................................................................................... 1.3.4 2D Crystallization ................................................................................................................................... 1.4 2D CRYSTALLIZATION METHOD.................................................................................................................... 1.4.1 2D Crystallization Procedure and Parameters ......................................................................................... 1.4.2 2D Samples Used in the Current Application ......................................................................................... 1.5 DETERMINATION OF THE CRYSTALLIZATION PARAMETERS.......................................................................... 1.5.1 Grid Preparation ......................................................................................................................................

1.5.2 Negative Staining .................................................................................................................................... 1.5.3 Assessment of 2D Crystals...................................................................................................................... 1.6 CRYO-ELECTRON MICROSCOPY AND 3D RECONSTRUCTION ........................................................................ 1.7 OBJECTIVES OF THE EUROPEAN PROJECT HT3DEM..................................................................................... 1.7.1 Motivations for the Automation of a Protein Analysis Pipeline.............................................................. 1.7.2 Motivations for Automatic Image Analysis ............................................................................................ 1.7.

Table of Contents

 of 1.1 INTRODUCTION ............................................................................................................................................. 1.2 BIOLOGICAL CONTEXT ................................................................................................................................. 1.3 PROTEIN ANALYSIS METHODS...................................................................................................................... 1.3.1 3D Crystallization ................................................................................................................................... 1.3.2 Nuclear Magnetic Resonance.................................................................................................................. 1.3.3 Single Particle Analysis ..........................................................................................................................

1.3.4 2D Crystallization ................................................................................................................................... 1.4 2D CRYSTALLIZATION METHOD.................................................................................................................... 1.4.1 2D Crystallization Procedure and Parameters ......................................................................................... 1.4.2 2D Samples Used in the Current Application ......................................................................................... 1.5 DETERMINATION OF THE CRYSTALLIZATION PARAMETERS.......................................................................... 1.5.1 Grid Preparation ...................................................................................................................................... 1.5.2 Negative Staining .................................................................................................................................... 1.5.3 Assessment of 2D Crystals...................................................................................................................... 1.6 CRYO-ELECTRON MICROSCOPY AND 3D RECONSTRUCTION ........................................................................ 1.7 OBJECTIVES OF THE EUROPEAN PROJECT HT3DEM..................................................................................... 1.7.1 Motivations for the Automation of a Protein Analysis Pipeline.............................................................. 1.7.2 Motivations for Automatic Image Analysis ............................................................................................ 1.7.3 HT3DEM Project .................................................................................................................................... 1.7.4 Thesis Motivations .................................................................................................................................. 1.8 TOWARDS AUTOMATIC ACQUISITION FOR MICROSCOPE CONTROL -STATE OF THE ART ............................ 1.9 TEM AUTOMATION FOR SPECIMEN ANALYSIS -GENERAL STRATEGY ......................................................... 1.9.1 Analysis of Low Magnification Acquisitions.......................................................................................... 1.9.2 Medium Magnification Analysis............................................................................................................. 1.9.3 High Magnification Analysis .................................................................................................................. 1.10 SPECIMEN CHARACTERIZATION.................................................................................................................... 1.11 CONCLUSION AND CONTRIBUTION OF THE THESIS.........................................................................................

  These arrays of periodically arranged proteins are called 2D crystals. Finding specific conditions where membrane protein crystallization occurs is long and tedious work requiring extensive screenings.

	Detergent-solubilized proteins are first mixed with a suitable lipid in the presence of detergent. The
	detergent is then progressively removed; protein hydrophobic zones become 'available' to be attached to
	lipids that surround proteins. The lipids rebuild a membrane in which the proteins find their natural
	formation again. When the lipid-protein structures occur in a regular array, it is called a 2D crystal. Critical
	parameters are: protein concentration, lipid nature, Lipid to Protein Ratio (LPR), pH (acidity of the
	solution), temperature, detergent nature, reconstitution kinetic and method of detergent removal. If detergent
	removal is rapid, proteins may be attracted to each other and instead of being nicely arranged in a flat lipidic
	membrane, they aggregate, yielding bulky structures. There are different detergent removal methods:

1.5.2 Negative Staining

	Biological objects are thin specimens with a low electron-scattering power because of their chemical
	composition. Electrons are therefore absorbed or scattered very little by biological material and contrast in
	electron microscopy is obtained mainly by electron scattering. To obtain a sufficient contrast, samples once
	deposited on the grid are stained. Stains are usually compounds of heavy metals of high atomic number that
	serve to scatter the electrons. A variety of electron dense materials is available for negative staining; the
	most commonly used stains are uranyl acetate, uranyl formate, sodium/potassium phosphotungstate,
	ammonium molybdate. Uranyl acetate is presently employed. Periodic organization of proteins-lipids within
	the membrane cannot be verified at high magnification unless they are stained by heavy mass atoms. To

  Table presenting the three magnification phases for the automatic microscope control

		Low Magnification	Medium Magnification	High Magnification
	Name	Grid map	Square view	ROI atomic view
	Image Example
	Magnification	~50	~3000	~50,000
	Pixel size	~1.5 µm / pixel	~10 nm / pixel	~0.5 nm / pixel
	Field of view	~1.5 mm / side	~10 µm / side	~0.5 µm / side
	Assessment objectives	Evaluate the grid quality	Membrane Characterization	Identify crystalline ROI
	Information for the next TEM acquisition stage	Target the non-damaged grid squares	Target Regions Of crystalline assessment Interest (ROI) for	none
	Figure 1.7

Table of Contents

 of 2.1 INTRODUCTION ............................................................................................................................................. 2.2 OBJECTIVES .................................................................................................................................................. 2.3 TEM IMAGE CHARACTERISTICS ................................................................................................................... 2.3.1 Contrast ................................................................................................................................................... 2.3.2 Noise ....................................................................................................................................................... 2.3.3 Membrane Regions .................................................................................................................................

2.3.4 Membrane Borders.................................................................................................................................. 2.3.5 Background ............................................................................................................................................. 2.4 REGION SEGMENTATION............................................................................................................................... 2.4.1 Motivations ............................................................................................................................................. 2.4.2 Coudray's Algorithm............................................................................................................................... 2.5 OVER-SEGMENTATION: INTRODUCTION TO THE PROBLEMATIC.................................................................... 2.5.1 Introductory Literature ............................................................................................................................ 2.5.2 False Contours......................................................................................................................................... 2.6 CONCLUSION ................................................................................................................................................

Table of Contents

 of 3.1 INTRODUCTION ............................................................................................................................................. 3.2 PROBLEM STATEMENT.................................................................................................................................. 3.2.1 State of the Art ........................................................................................................................................ 3.3 STATISTICAL CONTOUR EVALUATION: PRESENTATION OF THE ALGORITHM ................................................ 3.3.1 Algorithm Principle and Mathematical Notations................................................................................... 3.3.2 Description of the Algorithm .................................................................................................................. 3.4 RESULTS ....................................................................................................................................................... 3.4.1 Qualitative Analysis ................................................................................................................................ 3.4.2 Quantitative Analysis .............................................................................................................................. 3.4.3 Results Comparison with Classic Merging Method ................................................................................ 3.5 CONCLUSION ................................................................................................................................................

Table 3 .

 3

1 A summary of results throughout a statistical test

Table 3 . 2

 32 Percentages of FCP and TCP edge pixels within the contour segments of green window classification of Figure3.3

Table 3

 3 Table3.1 shows a summary performance analysis of the algorithm; for this purpose we applied an additive noise of different standard deviations to our synthetic image varying from 7 to 15, corresponding to a series of CNRs. For each of the CNR series, 100 examples with random noise were employed.

	ı noise CNR	4 1	7 0.60	8 0.50	9 0.45	10 0.40	12 0.30	15 0.25
	FCS detection	100% 94.4% 94.3% 93.1% 93.1% 93% 92.7%
	FCS elimination	100% 94.4% 94%	93%	92%	92%	92%

.3 Performance analysis of the algorithm

Table 4

 4 .1 show that the proportion of misclassification is non-null but very low. It shows that the background in 88% of the images has been well detected. The second class contains 8% of the images while the case of a background misclassification happens in 4% of the images. On the other hand, foreground objects are globally properly classified as such.

		PRINCIPAL	TOTAL
	CLASSES	BACKGROUND	BACKGROUND
		EXTRACTION	EXTRACTION
	WELL DETECTED BACKGROUND	88%	95%
	PARTIALLY DETECTED BACKGROUND	8%	5%
	BACKGROUND MISCLASSIFICATION	4%	0%

Table 4 . 1

 41 Quantitative performance measures of the background extraction algorithm for 45 representative images

Table C .

 C Features used in our classification algorithm

	Category	Features	Feature Vector Size
	Region Features	Stacking Level Dispersion	5
	Shape Descriptors	Elongation Form Factor	1 1
	Contour Features	Proportion of Rectilinear Contours Proportion of Curvilinear Contours	1 1

Figure 4.11 acquisition 1

Figure 4.12 acquisition 2

Figure 4.13 acquisition 3

Figure 4.14 acquisition 4

Figure 4.15 acquisition 5

Acknowledgments

This research project would not have been possible without the support and encouragement of many people as well as the financial support of the European Union (6 th framework Program, HT3DEM, LSHG-CT-2005-018811).

I wish to express my gratitude to my supervisor, Pr. Jean-Philippe Urban for his trust on me to carry out this research work. I acknowledge my advisors, Dr. Jean-Luc Buessler for his assistance and guidance, and Dr.

Edge Segment Elimination

To obtain a final segmentation, a false boundary elimination step is needed. There exist two main techniques for segment elimination: i) elimination by setting a threshold value and keeping only those that exceed this threshold value ii) iterative elimination by eliminating only one segment at each iteration. The first solution is very fast but its inconvenience is that topological coherence of regions is not guaranteed: we cannot remove a segment contour arbitrarily without first checking whether biased topologies are introduced, e.g. regions have a meaning if surrounded by edge segments that form closed contours.

In order to maintain topological coherence, regardless of the computation time, we chose to eliminate segments iteratively with the help of an adjacency matrix like other authors (Haris et al. 1998, Pavlidis and[START_REF] Pavlidis | Integrating region growing and edge detection[END_REF]. Each edge segment is qualified as true or false one according to its C(u,v) value. Problems arise when a region has more than one contour segment that is detected as FCS. In such a case, only an iterative elimination ensures meaningful regions. They are eliminated by finding for each region the segments the most appropriate to keep. That simply means that at each merging step, the weakest segment S ij is eliminated for region R i and R j if:

Conclusion

Over-segmentation reduction is a classical step in the domain of image processing and a major preliminary step before identifying regions. To detect false contour segments, we measure the correlation between graylevels of neighboring pixels to the contour and a reference signal in a slope-like form perpendicularly to the segment. The correlation measure is defined locally for each contour pixel and is averaged over the set of pixels of the considered segment. An empirical evaluation of the noise standard deviation in the image allows defining a validation threshold relative to a confidence coefficient. This method takes into account the number of pixels contributing to the correlation estimation and proves to be sufficient for long segments.

Smaller contour segments are validated when they are correlated to an important local contrast. Evaluated segments are then split into two categories: classified as True Contour Segments and as False Contour Segments. Suppression of FCSs is equivalent to the fusion of two adjacent regions; their elimination is therefore conditioned to an iterative test to preserve topological coherence. We verified the capability of the algorithm to recognize real contours from false ones. Our technique extracts and suitably selects the important contours, by finally eliminating efficiently spurious ones.

Chapter 4

Background-Foreground Distinction

Table of Contents

Quantum Determination

According to the hypotheses as detailed above, the quantum is computed in three steps:

Local minima search: mono-membrane regions are the brightest regions except for the background. In an image where the background is detected and ignored, the local minima are searched. This search is done in the average gray-level image. GR n is the gray-level of the considered region n. The local minima found are potentially mono-membrane regions according to our second hypothesis. Figure 4.5 shows these first-step results. In white, we can see all the local minima of the image.

Region selection for the quantum calculation: all local minima do not correspond to mono-membrane regions. Among the detected local minima, only those that are neighbors to the background are selected; as suggested in our third hypothesis. We can notice that multi-stacked regions could be present. Statistically, these selection errors are minor since they influence only the quantum calculation without affecting the final image classification. Figure 4.5 shows such a selection in white.

Quantum calculation: Quantum Q L1 is computed according to equation 4.1 and is defined as the weighted average of the differences between the region average gray-level GR n , and the average gray-level of the principal background region GR f . This weighting factor has the purpose to give more importance to large regions. )

with N the number of considered regions, and t rn the n-region size.

Once the quantum is calculated, a gray-level range can be defined. All regions with a mean gray-level included within this range are labeled as "level 1" regions and classified as such. Gray-level upper limit is defined as 150% of quantum Q 1 and the lower one as 20% of the quantum. These thresholds are only an initial experimental choice; future work and experiments on the prototype will help us to define more robust thresholds.

This second limit allows us to detect secondary background regions to complete the background extraction scheme.

Quantum Determination Improvement

Our algorithm provides a satisfactory detection of the background using the stacking representation in TEM The background is therefore considered locally. In equation 4.1, we replace the term GR f , the mean graylevel of the whole background, by GR fn . GR fn defines the mean gray-level of a 50x50 pixels area situated nearest to the considered region n (minimize Euclidean distance). In this case, the difference Crn = GR n -GR fn corresponds to the local contrast. To reduce intra-region inhomogeneity, a 50x50 area region is used when possible, e.g. when this area is large enough. Figure 4.7 shows some regions and associated background area boxes used for the quantum calculation. The previous figure was processed with the local approach and it can be noticed that the classification of regions is now corrected; membrane regions that were previously identified as background are now properly identified.

Multi-Level Stacking

A robust technique was elaborated for detecting mono-membrane regions and is extended for multi-level representation of membranes. For higher stacking-levels the quantum, as defined before for a "level 1" membrane, cannot be simply used. The main reason is that the gray-level of a "level 2" membrane is not necessarily twice the gray-level of a "level 1" one, and so on. Observations confirmed that the relation Twenty automatic experiments have been successfully processed, with automated grid loading and unloading. The process successfully identifies some crystalline regions on the three grids that enclosed crystals; Figure 4.11,Figure 4.12,Figure 4.13,Figure 4.14,Figure 4.15 show such selections where the process was fully automatic. The figures display: contour evaluation (false contours in red and true ones in blue) stacking-level representation, and automatic selections (red circle) of 2 of the largest "level 1" regions (only regions with an at least 500 nm average diameter were accepted). Other selections however pointed to non-crystalline regions; these selections still remain interesting for assessing the crystallization experiment when the biologist browses the registered images afterwards. In terms of time, the medium magnification phase runs for approximately 20 s as shown in the table below. The image processing algorithms takes only 10 s on average, depending on the image type, which is quite efficient, considering the complexity of the images and their large size.

Tasks Time

Medium Magnification Analysis

Include:

-stage displacement to selected squares -image acquisition -membrane targeting algorithms ~20 s / image The first set of tests realized with typical samples led to satisfactory results. Biologists will provide in the future a larger variety of specimen samples in order to test the robustness of our algorithms. A second set of tests is currently underway to carry on intensive in situ evaluation of the algorithms. Results from these tests are to be validated by the biologists to eventually enhance our technique. Once the whole process is validated, optimization of the algorithms will further reduce the execution time.

Conclusion and Perspectives

The image analysis techniques presented in this thesis have been developed in the context of a European project realizing an automatic membrane protein crystallization platform. In this project, a high volume of samples is simultaneously produced and then assessed by transmission electron microscopy. Automatic screening involves an on-line analysis of the acquired images, to ensure the microscope control by selecting the regions that will be observed at high magnification and to characterize the specimen.

The image analysis algorithms elaborated in this manuscript were therefore adapted to this particular context. The crystallization steps and sample preparation steps induce specific components that can be sorted in a limited number of categories. However, these objects present only a few characteristic features, appearing in an extremely noisy context with gray-level fluctuations. The artificial membranes, which represent the most important elements to identify, are transparent to electrons yielding thus to a weak contrast.

We have elaborated a number of different techniques for analysis and interpretation of electron microscope images of biological specimen at medium magnification. These methods assure fully-automatic acquisitions by providing a number of regions where the microscope zooms; regions are provided and selected by means of a processing chain that splits the image into regions and identifies those to be analyzed at higher magnification.

Thesis contribution

The significance of object identification in TEM images is essential for further characterization. Separating objects from the image background is therefore necessary. However, objects appear within a great diversity and do not have the required properties for such recognition. On the other hand, the background is globally presented large and bright; common criteria for a large quantity of images. This a priori information regarding the background aspect gave us the inspiration to build a framework, with segmented regions as a starting point; these regions are provided by a segmentation dedicated to TEM images. The segmentation strategy has as objective to detect a maximum of contours, to ensure even low-contrasted edge detection that correspond to mono-membrane regions. As a consequence, the multi-resolution algorithm will place contours to all the gradients of the image corresponding to transitions or not. Criteria application presumes a proper segmentation of the background region, fact that is generally not true.

Chapter 3 details the method elaborated in order to resolve the problem of the background oversegmentation. An original technique for region fusion is established independent from region comparisons.

Such comparisons are not applicable in our images as region criteria are not sufficient. A local gradient

The interest of the stacking level representation is therefore multiple: it allows detecting secondary background regions, tagging mono-membrane regions for their further characterization and providing an important feature for image object classification.

Perspectives

Throughout this thesis many problems were raised and a certain number of solutions were proposed and applied to our problematic. However, there still remain aspects of the work not completed opening up the way to several perspectives. These perspectives may concern: i) the classification of foreground image objects ii) intensive on-line tests of the acquisition chain procedure. These two main axes of perspectives are discussed more in detail below.

Foreground Image Object Classification

A microscopist while observing a sample in the microscope provides an evaluation synthesis of the sample.

Even though analysis at high magnification verifies the crystalline state of a membrane crystal, other elements may contribute to the assessment of 2D-crystallization experiments. Large 2D crystals with as less aggregates as possible on the grid indicate a correct and proper crystallization. The microscopist practically always searches for the existence of mono-membrane regions before going to higher magnification, and observes the protein aggregation on the grid as it implies a crystal malformation. Indeed, aggregation phenomena are related to the detergent nature and removal method while their presence often implicates proteins that are not stable and start to aggregate before crystallizing. All these observations are very important for the biologist as empirical conclusions may be established in order to adjust crystallization parameters. Different morphologies may be found in the form of vesicles, sheets, narrow tubes, aggregates or piles of sheets and vesicles. Their morphology is related to the type of proteins and lipids or other parameters that are involved into the crystallization procedure. Such examination may help the biologist to better comprehend the nature of proteins themselves and their crystallization "preferences".

The classification in sheet, vesicle and membrane classes constitutes an important validation step in the 2D crystallization process demanding elaborated techniques. We thus elaborated a whole chain process to assure the best conditions for the object classification. Our objectives are in accord with biologists' needs for membrane and specimen element characterization. For this purpose, we elaborate techniques in order to:

i) Relate image regions to world objects: by classifying objects according to their shape (sheets, vesicles, tubes, etc.) and stacking level (mono-membrane regions, aggregates, piles).

ii) Statistical information concerning the specimen such as type, size, quantity of specimen elements, aggregate surface percentage or mono-membrane surface percentage.

Many authors have dealt with the classification problem in various domains such as the domain of microscopy for cell classification [START_REF] Boland | Automated recognition of patterns characteristic of subcellular structures in fluorescence microscopy images[END_REF][START_REF] Boland | A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscopic images of HeLa cells[END_REF][START_REF] Minamikawa-Tachino | High-throughput classification of images of cells transfected with cDNA clones[END_REF][START_REF] Lezoray | Automatic segmentation and classification of cells from Broncho Alveolar Lavage[END_REF] or in medicine by classifying cells for automated disease diagnosis -we indicate only a the quality of the 2D-crystallization. During the next months following this project, the MIPS laboratory will go on with intensive tests to study and improve the relevance of the selections.

If some of the characterization tools are more specific to 2D-crystals, we note that the skeleton of the software, used to select targets and trigger new image acquisitions, is broad enough to be extended to the automatic image acquisition of other types of samples.

Appendix

Introduction

Chapter 4 detailed the distinction between background and foreground image regions. This discrimination provides a number of foreground regions that must be characterized precisely in this last step of the chain of analysis. Most objects present on the grid correspond to several segmented image regions, or even a large number of regions. Membranes may be found isolated or within groups of membrane aggregates or piles producing a large diversity of elements; thus we propose that the entity to be classified is a set of contiguous foreground regions. The correspondence between such entities and specimen elements is therefore searched presently and provided by classification methods informing at the same time the biologists about the type, size, quantity and other statistical information of the sample. The classification is carried out by applying decision rules that have been elaborated by asking an expert to explain his reasoning during a manual labeling session.

In this appendix, classification techniques will be reviewed and aspects of the problem will be considered for the choice of the classification method and the criteria used. Each foreground entity is analyzed and characteristics are extracted and used as input data for their classification based on rules developed in collaboration with biologists. These characteristics at the same time provide complementary information such as size, quantity and other statistical information that are gathered in the results database.

x Relate image regions to world objects: by classifying entities according to their shape (sheets, vesicles, tubes etc.) and stacking level (mono-membrane regions, aggregates, piles).

According to these characteristics, a classification method is employed to classify image entities. A vector of characteristics is therefore extracted for each object, derived with the help of biologists. In this way, an equivalent analysis to the qualitative one of an expert is developed.

x Statistical information concerning the specimen such as type, size, quantity of specimen elements, aggregate surface percentage or mono-membrane surface percentage. In terms of the HT3DEM project, a synthesis of the above statistics is saved into the database developed by WP6 (chapter 1, section 1.5). This database is created to assemble and present this statistical information that is extracted for each grid in combination with associated crystallization conditions, staining factors and acquisition conditions. Image analysis algorithms should therefore provide statistical information concerning the grid content.

B. Problematic -State of the art

Classification is concerned with establishing criteria that can be used to identify or distinguish different populations of objects that appear in images. It is a very well-known domain of computer vision that aims at categorizing data according to criteria that vary in conception and form:

x Example images of representatives of each class.

x Numeric parameters for measurement.

x Key features-descriptors.

Many authors have dealt with the classification problem in various domains such as the domain of microscopy for cell classification [START_REF] Boland | Automated recognition of patterns characteristic of subcellular structures in fluorescence microscopy images[END_REF][START_REF] Boland | A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscopic images of HeLa cells[END_REF][START_REF] Minamikawa-Tachino | High-throughput classification of images of cells transfected with cDNA clones[END_REF][START_REF] Lezoray | Automatic segmentation and classification of cells from Broncho Alveolar Lavage[END_REF] or in medicine by classifying cells for automated disease diagnosis -we indicate only a few of the most recent studies - [START_REF] Nasser Esgiar | Fractal analysis in the detection of colonic cancer images[END_REF][START_REF] Amaral | Classification of breast-tissue microarray spots using colour and local invariants[END_REF]. Different classifiers are found in the literature and may be categorized according to the model used for the classification: Classification Rules, Decision Trees [START_REF] Breiman | Classification and Regression Trees[END_REF], Instance Based Learning and Bayesian Classifiers [START_REF] Sierra | On Applying Supervised Classification Techniques in Medicine[END_REF]. Statistical methods are based on statistical characterization of data. Such statistical tools are Neural Networks [START_REF] Kung | Decision-based neural networks with signal/image classification applications[END_REF], Chappelle et al. 1999) as they build behaviour models starting from a collection of examples. The neural net, will, through a "learning" process, become a model for the characteristics and the class/behaviour to be assigned.

Classic classification techniques were employed as they seemed more appropriate to our problematic.

We take advantage of expert's knowledge and build a decision scheme. A primary stage of feature extraction is essential so that rules can be given. With our partner experts of the domain of biology Most of the approaches examine the exterior contour or otherwise occluding contour by extracting a different kind of information as the information hidden behind is much more interesting. It was shown (Koendrick 1984) that the exterior contour bears important information about 3D shape. Attneave (1954) showed while extracting "breakpoints" and connecting related points by straight lines that a global outline of the image object is generated and recognized by humans. He observed that the information found at extrema of contour curvature where the amplitude of curvature is maximum is of a great deal for shape analysis and representation [START_REF] Norman | Information concentration along the boundary contours of naturally shaped solid objects[END_REF]. There exist different types of extrema as positive maxima, negative minima and inflexion points depending on the sign of the curvature [START_REF] Koenderink | The shape of smooth objects and the way contours end[END_REF][START_REF] Koenderink | What the occluding contour tell us about solid shape[END_REF][START_REF] Hoffman | Parts of recognition[END_REF][START_REF] Richards | Codon constraints on closed 2d shapes[END_REF][START_REF] Leyton | Inferring causal history from shape[END_REF]. Parallel work on contour integration has demonstrated that the turning angles between successive local elements in a discrete curve play a wide role (Field et al. 1993[START_REF] Feldman | Curvilinearity, covariance, and regularity in perceptual groups[END_REF][START_REF] Pettet | Constraints on long range interactions mediating contour detection[END_REF][START_REF] Geisler | Edge cooccurrence in natural images predicts contour grouping performance[END_REF]; grouping is strongest for smallest turning angles and weakens with their increasing magnitude. All these techniques suggest thereby the most natural break points for segmenting a contour; widely used in shape analysis as shape or contour segmentation.

Break Point Detection

Many authors call these points as breakpoints as well and their detection may be applied to extract curve features for shape analysis or pattern recognition. These breakpoints have an important significance on the boundary and segment the digital curve into homogeneous contour portions to be analyzed further. Many dominant point detection methods have been proposed in the literature and can be classified into three categories according to [START_REF] Carmona Poyato | A New Algorithm for Dominant Point Detection by Quasi-collinear Break Points Supression[END_REF]):

-Those which search for dominant points by estimating directly the curvature in the original picture space [START_REF] Eom | Contour models for curvature estimation and shape decomposition[END_REF].

-Those which evaluate the curvature in curves smoothed and transformed into the Gaussian scale space (Garido et al. 1998[START_REF] Mokhtarian | A theory of multiscale-based shape representation for planar curves[END_REF][START_REF] Mokhtarian | Silhouette-based isolated object recognition through curvature scale space[END_REF][START_REF] Antonio | Shape Matching Using A Curvature Based Polygonal Approximation In Scale-Space[END_REF], Parisi-Baradad et al. 2005). They are suitable for noisy curves.

-Those which search for dominant points using some significant measure other than curvature [START_REF] Carmona Poyato | Dominant point detection: A new proposal[END_REF][START_REF] Cornic | Another look at dominant point detection of digital curves[END_REF][START_REF] Gao | Multiscale contour corner detection based on local natural scale and wavelet transform[END_REF], Masood et al. 2008, Masood 2008, Masood 2008[START_REF] Wu | An adaptive method for detecting dominant points[END_REF] As mentioned in the above literature overview, the greater the turning angle between two given contour fragments, the less likely it is that they will group perceptually into a single unit. If the representation of smooth contours is governed by a similar dependence, curvature extrema would constitute the ''weakest links'' along a contour -thereby suggesting that they provide the most natural break points for segmenting a contour. Moreover, due to digitalization, that makes contours look like stairs, our exterior contours of entities are digital discrete curves of 4-pixel connectivity (Figure Q).

This last observation gave us the intuition to use scale space curvature techniques to detect these points.

In this thesis, we wish to distinguish sheets or groups of sheets that are more susceptible to have linear shape, vesicles or groups of vesicles characterized by a circular shape usually appearing as "squeezed" circular objects and tubes having a "cylinder" shape. Scale space curvature analysis of the contour will in this way provide the points that allow segmenting the contour into several portions to be further characterized in terms either of linearity or circularity.

where: X(l,ı)=x(l)*g(l,ı), Y(l, ı)=y(l)*g(l,ı), * is the convolution operator and g(l,ı) denotes a

Gaussian function with a standard deviation ı.

The curvature k(l, w) of the smoothed contour M(u) w is defined as the variation of the smoothed contour tangent angle with the length l:

( , ) d K l w dl

Discrete curvature computation of a digital boundary depends on a neighbourhood, referred to as region of support. The derivative can be therefore computed in our case within the neighbourhood of 2 pixels around the pixel of interest i using the following approximation: dș = ș(i+1)ș(i-1). The angle ș is computed as:

Classes

The classes introduced by the expert are presented in the figure below: Three main classes are identifiable within our images; isolated objects may include sheets, vesicles, tubes or membranes where the difference between these classes relies on shape criteria. Then composed objects are divided into sheets, vesicles or membranes -aggregates or stacks; the difference between aggregates and stacks lies on the presence of level "4" within the entity. The dominant presence of dark regions within a biological object is essential information for experts concerning the mechanism of the 2D crystallization procedure; proteins aggregate principally when detergent removal was processed too fast. On the other hand, stacks are composed objects interesting to detect as they are "witnesses" of a succeeded crystallization. The "other" family consists of artefacts and unclassified objects that may be found within our image.

E. Preliminary Results

Several experiments were conducted on a variety of TEM images where the proposed method has provided classification results that were evaluated and compared with manual classifications of

Abstract

The image analysis techniques presented in the present thesis have been developed as part of a European project dedicated to the development of an automatic membrane protein crystallization pipeline. A large number of samples is simultaneously produced and assessed by transmission electron microscope (TEM) screening. Automating this last step implicates an on-line analysis of acquired images to assure the microscope control by selecting the regions to be observed at high magnification and identify the components for specimen characterization.

The observation of the sample at medium magnification provides the information that is essential to characterize the success of the 2D crystallization. The resulting objects, and especially the artificial membranes, are identifiable at this scale. These latter present only a few characteristic signatures, appearing in an extremely noisy context with gray-level fluctuations. Moreover they are practically transparent to electrons yielding low contrast.

This thesis presents an ensemble of image processing techniques to analyze medium magnification images (5-15 nm/pixel). The original contribution of this work lies in: i) a statistical evaluation of contours by measuring the correlation between gray-levels of neighbouring pixels to the contour and a gradient signal for over-segmentation reduction, ii) the recognition of foreground entities of the image and iii) an initial study for their classification. This chain has been already tested on-line on a prototype and is currently evaluated.

Keywords

Transmission Cette thèse présente un ensemble de techniques de traitement d'images pour leur analyse à moyen grossissement (5-15 nm/pixel). La contribution originale de ce travail est située sur i) une évaluation statistique des contours en mesurant la corrélation entre les niveaux de gris proche du contour et un filtre de gradient pour réduire le sur segmentation, ii) la reconnaissance des objets de l'image, iii) une étude préliminaire de classification. Cette chaîne est en cours de validation sur un prototype.

Mots Clés

Microscope Electronique en Transmission (MET), Cristallisation 2D, Détection de Membranes, Evaluation de Contours, Approches Statistiques, Représentation d'Empillement, Classification.