the technology has grown tremendously , and now there are numerous types of probes and target elements . target elements can include genes , oligonucleotides , or bacterial artificial chromosomes ( bacs ) and new microarray chips can contain on the order of a hundred thousand probes . due to the technology , the signal obtained is a combination of the biological signal and technological signal . array - based comparative genomic hybridization ( acgh ) technology is similar to cdna arrays and is an extension from conventional cgh that is used to identify and quantify dna copy number changes across the genome in a single experiment . the advantages of acgh include high - resolution and high - throughput measurement capability allowing for more quantitative analysis of the genomic aberrations . a thorough introduction to the design and manufacture of microarrays is provided in   while   provides an introduction to the statistical issues in analyzing microarray datasets . in bac acgh , the probes corresponding to locations on a genome are cloned ( grown ) in a bacterial culture and then arrayed to a glass slide . bac acgh technology can be employed to discover markers in diseases as in [ 58 ] and for detecting genomic imbalances in cancers as described in [ 920 ] . in bac acgh studies , the markers for cancer are often discovered by comparing the signal at a given chromosome loci between the tumor sample and a control sample . specifically , researchers often examine the logarithm ( base 2 ) of the ratio of the tumor sample to the control sample ( log  t / c ) . this value will allow researchers to determine the presence of an imbalance in copy number for a given marker between the tumor sample ( t ) and the control sample ( c ) . it is necessary to normalize the raw log  t / c values before subsequent analysis to determine regions of chromosomal imbalance . normalization procedures have been recognized as necessary for microarray experiments , and a recent search on pubmed ( http://pubmed.org ) reveals over 500 references to articles on microarray normalization . for two - channel microarrays , some of the normalization algorithms similar to the proposed analysis in this paper are cited in [ 21 , 22 ] . specifically , with regards to acgh datasets , recent normalization algorithms are proposed in [ 2327 ] and in r software packages [ 28 , 29 ] . our normalization approach for the log  t / c data will be similar to the approach in   but will feature several important differences in the estimation procedures for the technical effects . specifically , the goal of this paper is to isolate the biological signal in the log  t / c by removing the technological signal via the novel  smootharray the technological signal is composed of three major components : ( 1 ) signal due to the intensity of each scanning channel , ( 2 ) signal due to spatial ( array ) location , and ( 3 ) signal due to the spotting technology . we compare our procedure against other normalization procedures and use several metrics to measure the improvements of our method . although our results can be applied to any print - tip microarray setting , our examples were obtained from the roswell park cancer institute ( rpci ) acgh microarray facility . note our software is written in the r programming language   and is freely available at . in the rpci acgh microarray facility , differentially labeled total genomic dna from a  test  and a  reference  cell population are cohybridized to the bac clones . after hybridization , a genepix axon scanner generates two images of the array at the wavelengths of light corresponding to the two dyes ( cy3 and cy5 ) . the images are processed to generate a single number corresponding to each sample ( dye ) for each spot on the array . for the rpci facilities the resulting ratio of the fluorescent intensities at a location on the chromosomes is approximately proportional to the ratio of the copy numbers of the corresponding dna sequences in the test and reference genomes . a traditional experiment describing tumor extraction , preparation , and so on is described in . for our bac acgh studies , the rpci 19  k bac array was utilized containing 19,000  bac clones ( probes ) that were chosen by virtue of their sts content , paired bac end - sequence , and association with heritable disorders and cancer . reference and test sample genomic dna ( 1  g each ) were individually fluorescent labeled using the bioarray cgh labeling system ( enzo life sciences ) as described in . the hybridized bac - based acgh slides were scanned using a genepix 4200al scanner ( molecular devices ) to generate high - resolution ( 5  m ) images for both cy3 ( test ) and cy5 ( control ) channels . in bacterial artificial chromosome ( bac ) acgh technology , the target dna elements are physically arrayed in a two - dimensional grid on a chemically modified glass slide . note that the bac clones are stored in freezers on a total of 51 plates ( 384 wells per plate ) . another source of potential variation is due to the pin array process of printing the bac clones on the glass microarray slide . for our data , the 48 pins in the arrayer are arranged in a 12  4 matrix structure , approximately 4.5  mm on center , so that they transport the probes to the slide where each pin fills one region or the spots are approximately 80  m in diameter , with respective centers 150  m apart from each other to ensure no overlap between spots . the array has the spots laid out in a 116  348 array of 40368 spots . more specifically , each of the grids within the array ( corresponding to pin number ) has dimensions 29  29 thus there are 841 spots per grid ( pin ) . the array 's spot locations are consecutively labeled row - wise within each pin , first numbering within pin 1 ( 1841 ) , followed by the spots within pin 2 ( 8421682 ) , and so forth . thus , the spot location values range from 1 to 40368 . due to this geometry , each bac clone is repeated on the array ; in other words , each clone has two spots on each array . further , note that there are 384/48 = 8 spots per grid per plate . since 8  51  2 = 816 and each grid has 841 spots , there are 841  816 = 25 blank spots in each grid . each plate is used twice , as each spot is replicated within a grid on the array . put another way , this procedure can produce the intensity levels of 40,368/2  25  48 = 18984  bac clones per array arranged in a two - dimensional array on the slide that accommodates up to 40368 spots . the remaining 25  48 = 1200 spot locations remain unused and are , therefore , not considered in this analysis ( note that however , these unused spots may contain valuable information regarding the background and laser scanner settings ) . the data summary gives the intensity readings from the cy3- and cy5- labeled genetic material for each spot , as produced by the image processing software . since our dataset is related to oncology research , we will refer to the data in terms of the tumor channel ( t ) and the control channel ( c ) . for spot i on the array , we will be interested in the logarithmic ratio of the tumor channel ( ti ) to the control channel ( ci ) . in other words , we will focus on mi  log2ti / ci for a given spot i. we will define the vector m as the collection of mi values for a given sample . from   there are three major sources of possible systematic variation in m which are a consequence of the experimental procedure and do not contribute to differential expression . this bias is evident from figure 2 and is a noted source of variation in two - channel microarrays . in the data shown in figure 2 , this bias predominately produces a curvature shape where the lower intensity probes tend to have a large log  t / c value . the second source of variation is the physical layout on the glass slide ; one can imagine that there are spatial effects across the slide ( caused , e.g. , by the way the dye - labeled material is hybridized to the slide ) which would manifest as a pattern of row and/or column effects if the data were analyzed as 348    116 array . the third source of variation stems from the 384 well plates which are the source of the spots on the glass slide ; one can imagine that there are effects which are localized to one ( or more ) specific plates which would appear as localized effects on the glass slide . as stated in [ 23 , 36 ] , this bias  may be caused by the fact that different clones that are produced in the plates might have experienced slightly different physical conditions during the polymerase chain reaction ( pcr ) or in subsequent purification steps . also there are potential effects due to the 384 well - plate rows ( 16 rows ) and 384 well plate columns ( 24 columns ) . note that the localization is complicated because of the arrayer procedure described above ; recall the complex numbering scheme . one can easily imagine that the pins vary in shape , head size , or some other property that causes the observations to vary from quadrant to quadrant on the array . equally , one can imagine a serial ( in time ) correlation among the observations caused by , for example , the pins not being adequately cleaned between successive dips into the wells on the plates . this is not intended to be an exhaustive list of possible sources of systematic variation , but rather simply a short list of obvious possibilities . the key point here , and in all subsequent analysis , is that we assume a random spatial distribution of the probes on the microarray chip . in other words , we assume that there is no correlation between a probe 's genomic location and its spatial coordinates on the array . this random assumption is required in order for our normalization method to preserve the biological signal present in the chip . with this layout and the potential sources of technical variation note the goal of our  smootharray  algorithm will be to remove the technical variation in the log  t / c values . the data used to demonstrate the process consist of 219 head and neck tumor samples obtained from the rpci microarray facility . process consists of the steps described in table 1 applied sequentially to m , where m denotes the vector of mi spot values for a specific array . let a denote the vector of ai values , where 
 ( 1)ailog2(tici ) ,  i=1,2,3,  ,37968 . 
 a is the vector of logarithm products for the two channels in an cgh microarray experiment . in all subsequent analysis , we will use a typical sample from a dataset designed to examine head and neck tumors as obtained from the rpci microarray facility . the m vector of values from the scanner represents the input values for the  smootharray  algorithm . figure 1 shows the raw dataset ( m values ) in the form of an array image and a genomic plot . note that since each bac clone is printed twice on the array , the genomic plot is obtained by averaging the two mi values for each bac clone . the representation of m values and the ranked m values based on their location on the array image is a good way to view the spatial bias present on an array . the plot shown in figure 1(a ) is commonly referred to as an m - xy plot . by design , each sample is hybridized against a sex mismatch hence , as seen in the genomic plot ( figure 1(b ) ) , there is a gain ( increase ) in the ratio on the x chromosome mimicking a single - copy gain . the data is not centered around 0 in figure 1(b ) indicating a bias in the log  t / c values . the first step in the  smootharray  algorithm employs a loess smoother on the mi values using the ai values as the explanatory variable . in short , a loess smoother fits a polynomial surface determined by the set of explanatory variables . we consider this a global operation since the entire set of probes from an array is used in the loess fitting function , regardless of position on the array or genome . fitting is by ( weighted ) least squares using the  loess  function in the r package   stats . the result from this operation is a set of fitted values , gl(mi ) , according to the ai values , where we denote the loess function as gl ( ) . the fitted values , gl(mi ) , represent the intensity bias present at spot i due to the probe intensity ai . by subtracting the fitted values gl(mi ) we account for this technological bias . hence , we carry forth the m vector to the next step , where m is a vector consisting of  
 ( 2)mi=migl(mi ) ,  i=1,2,3,  ,37968 . m 
 i represents the signal for spot i after removing the technological signal due to the product of the intensities from the two channels . the goal in the next step is to remove the spatial bias present in the acgh technology . the next step in the algorithm removes the technological noise due to the spatial location in the array . it is reasonable to expect nearby spots to be correlated with each other due to the reagents process and the hybridization process in microarray technology . the goal in this step is to accurately determine the spatial pattern present in the array and thus remove it . the data depicted in figure 3 is the starting point for the spatial smoothing step . rather than perform the spatial smoothing on the data depicted on figure 3 , we will modify our approach to ensure we preserve the biological signal . the biological signal present in acgh data can be captured using the circular binary segmentation ( cbs ) algorithm described in . in short , the cbs algorithm can be applied to cluster the m values into segments of estimated equal copy number according to their location on the genome . after cbs , each probe i is a member of a specific segment where we will denote cbs(mi ) as the log  t / c group mean for the segment containing probe i. we apply the cbs function ( denoted by cbs ( ) ) to the values in figure 3 and compute the residuals from the cbs operation as m  cbs(m ) . the spatial two - dimensional kernel density smoother is applied to the residuals from the cbs operation . note that , by subtracting the cbs group we are , in a sense , removing the genome / biological signal to ensure that our kernel density estimate of the spatial signal has a minimal amount of biological signal and is only modeling the technical variation . this is one of the key ways in which our method differs from the algorithm described in . in short , smooth.2d  in the   fields library in r . the smoothing parameter ( bandwidth ) is chosen via a cross - validation ( cv ) procedure . namely , a random subset is removed from the data and the surface is fit . after fitting a surface , the absolute value   loss ( l1 loss ) or the sum of squares   loss ( l2 loss ) for the random subset is computed . the value of the smoothing parameter that yields the smallest sum of squares is used as the optimal value in the kernel smoothing algorithm . after estimating the spatial technical variation ( figure 4(a ) ) , we compute the resulting set of log  t / c values with the spatial variation removed . in other words for spot i , we compute  
 ( 3)mi=miks(micbs(mi ) ) , 
									 where ks represents the kernel smoothing function . mi represents the log  t / c value for spot i after the technical signal due to intensity and the spatial location has been removed . at this point , we remove the technological signal due to the spotting of the array . the technical issues of the spotting procedure noted in [ 22 , 34 , 39 ] demonstrate a solution for cgh microarray chips . similar to determining the kernel smoothed spatial surface , we determine the pin , plate , plate row , plate column and repetition effects for the vector m populated with mi for spot i. similar to the estimation of the spatial effect , we will estimate the spotting process effects on the dataset where we preserve the biological signal by employing the cbs algorithm . we apply the cbs function to the m data and then compute the residuals by subtracting the cbs segment mean . we compute a median of the residuals for each pin , plate , plate row , plate column , and repetition value and use that as our estimate for the spotting process effect . we obtain the final set of log  t / c values representing the remaining biological signal by subtracting the effect of the pin , plate , plate row , plate column and repetition effect from m , that is , the vector of log  t / c values after accounting for intensity and spatial effects . to assess our  smootharray  procedure , we compare it with five other normalization methods . specifically , we examine raw , grid loess , background subtraction , global median normalization , and quantile normalization . in the raw normalization method , we use the raw log ratios without performing any normalization . in a grid loess procedure , within each pin grid , a local polynomial regression fit   is performed on the mi values using the ai values as the explanatory variables . the normalized values from this procedure are obtained by taking the difference between the raw values ( mi ) and the fitted loess values . the loess fitting is done using the  loess  function in the r package   stats . the background subtraction method is employed by subtracting the estimated background intensity from the estimated foreground intensity of each spot before taking the logarithm ratios . the global median normalization procedure is performed by estimating the median log ratio on each array and then subtracting this value from the log ratios on the array . with a global median normalization procedure , the empirical ( sample ) the quantile normalization procedure is enacted according to the procedure outlined in . in this procedure the goal is to impose the same empirical distribution of log ratios to each array . a mean array is created by taking the sample mean across the sorted values of the log ratios in each array . then the distribution of the log ratios in the other arrays in the experiment is matched to the empirical distribution of log ratios in this hypothetical mean array . in the results section as a tool to quantify the results at each step of  smootharray  in terms of reduction of noise we employed the median absolute deviation ( mad ) on the x chromosome . the signal was estimated by the mad on the x chromosome since the x chromosome , by design , was always altered by the virtue of the sex - mismatched controls . further , we do not expect there to be any disease - specific imbalances to occur on the x chromosome . the mad as calculated on the x chromosome acts as a good measure of the performance of  smootharray  and other preprocessing algorithms since the genomic log  t / c data is ultimately used to call regions of genomic imbalance , and this measure is based on a subset of the genomic data . also note that none of the  smootharray  steps require knowledge of the genomic location for a given spot . we are assured that the improvement ( reduction ) in this measurement must be the result of the removal of technical variation and not biological variation . for the chosen array in our experiment , figure 2 shows the loess fit as a function of the intensity ( a ) . the intensity bias in this case is depicted by a convex curve ( red line ) that shows that probes with a small intensity are more likely to have a large log  t / c value . figure 3 shows the resulting m vector as function of location on the array ( figure 3(a ) ) and genomic location ( figure 3(b ) ) . for the genomic plot ( figure 3(b ) ) we averaged the two probes for each bac clone in order to obtain a value for the loci . the mad for the x chromosome after this step is 0.102 , indicating a slight increase from the starting mad on the x chromosome . the results from studying the spatial effects for this data are provided in figures 4 and 5 . figure 4(a ) is the spatial effect image as estimated from a kernel density smoother when using l2 loss as opposed to l1 loss . figure 4(b ) shows the ranked cross - validation ( cv ) values as estimated using an absolute error or l1 loss (  1  line ) and using a squared error or l2 loss (  2  line ) . note that , the smoothing parameter ( bandwidth ) for our kernel density smoother changes slightly when using l2 loss versus l1 loss . the m data is shown via array location in figure 5(a ) and genomic profile in figure 5(b ) . contrasting figure 5(a ) with the data in figure 1(a ) , it is clear that the technical spatial variation has been removed . in fact , the mad for the x chromosome after removing the spatial effect is 0.075 , indicating a reduction from the mad prior to this step . figures 6 , 7 , 8 and 9 show the side by - side box plots of each of the spotting process effects before and after their removal . the final dataset after the  smootharray  process is displayed in an array location image ( figure 10(a ) ) and genomic profile ( figure 10(b ) ) . from a closer analysis in figures 6(a)9(a ) , there is not an obvious outlier in terms of the spotting process effects after accounting for the intensity and spatial bias . the mad for the x chromosome after removing the spotting process effect is 0.069 , which represents a reduction when compared to the mad score of 0.075 prior to accounting for this effect . this 20 percent reduction of noise indicates that subsequent acgh calls of gains and losses should be improved due to the  smootharray  process . figure 10(c ) demonstrates this reduction in noise as a function of location on the chromosome . the raw genomic profile is represented in grey points , while the values after  smootharray  are represented in red . as a global measure of the normalization methods , we also examined the mad for the x chromosome for each of the 219 samples in the experiment designed to examine biomarkers in head and neck tumors . the median mad prior to the  smootharray  process ( raw normalization ) is 0.1465 , while the median mad across the 219 samples after table 2 displays the median mad across the 219 samples for each of the competing normalization methods while figure 11 shows the distribution of the mad for each method . from table 2 and figure 11 we also examined each of the normalization methods by using a nonparametric one - sample t test to examine the significance of the log ratios on the x chromosome . for each probe , we obtain a p value measuring the significance from a mean of 0 for each log ratio . since each sample is hybridized against a sex - mismatched control , we expect each log ratio on the x chromosome to have a significantly small p value . for this analysis , we employed the wilcoxon rank sum test ( a nonparametric test ) for each log ratio . the cutoff of.05 was chosen since , on a univariate level , this would indicate a single copy gain or loss . using this metric , we see that the  smootharray  normalization compares favorably to the grid loess , and global median normalization methods . as a further comparison of each normalization scheme , the use of m - xy plots allows the users a visual metric to compare the different normalization methods . figure 12 allows the user to determine if any spatial abnormalities are present on the array . from examining figure 12 , through a series of sequential steps we have developed an algorithm called  smootharray  which normalizes the logarithmic ratios from a cgh- based microarray platform . secondly the signal due to the spatial location on the microarray is accounted for and removed . each of these sources of signal is a well - documented problem in acgh literature . throughout the algorithm our philosophy was to employ parsimonious and straightforward approaches to correct for the technical effects at each step . we note the similarity of our  smootharray  process with the preprocessing defined in . specifically , both methods remove noise due to intensity effects , spatial effects , and plate effects . namely , we employ the cbs algorithm throughout our  smootharray  algorithm to ensure that we preserve the biological signal , in other words , to ensure that we only remove the technical variation present in the dataset . although minor , we use a loess rather than the lowess method as in   to remove the intensity bias . we also employ a kernel density smoother with bandwidth chosen via cross - validation to account for the spatial bias rather than a 11  11 window median smoother employed in . further , we remove the effects due to the pin arrayer procedure and additionally the plate row , plate column , and repetition effect . by comparing these results on the x chromosome we show the results of our  smootharray  algorithm . also , our  smootharray  algorithm has the flexibility to explore the cross - validation steps using an absolute ( l1 ) loss function or squared error ( l2 ) loss function . note that , currently we have a background option in acghplus which allows the user to subtract a weighted version of the background . future work will explore employing a step that takes into account the background in preprocessing cgh bac arrays . smootharray  via two ways ( 1 ) by using the blank spots at the end of each grid ( see section 2.1 ) or ( 2 ) by using images obtained from the genepix scanner where segmentation algorithms are applied to determine a background signal for each spot . by examining the spot process effect we can employ violin plots to examine the quality control for each array - design variable pin , plate , plate row , plate column , repetition . note that the repetition effect acts as a surrogate measure for a potential time effect . that is , the time elapsed in spotting the probes on the glass slide is represented by the  repetition  variable , since the second spot for each bac clone is not spotted until all other bac clones have been spotted once . violin plots have numerous references in current statistical literature as a way of combining the information available from local density estimates with the basic summary statistics inherent in standard box plots . combining the box plot and the density trace on a single plot , comparing the distributions of several variables via violin plots , is a great tool for cgh microarrays . for the rpci acgh lab firstly , the spot can be flagged for having a low signal - to - noise value . the signal - to - noise value is determined by taking the mean value of the pixels in the signal and dividing them by the standard deviation of the background . if this value is too low ( < 2.5 ) then the spot is of poor quality . secondly , the spot can be manually flagged by the user to determine poor quality , or thirdly , the spot can be flagged of poor quality because of a dim signal in one of the channels . dimness is determined by having a mean signal value under a prescribed cutoff in one of the channels . with this set of flags used for poor spot quality , for a given set of samples ( experiment ) , we can compute the rate for each spot being flagged with a qc problem . then we can examine the distribution of this percentage via violin plots across each spotting process variable . figure 13(a ) shows the distribution of the spot percentages across each   pin , where the spot percentage is the number of times the given spot passed quality control divided by the number of samples in the experiment ( 219 samples ) . similarly , figure 13(b ) shows the distribution of the percentage of bac clones from each   plate that passed the quality control . by studying figure 13 , it is shown that pins 1 and 3 may be suspect in terms of quality control , while several plates have a larger frequency of quality control problems . specifically , plate 28 consists of bac clones that are consistently flagged for quality control problems . by examining figures 13(c ) , 13(d ) , and 13(e ) for this experiment , there does not appear to be any obvious problems affecting the plate row , plate column , or repetition number for the bac clones . the concept of violin plots for quality control can further be extended to other commonly reported spot variables such as background mean , background standard deviation , and other potential outlier flags . for future work , we plan to examine another quality control measure relevant to the bac clones . due to the nature of the bac clones and the updates to the human genome , it is possible that the bac clones could be mismapped from their position on the genome . mismapped bac clones can manifest themselves as appearing as an outlier when viewed via their genomic profile . using a mixture model approach , we plan to subset the number of bac clones under consideration based on estimating the probability for a given bac clone to be mismapped . this approach shows great promise based on our early attempts at modeling mismapped bac clones . a key component in preprocessing acgh data lies in understanding the subsequent analysis steps . with our preprocessed data , the next step in the acgh bac analysis pipeline involves characterizing the genome in terms of detecting regions of chromosomal copy number variations ( gains and losses ) . the softwares and algorithms designed for this analysis include cghcall   and other breakpoint detection methods , for example , [ 4446 ] . a slightly different approach allows the researcher to analyze each chromosomal arm rather than examining within each arm for chromosomal breakpoints . this approach allows the researcher to characterize a chromosome in terms of overall imbalance ( with confidence ) rather than focus on specific regions of gains and losses . our  smootharray  method clearly shows improvement in reducing the noise for a dataset of 219 samples designed to study head and neck tumors . further , when using several quality control metrics , our method performs favorably to five other competing normalization methods described in materials and methods . for future work , we plan to extend our comparisons to quantify the amount of improvement over other competing pre - processing methods such as those in [ 2327 ] . experiments and comparisons such as those employed in [ 32 , 48 , 49 ] can be used to assess the performance and determine the best analysis routes for identifying genomic imbalances in bac acgh datasets . this future study would also compare the subsequent algorithms that assess gains and losses across the genome . this novel method compares favorably against several other normalization measures when evaluated using several quality control metrics . for this study , we focused on data obtained from the rpci microarray facility on a study of 200 head and neck tumor samples . for this experiment , our algorithm reduced the noise by approximately 23 percent . by removing the technological noise due to the intensity effect , spatial effect , and spotting process , the resulting data has reduced noise and is suitable for subsequent analyses to determine chromosomal regions of gains and losses . the  smootharray  method also offers the user the option to examine several quality control figures which allows the researcher to pinpoint problems that may arise in the spotting process .