extfieldsofstudy:
Medicine
Computer Science

text:
Is Intensity Inhomogeneity Correction Useful for Classification of Breast Cancer in Sonograms Using Deep Neural Network?

The sonogram is currently an effective cancer screening and diagnosis way due to the convenience and harmlessness in humans. Traditionally, lesion boundary segmentation is first adopted and then classification is conducted, to reach the judgment of benign or malignant tumor. In addition, sonograms often contain much speckle noise and intensity inhomogeneity. This study proposes a novel benign or malignant tumor classification system, which comprises intensity inhomogeneity correction and stacked denoising autoencoder (SDAE), and it is suitable for small-size dataset. A classifier is established by extracting features in the multilayer training of SDAE; automatic analysis of imaging features by the deep learning algorithm is applied on image classification, thus allowing the system to have high efficiency and robust distinguishing. In this study, two kinds of dataset (private data and public data) are used for deep learning models training. For each dataset, two groups of test images are compared: the original images and the images after intensity inhomogeneity correction, respectively. The results show that when deep learning algorithm is applied on the sonograms after intensity inhomogeneity correction, there is a significant increase of the tumor distinguishing accuracy. This study demonstrated that it is important to use preprocessing to highlight the image features and further give these features for deep learning models. In this way, the classification accuracy will be better to just use the original images for deep learning.

Introduction
In recent years, with dietary habit and the change of lifestyle, a quick rise has been seen in female's breast cancer. In the developed and developing countries, breast cancer has, respectively, become the first and second causes of women's cancer death. According to the article published in International Journal of Cancer, 1993, one in every 8 women in European and American countries has breast cancer [1]. American Cancer Society pointed out as early as in 1999 that breast cancer is the first cause of women's death in the developed countries [2]. In the various breast cancer diagnosis procedures, ultrasound diagnosis is regarded as a kind of "highly accepted" tool because of low price, convenience, universality, and nonradiation. Ultrasound has become a necessary diagnosis tool in all medical centers and a good tool for doctors to initially diagnose the breast cancer clinically.
However, ultrasonic images often contain lots of speckles, noises, and textures so that it is unable to clearly discriminate the detail changes of tissues, e.g., the tumor size and boundary. erefore, many studies in the past have put forward ultrasound tumor segmentation algorithm, including the well-known deformation model [3,4], clustering [5], overzero [6], thresholding [7], watershed method [8], and level set method [9,10]. e above methods can be roughly divided into two categories, respectively, based on pixel or region information [11]. However, in order to make the segmentation results well [12][13][14], many experiment steps and parameter adjustments are involved. It may have been affected by the developer's subjective ideas and habits so that these results may be not sufficiently objective, affecting the accuracy in doctor's judgment with naked eyes, and it is likely to cause misdiagnosis. erefore, the traditional computer-aided diagnosis tool is limited.
Recently, deep learning has been widely used in various applications such as facial recognition [15,16], object detection, and car identification. In medical imaging, deep learning still offers excellent performance on different fields [17,18]. With the development of deep learning, the system architecture of computer-aided diagnosis has been changed, and the advantages unavailable in traditional computer-aided diagnosis system have been added. In the deep learning method, it is only needed to specify the training data, and then, the features in the images can be automatically extracted, and more abstract feature descriptions can be extracted according to the nerve cell depth of each layer, including the features from point to edge, contour, and even a higher level so as to gradually reduce the nerve cells at the same time of improving the features. In the deep learning method, selection and extraction of features, as well as data classification, are established under the same structure, which has a higher accuracy than traditional feature extraction.
To address the problem of the breast sonogram detection, there are several approaches that can be used such as a convolutional neural network (CNN) [19][20][21], deep autoencoder (DAE), or stack denoised autoencoder (SDAE). Zhou et al. demonstrated the importance of feature extraction and selection for tumor classification and confirmed that the classification effect based on the Shearlet-based texture feature was better than those of the other four feature extraction methods by AdaBoost and SVM. In the classification of 200 benign and malignant tumors, the accuracy is up to 90%, indicating the effectiveness of using machine learning for breast tumor classification [22]. In 2017, Moon added adaptive filtering to the CAD system (A-CAD) to emphasize the characteristics of tumor size. CAD was more robust when classifying tumors larger than or equal to 1 cm. e results showed that the accuracy increased from 73.1% to 81.4% after adding the adaptive filter. It is suggested that the tumor classification effect is better after using adaptive filtering [23]. Due to the speckle noise of ultrasonic images, Raha used four kinds of filters (median, high boost, Sobel, and average filter) for image preprocessing, and watershed was used for segmentation. Finally, the k-nearest neighbors algorithm was performed for the classification of benign and malignant tumors with a high accuracy of 96.4% [24]. Abdel-Nasser extracted a highresolution (HR) image from a set of low-resolution (LR) images using super-resolution to overcome the speckle noise and artifacts problems. e 31 benign and 28 malignant tumors were divided into training and test data by leaveone-out cross validation (LOOCV) and classified by random forests [25]. e classification results with high accuracy were obtained. It was also emphasized that if the texture features were able to accurately represent the tumor information, it is very important to perform preprocessing. Cheng et al. conducted research on deep learning-based CAD. To avoid using inaccurate image processing results to achieve tumor classification, the team used SDAE for US breast cancer, pulmonary nodules in CT scans classification. Relying on the advantages of SDAE well equipped with the automatic feature exploration mechanism and noise tolerance advantage, the classification result can reach 82.4% accuracy. Cheng et al. reported that the research is the first CAD study based on deep learning, which can further study other image features for more accurate classification results [26]. Overall, it is important to do image preprocessing to highlight the image features and further give these features for deep learning models.
In general, CNN is an end-to-end architecture and suitable for various image classification tasks. However, it is relatively hard to find the useful and meaningful features in a constrained-scale training set. In breast sonogram classification task, the training set is usually smaller than the size of the general training images. For example, the number of training images in ILSVRC12 [19] is 1.2 million, while that of our training set is 170. Instead, the autoencoder families deal with feature extraction task as a data reconstruction task. By minimizing the reconstruction error, the primary feature of the training images should be detectable. In this study, therefore, we tend to adopt the autoencoder network to extract the useful features and follow by concatenating a classifier to distinguish the type of breast sonogram. SDAE [27] is one of the state-of-the-art autoencoder architectures. Xing et al. adopted SDAE for the extraction of tumor features and classification of benign and malignant tumors [27]. SDAE is about to find out many typical patterns in the input training data, to deal with the high change problem of tumor margin, and moreover, it is provided with multilayer training automatic feature extraction and noise reduction ability so that the problems likely to occur under traditional methods may be avoided. e accuracy of classifier is highly related to the quality of test images. As for ultrasonic image, due to the imaging principle and the properties of object to be tested, the research scholars are often encountered with two major problems, much noise and intensity inhomogeneity. Noiseresistant function has been added to SDAE when it is designed, to reduce the noise effect. In terms of intensity inhomogeneity, studies have found that after the preprocess of intensity inhomogeneity correction [28][29][30][31] of medical images, the imaging quality is effectively improved so as to reach a better image processing result.
ere are several intensity inhomogeneity correction methods. For example, the filter method [32,33] is to take intensity inhomogeneity as low frequency signal and eliminate the intensity inhomogeneity with the filter method, but such a method has the elimination risk of important low-frequency information; the surface fitting method [34] is generally used to establish a curve with polynomial or spline, and the characteristics in the image are fit on such a curve to serve as image intensity inhomogeneity; the histogram method is to take the observed image as the convolution of original image and intensity inhomogeneity, and the original image can be obtained only through deconvolution; the image segmentation method [35,36] is able to get the segmentation result and estimated intensity inhomogeneity; or the method combining with many concepts above is adopted [37]. In this study, intensity inhomogeneity correction is conducted before the input of ultrasonic testing images. en, the corrected images are input into the classifier system established on the basis of SDAE, hoping to overcome the effect of intensity inhomogeneity in ultrasonic imaging for improving the system resolution and quality.
Main aims of the paper are as follows: (i) A classification algorithm in breast sonogram without subjective influences is proposed (ii) Up to 85% accuracy can be obtained when deep learning was applied to the intensity inhomogeneity-corrected image (iii) e study overcomes drawbacks of sonogram and has reliable discrimination ability (iv) It has the potential to obtain the better classification results after image preprocessing

Input Data.
is study was approved by the Institutional Review Boards of the Taipei Veterans General Hospital. All the experimental methods were carried out in accordance with the approved guidelines. In this study, we used two kinds of dataset. One is the data collected by our team (a database of 96 malignant and 74 benign images) and the other is the public dataset on the website, Rodrigues, Paulo Sergio (2017), "Breast Ultrasound Image," Mendeley Data, v1 (a database of 150 malignant and 100 benign images) [38].

SDAE.
Artificial neural network is a kind of machine learning concept to simulate the learning of human's brain. As for each nerve cell, the nerve cell signals connected to it will influence the output of such nerve cell. e transmission of nerve signals is to add the received signals and then conduct nonlinear transformation, to get new output. e mutual connection in neural network decides the calculation method to make the actual output approach the expected output as much as possible through neural network adjustment. e training of one neural network is to limit the output value to be equal to the input value, and this indicates that the output layer has the same quantity of nerve elements as the input layer, and the error between the two layers is used to adjust the weight of each layer. e training of autoencoder (AE) is unsupervised, so no label information is needed. Denoising autoencoder (DAE) is improved on the basis of AE. It is assumed that the input data include noise, so DAE is suitable to learn the features from the data including noise. SDAE is stacked by DAE, to obtain higher level features. e network training of SDAE is layer-wise, for it is of independent training between each DAE. As for the SDAE network after training, the decoding layer is eliminated, while the encoding layer generating features is reserved. To classify the data, logistic regression (LR) layer is added to be taken as output layer, and LR is of supervised type, adding the expected output volume label information through backpropagation algorithm, and based on the error between actual output and expected output, the network weight between layers is fine-tuned. erefore, the feature learning of system is the result of combination between SDAE pretraining and LR adjustment.
It is supposed that the input data are x and DAE antinoise method is used to firstly set some input as 0 or add Gaussian noise to generate x as shown in Figure 1. e way of thinking is to make the input not perfect in the very beginning so as to get better features after training. erefore, a good result still can be achieved after the input is polluted. DAE encode is obtained according to one nonlinear transformation equation: where Z is the output of the hidden layer, called as feature description, w is the weight of input and hidden layers, b is the offset, and f θ is the activation function of the hidden layer. DAE decoding and reconstruction are realized through the mapping function: where x ′ denotes the transpose operation, x the output of DAE (the reconstruction), and w the corresponding weight. DAE may generate a depth network with many hidden layers through stacking. A standard SDAE structure includes two encoding layers and two decoding layers. In the encoding stage, the output of the first encoding layer is taken as the input of the second encoding layer. It is supposed that there are L hidden layers in the encoding stage, and we can obtain the activation function of number k encoding layer as follows: where Z 0 is the original input and Z L is the feature description of the highest layer. e decoding stage is the same as the encoding stage. e output of the first decoding layer is taken as the input of the second decoding layer, and in this way, the activation function of number k encoding layer can be obtained as follows: where x ′ 0 is the output of the last layer in the encoding stage and x ′ L is the reconstruction of x. e weight of SDAE after pretraining will be the initial weight of adjustment and classification stages. Once the import and discriminative features have learned by SDAE, the next step is to find a suitable classifier to correctly detect the type of the extracted feature of the input image. Since there are many different classifiers such as linear classifier, perceptron, SVM, random forest, and so on, it is necessary to select one of them for the best performance purpose. Most of them are different from SDAE, while the perceptron is also based on the neural network. Inspired by perceptron learning, we concatenate a sigmoid layer to SDAE and formulate it as a cross-entropy loss function to keep it as an end-to-end architecture.
In the LR stage, the expected output is added, and the weight after pretraining is fine-tuned through the supervised learning method, and the activation function of LR layer is S (sigmoid): where t is the output Z L of the final encoding layer and also the depth feature of pretraining of the SDAE method, and the output of S is the classification result, with the value ranging from 0 to 1. e backpropagation algorithm is one kind of gradient descent methods. After the gradient direction is worked out, the weight of the classifier system will advance towards the direction with the quickest gradient descent. In addition, it is a kind of the greedy method, for it always advances towards the steepest direction, seeking for the biggest decent extent.
To realize the slight adjustment of weight, the error calculation method shall be defined. Different error calculation methods have different weight upgrading rules. Here, the common square error will be adopted: where D indicates all input data, d indicates one of the input data, t is the expected output, and y is the actual output. e target is to find out a set of weights, to realize the smallest calculation error: where α indicates the learning rate. e weight upgraded mode is the batch mode. After all inputs are executed, the connecting weight will be changed.

Intensity Inhomogeneity Correction.
e algorithm was composed of two parts, constrained fuzzy cell-based bipartition and the intensity inhomogeneity modelling [39]. When correcting the intensity inhomogeneity, the polynomial surface fitting method is adopted to estimate the image intensity inhomogeneity. Curve fitting is a kind of concept to represent the existing data through the mathematical method. Fitting is to obtain the discrete information through sampling and testing in engineering. Based on the data obtained in this way, it is hoped to obtain a continuous function (polynomial) or the gathering (spline) of many discrete equations to be identical to the data. Surface fitting is the popularization of curve fitting. It is supposed that the region of interest (ROI) image is composed of foreground (F) and background (B), and each area is regarded as the homogeneous area. e observed gray level of image can be expressed as follows: where O i is the gray level of number i pixel, μ ψ is the average value of area where the pixel is located, ψε F, B { } represents the area where the pixel is located, and the latter two items indicate intensity inhomogeneity and noise. Here, intensity inhomogeneity is supposed as the normal distribution of spatial changes, which is composed of a fixed variance and the average value P changing with the space, and it is represented with polynomial surface form. P i represents the value of polynomial in number i pixel, that is, the average value of intensity inhomogeneity of number i pixel in the image, and n i is the composition of change and noise in the polynomial surface indicating intensity inhomogeneity.
Adopt the least square method to minimize the cost function and estimate the polynomial surface as follows: Reconstructed error = 2 xx  Journal of Healthcare Engineering When the pixel is in the foreground area, μ ψ � μ F , and when the pixel is in the background area, μ ψ � μ B . P i � P(x i , y i ), in which P is N time polynomial. After the polynomial surface is obtained, the image intensity will be adjusted based on results, to complete the image correction. e image training and classification steps are shown as follows: (1) Input training image and initialize the neural network weight by use of SDAE (2) Remove decoding part and add LR structure, to establish SDAE-LR system (3) Add expected input volume label and slightly adjust the network weight, to complete the classifier We use the original images of the two different databases, i.e., private dataset and BUSIS dataset, and conduct intensity inhomogeneity correction for the original images. erefore, each database will generate two groups of images (the original image and the corrected image) and their labels as training data.
ese materials are trained in the deep learning model and then performed testing to predict the benign/malignant lesions in the image. In this study, we use five different models to compare the results. Figure 2 shows the flowchart for deep learning.

Results and Discussion
Intensity inhomogeneity often causes unclear contour of tumors in sonogram so that it is uneasy to judge the type of tumors clinically. e correction method is proposed for improving image quality and increasing the accuracy in classification. Figure 3 shows the comparison images of one malignant tumor with/without intensity inhomogeneity. Before correction, it is uneasy to find the whole edge of the tumor. ere is a missing boundary part at the left bottom of the tumor so that a wrong judgment may be caused. However, after correction, the missing part can be clearly seen, so it could be correctly judged as a malignant tumor.
Certainly, intensity inhomogeneity correction also works well in benign tumor images. Figure 4 shows the comparison images of benign tumor with/without intensity inhomogeneity. e original image with intensity inhomogeneity has unclear boundary at the right and left of the tumor that would cause wrong classification result. After correction, the unclear boundary part can be obviously distinguished so as to classify the type of the tumor correctly.
In this study, we used two kinds of dataset. One is the data collected by our team, named private dataset (96 malignant and 74 benign images), and the other is the public dataset on the website, named BUSIS dataset (150 malignant and 100 benign images). In the private dataset, 143 images and 27 images are selected randomly as training data and test data, respectively. In the BUSIS dataset, 210 images and 40 images are selected randomly as training data and test data, respectively.
First, we feed ultrasound tumor ROI images into SDAE network sized of 28 × 28. In the private dataset, 143 images and 27 images are selected randomly as training data and test data, respectively. In the BUSIS dataset, 210 images and 40 images are selected randomly as training data and test data, respectively. Second, a backpropagation algorithm is used on SDAE to learn the weights and the feature representation.
ird, the images are labeled manually and conducted the second supervised training to classify the malignant/benign tumor images. e experiments of benign/malignant classification are grouped into two groups: (1) original images and (2) contrast-enhanced images by intensity inhomogeneity correction, which are shown in Figures 5-8, respectively.
For the private dataset, the classification accuracy values of testing results of two groups of images are, respectively, 63% and 82%. For the BUSIS dataset, the classification accuracy values of testing results of two groups of images are, respectively, 75% and 83%.
e results indicate the improvement of image quality (contrast-enhanced images) actually achieve better classification performance. e results also show that because the image complexity in the private database is higher, the accuracy increases noticeably after intensity inhomogeneity correction.
As we stated previously, CNN may fail when the number of the training images is relatively small. To verify this, we also conduct an experiment of CNN for detecting benign and malignant tumors. e labeled training images are directly fed into AlexNet [19] to learn the prediction results (i.e., benign or malignant tumors). Similarly, the backpropagation algorithm is used to train the weights of AlexNet. Finally, the classification accuracy values of testing results of two groups of private dataset images are 44% and 37%, while the training accuracy values are 99.8% and 99.4%, respectively. Apparently, such CNN architecture is not suitable for such small-scale training images, leading to a serious overfitting problem. It also verified that the proposed SDAE is a proper choice for analyzing malignant/benign tumor images classification. In addition, three models (Inception v3 [40], ResNet [41], and DenseNet [42]) are also used for comparison. Table 1 shows the detection results of the original images (without correction) and the corrected images (with correction) of the private database, in which the displayed value means with/without intensity inhomogeneity correction, respectively. Table 2 shows the results of the BUSIS dataset. To sum up, because the complexity of the images in the BUSIS database is lower, the classification accuracy is higher than that of the private database before or after intensity inhomogeneity correction.
It is clear that the DenseNet achieves the best performance. However, it is also shown that the performance of the different CNN methods cannot achieve good performance, compared with SDAE approach. Although the number of the parameters of AlexNet is relatively low, the total number of parameters is still greatly large than that of SDAE. By fair comparison, we have chosen Resnet with 54 layers and Inception v3 with 34 layers to learn the classifiers with the default settings suggested by their approach. However, such lot parameters in these two networks lead to serious

Conclusion
is study reports that the proposed algorithm overcomes the problem of intensity inhomogeneity in the sonogram, and in combination with the deep learning method for tumor classification. is study compared five deep learning models, and SDAE achieved the best identification accuracy. Since this is a small amount of data, SDAE is a suitable choice for analyzing the small dataset.
For the original image group and the corrected image group in the private database, the accuracy values of identifying the benign and malignant tumors are 63% and 82%, respectively. For the same group in the public database, the accuracy values are 75% and 83%, respectively. e results refer that increasing the image quality will help improve the accuracy of classification. e image complexity in the private database is higher. From the results, it can be inferred that for images with high complexity, the classification accuracy increases noticeably after image preprocessing.
is work is different from the traditional algorithm to classify the type of lesions, i.e., benign or malignant in the sonogram. In the traditional way, the tumor contours must be delineated first by the segmentation technique before they can be classified. However, many segmentation algorithms need to adjust many parameters or depend on the developer's subjective ideas. In this way, the result of the segmentation in turn affects the classification result. e proposed method reduced the impact of segmentation processing steps to lead an objective classification result via deep learning. Moreover, this paper combines an intensity inhomogeneity correction method to make the classification result better. is way is advantageous, and it has the potential to obtain the better classification results after image preprocessing.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.

Authors' Contributions
Chia-Yen Lee and Chih-Chung Hsu conceptualized the study. Chia-Yen Lee was responsible for funding acquisition. Chia-Yen Lee, Chih-Chung Hsu, and Guan-Lin Chen performed the methodology. Chia-Yen Lee and Yi-Hong Chou administered the project. Yi-Hong Chou collected the


extfieldsofstudy:
Mathematics
Computer Science

text:
Learning from Delayed Outcomes with Intermediate Observations

Optimizing for long term value is desirable in many practical applications, e.g. recommender systems. The most common approach for long term value optimization is supervised learning using long term value as the target. Unfortunately, long term metrics take a long time to measure (e.g., will customers finish reading an ebook?), and vanilla forecasters cannot learn from examples until the outcome is observed. In practical systems where new items arrive frequently, such delay can increase the training-serving skew, thereby negatively affecting the model's predictions for new products. We argue that intermediate observations (e.g., if customers read a third of the book in 24 hours) can improve a model's predictions. We formalize the problem as a semi-stochastic model, where instances are selected by an adversary but, given an instance, the intermediate observation and the outcome are sampled from a factored joint distribution. We propose an algorithm that exploits intermediate observations and theoretically quantify how much it can outperform any prediction method that ignores the intermediate observations. Motivated by the theoretical analysis, we propose two neural network architectures: Factored Forecaster (FF) which is ideal if our assumptions are satisfied, and Residual Factored Forecaster (RFF) that is more robust to model mis-specification. Experiments on two real world datasets, a dataset derived from GitHub repositories and another dataset from a popular marketplace, show that RFF outperforms both FF as well as an algorithm that ignores intermediate observations.

Introduction
Long term utility of recommendations is an important indicator of recommendation quality (cf. [Wu et al., 2017]); hence optimizing for long term value is desirable in many practical applications. For instance, an online book seller may want to maximize the sale of ebooks that customers finish reading (assuming that customers who engage with purchased content are likely to return to the store). Similarly, an online video subscription service may want to maximize the number of videos a customer finishes watching (to avoid recommending 'click-bait' content). Unfortunately, long term value is only revealed after a significant delay, which is problematic when training a model to predict long term value based on historical data. The delay causes training-serving skew [Zinkevich, 2017], and hence the model's predictions for new products may be inaccurate, particularly in large-scale systems where new items arrive frequently. While generalization can help to mitigate this problem, per product memorization is important for large scale recommender systems [Cheng et al., 2016].
Motivating Example: Consider an online marketplace for ebooks. We define an engagement event to occur when a customer finishes an ebook within 90 days of conversion. Similar to YouTube video recommendations [Davidson et al., 2010], when a customer visits, the marketplace (1) generates a small set of ebook candidates, (2) scores each of the candidates, and (3) presents them in descending order of predicted probability of engagement. If an ebook is purchased, whether or not a successful engagement will occur is unknown. The approach taken by much of the prior work on learning with delay [Weinberger and Ordentlich, 2002, Mesterharm, 2005, Joulani et al., 2013, Quanrud and Khashabi, 2015 would imply waiting the whole 90 days to see the outcome. However, new books are added to the marketplace every day. If we waited 90 days before we can accurately predict the engagement probability of a new book, the marketplace will miss out on many potential sales. An alternative approach could make use of intermediate observations, which we denote by ISym (for intermediate symbol) throughout this paper. For example, one day after a purchase we can define ISyms based on the furthest page reached in the ebook. Clearly, these ISyms provide some information about the eventual outcome.
Intermediate Observations and Outcomes: To exploit ISyms, we need to formalize their relationship to outcomes. For example, ISyms based on independent, random noise obviously do not provide useful information about outcomes. The key assumption we use is that the outcome distribution can be factored into two models: (1) one that generates ISyms given an instance, and (2) one that generates outcomes given an ISym. An intuitive algorithm could try to learn both models. The first model can be updated as soon as an ISym is available, while the second model is slower to learn but generalizes across instances.
The Problem: In the ebook marketplace, ebook candidates are not generated by a fixed distribution, since new books are being added over time. Furthermore, a useful scoring algorithm should be agnostic to the candidate retrieval process. For these reasons, we model the problem as a semistochastic online learning problem with ISyms, where instances may be selected by an adversary but, given an instance, the outcome and the ISyms are sampled from a joint distribution. In our setting, if ISyms are ignored, a lower bound on the cumulative error is Ω(DN + √ N T ) where D ≥ 0 is the maximum outcome delay, N is the number of instances (e.g., total corpus size), and T is the performance horizon. Note that in a recommender system where we make 10,000 predictions per day and only update the forecaster each evening, a 90 day delay would translate into a lower bound on cumulative error proportional to 10, 000 × 90 × N + √ N T . We show that a much improved O(D + N + Z √ N T ) cumulative error can be achieved by a predictor with a proper factored structure, where Z is the number of possible ISyms and the tilde indicates that we have suppressed logarithmic factors. Examining these two bounds, we can see that ISyms help when the delay D and number of instances N are large and the number of ISyms Z is small.
Practical Deep Implementations: Using this intuition we propose a neural network-based online learner (FF) with two modules learning the above two models separately. However, we found that the factorization assumption is violated in both of our experimental domains. To mitigate this problem, we introduce a second neural network-based learner (RFF) that introduces a residual correction term. We compare both of these factored algorithms to an algorithm that ignores ISyms (DF) on two experimental domains: (1) predicting commit activity for GitHub repositories, and (2) predicting engagement with items acquired from a popular marketplace. In both of these domains, RFF outperforms both DF and FF.
Contribution: This paper offers three main contributions: 1. We formalize the problem of learning from delayed outcomes with ISyms as a semi-stochastic online learning problem. Many prior works have analyzed learning with delayed outcomes [e.g., Weinberger and Ordentlich, 2002, Mesterharm, 2005, Joulani et al., 2013, however, we believe this is the first work to consider a setting where ISyms are exploited to mitigate the impact of delay.
2. We quantify the potential gain for using ISyms under an assumption on the relationship between ISyms and outcomes. In particular, exploiting ISyms helps most when the outcome delay is large and the number of ISyms is small.
3. Finally, we introduce a practical neural network implementation, RFF, for exploiting ISyms. Our experiments provide evidence that RFF outperforms a delayed learner even when the assumptions required by our analysis are violated.

Formal Problem Description & Approach
Let X , Y, Z be finite, nonempty sets representing the instances, outcomes, and ISyms, respectively. We denote the number of instances |X | by N , the number of labels |Y| by C, the number of different ISyms |Z| by Z, and the d-dimensional simplex by ∆ d for any d ≥ 2. The outcome distribution function is p * : X → ∆ C . Let T, D ≥ 0 where T is the number of prediction rounds and D is the maximum number of steps that an outcome can be delayed (the delay is defined to be 0 if the label is received at the end of the round). At each round, the environment generates (x t , y t , z t ) ∈ X × Y × Z such that x t is chosen by an adversary and y t is sampled according to p * (·|x t ), independently in every round t. 1 We will discuss how ISyms are generated below. Using the instance x t , the forecaster makes a prediction p t ∈ ∆ C and incurs instantaneous loss (p t , x t , y t ) = |p t (y t |x t ) − p * (y t |x t )|. At the end of round t ∈ {1, 2, . . . , T }, the forecaster receives a possibly empty set D t ⊂ [t] × Y containing pairs (s, y s ) where s indicates the round that the label y s was generated on. The goal of the forecaster is to minimize cumulative error ξ(T ) = T t=1 |p t (y t |x t ) − p * (y t |x t )| over T rounds. Joulani et al. [2013] introduce a strategy for converting a base online learner into a delayed online learner, both for the adversarial and stochastic setting (but not for the hybrid setting we consider). We refer to this approach or any other approach that learns a direct relationship between instances and outcomes as a direct forecaster for delayed outcomes. As mentioned in the introduction, a lower bound on the cumulative error in our setting where ISyms are ignored is Ω(DN + √ N T ). Thus, if we want to do better, we need to make an assumption about the relationship between how the outcome and the ISyms are generated.
Assumption 1. At round t the environment generates (x t , y t , z t ) ∈ X × Y × Z, x t is revealed at the beginning of the round, z t is revealed at the end of the round, 2 and y t is revealed after at most D additional rounds. Given x t , z t ∼ h(·|x t ) and given z t , y t ∼ g(·|z t ), independently for all rounds.
Note that Assumption 1 implies that for all x, y ∈ X × Y, the probability of y is (1) Assumption 1 says that the outcome is generated by a stochastic process that can be factorized into two conditional probability distributions g and h. Since g does not depend on an instance it can be estimated regardless of the sequence of instances chosen by the adversary. On the other hand, h does depend on an instance, but the ISyms are revealed by the environment much sooner than the label. So h can also be updated more quickly than trying to estimate p * directly from labels.
The formal setting in this paper is motivated by online marketplaces where we want to predict consumer engagement with purchased products. An instance x ∈ X represents features about the users and the content being considered for recommendation, while a label y ∈ Y indicates what kind of engagement occurred, with the simplest being Y = {0, 1} where y = 0 represents "no engagement" and y = 1 represents "successful engagement". While it is tempting to analyze this as a bandit problem, real online market places often rank content based on multiple factors. Since multiple factors are used to determine which items are recommended, there is rarely a predictable distribution over future instances available. Thus, we allow instances to be chosen adversarially.
Proposed Approach: The basic idea is to exploit the factored model for the outcome distribution introduced in Assumption 1. Algorithm 1 learns empirical estimates of g(·|z) and h(·|x) and then uses the factorization (1) to make predictions. Since g does not depend on an instance x ∈ X , the algorithm can improve the estimate of g no matter what sequence of instances the adversary selects.
Although h does depend on an instance, the ISym is revealed on the same round. So the agent can update h quickly when the adversary selects an instance that has not been observed in the past.
Before we analyze Algorithm 1, we need one additional assumption.
Assumption 2. Let ζ > 0 and A ⊂ ∆ N be a subset of distributions over documents such that , the adversary first chooses α t ∈ A and then selects the instance x t ∼ α t (·), independently at all rounds.

Algorithm 1 Factored Forecaster for Delayed Labels
Require: A (g) and A (h) {Forecasters for g and h.} history ← {Stores observation history.} for t = 1, 2, . . . , T do Environment generates (x t , y t , z t ) and reveals Assumption 2 is needed to ensure that we can estimate g quickly. If ζ = 0, the adversary might select instances where some ISym z ∈ Z does not occur and then towards the end of the episode start introducing instances where the observation z is very probable causing any algorithm to have to wait for the label rather than approximating (1).
Theorem 1. Given Assumptions 1 and 2, for any delay D ≥ 0, the expected cumulative error of Al- While the analysis is straightforward, can be derived. It is well-known that estimating the mean of a Bernoulli random variable from T samples results in an Ω( √ T ) error (see, e.g., Devroye et al. [1996]). If the outcomes are delayed by D rounds, at least a constant positive loss is suffered on average in the first D + 1 rounds (before any observation is received). This implies that in the delayed case, a lower bound on the error is Ω(D + √ T ) for a single Bernoulli distribution. Now consider the case that we have N different instances, and each is repeated T /N times. In each of these segments the minimum loss suffered is Ω(D + T /N ), and so the cumulative loss is at least Ω(DN + √ T N ).
Compared to the lower bound that ignores intermediate observations, the bound of Theorem 1 scales more favorably when the delay and number of instances are large, since it depends on D + N rather than DN . However, we pay an additional O Z √ T N + T ζ + 1 ζ price for learning with ISyms. If ζ is very small (meaning that there is at least one ISym that is unlikely to be observed), it can take many rounds to learn a good approximation of g(·|z) for each z ∈ Z. Furthermore, we also pay a price for introducing a large number of ISyms.

Neural Network Architectures for Learning from Delayed Outcomes
Based on our analysis, we propose three neural network architectures for learning from delayed outcomes with intermediate observations.
Direct forecaster (DF): We use a single neural network to predict the distribution over outcomes given an instance (Figure 1(a)). This approach ignores ISyms.
Factored forecaster (FF): This approach learns two neural networks (Figure 1(b) and 1(c)). The first (Figure 1(b)) predicts a distribution over ISyms given an instance, while the second (Figure 1(c)) predicts an outcome distribution given an ISym. This approach is similar to Algorithm 1.

Factored with Residual forecaster (RFF):
Similarly to the factored approach, this approach learns two neural networks (Figure 1(b) and 1(d)). However, the neural network that predicts an outcome distribution has two towers. The first tower only uses the ISym to predict the logits, while the second tower is an instance-dependent residual correction that can help to correct predictions when Assumption 1 does not hold exactly. Furthermore, we train the residual tower with a separate loss and stop backpropogation from that loss to the first tower. This ensures that the second tower is treated as a residual correction helping to preserve generalization across instances. For all experiments and unless stated otherwise, we update network weights using Stochastic Gradient Descent 3 with a learning rate of 0.1 minimizing the log-loss. Except for the networks predicting the outcome distribution from ISyms (Figure 1(c) and left tower in Figure 1(d)), all network towers have two hidden layers. Their output layer is sized appropriately (to output the correct number of class logits) and use a softmax activation. We apply L 2 regularization on the weights with a scale parameter of 0.01. The networks predicting the outcome distribution from ISyms do not contain hidden layers, they do not use regularization or bias in their output layer. Training samples are stored in a fixed-sized FIFO replay buffer from which we sample uniformly.
For the experiment explained in Section 3.1, the networks predicting the outcome distribution from ISyms use a learning rate of 1. Network towers have two hidden layers with 40 and 20 units. The training buffer has a size of 1,000. We start training once we have 128 examples in the buffer and perform one gradient step with a batch size of 128 every four rounds.
For the experiment explained in Section 3.2, the networks predicting the outcome distribution from ISyms use a learning rate of 0.1. Network towers have two hidden layers with 20 and 10 units. The training buffer has a size of 3,000. We start training once we have 500 examples in the buffer and perform 20 gradient steps with a batch size of 128 every 1,000 rounds. 4 The parameters were tuned using grid search.

Experiments & Results
We compare cumulative error of DF, FF, and RFF in two domains: (1) predicting the commit activity of GitHub repositories, and (2) predicting engagement with items acquired from a marketplace.

GitHub Commit Activity
The goal is to predict the number of commits made to repositories from GitHub 5 -a popular website hosting open source software projects. Given a repository, the question we want the online learner to answer is "will there be at least three commits in the next three weeks?". This information could be used to predict churn rate. For example, GitHub could potentially intervene by sending a reminder email. We obtained historical information about commits to GitHub repositories from the BigQuery GitHub database. 6 We started with 100,000 repositories and filtered out repositories with fewer than 3 We tried to use other optimizers but found that most were not sensitive enough to changes in the distribution over instances as they keep track of an historical average over gradients. 4 To simulate less frequent updates to the networks. 5 http://www.github.com 6 https://cloud.google.com/bigquery/public-data/github five unique days with commits between May 1, 2017 and January 8, 2018. This resulted in about 8,300 repositories. In our experiments, an adversary selects both a repository and a timestamp. The outcome is one if there were at least three commits over the 21 days following the chosen timestamp and zero otherwise. The ISym is based on the number of commits over the first seven days following the chosen timestamp. These were mapped to three ISyms: (1) no commits, (2) one commit, and (3) more than one commits. The ISym is delayed by one week and the outcomes are delayed by three weeks. The forecaster receives the equivalent of one sample every 10 minutes.
The adversary initially selects repositories from a subset with a low number of commits -one or fewer commits over the past month. After four and a half weeks, the adversary switches to a distribution that samples from repositories with two or more commits within the past week. Figure 2 shows the outcome probabilities for each ISym under the two distributions used by the adversary. Since the outcome probabilities given an ISym are not equal, this violates Assumption 1. Thus, this makes the task more difficult for the factored architectures but does not matter for the direct architecture. In this experiment, the historical information about a repository defines an instance. We used used binary features to represent the programming languages present in a repository as well as time bucketized counts of historical commit activity for that repository.
To generate the cumulative error, we subtract the loss of an optimal forecaster. Since we do not have access to an optimal forecaster, we trained two models with the same architecture as DF on both modes used by the adversary. We trained these models using the Adam optimizer for 10,000 steps and using an initial learning rate of 0.0005. Figure 3(a) compares the loss 7 of the direct and factored architectures averaged over 200 independent trials. The vertical dashed line indicates the time at which the adversary switches from its initial distribution to a distribution over high commit repositories. Due to the outcome delay, all algorithms suffer the same initial loss for roughly 3 weeks. Then all three algorithms quickly achieve a low loss until the adversary changes the distribution over repositories. Finally, the factored architectures (FF and RFF) recover roughly 2.5 weeks more quickly than DF (as can be seen in Figure 3(a) from week 5.5-8). Figure 3(b) shows the cumulative error for the same experiment. FF and RFF achieve smaller cumulative error as they are better able to adapt to changing distributions. We can also clearly observe that FF is not able to maintain a low loss (when compared to DF and RFF) as it is unable to cope with the slightly incorrect factorization assumption.

Engagement with Marketplace Items
For a popular marketplace with personalized recommendations, we want to predict the probability that after acquiring an item a user will (1) engage with that item more than once, and (2) not delete that item within 7 days after acquiring it (e.g., a user that downloads a new ebook will read at least two chapters and not delete this ebook from their device). The ISyms are measured two days after an item is acquired and the possible outcomes are: (1) Deleted: The item was deleted. (2) Zero Engagements: The user engaged with the item zero times but did not delete it. (3) One Engagement: The user engaged with the item exactly once and did not delete it. (4) Many Engagements: The user engaged with the item two or more times and did not delete it. The outcomes are the same as the ISyms but measured seven days after conversion. We collected 21 days of data between the 10th and 31st of January 2018. For each item with at least 100 conversions, we stored the empirical probability of each ISym and the probability of each outcome conditioned on ISyms. In addition, we stored the empirical probability that an item was acquired on each of the 21 days. We consider two distribution schedules (Figure 4). The first schedule, which we refer to as Staggered (Figure 4(a)), subsampled 100 items for each day (at noon) that appear for the first time on that day in the marketplace (i.e., the candidate items with higher indices only appear on increasingly later days). This schedule simulates items continually being added to the marketplace. This is a very hard setting, which we expect DF to perform poorly in due the need to constantly make predictions about new instances. The second schedule, which we refer to as Uniform, is derived by subsampling 100 items uniformly (Figure 4(b)) and creating a 21 day schedule with half-day intervals by interpolating between the empirical distribution for each item based on the frequencies from the 21 logged days. This schedule is more favorable for DF because instances are sampled according to a slowly shifting distribution. Overall, the ISyms are delayed by two days and the outcomes are delayed by seven days. The forecaster receives the equivalent of one sample every 40 seconds.
In this experiment, we encoded an item/instance using a 100-dimensional one-hot encoding specifying items by index. Similarly to the GitHub experiment, to generate the cumulative error, we subtract the loss of an optimal forecaster. We trained 42 models (one for each half day interval) with the same architecture as DF on all modes used by the adversary (one model for each half-day interval). We train these models using the Adam optimizer and trained for 40,000 steps using an initial learning rate of 0.0005. Figure 5 compares the average cumulative error of all three forecasters over 200 independent trials for both adversarial schedules. For the Staggered schedule ( Figure 5(a)), FF and RFF outperform DF, as expected. DF must wait until the outcomes become available, but FF and RFF are able to generalize to new instances. RFF performs slightly better than FF, indicating that RFF can mitigate the incorrect factorization assumption as delayed outcomes become available.
The Uniform schedule ( Figure 5(b)) is easier since the distribution over instances is shifting slowly. DF, FF, and RFF all achieve smaller cumulative error compared to the Staggered schedule. In the Uniform schedule, DF outperforms FF because the factorization assumption is violated by the data. However, RFF achieves similar results to DF because its residual tower allows it to mitigate the error introduced by the incorrect factorization assumption.
4 Related Work Chapelle [2014] proposes a model for learning from delayed conversions. However, this approach does not take advantage of potential intermediate observations. Learning from delayed labels is also related to survival analysis [Yu et al., 2011, Fernández et al., 2016, where the goal is to model the time until a delayed event. A significant difference of our work is the use of intermediate observations. In a stochastic online learning setting, where the instances and labels are sampled from the same distribution at each round, the regret [Joulani et al., 2013] or mistake bounds [Mesterharm, 2005] scale with D + N (i.e., delay plus number of instances; the exact dependence on the time horizon T depends on the loss function). However, the stochastic assumption is not realistic for online marketplaces because new products are being added on a regular basis. Thus, the distribution over instances is not independent and identically distributed from day to day.

Discussion
We present a way to leverage intermediate observations to learn faster in scenarios where the long term labels are delayed. Our theoretical analysis shows that the cumulative error of the factored approach which exploits intermediate observations scales as D + N unlike a naive approach that scales as D × N . We present experimental results on a dataset from GitHub as well as a dataset from a real marketplace, and show that our algorithms can learn faster when intermediate observations are helpful, and can gracefully recover the baseline performance when intermediate observations are unhelpful. We believe that the proposed approach can be beneficial in many real-world applications where the goal is to optimize for long term value. It would be interesting to extend our theoretical analysis to ranking measures.

Learning from Delayed Outcomes with Intermediate Observations: Supplementary Material
A Proof of Theorem 1 Proof. Let T t−1,zy denote the number of times we observe a feedback-label pair (z, y) by the end of round t − 1, and T t−1,z the number of times we observe an ISyms z. Note that T t−1−D,z = y∈Y T t−1,zy . Furthermore, let T t−1,xz and T t−1,x denote the number of times we observe a side information-feedback pair (x, z) and, respectively, a feature vector x by the end of round t − 1 (note that T t−1,x = z∈Z T t−1,xz ).
Define the estimators g t (y|z) = T t−1,yz /T t−1−D,z and h t (z|x) = T t−1,xz /T t−1,x ; note that T t−1−D,z is the number of times we observe labels for the feedback z before the tth prediction is made. Let δ t = 3δ π 2 t 2 (so that ∞ t=1 δ t = δ/2). By the union bound and the Hoeffding-Azuma inequality [Devroye et al., 1996], we can obtain concentration bounds for g t and h t . Let E 1 denote the event that hold simultaneously for all 1 ≤ t ≤ T, x ∈ X , y ∈ Y, z ∈ Z, where the right hand side is defined to be infinity when the corresponding counts in the denominator (T t−1−D,z and, resp., T t−1,x ) are zero. Then E 1 holds with probability at least 1 − δ/2.
The error of the estimate at time t for any x ∈ X and y ∈ Y can be bounded as Given E 1 , if T t−1,x ≥ 1, the first term can be bounded by Z 1 2Tt−1,x log 2ZN δ T . Furthermore, taking into account that z∈Z h * (z|x) = 1, the second term is bounded by 1 2 min z∈Z T t−1−D,z log 2ZC δt as long as min z∈Z T t−1−D,z ≥ 1.
Bounding the first expression for all t is simple, as we get the ISyms z t immediately after making a prediction. For any fixed side information x, we can use the concentration bounds for our estimates from the second time x is observed. Thus, given E 1 , Summing up for all x ∈ X and using Jensen's inequality with the concavity of the square root function and x∈X T T −1,x = T − 1, we get n t=1 z∈Z


Translating Indonesian Notarial Documents into English: Issues and Its Strategies

─Notary deed is an authentic deed drawn up by or before a notary. As a legal document, notary deed is drawn up using Indonesian legal language. Due to many foreigners entering into notary deeds in Indonesia, translation of notary deeds into foreign languages (particularly English) is indispensable. This notary deed is normally translated by certified translators. The aims of this research are to find out the issues faced by the translators in translating notary deeds and the strategies applied to translate them. The data used for this research are taken from 10 (ten) notary deeds obtained from notary offices in charge in 4 (four) regencies and 1 (one) city. The ten notarial documents consist of 3 lease agreements, 2 powers of attorney, 2 prenuptial agreements and 3 affidavits. To find out the issues faced by the translators, questionnaires were given to the translators frequently translating notary deeds and to recognize the strategies used by the translators, comparative study was carried out to source and target language. Based on the result of analysis, it was found 2 causes producing issues to the translators: linguistic and non-linguistic issues. While strategies applied in translating notary deeds are literal translation, established translation, calque, transference, cultural translation and official translation in word level, while in sentence level, literal translation, transposition and adaptation translation were commonly applied. Keywords─authentic deed; notary deed; legal system; certified translator;

I. INTRODUCTION
The advanced development of technology and information has changed various aspects of life, including human flows, business behavior and the world economy. With these developments, the legal field is required to be able to counterbalance; in this case, Indonesian law is required to be able to adapt the phenomenon of international cooperation, of which goal is for the sake of mutual prosperity. Indonesian Economic Laws should also be capable to anticipate the effects of new developments, as a result of globalization indicated by the increasingly thin boundaries between countries resulting from the development of information and technology. These new developments continuously enhance the link of the Indonesian economy to the world economy, so that the Indonesian economy could not close itself to the influence and demands of globalization. As the result, free competition and business activities will increase very tremendously. Those doing business in Indonesia are not only Indonesian but foreigners from all over the world. To promote this atmosphere, the government must prepare, and provide adequate facilities and infrastructure in various fields, particularly legal protection and legal certainty as well as human resources in legal field to attract foreign investors and investment in Indonesia.
Notary Public (Openbaar Ambtennar) is one of the public officers required by foreigners who intend to draw up notarial documents (agreement or contract). He must be professional as he is representing the state in carrying out his duties and social functions in drawing up notarial deeds in the form of "Authentic Act". Notarial deeds are made by the parties before a notary public as a means of obtaining legal protection and certainty regarding the rights and obligations as well as roles and functions of each party.
A notarial document (deed) as an authentic deed under the prevailing law in Indonesia shall be drawn up in Indonesian language. It emerges an issue for foreigners who are the parties to the notarial deed, not only to understand the contents of the document to be executed, but also to comprehend legal terminology in different language and legal system as well as culture. To settle these issues, a notary usually asks for the service of a certified translator to translate the deed. In this case the translator plays an important role to bridge the differences between the legal cultures and more particularly between legal systems (Kocbek, 2012) in rendering the message of the source language (SL) into the target language (TL) (Cao, 2007;Newmark, 1998). Translating a notarial deed is not an easy job as it is a legal document, it is determined not only by particular factors such as, different culture, linguistic and legal environment where this legal document is drawn up but on the target users having their own culture and knowledge (Gotti, 2016). Therefore a certified translator translating a notarial document should not merely transfer the message from the source language (SL) into the target language (TL) which contains legal language, system and culture but he must also render the legal language, system and culture of SL into TL; since the mistake in rendering the message may cause legal consequence to the parties to the contract in the future (Cao, 2007;Sarcevic, 2000).
Based on the aforementioned background, this research is deemed very necessary to find out the issues faced by the certified translators in translating notarial documents and the strategies applied to cope with the problems.

II. THEORETICAL REVIEW
The translation of notarial document is categorized as legal translation. It is a specific institutional translation as it closely bound by specific culture (Newmark, 1998) of certain country. Therefore legal translation is considered as the most difficult types of translation (Murici, 2016) and challenging (Müllerová-Shiflett, 2012) as it has its own and particular register and the translators face various legal system between SL and TL (Sarcevic, 2000). It is supported by Rackeviciene & Sliogeriene, (2014) stating that one of the affecting factors in legal translation is the reality that it is not merely a translation between two or more languages but it is between legal systems and cultures. Thus, the duty of a translator is onerous as he has to be faithful with the intent, style and form of source text (SL) while the message in the TL must be obvious and understandable without any deviation as well as legally acceptable (Zidan, 2015). Because of these particularities, issues and difficulties are inevitable for the translator.
To analyze the issues faced by the translators in translating notarial documents, the idea of Cao (2007) is used. She identified three sources of issues faced by the translators; they are difference of legal system involved, linguistic and cultural difference between SL and TL. While to analyze the strategies applied by the translators of notarial documents, Newmark's strategies are applied. Newmark (1998) proposed 8 translation strategies split into two poles. Four strategies (word-for-word, literal, faithful and semantic translation) are SL-oriented translation and the other four (adaptation, free, idiomatic and communicative translation) are TL-oriented translation. It complied with what he stated that foreign laws are translated for information-specific only, and for such type, Newmark (1998) suggested literal or semantic approach to translation. On the other hand, he also considered that formal register of TL must be observed when handling documents that are to be valid at the same time in the TL society. In this case, according to Newmark's view requires communicative approach (TL-oriented). Although the main theory applied in this research is Newmark's, however it is not impossible for the translators to use other strategies. Therefore, the idea of Molina & Albir (2002) is also used as the supporting theory.

III. METHODOLOGY
This research aims at answering two problems. This first problem is the issues faced by the translators and the second one is to find out the strategies used to cope with the issues in translating notarial documents. For the first problem, questionnaires were delivered to 7 translators commonly provided notarial document translation services. Of the 7 questionnaires sent, 5 questionnaires were returned (answered) and the results were further analyzed and discussed in the findings and discussion. To answer the second problem, the researcher used the corpus consisting of ten notarial documents obtained from notary public in charge in four regencies and one city in Bali, Indonesia. The ten notarial documents consist of 3 lease agreements, 2 powers of attorney, 2 prenuptial agreements, and 3 affidavits.
The method applied for this research is descriptive method as it is a product-oriented translation proper (Jakobson in Venuti, 2004). The data is qualitatively analyzed by segmenting the document into segments (term commonly used in Computer Assisted Translation (CAT) Tools to refer to a chunk or a sentence), as described in Table 1. Then, each segment of SL and TL are compared to recognized the translation strategies applied by the translators.

A. Issues encountered by the Translators in Translating
Notarial Documents Indonesian notarial document as described above is drawn up in Indonesian Language under Indonesian Law. It surely has its own legal system and terminology, as Sarcevic stated that if a legal translation is considered as a communicative act in legal context the background to its law is closely related to the place where the legal system or tradition is produced and enforced as well as the language forming the shape of the legal language (Sarcevic, 2000, p. 56). As a result, such condition causes difficulties to find equivalence in legal terminology between Indonesian language to English. This is a serious issue among legal translators. Further issue is difficulty in finding equivalence in TL. In relation to this lack of equivalence, then Matilla (2006) conveyed two main dangers: void literal translation and misleading literal translation. Void literal translation occurs if a legal concept or institution in TL is meaningless to a foreign party. And the most serious danger is misleading literal translation as it shifts the whole status and function of the legal concept in the TL.

Advances in Social Science, Education and Humanities Research, volume 166
The result of this research showed that some challenges were encountered by the legal translators in translating Indonesian notarial documents into English. They can be divided into two, namely, linguistics and non-linguistics issues.

1) Linguistics issues
All respondents reported that they encountered the main problem of finding terminological equivalence between Indonesian notarial documents (SL) and English (TL). For instance, surat kuasa, pemberi kuasa, penerima kuasa, and so forth, are thimbleful terms of art that are difficult for the translators to come across the exact terms in English. Some translators translated surat kuasa into letter of authority, authorization letter and others translated it into power of attorney. So, which is closer to English legal term of art? It is hard to determine. However, from the translated text of the notarial documents most translators opted power of attorney. Similar condition occurs in the collocated terms pemberi kuasa and penerima kuasa. The equivalent legal terms in the TL are even more difficult to find. Some translated them literally into authority giver and authority receiver; some translated them into authorizer and authorizee, and others into grantor and grantee. This issues as David and Brierly (in Cao, 2007) said that the absence of exact equivalence of legal concept and category is the greatest problem in analyzing comparative law. If comparative law is hard to be conducted, its translation is difficult to do.
Syntactic issue is inevitable in the translation of Indonesian notarial documents into English. Indonesian legal Language of notarial document is featured with lengthy and complex sentences. A single sentence is a notary document may be so complex and long, so it looks like a paragraph (Anditya, 2014;Cao, 2007). This lengthy and complex syntactical structure may affect readers' and translators' comprehension and hence makes it difficult in rendering into TL (Anditya, 2014, p.25).

2) Non-Linguistics issues
There was no report from the respondents with regard to the issues of different legal system between Indonesian and English legal system. It seems that such difference also disembogues and is realized in the difference of terms of art. Relating to the cultural difference, two respondents reported that such difference contributed issues in translation but it is implied in the terminological equivalence. Sarjana Hukum, for instance, as the degree of the notary public is tough to find the exact equivalence. As the result, some translators translated into Law Scholar, some translated it into Bachelor of Law and others into Graduate at Law.

B. Strategies Applied in Notarial Documents Translation
Two issues have been illustrated above, but, in fact, all the afore issues are involved in the legal terms of art, style and syntax. It has to be carefully observed in the translation of notarial documents. The procedures to cope with these issues in sentence, phrase or even in the word level (Newmark, 1998) is referred to as strategies. To avoid confusion and misleading, in this research, the strategy also includes method of translation, namely, the technique picked up by the translator (Molina & Albir, 2002) in translating notarial documents. In this research, both terms are considered to have similar definition and used interchangeably. Principally, any methods or strategies can be used in translating notarial documents, but some strategies are applicable and frequently used. In translating practice, a translator may choose one or more appropriate strategies in accordance with to whom and for what purpose the translation of the notarial documents is commissioned.
Based on the result of the research, the strategies applied in translating notarial documents can be distinguished into two, namely, translation strategies in word level and sentence level.

1) Translation Strategies in Word Level
Each translation expert has his/her own terms in determining the translation strategies, even they often tend to be overlapping between a strategy of one expert to the others, as the same strategies may be differently termed. Of the translated notarial documents, the following strategies were dominantly applied in the translation of notarial documents.

a) Literal Translation
Literal translation is the basic strategies automatically applied by a translator if appropriate. "If appropriate" means that the meaning or message of SL can be well rendered and the structure of the translation is compatible with the grammatical structure of the TL. In the event that such condition cannot be attained, then other strategies will be selected. It is different from what Alcaraz & Hughes (2002) stated that they did not recommend literal translation. But, it is not always so. As seen in the example below: The above examples indicate the words or phrase in SL is word-for-word translated into the TL without any context. The translators only tried to reconstruct the SL structure into the grammatical structure of the TL. In practice, it is commonly acceptable and understandable and the transaction at the Notary Office took place. Thus, what Alcaraz & Hughes suggested in this example and condition did not work. This strategy is the most frequently used in the notarial document translation.

b) Established equivalence
This strategy is popular and familiar among translators in notarial document translation. The terms of art used by the translators are commonly found and well accepted by the foreign readers. These terms of art are normally found in the Black Law's Dictionary or in the event such terms are not found, neologism are normally applied but the terms are familiar and well accepted by readers. The lessor agrees to lease out the land … and the lessee agrees to rent from the lessor a plot of Perjanjian ini dilangsungkan dengan syarat dan ketentuan sebagai berikut:

Advances in Social Science, Education and Humanities Research, volume 166
This agreement is entered into under the terms and conditions as follows: Yang Menyewakan dan penyewa in SL are collocation and then translated into the lessor and the lessee which collocates. The term lessor cannot be found in the dictionary but commonly accepted by the parties in the contract as well as the readers in general as the term collocates with lessee. Similarly, terms and conditions are commonly used in the notarial documents and accepted by the readers. The use of this strategy occupies the second position after literal translation strategy.

c) Calque
This translation strategy is applied when translating phrase or words of SL into TL literally. Newmark (1998) called this this strategy as through translation or "the literal translation of common collocation, or names of organization. Vinay & Darbelnet (1995, p. 129) stated that calque is a special kind of borrowing whereby a language borrows an expression form of another, but then translates literally each of its elements, the result is either lexical calque or structural calque. Note that calque uses the expression or structure of SL. Target Language (TL) -Biaya pemeliharaan wajib dibayar sebelum tanggal 10 setiap bulan kepada managemen vila.
-The costs of maintenance shall be paid for prior the 10 th of each month to Villa Management.
This translation strategy just transfers the phrase managemen vila into Villa Management by adjusting the spelling and structure of SL into TL.

d) Transference
The translators used the SL words directly without any changes and adaptation at all. Newmark (1998) stated that transference strategy is used if the equivalence of SL word cannot be found or unknown in TL, or the word refers to a specific place of situation in the SL. It is supported by Molina, & Albir (2002) that to resolve the unknown concept in TL, borrowing or translation using loan word (Baker, 2001) is the easiest strategies to apply. As exemplified above, the transference strategy is used by the translators in translating notarial documents as they cannot find any term having similar or even closer meaning to the term in TL, that's why the translators just picked the SL word purely without any changes or adaptation in the TL as (Hariyanto, 2015;Suryawinata & Hariyanto, 2016) referred to as transliteration. In this research, this strategy is applied for the level of territory in Indonesia as it is hard to find its equivalence in the TL.

e) Cultural equivalence
Cultural equivalence is a translation strategy by substituting the cultural words or phrase of SL with the cultural words or phrase of TL (Newmark, 1998, p. 82). The benefit of this strategy is that the translation is easier to be understood and comprehended by the TL readers. The concept, message and condition implied in the SL can be depicted in the TL. Foreigners are easier to understand District Court with the assumption that this court is having jurisdiction in the regency level than if it is translated into State Court of Gianyar. And the same case occurs in the translation of demikianlah. If it is translated into "that all" the foreign readers will be confused to comprehend the intended message. Fortunately, the cultural equivalence exists in the TL for demikianlah, so it can be easily understood that this expression is used to close the deed formally.

f) Official Translation
In official translation strategy, the translators use the terms of art of which use has been officially legalized by a certain institution. The translated terms are usually names of institutions or technical terms used by a certain institution.

Advances in Social Science, Education and Humanities Research, volume 166
The literal strategy used by the translator in word level is the same as in sentence level. Applying this strategy, the translators translate the SL of notarial documents without modifying the meaning of the SL. The translators may adjust the grammatical structure of the TL. From the segment above, it is clear that the translator only renders the SL sentence into English. No modification was made by the translator in term of meaning. This result of this translation strategy seems unnatural and therefore not recommended but it is acceptable and comprehendible. Even this translation strategy is mostly applied by translators in translating Indonesian notarial document into English.

b) Transposition strategy
This translation strategy is used in translating clauses or sentences. This strategy is applied upon the reason of style. It includes change of plural to singular form to structural change of sentence (Newmark, 1998, p. 85). The split of a sentence or joining two sentences or more into one sentence includes in this strategy. Under this strategy, the translators translate sentences in different grammatical structure of SL but has similar semantic density. From the example above, the translator changed the structure of the SL sentence in TL sentence. The SL sentence structure is in active form while the structure in the TL is in passive form. In the later example, both SL and TL sentence are in passive form, but there is different focus between SL and TL. SL focuses on the land while TL focuses on several units of villas.

c) Adaptation
Due to different background between SL and TL, the equivalence expression or concept of SL is frequently difficult to find (Bell, 1991, p. 71). In this case, the strategy used by the translators is the use of equivalence expression or words commonly used in the same circumstance to translate an expression or words. Of the above examples, the translators use TL equivalence expressions commonly used by the foreigners for the SL expressions or concept. The exact equivalence for demikianlah, menghadap kepada saya, and selanjutnya di dalam akta ini disebut sebagai Pihak Pertama are not in witness whereof, appear before me, hereinafter in this deed … respectively, however these expressions are commonly used by the translators and well understood and comprehended by the foreign users.

V. CONCLUSION
Translators encountered some issues in translating notarial documents, such as, linguistic and non-linguistic issues but they did not realize that the linguistic issues faced were derived from different legal system and culture between SL and TL. To cope with such issues, some translation strategies were used, namely, literal translation, established translation, calque, transference, cultural translation and official translation in word level, while in sentence level, literal translation, transposition and adaptation translation were applied.