<commit_before># Deploy a Linux HDInsight cluster with new Data Lake Store and Storage accounts.

<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fhdinsight-new-datalake-store-azure-storage%2Fazuredeploy.json" target="_blank">
    <img src="http://azuredeploy.net/deploybutton.png"/>
</a>

This template allows you to deploy a new Linux HDInsight cluster with new Data Lake Store and Storage accounts.

### Prerequisites ###

**PFX Certificate and Service Principal**

In order to properly deploy this ARM template, you need to first create a service principal in your Azure Active directory.

This service principal needs to be configured to use a password-protected PFX certificate for authentication.
 
Below are instructions for creating the certificate and service principal.

1. Create a password-protected PFX certificate.
   
    In Windows, you can do this using Azure PowerShell.

    First, install the Windows SDK portion of [Visual Studio](https://www.visualstudio.com/downloads/download-visual-studio-vs). 
    
        $certFolder = "C:\certificates"
        $certFilePath = "$certFolder\certFile.pfx"
        $certStartDate = (Get-Date).Date
        $certStartDateStr = $certStartDate.ToString("MM/dd/yyyy")
        $certEndDate = $certStartDate.AddYears(1)
        $certEndDateStr = $certEndDate.ToString("MM/dd/yyyy")
        $certName = "HDI-ADLS-SPI"
        $certPassword = "<PASSWORD>"
        
        mkdir $certFolder
        cd "C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin"
        
        # When prompted, enter a new password to generate the private key.
        .\makecert.exe -sv $certFolder\myKey.pvk -n "cn=$certName" $certFolder\certFile.cer -b $certStartDateStr -e $certEndDateStr -r -len 2048
        
        # When prompted, enter the same password as before.
        .\pvk2pfx.exe -pvk $certFolder\myKey.pvk -spc $certFolder\certFile.cer -pfx $certFolder\certFile.pfx -po $certPassword

2. Create a service principal using the certificate.

    In Windows, you can do this using Azure PowerShell.

        $clusterName = "<your cluster name>"
        $certificatePFX = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($certFilePath, $certPassword)
        $credential = [System.Convert]::ToBase64String($certificatePFX.GetRawCertData())
        
        $application = New-AzureRmADApplication -DisplayName $certName `
                                -HomePage "https://$clusterName.azurehdinsight.net" -IdentifierUris "https://$clusterName.azurehdinsight.net"  `
                                -KeyValue $credential -KeyType "AsymmetricX509Cert" -KeyUsage "Verify"  `
                                -StartDate $certStartDate -EndDate $certEndDate
                                
        $servicePrincipal = New-AzureRmADServicePrincipal -ApplicationId $application.ApplicationId

3. Obtain the service principal information needed for the ARM template deployment.

    In Windows, you can do this using Azure PowerShell.

    * Application ID: ``$servicePrincipal.ApplicationId``
    * Object ID: ``$servicePrincipal.Id``
    * AAD Tenant ID: ``(Get-AzureRmContext).Tenant.TenantId``
    * Base-64 PFX file contents: ``[Convert]::ToBase64String(Get-Content $certFilePath -Encoding Byte)``
    * PFX password: ``<PASSWORD>``<commit_msg>Updated certificate creation instructions to use PowerShell's New-SelfSignedCertificate and Export-PfxCertificate cmdlets.<commit_after># Deploy a Linux HDInsight cluster with new Data Lake Store and Storage accounts.

<a href="https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fhdinsight-new-datalake-store-azure-storage%2Fazuredeploy.json" target="_blank">
    <img src="http://azuredeploy.net/deploybutton.png"/>
</a>

This template allows you to deploy a new Linux HDInsight cluster with new Data Lake Store and Storage accounts.

### Prerequisites ###

**PFX Certificate and Service Principal**

In order to properly deploy this ARM template, you need to first create a service principal in your Azure Active directory.

This service principal needs to be configured to use a password-protected PFX certificate for authentication.
 
Below are instructions for creating the certificate and service principal.

1. Create a password-protected PFX certificate.
   
    In Windows, you can do this using Azure PowerShell.

    First, install the Windows SDK portion of [Visual Studio](https://www.visualstudio.com/downloads/download-visual-studio-vs). 
    
        $certFolder = "C:\certificates"
        $certFilePath = "$certFolder\certFile.pfx"
        $certStartDate = (Get-Date).Date
        $certStartDateStr = $certStartDate.ToString("MM/dd/yyyy")
        $certEndDate = $certStartDate.AddYears(1)
        $certEndDateStr = $certEndDate.ToString("MM/dd/yyyy")
        $certName = "HDI-ADLS-SPI"
        $certPassword = "<PASSWORD>"
        $certPasswordSecureString = ConvertTo-SecureString $certPassword -AsPlainText -Force
        
        mkdir $certFolder
        
        $cert = New-SelfSignedCertificate -DnsName $certName -CertStoreLocation cert:\CurrentUser\My -KeySpec KeyExchange -NotAfter $certEndDate -NotBefore $certStartDate
        $certThumbprint = $cert.Thumbprint
        $cert = (Get-ChildItem -Path cert:\CurrentUser\My\$certThumbprint)
        
        Export-PfxCertificate -Cert $cert -FilePath $certFilePath -Password $certPasswordSecureString

2. Create a service principal using the certificate.

    In Windows, you can do this using Azure PowerShell.

        $clusterName = "cluster-name-here"
        $certificatePFX = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($certFilePath, $certPasswordSecureString)
        $credential = [System.Convert]::ToBase64String($certificatePFX.GetRawCertData())
        
        $application = New-AzureRmADApplication -DisplayName $certName `
                                -HomePage "https://$clusterName.azurehdinsight.net" -IdentifierUris "https://$clusterName.azurehdinsight.net"  `
                                -KeyValue $credential -KeyType "AsymmetricX509Cert" -KeyUsage "Verify"  `
                                -StartDate $certStartDate -EndDate $certEndDate
                                
        $servicePrincipal = New-AzureRmADServicePrincipal -ApplicationId $application.ApplicationId

3. Obtain the service principal information needed for the ARM template deployment.

    In Windows, you can do this using Azure PowerShell.

    * Application ID: ``$servicePrincipal.ApplicationId``
    * Object ID: ``$servicePrincipal.Id``
    * AAD Tenant ID: ``(Get-AzureRmContext).Tenant.TenantId``
    * Base-64 PFX file contents: ``[Convert]::ToBase64String(Get-Content $certFilePath -Encoding Byte)``
    * PFX password: ``<PASSWORD>``
{'fraction_non_alphanumeric': 0.09662598707824839, 'fraction_numerical': 0.00933237616654702, 'mean_word_length': 3.313312693498452, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2490339', 'n_tokens_mistral': 2011, 'n_tokens_neox': 1930, 'n_words': 531}