Skip to content

Commit

Permalink
Merge pull request Azure#7675 from sassoftware/sas-develop
Browse files Browse the repository at this point in the history 
AZUREVIYA 1.4.4
  • Loading branch information
@bmoore-msft
bmoore-msft authored Jun 11, 2020
2 parents bbf6e71 + 12eb336 commit 5aa3449
Show file tree
Hide file tree
Showing 4 changed files with 38 additions and 38 deletions.
70 changes: 35 additions & 35 deletions sas-viya/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ Figure 2: Quickstart architecture for SAS Viya on Azure in an MPP Environment

For details, see [SAS Viya 3.5 for Linux: Deployment Guide](https://go.documentation.sas.com/?docsetId=dplyml0phy0lax&docsetTarget=soe.htm&docsetVersion=3.5&locale=en).

<a name="Costs">
<a name="Costs"></a>
### Costs and Licenses
You are responsible for the cost of the Azure services used while running this Quickstart deployment. There is no additional cost for using the Quickstart.
You will need a SAS license to launch this Quickstart. Your SAS account team and the SAS Enterprise Excellence Center can advise on the appropriate software licensing and sizing to meet workload and performance needs.
Expand All @@ -121,7 +121,7 @@ Here are some recommended example VM sizes based on the number of licensed cores

If you are installing VDMML or a similarly large installation, we recommend that you use at least the Standard E16s_v3 VM size.

<a name="Prerequisites">
<a name="Prerequisites"></a>
## Prerequisites
Before deploying SAS Viya Quickstart Template for Azure, you must have the following:
* Azure user account with Contributor and Admin Roles
Expand All @@ -136,7 +136,7 @@ Before deploying SAS Viya Quickstart Template for Azure, you must have the follo
["Application Gateway limits."](https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits?toc=%2fazure%2fapplication-gateway%2ftoc.json#application-gateway-limits)
* A resource group that does not already contain a Quickstart deployment. For more information, see [Resource groups](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#resource-groups).

<a name="License">
<a name="License"></a>
### Upload the License File to an Azure Blob
When you run the deployment, you will need the blob Shared Access Signature (SAS) URL as a parameter.

Expand All @@ -151,7 +151,7 @@ Before you run the deployment:

For details, see ["Using Shared Access Signatures."](https://docs.microsoft.com/en-us/azure/storage/common/storage-dotnet-shared-access-signature-part-1)

<a name="Mirror">
<a name="Mirror"></a>
### (Optional) Create a Mirror Repository
For your repository, you can do either:
* Use the default method, which downloads the installation files directly from SAS.
Expand All @@ -172,29 +172,29 @@ az storage blob upload-batch --account-name "$STORAGE_ACCOUNT" --account-key "$S

**Note** For the mirror storage, use the same storage account that you used for the license file in ["Upload the License Zip file."](#license)

<a name="Best-Practices">
<a name="Best-Practices"></a>
## Best Practices When Deploying SAS Viya on Azure
We recommend the following as best practices:
* Create a separate resource group for each Quickstart deployment. For more information, see [Resource groups](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#resource-groups).
* In resource groups that contain a Quickstart deployment, include only the Quickstart deployment in the resource group to facilitate the deletion of the deployment as a unit.

<a name="Deployment">
<a name="Deployment"></a>
## Deployment Steps
You can click the "Deploy to Azure" button at the beginning of this document or follow the instructions for a command-line (CLI) deployment using the scripts in the root of this repository.

The deployment takes between 1 and 4 hours, depending on the quantity of software licensed.

<a name="deployment-details">
<a name="deployment-details"></a>

## Additional Deployment Details
## User Accounts
<a name="useraccounts">
<a name="useraccounts"></a>
The *vmuser* host operating system account is created during deployment. Use this account to log in via SSH to any of the machines.

The *sasadmin* and *sasuser* SAS Viya user accounts are also created during deployment. These accounts exist in LDAP, and are the default user accounts for logging in to SAS Viya. You cannot directly log on to the host operating system with these accounts.

### Important File and Folder Locations
<a name="filefolderlocations">
<a name="filefolderlocations"></a>

Here are the location and sizes of key files and folders that are useful for troubleshooting and performing maintenance tasks:

Expand All @@ -209,9 +209,9 @@ Here are the location and sizes of key files and folders that are useful for tro
|SASCACHE|Location of CAS disk cache.|*Controller VM:* /sastmp/cascache<br>Symlink to /mnt/resource/sastmp/cascache, <br>which is on the ephemeral disks for this machine. <br>Size depends on the machine selected and is set by Azure.|
|SASLOGS|Location of the SAS application log files.|*Services VM* and *Controller VM:*<br>/opt/sas/viya/config/var/log<br>(also at /var/log/sas/viya)|
|SASBACKUP|Location for SAS Backup and Recovery Tool backup vault.|*Services VM:*<br>/opt/sas/backups<br>(part of the 256 GB of /opt/sas)|
<a name="Post-Deployment">
<a name="Post-Deployment"></a>
## Optional Post-Deployment
<a name="DNS">
<a name="DNS"></a>
### Configure a Certificate Authority-Signed Digital Certificate and Custom DNS Name

By default, the Quickstart deployment generates a highly unique DNS name for your deployment and a self-signed certificate for secure connections. This is enough for limited use cases or proofs of concept. However, because a self-signed certificate provides limited protection against man-in-the-middle attacks and the default DNS is computer-readable, it is recommended that you change the DNS name and provide a Trusted Root signed certificate.
Expand All @@ -221,7 +221,7 @@ After acquiring a domain name and a TLS certificate from your corporate IT depar

If you have acquired a new domain name or are using an existing domain name, you can upload a trusted root signed certificate for that domain to the application gateway. For details, see ["Renew Application Gateway certificates."](https://docs.microsoft.com/en-us/azure/application-gateway/renew-certificates)

<a name="DataSources">
<a name="DataSources"></a>
### Enable Access to Existing Data Sources
To access an existing data source from your SAS Viya deployment, add an inbound rule to each security group or firewall for the data source as follows:
* If your data source is accessed via the public internet, add a public IP to the SAS Viya services VM and SAS Viya controller VM. Add an Allow rule to your data source for both the services VM and controller VM public IP addresses. When creating the public IP addresses for each SAS Viya VM, a Static IP using the "Standard" SKU is recommended. For details, see
Expand All @@ -235,15 +235,15 @@ To access an existing data source from your SAS Viya deployment, add an inbound

Data sources accessed through SAS/ACCESS should use the [SAS Data Agent for Linux Deployment Guide](https://go.documentation.sas.com/?docsetId=dplyml0phy0lax&docsetTarget=titlepage.htm&docsetVersion=3.5&locale=en) instructions to ["Configure Data Access"](https://go.documentation.sas.com/?docsetId=dplyml0phy0lax&docsetTarget=p03m8khzllmphsn17iubdbx6fjpq.htm&docsetVersion=3.5&locale=en) and ["Validate the Deployment."](https://go.documentation.sas.com/?docsetId=dplydagent0phy0lax&docsetTarget=n1v7mc6ox8omgfn1qzjjjektc7te.htm&docsetVersion=2.5&locale=en)

<a name="ACCESSCertWarn">
<a name="ACCESSCertWarn"></a>
### Validate the Server Certificate if Using SAS/ACCESS
If you are using SAS/ACCESS with SSL/TLS, unvalidated SSL certificates are not supported. In this case, a trust store must be explicitly provided.

**Note:** For most Azure-managed data sources, the standard OpenSSL trust store validates the data source certificate:
```
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
```
<a name="MSSQL">
<a name="MSSQL"></a>
### Set Up ODBC and Microsoft SQL Server
1. Locate the following two odbc.ini files:
* CAS controller: /opt/sas/viya/home/lib64/accessclients/odbc.ini
Expand Down Expand Up @@ -283,7 +283,7 @@ EnableQuotedIdentifiers=0
```
4. Save the odbc.ini files.

<a name="DataAgent">
<a name="DataAgent"></a>
### Set Up SAS Data Agent

1. Perform the pre-installation and installation steps in [SAS Data Agent for Linux: Deployment Guide.](https://go.documentation.sas.com/?docsetId=dplydagent0phy0lax&docsetTarget=p06vsqpjpj2motn1qhi5t40u8xf4.htm&docsetVersion=2.5&locale=en) For the post-installation tasks, you can either:
Expand Down Expand Up @@ -352,7 +352,7 @@ in the SAS Data Agent for Linux: Deployment Guide.
9. Validate the environment, including round-trip communication. For details, see the ["Validation"](https://go.documentation.sas.com/?docsetId=dplydagent0phy0lax&docsetTarget=n1v7mc6ox8omgfn1qzjjjektc7te.htm&docsetVersion=2.5&locale=en ) chapter in the SAS Data Agent for Linux: Deployment Guide.
<a name="Usage">
<a name="Usage"></a>
## Usage
* To log in to any machine via SSH to check on a deployment or to perform maintenance, log in as *vmuser*.
Expand All @@ -376,7 +376,7 @@ ssh vmuser@worker01
ssh vmuser@worker02
```
<a name="Tshoot">
<a name="Tshoot"></a>
## Troubleshooting
If your deployment fails:
1. Check to ensure that the location of your license is accessible.
Expand All @@ -390,7 +390,7 @@ For more information, see ["Upload the License Zip file."](#License)
* If the error comes from a sub-deployment (for example, “AnsiblePhase4PreViyaInstall”), review the log files.
<a name="ReviewLog">
<a name="ReviewLog"></a>
### Review the Log Files
Ansible is the primary virtual machine that is used for the installation. Most of the deployment log files reside on the Ansible virtual machine.
#### Ansible Server Log Files:
Expand All @@ -411,7 +411,7 @@ The /var/log/sas/install directory is the primary deployment log directory. Othe
* /var/log/sas/install
* prerequisites.log: log that is created by the first setup script on the CAS Controller VM that prepares it for Ansible to run against and to mount /mnt/viyashare
<a name="RestartServices">
<a name="RestartServices"></a>
### Restarting the SAS Services
While all the services can be started on each box independently, the Viya-Ark toolkit provides an efficient way to restart all the services across all the boxes from the Ansible controller.
Expand All @@ -428,27 +428,27 @@ Viya-Ark can restart all of the services by issuing the following commands as th
cd /sas/install/ansible/sas_viya_playbook/
ansible-playbook viya-ark/playbooks/viya-mmsu/viya-services-restart.yml -e enable_stray_cleanup=true
```
<a name="UncommonErrors">
<a name="UncommonErrors"></a>
If you encounter the following errors, remove the deployment and redeploy the software again.
#### SSH Error: data could not be sent to remote host
This error is the result of issues related to dns/network during some critical times during the build. (In most cases, the system is set to reconnect or retry.).
#### Yum repo errors “Error: Package 'package' Requires: 'another package' Available"
This error is the result of RHEL RPM mirrors that are not always correct in Azure. The error is rare. Typically the repositories are back in sync within a few hours.
<a name="AddA">
<a name="AddA"></a>
## Appendix A: Configuring the Identities Service
<a name="AddAVerify">
<a name="AddAVerify"></a>
### Verify Security Settings
Ensure that the correct port on your Lightweight Directory Access Protocol (LDAP) or secure LDAP (LDAPS) machine can be accessed by the SAS Viya machines:
* Port 389 if using LDAP
* Port 636 if using secure LDAP (LDAPS). For more information about securing LDAP connections, see [Encrypt LDAP Connections](https://go.documentation.sas.com/?docsetId=calencryptmotion&docsetTarget=n1xdqv1sezyrahn17erzcunxwix9.htm&docsetVersion=3.5&locale=en#p1bai319815977n1bzdyyxr3d5he) in the Encryption in SAS Viya: Data in Motion.
<a name="AddACreateServiceAccount">
<a name="AddACreateServiceAccount"></a>
### Create a Service Account
Create a service account in your LDAP system. The service account must have permission to read the users and groups that will log on to the system.
<a name="AddAConfigureIdentitiesService">
<a name="AddAConfigureIdentitiesService"></a>
### Configure the Identities Service
**Note:** OpenLDAP systems and customized AD setups might require additional configuration that is beyond the scope of this guide.
* See [Configure the Connection to Your Identity Provider ](https://go.documentation.sas.com/?docsetId=dplyml0phy0lax&docsetTarget=p0dt267jhkqh3un178jzupyyetsa.htm&docsetVersion=3.5&locale=en#n1p4yydj6grbban1kl1te52gv0kf) in the SAS Viya for Linux: Deployment Guide for more information about configuring the identities service.
Expand All @@ -469,7 +469,7 @@ In the SAS Environment Manager, on the Configuration tab, select the Identities
Set the default values to work with a standard Microsoft Active Directory system.
 
<a name="AddAVerifyTheConfiguration">
<a name="AddAVerifyTheConfiguration"></a>
### Verify the Configuration
1. Log in to SAS Viya with your LDAP accounts. You might need to restart SAS Viya for the LDAP changes to take effect.
Expand All @@ -480,7 +480,7 @@ Set the default values to work with a standard Microsoft Active Directory system
4. Enter the password to your LDAP service account.
If verification is successful, the list of your users and groups is displayed.
<a name="AddAConfigurePam">
<a name="AddAConfigurePam"></a>
### Configure PAM for SAS Studio
Because SAS Studio does not use the SAS Logon Manager, it has different requirements for integration with an LDAP system. SAS Studio manages authentication through a pluggable authentication module (PAM). You can use System Security Services Daemon (SSSD) to integrate the PAM configuration on your services machine with your LDAP system.
To access SAS Studio, the following conditions must be met:
Expand All @@ -490,10 +490,10 @@ To access SAS Studio, the following conditions must be met:
For details about configuring SSSD against your LDAP setup, see the RedHat documentation. In many cases SSSD is configured to automatically create home directories when a user logs on to the system locally or via SSH. Because SAS Studio does not do this; you must manually create home directories for each remote user.
After SSSD has been configured, you might need to restart the services machine.
<a name="AddB">
<a name="AddB"></a>
## Appendix B: Managing Users for the Provided OpenLDAP Server
<a name="AddBLoginAndList">
<a name="AddBLoginAndList"></a>
### List All Users and Groups
From the Ansible controller VM, log in to the services VM:
```
Expand All @@ -505,7 +505,7 @@ To list all users and groups:
ldapsearch -x -h localhost -b "dc=sasviya,dc=com"
```
<a name="AddBAddUser">
<a name="AddBAddUser"></a>
### Add a User
1. Create a user file that contains all the user info:
Expand Down Expand Up @@ -559,22 +559,22 @@ ldapsearch -x -h localhost -b "dc=sasviya,dc=com"
exit
```
<a name="AddBPassword">
<a name="AddBPassword"></a>
### Change or Set a Password
```
ldappasswd -h localhost -s USERPASSWORD -W -D cn=admin,dc=sasviya,dc=com -x "uid=newuser,ou=users,dc=sasviya,dc=com"
```
**Note:** To prevent the command from being saved to the bash history, preface this command with a space. The string following the -x should match the dn: attribute of the user.
<a name="AddBDeleteUser">
<a name="AddBDeleteUser"></a>
### Delete a User
```
ldapdelete –h localhost -W -D "cn=admin,dc=sasviya,dc=com" "uid=newuser,ou=users,dc=sasviya,dc=com"
```
<a name="Security"></a>
## Appendix C: Security Considerations
<a name="nsc">
<a name="nsc"></a>
### Network Security Groups
SAS Viya Quickstart for Azure uses the following network security groups to control access to the servers and load balancers from sources outside the virtual network. All server to server communication between subnets in the SAS Viya virtual network is permitted.
Expand All @@ -584,17 +584,17 @@ SAS Viya Quickstart for Azure uses the following network security groups to cont
|PrimaryViyaLoadbalancer_NetworkSecurityGroup | Allow port 443/tcp from CIDR prefix specified in the "WebIngressLocation" parameter. Deny all others. | Allow All | PrimaryViyaLoadbalancer |The primary load balancer can be connected to only through https. |
| Viya_NetworkSecurityGroup | Deny All | Allow All |Services Controller | No external connections can be directly made to the SAS Viya servers.
<a name="hard">
<a name="hard"></a>
### Hardening Provided OpenLDAP Security
By default, the OpenLDAP provider is set up if you provide a user password that does not use TLS to secure the communications between the controller and the OpenLDAP server. Most connections should be authenticated by the OAuth provider in SASLogon, which communicates by loopback with the OpenLDAP server. In a production environment, it is recommended that you enable TLS encryption for OpenLDAP queries. For information about enabling LDAPS, refer to the
[ OpenLDAP documentation.](https://www.openldap.org/doc/admin24/tls.html)
<a name="datasec">
<a name="datasec"></a>
### Data Security
The Quickstart deployment is built to get you "up and running" quickly. However, the deployment trades some security for the assurances that a large quantity of SAS licensed products can be loaded without issue. Before you load highly valuable data, it is recommended that you:
* Lock down the communication between the servers to allow only those ports that your licensed products are using.
* Ensure that the user rights of created users are as minimal as possible.
<a name="updates">
<a name="updates"></a>
### Updating the Operating System
During installation, yum updates servers but will not automatically apply patches after deployment is complete. To apply patches either:
* Schedule updates on the boxes through cron
Expand Down
2 changes: 1 addition & 1 deletion sas-viya/ansible/roles/prepare_deployment/deployment_mirror/tasks/main.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
state: "latest"
with_items:
- libunwind
- icu
- libicu
- rsync

- name: create install folder
Expand Down
2 changes: 1 addition & 1 deletion sas-viya/azuredeploy.parameters.json
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"DeploymentDataLocation": {
Expand Down
2 changes: 1 addition & 1 deletion sas-viya/metadata.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"description": "The SAS Viya Quickstart Template for Azure deploys these products on the cloud: SAS Visual Analytics 8.5 on Linux, SAS Visual Statistics 8.5 on Linux, and SAS Visual Data Mining and Machine Learning 8.5 on Linux. This Quickstart is a reference architecture for users who want to deploy the SAS Viya platform, using microservices and other cloud-friendly technologies. By deploying the SAS platform on Azure, you get SAS analytics, data visualization, and machine learning capabilities in an Azure-validated environment. SAS Viya is a cloud-enabled, in-memory analytics engine. It uses elastic, scalable, and fault-tolerant processing to address complex analytical challenges. SAS Viya provides faster processing for analytics by using a standardized code base that supports programming in SAS, Python, R, Java, and Lua. It also supports cloud, on-premises, or hybrid environments and deploys seamlessly to any infrastructure or application ecosystem.",
"summary": "This Quickstart is a reference architecture for users who want to deploy the SAS Viya platform, using microservices and other cloud-friendly technologies.",
"githubUsername": "sassoftware",
"dateUpdated": "2020-03-05",
"dateUpdated": "2020-06-10",
"environments": [
"AzureCloud"
]
Expand Down

0 comments on commit 5aa3449

Please sign in to comment.