Data Warehouse

Tutorial Videos

API

Storage and Compute

Data Sources

CDC Setup

Transform

KPI

Models
Segments

Dashboard

Drill Down

Explores

Machine Learning

Sharing

Scheduling

Notifications

View Activity

Admin

Launch On Cloud

FAQs

FAQ's

Security

Feedback

Option to take feedback from UI

Release Notes

Release Notes

Kubernetes Compute

Models

Segments

Overview

In the Sprinkle console, you need to create drivers for the Compute and the Notebook. You need to update the drivers in the Project settings to use newly created Drivers. This document will describe how to create these drivers and update required project settings.

Create Compute 

  1. Open the Sprinkle console and choose Admin and then Drivers from the Left Panel.
  2. Select Create New Compute from the Compute and then select type as Compute Kubernetes.
  3. In the configuration screen, provide some distinct name.
  4. In the case of GKE, you can get the cluster url from the Google console. Goto Kubernetes clusters, choose your cluster and then from the Details tab copy Endpoint and update with https protocol.
  5. In the case of EKS, you can get the cluster url from the AWS console. Goto EKS clusters, choose your cluster and then select the Configurations tab, then Details and copy API Server Endpoint.
  6. In the case of AKS, you can get the cluster URL from the Azure console. Go to Kubernetes Services, choose your cluster and then go to the overview page. There you can copy the API Server Address and create the cluster URL as ‘https://<API Server Address>’
  7. Provide Cluster CA Certificate and whether it is encoded or not. By default, AWS provides certificates in encoded format. GCP and AKS are provided in decoded format.
  8. In AKS to get the cluster CA certificate go to the Kubernetes Resources → Configurations → Secret. Here users can copy all the credentials like certificate (ca-crt), user-token and namespace by clicking on the view icon.
  9. You need to provide a User token and namespace details. You can refer to the Kubernetes Documentation for token authentication.
  10. You can provide node group labels as well if you want Sprinkle to launch pods only in specific nodegroups.
  11. Once all details are provided, you can try Test Connection and then Create if Test Connection is successful.

Create Notebook Compute

  1. Open the Sprinkle console and choose Admin and then Drivers from the Left Panel.
  2. Select Create New Notebook Compute from the Notebook Compute and then select type as Kubernetes Notebook.
  3. In the configuration screen, provide some distinct name.
  4. Provide Notebook Docker Container Url, if you want to use your own docker image for notebook. We suggest using default values.
  5. In the case of GKE, you can get the cluster url from the Google console. Goto Kubernetes clusters, choose your cluster and then from the Details tab copy Endpoint and update with https protocol.
  6. In the case of EKS, you can get the cluster url from the AWS console. Goto EKS clusters, choose your cluster and then select the Configurations tab, then Details and copy API Server Endpoint.
  7. In the case of AKS, you can get the cluster URL from the Azure console. Go to Kubernetes Services, choose your cluster and then go to the overview page. There you can copy the API Server Address and create the cluster URL as ‘https://<API Server Address>’
  8. Provide Cluster CA Certificate and whether it is encoded or not. By default, AWS provides certificates in encoded format. GCP and AKS are provided in decoded format.
  9. In AKS to get the cluster CA certificate go to the Kubernetes Resources → Configurations → Secret. Here users can copy all the credentials like certificate (ca-crt), user-token and namespace by clicking on the view icon.
  10. Provide the same User token and namespace provided in the previous step.
  11. You can provide node group labels as well if you want Sprinkle to launch pods only in specific nodegroups.
  12. Provide Notebook timeout in mins, this will stop the notebook server if it is ideal for specified time. 
  13. Provide Sprinkle console url as Sprinkle API url
  14. You need to create an API key. Goto Admin from the left panel and select Permission. Choose API Keys Left side menu and choose New and then provide some Name and choose Create. Copy the API Key and Secret.
  15. Provide the same API key and secret created in the previous step.
  16. Once all details are provided, you can try Test Connection and then Create if Test Connection is successful.


Update Project Setting

  1. Open the Sprinkle console and choose Admin and then Projects from the Left Panel.
  2. Choose the Project list and select the newly created Compute and Notebook from the drop down menu.
  3. Choose Save to update the settings.

import requests
from requests.auth import HTTPBasicAuth

auth =  HTTPBasicAuth(<API_KEY>, <API_SECRET>)
response = requests.get("https://<hostname>/api/v0.4/explore/streamresult/<EXPLORE_ID>", auth)

print(response.content)

library('httr')

username = '<API KEY>'
password = '<API SECRET>'

temp = GET("https://<hostname>/api/v0.4/explore/streamresult/<EXPLORE ID>",
           authenticate(username,password, type = "basic"))

temp = content(temp, 'text')
temp = textConnection(temp)
temp = read.csv(temp)

/*Download the Data*/

filename resp temp;
proc http
url="https://<hostname>/api/v0.4/explore/streamresult/<EXPLORE ID>"
   method= "GET"  
   WEBUSERNAME = "<API KEY>"
   WEBPASSWORD = "<API SECRET>"
   out=resp;
run;

/*Import the data in to csv dataset*/
proc import
   file=resp
   out=csvresp
   dbms=csv;
run;

/*Print the data */
PROC PRINT DATA=csvresp;
RUN;

import requests
import json

url='http://hostname/api/v0.4/createCSV'

username='API_KEY'
password='API_SECRET'

files={'file':open('FILE_PATH.csv','rb')}
values={'projectname':PROJECT_NAME','name':'CSV_DATASOURCE_NAME'}

r=requests.post(url, files=files, data=values, auth=(username,password))

res_json=json.loads(r.text)

print(res_json['success'])

import requests
import json

url='http://hostname/api/v0.4/updateCSV'

username='API_KEY'
password='API_SECRET'

files={'file':open('FILE_PATH.csv','rb')}
values={'projectname':PROJECT_NAME','name':'CSV_DATASOURCE_NAME'}

r=requests.post(url, files=files, data=values, auth=(username,password))

res_json=json.loads(r.text)

print(res_json['success'])

import requests

url='https://<hostname>/api/v0.4/explore/streamresult/<EXPLORE ID>'

username='API_KEY'
password='API_SECRET'

r=requests.get(url,auth=(username,password))
print(r)
print(r.text)

import requests
import pandas as pd

url='https://<hostname>/api/v0.4/explores/infoByFolder/<SPACE_ID>'

username='API_KEY'
password='API_SECRET'

r=requests.get(url,auth=(username,password)).json()
df = pd.DataFrame(r)
print(df)

import requests
import pandas as pd

url='https://<hostname>/api/v0.4/folders/byOrgName/<ORG_NAME>'

username='API_KEY'
password='API_SECRET'

r=requests.get(url,auth=(username,password)).json()
df = pd.DataFrame(r)
print(df.loc[:,['name','id']])

import requests

import pandas as pd

import io

url='https://<hostname>/api/v0.4/explore/streamresult/<EXPLORE ID>'

secret='API_SECRET'

r=requests.get(url,headers = {'Authorization': 'SprinkleUserKeys ' +secret } )

df = pd.read_csv(io.StringIO(r.text),sep=',')

import requests

import pandas as pd

import io

url='https://<hostname>/api/v0.4/segment/streamresult/<SEGMENT ID>'

secret='API_SECRET'

r=requests.get(url,headers = {'Authorization': 'SprinkleUserKeys ' +secret } )

df = pd.read_csv(io.StringIO(r.text),sep=',')

import requests

import json

url='http://hostname/api/v.o4/createCSV'

files={'file':open('path/file.csv’')}

values={'projectname':PROJECT_NAME,'name':'csv_datasource_name/table_name'}

secret='API_SECRET'

r=requests.post(url, files=files, data=values, headers = {'Authorization': 'SprinkleUserKeys ' +secret } )

res_json=json.loads(r.text)

import requests

import json

url='http://hostname/api/v.o4/updateCSV'

files={'file':open('path/file.csv’')}

values={'projectname':PROJECT_NAME,'name':'csv_datasource_name/table_name'}

secret='API_SECRET'

r=requests.post(url, files=files, data=values,headers = {'Authorization': 'SprinkleUserKeys ' +secret } )

res_json=json.loads(r.text)