das.helpers package

Subpackages

das.helpers.schemas package

Submodules

das.helpers.registration module

class das.helpers.registration.AzureFunction(name, description, help_text, trigger_url, credential, request_contains_timestamps=False, response_contains_timestamps=False, method='POST')

Bases: BaseService

AzureFunction class.

Class represents an Azure function deployed in function-app and can be used to trigger the function as it would be triggered from IoT-TICKET.

Attributes

base_body: Property method to get the base information from this class.
input_schema
output_schema

Methods

`as_dict`()	Method to get the registration class as a dict.
`authentication`()	Returns client_id or headers that can be used for authentication.
`from_json`(value, credential)	Instantiates an AzureFunction object from given value.
`try_out`(data)	Try out method.
`try_out_infer`(input_data[, inplace])	Try_out_infer method.
`validate_url`(value)	Url validator.

classmethod from_json(value, credential)

Instantiates an AzureFunction object from given value. Designed to accept JSON object from Azure Function API/related endpoint (see example).

Parameters

valuedict: Content to be used to instantiate the object. Content will be validated against the job schema (inherited from DB jobs api version 2.1).
credentialclass instance from das.credentials: Credential and token provider object.

Returns

AzureFunction object: Instantiated object.

Examples

>>> from das import AzureFunctionsClient, AzureFunction
>>> from das.credential import AzureFunctionCredential
>>> client = AzureFunctionsClient()
>>> cred = AzureFunctionCredential()
>>> func_list = client.list_functions()['value']
>>> # ... find proper function ...
>>> function = AzureFunction.from_json(function_list[0], cred)

try_out(data)

Try out method. Invoke related Azuren Function endpoint with given data. Designed to mimic the request that IoT-TICKET will send to Azure Function after triggering.

Parameters

datadict or list: Invocation data to be send inside request payload.

Returns

requests.Response object: Response from the corresponding Azure Function.

class das.helpers.registration.BaseRegistration(name, description, help_text, trigger_url, credential, method='POST')

Bases: object

Base class for registrating services and models to IoT-TICKET, not mentioned to be used directly.

Attributes

base_body: Property method to get the base information from this class.
type

Methods

`authentication`()	Returns client_id or headers that can be used for authentication.
`validate_url`(value)	Url validator.

accepted_methods = ['GET', 'POST', 'PUT']

authentication(): Returns client_id or headers that can be used for authentication.

property base_body

Property method to get the base information from this class.

Returns

base_bodydict: Containing name, description, help_text, trigger_url, method, type and either headers or client_id for authentication

type = None

validate_url(value)

Url validator. Validates the given url, bases on https://stackoverflow.com/questions/7160737/how-to-validate-a-url-in-python-malformed-or-not

Parameters

value: str: Value of the url.

Returns

ret_val: bool: True if url is valid, else False.

class das.helpers.registration.BaseService(name, description, help_text, trigger_url, credential, request_contains_timestamps=False, response_contains_timestamps=False, method='POST')

Bases: BaseRegistration

Base class for registrating services to IoT-TICKET class, not intended to be used directly.

Attributes

base_body: Property method to get the base information from this class.
input_schema
output_schema

Methods

`as_dict`()	Method to get the registration class as a dict.
`authentication`()	Returns client_id or headers that can be used for authentication.
`try_out_infer`(input_data[, inplace])	Try_out_infer method.
`validate_url`(value)	Url validator.

try_out

as_dict()

Method to get the registration class as a dict.

Returns

dict: Containing the keys name, description, help_text, trigger_url, access_token, method, request_contains_timestamps, response_contains_timestamps, input_schema, output_schema and client_id or headers.

property base_body

Property method to get the base information from this class.

Returns

base_bodydict: Containing name, description, help_text, trigger_url, access_token, method, request_contains_timestamps and response_contains_timestamps and client_id or headers.

property input_schema

property output_schema

try_out(data)

try_out_infer(input_data, inplace=False)

Try_out_infer method. Calls try_out method and tries to infer schema from both input_data and output_data. Output_data is the response gotten from the service. Designed to be used to create input_schema and output_schema.

Parameters

input_datadict or list: Data to be sent to the service.
inplacebool: Whether inferred schemas should be set to attributes of not. If True then no value is returned.

Returns

dict: input_schema and output_schema inside a dict (if inplace is False).

type = 'service'

class das.helpers.registration.DatabricksJob(name, description, help_text, trigger_url, credential, job_id)

Bases: BaseRegistration

DatabricksJob class.

Class represents a job Databricks’ workflow view and can be used to trigger job as it would be triggered from IoT-TICKET.

Attributes

base_body: Property method to get the base information from this class.
notebook_inputs
notebook_outputs
notebook_params

Methods

`as_dict`()	Method to get the registration class as a dict.
`authentication`()	Returns client_id or headers that can be used for authentication.
`from_json`(value, credential)	Instantiates a DatabricksJob object from given value.
`try_out`([notebook_params, jar_params, ...])	Try out method.
`validate_url`(value)	Url validator.

as_dict()

Method to get the registration class as a dict.

Returns

dict: Containing the keys name, description, help_text, trigger_url, access_token, method, job_id, client_id, inputs and outputs.

property base_body

Property method to get the base information from this class.

Returns

base_bodydict: Containing the keys name, description, help_text, trigger_url, access_token, method, job_id and client_id.

classmethod from_json(value, credential)

Instantiates a DatabricksJob object from given value. Designed to accept JSON object from Databricks Jobs API (see example).

Parameters

valuedict: Content to be used to instantiate the object. Content will be validated against the job schema (inherited from DB jobs api version 2.1).
credentialclass instance from das.credentials: Credential and token provider object.

Returns

ret_val: DatabricksJob object: Instantiated object.

Examples

>>> from das import IoTClient, DatabricksJob
>>> from das.credential import AzureDatabricksCredential
>>> client = IoTClient()
>>> cred = AzureDatabricksCredential()
>>> job_list = client.db.list_jobs()
>>> # ... find proper job ...
>>> job = DatabricksJob.from_json(job_list[0], cred)

property notebook_inputs

property notebook_outputs

property notebook_params

try_out(notebook_params=None, jar_params=None, python_params=None, spark_submit_params=None, python_named_params=None, pipeline_params=None, sql_params=None, dbt_commands=None)

Try out method. Validates (asserts) that given parameters matches the structure of corresponding parameters and sends requests to Databricks REST API jobs/run-now endpoint. Designed to mimic the request that IoT-TICKET will send to Databricks after triggering.

Parameters

notebook_paramslist
jar_params:
python_params:
spark_submit_params:
python_named_params:
pipeline_params:
sql_params:
dbt_commands:

Returns

requests.Response object: Response from the Databricks REST API.

type = 'job'

class das.helpers.registration.DatabricksServing(name, description, help_text, trigger_url, credential, request_contains_timestamps=False, response_contains_timestamps=False, method='POST')

Bases: BaseService

DatabricksServing class.

Class represents an exposed Databricks model (from MLflow registry) and can be used to trigger the model as it would be triggered from IoT-TICKET.

Attributes

base_body: Property method to get the base information from this class.
input_schema
output_schema

Methods

`as_dict`()	Method to get the registration class as a dict.
`authentication`()	Returns client_id or headers that can be used for authentication.
`try_out`(data)	Try out method.
`try_out_infer`(input_data[, inplace])	Try_out_infer method.
`validate_url`(value)	Url validator.

from_json

classmethod from_json(value, cred)

try_out(data)

Try out method. Invoke related Databricks model serving endpoint with given data. Designed to mimic the request that IoT-TICKET will send to Databricks model serving after triggering.

Parameters

datadict or list: Invocation data to be send inside request payload.

Returns

requests.Response object: Response from the corresponding model serving endpoint.

das.helpers.registration.infer_schema(data)

Generates a schema from given data. Helper function for instantiating service objects’ schema from example data.

Parameters

datadict, list or pandas.DataFrame: Data from which the schema is built.

Returns

schemadict: Generated schema.

Raises

TypeError: If the given data type is not accepted.

das.helpers package

Subpackages

Submodules

das.helpers.registration module

Module contents