airt.client
APIKey
¤
A class for managing the APIKeys in the server.
Both the APIKey and the token can be used for accessing the airt services. However, there is a slight difference in generating and managing the two.
For generating the APIKey, you first need to get the developer token. Please refer to Client.get_token method documentation to generate one.
After logging in with your developer token, you can create any number of new APIKeys and can set an expiration date individually. You can also access other methods available in the APIKey class to list, revoke the APIKey at any time.
Here's an example of how to use the APIKey class to create a new key and use it to access the airt service.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a new key with the given name
key_name = "{fill in key_name}"
new_key = APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# Call the set_token method to set the newly generated key
Client.set_token(token=new_key["access_token"])
# Print the logged-in user details
# If set_token fails, the line below will throw an error.
print(User.details())
__init__(self, uuid, name=None, expiry=None, disabled=None, created=None)
special
¤
Constructs a new APIKey instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
APIKey uuid. |
required |
name |
Optional[str] |
APIKey name. |
None |
expiry |
Optional[str] |
APIKey expiry date. |
None |
disabled |
Optional[bool] |
Flag to indicate the status of the APIKey. |
None |
created |
Optional[str] |
APIKey creation date. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
name: Optional[str] = None,
expiry: Optional[str] = None,
disabled: Optional[bool] = None,
created: Optional[str] = None,
):
"""Constructs a new APIKey instance.
Args:
uuid: APIKey uuid.
name: APIKey name.
expiry: APIKey expiry date.
disabled: Flag to indicate the status of the APIKey.
created: APIKey creation date.
"""
self.uuid = uuid
self.name = name
self.expiry = expiry
self.disabled = disabled
self.created = created
as_df(ax)
staticmethod
¤
Return the details of APIKey instances in a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ax |
List[APIKey] |
List of APIKey instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the APIKeys in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
An example of displaying the APIKeys generated by the currently logged-in user in a dataframe
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display all the APIKey instance details in a pandas dataframe
df = APIKey.as_df(APIKey.ls())
print(df)
Source code in airt/client.py
@staticmethod
def as_df(ax: List["APIKey"]) -> pd.DataFrame:
"""Return the details of APIKey instances in a pandas dataframe.
Args:
ax: List of APIKey instances.
Returns:
Details of all the APIKeys in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
An example of displaying the APIKeys generated by the currently logged-in user in a dataframe
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display all the APIKey instance details in a pandas dataframe
df = APIKey.as_df(APIKey.ls())
print(df)
```
"""
lists = get_attributes_from_instances(ax, APIKey.API_KEY_COLS) # type: ignore
return generate_df(lists, APIKey.API_KEY_COLS)
create(name, expiry=None, otp=None)
staticmethod
¤
Create a new APIKey
In order to access the airt service with the newly generated APIKey, please call the Client.set_token method
or set the APIKey value in the AIRT_SERVICE_TOKEN environment variable.
Note
-
The APIKey's name must be unique. If not, an exception will be raised while creating a new key with the name of an existing key. However, you can create a new key with the name of a revoked key.
-
The expiry for an APIKey is optional, if not passed then the default value None will be used to create an APIKey with no expiry date!
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name |
str |
The name of the APIKey. |
required |
expiry |
Union[int, datetime.timedelta, datetime.datetime] |
The validity for the APIKey. This can be an integer representing the number of days till expiry, can be an instance of timedelta (timedelta(days=x)) or can be an instance of datetime. If not passed, then the default value None will be used to create a APIKey that will never expire! |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if the MFA is enabled for your account. |
None |
Returns:
| Type | Description |
|---|---|
Dict[str, str] |
The APIKey and its type as a dictionary. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the user is not authenticated. |
ValueError |
If the user tries to create a new APIKey with an existing key name. |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
In the following example, a new APIKey is created with a 10-day expiration date and used to access the airt service.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key with the given name and set the expiry to 10 days from now.
# If the expiry parameter is not specified, a key with no expiry date is created.
key_name = "{fill in key_name}"
new_key_details = APIKey.create(name=key_name, expiry=10)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# If a new key with the same name is created, an exception will be raised.
# However, you can create a new key with the name of a revoked key.
try:
APIKey.create(name=key_name, expiry=10)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
# Finally, either call the below method to set the newly generated key
# or store it in the AIRT_SERVICE_TOKEN environment variable.
Client.set_token(token=new_key_details["access_token"])
# If set_token fails, the line below will throw an error.
print(APIKey.details(apikey=key_name))
Source code in airt/client.py
@staticmethod
def create(
name: str,
expiry: Optional[Union[int, timedelta, datetime]] = None,
otp: Optional[str] = None,
) -> Dict[str, str]:
"""Create a new APIKey
In order to access the airt service with the newly generated APIKey, please call the `Client.set_token` method
or set the APIKey value in the **AIRT_SERVICE_TOKEN** environment variable.
!!! note
- The APIKey's name must be unique. If not, an exception will be raised while creating a new key with the name of an existing key.
However, you can create a new key with the name of a revoked key.
- The expiry for an APIKey is optional, if not passed then the default value **None** will be used to create an APIKey with no expiry date!
Args:
name: The name of the APIKey.
expiry: The validity for the APIKey. This can be an integer representing the number of days till expiry, can be
an instance of timedelta (timedelta(days=x)) or can be an instance of datetime. If not passed, then the default value
**None** will be used to create a APIKey that will never expire!
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if the MFA is enabled for your account.
Returns:
The APIKey and its type as a dictionary.
Raises:
ValueError: If the user is not authenticated.
ValueError: If the user tries to create a new APIKey with an existing key name.
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
In the following example, a new APIKey is created with a 10-day expiration date and used to access the airt service.
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key with the given name and set the expiry to 10 days from now.
# If the expiry parameter is not specified, a key with no expiry date is created.
key_name = "{fill in key_name}"
new_key_details = APIKey.create(name=key_name, expiry=10)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# If a new key with the same name is created, an exception will be raised.
# However, you can create a new key with the name of a revoked key.
try:
APIKey.create(name=key_name, expiry=10)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
# Finally, either call the below method to set the newly generated key
# or store it in the AIRT_SERVICE_TOKEN environment variable.
Client.set_token(token=new_key_details["access_token"])
# If set_token fails, the line below will throw an error.
print(APIKey.details(apikey=key_name))
```
"""
if expiry is None:
expiry_date = expiry
else:
if isinstance(expiry, int):
delta = datetime.now() + timedelta(days=expiry)
elif isinstance(expiry, timedelta):
delta = datetime.now() + expiry
else:
delta = expiry
expiry_date = delta.strftime("%Y-%m-%dT%H:%M")
return Client._post_data(
relative_url="/apikey",
json=dict(name=name, expiry=expiry_date, otp=otp),
)
details(apikey)
staticmethod
¤
Return details of an APIKey.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
apikey |
str |
APIKey uuid/name. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas Dataframe encapsulating the details of the APIKey. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the APIKey uuid is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
An example to get details of an APIKey
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# To display the details of all keys created by the user, use the method below.
df = APIKey.as_df(APIKey.ls())
print(df)
Source code in airt/client.py
@staticmethod
def details(apikey: str) -> pd.DataFrame:
"""Return details of an APIKey.
Args:
apikey: APIKey uuid/name.
Returns:
A pandas Dataframe encapsulating the details of the APIKey.
Raises:
ValueError: If the APIKey uuid is invalid.
ConnectionError: If the server address is invalid or not reachable.
An example to get details of an APIKey
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# To display the details of all keys created by the user, use the method below.
df = APIKey.as_df(APIKey.ls())
print(df)
```
"""
details = Client._get_data(relative_url=f"/apikey/{apikey}")
return pd.DataFrame(details, index=[0])[APIKey.API_KEY_COLS]
ls(user=None, offset=0, limit=100, include_disabled=False)
staticmethod
¤
Return the list of APIKeys instances.
Please do not pass the user parameter unless you are a super user. Only a super user can view the APIKeys created by other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Optional[str] |
user_uuid/username associated with the APIKey. Please call |
None |
offset |
int |
The number of APIKeys to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of APIKeys to return from the server. If None, then the default value 100 will be used. |
100 |
include_disabled |
bool |
If set to True, then the disabled APIKeys will also be included in the result. |
False |
Returns:
| Type | Description |
|---|---|
List[APIKey] |
A list of APIKey instances. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
ValueError |
If the user_uuid is invalid. |
An example of displaying the APIKeys generated by the currently logged-in user
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Get the list of all APIKey instances created by the currently logged-in user.
# If you are a super user, you can view the APIkeys created by other users by
# passing their uuid/username in the user parameter.
ax = APIKey.ls()
print(ax)
# Display the details of the instances in a pandas dataframe
df = APIKey.as_df(ax)
print(df)
Source code in airt/client.py
@staticmethod
def ls(
user: Optional[str] = None,
offset: int = 0,
limit: int = 100,
include_disabled: bool = False,
) -> List["APIKey"]:
"""Return the list of APIKeys instances.
Please do not pass the **user** parameter unless you are a super user. Only a super user can view
the APIKeys created by other users.
Args:
user: user_uuid/username associated with the APIKey. Please call `User.details` method of the User class to get your user_uuid.
If not passed, then the currently logged-in user_uuid will be used.
offset: The number of APIKeys to offset at the beginning. If None, then the default value 0 will be used.
limit: The maximum number of APIKeys to return from the server. If None, then the default value 100 will be used.
include_disabled: If set to **True**, then the disabled APIKeys will also be included in the result.
Returns:
A list of APIKey instances.
Raises:
ConnectionError: If the server address is invalid or not reachable.
ValueError: If the user_uuid is invalid.
An example of displaying the APIKeys generated by the currently logged-in user
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Get the list of all APIKey instances created by the currently logged-in user.
# If you are a super user, you can view the APIkeys created by other users by
# passing their uuid/username in the user parameter.
ax = APIKey.ls()
print(ax)
# Display the details of the instances in a pandas dataframe
df = APIKey.as_df(ax)
print(df)
```
"""
user_uuid = User.details(user=user)["uuid"]
apikeys = Client._get_data(
relative_url=f"/{user_uuid}/apikey?include_disabled={include_disabled}&offset={offset}&limit={limit}"
)
ax = [
APIKey(
uuid=apikey["uuid"],
name=apikey["name"],
expiry=apikey["expiry"],
disabled=apikey["disabled"],
created=apikey["created"],
)
for apikey in apikeys
]
return ax
revoke(keys, user=None, otp=None)
staticmethod
¤
Revoke one or more APIKeys
Please do not pass the user parameter unless you are a super user. Only a super user can revoke the APIKeys created by other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keys |
Union[str, List[str], List[APIKey]] |
APIKey uuid/name to revoke. To revoke multiple keys, either pass a list of APIKey uuid/names or a list of APIKey instances. |
required |
user |
Optional[str] |
user_uuid/username associated with the APIKey. Please call |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if the MFA is enabled for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas Dataframe encapsulating the details of the deleted APIKey(s). |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the APIKey uuid is invalid. |
ValueError |
If the user_uuid is invalid. |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
An example to revoke a single APIKey by name
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Check that the newly created key exists
print([key.name for key in APIKey.ls()])
# Revoke the newly created key
# To delete multiple keys, pass a list of key names or key instances
APIKey.revoke(keys=key_name)
# Check that the newly created key does not exists
print([key.name for key in APIKey.ls()])
Here's an example of a super user revoking all APIkeys generated by a specific user.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# List the APIKeys generated by a specific user
user = "{fill in other_username}"
ax = APIKey.ls(user=user)
print([key.name for key in ax])
# Revoke the APIKeys
APIKey.revoke(keys=ax, user=user)
# Check that all APIkeys have been revoked
print([key.name for key in APIKey.ls(user=user)])
Source code in airt/client.py
@staticmethod
def revoke(
keys: Union[str, List[str], List["APIKey"]],
user: Optional[str] = None,
otp: Optional[str] = None,
) -> pd.DataFrame:
"""Revoke one or more APIKeys
Please do not pass the **user** parameter unless you are a super user. Only a super user can revoke the
APIKeys created by other users.
Args:
keys: APIKey uuid/name to revoke. To revoke multiple keys, either pass a list of APIKey uuid/names or a list of APIKey instances.
user: user_uuid/username associated with the APIKey. Please call `User.details` method of the User class to get your user_uuid/username.
If not passed, then the currently logged-in user will be used.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if the MFA is enabled for your account.
Returns:
A pandas Dataframe encapsulating the details of the deleted APIKey(s).
Raises:
ValueError: If the APIKey uuid is invalid.
ValueError: If the user_uuid is invalid.
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
An example to revoke a single APIKey by name
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Check that the newly created key exists
print([key.name for key in APIKey.ls()])
# Revoke the newly created key
# To delete multiple keys, pass a list of key names or key instances
APIKey.revoke(keys=key_name)
# Check that the newly created key does not exists
print([key.name for key in APIKey.ls()])
```
Here's an example of a super user revoking all APIkeys generated by a specific user.
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# List the APIKeys generated by a specific user
user = "{fill in other_username}"
ax = APIKey.ls(user=user)
print([key.name for key in ax])
# Revoke the APIKeys
APIKey.revoke(keys=ax, user=user)
# Check that all APIkeys have been revoked
print([key.name for key in APIKey.ls(user=user)])
```
"""
user_uuid = User.details(user=user)["uuid"]
_keys = APIKey._get_key_names(keys)
response_list = []
for key_uuid in _keys:
url = f"/{user_uuid}/apikey/{key_uuid}"
response = Client._delete_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
response_list.append(response)
return generate_df(response_list, APIKey.API_KEY_COLS)
Client
¤
A class for authenticating and accessing the airt service.
To access the airt service, you must first create a developer account. To obtain one, please contact us at info@airt.ai.
After successful verification, you will receive an email with the username and password for the developer account.
Once you have the credentials, use them to get an access token by calling get_token method. It is necessary to get
an access token; otherwise, you won't be able to access all of the airt service's APIs. You can either pass the username, password, and server
address as parameters to the get_token method or store them in the environment variables AIRT_SERVICE_USERNAME,
AIRT_SERVICE_PASSWORD, and AIRT_SERVER_URL.
In addition to the regular authentication with credentials, you can also enable multi-factor authentication (MFA) and single sign-on (SSO) for generating tokens.
To help protect your account, we recommend that you enable multi-factor authentication (MFA). MFA provides additional security by requiring you to provide unique verification code (OTP) in addition to your regular sign-in credentials when performing critical operations.
Your account can be configured for MFA in just two easy steps:
-
To begin, you need to enable MFA for your account by calling the
User.enable_mfamethod, which will generate a QR code. You can then scan the QR code with an authenticator app, such as Google Authenticator and follow the on-device instructions to finish the setup in your smartphone. -
Finally, activate MFA for your account by calling
User.activate_mfaand passing the dynamically generated six-digit verification code from your smartphone's authenticator app.
After activating MFA for your account, you must pass the dynamically generated six-digit verification code, along with your username and password,
to the get_token method to generate new tokens.
Single sign-on (SSO) can be enabled for your account in three simple steps:
-
Enable the SSO for a provider by calling the
User.enable_ssomethod with the SSO provider name and an email address. At the moment, we only support "google" and "github" as SSO providers. We intend to support additional SSO providers in future releases. -
Before you can start generating new tokens with SSO, you must first authenticate with the SSO provider. Call the
get_tokenwith the same SSO provider you have enabled in the step above to generate an SSO authorization URL. Please copy and paste it into your preferred browser and complete the authentication process with the SSO provider. -
After successfully authenticating with the SSO provider, call the
set_sso_tokenmethod to generate a new token and use it automatically in all future interactions with the airt server.
Here's an example of how to use the Client class to authenticate and display the details of the currently logged-in user.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
# MFA enabled users must pass the OTP along with the username and password
# to the get_token method.
Client.get_token(username="{fill in username}", password="{fill in password}")
# Print the logged-in user details
print(User.details())
get_token(*, username=None, password=None, server=None, sso_provider=None, otp=None)
classmethod
¤
Get application token for airt service from a username/password pair.
This methods validates the developer credentials and returns an auth token. The returned auth token is implicitly used in all the interactions with the server.
If you've already enabled multi-factor authentication (MFA) for your account, you'll need to pass the dynamically generated six-digit verification code along with your username and password to generate new tokens.
If the token is requested using Single sign-on (SSO), an authorization URL will be returned. Please copy and paste it into your preferred browser and complete the SSO provider authentication within 10 minutes. Otherwise, the SSO login will time out and you will need to re-request the token.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
username |
Optional[str] |
Username for the developer account. If None (default value), then the value from AIRT_SERVICE_USERNAME environment variable is used. |
None |
password |
Optional[str] |
Password for the developer account. If None (default value), then the value from AIRT_SERVICE_PASSWORD environment variable is used. |
None |
server |
Optional[str] |
The airt server uri. If None (default value), then the value from AIRT_SERVER_URL environment variable is used. If the variable is not set as well, then the default public server will be used. Please leave this setting to default unless you are running the service in your own server (please email us to info@airt.ai for that possibility). |
None |
sso_provider |
Optional[str] |
Name of the Single sign-on (SSO) provider. Please pass this parameter only if you have successfully enabled SSO for this provider. At present, the API only supports "google" and "github" as valid SSO providers. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app or the OTP you have received via SMS. |
None |
Returns:
| Type | Description |
|---|---|
Optional[str] |
The authorization url if the token is requested using Single sign-on (SSO). |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the username/password pair does not match. |
ConnectionError |
If the server address is invalid or not reachable. |
KeyError |
If username/password is neither passed as parameters nor stored in environment variables. |
Here's an example of a non-MFA user authenticating and generating a new token
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Print the logged-in user details
print(User.details())
Here's an example of a MFA user authenticating using SMS OTP and generating a new token
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to authenticate
# If you want to use the OTP from the authenticator app, skip this step and
# don't generate an SMS OTP; instead, pass the OTP from the authenticator
# app to the get_token method below
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Print the logged-in user details
print(User.details())
Source code in airt/client.py
@classmethod
def get_token( # type: ignore
cls,
*,
username: Optional[str] = None,
password: Optional[str] = None,
server: Optional[str] = None,
sso_provider: Optional[str] = None,
otp: Optional[str] = None,
) -> Optional[str]:
"""Get application token for airt service from a username/password pair.
This methods validates the developer credentials and returns an auth token. The returned auth
token is implicitly used in all the interactions with the server.
If you've already enabled multi-factor authentication (MFA) for your account, you'll need to
pass the dynamically generated six-digit verification code along with your username and
password to generate new tokens.
If the token is requested using Single sign-on (SSO), an authorization URL will be returned.
Please copy and paste it into your preferred browser and complete the SSO provider
authentication within 10 minutes. Otherwise, the SSO login will time out and you will need
to re-request the token.
Args:
username: Username for the developer account. If None (default value), then the value from
**AIRT_SERVICE_USERNAME** environment variable is used.
password: Password for the developer account. If None (default value), then the value from
**AIRT_SERVICE_PASSWORD** environment variable is used.
server: The airt server uri. If None (default value), then the value from **AIRT_SERVER_URL** environment variable
is used. If the variable is not set as well, then the default public server will be used. Please leave this
setting to default unless you are running the service in your own server (please email us to info@airt.ai
for that possibility).
sso_provider: Name of the Single sign-on (SSO) provider. Please pass this parameter only if you have successfully
enabled SSO for this provider. At present, the API only supports "google" and "github" as valid SSO providers.
otp: Dynamically generated six-digit verification code from the authenticator app or the OTP you have received via SMS.
Returns:
The authorization url if the token is requested using Single sign-on (SSO).
Raises:
ValueError: If the username/password pair does not match.
ConnectionError: If the server address is invalid or not reachable.
KeyError: If username/password is neither passed as parameters nor stored in environment variables.
Here's an example of a non-MFA user authenticating and generating a new token
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Print the logged-in user details
print(User.details())
```
Here's an example of a MFA user authenticating using SMS OTP and generating a new token
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to authenticate
# If you want to use the OTP from the authenticator app, skip this step and
# don't generate an SMS OTP; instead, pass the OTP from the authenticator
# app to the get_token method below
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Print the logged-in user details
print(User.details())
```
"""
cls.server = get_base_url(server)
username, password = _get_credentials(username, password)
if otp is not None:
password = json.dumps({"password": password, "user_otp": otp})
if sso_provider is None:
response = post_data(
url=f"{cls.server}/token",
data=dict(username=username, password=password),
token=None,
)
cls.auth_token = response["access_token"]
else:
response = post_data(
url=f"{cls.server}/sso/initiate",
data=json.dumps( # type: ignore
dict(
username=username, password=password, sso_provider=sso_provider
)
),
token=None,
)
cls.sso_authorization_url = response["authorization_url"]
return cls.sso_authorization_url
set_sso_token()
classmethod
¤
Set the application token generated using Single sign-on (SSO).
The token set using this method will be implicitly used in all the interactions with the server.
Please call this method only if you successfully enabled and completed the login with the Single
sign-on (SSO) provider. If not, please call the get_token method with an appropriate
sso_provider to initiate the SSO authentication.
Here's an example of authenticating with Single sign-on (SSO) using google and setting the newly generated token to interact with the airt service.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Enable single sign-on (SSO) and use google as the provider
sso_provider="google"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
# Authenticate using Single sign-on (SSO)
# To generate a token using SSO, you must first authenticate with the provider.
# The command below will generate an authorization URL for you.
# Please copy and paste it into your preferred browser and complete the
# SSO provider authentication within 10 minutes. Otherwise, the SSO login
# will time out and you will need to call the get_token method again.
sso_url = Client.get_token(sso_provider=sso_provider)
print(sso_url)
# Once the provider authentication is successful, call the below method to
# set the generated token
Client.set_sso_token()
# If set_sso_token fails, the line below will throw an error.
print(User.details())
Source code in airt/client.py
@classmethod
def set_sso_token(cls):
"""Set the application token generated using Single sign-on (SSO).
The token set using this method will be implicitly used in all the interactions with the server.
Please call this method only if you successfully enabled and completed the login with the Single
sign-on (SSO) provider. If not, please call the `get_token` method with an appropriate
sso_provider to initiate the SSO authentication.
Here's an example of authenticating with Single sign-on (SSO) using google and setting the
newly generated token to interact with the airt service.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Enable single sign-on (SSO) and use google as the provider
sso_provider="google"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
# Authenticate using Single sign-on (SSO)
# To generate a token using SSO, you must first authenticate with the provider.
# The command below will generate an authorization URL for you.
# Please copy and paste it into your preferred browser and complete the
# SSO provider authentication within 10 minutes. Otherwise, the SSO login
# will time out and you will need to call the get_token method again.
sso_url = Client.get_token(sso_provider=sso_provider)
print(sso_url)
# Once the provider authentication is successful, call the below method to
# set the generated token
Client.set_sso_token()
# If set_sso_token fails, the line below will throw an error.
print(User.details())
```
"""
quoted_authorization_url = urllib.parse.quote(cls.sso_authorization_url)
response = get_data(
url=f"{cls.server}/sso/token/?authorization_url={quoted_authorization_url}",
token=None,
)
cls.auth_token = response["access_token"]
set_token(token=None, server=None)
classmethod
¤
Set application token for airt service.
If you already have a valid token, you can call this method to set it and use it in all subsequent interactions with the airt server.
Please call this method only if you already have a token. If not, please call the get_token method to generate one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
token |
Optional[str] |
The application token obtained by calling the |
None |
server |
Optional[str] |
The airt server uri. If None (default value), then the value from AIRT_SERVER_URL environment variable is used. If the variable is not set as well, then the default public server will be used. Please leave this setting to default unless you are running the service in your own server (please email us to info@airt.ai for that possibility). |
None |
An example to set an existing token:
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: For demonstration purpose, generate a new token
# When you generate a new token with the get_token method, you do not
# need to explicitly call set_token. It is shown here for demo purposes only.
# Skip this step if you already have a valid token and pass it directly to
# the set_token method below
Client.get_token(username="{fill in username}", password="{fill in password}")
# Setting a valid token
Client.set_token(token=Client.auth_token)
# If set_token fails, the line below will throw an error.
print(User.details())
Source code in airt/client.py
@classmethod
def set_token(cls, token: Optional[str] = None, server: Optional[str] = None):
"""Set application token for airt service.
If you already have a valid token, you can call this method to set it and use it in all
subsequent interactions with the airt server.
Please call this method only if you already have a token. If not, please call the `get_token` method to generate one.
Args:
token: The application token obtained by calling the `get_token` method, or an APIKey obtained by calling
the `APIKey.create` method. If None (default value), then the value from **AIRT_SERVICE_TOKEN** environment variable is used.
server: The airt server uri. If None (default value), then the value from **AIRT_SERVER_URL** environment variable
is used. If the variable is not set as well, then the default public server will be used. Please leave this
setting to default unless you are running the service in your own server (please email us to info@airt.ai
for that possibility).
An example to set an existing token:
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: For demonstration purpose, generate a new token
# When you generate a new token with the get_token method, you do not
# need to explicitly call set_token. It is shown here for demo purposes only.
# Skip this step if you already have a valid token and pass it directly to
# the set_token method below
Client.get_token(username="{fill in username}", password="{fill in password}")
# Setting a valid token
Client.set_token(token=Client.auth_token)
# If set_token fails, the line below will throw an error.
print(User.details())
```
"""
auth_token = token if token is not None else os.environ.get(SERVICE_TOKEN)
if not auth_token:
raise KeyError(
f"The token is neither passed as parameter nor set in the environment variable {SERVICE_TOKEN}."
)
cls.auth_token = auth_token
cls.server = get_base_url(server)
version()
staticmethod
¤
Return the client and server versions.
Returns:
| Type | Description |
|---|---|
dict |
A dict containing the client and server versions. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
An example to get the client and server versions:
Examples:
# Importing necessary libraries
from airt.client import Client
# Get the client and server versions
print(Client.version())
Source code in airt/client.py
@staticmethod
def version() -> dict:
"""Return the client and server versions.
Returns:
A dict containing the client and server versions.
Raises:
ConnectionError: If the server address is invalid or not reachable.
An example to get the client and server versions:
Example:
```python
# Importing necessary libraries
from airt.client import Client
# Get the client and server versions
print(Client.version())
```
"""
response = Client._get_data(relative_url=f"/version")
version = {
# nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import
"client": importlib.import_module(CLIENT_NAME).__version__,
"server": response["airt_service"],
}
return version
DataBlob
¤
A class for importing and processing data from sources such as CSV/parquet files, databases, AWS S3 buckets, and Azure Blob Storage.
Currently, the only way to instantiate the DataBlob class is to call one of the following static methods
from_local, from_mysql, from_clickhouse, from_s3, or from_azure_blob_storage which imports the data in
the parquet file format from:
-
a local CSV/parquet file,
-
a MySql database,
-
a ClickHouse database
-
an AWS S3 bucket, and
-
an Azure Blob Storage respectively.
We intend to support additional databases and storage mediums in future releases.
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
__init__(self, uuid, type=None, source=None, region=None, cloud_provider=None, datasources=None, total_steps=None, completed_steps=None, folder_size=None, disabled=None, pulled_on=None, user=None, tags=None, error=None)
special
¤
Constructs a new DataBlob instance.
Warning
Do not construct this object directly by calling the constructor, please use from_s3, from_azure_blob_storage,
from_mysql, from_clickhouse or from_local methods instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
Datablob uuid. |
required |
source |
Optional[str] |
The URI of the data that was used to create the datablob. |
None |
type |
Optional[str] |
The type of source used to generate the datablob. Depending on the source type, one of the following values will be assigned: "s3", "local", "db", or "azure_blob_storage". |
None |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider. |
None |
cloud_provider |
Optional[str] |
Cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. |
None |
datasources |
Optional[List[str]] |
The uuids of the datasources created from the datablob. |
None |
total_steps |
Optional[int] |
The number of steps required to upload the datablob to the server. |
None |
completed_steps |
Optional[int] |
The number of steps completed during the datablob's upload to the server. |
None |
folder_size |
Optional[int] |
The uploaded datablob's size in bytes. |
None |
disabled |
Optional[bool] |
A flag that indicates the datablob's status. If the datablob is deleted, then False will be set. |
None |
pulled_on |
Optional[str] |
The most recent date the datablob was uploaded. |
None |
user |
Optional[str] |
The uuid of the user who created the datablob. |
None |
tags |
Optional[List] |
Tag names associated with the datablob. |
None |
error |
Optional[str] |
Contains the error message if the processing of the datablob fails. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
type: Optional[str] = None,
source: Optional[str] = None,
region: Optional[str] = None,
cloud_provider: Optional[str] = None,
datasources: Optional[List[str]] = None,
total_steps: Optional[int] = None,
completed_steps: Optional[int] = None,
folder_size: Optional[int] = None,
disabled: Optional[bool] = None,
pulled_on: Optional[str] = None,
user: Optional[str] = None,
tags: Optional[List] = None,
error: Optional[str] = None,
):
"""Constructs a new DataBlob instance.
Warning:
Do not construct this object directly by calling the constructor, please use `from_s3`, `from_azure_blob_storage`,
`from_mysql`, `from_clickhouse` or `from_local` methods instead.
Args:
uuid: Datablob uuid.
source: The URI of the data that was used to create the datablob.
type: The type of source used to generate the datablob. Depending on the source type, one of the following
values will be assigned: "s3", "local", "db", or "azure_blob_storage".
region: The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider.
cloud_provider: Cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
datasources: The uuids of the datasources created from the datablob.
total_steps: The number of steps required to upload the datablob to the server.
completed_steps: The number of steps completed during the datablob's upload to the server.
folder_size: The uploaded datablob's size in bytes.
disabled: A flag that indicates the datablob's status. If the datablob is deleted, then **False** will be set.
pulled_on: The most recent date the datablob was uploaded.
user: The uuid of the user who created the datablob.
tags: Tag names associated with the datablob.
error: Contains the error message if the processing of the datablob fails.
"""
self.uuid = uuid
self.type = type
self.source = source
self.region = region
self.cloud_provider = cloud_provider
self.datasources = datasources
self.total_steps = total_steps
self.completed_steps = completed_steps
self.folder_size = folder_size
self.disabled = disabled
self.pulled_on = pulled_on
self.user = user
self.tags = tags
self.error = error
as_df(dbx)
staticmethod
¤
Return the details of datablob instances as a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dbx |
List[DataBlob] |
List of datablob instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the datablobs in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
@staticmethod
def as_df(dbx: List["DataBlob"]) -> pd.DataFrame:
"""Return the details of datablob instances as a pandas dataframe.
Args:
dbx: List of datablob instances.
Returns:
Details of all the datablobs in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
db_lists = get_attributes_from_instances(dbx, DataBlob.ALL_DB_COLS) # type: ignore
for db in db_lists:
db = DataBlob._get_tag_name_and_datasource_id(db)
lists_df = generate_df(db_lists, DataBlob.BASIC_DB_COLS)
df = add_ready_column(lists_df)
df = df.rename(columns=DataBlob.COLS_TO_RENAME)
return df
delete(self)
¤
Delete a datablob from the server.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the deleted datablob. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
@patch
def delete(self: DataBlob) -> pd.DataFrame:
"""Delete a datablob from the server.
Returns:
A pandas DataFrame encapsulating the details of the deleted datablob.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._delete_data(relative_url=f"/datablob/{self.uuid}")
response = DataBlob._get_tag_name_and_datasource_id(response)
df = pd.DataFrame([response])[DataBlob.BASIC_DB_COLS]
df = df.rename(columns=DataBlob.COLS_TO_RENAME)
return add_ready_column(df)
details(self)
¤
Return details of a datablob.
Returns:
| Type | Description |
|---|---|
DataFrame |
The datablob details as a pandas dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Columns in the resulting dataframe are: uuid, datasources, type, source, region, cloud_provider, tags, pulled_on, completed_steps, total_steps, folder_size, user, error, disabled.
Source code in airt/client.py
@patch
def details(self: DataBlob) -> pd.DataFrame:
"""Return details of a datablob.
Returns:
The datablob details as a pandas dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
details = Client._get_data(relative_url=f"/datablob/{self.uuid}")
details = DataBlob._get_tag_name_and_datasource_id(details)
details_df = pd.DataFrame([details])[DataBlob.ALL_DB_COLS]
details_df = details_df.rename(columns=DataBlob.COLS_TO_RENAME)
return add_ready_column(details_df)
from_azure_blob_storage(uri, credential, cloud_provider=None, region=None, tag=None)
classmethod
¤
Create and return a datablob that encapsulates the data from an Azure Blob Storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri |
str |
Azure Blob Storage URI of the source file. |
required |
credential |
str |
Credential to access the Azure Blob Storage. |
required |
cloud_provider |
Optional[str] |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. If None (default value), then azure will be used as the cloud storage provider. |
None |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider. In the case of aws, eu-west-1 will be used and in the case of azure, westeurope will be used. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
tag |
Optional[str] |
A string to tag the datablob. If not passed, then the tag latest will be assigned to the datablob. |
None |
Returns:
| Type | Description |
|---|---|
DataBlob |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
To create a Datablob from Azure Blob Storage, you must have a valid Azure Blob Storage credential.
If you don't know how to get the Azure Blob Storage credential, you can follow the below python example. It's one of the ways to get the Azure Blob Storage credential.
-
If you don't already have it, please install the Azure Storage Management (azure-mgmt-storage) and Azure Resource Management (azure-mgmt-resource) python client libraries using pip.
-
Ensure the following four environment variables are set into your current working environment with appropriate values.
-
AZURE_TENANT_ID
-
AZURE_CLIENT_ID
-
AZURE_CLIENT_SECRET
-
AZURE_SUBSCRIPTION_ID
-
-
Assign the resource group name in the GROUP_NAME variable and the storage account name in the STORAGE_ACCOUNT_NAME variable.
-
Below is a sample code to create a datablob and storing it in S3. Please copy it and replace the placeholders with appropriate values
Examples:
# Importing necessary libraries
import os
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob
# Create a credential for accessing Azure Blob Storage
# Setting the required environment variables
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
# Setting the resource group name and storage account name
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
# Retrieving the credential
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
credential = azure_storage_keys['key1']
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-1 (default), feel free to change the cloud provider and
# the region to suit your needs.
db = DataBlob.from_azure_blob_storage(
uri="{fill in uri}",
cloud_provider="aws",
credential=credential
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@classmethod
def from_azure_blob_storage(
cls,
uri: str,
credential: str,
cloud_provider: Optional[str] = None,
region: Optional[str] = None,
tag: Optional[str] = None,
) -> "DataBlob":
"""Create and return a datablob that encapsulates the data from an Azure Blob Storage.
Args:
uri: Azure Blob Storage URI of the source file.
credential: Credential to access the Azure Blob Storage.
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
If **None** (default value), then **azure** will be used as the cloud storage provider.
region: The destination cloud provider's region to store the datablob. If **None** (default value) then the default region will be assigned based on the cloud
provider. In the case of **aws**, **eu-west-1** will be used and in the case of **azure**, **westeurope** will be used. The supported AWS regions
are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1,
us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast,
brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast,
japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia,
switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
tag: A string to tag the datablob. If not passed, then the tag **latest** will be assigned to the datablob.
Returns:
An instance of the `DataBlob` class.
Raises:
ValueError: If parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
To create a Datablob from Azure Blob Storage, you must have a valid Azure Blob Storage credential.
If you don't know how to get the Azure Blob Storage credential, you can follow the below python example. It's one of the ways to get the Azure Blob Storage credential.
- If you don't already have it, please install the Azure Storage Management (azure-mgmt-storage) and Azure Resource Management (azure-mgmt-resource) python client libraries using pip.
- Ensure the following four environment variables are set into your current working environment with appropriate values.
- AZURE_TENANT_ID
- AZURE_CLIENT_ID
- AZURE_CLIENT_SECRET
- AZURE_SUBSCRIPTION_ID
- Assign the resource group name in the GROUP_NAME variable and the storage account name in the STORAGE_ACCOUNT_NAME variable.
- Below is a sample code to create a datablob and storing it in S3. Please copy it and replace the placeholders with appropriate values
Example:
```python
# Importing necessary libraries
import os
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob
# Create a credential for accessing Azure Blob Storage
# Setting the required environment variables
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
# Setting the resource group name and storage account name
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
# Retrieving the credential
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
credential = azure_storage_keys['key1']
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-1 (default), feel free to change the cloud provider and
# the region to suit your needs.
db = DataBlob.from_azure_blob_storage(
uri="{fill in uri}",
cloud_provider="aws",
credential=credential
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
cloud_provider, region = DataBlob._get_cloud_provider_and_region(cloud_provider=cloud_provider, region=region, default_cloud_provider="azure") # type: ignore
response = Client._post_data(
relative_url="/datablob/from_azure_blob_storage",
json=dict(
uri=uri,
credential=credential,
region=region,
cloud_provider=cloud_provider,
tag=tag,
),
)
return DataBlob(
uuid=response["uuid"], type=response["type"], source=response["source"]
)
from_clickhouse(*, host, database, table, protocol, index_column, timestamp_column, port=0, cloud_provider=None, region=None, username=None, password=None, filters=None, tag=None)
staticmethod
¤
Create and return a datablob that encapsulates the data from a ClickHouse database.
If the database requires authentication, pass the username/password as parameters or store it in the CLICKHOUSE_USERNAME and CLICKHOUSE_PASSWORD environment variables.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
host |
str |
Remote database host name. |
required |
database |
str |
Database name. |
required |
table |
str |
Table name. |
required |
protocol |
str |
Protocol to use. The valid values are "native" and "http". |
required |
index_column |
str |
The column to use as index (row labels). |
required |
timestamp_column |
str |
Timestamp column name in the tabel. |
required |
port |
int |
Host port number. If not passed, then the default value 0 will be used. |
0 |
cloud_provider |
Optional[str] |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. If None (default value), then aws will be used as the cloud storage provider. |
None |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider. In the case of aws, eu-west-1 will be used and in the case of azure, westeurope will be used. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
username |
Optional[str] |
Database username. If not passed, the default value "root" will be used unless the value is explicitly set in the environment variable CLICKHOUSE_USERNAME. |
None |
password |
Optional[str] |
Database password. If not passed, the default value "root" will be used unless the value is explicitly set in the environment variable CLICKHOUSE_PASSWORD. |
None |
filters |
Optional[Dict[str, Any]] |
Additional parameters to be used when importing data. For example, if you want to filter and extract data only for a specific user_id, pass {"user_id": 1}. |
None |
tag |
Optional[str] |
A string to tag the datablob. If not passed, then the tag latest will be assigned to the datablob. |
None |
Returns:
| Type | Description |
|---|---|
DataBlob |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of how to create a Datablob from a ClickHouse database:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_clickhouse(
username="{fill in database_username}",
password="{fill in database_password}",
host="{fill in host}",
database="{fill in database}",
table="{fill in table}",
index_column="{fill in index_column}",
timestamp_column="{fill in timestamp_column}",
port="{fill in port}",
filters={fill in filters},
protocol="native",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@staticmethod
def from_clickhouse(
*,
host: str,
database: str,
table: str,
protocol: str,
index_column: str,
timestamp_column: str,
port: int = 0,
cloud_provider: Optional[str] = None,
region: Optional[str] = None,
username: Optional[str] = None,
password: Optional[str] = None,
filters: Optional[Dict[str, Any]] = None,
tag: Optional[str] = None,
) -> "DataBlob":
"""Create and return a datablob that encapsulates the data from a ClickHouse database.
If the database requires authentication, pass the username/password as parameters or store it in
the **CLICKHOUSE_USERNAME** and **CLICKHOUSE_PASSWORD** environment variables.
Args:
host: Remote database host name.
database: Database name.
table: Table name.
protocol: Protocol to use. The valid values are "native" and "http".
index_column: The column to use as index (row labels).
timestamp_column: Timestamp column name in the tabel.
port: Host port number. If not passed, then the default value **0** will be used.
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
If **None** (default value), then **aws** will be used as the cloud storage provider.
region: The destination cloud provider's region to store the datablob. If **None** (default value) then the default region will be assigned based on the cloud
provider. In the case of **aws**, **eu-west-1** will be used and in the case of **azure**, **westeurope** will be used. The supported AWS regions
are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1,
us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast,
brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast,
japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia,
switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
username: Database username. If not passed, the default value "root" will be used unless the value is explicitly set in the environment variable
**CLICKHOUSE_USERNAME**.
password: Database password. If not passed, the default value "root" will be used unless the value is explicitly set in the environment variable
**CLICKHOUSE_PASSWORD**.
filters: Additional parameters to be used when importing data. For example, if you want to filter and extract data only for a specific user_id, pass {"user_id": 1}.
tag: A string to tag the datablob. If not passed, then the tag **latest** will be assigned to the datablob.
Returns:
An instance of the `DataBlob` class.
Raises:
ValueError: If parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of how to create a Datablob from a ClickHouse database:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_clickhouse(
username="{fill in database_username}",
password="{fill in database_password}",
host="{fill in host}",
database="{fill in database}",
table="{fill in table}",
index_column="{fill in index_column}",
timestamp_column="{fill in timestamp_column}",
port="{fill in port}",
filters={fill in filters},
protocol="native",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
username = (
username
if username is not None
else os.environ.get("CLICKHOUSE_USERNAME", "root")
)
password = (
password
if password is not None
else os.environ.get("CLICKHOUSE_PASSWORD", "")
)
cloud_provider, region = DataBlob._get_cloud_provider_and_region(cloud_provider, region) # type: ignore
json_req = dict(
host=host,
database=database,
table=table,
protocol=protocol,
port=port,
username=username,
password=password,
index_column=index_column,
timestamp_column=timestamp_column,
filters=filters,
region=region,
cloud_provider=cloud_provider,
tag=tag,
)
response = Client._post_data(
relative_url=f"/datablob/from_clickhouse", json=json_req
)
return DataBlob(
uuid=response["uuid"], type=response["type"], source=response["source"]
)
from_local(path, cloud_provider=None, region=None, tag=None, show_progress=True)
staticmethod
¤
Create and return a datablob from local file.
The API currently allows users to create datablobs from CSV or Parquet files. We intend to support additional file formats in future releases.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path |
Union[str, pathlib.Path] |
The relative or absolute path to a local file or to a directory containing the source files. |
required |
cloud_provider |
Optional[str] |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. If None (default value), then aws will be used as the cloud storage provider. |
None |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider. In the case of aws, eu-west-1 will be used and in the case of azure, westeurope will be used. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
tag |
Optional[str] |
A string to tag the datablob. If not passed, then the tag latest will be assigned to the datablob. |
None |
show_progress |
Optional[bool] |
Flag to set the progressbar visibility. If not passed, then the default value True will be used. |
True |
Returns:
| Type | Description |
|---|---|
DataBlob |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of how to create a Datablob from a local file:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_local(
path="{fill in path}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@staticmethod
def from_local(
path: Union[str, Path],
cloud_provider: Optional[str] = None,
region: Optional[str] = None,
tag: Optional[str] = None,
show_progress: Optional[bool] = True,
) -> "DataBlob":
"""Create and return a datablob from local file.
The API currently allows users to create datablobs from CSV or Parquet files. We intend to support additional file formats in future releases.
Args:
path: The relative or absolute path to a local file or to a directory containing the source files.
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
If **None** (default value), then **aws** will be used as the cloud storage provider.
region: The destination cloud provider's region to store the datablob. If **None** (default value) then the default region will be assigned based on the cloud
provider. In the case of **aws**, **eu-west-1** will be used and in the case of **azure**, **westeurope** will be used. The supported AWS regions
are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1,
us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast,
brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast,
japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia,
switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
tag: A string to tag the datablob. If not passed, then the tag **latest** will be assigned to the datablob.
show_progress: Flag to set the progressbar visibility. If not passed, then the default value **True** will be used.
Returns:
An instance of the `DataBlob` class.
Raises:
ValueError: If parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of how to create a Datablob from a local file:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_local(
path="{fill in path}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
path = Path(path)
cloud_provider, region = DataBlob._get_cloud_provider_and_region(cloud_provider, region) # type: ignore
# Step 1: get presigned URL
_path = f"local:{str(path)}"
response = Client._post_data(
relative_url=f"/datablob/from_local/start",
json=dict(
path=_path, region=region, cloud_provider=cloud_provider, tag=tag
),
)
# Step 2: download the csv to the s3 bucket
files = list(path.glob("*")) if path.is_dir() else [path]
# Initiate progress bar
t = tqdm(total=len(files), disable=not show_progress)
for file_to_upload in files:
DataBlob._upload_to_s3_with_retry(
file_to_upload=file_to_upload,
presigned_url=response["presigned"]["url"],
presigned_fields=response["presigned"]["fields"],
)
t.update()
t.close()
return DataBlob(uuid=response["uuid"], type=response["type"])
from_mysql(*, host, database, table, port=3306, cloud_provider=None, region=None, username=None, password=None, tag=None)
staticmethod
¤
Create and return a datablob that encapsulates the data from a mysql database.
If the database requires authentication, pass the username/password as parameters or store it in the AIRT_CLIENT_DB_USERNAME and AIRT_CLIENT_DB_PASSWORD environment variables.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
host |
str |
Remote database host name. |
required |
database |
str |
Database name. |
required |
table |
str |
Table name. |
required |
port |
int |
Host port number. If not passed, then the default value 3306 will be used. |
3306 |
cloud_provider |
Optional[str] |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. If None (default value), then aws will be used as the cloud storage provider. |
None |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. If None (default value) then the default region will be assigned based on the cloud provider. In the case of aws, eu-west-1 will be used and in the case of azure, westeurope will be used. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
username |
Optional[str] |
Database username. If not passed, the default value "root" will be used unless the value is explicitly set in the environment variable AIRT_CLIENT_DB_USERNAME. |
None |
password |
Optional[str] |
Database password. If not passed, the default value "" will be used unless the value is explicitly set in the environment variable AIRT_CLIENT_DB_PASSWORD. |
None |
tag |
Optional[str] |
A string to tag the datablob. If not passed, then the tag latest will be assigned to the datablob. |
None |
Returns:
| Type | Description |
|---|---|
DataBlob |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of how to create a Datablob from a MySQL database:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_mysql(
username="{fill in database_username}",
password="{fill in database_password}",
host="{fill in host}",
database="{fill in database}",
table="{fill in table}",
port="{fill in port}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@staticmethod
def from_mysql(
*,
host: str,
database: str,
table: str,
port: int = 3306,
cloud_provider: Optional[str] = None,
region: Optional[str] = None,
username: Optional[str] = None,
password: Optional[str] = None,
tag: Optional[str] = None,
) -> "DataBlob":
"""Create and return a datablob that encapsulates the data from a mysql database.
If the database requires authentication, pass the username/password as parameters or store it in
the **AIRT_CLIENT_DB_USERNAME** and **AIRT_CLIENT_DB_PASSWORD** environment variables.
Args:
host: Remote database host name.
database: Database name.
table: Table name.
port: Host port number. If not passed, then the default value **3306** will be used.
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
If **None** (default value), then **aws** will be used as the cloud storage provider.
region: The destination cloud provider's region to store the datablob. If **None** (default value) then the default region will be assigned based on the cloud
provider. In the case of **aws**, **eu-west-1** will be used and in the case of **azure**, **westeurope** will be used. The supported AWS regions
are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1,
us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast,
brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast,
japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia,
switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
username: Database username. If not passed, the default value **"root"** will be used unless the value is explicitly set in the environment variable
**AIRT_CLIENT_DB_USERNAME**.
password: Database password. If not passed, the default value **""** will be used unless the value is explicitly set in the environment variable
**AIRT_CLIENT_DB_PASSWORD**.
tag: A string to tag the datablob. If not passed, then the tag **latest** will be assigned to the datablob.
Returns:
An instance of the `DataBlob` class.
Raises:
ValueError: If parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of how to create a Datablob from a MySQL database:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_mysql(
username="{fill in database_username}",
password="{fill in database_password}",
host="{fill in host}",
database="{fill in database}",
table="{fill in table}",
port="{fill in port}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
username = (
username
if username is not None
else os.environ.get(CLIENT_DB_USERNAME, "root")
)
password = (
password if password is not None else os.environ.get(CLIENT_DB_PASSWORD, "")
)
cloud_provider, region = DataBlob._get_cloud_provider_and_region(cloud_provider, region) # type: ignore
json_req = dict(
host=host,
port=port,
username=username,
password=password,
database=database,
table=table,
region=region,
cloud_provider=cloud_provider,
tag=tag,
)
response = Client._post_data(
relative_url=f"/datablob/from_mysql", json=json_req
)
return DataBlob(
uuid=response["uuid"], type=response["type"], source=response["source"]
)
from_s3(*, uri, access_key=None, secret_key=None, cloud_provider=None, region=None, tag=None)
staticmethod
¤
Create and return a datablob that encapsulates the data from an AWS S3 bucket.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri |
str |
AWS S3 bucket uri. |
required |
access_key |
Optional[str] |
Access key for the S3 bucket. If None (default value), then the value from AWS_ACCESS_KEY_ID environment variable will be used. |
None |
secret_key |
Optional[str] |
Secret key for the S3 bucket. If None (default value), then the value from AWS_SECRET_ACCESS_KEY environment variable will be used. |
None |
cloud_provider |
Optional[str] |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. If None (default value), then aws will be used as the cloud storage provider. |
None |
region |
Optional[str] |
The region of the destination cloud provider where the datablob will be stored. If None (default value) then the default region will be assigned based on the cloud provider. In the case of aws, the datablob's source bucket region will be used, whereas azure will use westeurope. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
tag |
Optional[str] |
A string to tag the datablob. If not passed, then the tag latest will be assigned to the datablob. |
None |
Returns:
| Type | Description |
|---|---|
DataBlob |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of how to create a Datablob from an AWS S3 bucket:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the access_key and the secret_key are set in the
# AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@staticmethod
def from_s3(
*,
uri: str,
access_key: Optional[str] = None,
secret_key: Optional[str] = None,
cloud_provider: Optional[str] = None,
region: Optional[str] = None,
tag: Optional[str] = None,
) -> "DataBlob":
"""Create and return a datablob that encapsulates the data from an AWS S3 bucket.
Args:
uri: AWS S3 bucket uri.
access_key: Access key for the S3 bucket. If **None** (default value), then the value
from **AWS_ACCESS_KEY_ID** environment variable will be used.
secret_key: Secret key for the S3 bucket. If **None** (default value), then the value
from **AWS_SECRET_ACCESS_KEY** environment variable will be used.
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
If **None** (default value), then **aws** will be used as the cloud storage provider.
region: The region of the destination cloud provider where the datablob will be stored. If **None** (default value) then the default region will be assigned based on
the cloud provider. In the case of **aws**, the datablob's source bucket region will be used, whereas **azure** will use **westeurope**. The supported AWS regions
are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1,
us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast,
brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast,
japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia,
switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
tag: A string to tag the datablob. If not passed, then the tag **latest** will be assigned to the datablob.
Returns:
An instance of the `DataBlob` class.
Raises:
ValueError: If parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of how to create a Datablob from an AWS S3 bucket:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the access_key and the secret_key are set in the
# AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
access_key = (
access_key if access_key is not None else os.environ["AWS_ACCESS_KEY_ID"]
)
secret_key = (
secret_key
if secret_key is not None
else os.environ["AWS_SECRET_ACCESS_KEY"]
)
cloud_provider, region = DataBlob._get_cloud_provider_and_region(cloud_provider=cloud_provider, region=region, set_source_region=True) # type: ignore
response = Client._post_data(
relative_url="/datablob/from_s3",
json=dict(
uri=uri,
access_key=access_key,
secret_key=secret_key,
region=region,
cloud_provider=cloud_provider,
tag=tag,
),
)
return DataBlob(
uuid=response["uuid"], type=response["type"], source=response["source"]
)
is_ready(self)
¤
Check if the method's progress is complete.
Info
This method will return True immediately and will not wait for the progress to finish
if the datablob is created using the from_local method.
Returns:
| Type | Description |
|---|---|
bool |
True if the upload progress is completed, else False. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
def is_ready(self) -> bool:
"""Check if the method's progress is complete.
!!! info
This method will return `True` immediately and will not wait for the progress to finish
if the datablob is created using the `from_local` method.
Returns:
**True** if the upload progress is completed, else **False**.
"""
if self.type in ["local"]:
return True
progress_status = ProgressStatus(relative_url=f"/datablob/{self.uuid}")
return progress_status.is_ready()
ls(offset=0, limit=100, disabled=False, completed=False)
staticmethod
¤
Return the list of DataBlob instances
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int |
The number of datablobs to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of datablobs to return from the server. If None, then the default value 100 will be used. |
100 |
disabled |
bool |
If set to True, then only the deleted datablobs will be returned. Else, the default value False will be used to return only the list of active datablobs. |
False |
completed |
bool |
If set to True, then only the datablobs that are successfully downloaded to the server will be returned. Else, the default value False will be used to return all the datablobs. |
False |
Returns:
| Type | Description |
|---|---|
List[DataBlob] |
A list of DataBlob instances available in the server. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
@staticmethod
def ls(
offset: int = 0,
limit: int = 100,
disabled: bool = False,
completed: bool = False,
) -> List["DataBlob"]:
"""Return the list of DataBlob instances
Args:
offset: The number of datablobs to offset at the beginning. If **None**,
then the default value **0** will be used.
limit: The maximum number of datablobs to return from the server. If **None**,
then the default value **100** will be used.
disabled: If set to **True**, then only the deleted datablobs will be returned.
Else, the default value **False** will be used to return only the list
of active datablobs.
completed: If set to **True**, then only the datablobs that are successfully downloaded
to the server will be returned. Else, the default value **False** will be used to
return all the datablobs.
Returns:
A list of DataBlob instances available in the server.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
lists = Client._get_data(
relative_url=f"/datablob/?disabled={disabled}&completed={completed}&offset={offset}&limit={limit}"
)
dbx = [
DataBlob(
uuid=db["uuid"],
type=db["type"],
source=db["source"],
region=db["region"],
cloud_provider=db["cloud_provider"],
datasources=db["datasources"],
total_steps=db["total_steps"],
completed_steps=db["completed_steps"],
folder_size=db["folder_size"],
disabled=db["disabled"],
pulled_on=db["pulled_on"],
user=db["user"],
tags=db["tags"],
error=db["error"],
)
for db in lists
]
return dbx
progress_bar(self, sleep_for=5, timeout=0)
¤
Blocks the execution and displays a progress bar showing the remote action progress.
Info
This method will not check the progress if the datablob is created using the
from_local method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sleep_for |
Union[int, float] |
The time interval in seconds between successive API calls. |
5 |
timeout |
int |
The maximum time allowed in seconds for the asynchronous call to complete. If not the progressbar will be terminated. |
0 |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of connection timeout. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
def progress_bar(self, sleep_for: Union[int, float] = 5, timeout: int = 0):
"""Blocks the execution and displays a progress bar showing the remote action progress.
!!! info
This method will not check the progress if the datablob is created using the
`from_local` method.
Args:
sleep_for: The time interval in seconds between successive API calls.
timeout: The maximum time allowed in seconds for the asynchronous call to complete. If not the
progressbar will be terminated.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of connection timeout.
"""
if self.type not in ["local"]:
progress_status = ProgressStatus(
relative_url=f"/datablob/{self.uuid}",
sleep_for=sleep_for,
timeout=timeout,
)
progress_status.progress_bar()
set_default_cloud_provider(cls, cloud_provider, region=None)
¤
Sets the default destination value for the cloud_provider and the region.
Whenever you call the from_* methods of the DataBlob class inside this context manager, the destination cloud_provider and region set in this context
will be passed to the from_* methods, unless you explicitely override it in the parameter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cloud_provider |
str |
The destination cloud storage provider's name to store the datablob. Currently, the API only supports aws and azure as cloud storage providers. |
required |
region |
Optional[str] |
The destination cloud provider's region to store the datablob. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia, centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth, northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth, switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2. |
None |
Returns:
| Type | Description |
|---|---|
Iterator[NoneType] |
A context manager that specifies the cloud provider and region to use. |
Here's an example of creating a datablob from Azure Blob Storage and storing it in AWS S3:
Examples:
# Importing necessary libraries
import os
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob
# Create a credential for accessing Azure Blob Storage
# Setting the required environment variables
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
# Setting the resource group name and storage account name
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
# Retrieving the credential
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
credential = azure_storage_keys['key1']
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablobs created inside the context manager will be
# stored in an AWS S3 bucket with the region set to eu-west-3.
with DataBlob.set_default_cloud_provider(
cloud_provider="aws",
region="eu-west-3"
):
db = DataBlob.from_azure_blob_storage(
uri="{fill in uri}",
credential=credential
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@patch(cls_method=True)
@contextmanager
def set_default_cloud_provider(
cls: DataBlob, cloud_provider: str, region: Optional[str] = None
) -> Iterator[None]:
"""Sets the default destination value for the cloud_provider and the region.
Whenever you call the from_\* methods of the `DataBlob` class inside this context manager, the destination cloud_provider and region set in this context
will be passed to the from_\* methods, unless you explicitely override it in the parameter.
Args:
cloud_provider: The destination cloud storage provider's name to store the datablob. Currently, the API only supports **aws** and **azure** as cloud storage providers.
region: The destination cloud provider's region to store the datablob. The supported AWS regions are: ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1,
ap-southeast-2, ca-central-1, eu-central-1, eu-north-1, eu-west-1, eu-west-2, eu-west-3, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2. The supported
Azure Blob Storage regions are: australiacentral, australiacentral2, australiaeast, australiasoutheast, brazilsouth, canadacentral, canadaeast, centralindia,
centralus, eastasia, eastus, eastus2, francecentral, francesouth, germanynorth, germanywestcentral, japaneast, japanwest, koreacentral, koreasouth,
northcentralus, northeurope, norwayeast, norwaywest, southafricanorth, southafricawest, southcentralus, southeastasia, southindia, switzerlandnorth,
switzerlandwest, uaecentral, uaenorth, uksouth, ukwest, westcentralus, westeurope, westindia, westus, westus2.
Returns:
A context manager that specifies the cloud provider and region to use.
Here's an example of creating a datablob from Azure Blob Storage and storing it in AWS S3:
Example:
```python
# Importing necessary libraries
import os
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob
# Create a credential for accessing Azure Blob Storage
# Setting the required environment variables
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
# Setting the resource group name and storage account name
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
# Retrieving the credential
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
credential = azure_storage_keys['key1']
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablobs created inside the context manager will be
# stored in an AWS S3 bucket with the region set to eu-west-3.
with DataBlob.set_default_cloud_provider(
cloud_provider="aws",
region="eu-west-3"
):
db = DataBlob.from_azure_blob_storage(
uri="{fill in uri}",
credential=credential
)
# Display the status in a progress bar
db.progress_bar()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
cls._default_provider_and_regions.append((cloud_provider, region)) # type: ignore
yield
cls._default_provider_and_regions.pop()
tag(self, name)
¤
Tag an existing datablob in the server.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name |
str |
A string to tag the datablob. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas dataframe with the details of the tagged datablob. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
@patch
def tag(self: DataBlob, name: str) -> pd.DataFrame:
"""Tag an existing datablob in the server.
Args:
name: A string to tag the datablob.
Returns:
A pandas dataframe with the details of the tagged datablob.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._post_data(
relative_url=f"/datablob/{self.uuid}/tag", json=dict(name=name)
)
response = DataBlob._get_tag_name_and_datasource_id(response)
df = pd.DataFrame([response])[DataBlob.BASIC_DB_COLS]
df = df.rename(columns=DataBlob.COLS_TO_RENAME)
return add_ready_column(df)
to_datasource(self, *, file_type, index_column, sort_by, deduplicate_data=False, blocksize='256MB', **kwargs)
¤
Process the datablob and return a datasource object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
file_type |
str |
The file type of the datablob. Currently, the API only supports "csv" and "parquet" as file types. |
required |
index_column |
str |
The column to use as index (row labels). |
required |
sort_by |
Union[str, List[str]] |
The column(s) to sort the data. Can either be a string or a list of strings. |
required |
deduplicate_data |
bool |
If set to True (default value False), the datasource will be created with duplicate rows removed. |
False |
blocksize |
str |
The number of bytes used to split larger files. If None, then the default value 256MB will be used. |
'256MB' |
kwargs |
Additional keyword arguments to use while processing the data.e.g: To skip 100 lines from the bottom of file, pass **{"skipfooter": 100} |
{} |
Returns:
| Type | Description |
|---|---|
DataSource |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the CSV file processing fails. |
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
@patch
def to_datasource(
self: DataBlob,
*,
file_type: str,
index_column: str,
sort_by: Union[str, List[str]],
deduplicate_data: bool = False,
blocksize: str = "256MB",
**kwargs,
) -> DataSource:
"""Process the datablob and return a datasource object.
Args:
file_type: The file type of the datablob. Currently, the API only supports **"csv"** and **"parquet"** as file types.
index_column: The column to use as index (row labels).
sort_by: The column(s) to sort the data. Can either be a string or a list of strings.
deduplicate_data: If set to **True** (default value **False**), the datasource will be created with duplicate rows removed.
blocksize: The number of bytes used to split larger files. If None, then the default value **256MB** will be used.
kwargs: Additional keyword arguments to use while processing the data.e.g: To skip 100 lines from the bottom of file,
pass **{"skipfooter": 100}
Returns:
An instance of the `DataSource` class.
Raises:
ValueError: If the CSV file processing fails.
ConnectionError: If the server address is invalid or not reachable.
"""
json_req = dict(
file_type=file_type,
deduplicate_data=deduplicate_data,
index_column=index_column,
sort_by=sort_by,
blocksize=blocksize,
kwargs=kwargs,
)
response = Client._post_data(
relative_url=f"/datablob/{self.uuid}/to_datasource", json=json_req
)
return DataSource(uuid=response["uuid"])
wait(self, sleep_for=1, timeout=0)
¤
Blocks execution while waiting for the remote action to complete.
Info
This method will not check the progress if the datablob is created using the
from_local method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sleep_for |
Union[int, float] |
The time interval in seconds between successive API calls. |
1 |
timeout |
int |
The maximum time allowed in seconds for the asynchronous call to complete. If not the progressbar will be terminated. |
0 |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of timeout. |
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
db.progress_bar()
# Display the ready status
# If the datablob is successfully uploaded, True will be returned.
print(db.is_ready())
# Print the details of the newly created datablob
print(db.details())
# Display the details of all datablob created by the currently
# logged-in user
print(DataBlob.as_df(DataBlob.ls()))
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Display the head of the data to ensure everything is fine.
print(ds.head())
# Tag the datablob
print(db.tag(name="{fill in tag_name}"))
# Delete the datablob
print(db.delete())
Source code in airt/client.py
def wait(self, sleep_for: Union[int, float] = 1, timeout: int = 0):
"""Blocks execution while waiting for the remote action to complete.
!!! info
This method will not check the progress if the datablob is created using the
`from_local` method.
Args:
sleep_for: The time interval in seconds between successive API calls.
timeout: The maximum time allowed in seconds for the asynchronous call to complete. If not the
progressbar will be terminated.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of timeout.
"""
if self.type not in ["local"]:
progress_status = ProgressStatus(
relative_url=f"/datablob/{self.uuid}",
sleep_for=sleep_for,
timeout=timeout,
)
progress_status.wait()
DataSource
¤
A class for managing datasources and training ML models on them.
To instantiate the DataSource class, please call DataBlob.to_datasource method of the DataBlob class.
The DataSource class has two categories of methods,
- Methods for managing the datasources.
- Method for training a model against a datasource.
Methods such as delete, ls, details, head, etc., can be used to manage and obtain additional information from a datasource instance.
And, the train method can be used to train a new model against a datasource instance.
All the function calls to the library are asynchronous and they return immediately. To manage completion, methods inside the returned object will return a status object indicating the completion status and a method to display an interactive progress bar that can be called to check the progress.
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
dtypes: DataFrame
property
readonly
¤
Return the dtypes of the datasource.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame containing the data type of each column. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
__init__(self, uuid, datablob=None, folder_size=None, no_of_rows=None, error=None, disabled=None, created=None, pulled_on=None, user=None, hash=None, region=None, cloud_provider=None, tags=None, total_steps=None, completed_steps=None)
special
¤
Constructs a new DataSource instance.
Warning
Do not construct this object directly by calling the constructor, please use DataBlob.to_datasource method instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
DataSource uuid. |
required |
datablob |
Optional[str] |
Datablob uuid. |
None |
folder_size |
Optional[int] |
The uploaded datasource's size in bytes. |
None |
no_of_rows |
Optional[int] |
The number of records in the datasource. |
None |
error |
Optional[str] |
Contains the error message if the processing of the datasource fails. |
None |
disabled |
Optional[bool] |
A flag that indicates the datasource's status. If the datasource is deleted, then False will be set. |
None |
created |
Optional[str] |
The datasource creation date. |
None |
pulled_on |
Optional[str] |
The most recent date the datasource was uploaded. |
None |
user |
Optional[str] |
The uuid of the user who created the datasource. |
None |
hash |
Optional[str] |
The datasource hash. |
None |
region |
Optional[str] |
The region name of the cloud provider where the datasource is stored |
None |
cloud_provider |
Optional[str] |
The name of the cloud storage provider where the datasource is stored. |
None |
tags |
Optional[List[Dict[str, str]]] |
Tag names associated with the datasource. |
None |
total_steps |
Optional[int] |
The number of steps required to upload the datasource to the server. |
None |
completed_steps |
Optional[int] |
The number of steps completed during the datasource's upload to the server. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
datablob: Optional[str] = None,
folder_size: Optional[int] = None,
no_of_rows: Optional[int] = None,
error: Optional[str] = None,
disabled: Optional[bool] = None,
created: Optional[str] = None,
pulled_on: Optional[str] = None,
user: Optional[str] = None,
hash: Optional[str] = None,
region: Optional[str] = None,
cloud_provider: Optional[str] = None,
tags: Optional[List[Dict[str, str]]] = None,
total_steps: Optional[int] = None,
completed_steps: Optional[int] = None,
):
"""Constructs a new `DataSource` instance.
Warning:
Do not construct this object directly by calling the constructor, please use `DataBlob.to_datasource` method instead.
Args:
uuid: DataSource uuid.
datablob: Datablob uuid.
folder_size: The uploaded datasource's size in bytes.
no_of_rows: The number of records in the datasource.
error: Contains the error message if the processing of the datasource fails.
disabled: A flag that indicates the datasource's status. If the datasource is deleted, then **False** will be set.
created: The datasource creation date.
pulled_on: The most recent date the datasource was uploaded.
user: The uuid of the user who created the datasource.
hash: The datasource hash.
region: The region name of the cloud provider where the datasource is stored
cloud_provider: The name of the cloud storage provider where the datasource is stored.
tags: Tag names associated with the datasource.
total_steps: The number of steps required to upload the datasource to the server.
completed_steps: The number of steps completed during the datasource's upload to the server.
"""
self.uuid = uuid
self.datablob = datablob
self.folder_size = folder_size
self.no_of_rows = no_of_rows
self.error = error
self.disabled = disabled
self.created = created
self.pulled_on = pulled_on
self.user = user
self.hash = hash
self.region = region
self.cloud_provider = cloud_provider
self.tags = tags
self.total_steps = total_steps
self.completed_steps = completed_steps
as_df(dsx)
staticmethod
¤
Return the details of DataSource instances as a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dsx |
List[DataSource] |
List of |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of the datasources in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@staticmethod
def as_df(dsx: List["DataSource"]) -> pd.DataFrame:
"""Return the details of `DataSource` instances as a pandas dataframe.
Args:
dsx: List of `DataSource` instances.
Returns:
Details of the datasources in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
ds_lists = [{i: getattr(ds, i) for i in DataSource.ALL_DS_COLS} for ds in dsx]
for ds in ds_lists:
ds["tags"] = get_values_from_item(ds["tags"], "name")
lists_df = generate_df(ds_lists, DataSource.BASIC_DS_COLS)
df = add_ready_column(lists_df)
df = df.rename(columns=DataSource.COLS_TO_RENAME)
return df
delete(self)
¤
Delete a datasource from the server.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the deleted datasource. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def delete(self: DataSource) -> pd.DataFrame:
"""Delete a datasource from the server.
Returns:
A pandas DataFrame encapsulating the details of the deleted datasource.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._delete_data(relative_url=f"/datasource/{self.uuid}")
response["tags"] = get_values_from_item(response["tags"], "name")
df = pd.DataFrame([response])[DataSource.BASIC_DS_COLS]
df = df.rename(columns=DataSource.COLS_TO_RENAME)
return add_ready_column(df)
details(self)
¤
Return details of a datasource.
Returns:
| Type | Description |
|---|---|
DataFrame |
The datasource details as a pandas dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def details(self: DataSource) -> pd.DataFrame:
"""Return details of a datasource.
Returns:
The datasource details as a pandas dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/datasource/{self.uuid}")
response["tags"] = get_values_from_item(response["tags"], "name")
df = pd.DataFrame([response])[DataSource.ALL_DS_COLS]
df = df.rename(columns=DataSource.COLS_TO_RENAME)
return add_ready_column(df)
head(self)
¤
Return the first few rows of the datasource.
Returns:
| Type | Description |
|---|---|
DataFrame |
The first few rows of the datasource as a pandas dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def head(self: DataSource) -> pd.DataFrame:
"""Return the first few rows of the datasource.
Returns:
The first few rows of the datasource as a pandas dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/datasource/{self.uuid}/head")
df = dict_to_df(response)
return df
is_ready(self)
¤
Check if the method's progress is complete.
Returns:
| Type | Description |
|---|---|
bool |
True if the progress is completed, else False. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def is_ready(
self: DataSource,
) -> bool:
"""Check if the method's progress is complete.
Returns:
**True** if the progress is completed, else **False**.
"""
progress_status = ProgressStatus(relative_url=f"/datasource/{self.uuid}")
return progress_status.is_ready()
ls(offset=0, limit=100, disabled=False, completed=False)
staticmethod
¤
Return the list of DataSource instances available in server.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int |
The number of datasources to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of datasources to return from the server. If None, then the default value 100 will be used. |
100 |
disabled |
bool |
If set to True, then only the deleted datasources will be returned. Else, the default value False will be used to return only the list of active datasources. |
False |
completed |
bool |
If set to True, then only the datasources that are successfully processed in server will be returned. Else, the default value False will be used to return all the datasources. |
False |
Returns:
| Type | Description |
|---|---|
List[DataSource] |
A list of |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@staticmethod
def ls(
offset: int = 0,
limit: int = 100,
disabled: bool = False,
completed: bool = False,
) -> List["DataSource"]:
"""Return the list of `DataSource` instances available in server.
Args:
offset: The number of datasources to offset at the beginning. If **None**,
then the default value **0** will be used.
limit: The maximum number of datasources to return from the server. If **None**,
then the default value **100** will be used.
disabled: If set to **True**, then only the deleted datasources will be returned.
Else, the default value **False** will be used to return only the list
of active datasources.
completed: If set to **True**, then only the datasources that are successfully processed
in server will be returned. Else, the default value **False** will be used to
return all the datasources.
Returns:
A list of `DataSource` instances available in server.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
lists = Client._get_data(
relative_url=f"/datasource/?disabled={disabled}&completed={completed}&offset={offset}&limit={limit}"
)
dsx = [
DataSource(
uuid=ds["uuid"],
datablob=ds["datablob"],
folder_size=ds["folder_size"],
no_of_rows=ds["no_of_rows"],
region=ds["region"],
cloud_provider=ds["cloud_provider"],
error=ds["error"],
disabled=ds["disabled"],
created=ds["created"],
pulled_on=ds["pulled_on"],
user=ds["user"],
hash=ds["hash"],
tags=ds["tags"],
total_steps=ds["total_steps"],
completed_steps=ds["completed_steps"],
)
for ds in lists
]
return dsx
progress_bar(self, sleep_for=5, timeout=0)
¤
Blocks the execution and displays a progress bar showing the remote action progress.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sleep_for |
Union[int, float] |
The time interval in seconds between successive API calls. |
5 |
timeout |
int |
The maximum time allowed in seconds for the asynchronous call to complete. If not the progressbar will be terminated. |
0 |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of connection timeout. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def progress_bar(self: DataSource, sleep_for: Union[int, float] = 5, timeout: int = 0):
"""Blocks the execution and displays a progress bar showing the remote action progress.
Args:
sleep_for: The time interval in seconds between successive API calls.
timeout: The maximum time allowed in seconds for the asynchronous call to complete. If not the
progressbar will be terminated.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of connection timeout.
"""
progress_status = ProgressStatus(
relative_url=f"/datasource/{self.uuid}", sleep_for=sleep_for, timeout=timeout
)
progress_status.progress_bar()
tag(self, name)
¤
Tag an existing datasource in server.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name |
str |
A string to tag the datasource. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas dataframe with the details of the tagged datasource. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def tag(self: DataSource, name: str) -> pd.DataFrame:
"""Tag an existing datasource in server.
Args:
name: A string to tag the datasource.
Returns:
A pandas dataframe with the details of the tagged datasource.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._post_data(
relative_url=f"/datasource/{self.uuid}/tag", json=dict(name=name)
)
response["tags"] = get_values_from_item(response["tags"], "name")
df = pd.DataFrame([response])[DataSource.BASIC_DS_COLS]
df = df.rename(columns=DataSource.COLS_TO_RENAME)
return add_ready_column(df)
train(self, *, client_column, timestamp_column=None, target_column, target, predict_after)
¤
Train a model against the datasource.
This method trains the model for predicting which clients are most likely to have a specified event in the future.
The call to this method is asynchronous and the progress can be checked using the progress bar method
or the status flag attribute available in the DataSource class.
For more model specific information, please check the documentation of Model class.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client_column |
str |
The column name that uniquely identifies the users/clients. |
required |
timestamp_column |
Optional[str] |
The timestamp column indicating the time of an event. If not passed, then the default value None will be used. |
None |
target_column |
str |
Target column name that indicates the type of the event. |
required |
target |
str |
Target event name to train and make predictions. You can pass the target event as a string or as a regular expression for predicting more than one event. For example, passing *checkout will train a model to predict any checkout event. |
required |
predict_after |
timedelta |
Time delta in hours of the expected target event. |
required |
Returns:
| Type | Description |
|---|---|
Model |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the input parameters to the API are invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def train(
self: DataSource,
*,
client_column: str,
timestamp_column: Optional[str] = None,
target_column: str,
target: str,
predict_after: timedelta,
) -> Model:
"""Train a model against the datasource.
This method trains the model for predicting which clients are most likely to have a specified
event in the future.
The call to this method is asynchronous and the progress can be checked using the progress bar method
or the status flag attribute available in the `DataSource` class.
For more model specific information, please check the documentation of `Model` class.
Args:
client_column: The column name that uniquely identifies the users/clients.
timestamp_column: The timestamp column indicating the time of an event. If not passed,
then the default value **None** will be used.
target_column: Target column name that indicates the type of the event.
target: Target event name to train and make predictions. You can pass the target event as a string or as a
regular expression for predicting more than one event. For example, passing ***checkout** will
train a model to predict any checkout event.
predict_after: Time delta in hours of the expected target event.
Returns:
An instance of the `Model` class.
Raises:
ValueError: If the input parameters to the API are invalid.
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._post_data(
relative_url=f"/model/train",
json=dict(
data_uuid=self.uuid,
client_column=client_column,
target_column=target_column,
target=target,
predict_after=int(predict_after.total_seconds()),
),
)
return Model(uuid=response["uuid"])
wait(self, sleep_for=1, timeout=0)
¤
Blocks execution while waiting for the remote action to complete.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sleep_for |
Union[int, float] |
The time interval in seconds between successive API calls. |
1 |
timeout |
int |
The maximum time allowed in seconds for the asynchronous call to complete. If not the progressbar will be terminated. |
0 |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of timeout. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, DataSource
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
# Call the wait method to wait for the progress to finish but
# without displaying an interactive progress bar.
ds.progress_bar()
# Display the ready status
print(ds.is_ready())
# Display the data types of the datasource's columns.
print(ds.dtypes)
# Display the details of the datasource
print(ds.details())
# Display the details of all datasource created by the currently
# logged-in user
print(DataSource.as_df(DataSource.ls()))
# Display the first few records of the datasource
print(ds.head())
# Train a model against the datasource.
# This example predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the training status in a progress bar
model.progress_bar()
# Display the details of the newly created model
print(model.details())
# Tag the datasource
print(ds.tag(name="{fill in tag_name}"))
# Delete the datasource
print(ds.delete())
Source code in airt/client.py
@patch
def wait(self: DataSource, sleep_for: Union[int, float] = 1, timeout: int = 0):
"""Blocks execution while waiting for the remote action to complete.
Args:
sleep_for: The time interval in seconds between successive API calls.
timeout: The maximum time allowed in seconds for the asynchronous call to complete. If not the
progressbar will be terminated.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of timeout.
"""
progress_status = ProgressStatus(
relative_url=f"/datasource/{self.uuid}", sleep_for=sleep_for, timeout=timeout
)
progress_status.wait()
Model (ProgressStatus)
¤
A class for querying the model training, evaluation, and prediction status.
The Model class is instantiated automatically when the DataSource.train method is called on a datasource. Currently,
it is the only way to instantiate the Model class.
The model is trained to predict a specific event in the future and we assume the input data to have:
- a column identifying a client (client_column). E.g: person, car, business, etc.,
- a column specifying a type of event to predict (target_column). E.g: buy, checkout, etc.,
- a timestamp column (timestamp_column) specifying the time of an occurred event.
Along with the above mandatory columns, the input data can have additional columns of any type (int, category, float, datetime type, etc.,). These additional columns will be used in the model training for making more accurate predictions.
Finally, we need to know how much ahead we wish to make predictions. This lead time varies widely for each use case and can be in minutes for a webshop or even several weeks for a banking product such as a loan.
As always, the model training and prediction is an asynchronous process and can take a few hours to finish depending
on the size of your dataset. The progress for the same can be checked by calling the ProgressStatus.is_ready method on the Model
instance. Alternatively, you can call the ProgressStatus.progress_bar method to monitor the status interactively.
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
__init__(self, uuid, datasource=None, client_column=None, target_column=None, target=None, predict_after=None, timestamp_column=None, total_steps=None, completed_steps=None, region=None, cloud_provider=None, error=None, disabled=None, created=None, user=None)
special
¤
Constructs a new Model instance
Warning
Do not construct this object directly by calling the constructor, please use
DataSource.train method instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
Model uuid. |
required |
datasource |
Optional[str] |
DataSource uuid. |
None |
client_column |
Optional[str] |
The column name that uniquely identifies the users/clients. |
None |
target_column |
Optional[str] |
Target column name that indicates the type of the event. |
None |
target |
Optional[str] |
Target event name to train and make predictions. You can pass the target event as a string or as a regular expression for predicting more than one event. For example, passing *checkout will train a model to predict any checkout event. |
None |
predict_after |
Optional[str] |
Time delta in hours of the expected target event. |
None |
timestamp_column |
Optional[str] |
The timestamp column indicating the time of an event. If not passed, then the default value None will be used. |
None |
total_steps |
Optional[int] |
No of steps needed to complete the model training. |
None |
completed_steps |
Optional[int] |
No of steps completed so far in the model training. |
None |
region |
Optional[str] |
The region name of the cloud provider where the model is stored. |
None |
cloud_provider |
Optional[str] |
The name of the cloud storage provider where the model is stored. |
None |
error |
Optional[str] |
Contains the error message if the training of the model fails. |
None |
disabled |
Optional[bool] |
A flag that indicates the model's status. If the model is deleted, then False will be set. |
None |
created |
Optional[str] |
Model creation date. |
None |
user |
Optional[str] |
The uuid of the user who created the model. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
datasource: Optional[str] = None,
client_column: Optional[str] = None,
target_column: Optional[str] = None,
target: Optional[str] = None,
predict_after: Optional[str] = None,
timestamp_column: Optional[str] = None,
total_steps: Optional[int] = None,
completed_steps: Optional[int] = None,
region: Optional[str] = None,
cloud_provider: Optional[str] = None,
error: Optional[str] = None,
disabled: Optional[bool] = None,
created: Optional[str] = None,
user: Optional[str] = None,
):
"""Constructs a new `Model` instance
Warning:
Do not construct this object directly by calling the constructor, please use
`DataSource.train` method instead.
Args:
uuid: Model uuid.
datasource: DataSource uuid.
client_column: The column name that uniquely identifies the users/clients.
target_column: Target column name that indicates the type of the event.
target: Target event name to train and make predictions. You can pass the target event as a string or as a
regular expression for predicting more than one event. For example, passing ***checkout** will
train a model to predict any checkout event.
predict_after: Time delta in hours of the expected target event.
timestamp_column: The timestamp column indicating the time of an event. If not passed,
then the default value **None** will be used.
total_steps: No of steps needed to complete the model training.
completed_steps: No of steps completed so far in the model training.
region: The region name of the cloud provider where the model is stored.
cloud_provider: The name of the cloud storage provider where the model is stored.
error: Contains the error message if the training of the model fails.
disabled: A flag that indicates the model's status. If the model is deleted, then **False** will be set.
created: Model creation date.
user: The uuid of the user who created the model.
"""
self.uuid = uuid
self.datasource = datasource
self.client_column = client_column
self.target_column = target_column
self.target = target
self.predict_after = predict_after
self.timestamp_column = timestamp_column
self.total_steps = total_steps
self.completed_steps = completed_steps
self.region = region
self.cloud_provider = cloud_provider
self.error = error
self.disabled = disabled
self.created = created
self.user = user
ProgressStatus.__init__(self, relative_url=f"/model/{self.uuid}")
as_df(mx)
staticmethod
¤
Return the details of Model instances as a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mx |
List[Model] |
List of Model instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the models in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@staticmethod
def as_df(mx: List["Model"]) -> pd.DataFrame:
"""Return the details of Model instances as a pandas dataframe.
Args:
mx: List of Model instances.
Returns:
Details of all the models in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
model_lists = get_attributes_from_instances(mx, Model.BASIC_MODEL_COLS) # type: ignore
df = generate_df(model_lists, Model.BASIC_MODEL_COLS)
df = df.rename(columns=Model.COLS_TO_RENAME)
return add_ready_column(df)
delete(self)
¤
Delete a model from the server.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the deleted model. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@patch
def delete(self: Model) -> pd.DataFrame:
"""Delete a model from the server.
Returns:
A pandas DataFrame encapsulating the details of the deleted model.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._delete_data(relative_url=f"/model/{self.uuid}")
df = pd.DataFrame(response, index=[0])[Model.BASIC_MODEL_COLS]
df = df.rename(columns=Model.COLS_TO_RENAME)
return add_ready_column(df)
details(self)
¤
Return the details of a model.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the model. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@patch
def details(self: Model) -> pd.DataFrame:
"""Return the details of a model.
Returns:
A pandas DataFrame encapsulating the details of the model.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/model/{self.uuid}")
df = pd.DataFrame(response, index=[0])[Model.ALL_MODEL_COLS]
df = df.rename(columns=Model.COLS_TO_RENAME)
return add_ready_column(df)
evaluate(self)
¤
Return the evaluation metrics of the trained model.
Currently, this method returns the model's accuracy, precision, and recall. In the future, more performance metrics will be added.
Returns:
| Type | Description |
|---|---|
DataFrame |
The performance metrics of the trained model as a pandas series. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@patch
def evaluate(self: Model) -> pd.DataFrame:
"""Return the evaluation metrics of the trained model.
Currently, this method returns the model's accuracy, precision, and recall. In the
future, more performance metrics will be added.
Returns:
The performance metrics of the trained model as a pandas series.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
model_evaluate = Client._get_data(relative_url=f"/model/{self.uuid}/evaluate")
return pd.DataFrame(dict(model_evaluate), index=[0]).T.rename(columns={0: "eval"})
ls(offset=0, limit=100, disabled=False, completed=False)
staticmethod
¤
Return the list of Model instances available in the server.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int |
The number of models to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of models to return from the server. If None, then the default value 100 will be used. |
100 |
disabled |
bool |
If set to True, then only the deleted models will be returned. Else, the default value False will be used to return only the list of active models. |
False |
completed |
bool |
If set to True, then only the models that are successfully processed in server will be returned. Else, the default value False will be used to return all the models. |
False |
Returns:
| Type | Description |
|---|---|
List[Model] |
A list of Model instances available in the server. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@staticmethod
def ls(
offset: int = 0,
limit: int = 100,
disabled: bool = False,
completed: bool = False,
) -> List["Model"]:
"""Return the list of Model instances available in the server.
Args:
offset: The number of models to offset at the beginning. If None, then the default value **0** will be used.
limit: The maximum number of models to return from the server. If None,
then the default value **100** will be used.
disabled: If set to **True**, then only the deleted models will be returned. Else, the default value
**False** will be used to return only the list of active models.
completed: If set to **True**, then only the models that are successfully processed in server will be returned.
Else, the default value **False** will be used to return all the models.
Returns:
A list of Model instances available in the server.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
lists = Client._get_data(
relative_url=f"/model/?disabled={disabled}&completed={completed}&offset={offset}&limit={limit}"
)
mx = [
Model(
uuid=model["uuid"],
datasource=model["datasource"],
client_column=model["client_column"],
target_column=model["target_column"],
target=model["target"],
predict_after=model["predict_after"],
timestamp_column=model["timestamp_column"],
total_steps=model["total_steps"],
completed_steps=model["completed_steps"],
region=model["region"],
cloud_provider=model["cloud_provider"],
error=model["error"],
disabled=model["disabled"],
created=model["created"],
user=model["user"],
)
for model in lists
]
return mx
predict(self, data_uuid=0)
¤
Run predictions against the trained model.
The progress for the same can be checked by calling the is_ready method on the Model instance.
Alternatively, you can call the progress_bar method to monitor the status interactively.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_uuid |
Optional[int] |
The datasource uuid to run the predictions. If not set, then the datasource used for training the model will be used for prediction aswell. |
0 |
Returns:
| Type | Description |
|---|---|
Prediction |
An instance of the |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
from datetime import timedelta
from airt.client import Client, DataBlob, Model
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Print the details of the newly created model
print(model.details())
# Display the details of all models created by the currently
# logged-in user
print(Model.as_df(Model.ls()))
# Evaluate the newly created model
print(model.evaluate())
# Run predictions on the newly created model
prediction = model.predict()
# Display the prediction status in a progress bar
prediction.progress_bar()
# Display details of the predictions
print(prediction.details())
# Delete the newly created model
print(model.delete())
Source code in airt/client.py
@patch
def predict(self: Model, data_uuid: Optional[int] = 0) -> Prediction:
"""Run predictions against the trained model.
The progress for the same can be checked by calling the `is_ready` method on the `Model` instance.
Alternatively, you can call the `progress_bar` method to monitor the status interactively.
Args:
data_uuid: The datasource uuid to run the predictions. If not set, then the datasource used for training
the model will be used for prediction aswell.
Returns:
An instance of the `Prediction` class.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
req_json = dict(data_uuid=data_uuid) if data_uuid else None
response = Client._post_data(
relative_url=f"/model/{self.uuid}/predict", json=req_json
)
return Prediction(uuid=response["uuid"], datasource=response["datasource"])
Prediction (ProgressStatus)
¤
A class to manage and download the predictions.
The Prediction class is automatically instantiated by calling the Model.predict method of a Model instance.
Currently, it is the only way to instantiate this class.
At the moment, the prediction results can only be
-
downloaded to a local folder in parquet file format
-
pushed to Azure Blob Storage or an AWS S3 bucket
-
pushed to MySql or ClickHouse database
We intend to support additional databases and storage mediums in future releases.
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
__init__(self, uuid, datasource=None, model=None, created=None, total_steps=None, completed_steps=None, region=None, cloud_provider=None, error=None, disabled=None)
special
¤
Constructs a new Prediction instance
Warning
Do not construct this object directly by calling the constructor, instead please use
Model.predict method of the Model instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
Prediction uuid. |
required |
datasource |
Optional[str] |
DataSource uuid. |
None |
model |
Optional[str] |
Model uuid. |
None |
created |
Optional[str] |
Prediction creation date. |
None |
total_steps |
Optional[int] |
No of steps needed to complete the model prediction. |
None |
completed_steps |
Optional[int] |
No of steps completed so far in the model prediction. |
None |
region |
Optional[str] |
The region name of the cloud provider where the prediction is stored. |
None |
cloud_provider |
Optional[str] |
The name of the cloud storage provider where the prediction is stored. |
None |
error |
Optional[str] |
Contains the error message if running the predictions fails. |
None |
disabled |
Optional[bool] |
A flag that indicates the prediction's status. If the prediction is deleted, then False will be set. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
datasource: Optional[str] = None,
model: Optional[str] = None,
created: Optional[str] = None,
total_steps: Optional[int] = None,
completed_steps: Optional[int] = None,
region: Optional[str] = None,
cloud_provider: Optional[str] = None,
error: Optional[str] = None,
disabled: Optional[bool] = None,
):
"""Constructs a new **Prediction** instance
Warning:
Do not construct this object directly by calling the constructor, instead please use
`Model.predict` method of the Model instance.
Args:
uuid: Prediction uuid.
datasource: DataSource uuid.
model: Model uuid.
created: Prediction creation date.
total_steps: No of steps needed to complete the model prediction.
completed_steps: No of steps completed so far in the model prediction.
region: The region name of the cloud provider where the prediction is stored.
cloud_provider: The name of the cloud storage provider where the prediction is stored.
error: Contains the error message if running the predictions fails.
disabled: A flag that indicates the prediction's status. If the prediction is deleted, then **False** will be set.
"""
self.uuid = uuid
self.datasource = datasource
self.model = model
self.created = created
self.total_steps = total_steps
self.completed_steps = completed_steps
self.region = region
self.cloud_provider = cloud_provider
self.error = error
self.disabled = disabled
ProgressStatus.__init__(self, relative_url=f"/prediction/{self.uuid}")
as_df(predx)
staticmethod
¤
Return the details of prediction instances as a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
predx |
List[Prediction] |
List of prediction instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the prediction in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@staticmethod
def as_df(predx: List["Prediction"]) -> pd.DataFrame:
"""Return the details of prediction instances as a pandas dataframe.
Args:
predx: List of prediction instances.
Returns:
Details of all the prediction in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = get_attributes_from_instances(predx, Prediction.BASIC_PRED_COLS) # type: ignore
df = generate_df(response, Prediction.BASIC_PRED_COLS)
df = df.rename(columns=Prediction.COLS_TO_RENAME)
return add_ready_column(df)
delete(self)
¤
Delete a prediction from the server.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the deleted prediction. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def delete(self: Prediction) -> pd.DataFrame:
"""Delete a prediction from the server.
Returns:
A pandas DataFrame encapsulating the details of the deleted prediction.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._delete_data(relative_url=f"/prediction/{self.uuid}")
df = pd.DataFrame(response, index=[0])[Prediction.BASIC_PRED_COLS]
df = df.rename(columns=Prediction.COLS_TO_RENAME)
return add_ready_column(df)
details(self)
¤
Return the details of a prediction.
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the prediction. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def details(self: Prediction) -> pd.DataFrame:
"""Return the details of a prediction.
Returns:
A pandas DataFrame encapsulating the details of the prediction.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/prediction/{self.uuid}")
df = pd.DataFrame(response, index=[0])[Prediction.ALL_PRED_COLS]
df = df.rename(columns=Prediction.COLS_TO_RENAME)
return add_ready_column(df)
ls(offset=0, limit=100, disabled=False, completed=False)
staticmethod
¤
Return the list of Prediction instances available in the server.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int |
The number of predictions to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of predictions to return from the server. If None, then the default value 100 will be used. |
100 |
disabled |
bool |
If set to True, then only the deleted predictions will be returned. Else, the default value False will be used to return only the list of active predictions. |
False |
completed |
bool |
If set to True, then only the predictions that are successfully processed in server will be returned. Else, the default value False will be used to return all the predictions. |
False |
Returns:
| Type | Description |
|---|---|
List[Prediction] |
A list of Prediction instances available in the server. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@staticmethod
def ls(
offset: int = 0,
limit: int = 100,
disabled: bool = False,
completed: bool = False,
) -> List["Prediction"]:
"""Return the list of Prediction instances available in the server.
Args:
offset: The number of predictions to offset at the beginning. If None, then the default value **0** will be used.
limit: The maximum number of predictions to return from the server. If None,
then the default value **100** will be used.
disabled: If set to **True**, then only the deleted predictions will be returned. Else, the default value
**False** will be used to return only the list of active predictions.
completed: If set to **True**, then only the predictions that are successfully processed in server will be returned.
Else, the default value **False** will be used to return all the predictions.
Returns:
A list of Prediction instances available in the server.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
lists = Client._get_data(
relative_url=f"/prediction/?disabled={disabled}&completed={completed}&offset={offset}&limit={limit}"
)
predx = [
Prediction(
uuid=pred["uuid"],
model=pred["model"],
datasource=pred["datasource"],
created=pred["created"],
total_steps=pred["total_steps"],
completed_steps=pred["completed_steps"],
region=pred["region"],
cloud_provider=pred["cloud_provider"],
error=pred["error"],
disabled=pred["disabled"],
)
for pred in lists
]
return predx
to_azure_blob_storage(self, uri, credential)
¤
Push the prediction results to the target Azure Blob Storage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri |
str |
Target Azure Blob Storage uri. |
required |
credential |
str |
Credential to access the Azure Blob Storage. |
required |
Returns:
| Type | Description |
|---|---|
ProgressStatus |
An instance of |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_azure_blob_storage(
self: Prediction,
uri: str,
credential: str,
) -> ProgressStatus:
"""Push the prediction results to the target Azure Blob Storage.
Args:
uri: Target Azure Blob Storage uri.
credential: Credential to access the Azure Blob Storage.
Returns:
An instance of `ProgressStatus` class.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._post_data(
relative_url=f"/prediction/{self.uuid}/to_azure_blob_storage",
json=dict(uri=uri, credential=credential),
)
return ProgressStatus(relative_url=f"/prediction/push/{response['uuid']}")
to_clickhouse(self, *, host, database, table, protocol, port=0, username=None, password=None)
¤
Push the prediction results to a clickhouse database.
If the database requires authentication, pass the username/password as parameters or store it in the CLICKHOUSE_USERNAME and CLICKHOUSE_PASSWORD environment variables.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
host |
str |
Remote database host name. |
required |
database |
str |
Database name. |
required |
table |
str |
Table name. |
required |
protocol |
str |
Protocol to use (native/http). |
required |
port |
int |
Host port number. If not passed, then the default value 0 will be used. |
0 |
username |
Optional[str] |
Database username. If not passed, then the value set in the environment variable CLICKHOUSE_USERNAME will be used else the default value "root" will be used. |
None |
password |
Optional[str] |
Database password. If not passed, then the value set in the environment variable CLICKHOUSE_PASSWORD will be used else the default value "" will be used. |
None |
Returns:
| Type | Description |
|---|---|
ProgressStatus |
An instance of |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_clickhouse(
self: Prediction,
*,
host: str,
database: str,
table: str,
protocol: str,
port: int = 0,
username: Optional[str] = None,
password: Optional[str] = None,
) -> ProgressStatus:
"""Push the prediction results to a clickhouse database.
If the database requires authentication, pass the username/password as parameters or store it in
the **CLICKHOUSE_USERNAME** and **CLICKHOUSE_PASSWORD** environment variables.
Args:
host: Remote database host name.
database: Database name.
table: Table name.
protocol: Protocol to use (native/http).
port: Host port number. If not passed, then the default value **0** will be used.
username: Database username. If not passed, then the value set in the environment variable
**CLICKHOUSE_USERNAME** will be used else the default value "root" will be used.
password: Database password. If not passed, then the value set in the environment variable
**CLICKHOUSE_PASSWORD** will be used else the default value "" will be used.
Returns:
An instance of `ProgressStatus` class.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
username = (
username
if username is not None
else os.environ.get("CLICKHOUSE_USERNAME", "root")
)
password = (
password if password is not None else os.environ.get("CLICKHOUSE_PASSWORD", "")
)
req_json = dict(
host=host,
database=database,
table=table,
protocol=protocol,
port=port,
username=username,
password=password,
)
response = Client._post_data(
relative_url=f"/prediction/{self.uuid}/to_clickhouse", json=req_json
)
return ProgressStatus(relative_url=f"/prediction/push/{response['uuid']}")
to_local(self, path, show_progress=True)
¤
Download the prediction results to a local directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path |
Union[str, pathlib.Path] |
Local directory path. |
required |
show_progress |
Optional[bool] |
Flag to set the progressbar visibility. If not passed, then the default value True will be used. |
True |
Exceptions:
| Type | Description |
|---|---|
FileNotFoundError |
If the path is invalid. |
HTTPError |
If the presigned AWS s3 uri to download the prediction results are invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_local(
self: Prediction,
path: Union[str, Path],
show_progress: Optional[bool] = True,
) -> None:
"""Download the prediction results to a local directory.
Args:
path: Local directory path.
show_progress: Flag to set the progressbar visibility. If not passed, then the default value **True** will be used.
Raises:
FileNotFoundError: If the **path** is invalid.
HTTPError: If the presigned AWS s3 uri to download the prediction results are invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/prediction/{self.uuid}/to_local")
# Initiate progress bar
t = tqdm(total=len(response), disable=not show_progress)
for file_name, url in response.items():
Prediction._download_prediction_file_to_local(file_name, url, Path(path))
t.update()
t.close()
to_mysql(self, *, host, database, table, port=3306, username=None, password=None)
¤
Push the prediction results to a mysql database.
If the database requires authentication, pass the username/password as parameters or store it in the AIRT_CLIENT_DB_USERNAME and AIRT_CLIENT_DB_PASSWORD environment variables.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
host |
str |
Database host name. |
required |
database |
str |
Database name. |
required |
table |
str |
Table name. |
required |
port |
int |
Host port number. If not passed, then the default value 3306 will be used. |
3306 |
username |
Optional[str] |
Database username. If not passed, then the value set in the environment variable AIRT_CLIENT_DB_USERNAME will be used else the default value "root" will be used. |
None |
password |
Optional[str] |
Database password. If not passed, then the value set in the environment variable AIRT_CLIENT_DB_PASSWORD will be used else the default value "" will be used. |
None |
Returns:
| Type | Description |
|---|---|
ProgressStatus |
An instance of |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_mysql(
self: Prediction,
*,
host: str,
database: str,
table: str,
port: int = 3306,
username: Optional[str] = None,
password: Optional[str] = None,
) -> ProgressStatus:
"""Push the prediction results to a mysql database.
If the database requires authentication, pass the username/password as parameters or store it in
the **AIRT_CLIENT_DB_USERNAME** and **AIRT_CLIENT_DB_PASSWORD** environment variables.
Args:
host: Database host name.
database: Database name.
table: Table name.
port: Host port number. If not passed, then the default value **3306** will be used.
username: Database username. If not passed, then the value set in the environment variable
**AIRT_CLIENT_DB_USERNAME** will be used else the default value "root" will be used.
password: Database password. If not passed, then the value set in the environment variable
**AIRT_CLIENT_DB_PASSWORD** will be used else the default value "" will be used.
Returns:
An instance of `ProgressStatus` class.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
username = (
username if username is not None else os.environ.get(CLIENT_DB_USERNAME, "root")
)
password = (
password if password is not None else os.environ.get(CLIENT_DB_PASSWORD, "")
)
req_json = dict(
host=host,
port=port,
username=username,
password=password,
database=database,
table=table,
)
response = Client._post_data(
relative_url=f"/prediction/{self.uuid}/to_mysql", json=req_json
)
return ProgressStatus(relative_url=f"/prediction/push/{response['uuid']}")
to_pandas(self)
¤
Return the prediction results as a pandas DataFrame
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the results of the prediction. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_pandas(self: Prediction) -> pd.DataFrame:
"""Return the prediction results as a pandas DataFrame
Returns:
A pandas DataFrame encapsulating the results of the prediction.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
response = Client._get_data(relative_url=f"/prediction/{self.uuid}/pandas")
keys = list(response.keys())
keys.remove("Score")
index_name = keys[0]
return (
pd.DataFrame(response)
.set_index(index_name)
.sort_values("Score", ascending=False)
)
to_s3(self, uri, access_key=None, secret_key=None)
¤
Push the prediction results to the target AWS S3 bucket.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uri |
str |
Target S3 bucket uri. |
required |
access_key |
Optional[str] |
Access key for the target S3 bucket. If None (default value), then the value from AWS_ACCESS_KEY_ID environment variable is used. |
None |
secret_key |
Optional[str] |
Secret key for the target S3 bucket. If None (default value), then the value from AWS_SECRET_ACCESS_KEY environment variable is used. |
None |
Returns:
| Type | Description |
|---|---|
ProgressStatus |
An instance of |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
Examples:
# Importing necessary libraries
import os
import tempfile
from datetime import timedelta
from azure.identity import DefaultAzureCredential
from azure.mgmt.storage import StorageManagementClient
from airt.client import Client, DataBlob, DataSource, Model, Prediction
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The
# access_key and the secret_key are set in the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY environment variables, and the region is set to
# eu-west-3; feel free to change the cloud provider and the region to
# suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Display the status in a progress bar
db.progress_bar()
# Create a datasource
ds = db.to_datasource(
file_type="{fill in file_type}",
index_column="{fill in index_column}",
sort_by="{fill in sort_by}",
)
# Display the status in a progress bar
ds.progress_bar()
# Train a model to predicts which users will perform a purchase
# event ("*purchase") three hours before they actually do it.
model = ds.train(
client_column="{fill in client_column}",
target_column="{fill in target_column}",
target="*purchase",
predict_after=timedelta(hours=3)
)
# Display the status in a progress bar
model.progress_bar()
# Run predictions
prediction = model.predict()
prediction.progress_bar()
# Print the details of the newly created prediction
print(prediction.details())
# Get the list of all prediction instances created by the currently logged-in user
predx = Prediction.ls()
print(predx)
# Display the details of the prediction instances in a pandas dataframe
df = Prediction.as_df(predx)
print(df)
# Display the prediction results in a pandas DataFrame
print(prediction.to_pandas())
# Push the prediction results to an AWS S3 bucket
s3_status = prediction.to_s3(uri="{fill in s3_target_uri}")
# Push the prediction results to an Azure Blob Storage
os.environ["AZURE_SUBSCRIPTION_ID"] = "{fill in azure_subscription_id}"
os.environ["AZURE_CLIENT_ID"] = "{fill in azure_client_id}"
os.environ["AZURE_CLIENT_SECRET"] = "{fill in azure_client_secret}"
os.environ["AZURE_TENANT_ID"]= "{fill in azure_tenant_id}"
azure_group_name = "{fill in azure_group_name}"
azure_storage_account_name = "{fill in azure_storage_account_name}"
azure_storage_client = StorageManagementClient(
DefaultAzureCredential(), os.environ["AZURE_SUBSCRIPTION_ID"]
)
azure_storage_keys = azure_storage_client.storage_accounts.list_keys(
azure_group_name, azure_storage_account_name
)
azure_storage_keys = {v.key_name: v.value for v in azure_storage_keys.keys}
azure_credential = azure_storage_keys['key1']
azure_status = prediction.to_azure_blob_storage(
uri="{fill in azure_target_uri}",
credential=azure_credential
)
# Push the prediction results to a MySQL database
mysql_status = prediction.to_mysql(
username="{fill in mysql_db_username}",
password="{fill in mysql_db_password}",
host="{fill in mysql_host}",
database="{fill in mysql_database}",
table="{fill in mysql_table}",
)
# Push the prediction results to a ClickHouse database
clickhouse_status = prediction.to_clickhouse(
username="{fill in clickhouse_db_username}",
password="{fill in clickhouse_db_password}",
host="{fill in clickhouse_host}",
database="{fill in clickhouse_database}",
table="{fill in clickhouse_table}",
protocol="native",
)
# Download the predictions to a local directory
# In this example, the prediction results are downloaded
# to a temporary directory
with tempfile.TemporaryDirectory(prefix="predictions_results_") as d:
prediction.to_local(path=d)
# Check the downloaded prediction files
downloaded_files = sorted(list(os.listdir(d)))
print(downloaded_files)
# Check the status
s3_status.wait()
azure_status.progress_bar()
mysql_status.progress_bar()
clickhouse_status.progress_bar()
# Delete the prediction
prediction.delete()
Source code in airt/client.py
@patch
def to_s3(
self: Prediction,
uri: str,
access_key: Optional[str] = None,
secret_key: Optional[str] = None,
) -> ProgressStatus:
"""Push the prediction results to the target AWS S3 bucket.
Args:
uri: Target S3 bucket uri.
access_key: Access key for the target S3 bucket. If **None** (default value), then the value
from **AWS_ACCESS_KEY_ID** environment variable is used.
secret_key: Secret key for the target S3 bucket. If **None** (default value), then the value
from **AWS_SECRET_ACCESS_KEY** environment variable is used.
Returns:
An instance of `ProgressStatus` class.
Raises:
ConnectionError: If the server address is invalid or not reachable.
"""
access_key = (
access_key if access_key is not None else os.environ["AWS_ACCESS_KEY_ID"]
)
secret_key = (
secret_key if secret_key is not None else os.environ["AWS_SECRET_ACCESS_KEY"]
)
response = Client._post_data(
relative_url=f"/prediction/{self.uuid}/to_s3",
json=dict(uri=uri, access_key=access_key, secret_key=secret_key),
)
return ProgressStatus(relative_url=f"/prediction/push/{response['uuid']}")
ProgressStatus
¤
A base class for querying status of a remote operation.
Here's an example of using the ProgressStatus class to checking the upload status of the datablob
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Check the ready status of the datablob
print(db.is_ready())
# Display the status in a progress bar
db.progress_bar()
# Check the ready status of the datablob
# If the upload is successful, True will be returned
print(db.is_ready())
__init__(self, relative_url, sleep_for=5, timeout=0)
special
¤
Constructs a new ProgressStatus instance.
Warning
Do not construct this object directly by calling the constructor, please use either progress_bar,
is_ready, or wait methods of DataBlob, DataSource, Model or Prediction classes instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
relative_url |
str |
Relative URI to query the status of the remote operation. |
required |
sleep_for |
Union[int, float] |
The time interval in seconds between successive API calls. |
5 |
timeout |
int |
The maximum time allowed in seconds for the asynchronous call to complete. If not the progressbar will be terminated. |
0 |
Exceptions:
| Type | Description |
|---|---|
TimeoutError |
in case of connection timeout. |
Source code in airt/client.py
def __init__(
self, relative_url: str, sleep_for: Union[int, float] = 5, timeout: int = 0
):
"""Constructs a new ProgressStatus instance.
Warning:
Do not construct this object directly by calling the constructor, please use either progress_bar,
is_ready, or wait methods of `DataBlob`, `DataSource`, `Model` or `Prediction` classes instead.
Args:
relative_url: Relative URI to query the status of the remote operation.
sleep_for: The time interval in seconds between successive API calls.
timeout: The maximum time allowed in seconds for the asynchronous call to complete. If not the
progressbar will be terminated.
Raises:
TimeoutError: in case of connection timeout.
"""
self.relative_url = relative_url
self.sleep_for = sleep_for
self.timeout = timeout
is_ready(self)
¤
Check if the method's progress is complete.
Returns:
| Type | Description |
|---|---|
bool |
True if the progress is completed, else False. |
Here's an example of checking the upload status of the datablob:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Check the ready status of the datablob
print(db.is_ready())
# Display the status in a progress bar
db.progress_bar()
# Check the ready status of the datablob
# If the upload is successful, True will be returned
print(db.is_ready())
Source code in airt/client.py
def is_ready(self) -> bool:
"""Check if the method's progress is complete.
Returns:
**True** if the progress is completed, else **False**.
Here's an example of checking the upload status of the datablob:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Check the ready status of the datablob
print(db.is_ready())
# Display the status in a progress bar
db.progress_bar()
# Check the ready status of the datablob
# If the upload is successful, True will be returned
print(db.is_ready())
```
"""
response = Client._get_data(relative_url=self.relative_url)
return response["completed_steps"] == response["total_steps"]
progress_bar(self)
¤
Blocks the execution and displays a progress bar showing the remote action progress.
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of connection timeout. |
Here's an example of checking the upload status of the datablob:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Check the ready status of the datablob
print(db.is_ready())
# Display the status in a progress bar
db.progress_bar()
# Check the ready status of the datablob
# If the upload is successful, True will be returned
print(db.is_ready())
Source code in airt/client.py
def progress_bar(self):
"""Blocks the execution and displays a progress bar showing the remote action progress.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of connection timeout.
Here's an example of checking the upload status of the datablob:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Check the ready status of the datablob
print(db.is_ready())
# Display the status in a progress bar
db.progress_bar()
# Check the ready status of the datablob
# If the upload is successful, True will be returned
print(db.is_ready())
```
"""
total_steps = Client._get_data(relative_url=self.relative_url)["total_steps"]
with tqdm(total=total_steps) as pbar:
started_at = datetime.now()
while True:
if (0 < self.timeout) and (datetime.now() - started_at) > timedelta(
seconds=self.timeout
):
raise TimeoutError()
response = Client._get_data(relative_url=self.relative_url)
completed_steps = response["completed_steps"]
pbar.update(completed_steps)
if completed_steps == total_steps:
break
sleep(self.sleep_for)
wait(self)
¤
Blocks execution while waiting for the remote action to complete.
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
TimeoutError |
in case of timeout. |
Here's an example of preventing further calls from being executed until the datablob upload is complete:
Examples:
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Further calls to the API will be blocked until the datablob upload is complete.
db.wait()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
Source code in airt/client.py
@patch
def wait(self: ProgressStatus):
"""Blocks execution while waiting for the remote action to complete.
Raises:
ConnectionError: If the server address is invalid or not reachable.
TimeoutError: in case of timeout.
Here's an example of preventing further calls from being executed until the datablob upload is complete:
Example:
```python
# Importing necessary libraries
from airt.client import Client, DataBlob
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a datablob
# In this example, the datablob will be stored in an AWS S3 bucket. The region
# is set to eu-west-3, feel free to change the cloud provider and the region
# to suit your needs.
db = DataBlob.from_s3(
uri="{fill in uri}",
cloud_provider="aws",
region="eu-west-3"
)
# Further calls to the API will be blocked until the datablob upload is complete.
db.wait()
# Print the details of the newly created datablob
# If the upload is successful, the ready flag should be set to True
print(db.details())
```
"""
started_at = datetime.now()
while True:
if (0 < self.timeout) and (datetime.now() - started_at) > timedelta(
seconds=self.timeout
):
raise TimeoutError()
if self.is_ready():
return
sleep(self.sleep_for)
User
¤
A class for creating, managing, and updating users on the server.
The User class has two types of methods:
- Methods for creating and managing users.
- Method for updating and adding additional security to user accounts.
Methods such as create, enable, disable, and ls can be used to manage user accounts on the server, but access to them requires super user privileges.
The remaining methods do not require super user privileges and are used to update/additionally secure user accounts.
In addition to the regular authentication with credentials, the users can enable multi-factor authentication (MFA) and single sign-on (SSO) for their accounts.
To help protect your account, we recommend that you enable multi-factor authentication (MFA). MFA provides additional security by requiring you to provide unique verification code (OTP) in addition to your regular sign-in credentials when performing critical operations.
Your account can be configured for MFA in just two easy steps:
-
To begin, you need to enable MFA for your account by calling the
enable_mfamethod, which will generate a QR code. You can then scan the QR code with an authenticator app, such as Google Authenticator and follow the on-device instructions to finish the setup in your smartphone. -
Finally, activate MFA for your account by calling
activate_mfaand passing the dynamically generated six-digit verification code from your smartphone's authenticator app.
Single sign-on (SSO) can be enabled for your account in three simple steps:
-
Enable the SSO for a provider by calling the
enable_ssomethod with the SSO provider name and an email address. At the moment, we only support "google" and "github" as SSO providers. We intend to support additional SSO providers in future releases. -
Before you can start generating new tokens with SSO, you must first authenticate with the SSO provider. Call the get_token with the same SSO provider you have enabled in the step above to generate an SSO authorization URL. Please copy and paste it into your preferred browser and complete the authentication process with the SSO provider.
-
After successfully authenticating with the SSO provider, call the
Client.set_sso_tokenmethod to generate a new token and use it automatically in all future interactions with the airt server.
Here's an example of using the User class's methods to display the logged-in user's uuid
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the uuid of logged-in user
user_details = User.details()
print(user_details["uuid"])
__init__(self, uuid, username=None, first_name=None, last_name=None, email=None, subscription_type=None, super_user=None, disabled=None, created=None, is_mfa_active=None, phone_number=None, is_phone_number_verified=None)
special
¤
Constructs a new User instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
User uuid. |
required |
username |
Optional[str] |
The username of the user. |
None |
first_name |
Optional[str] |
The first name of the user. |
None |
last_name |
Optional[str] |
The last name of the user. |
None |
email |
Optional[str] |
The email address of the user. |
None |
subscription_type |
Optional[str] |
User subscription type. Currently, the API supports only the following subscription types small, medium and large. |
None |
super_user |
Optional[bool] |
Flag to indicate the user type. |
None |
disabled |
Optional[str] |
Flag to indicate the status of the user. |
None |
created |
Optional[str] |
User creation date. |
None |
is_mfa_active |
Optional[bool] |
Flag to indicate the user's MFA status. |
None |
phone_number |
Optional[str] |
The registered phone number of the user. The phone number should follow the pattern of country code followed by your phone number. For example, 440123456789, +440123456789, 00440123456789, +44 0123456789, and (+44) 012 345 6789 are all valid formats for registering a UK phone number. |
None |
is_phone_number_verified |
Optional[bool] |
Flag to indicate the phone number registration status. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
username: Optional[str] = None,
first_name: Optional[str] = None,
last_name: Optional[str] = None,
email: Optional[str] = None,
subscription_type: Optional[str] = None,
super_user: Optional[bool] = None,
disabled: Optional[str] = None,
created: Optional[str] = None,
is_mfa_active: Optional[bool] = None,
phone_number: Optional[str] = None,
is_phone_number_verified: Optional[bool] = None,
):
"""Constructs a new User instance.
Args:
uuid: User uuid.
username: The username of the user.
first_name: The first name of the user.
last_name: The last name of the user.
email: The email address of the user.
subscription_type: User subscription type. Currently, the API supports only the following subscription
types **small**, **medium** and **large**.
super_user: Flag to indicate the user type.
disabled: Flag to indicate the status of the user.
created: User creation date.
is_mfa_active: Flag to indicate the user's MFA status.
phone_number: The registered phone number of the user. The phone number should follow the pattern of country
code followed by your phone number. For example, **440123456789, +440123456789, 00440123456789, +44 0123456789,
and (+44) 012 345 6789** are all valid formats for registering a UK phone number.
is_phone_number_verified: Flag to indicate the phone number registration status.
"""
self.uuid = uuid
self.username = username
self.first_name = first_name
self.last_name = last_name
self.email = email
self.subscription_type = subscription_type
self.super_user = super_user
self.disabled = disabled
self.created = created
self.is_mfa_active = is_mfa_active
self.phone_number = phone_number
self.is_phone_number_verified = is_phone_number_verified
activate_mfa(otp)
staticmethod
¤
Activate MFA for the user
Adding MFA to your account is a two-step process. To begin, you must enable MFA for your account
by calling the enable_mfa method, then call the activate_mfa method and pass the current OTP
from the authenticator application to verify and activate MFA for your account.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
otp |
int |
Dynamically generated six-digit verification code from the authenticator app |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the MFA activated user details |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the user entered six-digit verification code is invalid |
Here's an example to enable and activate MFA for the user
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Enable MFA for the user
# The line below will generate a QR code. To finish the setup on your smartphone,
# scan the QR code with an authenticator app like Google Authenticator and
# follow the on-device instructions
User.enable_mfa()
# After you've completed the setup, enter the current OTP from the authenticator
# app to verify and enable MFA for your account
User.activate_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to True, if the setup is successful
print(User.details()["is_mfa_active"])
Source code in airt/client.py
@staticmethod
def activate_mfa(otp: int) -> pd.DataFrame:
"""Activate MFA for the user
Adding MFA to your account is a two-step process. To begin, you must enable MFA for your account
by calling the `enable_mfa` method, then call the `activate_mfa` method and pass the current OTP
from the authenticator application to verify and activate MFA for your account.
Args:
otp: Dynamically generated six-digit verification code from the authenticator app
Returns:
A pandas DataFrame encapsulating the MFA activated user details
Raises:
ValueError: If the user entered six-digit verification code is invalid
Here's an example to enable and activate MFA for the user
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Enable MFA for the user
# The line below will generate a QR code. To finish the setup on your smartphone,
# scan the QR code with an authenticator app like Google Authenticator and
# follow the on-device instructions
User.enable_mfa()
# After you've completed the setup, enter the current OTP from the authenticator
# app to verify and enable MFA for your account
User.activate_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to True, if the setup is successful
print(User.details()["is_mfa_active"])
```
"""
response = Client._post_data(
relative_url="/user/mfa/activate",
json=dict(user_otp=otp),
)
return pd.DataFrame(response, index=[0])
as_df(ux)
staticmethod
¤
Return the details of User instances as a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ux |
List[User] |
List of user instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the User in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
An example of displaying the details of all active users
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
# Only super users can get the list of available users
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Display the details of all active users
# Set the disabled parameter to True to display the details of inactive users
ux = User.ls()
print(User.as_df(ux))
Source code in airt/client.py
@staticmethod
def as_df(ux: List["User"]) -> pd.DataFrame:
"""Return the details of User instances as a pandas dataframe.
Args:
ux: List of user instances.
Returns:
Details of all the User in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
An example of displaying the details of all active users
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
# Only super users can get the list of available users
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Display the details of all active users
# Set the disabled parameter to True to display the details of inactive users
ux = User.ls()
print(User.as_df(ux))
```
"""
lists = get_attributes_from_instances(ux, User.USER_COLS) # type: ignore
return generate_df(lists, User.USER_COLS)
create(*, username, first_name, last_name, email, password, subscription_type, super_user=False, phone_number=None, otp=None)
staticmethod
¤
Create a new user in the server.
To access this method, you must have super user privileges.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
username |
str |
The new user's username. The username must be unique or an exception will be thrown. |
required |
first_name |
str |
The new user's first name. |
required |
last_name |
str |
The new user's last name. |
required |
email |
str |
The new user's email. The email must be unique or an exception will be thrown. |
required |
password |
str |
The new user's password. |
required |
subscription_type |
str |
User subscription type. Currently, the API supports only the following subscription types small, medium and large. |
required |
super_user |
bool |
If set to True, then the new user will have super user privilages. If None, then the default value False will be used to create a non-super user. |
False |
phone_number |
Optional[str] |
Phone number to be added to the user account. The phone number should follow the pattern of the country code followed by your phone number. For example, 440123456789, +440123456789, 00440123456789, +44 0123456789, and (+44) 012 345 6789 are all valid formats for registering a UK phone number. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the newly created user. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
ValueError |
If the OTP is invalid. |
ValueError |
If the username or email is already present in the server. |
Below is an example of creating a new user
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Details required to create a new user
username = "{fill in username}"
first_name = "{fill in first_name}"
last_name = "{fill in last_name}"
email = "{fill in email}"
password = "{fill in password}"
super_user = "{fill in super_user}"
subscription_type = "{fill in subscription_type}"
# Create a new user. To access this method, you must have super user privileges.
new_user = User.create(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
password=password,
super_user=super_user,
subscription_type=subscription_type,
)
# Display the details of the newly created user
print(new_user)
# An exception will be raised if you attempt to create a new user
# with an already-used username or email address.
try:
User.create(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
password=password,
super_user=super_user,
subscription_type=subscription_type,
)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
Source code in airt/client.py
@staticmethod
def create(
*,
username: str,
first_name: str,
last_name: str,
email: str,
password: str,
subscription_type: str,
super_user: bool = False,
phone_number: Optional[str] = None,
otp: Optional[str] = None,
) -> pd.DataFrame:
"""Create a new user in the server.
To access this method, you must have super user privileges.
Args:
username: The new user's username. The username must be unique or an exception will be thrown.
first_name: The new user's first name.
last_name: The new user's last name.
email: The new user's email. The email must be unique or an exception will be thrown.
password: The new user's password.
subscription_type: User subscription type. Currently, the API supports only the following subscription
types **small**, **medium** and **large**.
super_user: If set to **True**, then the new user will have super user privilages.
If **None**, then the default value **False** will be used to create a non-super user.
phone_number: Phone number to be added to the user account. The phone number should follow the pattern of the country
code followed by your phone number. For example, **440123456789, +440123456789, 00440123456789, +44 0123456789,
and (+44) 012 345 6789** are all valid formats for registering a UK phone number.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the details of the newly created user.
Raises:
ConnectionError: If the server address is invalid or not reachable.
ValueError: If the OTP is invalid.
ValueError: If the username or email is already present in the server.
Below is an example of creating a new user
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Details required to create a new user
username = "{fill in username}"
first_name = "{fill in first_name}"
last_name = "{fill in last_name}"
email = "{fill in email}"
password = "{fill in password}"
super_user = "{fill in super_user}"
subscription_type = "{fill in subscription_type}"
# Create a new user. To access this method, you must have super user privileges.
new_user = User.create(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
password=password,
super_user=super_user,
subscription_type=subscription_type,
)
# Display the details of the newly created user
print(new_user)
# An exception will be raised if you attempt to create a new user
# with an already-used username or email address.
try:
User.create(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
password=password,
super_user=super_user,
subscription_type=subscription_type,
)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
```
"""
if phone_number is not None:
phone_number = standardize_phone_number(phone_number)
req_json = dict(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
subscription_type=subscription_type,
super_user=super_user,
password=password,
phone_number=phone_number,
otp=otp,
)
response = Client._post_data(relative_url=f"/user/", json=req_json)
return pd.DataFrame(response, index=[0])[User.USER_COLS]
details(user=None)
staticmethod
¤
Get user details
Please do not pass the optional user parameter unless you are a super user. Only a super user can view details for other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Optional[str] |
Account user_uuid/username to get details. If not passed, then the currently logged-in details will be returned. |
None |
Returns:
| Type | Description |
|---|---|
Dict[str, Union[str, bool]] |
A dict containing the details of the user |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the user_uuid/username is invalid or the user have insufficient permission to access other user's data |
ConnectionError |
If the server address is invalid or not reachable. |
An example of displaying the logged-in user's uuid
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the uuid of logged-in user
# If you are a super user, you can view the details of other users by
# passing their uuid/username in the user parameter.
user_details = User.details()
print(user_details["uuid"])
# If a Non-super user tries to access other user's detail,
# an exception will be thrown.
try:
User.details(user="some_other_username")
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
Source code in airt/client.py
@staticmethod
def details(user: Optional[str] = None) -> Dict[str, Union[str, bool]]:
"""Get user details
Please do not pass the optional user parameter unless you are a super user. Only a
super user can view details for other users.
Args:
user: Account user_uuid/username to get details. If not passed, then the currently logged-in
details will be returned.
Returns:
A dict containing the details of the user
Raises:
ValueError: If the user_uuid/username is invalid or the user have insufficient permission to access other user's data
ConnectionError: If the server address is invalid or not reachable.
An example of displaying the logged-in user's uuid
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the uuid of logged-in user
# If you are a super user, you can view the details of other users by
# passing their uuid/username in the user parameter.
user_details = User.details()
print(user_details["uuid"])
# If a Non-super user tries to access other user's detail,
# an exception will be thrown.
try:
User.details(user="some_other_username")
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
```
"""
relative_url = (
f"/user/details?user_uuid_or_name={user}"
if user is not None
else f"/user/details"
)
return Client._get_data(relative_url=relative_url)
disable(user, otp=None)
staticmethod
¤
Disable one or more users.
To access this method, you must have super user privileges.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Union[str, List[str]] |
user_uuid/username to disabled. To disable multiple users, please pass the uuids/names as a list. |
required |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the disabled user. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example to disable a user
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Optional Step: For demonstration purpose, create a new user
username = "{fill in username}"
User.create(
username=username,
first_name="{fill in first_name}",
last_name="{fill in last_name}",
email="{fill in email}",
password="{fill in password}",
subscription_type="{fill in subscription_type}",
)
# Display the details of the user you want to disable using their username/uuid
print(User.details(username))
# Disable the user
# To disable multiple users, pass a list of username/uuid
User.disable(user=username)
# Check whether the user has been disabled
# The disabled flag should be set to True, if the disable was sucessful
print(User.details(username))
Source code in airt/client.py
@staticmethod
def disable(user: Union[str, List[str]], otp: Optional[str] = None) -> pd.DataFrame:
"""Disable one or more users.
To access this method, you must have super user privileges.
Args:
user: user_uuid/username to disabled. To disable multiple users, please pass the uuids/names as a list.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the details of the disabled user.
Raises:
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example to disable a user
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Optional Step: For demonstration purpose, create a new user
username = "{fill in username}"
User.create(
username=username,
first_name="{fill in first_name}",
last_name="{fill in last_name}",
email="{fill in email}",
password="{fill in password}",
subscription_type="{fill in subscription_type}",
)
# Display the details of the user you want to disable using their username/uuid
print(User.details(username))
# Disable the user
# To disable multiple users, pass a list of username/uuid
User.disable(user=username)
# Check whether the user has been disabled
# The disabled flag should be set to True, if the disable was sucessful
print(User.details(username))
```
"""
_users = user if isinstance(user, list) else [user]
response_list = []
for user in _users:
user_uuid = User.details(user=user)["uuid"]
url = f"/user/{user_uuid}"
response = Client._delete_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
response_list.append(response)
return pd.DataFrame(response_list)[User.USER_COLS]
disable_mfa(user=None, otp=None)
staticmethod
¤
Disable MFA for the user
We currently support two types of OTPs for disabling multi-factor authentication on your account.
If you have access to the authenticator app, enter the app's dynamically generated six-digit verification code(OTP). If you don't have access to the authentication app, you can have an OTP sent to your registered phone number via SMS.
To receive OTP via SMS, first call the send_sms_otp method, which will send the OTP to your registered
phone number. Once you have the OTP, call the disable_mfa method to deactivate MFA for your account.
Currently, we only support the above two methods for disabling MFA. If you do not have access to the authenticator app or your registered phone number, please contact your administrator.
Note
Please do not pass the user parameter unless you are a super user. Only a super user can disable MFA for other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Optional[str] |
Account user_uuid/username to disable MFA. If not passed, then the default value None will be used to disable MFA for the currently logged-in user. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app or the OTP you have received via SMS. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the MFA disabled user details |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If a non-super user tries to disable MFA for other users |
ValueError |
If the OTP is invalid. |
ValueError |
If the user_uuid/username is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of disabling MFA for the currently logged in user using the verification code generated by the authentication application. The example assumes that you have already activated the MFA on your account and have access to the authentication application.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Skip this step if you've already logged-in
# Authenticate. Pass the current OTP from the authenticator app below
otp="{fill in otp}"
Client.get_token(
username="{fill in username}",
password="{fill in password}",
otp=otp
)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Disable MFA for the user
User.disable_mfa(otp=otp)
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
Here's an example of disabling MFA for the currently logged in user using the SMS OTP. The example assumes that you have already registered and validated your phone number on our servers.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Request OTP via SMS to authenticate
# If you've already logged in, you can skip the two optional steps
# If you've already have a valid token, skip the below optional step
# and call Client.set_token instead of Client.get_token
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Optional Step: Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Request OTP via SMS to disable MFA
User.send_sms_otp(
username=username,
message_template_name="disable_mfa" # Don't change the message_template_name
)
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
User.disable_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
Source code in airt/client.py
@staticmethod
def disable_mfa(
user: Optional[str] = None, otp: Optional[str] = None
) -> pd.DataFrame:
"""Disable MFA for the user
We currently support two types of OTPs for disabling multi-factor authentication on your account.
If you have access to the authenticator app, enter the app's dynamically generated six-digit verification
code(OTP). If you don't have access to the authentication app, you can have an OTP sent to your registered
phone number via SMS.
To receive OTP via SMS, first call the `send_sms_otp` method, which will send the OTP to your registered
phone number. Once you have the OTP, call the `disable_mfa` method to deactivate MFA for your account.
Currently, we only support the above two methods for disabling MFA. If you do not have access to the authenticator
app or your registered phone number, please contact your administrator.
!!! note
Please do not pass the user parameter unless you are a super user. Only
a super user can disable MFA for other users.
Args:
user: Account user_uuid/username to disable MFA. If not passed, then the default
value **None** will be used to disable MFA for the currently logged-in user.
otp: Dynamically generated six-digit verification code from the authenticator app or
the OTP you have received via SMS.
Returns:
A pandas DataFrame encapsulating the MFA disabled user details
Raises:
ValueError: If a non-super user tries to disable MFA for other users
ValueError: If the OTP is invalid.
ValueError: If the user_uuid/username is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of disabling MFA for the currently logged in user using the verification code generated by the authentication application.
The example assumes that you have already activated the MFA on your account and have access to the authentication application.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Skip this step if you've already logged-in
# Authenticate. Pass the current OTP from the authenticator app below
otp="{fill in otp}"
Client.get_token(
username="{fill in username}",
password="{fill in password}",
otp=otp
)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Disable MFA for the user
User.disable_mfa(otp=otp)
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
```
Here's an example of disabling MFA for the currently logged in user using the SMS OTP. The example assumes that you have
already registered and validated your phone number on our servers.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Request OTP via SMS to authenticate
# If you've already logged in, you can skip the two optional steps
# If you've already have a valid token, skip the below optional step
# and call Client.set_token instead of Client.get_token
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Optional Step: Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Request OTP via SMS to disable MFA
User.send_sms_otp(
username=username,
message_template_name="disable_mfa" # Don't change the message_template_name
)
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
User.disable_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
```
"""
user_uuid = User.details(user=user)["uuid"]
url = f"/user/mfa/{user_uuid}/disable"
response = Client._delete_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
return pd.DataFrame(response, index=[0])
disable_sso(sso_provider, user=None, otp=None)
staticmethod
¤
Disable Single sign-on (SSO) for the user
Please do not pass the user parameter unless you are a super user. Only a super user can disable SSO for other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sso_provider |
str |
The name of the Single sign-on (SSO) provider you want to disable. At present, the API only supports "google" and "github" as valid SSO identity providers. |
required |
user |
Optional[str] |
Account user_uuid/username to disable SSO. If not passed, then the default value None will be used to disable SSO for the currently logged-in user. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
str |
A pandas DataFrame encapsulating the SSO disabled user details |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If a non-super user tries to disable SSO for other users |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example to disable the Single sign-on (SSO)
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Optional Step: For demonstration purpose, enable Single sign-on (SSO)
# for the user
sso_provider="{fill in sso_provider}"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
sso_url = Client.get_token(sso_provider=sso_provider) # Authenticate using SSO
print(sso_url)
Client.set_sso_token() # Set SSO token
# Disable the Single sign-on (SSO) for the provider
print(User.disable_sso(sso_provider=sso_provider))
# If you try to disable an already disabled SSO provider, an exception
# will be raised
try:
User.disable_sso(sso_provider=sso_provider)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
Source code in airt/client.py
@staticmethod
def disable_sso(
sso_provider: str,
user: Optional[str] = None,
otp: Optional[str] = None,
) -> str:
"""Disable Single sign-on (SSO) for the user
Please do not pass the user parameter unless you are a super user. Only
a super user can disable SSO for other users.
Args:
sso_provider: The name of the Single sign-on (SSO) provider you want to disable.
At present, the API only supports **"google"** and **"github"** as valid SSO identity providers.
user: Account user_uuid/username to disable SSO. If not passed, then the default
value **None** will be used to disable SSO for the currently logged-in user.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the SSO disabled user details
Raises:
ValueError: If a non-super user tries to disable SSO for other users
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example to disable the Single sign-on (SSO)
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Optional Step: For demonstration purpose, enable Single sign-on (SSO)
# for the user
sso_provider="{fill in sso_provider}"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
sso_url = Client.get_token(sso_provider=sso_provider) # Authenticate using SSO
print(sso_url)
Client.set_sso_token() # Set SSO token
# Disable the Single sign-on (SSO) for the provider
print(User.disable_sso(sso_provider=sso_provider))
# If you try to disable an already disabled SSO provider, an exception
# will be raised
try:
User.disable_sso(sso_provider=sso_provider)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
```
"""
user_uuid = User.details(user=user)["uuid"]
url = f"/user/sso/{user_uuid}/disable/{sso_provider}"
response = Client._delete_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
success_msg = f"Single sign-on (SSO) is successfully disabled for {response['sso_provider']}."
return success_msg
enable(user, otp=None)
staticmethod
¤
Enable one or more disabled users.
To access this method, you must have super user privileges.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Union[str, List[str]] |
user_uuid/username to enable. To enable multiple users, please pass the uuids/names as a list. |
required |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the details of the enabled user. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example to enable a user
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Optional Step: For demonstration purpose, create a new user and disable them
username = "{fill in username}"
User.create(
username=username,
first_name="{fill in first_name}",
last_name="{fill in last_name}",
email="{fill in email}",
password="{fill in password}",
subscription_type="{fill in subscription_type}",
)
User.disable(user=username)
# Display the details of the user you want to enable using their username/uuid
print(User.details(username))
# Enable the user
# To enable multiple users, pass a list of username/uuid
User.enable(user=username)
# Check whether the user has been enabled
# The disabled flag should be set to False, if the enable was sucessful
print(User.details(username))
Source code in airt/client.py
@staticmethod
def enable(user: Union[str, List[str]], otp: Optional[str] = None) -> pd.DataFrame:
"""Enable one or more disabled users.
To access this method, you must have super user privileges.
Args:
user: user_uuid/username to enable. To enable multiple users, please pass the uuids/names as a list.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the details of the enabled user.
Raises:
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example to enable a user
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Optional Step: For demonstration purpose, create a new user and disable them
username = "{fill in username}"
User.create(
username=username,
first_name="{fill in first_name}",
last_name="{fill in last_name}",
email="{fill in email}",
password="{fill in password}",
subscription_type="{fill in subscription_type}",
)
User.disable(user=username)
# Display the details of the user you want to enable using their username/uuid
print(User.details(username))
# Enable the user
# To enable multiple users, pass a list of username/uuid
User.enable(user=username)
# Check whether the user has been enabled
# The disabled flag should be set to False, if the enable was sucessful
print(User.details(username))
```
"""
_users = user if isinstance(user, list) else [user]
response_list = []
for user in _users:
user_uuid = User.details(user=user)["uuid"]
url = f"/user/{user_uuid}/enable"
response = Client._get_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
response_list.append(response)
return pd.DataFrame(response_list)[User.USER_COLS]
enable_mfa(otp=None)
staticmethod
¤
Enable MFA for the user
This method will generate a QR code. To finish the setup on your smartphone, scan the
QR code with an authenticator app such as Google Authenticator and follow the on-device
instructions. After you've completed the setup, call the activate_mfa method and pass the
current OTP from the authenticator application to verify and enable MFA for your account
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
PilImage |
The generated QR code |
Here's an example to enable and activate MFA for the user
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Enable MFA for the user
# The line below will generate a QR code. To finish the setup on your smartphone,
# scan the QR code with an authenticator app like Google Authenticator and
# follow the on-device instructions
User.enable_mfa()
# After you've completed the setup, enter the current OTP from the authenticator
# app to verify and enable MFA for your account
User.activate_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to True, if the setup is successful
print(User.details()["is_mfa_active"])
Source code in airt/client.py
@staticmethod
def enable_mfa(otp: Optional[str] = None) -> PilImage:
"""Enable MFA for the user
This method will generate a QR code. To finish the setup on your smartphone, scan the
QR code with an authenticator app such as Google Authenticator and follow the on-device
instructions. After you've completed the setup, call the `activate_mfa` method and pass the
current OTP from the authenticator application to verify and enable MFA for your account
Args:
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
The generated QR code
Here's an example to enable and activate MFA for the user
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Enable MFA for the user
# The line below will generate a QR code. To finish the setup on your smartphone,
# scan the QR code with an authenticator app like Google Authenticator and
# follow the on-device instructions
User.enable_mfa()
# After you've completed the setup, enter the current OTP from the authenticator
# app to verify and enable MFA for your account
User.activate_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to True, if the setup is successful
print(User.details()["is_mfa_active"])
```
"""
qr_code = qrcode.make(User._get_mfa_provision_url(otp))
return qr_code
enable_sso(sso_provider, sso_email, otp=None)
staticmethod
¤
Enable Single sign-on (SSO) for the user
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sso_provider |
str |
Name of the Single sign-on (SSO) identity provider. At present, the API only supports "google" and "github" as valid SSO identity providers. |
required |
sso_email |
str |
Email id to be used for SSO authentication |
required |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
str |
A pandas DataFrame encapsulating the user details |
Here's an example of authenticating with Single sign-on (SSO) using google and setting the newly generated token to interact with the airt service.
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Enable single sign-on (SSO) and use google as the provider
sso_provider="google"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
# Authenticate using Single sign-on (SSO)
# To generate a token using SSO, you must first authenticate with the provider.
# The command below will generate an authorization URL for you.
# Please copy and paste it into your preferred browser and complete the
# SSO provider authentication within 10 minutes. Otherwise, the SSO login
# will time out and you will need to call the get_token method again.
sso_url = Client.get_token(sso_provider=sso_provider)
print(sso_url)
# Once the provider authentication is successful, call the below method to
# set the generated token
Client.set_sso_token()
# If set_sso_token fails, the line below will throw an error.
print(User.details())
Source code in airt/client.py
@staticmethod
def enable_sso(sso_provider: str, sso_email: str, otp: Optional[str] = None) -> str:
"""Enable Single sign-on (SSO) for the user
Args:
sso_provider: Name of the Single sign-on (SSO) identity provider.
At present, the API only supports **"google"** and **"github"** as valid SSO identity providers.
sso_email: Email id to be used for SSO authentication
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the user details
Here's an example of authenticating with Single sign-on (SSO) using google and
setting the newly generated token to interact with the airt service.
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Enable single sign-on (SSO) and use google as the provider
sso_provider="google"
sso_email="{fill in sso_email}"
User.enable_sso(sso_provider=sso_provider, sso_email=sso_email)
# Authenticate using Single sign-on (SSO)
# To generate a token using SSO, you must first authenticate with the provider.
# The command below will generate an authorization URL for you.
# Please copy and paste it into your preferred browser and complete the
# SSO provider authentication within 10 minutes. Otherwise, the SSO login
# will time out and you will need to call the get_token method again.
sso_url = Client.get_token(sso_provider=sso_provider)
print(sso_url)
# Once the provider authentication is successful, call the below method to
# set the generated token
Client.set_sso_token()
# If set_sso_token fails, the line below will throw an error.
print(User.details())
```
"""
response = Client._post_data(
relative_url=f"/user/sso/enable",
json=dict(sso_provider=sso_provider, sso_email=sso_email, otp=otp),
)
success_msg = f"Single sign-on (SSO) is successfully enabled for {sso_provider}. Please use {response['sso_email']} as the email address while authenticating with {sso_provider}."
return success_msg
ls(offset=0, limit=100, disabled=False)
staticmethod
¤
Return the list of User instances available in the server.
To access this method, you must have super user privileges.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
offset |
int |
The number of users to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of users to return from the server. If None, then the default value 100 will be used. |
100 |
disabled |
bool |
If set to True, then only the deleted users will be returned. Else, the default value False will be used to return only the list of active users. |
False |
Returns:
| Type | Description |
|---|---|
List[User] |
A list of User instances available in the server. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
An example of displaying the details of all active users
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Display the details of all active users
# Set the disabled parameter to True to display the details of inactive users
ux = User.ls()
print(User.as_df(ux))
Source code in airt/client.py
@staticmethod
def ls(
offset: int = 0,
limit: int = 100,
disabled: bool = False,
) -> List["User"]:
"""Return the list of User instances available in the server.
To access this method, you must have super user privileges.
Args:
offset: The number of users to offset at the beginning. If **None**, then the default value **0** will be used.
limit: The maximum number of users to return from the server. If None, then the default value 100 will be used.
disabled: If set to **True**, then only the deleted users will be returned. Else, the default value **False** will
be used to return only the list of active users.
Returns:
A list of User instances available in the server.
Raises:
ConnectionError: If the server address is invalid or not reachable.
An example of displaying the details of all active users
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# Display the details of all active users
# Set the disabled parameter to True to display the details of inactive users
ux = User.ls()
print(User.as_df(ux))
```
"""
users = Client._get_data(
relative_url=f"/user/?disabled={disabled}&offset={offset}&limit={limit}"
)
ux = [
User(
uuid=user["uuid"],
username=user["username"],
first_name=user["first_name"],
last_name=user["last_name"],
email=user["email"],
subscription_type=user["subscription_type"],
super_user=user["super_user"],
disabled=user["disabled"],
created=user["created"],
is_mfa_active=user["is_mfa_active"],
phone_number=user["phone_number"],
is_phone_number_verified=user["is_phone_number_verified"],
)
for user in users
]
return ux
register_phone_number(phone_number=None, otp=None)
staticmethod
¤
Register a phone number
Registering your phone number will help you to regain access in case you forget your password and cannot access
your account. To receive the OTP via SMS, you need to register and validate your phone number. Calling this
method will send an OTP via SMS to the phone number and you need to call the validate_phone_number method
with the OTP you have received to complete the registration and validation process.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
phone_number |
Optional[str] |
Phone number to register. The phone number should follow the pattern of the country code followed by your phone number. For example, 440123456789, +440123456789, 00440123456789, +44 0123456789, and (+44) 012 345 6789 are all valid formats for registering a UK phone number. If the phone number is not passed in the arguments, then the OTP will be sent to the phone number that was already registered to the user's account. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
Dict[str, Union[str, bool]] |
A dict containing the updated user details |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of registering and validating a new phone number for the currently logged-in user
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the phone number that is currently registered
# If no phone number has been registered, None will be displayed
print(User.details()["phone_number"])
# Register a new phone number.
# If you only want to validate an existing phone number, call the
# method below without the phone_number parameter
User.register_phone_number(phone_number="{fill in phone_number}")
# The above method will send the OTP via SMS to the specified phone number,
# which you must enter below to complete the registration process
User.validate_phone_number(otp="{fill in otp}")
# Check whether the phone number has been updated and verified
# The is_phone_number_verified flag should be set to True, if the
# registration is successful
user_details = User.details()
print(user_details["phone_number"])
print(user_details["is_phone_number_verified"])
Source code in airt/client.py
@staticmethod
def register_phone_number(
phone_number: Optional[str] = None,
otp: Optional[str] = None,
) -> Dict[str, Union[str, bool]]:
"""Register a phone number
Registering your phone number will help you to regain access in case you forget your password and cannot access
your account. To receive the OTP via SMS, you need to register and validate your phone number. Calling this
method will send an OTP via SMS to the phone number and you need to call the `validate_phone_number` method
with the OTP you have received to complete the registration and validation process.
Args:
phone_number: Phone number to register. The phone number should follow the pattern of the country
code followed by your phone number. For example, **440123456789, +440123456789,
00440123456789, +44 0123456789, and (+44) 012 345 6789** are all valid formats for registering a
UK phone number. If the phone number is not passed in the arguments, then the OTP will be sent to
the phone number that was already registered to the user's account.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A dict containing the updated user details
Raises:
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of registering and validating a new phone number for the currently logged-in user
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the phone number that is currently registered
# If no phone number has been registered, None will be displayed
print(User.details()["phone_number"])
# Register a new phone number.
# If you only want to validate an existing phone number, call the
# method below without the phone_number parameter
User.register_phone_number(phone_number="{fill in phone_number}")
# The above method will send the OTP via SMS to the specified phone number,
# which you must enter below to complete the registration process
User.validate_phone_number(otp="{fill in otp}")
# Check whether the phone number has been updated and verified
# The is_phone_number_verified flag should be set to True, if the
# registration is successful
user_details = User.details()
print(user_details["phone_number"])
print(user_details["is_phone_number_verified"])
```
"""
if phone_number is not None:
phone_number = standardize_phone_number(phone_number)
req_json = dict(phone_number=phone_number, otp=otp)
return Client._post_data(
relative_url="/user/register_phone_number", json=req_json
)
reset_password(username, new_password, otp)
staticmethod
¤
Resets the password of an account either using a TOTP or SMS OTP.
We currently support two types of OTPs to reset the password for your account and you don't have to be logged in to call this method
If you have already activated the MFA for your account, then you can either pass the dynamically generated six-digit verification code from the authenticator app (TOTP) or you can also request an OTP via SMS to your registered phone number.
If the MFA is not activated already, then you can only request the OTP via SMS to your registered phone number.
To get OTP by SMS, you must first call send_sms_otp method which will send the OTP to your registered
phone number. Once you have the OTP with you, then call this method with the OTP to reset your password.
Currently, we only support the above two methods for resetting the password. In case, you don't have MFA enabled or don't have access to your registered phone number, please contact your administrator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
username |
str |
Account username to reset the password |
required |
new_password |
str |
New password to set for the account |
required |
otp |
str |
Dynamically generated six-digit verification code from the authenticator app or the OTP you have received via SMS. |
required |
Returns:
| Type | Description |
|---|---|
str |
The password reset status message |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the username or OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example for resetting the password using the verification code generated by the authentication application. The example assumes that you have already activated the MFA on your account and have access to the authentication application.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Details required to reset the password
username = "{fill in username}"
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP generated by the authenticator app
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
Client.get_token(username=username, password=new_password, otp=otp)
# Check if get_token is successful
print(User.details())
Here's an example of a Non-MFA user resetting their password using the SMS OTP. The example assumes that you have already registered and validated your phone number on our servers.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to the registered phone number
# Please do not change the message_template_name
username="{fill in username}"
User.send_sms_otp(username=username, message_template_name="reset_password")
# The above method will send the OTP via SMS to the registered phone number,
# which you must fill below along with your new password
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP received via SMS
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
# MFA users must pass the otp generated by the authenticator app below
Client.get_token(username=username, password=new_password)
# Check if get_token is successful
print(User.details())
Source code in airt/client.py
@staticmethod
def reset_password(username: str, new_password: str, otp: str) -> str:
"""Resets the password of an account either using a TOTP or SMS OTP.
We currently support two types of OTPs to reset the password for your account and you don't have to be logged
in to call this method
If you have already activated the MFA for your account, then you can either pass the dynamically generated
six-digit verification code from the authenticator app (TOTP) or you can also request an OTP via SMS to your registered phone number.
If the MFA is not activated already, then you can only request the OTP via SMS to your registered phone number.
To get OTP by SMS, you must first call `send_sms_otp` method which will send the OTP to your registered
phone number. Once you have the OTP with you, then call this method with the OTP to reset your password.
Currently, we only support the above two methods for resetting the password. In case, you don't have MFA enabled or don't
have access to your registered phone number, please contact your administrator.
Args:
username: Account username to reset the password
new_password: New password to set for the account
otp: Dynamically generated six-digit verification code from the authenticator app or the OTP you have received
via SMS.
Returns:
The password reset status message
Raises:
ValueError: If the username or OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example for resetting the password using the verification code generated by the authentication application.
The example assumes that you have already activated the MFA on your account and have access to the authentication application.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Details required to reset the password
username = "{fill in username}"
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP generated by the authenticator app
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
Client.get_token(username=username, password=new_password, otp=otp)
# Check if get_token is successful
print(User.details())
```
Here's an example of a Non-MFA user resetting their password using the SMS OTP. The example assumes that you have already registered
and validated your phone number on our servers.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to the registered phone number
# Please do not change the message_template_name
username="{fill in username}"
User.send_sms_otp(username=username, message_template_name="reset_password")
# The above method will send the OTP via SMS to the registered phone number,
# which you must fill below along with your new password
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP received via SMS
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
# MFA users must pass the otp generated by the authenticator app below
Client.get_token(username=username, password=new_password)
# Check if get_token is successful
print(User.details())
```
"""
req_json = dict(username=username, new_password=new_password, otp=otp)
return Client._post_data(relative_url="/user/reset_password", json=req_json) # type: ignore
send_sms_otp(username, message_template_name)
staticmethod
¤
Send OTP via SMS to the user's registered phone number
This method does not require a login, and you should only use it to reset your password, disable MFA, or generate a new token using SMS OTP.
Calling this method will only send an OTP to your registered phone number via SMS. Following this method
call, you should explicitly call reset_password, disable_mfa, or Client.get_token to complete
the operation with the SMS OTP.
Please remember to pass a valid message_template_name along with the request. At present, the API supports "reset_password", "disable_mfa" and "get_token" as valid message templates.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
username |
str |
Account username to send the OTP |
required |
message_template_name |
str |
The message template to use while sending the OTP via SMS. At present, the API supports "reset_password", "disable_mfa" and "get_token" as valid message templates |
required |
Returns:
| Type | Description |
|---|---|
str |
The SMS status message |
Here's an example of a Non-MFA user resetting their password using the SMS OTP. The example assumes that you have already registered and validated your phone number on our servers.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to the registered phone number
# Please do not change the message_template_name
username="{fill in username}"
User.send_sms_otp(username=username, message_template_name="reset_password")
# The above method will send the OTP via SMS to the registered phone number,
# which you must fill below along with your new password
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP received via SMS
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
# MFA users must pass the otp generated by the authenticator app below
Client.get_token(username=username, password=new_password)
# Check if get_token is successful
print(User.details())
Here's an example of how to disable MFA with SMS OTP, assuming you've already registered and validated your phone number on our servers but don't have a valid token.
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Request OTP via SMS to authenticate
# If you've already logged in, you can skip the two optional steps
# If you've already have a valid token, skip the below optional step
# and call Client.set_token instead of Client.get_token
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Optional Step: Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Request OTP via SMS to disable MFA
User.send_sms_otp(
username=username,
message_template_name="disable_mfa" # Don't change the message_template_name
)
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
User.disable_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
Source code in airt/client.py
@staticmethod
def send_sms_otp(username: str, message_template_name: str) -> str:
"""Send OTP via SMS to the user's registered phone number
This method does not require a login, and you should only use it to reset your password,
disable MFA, or generate a new token using SMS OTP.
Calling this method will only send an OTP to your registered phone number via SMS. Following this method
call, you should explicitly call `reset_password`, `disable_mfa`, or `Client.get_token` to complete
the operation with the SMS OTP.
Please remember to pass a valid message_template_name along with the request. At present, the API
supports **"reset_password"**, **"disable_mfa"** and **"get_token"** as valid message templates.
Args:
username: Account username to send the OTP
message_template_name: The message template to use while sending the OTP via SMS. At present,
the API supports **"reset_password"**, **"disable_mfa"** and **"get_token"** as valid message templates
Returns:
The SMS status message
Here's an example of a Non-MFA user resetting their password using the SMS OTP. The example assumes that you have
already registered and validated your phone number on our servers.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Request OTP via SMS to the registered phone number
# Please do not change the message_template_name
username="{fill in username}"
User.send_sms_otp(username=username, message_template_name="reset_password")
# The above method will send the OTP via SMS to the registered phone number,
# which you must fill below along with your new password
new_password = "{fill in new_password}"
otp = "{fill in otp}" # OTP received via SMS
# Reset the password
User.reset_password(username=username, new_password=new_password, otp=otp)
# Authenticate using the new credentials
# MFA users must pass the otp generated by the authenticator app below
Client.get_token(username=username, password=new_password)
# Check if get_token is successful
print(User.details())
```
Here's an example of how to disable MFA with SMS OTP, assuming you've already registered
and validated your phone number on our servers but don't have a valid token.
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Optional Step: Request OTP via SMS to authenticate
# If you've already logged in, you can skip the two optional steps
# If you've already have a valid token, skip the below optional step
# and call Client.set_token instead of Client.get_token
username="{fill in username}"
User.send_sms_otp(
username=username,
message_template_name="get_token" # Don't change the message_template_name
)
# Optional Step: Authenticate using SMS OTP
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
password="{fill in password}"
otp="{fill in otp}"
Client.get_token(username=username, password=password, otp=otp)
# Check the current MFA status
print(User.details()["is_mfa_active"])
# Request OTP via SMS to disable MFA
User.send_sms_otp(
username=username,
message_template_name="disable_mfa" # Don't change the message_template_name
)
# The send_sms_otp method will send the OTP via SMS to the registered
# phone number, which you must fill below
User.disable_mfa(otp="{fill in otp}")
# Check the current MFA status
# The is_mfa_active flag should be set to False, if the disable was successful
print(User.details()["is_mfa_active"])
```
"""
url = f"/user/send_sms_otp?username={username}&message_template_name={message_template_name}"
return Client._get_data(relative_url=url)
update(user=None, username=None, first_name=None, last_name=None, email=None, otp=None)
staticmethod
¤
Update existing user details in the server.
Please do not pass the optional user parameter unless you are a super user. Only a super user can update details for other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Optional[str] |
Account user_uuid/username to update. If not passed, then the default value None will be used to update the currently logged-in user details. |
None |
username |
Optional[str] |
New username for the user. |
None |
first_name |
Optional[str] |
New first name for the user. |
None |
last_name |
Optional[str] |
New last name for the user. |
None |
email |
Optional[str] |
New email for the user. |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if you have activated the MFA for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas DataFrame encapsulating the updated user details. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the user_uuid/username is invalid or the user have insufficient permission to access other user's data |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of updating the email address of the logged-in user
Examples:
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the registered email address of the logged-in user
print(User.details()["email"])
# Update the logged-in user's email address
# If you are a super user, you can update the details of other users by
# passing their uuid/username in the user parameter
email = "{fill in new_email}"
User.update(email=email)
# Check whether the email address has been updated for the logged-in user
print(User.details()["email"])
# If you try to use an already used email address, an exception will be raised.
try:
User.update(email=email)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
Source code in airt/client.py
@staticmethod
def update(
user: Optional[str] = None,
username: Optional[str] = None,
first_name: Optional[str] = None,
last_name: Optional[str] = None,
email: Optional[str] = None,
otp: Optional[str] = None,
) -> pd.DataFrame:
"""Update existing user details in the server.
Please do not pass the optional user parameter unless you are a super user. Only a
super user can update details for other users.
Args:
user: Account user_uuid/username to update. If not passed, then the default
value **None** will be used to update the currently logged-in user details.
username: New username for the user.
first_name: New first name for the user.
last_name: New last name for the user.
email: New email for the user.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if you have activated the MFA for your account.
Returns:
A pandas DataFrame encapsulating the updated user details.
Raises:
ValueError: If the user_uuid/username is invalid or the user have insufficient permission to access other user's data
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of updating the email address of the logged-in user
Example:
```python
# Importing necessary libraries
from airt.client import User, Client
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the registered email address of the logged-in user
print(User.details()["email"])
# Update the logged-in user's email address
# If you are a super user, you can update the details of other users by
# passing their uuid/username in the user parameter
email = "{fill in new_email}"
User.update(email=email)
# Check whether the email address has been updated for the logged-in user
print(User.details()["email"])
# If you try to use an already used email address, an exception will be raised.
try:
User.update(email=email)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
```
"""
user_uuid = User.details(user=user)["uuid"]
req_json = dict(
username=username,
first_name=first_name,
last_name=last_name,
email=email,
otp=otp,
)
response = Client._post_data(
relative_url=f"/user/{user_uuid}/update", json=req_json
)
return pd.DataFrame(response, index=[0])[User.USER_COLS]
validate_phone_number(otp=None)
staticmethod
¤
Validate a registered phone number
Please call the register_phone_number method to get the OTP via SMS and then call this method
with the OTP to complete the registration and validation process.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
otp |
Optional[str] |
The OTP you have received on your registered phone number. |
None |
Returns:
| Type | Description |
|---|---|
Dict[str, Union[str, bool]] |
A dict containing the updated user details |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
Here's an example of registering and validating a new phone number for the currently logged-in user
Examples:
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the phone number that is currently registered
# If no phone number has been registered, None will be displayed
print(User.details()["phone_number"])
# Register a new phone number.
# If you only want to validate an existing phone number, call the
# method below without the phone_number parameter
User.register_phone_number(phone_number="{fill in phone_number}")
# The above method will send the OTP via SMS to the specified phone number,
# which you must enter below to complete the registration process
User.validate_phone_number(otp="{fill in otp}")
# Check whether the phone number has been updated and verified
# The is_phone_number_verified flag should be set to True, if the
# registration is successful
user_details = User.details()
print(user_details["phone_number"])
print(user_details["is_phone_number_verified"])
Source code in airt/client.py
@staticmethod
def validate_phone_number(
otp: Optional[str] = None,
) -> Dict[str, Union[str, bool]]:
"""Validate a registered phone number
Please call the `register_phone_number` method to get the OTP via SMS and then call this method
with the OTP to complete the registration and validation process.
Args:
otp: The OTP you have received on your registered phone number.
Returns:
A dict containing the updated user details
Raises:
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
Here's an example of registering and validating a new phone number for the currently logged-in user
Example:
```python
# Importing necessary libraries
from airt.client import Client, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Display the phone number that is currently registered
# If no phone number has been registered, None will be displayed
print(User.details()["phone_number"])
# Register a new phone number.
# If you only want to validate an existing phone number, call the
# method below without the phone_number parameter
User.register_phone_number(phone_number="{fill in phone_number}")
# The above method will send the OTP via SMS to the specified phone number,
# which you must enter below to complete the registration process
User.validate_phone_number(otp="{fill in otp}")
# Check whether the phone number has been updated and verified
# The is_phone_number_verified flag should be set to True, if the
# registration is successful
user_details = User.details()
print(user_details["phone_number"])
print(user_details["is_phone_number_verified"])
```
"""
url = "/user/validate_phone_number"
return Client._get_data(relative_url=check_and_append_otp_query_param(url, otp))
cls
¤
A class for managing the APIKeys in the server.
Both the APIKey and the token can be used for accessing the airt services. However, there is a slight difference in generating and managing the two.
For generating the APIKey, you first need to get the developer token. Please refer to Client.get_token method documentation to generate one.
After logging in with your developer token, you can create any number of new APIKeys and can set an expiration date individually. You can also access other methods available in the APIKey class to list, revoke the APIKey at any time.
Here's an example of how to use the APIKey class to create a new key and use it to access the airt service.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey, User
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a new key with the given name
key_name = "{fill in key_name}"
new_key = APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# Call the set_token method to set the newly generated key
Client.set_token(token=new_key["access_token"])
# Print the logged-in user details
# If set_token fails, the line below will throw an error.
print(User.details())
__init__(self, uuid, name=None, expiry=None, disabled=None, created=None)
special
¤
Constructs a new APIKey instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
uuid |
str |
APIKey uuid. |
required |
name |
Optional[str] |
APIKey name. |
None |
expiry |
Optional[str] |
APIKey expiry date. |
None |
disabled |
Optional[bool] |
Flag to indicate the status of the APIKey. |
None |
created |
Optional[str] |
APIKey creation date. |
None |
Source code in airt/client.py
def __init__(
self,
uuid: str,
name: Optional[str] = None,
expiry: Optional[str] = None,
disabled: Optional[bool] = None,
created: Optional[str] = None,
):
"""Constructs a new APIKey instance.
Args:
uuid: APIKey uuid.
name: APIKey name.
expiry: APIKey expiry date.
disabled: Flag to indicate the status of the APIKey.
created: APIKey creation date.
"""
self.uuid = uuid
self.name = name
self.expiry = expiry
self.disabled = disabled
self.created = created
as_df(ax)
staticmethod
¤
Return the details of APIKey instances in a pandas dataframe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ax |
List[APIKey] |
List of APIKey instances. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
Details of all the APIKeys in a dataframe. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
An example of displaying the APIKeys generated by the currently logged-in user in a dataframe
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display all the APIKey instance details in a pandas dataframe
df = APIKey.as_df(APIKey.ls())
print(df)
Source code in airt/client.py
@staticmethod
def as_df(ax: List["APIKey"]) -> pd.DataFrame:
"""Return the details of APIKey instances in a pandas dataframe.
Args:
ax: List of APIKey instances.
Returns:
Details of all the APIKeys in a dataframe.
Raises:
ConnectionError: If the server address is invalid or not reachable.
An example of displaying the APIKeys generated by the currently logged-in user in a dataframe
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display all the APIKey instance details in a pandas dataframe
df = APIKey.as_df(APIKey.ls())
print(df)
```
"""
lists = get_attributes_from_instances(ax, APIKey.API_KEY_COLS) # type: ignore
return generate_df(lists, APIKey.API_KEY_COLS)
create(name, expiry=None, otp=None)
staticmethod
¤
Create a new APIKey
In order to access the airt service with the newly generated APIKey, please call the Client.set_token method
or set the APIKey value in the AIRT_SERVICE_TOKEN environment variable.
Note
-
The APIKey's name must be unique. If not, an exception will be raised while creating a new key with the name of an existing key. However, you can create a new key with the name of a revoked key.
-
The expiry for an APIKey is optional, if not passed then the default value None will be used to create an APIKey with no expiry date!
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name |
str |
The name of the APIKey. |
required |
expiry |
Union[int, datetime.timedelta, datetime.datetime] |
The validity for the APIKey. This can be an integer representing the number of days till expiry, can be an instance of timedelta (timedelta(days=x)) or can be an instance of datetime. If not passed, then the default value None will be used to create a APIKey that will never expire! |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if the MFA is enabled for your account. |
None |
Returns:
| Type | Description |
|---|---|
Dict[str, str] |
The APIKey and its type as a dictionary. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the user is not authenticated. |
ValueError |
If the user tries to create a new APIKey with an existing key name. |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
In the following example, a new APIKey is created with a 10-day expiration date and used to access the airt service.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key with the given name and set the expiry to 10 days from now.
# If the expiry parameter is not specified, a key with no expiry date is created.
key_name = "{fill in key_name}"
new_key_details = APIKey.create(name=key_name, expiry=10)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# If a new key with the same name is created, an exception will be raised.
# However, you can create a new key with the name of a revoked key.
try:
APIKey.create(name=key_name, expiry=10)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
# Finally, either call the below method to set the newly generated key
# or store it in the AIRT_SERVICE_TOKEN environment variable.
Client.set_token(token=new_key_details["access_token"])
# If set_token fails, the line below will throw an error.
print(APIKey.details(apikey=key_name))
Source code in airt/client.py
@staticmethod
def create(
name: str,
expiry: Optional[Union[int, timedelta, datetime]] = None,
otp: Optional[str] = None,
) -> Dict[str, str]:
"""Create a new APIKey
In order to access the airt service with the newly generated APIKey, please call the `Client.set_token` method
or set the APIKey value in the **AIRT_SERVICE_TOKEN** environment variable.
!!! note
- The APIKey's name must be unique. If not, an exception will be raised while creating a new key with the name of an existing key.
However, you can create a new key with the name of a revoked key.
- The expiry for an APIKey is optional, if not passed then the default value **None** will be used to create an APIKey with no expiry date!
Args:
name: The name of the APIKey.
expiry: The validity for the APIKey. This can be an integer representing the number of days till expiry, can be
an instance of timedelta (timedelta(days=x)) or can be an instance of datetime. If not passed, then the default value
**None** will be used to create a APIKey that will never expire!
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if the MFA is enabled for your account.
Returns:
The APIKey and its type as a dictionary.
Raises:
ValueError: If the user is not authenticated.
ValueError: If the user tries to create a new APIKey with an existing key name.
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
In the following example, a new APIKey is created with a 10-day expiration date and used to access the airt service.
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key with the given name and set the expiry to 10 days from now.
# If the expiry parameter is not specified, a key with no expiry date is created.
key_name = "{fill in key_name}"
new_key_details = APIKey.create(name=key_name, expiry=10)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# If a new key with the same name is created, an exception will be raised.
# However, you can create a new key with the name of a revoked key.
try:
APIKey.create(name=key_name, expiry=10)
print("Should not print this, the above line should raise an exception")
raise RuntimeException()
except ValueError as e:
print("Expected to fail, everything is fine")
# Finally, either call the below method to set the newly generated key
# or store it in the AIRT_SERVICE_TOKEN environment variable.
Client.set_token(token=new_key_details["access_token"])
# If set_token fails, the line below will throw an error.
print(APIKey.details(apikey=key_name))
```
"""
if expiry is None:
expiry_date = expiry
else:
if isinstance(expiry, int):
delta = datetime.now() + timedelta(days=expiry)
elif isinstance(expiry, timedelta):
delta = datetime.now() + expiry
else:
delta = expiry
expiry_date = delta.strftime("%Y-%m-%dT%H:%M")
return Client._post_data(
relative_url="/apikey",
json=dict(name=name, expiry=expiry_date, otp=otp),
)
details(apikey)
staticmethod
¤
Return details of an APIKey.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
apikey |
str |
APIKey uuid/name. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas Dataframe encapsulating the details of the APIKey. |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the APIKey uuid is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
An example to get details of an APIKey
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# To display the details of all keys created by the user, use the method below.
df = APIKey.as_df(APIKey.ls())
print(df)
Source code in airt/client.py
@staticmethod
def details(apikey: str) -> pd.DataFrame:
"""Return details of an APIKey.
Args:
apikey: APIKey uuid/name.
Returns:
A pandas Dataframe encapsulating the details of the APIKey.
Raises:
ValueError: If the APIKey uuid is invalid.
ConnectionError: If the server address is invalid or not reachable.
An example to get details of an APIKey
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Display the details of the newly created key
print(APIKey.details(apikey=key_name))
# To display the details of all keys created by the user, use the method below.
df = APIKey.as_df(APIKey.ls())
print(df)
```
"""
details = Client._get_data(relative_url=f"/apikey/{apikey}")
return pd.DataFrame(details, index=[0])[APIKey.API_KEY_COLS]
ls(user=None, offset=0, limit=100, include_disabled=False)
staticmethod
¤
Return the list of APIKeys instances.
Please do not pass the user parameter unless you are a super user. Only a super user can view the APIKeys created by other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
user |
Optional[str] |
user_uuid/username associated with the APIKey. Please call |
None |
offset |
int |
The number of APIKeys to offset at the beginning. If None, then the default value 0 will be used. |
0 |
limit |
int |
The maximum number of APIKeys to return from the server. If None, then the default value 100 will be used. |
100 |
include_disabled |
bool |
If set to True, then the disabled APIKeys will also be included in the result. |
False |
Returns:
| Type | Description |
|---|---|
List[APIKey] |
A list of APIKey instances. |
Exceptions:
| Type | Description |
|---|---|
ConnectionError |
If the server address is invalid or not reachable. |
ValueError |
If the user_uuid is invalid. |
An example of displaying the APIKeys generated by the currently logged-in user
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Get the list of all APIKey instances created by the currently logged-in user.
# If you are a super user, you can view the APIkeys created by other users by
# passing their uuid/username in the user parameter.
ax = APIKey.ls()
print(ax)
# Display the details of the instances in a pandas dataframe
df = APIKey.as_df(ax)
print(df)
Source code in airt/client.py
@staticmethod
def ls(
user: Optional[str] = None,
offset: int = 0,
limit: int = 100,
include_disabled: bool = False,
) -> List["APIKey"]:
"""Return the list of APIKeys instances.
Please do not pass the **user** parameter unless you are a super user. Only a super user can view
the APIKeys created by other users.
Args:
user: user_uuid/username associated with the APIKey. Please call `User.details` method of the User class to get your user_uuid.
If not passed, then the currently logged-in user_uuid will be used.
offset: The number of APIKeys to offset at the beginning. If None, then the default value 0 will be used.
limit: The maximum number of APIKeys to return from the server. If None, then the default value 100 will be used.
include_disabled: If set to **True**, then the disabled APIKeys will also be included in the result.
Returns:
A list of APIKey instances.
Raises:
ConnectionError: If the server address is invalid or not reachable.
ValueError: If the user_uuid is invalid.
An example of displaying the APIKeys generated by the currently logged-in user
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Get the list of all APIKey instances created by the currently logged-in user.
# If you are a super user, you can view the APIkeys created by other users by
# passing their uuid/username in the user parameter.
ax = APIKey.ls()
print(ax)
# Display the details of the instances in a pandas dataframe
df = APIKey.as_df(ax)
print(df)
```
"""
user_uuid = User.details(user=user)["uuid"]
apikeys = Client._get_data(
relative_url=f"/{user_uuid}/apikey?include_disabled={include_disabled}&offset={offset}&limit={limit}"
)
ax = [
APIKey(
uuid=apikey["uuid"],
name=apikey["name"],
expiry=apikey["expiry"],
disabled=apikey["disabled"],
created=apikey["created"],
)
for apikey in apikeys
]
return ax
revoke(keys, user=None, otp=None)
staticmethod
¤
Revoke one or more APIKeys
Please do not pass the user parameter unless you are a super user. Only a super user can revoke the APIKeys created by other users.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
keys |
Union[str, List[str], List[APIKey]] |
APIKey uuid/name to revoke. To revoke multiple keys, either pass a list of APIKey uuid/names or a list of APIKey instances. |
required |
user |
Optional[str] |
user_uuid/username associated with the APIKey. Please call |
None |
otp |
Optional[str] |
Dynamically generated six-digit verification code from the authenticator app. Please pass this parameter only if the MFA is enabled for your account. |
None |
Returns:
| Type | Description |
|---|---|
DataFrame |
A pandas Dataframe encapsulating the details of the deleted APIKey(s). |
Exceptions:
| Type | Description |
|---|---|
ValueError |
If the APIKey uuid is invalid. |
ValueError |
If the user_uuid is invalid. |
ValueError |
If the OTP is invalid. |
ConnectionError |
If the server address is invalid or not reachable. |
An example to revoke a single APIKey by name
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Check that the newly created key exists
print([key.name for key in APIKey.ls()])
# Revoke the newly created key
# To delete multiple keys, pass a list of key names or key instances
APIKey.revoke(keys=key_name)
# Check that the newly created key does not exists
print([key.name for key in APIKey.ls()])
Here's an example of a super user revoking all APIkeys generated by a specific user.
Examples:
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# List the APIKeys generated by a specific user
user = "{fill in other_username}"
ax = APIKey.ls(user=user)
print([key.name for key in ax])
# Revoke the APIKeys
APIKey.revoke(keys=ax, user=user)
# Check that all APIkeys have been revoked
print([key.name for key in APIKey.ls(user=user)])
Source code in airt/client.py
@staticmethod
def revoke(
keys: Union[str, List[str], List["APIKey"]],
user: Optional[str] = None,
otp: Optional[str] = None,
) -> pd.DataFrame:
"""Revoke one or more APIKeys
Please do not pass the **user** parameter unless you are a super user. Only a super user can revoke the
APIKeys created by other users.
Args:
keys: APIKey uuid/name to revoke. To revoke multiple keys, either pass a list of APIKey uuid/names or a list of APIKey instances.
user: user_uuid/username associated with the APIKey. Please call `User.details` method of the User class to get your user_uuid/username.
If not passed, then the currently logged-in user will be used.
otp: Dynamically generated six-digit verification code from the authenticator app. Please pass this
parameter only if the MFA is enabled for your account.
Returns:
A pandas Dataframe encapsulating the details of the deleted APIKey(s).
Raises:
ValueError: If the APIKey uuid is invalid.
ValueError: If the user_uuid is invalid.
ValueError: If the OTP is invalid.
ConnectionError: If the server address is invalid or not reachable.
An example to revoke a single APIKey by name
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate
Client.get_token(username="{fill in username}", password="{fill in password}")
# Create a key without an expiry date in the given name
key_name = "{fill in key_name}"
APIKey.create(name=key_name)
# Check that the newly created key exists
print([key.name for key in APIKey.ls()])
# Revoke the newly created key
# To delete multiple keys, pass a list of key names or key instances
APIKey.revoke(keys=key_name)
# Check that the newly created key does not exists
print([key.name for key in APIKey.ls()])
```
Here's an example of a super user revoking all APIkeys generated by a specific user.
Example:
```python
# Importing necessary libraries
from airt.client import Client, APIKey
# Authenticate with super user privileges
Client.get_token(
username="{fill in super_user_username}",
password="{fill in super_user_password}"
)
# List the APIKeys generated by a specific user
user = "{fill in other_username}"
ax = APIKey.ls(user=user)
print([key.name for key in ax])
# Revoke the APIKeys
APIKey.revoke(keys=ax, user=user)
# Check that all APIkeys have been revoked
print([key.name for key in APIKey.ls(user=user)])
```
"""
user_uuid = User.details(user=user)["uuid"]
_keys = APIKey._get_key_names(keys)
response_list = []
for key_uuid in _keys:
url = f"/{user_uuid}/apikey/{key_uuid}"
response = Client._delete_data(
relative_url=check_and_append_otp_query_param(url, otp)
)
response_list.append(response)
return generate_df(response_list, APIKey.API_KEY_COLS)