SUMM.AI API

Apps

Configuration of SUMM.AI API app

class integreat_cms.summ_ai_api.apps.SummAiApiConfig(app_name, app_module)[source]

Bases: AppConfig

SUMM.AI API config inheriting the django AppConfig

name: Final[str] = 'integreat_cms.summ_ai_api'[source]: Full Python path to the application

ready() → None[source]

Inform about the SUMM.AI configuration

Return type:: None

verbose_name: Final[Promise] = 'SUMM.AI API'[source]: Human-readable name for the application

SUMM.AI API Client

This module contains the API client to interact with the SUMM.AI API

class integreat_cms.summ_ai_api.summ_ai_api_client.SummAiApiClient(request: HttpRequest, form_class: ModelFormMetaclass)[source]

Bases: MachineTranslationApiClient

SUMM.AI API client to get German pages in Easy German language.

Parameters:

request (HttpRequest)
form_class (ModelFormMetaclass)

__init__(request: HttpRequest, form_class: ModelFormMetaclass) → None[source]

Constructor initializes the class variables

Parameters:

region – The current region
form_class (ModelFormMetaclass) – The CustomContentModelForm subclass of the current content type
request (HttpRequest)

Return type:

None

static check_internal_server_error(response_status: int) → bool[source]

Checks if we got a HTTP 500 error

Parameters:: response_status (int) – The response-status form SummAiApi-Request
Returns:: False (if the response_status is not 500)
Raises:: SummAiRuntimeError – If the response_status is 500
Return type:: bool

static check_rate_limit_exceeded(response_status: int) → bool[source]

Checks if the limit of requests was exceeded (triggered by response_status=429 or 529) and logs this occurrence

Parameters:: response_status (int) – The response-status form SummAiApi-Request
Returns:: False (if the response_status is neither 429 nor 529)
Raises:: SummAiRateLimitingExceeded – If the response_status is 429 or 529
Return type:: bool

filter_exceeds_limit() → None[source]

This method removes content objects from the main queryset if translating them would exceed the translation budget.

The removed elements are stored in order to show users batched error messages after all objects have been handled.

Parameters:: source_language – The source language slug
Return type:: None

static get_target_language_key(_target_language: Language) → str[source]

Dummy placeholder for the (unused) abstract method from MachineTranslationApiClient

Parameters:: _target_language (Language)
Return type:: str

invoke_translation_api() → None[source]

Dummy placeholder for the (unused) abstract method from MachineTranslationApiClient

Return type:: None

translate_queryset(queryset: list[Page], language_slug: str) → None[source]

Translate a queryset of content objects from German into Easy German. To increase the speed of the translations, all operations are parallelized.

Parameters:

queryset (list[Page]) – The queryset which should be translated
language_slug (str) – The target language slug to translate into

Return type:

None

async translate_text_field(session: ClientSession, text_field: TextField) → TextField[source]

Uses aiohttp.ClientSession.post() to perform an asynchronous POST request to the SUMM.AI API. After the translation is finished, the processing is delegated to the specific textfield’s translate().

Parameters:

session (ClientSession) – The session object which is used for the request
text_field (TextField) – The text field to be translated

Returns:

The modified text field containing the translated text

Return type:

TextField

Note that worker() currently not only counts SummAiRateLimitingExceeded but also SummAiInvalidJSONError as a rate limit hit and enqueues the task again.

Raises:

SummAiRuntimeError – If text_field is none or text is empty
SummAiInvalidJSONError – Invalid JSON response returned by the API
SummAiRateLimitingExceeded – If query runs into rate limit (429 or 529 response)

Parameters:

session (ClientSession)
text_field (TextField)

Return type:

TextField

async translate_text_fields(loop: AbstractEventLoop, text_fields: Iterator[TextField]) → chain[list[TextField]][source]

Translate a list of text fields from German into Easy German. Create an async task translate_text_field() for each entry.

Parameters:

loop (AbstractEventLoop) – The asyncio event loop
text_fields (Iterator[TextField]) – The text fields to be translated

Returns:

The list of completed text fields

Return type:

chain[list[TextField]]

classmethod validate_response(response_data: dict, response_status: int) → bool[source]

Checks if translated text is found in SummAiApi-response

Parameters:

response_data (dict) – The response-data from SummAiApi
response_status (int) – The response-status form SummAiApi-Request

Returns:

True or False

Raises:

SummAiRuntimeError – The response doesn’t contain the field translated_text.

Return type:

bool

SUMM.AI Provider

class integreat_cms.summ_ai_api.summ_ai_provider.SummAiProvider[source]

Bases: MachineTranslationProvider

The provider for SUMM.AI machine translations

api_client[source]

The API client class for this provider

alias of SummAiApiClient

bulk_only_for_staff: bool = True[source]: Whether to require the staff permission for bulk actions

enabled: bool = False[source]: Whether the provider is globally enabled

name: str = 'SUMM.AI'[source]: The readable name for this provider

region_enabled_attr: str | None = 'summ_ai_enabled'[source]: The name of the region attribute which denotes whether the provider is enabled in a region

supported_source_languages: list[str] = ['de'][source]: The supported source languages

supported_target_languages: list[str] = ['de-si'][source]: The supported target languages

Utils

This module contains helpers for the SUMM.AI API client

class integreat_cms.summ_ai_api.utils.HTMLField(name: str, translation: PageTranslation)[source]

Bases: object

A class for more complex HTML fields which are splitted into segments

Parameters:

name (str)
translation (PageTranslation)

__init__(name: str, translation: PageTranslation) → None[source]

Parse the HTML string into an lxml tree object and split into segments

Parameters:

html – The HTML string content of this field
name (str)
translation (PageTranslation)

Return type:

None

property exception: SummAiException | None[source]

Check if any of the segments experienced an error

Returns:: The first exception of this HTML field

html: HtmlElement = None[source]: The current HTML stream

name: str[source]: The name of the corresponding model field

segments: list[HtmlElement] = [][source]: The list of HTML segments

property translated_text: str | None[source]

Assemble the content of the HTML segments into a HTML string again

Returns:: The translated HTML

class integreat_cms.summ_ai_api.utils.HTMLSegment(segment: HtmlElement)[source]

Bases: TextField

A class for translatable HTML segments

Parameters:: segment (HtmlElement)

__init__(segment: HtmlElement) → None[source]

Convert the lxml tree element to a flat text string. Preserve <br> tags as new lines characters. Remove all inner tags but keep their text content. Unescape all special HTML entities into unicode characters.

Parameters:: segment (HtmlElement) – The current HTML segment
Return type:: None

segment: HtmlElement[source]: The current HTML segment

translate(translated_text: str) → None[source]

Translate the current HTML segment and create new sub elements for line breaks

Parameters:: translated_text (str) – The translated text
Return type:: None

class integreat_cms.summ_ai_api.utils.PatientTaskQueue(tasks: list[T], wait_time: float = 30.0, max_retries: int = 5, abort_function: Callable | None = None)[source]

Bases: deque, Generic[T]

A ‘patient’ task queue which only hands out sleep tasks after a task was reported as failed.

Parameters:

last_rate_limit – The UNIX timestamp when the last rate limited request occurred
wait_time (float) – Seconds to wait after running into the rate limit before sending the next requests
max_retries (int) – Maximum amount of retries for a string to translate before giving up
tasks (list[T]) – List of request tasks
abort_function (Callable | None) – Function to call for each unfinished task if the queue is aborted

__init__(tasks: list[T], wait_time: float = 30.0, max_retries: int = 5, abort_function: Callable | None = None) → None[source]

Constructor initializes the class variables

Parameters:

tasks (list[T]) – List of request tasks
wait_time (float) – Waiting time until start next request in seconds
max_retries (int) – Maximum retries before giving up
abort_function (Callable | None) – Function to call for each unfinished task if the queue is aborted. Takes two arguments: The task (asyncio.Future) and the reason given (str). Can be None instead to do nothing.

Return type:

None

abort(reason: str = 'Aborted') → None[source]

Abort the Queue, handling unfinished task according to the supplied abort function.

Parameters:: reason (str) – The reason why the queue was aborted that is to be handed to the supplied abort function.
Return type:: None

completed(task: T) → None[source]

A task was completed, reset the retry counter

Parameters:: task (T) – The task that failed because of the rate limiting
Return type:: None

hit_rate_limit(task: T) → None[source]

A task hit the rate limit, so wait a bit and reschedule the task

Parameters:: task (T) – The task that failed because of the rate limiting
Return type:: None

last_rate_limit: float | None = None[source]: The UNIX timestamp when the last rate limited request occurred

max_retries: int = 5[source]: Maximum amount of retries for a string to translate before giving up

wait_time: float = 30.0[source]: Seconds to wait after running into the rate limit before sending the next requests

exception integreat_cms.summ_ai_api.utils.SummAiException[source]

Bases: Exception

Base class for custom SUMM.AI exceptions

exception integreat_cms.summ_ai_api.utils.SummAiInvalidJSONError[source]

Bases: SummAiException

Custom Exception class for faulty responses from SUMM.AI

exception integreat_cms.summ_ai_api.utils.SummAiRateLimitingExceeded[source]

Bases: SummAiException

Custom Exception class for running into rate limit in SUMM.AI

exception integreat_cms.summ_ai_api.utils.SummAiRuntimeError[source]

Bases: SummAiException

Custom Exception class for any other errors during interaction with SUMM.AI

class integreat_cms.summ_ai_api.utils.TextField(name: str, translation: PageTranslation)[source]

Bases: object

A class for simple text fields

Parameters:

name (str)
translation (PageTranslation)

__init__(name: str, translation: PageTranslation) → None[source]

Constructor initializes the class variables

Parameters:

text – The text to be translated
name (str)
translation (PageTranslation)

Return type:

None

exception: Exception | None = None[source]: The exception which occurred during translation, if any

name: str[source]: The name of the corresponding model field

text: str[source]: The source text

translate(translated_text: str) → None[source]

Translate the text of the current text field

Parameters:: translated_text (str) – The translated text
Return type:: None

translated_text: str = ''[source]: The translated text

class integreat_cms.summ_ai_api.utils.TranslationHelper(request: HttpRequest, form_class: ModelFormMetaclass, object_instance: Page)[source]

Bases: object

Custom helper class for interaction with SUMM.AI

Parameters:

request (HttpRequest) – The current request
form_class (ModelFormMetaclass) – The subclass of the current content type
object_instance (Page) – The current object instance to be translated
german_translation – The German source translation of the object instance
valid – Wether or not the translation was successful
text_fields – The text fields of this helper
html_fields – The HTML fields of this helper

__init__(request: HttpRequest, form_class: ModelFormMetaclass, object_instance: Page) → None[source]

Constructor initializes the class variables

Parameters:

request (HttpRequest) – current request
form_class (ModelFormMetaclass) – The CustomContentModelForm subclass of the current content type
object_instance (Page) – The current object instance

Return type:

None

commit(easy_german: Language) → bool[source]

Save the translated changes to the database

Parameters:: easy_german (Language) – The language object of Easy German
Returns:: Whether the commit was successful
Return type:: bool

property fields: list[HTMLField | TextField][source]

Get all fields of this helper instance

Returns:: All fields which need to be translated

get_text_fields() → list[HTMLSegment][source]

Get all text fields of this helper instance (all native text_fields combined with all segments of all html_fields)

Returns:: All text fields and segments which need to be translated
Return type:: list[HTMLSegment]

valid: bool = True[source]: Wether or not the translation was successful

async integreat_cms.summ_ai_api.utils.worker(loop: asyncio.AbstractEventLoop, task_generator: PatientTaskQueue[partial], identifier: str) → list[Any][source]

Continuously gets a task from the queue and executes it. Stops once no more tasks are available. This form makes it easy to always have at most n concurrent tasks as well as intermittent wait times through the task generator.

Catches SummAiRateLimitingExceeded and SummAiInvalidJSONError and counts them as rate limit hits in order enqueue them again.

Parameters:

loop (asyncio.AbstractEventLoop) – The asyncio event loop to execute tasks in
task_generator (PatientTaskQueue[partial]) – Queue to execute tasks from
identifier (str) – Identifyer of the worker (for logging purposes)

Returns:

A list of task-results

Return type:

list[Any]