SUMM.AI API

Apps

Configuration of SUMM.AI API app

class integreat_cms.summ_ai_api.apps.SummAiApiConfig(app_name, app_module)

Bases: AppConfig

SUMM.AI API config inheriting the django AppConfig

__init__(app_name, app_module)

classmethod create(entry)

Factory that creates an app config from an entry in INSTALLED_APPS.

default_auto_field

get_model(model_name, require_ready=True)

Return the model with the given case-insensitive model_name.

Raise LookupError if no model exists with this name.

get_models(include_auto_created=False, include_swapped=False)

Return an iterable of models.

By default, the following models aren’t included:

• auto-created models for many-to-many relations without an explicit intermediate table,

• models that have been swapped out.

Set the corresponding keyword argument to True to include such models. Keyword arguments aren’t documented; they’re a private API.

import_models()

name: Final[str] = 'integreat_cms.summ_ai_api'

Full Python path to the application

ready() → None

Inform about the SUMM.AI configuration

Return type:

None

verbose_name: Final[Promise] = 'SUMM.AI API'

Human-readable name for the application

SUMM.AI API Client

This module contains the API client to interact with the SUMM.AI API

class integreat_cms.summ_ai_api.summ_ai_api_client.SummAiApiClient(request: HttpRequest, form_class: ModelFormMetaclass)

Bases: MachineTranslationApiClient

SUMM.AI API client to get German pages in Easy German language.

Parameters:

• request (HttpRequest)

• form_class (ModelFormMetaclass)

__init__(request: HttpRequest, form_class: ModelFormMetaclass) → None

Constructor initializes the class variables

Parameters:

• region – The current region

• form_class (ModelFormMetaclass) – The CustomContentModelForm subclass of the current content type

• request (HttpRequest)

Return type:

None

static check_internal_server_error(response_status: int) → bool

Checks if we got a HTTP 500 error

Parameters:

response_status (int) – The response-status form SummAiApi-Request

Returns:

False (if the response_status is not 500)

Raises:

SummAiRuntimeError – If the response_status is 500

Return type:

bool

static check_rate_limit_exceeded(response_status: int) → bool

Checks if the limit of requests was exceeded (triggered by response_status=429 or 529) and logs this occurrence

Parameters:

response_status (int) – The response-status form SummAiApi-Request

Returns:

False (if the response_status is neither 429 nor 529)

Raises:

SummAiRateLimitingExceeded – If the response_status is 429 or 529

Return type:

bool

check_usage(region: Region, source_translation: EventTranslation | PageTranslation | POITranslation) → tuple[bool, int]

This function checks if the attempted translation would exceed the region’s word limit

Parameters:

• region (Region) – region for which to check usage

• source_translation (EventTranslation | (PageTranslation | POITranslation)) – single content object

Returns:

translation would exceed limit, region budget, attempted translation word count

Return type:

tuple[bool, int]

form_class: ModelFormMetaclass

The CustomContentModelForm

region: Region

The current region

request: HttpRequest

The current request

translate_object(obj: Event | Page | POI, language_slug: str) → None

This function translates one content object

Parameters:

• obj (Event | Page | POI) – The content object

• language_slug (str) – The target language slug

Return type:

None

translate_queryset(queryset: list[Page], language_slug: str) → None

Translate a queryset of content objects from German into Easy German. To increase the speed of the translations, all operations are parallelized.

Parameters:

• queryset (list[Page]) – The queryset which should be translated

• language_slug (str) – The target language slug to translate into

Return type:

None

async translate_text_field(session: ClientSession, text_field: TextField) → TextField

Uses aiohttp.ClientSession.post() to perform an asynchronous POST request to the SUMM.AI API. After the translation is finished, the processing is delegated to the specific textfield’s translate().

Parameters:

• session (ClientSession) – The session object which is used for the request

• text_field (TextField) – The text field to be translated

Returns:

The modified text field containing the translated text

Return type:

TextField

Note that worker() currently not only counts SummAiRateLimitingExceeded but also SummAiInvalidJSONError as a rate limit hit and enqueues the task again.

Raises:

• SummAiRuntimeError – If text_field is none or text is empty

• SummAiInvalidJSONError – Invalid JSON response returned by the API

• SummAiRateLimitingExceeded – If query runs into rate limit (429 or 529 response)

Parameters:

• session (ClientSession)

• text_field (TextField)

Return type:

TextField

async translate_text_fields(loop: AbstractEventLoop, text_fields: Iterator[TextField]) → chain[list[TextField]]

Translate a list of text fields from German into Easy German. Create an async task translate_text_field() for each entry.

Parameters:

• loop (AbstractEventLoop) – The asyncio event loop

• text_fields (Iterator[TextField]) – The text fields to be translated

Returns:

The list of completed text fields

Return type:

chain[list[TextField]]

classmethod validate_response(response_data: dict, response_status: int) → bool

Checks if translated text is found in SummAiApi-response

Parameters:

• response_data (dict) – The response-data from SummAiApi

• response_status (int) – The response-status form SummAiApi-Request

Returns:

True or False

Raises:

SummAiRuntimeError – The response doesn’t contain the field translated_text.

Return type:

bool

SUMM.AI Provider

class integreat_cms.summ_ai_api.summ_ai_provider.SummAiProvider

Bases: MachineTranslationProvider

The provider for SUMM.AI machine translations

__init__()

api_client

The API client class for this provider

alias of SummAiApiClient

bulk_only_for_staff: bool = True

Whether to require the staff permission for bulk actions

enabled: bool = False

Whether the provider is globally enabled

classmethod is_enabled(region: Region, language: Language) → bool

Whether this provider is enabled for a given region and language. Call this from the parent class.

Parameters:

• region (Region) – The given region

• language (Language) – The given language

Returns:

Wether this provider is enabled for the given region and language

Return type:

bool

is_needed(queryset: QuerySet[Event | Page | POI], target_language: Language) → list[Event | Page | POI]

Checks if a machine translation is needed, thus checking if the translation status is UP_TO_DATE or MACHINE_TRANSLATED and then returns a lit of translations which are to be updated

Parameters:

• queryset (QuerySet[Event | Page | POI]) – The content model which should be translated

• target_language (Language) – The target language

Returns:

translations which need to be translated and updated

Return type:

list[Event | Page | POI]

static is_permitted(region: Region, user: SimpleLazyObject, content_type: ModelBase) → bool

Checks if a machine translation is permitted, i.e. if for the given region, MT of the given content type is allowed and MT into the target language is enabled for the requesting user.

Parameters:

• region (Region) – The current region

• user (SimpleLazyObject) – The current user

• content_type (ModelBase) – The content model which should be translated

Returns:

Whether the translation is permitted

Return type:

bool

name: str = 'SUMM.AI'

The readable name for this provider

region_enabled_attr: str | None = 'summ_ai_enabled'

The name of the region attribute which denotes whether the provider is enabled in a region

supported_source_languages: list[str] = ['de']

The supported source languages

supported_target_languages: list[str] = ['de-si']

The supported target languages

Utils

This module contains helpers for the SUMM.AI API client

class integreat_cms.summ_ai_api.utils.HTMLField(name: str, translation: PageTranslation)

Bases: object

A class for more complex HTML fields which are splitted into segments

Parameters:

• name (str)

• translation (PageTranslation)

__init__(name: str, translation: PageTranslation) → None

Parse the HTML string into an lxml tree object and split into segments

Parameters:

• html – The HTML string content of this field

• name (str)

• translation (PageTranslation)

Return type:

None

property exception: SummAiException | None

Check if any of the segments experienced an error

Returns:

The first exception of this HTML field

html: HtmlElement = None

The current HTML stream

name: str

The name of the corresponding model field

segments: list[HtmlElement] = []

The list of HTML segments

property translated_text: str | None

Assemble the content of the HTML segments into a HTML string again

Returns:

The translated HTML

class integreat_cms.summ_ai_api.utils.HTMLSegment(segment: HtmlElement)

Bases: TextField

A class for translatable HTML segments

Parameters:

segment (HtmlElement)

__init__(segment: HtmlElement) → None

Convert the lxml tree element to a flat text string. Preserve <br> tags as new lines characters. Remove all inner tags but keep their text content. Unescape all special HTML entities into unicode characters.

Parameters:

segment (HtmlElement) – The current HTML segment

Return type:

None

exception: Exception | None = None

The exception which occurred during translation, if any

name: str

The name of the corresponding model field

segment: HtmlElement

The current HTML segment

text: str

The source text

translate(translated_text: str) → None

Translate the current HTML segment and create new sub elements for line breaks

Parameters:

translated_text (str) – The translated text

Return type:

None

translated_text: str = ''

The translated text

class integreat_cms.summ_ai_api.utils.PatientTaskQueue(tasks: list[T], wait_time: float = 30.0, max_retries: int = 5, abort_function: Callable | None = None)

Bases: deque, Generic[T]

A ‘patient’ task queue which only hands out sleep tasks after a task was reported as failed.

Parameters:

• last_rate_limit – The UNIX timestamp when the last rate limited request occurred

• wait_time (float) – Seconds to wait after running into the rate limit before sending the next requests

• max_retries (int) – Maximum amount of retries for a string to translate before giving up

• tasks (list[T]) – List of request tasks

• abort_function (Callable | None) – Function to call for each unfinished task if the queue is aborted

__init__(tasks: list[T], wait_time: float = 30.0, max_retries: int = 5, abort_function: Callable | None = None) → None

Constructor initializes the class variables

Parameters:

• tasks (list[T]) – List of request tasks

• wait_time (float) – Waiting time until start next request in seconds

• max_retries (int) – Maximum retries before giving up

• abort_function (Callable | None) – Function to call for each unfinished task if the queue is aborted. Takes two arguments: The task (asyncio.Future) and the reason given (str). Can be None instead to do nothing.

Return type:

None

abort(reason: str = 'Aborted') → None

Abort the Queue, handling unfinished task according to the supplied abort function.

Parameters:

reason (str) – The reason why the queue was aborted that is to be handed to the supplied abort function.

Return type:

None

append()

Add an element to the right side of the deque.

appendleft()

Add an element to the left side of the deque.

clear()

Remove all elements from the deque.

completed(task: T) → None

A task was completed, reset the retry counter

Parameters:

task (T) – The task that failed because of the rate limiting

Return type:

None

copy()

Return a shallow copy of a deque.

count()

D.count(value) – return number of occurrences of value

extend()

Extend the right side of the deque with elements from the iterable

extendleft()

Extend the left side of the deque with elements from the iterable

hit_rate_limit(task: T) → None

A task hit the rate limit, so wait a bit and reschedule the task

Parameters:

task (T) – The task that failed because of the rate limiting

Return type:

None

index()

D.index(value, [start, [stop]]) – return first index of value. Raises ValueError if the value is not present.

insert()

D.insert(index, object) – insert object before index

last_rate_limit: float | None = None

The UNIX timestamp when the last rate limited request occurred

max_retries: int = 5

Maximum amount of retries for a string to translate before giving up

maxlen

maximum size of a deque or None if unbounded

pop()

Remove and return the rightmost element.

popleft()

Remove and return the leftmost element.

remove()

D.remove(value) – remove first occurrence of value.

reverse()

D.reverse() – reverse IN PLACE

rotate()

Rotate the deque n steps to the right (default n=1). If n is negative, rotates left.

wait_time: float = 30.0

Seconds to wait after running into the rate limit before sending the next requests

exception integreat_cms.summ_ai_api.utils.SummAiException

Bases: Exception

Base class for custom SUMM.AI exceptions

__init__(*args, **kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception integreat_cms.summ_ai_api.utils.SummAiInvalidJSONError

Bases: SummAiException

Custom Exception class for faulty responses from SUMM.AI

__init__(*args, **kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception integreat_cms.summ_ai_api.utils.SummAiRateLimitingExceeded

Bases: SummAiException

Custom Exception class for running into rate limit in SUMM.AI

__init__(*args, **kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception integreat_cms.summ_ai_api.utils.SummAiRuntimeError

Bases: SummAiException

Custom Exception class for any other errors during interaction with SUMM.AI

__init__(*args, **kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class integreat_cms.summ_ai_api.utils.TextField(name: str, translation: PageTranslation)

Bases: object

A class for simple text fields

Parameters:

• name (str)

• translation (PageTranslation)

__init__(name: str, translation: PageTranslation) → None

Constructor initializes the class variables

Parameters:

• text – The text to be translated

• name (str)

• translation (PageTranslation)

Return type:

None

exception: Exception | None = None

The exception which occurred during translation, if any

name: str

The name of the corresponding model field

text: str

The source text

translate(translated_text: str) → None

Translate the text of the current text field

Parameters:

translated_text (str) – The translated text

Return type:

None

translated_text: str = ''

The translated text

class integreat_cms.summ_ai_api.utils.TranslationHelper(request: HttpRequest, form_class: ModelFormMetaclass, object_instance: Page)

Bases: object

Custom helper class for interaction with SUMM.AI

Parameters:

• request (HttpRequest) – The current request

• form_class (ModelFormMetaclass) – The subclass of the current content type

• object_instance (Page) – The current object instance to be translated

• german_translation – The German source translation of the object instance

• valid – Wether or not the translation was successful

• text_fields – The text fields of this helper

• html_fields – The HTML fields of this helper

__init__(request: HttpRequest, form_class: ModelFormMetaclass, object_instance: Page) → None

Constructor initializes the class variables

Parameters:

• request (HttpRequest) – current request

• form_class (ModelFormMetaclass) – The CustomContentModelForm subclass of the current content type

• object_instance (Page) – The current object instance

Return type:

None

commit(easy_german: Language) → bool

Save the translated changes to the database

Parameters:

easy_german (Language) – The language object of Easy German

Returns:

Whether the commit was successful

Return type:

bool

property fields: list[HTMLField | TextField]

Get all fields of this helper instance

Returns:

All fields which need to be translated

get_text_fields() → list[HTMLSegment]

Get all text fields of this helper instance (all native text_fields combined with all segments of all html_fields)

Returns:

All text fields and segments which need to be translated

Return type:

list[HTMLSegment]

valid: bool = True

Wether or not the translation was successful

async integreat_cms.summ_ai_api.utils.worker(loop: asyncio.AbstractEventLoop, task_generator: PatientTaskQueue[partial], identifier: str) → list[Any]

Continuously gets a task from the queue and executes it. Stops once no more tasks are available. This form makes it easy to always have at most n concurrent tasks as well as intermittent wait times through the task generator.

Catches SummAiRateLimitingExceeded and SummAiInvalidJSONError and counts them as rate limit hits in order enqueue them again.

Parameters:

• loop (asyncio.AbstractEventLoop) – The asyncio event loop to execute tasks in

• task_generator (PatientTaskQueue[partial]) – Queue to execute tasks from

• identifier (str) – Identifyer of the worker (for logging purposes)

Returns:

A list of task-results

Return type:

list[Any]