Core

Introduction

Core components. Most notably ALM which is the Class from which all models derive. All functions that are common amongst models are here.

ALM DIY

Override build_prompt and at least one of create_native_generator create_native_completion. Implementing create_native_generator but not create_native_completion will not lead to automatic availability of the latter.

Ideally you also override detokenize tokenize tokenize_as_str get_n_tokens

The easiest example of all this, is the OpenAI class.

Documentation

class pyalm.internal.alm.ALM(model_path_or_name, verbose=0, enable_functions=False)

Base class. Don’t instantiate on its own

add_tracker_entry(*args, **kwargs)

Add a new entry to the tracker. More info is in https://github.com/finnschwall/PyALM/blob/main/format_specifications.md

adopt_from_alm(other: Type[ALM])

Copy state from other ALM into this. This is not a deep copy!

Parameters:

other – Other ALM

autoconverse(interactions=3)
available_functions

Function names and the callable functions available for the model

abstract build_prompt(preserve_flow=False)

Build prompt in format native to library

Parameters:

preserve_flow – Block suffix for purely text based models

Returns:

prompt obj

build_prompt_as_str(new_lines_per_role=1, new_lines_afer_role=0, block_gen_prefix=False, raw=False, include_system_msg=True, use_build_prompt=False)

Build a prompt in string form :param new_lines_per_role: Newlines after Role+message :param new_lines_afer_role: Newlines after role :param block_gen_prefix: Whether to add generation prefix. Normally this leads a prompt to end with e.g. Assistant: so that the model does not continue to write as a user. :param raw: If true don’t resolve symbols :return: str

code_callback

Function that will be called to execute code

create_completion(text_obj=None, verbose=None, enable_function_calls=None, chat=True, token_prob_delta=None, token_prob_abs=None, handle_functions=None, stop=None, **kwargs)

Create completion with automatic prompt building, function calling etc.

Parameters:

  • text – Prompt or prompt obj

  • max_tokens – maximum tokens generated in completion

  • stop – Additional stop sequences

  • chat – Only applicable for pure text models. Will pass just text to model without building a history.

  • keep_dict – If library or API returns something else than raw tokens, whether to return native format

  • token_prob_delta – dict, relative added number for token logits

  • token_prob_abs – dict, Absolute logits for tokens

  • log_probs – int, when not None return the top X log probs and their tokens

  • handle_functions – Whether to call functions or return list with attemptedcalls

  • kwargs – kwargs

Returns:

completion

create_completion_plugin(conv_tracker=None, context=None, func_list=None, system_msg=None, code_calls=0, chat_store_loc=None, username=None, temp=None, metadata=None)
create_generator(text_obj=None, verbose=None, enable_function_calls=None, chat=True, token_prob_delta=None, token_prob_abs=None, handle_functions=True, stop=None, **kwargs)

Streaming version of create_generator. Returns generator with automatic prompt building, function calling etc.

Parameters:

  • text – Prompt or prompt obj

  • max_tokens – maximum tokens generated in completion

  • stop – Additional stop sequences

  • chat – Only applicable for pure text models. Will pass just text to model without building a history.

  • keep_dict – If library or API returns something else than raw tokens, whether to return native format

  • token_prob_delta – dict, relative added number for token logits

  • token_prob_abs – dict, Absolute logits for tokens

  • log_probs – int, when not None return the top X log probs and their tokens

  • handle_functions – Whether to call functions or return list with attemptedcalls

  • kwargs – kwargs

Returns:

completion

abstract create_native_completion(text, max_tokens=256, stop=None, keep_dict=False, token_prob_delta=None, token_prob_abs=None, log_probs=None, **kwargs)

Library native completion retriever. Different for each library. No processing of output is done

Parameters:

  • text – Prompt or prompt obj

  • max_tokens – maximum tokens generated in completion

  • stop – Additional stop sequences

  • keep_dict – If library or API returns something else than raw tokens, whether to return native format

  • token_prob_delta – dict, relative added number for token logits

  • token_prob_abs – dict, Absolute logits for tokens

  • log_probs – int, when not None return the top X log probs and their tokens

  • kwargs – kwargs

Returns:

completion

abstract create_native_generator(text, keep_dict=False, token_prob_delta=None, token_prob_abs=None, **kwargs)

Library native generator for tokens. Different for each library. No processing of output is done

Parameters:

  • text – Prompt or prompt obj

  • keep_dict – If library or API returns something else than raw tokens, whether to return native format

  • token_prob_delta – dict, Absolute logits for tokens

  • token_prob_abs – dict, relative added number for token logits

  • kwargs – kwargs

Returns:

generator

enable_automatic_function_calls()
abstract get_n_tokens(text)

How many tokens are in a string

Parameters:

text – tokenizable text

Returns:

amount

init_gui(num_messages_included=4, monkeypatch=True)
load_history(path_or_text, is_file=True)

Loads a conversation history

Parameters:

path_or_text – Either path to a file or a yaml like string

pop_entry()

Remove last element from conversation tracker. Automatically takes care of e.g. split messages due to function calls. :return: The popped entry

property preserved_sequences
replace_symbols(text, entry=None, additional_symbols=None)

Replace symbols in a conv history entry or text

Parameters:

  • text – text with symbols in it

  • entry – optional, conv history entry to use for replacement

  • temp_symbols – additional symbols to be replaced

Returns:

text with substitutions

reset_tracker(purge=False)

Remove all tracker entries

Parameters:

purge – Complete reset including e.g. system message

save_history(path=None)

Saves the ALMs conversation history

Parameters:

path – Where to save. If not specified string is returned

Returns:

None or history as yaml like string

save_state(path=None)

Saves the ALMs entire state (excluding the model itself)

Parameters:

path – Where to save. If not specified string is returned

Returns:

None or state as yaml

set_system_message(msg='', prepend_function_support=False)

Change the system message

Parameters:

  • msg – new message

  • prepend_function_support – Automatically change message to include function calling

property symbols
property system_msg
abstract tokenize(text)

Text to token as vector representation

Parameters:

text –

Returns:

List of tokens as ints

abstract tokenize_as_str(text)

Text to token as vector representation but each token is converted to string

Parameters:

text –

Returns:

List of tokens as strings

update_gui()
user_symbols

Variable symbols that will get replaced when building prompt. Either string or function pointer

property verbose
class pyalm.internal.alm.FunctionFormat(value)

An enumeration.

JSON = 'JSON'
MODEL_SPECIFIC = 'MODEL_SPECIFIC'
PYDOC = 'PYDOC'
class pyalm.internal.alm.ParseStatus(value)

Used to classify outcomes when trying to handle LLMs function call

NO_FUNC_SEQUENCE_FOUND = 'NO_FUNC_SEQUENCE_FOUND'
PARSED_DICT_RETURN = 'PARSED_DICT_RETURN'
PARSED_EXECUTED_ERR = 'PARSED_EXECUTED_ERR'
PARSED_EXECUTED_OK = 'PARSED_EXECUTED_OK'
UNDEFINED = 'UNDEFINED'
UNPARSEABLE_FUNC_FOUND = 'UNPARSEABLE_FUNC_FOUND'
class pyalm.internal.alm.Symbols
pyalm.internal.alm.change_latex_delimiters(latex_str)