ares package

Subpackages

Submodules

ares.cli module

CLI tool

ares.cli.evaluate(config: Path, limit: Annotated[bool, typer.Option] = False, first_n: Annotated[int, typer.Option] = -1, dashboard: Annotated[bool, typer.Option] = False, verbose: Annotated[bool, typer.Option] = False) → None

Execute a red-teaming evaluation.

Parameters:

config – path to main config YAML file
dashboard – bool indicating whether or not to deploy the report via streamlit
limit – bool indicating whether or not to run ARES in a test mode (limited seeds number)
first_n – int indicating the number of the first N seeds to use
verbose – bool indicating whether or not to show configs information during execution

Example:

>>> ares evaluate example_configs/minimal.yaml
>>> ares evaluate example_configs/minimal.yaml --dashboard
>>> ares evaluate example_configs/minimal.yaml --limit
>>> ares evaluate example_configs/minimal.yaml --limit --first 3
>>> ares evaluate example_configs/minimal.yaml -v

ares.cli.install_plugin(plugin_name: str) → None

Installs a plugin from ares-internal-plugins

Parameters:: plugin_name – Name of the plugin to install

ares.cli.main() → None: ARES CLI Application

ares.cli.show_report(config: Path, dashboard: Annotated[bool, typer.Option] = False) → None

Takes a list of configs and generates a report.

Parameters:

config – path to main config YAML file
dashboard – bool indicating whether or not to deploy the report via streamlit

ares.redteam module

ARES RedTeaming class

class ares.redteam.RedTeamConfig(user_config: dict, goal: dict, evaluation: dict, strategy: dict)[source]

Bases: object

Static configuration for the red-teaming exercise

evaluation: dict

goal: dict

strategy: dict

user_config: dict

class ares.redteam.RedTeamer(user_config: dict[str, Any], connectors: dict[str, Any], verbose: bool = False)[source]

Bases: object

The main red-teaming class, supporting: target goal strategy evaluate

add_evaluation_input_path(new_path: str) → None[source]

Adds input path with attacks results to evaluation config

Parameters:: new_path – the path to add to the evaluation config

base_tmp_dir = 'results/tmp'

clean_temp() → None[source]: Temporary files manager that cleans all files added to tmp_files_manager list

config: RedTeamConfig

dashboard(config_file: Path) → None[source]

Run the streamlit app for visual reporting

Parameters:: config_file – Path path to user configuration, must be specified in dashboard is True

evaluate() → None[source]: Setup evaluations

goal(limit: bool | None = False, first_n: int | None = None) → None[source]: Setup goals

init_strategy(strategy_config: dict[str, Any], target_connector: Connector, strategy_class: Any) → Any[source]

Initialize the strategy class depending on black vs white-box strategy

Parameters:

strategy_config – the strategy configuration
target_connector – the target connector
strategy_class – the strategy class

Returns:

the initialized strategy class

input_guardrail_name: str | None = None

load_external_connectors(user_config: dict) → dict[source]

Checks for connectors’ templates in installed plugins

Parameters:: user_config – user config with ARES core connectos templates
Returns:: updated user_config with connectors from plugins

metrics: dict[str, dict[str, Any]] = {}

output_guardrail_name: str | None = None

redteam(limit: bool = False, first_n: int = -1) → None[source]: Run the complete redteaming exercise, shortcut for the more granular approach Exceptions could be thrown:

report() → None[source]

Method to run the Streamlit app for visual reporting.

This function deploys a report via Streamlit when dashboard is set to True. It requires the path to the user configuration (config_file) to be specified when using the dashboard.

Parameters:

dashboard (bool) – Boolean indicating whether or not to deploy the report via Streamlit.
config_file (str) – Path to the user configuration, required if dashboard is True.

strategy() → Path[source]

Setup strategies

Returns:: Path to the strategy file

target() → None[source]: Setup the target connector

target_connector: Connector

tmp_files_manager: list[Path] = []

ares.utils module

Global utilities

class ares.utils.Plugin[source]

Bases: object

Imports and loads classes from a package path

static find_spec(the_spec: str) → bool[source]

Wraps importlib.find_spec function.

This function searches for a specified spec and returns True if found, False otherwise.

Parameters:: the_spec (str) – The spec to search for.
Returns:: bool - True if the spec is found, False otherwise.

static load(class_name: str, required: list | None = None, reload: bool = False) → type[source]

Import a module or class.

This function imports a specified class from a given package path. It also allows reloading the module if required.

Parameters:

class_name – str - A full package path to the class.
reload – bool - Whether to reload the module (default: True).

Returns:

type - The class for instantiation.

static module_checker(class_name: str) → None[source]

Checks if required config plugins are present before running the ARES pipeline.

This function verifies the presence of a specified plugin before proceeding with the ARES pipeline execution.

Parameters:: class_name – The name of the plugin to check for.

ares.utils.dict_contains(big: dict[str, Any], small: dict[str, Any]) → None[source]

Checks if all key-value pairs in a ‘small’ dictionary exist in a ‘big’ dictionary, recursively into sub-dictionaries.

Parameters:

big (dict) – The dictionary to validate against.
small (dict) – The dictionary driving the validation.

Example:

>>> dict_contains({'goal': {'config': {'base_path': os.path.isfile}}},  {'goal': {'config': {'base_path': "goals.csv"}}})

ares.utils.extract_intent_config(user_config: dict[str, Any] | list[dict[str, Any]], intents: dict[str, Any]) → dict[str, Any][source]

Extracts default/user-defined intent config from user config

Parameters:

user_config – user provided config from YAML
intents – ARES default intents

Returns:

requested intent config

ares.utils.extract_strategy_from_config(strategy_config: dict[str, Any] | list[dict[str, Any]] | list[str], all_strategies: dict[str, Any]) → dict[str, Any][source]

Extracts default and user-provided strategies from the configuration.

This function takes a user-provided configuration (strategy_config) from a YAML file and the set of ARES default strategies (all_strategies). It returns a combined configuration of requested strategies.

Parameters:

strategy_config (dict) – User-provided configuration from YAML.
all_strategies (list) – The set of ARES default strategies.

Returns:

Combined configuration of requested strategies.

Return type:

dict

ares.utils.find_connector(the_config: dict[str, Any], user_connector: dict[str, Any]) → dict[str, Any][source]

Find the target connection by name in the connections list.

Parameters:

the_config – full configuration
user_connector – user specified connector

Returns:

a dictionary of the target connections configuration

ares.utils.find_external(module_name: str = 'strategies', base_class_name: str = 'AttackStrategy', suffix: str = 'strategy', plugin_prefix: str = 'ares-') → dict[str, dict[str, Any]][source]

This function finds all external ARES modules (connectors, strategies, goals, evals) in the plugins.

Parameters:

module_name – name of the module to search for
base_class_name – name of the base class to search for
suffix – suffix of the module to search for
plugin_prefix – prefix of the plugin name

Returns:

a dictionary of external modules

ares.utils.generate_filename_with_timestamp(file_tag: str, extension: str = 'json') → Path[source]

The function generates a filename by appending a timestamp to the provided file_tag. The timestamp is formatted as YYYYMMDD_HHMMSS

Parameters:

file_tag – The base name of the file
extension – extension to a filename, defaults to json

Returns:

A Path object representing the generated filename

ares.utils.get_external_templates(module: Any, templates: dict[str, dict], base_class_name: str = 'Connector', suffix: str = 'connector') → dict[str, dict[str, Any]][source]

Find a connector class from a given module that inherits from the specified base class.

This function searches for a class in the provided module that inherits from the specified base class name (defaults to “Connector”). If found, it returns the class object.

Parameters:

module – The module to search for the base_class_name class.
base_class_name – The name of the base class to check for inheritance (default is “Connector”).
suffix – The suffix to remove from the class name (default is “connector”).

Returns:

a dictionary of the module configuration

ares.utils.log_dict(dictionary: dict) → str[source]

This function converts a dictionary to a string

Parameters:: dictionary – dictionary to be converted to string
Returns:: string representation of the dictionary

ares.utils.log_table(rich_table: rich.table.Table) → rich.text.Text[source]

Generate an ascii formatted presentation of a Rich table without styling

Parameters:: rich_table – Table
Returns:: a rich text table

ares.utils.nested_update(d1: dict[str, Any], d2: dict[str, Any]) → dict[str, Any][source]

Nested/recursive update of d1, with d2. d2 entries overwrite d1 in place.

Parameters:

d1 – dict to merge into
d2 – dict to merge from

Returns:

d1, updated with d2

ares.utils.parse_config(file_path: Path, reqs: dict[str, Any] | None = None) → dict[str, Any][source]

Parses the main configuration YAML file.

This function reads and parses the main configuration YAML file, ensuring that the specified required dictionary entries are present.

Parameters:

file_path (str) – The path to the main configuration YAML file.
reqs (dict) – A dictionary specifying required entries in the YAML file.

Returns:

A dictionary containing the parsed configuration.

Example:

>>> parse_config(Path("config.yaml"), reqs={'goal': {'config': {'base_path': os.path.isfile}}, 'strategy': None, 'evaluation': None})

ares.utils.parse_connectors(file_path: Path) → dict[str, Any][source]: Parses the main connectors YAML file :param file_path: path to the main configuration YAML file, search from this path looking for a connectors.yaml :return: a dictionary of the connectors

ares.utils.print_items(items: dict | list | None, console: rich.console.Console, color_code: str = '#0f62fe') → None[source]

This function prints list of available ARES components

Parameters:

items – dictionary or list of available items
console – Console object to print the output
color_code – Color code to use for printing the items (default is “#0f62fe”)

ares.utils.save_file(output_path: Path, contents: Any) → None[source]

Save a contents to file

Parameters:

output_path – str of the output path to save the file
contents – the content to save

Returns:

None

ares.show module

ARES show CLI

class ares.show.ColorCodes(core: str = '#0f62fe', external: str = '#198038', not_found: str = '#8a3ffc')[source]

Bases: object

Color codes for rich text output

core: str = '#0f62fe'

external: str = '#198038'

not_found: str = '#8a3ffc'

ares.show.connectors(name: Annotated[str, typer.Option] = None) → None

This command displays information about available connectors and optionally shows example configuration of a specific connector.

Parameters:: name – The name of the connector to configure. Defaults to None.

ares.show.evals(name: Annotated[str, typer.Option] = None) → None

This command displays information about available evaluators and optionally shows example configuration of an evaluator.

Parameters:: name – The name of the evaluator to configure. Defaults to None.

ares.show.goals(name: Annotated[str, typer.Option] = None) → None

This command displays information about available goals and optionally shows example configuration of an attack goal module.

Parameters:: name – The name of the attack goal module to configure. Defaults to None.

ares.show.modules() → None: This command displays information about available ARES modules.

ares.show.print_config(name: str, config: dict, console: rich.console.Console, color_code: str = '#0f62fe') → None[source]

This function prints the configuration of a specific module.

Parameters:

name – the name of the module to display the configuration for
config – the configuration of the module
console – the console object to use for printing
color_code – the color code to use for the module name

ares.show.show_module(module: str, name: str, core: dict, external: dict | None = None) → None[source]

This function displays information about a specific ARES module.

Parameters:

module – The name of the ARES module to display information about.
name – The name of the module implementation to display information about.
external – A dictionary containing external module templates.

ares.show.strategies(name: Annotated[str, typer.Option] = None) → None

This command displays information about available attack strategies and optionally shows example configuration of a strategy.

Parameters:: name – The name of the strategy to configure. Defaults to None.

Module contents

ARES: AI Robustness Evaluation System