Note: This documentation is in the process of being updated to AIITK v2.5, examples can be found in the content folder of the plugin. For direct support, visit Discord: AIITK Support

INTRODUCTION

The AI Integration Toolkit plugin for Unreal Engine is a comprehensive suite of tools designed to seamlessly integrate OpenAI's GPT, Dall-E, Whisper, and TTS APIs into your Unreal Engine projects as well as Eleven Labs text to speech for a variety of realistic voices. This plugin offers an extensive range of functions and events that make it easy to incorporate chat and function-based AI, image generation, speech generation, and audio transcription services into your games. My goal is to make it as easy as possible to integrate generative AI into your projects and with AIITK you can get started with as little as 2 nodes. You can find it on the UE Marketplace here: AIITK For UE

Philosophy

"I believe that integrating these new tools directly into the frontend of the experiences we create expands the possibility spaces of game design tenfold. Generative AI utilization should not be focused on replacing what a human could do, but about engaging with video games in brand new ways."

Main C++ Classes

GPTChatAPI

This class enables communication with OpenAI's GPT API, allowing you to send chat prompts and receive generated responses. Include previous messages for more context-aware responses.

DallEAPI

This class provides the ability to generate images using OpenAI's Dall-E API. You can generate images based on textual prompts, specifying the number of images, size, and encoding format.

WhisperAPI

This class allows you to leverage OpenAI's Whisper API for audio transcription. You can transcribe audio files or capture, save, and transcribe audio in real-time.

TextToSpeechAPI

This class allows you to generate natural sounding speech for your characters at runtime by utilizing the ElevenLabs API and the OpenAI TTS endpoint (Experimental).

OpenAIFunctionLibrary

This utility class offers a range of helper functions, including converting base64-encoded images to textures, saving textures to disk, managing/parsing requests, etc.

AIITKComponent

The AIITKComponent is designed to be able to be attached to any actor to enable AI generative capabilities, allowing developers to easily create replicable and instance-able generation patterns for their games.

Adding API keys

Navigate to Project settings->Plugins->AIITKDeveloperSettings.
Keys are automatically encrypted and stored in: YourProject\Config\DefaultGame.ini.

Alternatively, use SetAPIKey at runtime

Where do I get my API keys?

For OpenAI: https://platform.openai.com/api-keys

For ElevenLabs: https://elevenlabs.io/