DallE

DallE is the image generation service developed by OpenAI. AIITK gives you access to this API with the flexibility to request image generation services at runtime and convert it to an Unreal Engine usable texture for a wide variety of customization options, automation, or integrating responsive gameplay elements.

• DallEParams
  • Request Type: Type of request (Image/Variation/Edit).
  • APIKey: Key for OpenAI DALL-E API authentication. Set in Project settings.
  • Prompt: Description or text prompt for image generation.
  • NumberOfImages: Number of images to generate. Default is 1.
  • ImageSize: Size of the generated image. Dall-e-2 supports 256x256, 512x512, and 1024x1024. Dall-e-3 supports 1024x1024, 1792x1024, and 1024x1792
  • Encoding: Format of the response (e.g., URL, Base64). Dall-e-3 only supports URL responses due to webp output. Dall-e-2 supports both methods.
  • Endpoint: DALL-E API URL. Default: https://api.openai.com/v1/images/generations
  • Model: Name of the DALL-E model for image generation. "dall-e-2" or “dall-e-3”.
  • Quality: Quality setting for image generation. Only ‘standard’ and ‘hd’ are supported at the moment.
  • Timeout: Time in seconds before the request times out. Default is 60 seconds and maxes out at 120 to avoid API errors.
  • ImageToSend: Optional parameter for sending an image for variation or edit requests. Input the file path for the image to send
  • MaskToSend: Optional parameter for sending a mask image for edit requests. Example masks can be found in: Plugins\AIIntegrationToolkit\Resources

Image Generation

SendDallEImageRequest/SendRequestToDallE (AIITKComponentBP)

The SendDallEImageRequest function is used to send a text prompt to the Dall-E API, which generates an image based on the provided description. This function is particularly useful when you need the AI model to create unique and contextually relevant images for your game.

Handling Responses From DallE

ImageURLs will either contain raw Base64 encoded image data or a URL for each image generated based on your selected encoding format. Set a texture parameter value for a dynamic material instance, save the image to the disk, etc, DownloadImageFromURL from the AIITK library outputs the downloaded image as a Texture2D object (You can use the native DownloadImage async node as well if you want, DownloadImageFromURL just gives an event to bind to).

URL: You can use the function Download Image From URL and hook up the “On image downloaded” delegate to an event. This is returned as a Texture2D object.

Base64: You can use the functions ExtractBase64FromDataURL to get the image data from the response JSON and then convert that data with Base64ToTexture which effectively converts it into a texture Unreal Engine can use.

*Use SaveTextureToDisk and supply a valid file path to save any texture to any desired location (as a png), not limited to DallE responses. Output bool set to true on successful save.