Umbraco Commerce Technical Team Lead @ Umbraco HQ | Co-founder of CODECABIN | Creator of UMB.FYI
Matt Brailsford announces the integration of speech-to-text functionality in Umbraco.AI, enhancing user interaction with new microphone features in Copilot and Tiptap. This capability allows users to dictate content seamlessly, supported by a management API and a dedicated service for developers. Future enhancements may include real-time transcription and text-to-speech features.
Matt Brailsford discusses the integration of Microsoft.Extensions.AI (M.E.AI) and the Microsoft Agent Framework (MAF) in developing AI capabilities for Umbraco. The use of IChatClient and middleware patterns enabled efficient feature development and agent orchestration, allowing for scalable, transferable skills while maintaining a cohesive architecture.
Matt Brailsford discusses the implementation of guardrails in Umbraco.AI, which are safety policies that evaluate content before and after AI model interactions. Guardrails consist of evaluators, phases, and actions to manage inappropriate content. They are customizable and essential for maintaining content integrity in AI-driven systems.
Matt Brailsford introduces Umbraco.AI.Search, a semantic vector search add-on for Umbraco that enhances search capabilities by understanding user intent rather than relying solely on keyword matching. This tool addresses limitations of traditional searches, supports multilingual queries, and integrates with Umbraco.Cms.Search, offering improved content discovery and related content features.
Matt Brailsford discusses the new file upload feature in Umbraco.AI, allowing users to drag and drop documents into Copilot. The system supports various file types, enabling text extraction from Office documents and direct processing of images and PDFs. The extensible architecture allows for additional file formats to be integrated seamlessly.
Matt Brailsford discusses the Umbraco.AI.Prompt package, detailing how content editors can utilize prompt templates for AI-generated tasks like SEO titles and alt text. He explains the syntax, available variables, and the importance of using the image: prefix for image prompts, alongside practical examples and tips for maximizing AI capabilities.
Matt Brailsford discusses the concept of contexts in Umbraco AI, which serve as reusable containers of knowledge that enhance AI output by providing essential background information. Contexts can include various resource types, such as Brand Voice and Text, and can be assigned at multiple levels, improving the relevance and quality of generated content.
Matt Brailsford introduces a streamlined installation script for Umbraco AI, simplifying the setup process by automatically installing necessary packages and seed data. This allows users to quickly create a functional demo site, explore features, and customize AI integrations without extensive configuration, enhancing accessibility for new users.