Automatic Alt Text for images using Optimizely CMS 12
Even if ChatGPT and other LLMs (Large Language Models) have gotten a lot of attention lately, neither the use of AI nor integrating AI in tools like Optimizely CMS is new. I released an addon in January 2020, that uses AI and machine learning, to provide various metadata for images uploaded in Optimizely CMS. Since then the addon has been improved and updated to support Optimizely CMS 12 and .NET 6.
This addon is versatile, with several potential use cases. However, for the sake of simplicity, this blog post will focus on one specific use: how to use the addon to add alt text to images on an Optimizely CMS 12 Alloy demo site.
Create a fresh Alloy demo site
If you want a new Alloy demo site, update to the latest version of the Optimizely templates, create a new project, and run it.
dotnet new install EPiServer.Templates
dotnet new epi-alloy-mvc
dotnet runMake sure the site runs, and create an admin account when the site loads for the first time.
Install NuGet package
Install the NuGet package Gulla.Episerver.AutomaticImageDescription that contains code for generating metadata. You may use the GUI in a tool like Visual Studio, or install from the command line.
Install-Package Gulla.Episerver.AutomaticImageDescriptionGenerate alt text
In order to save the alt text for an image, let's update the image file model Models.Media.ImageFile, so the result looks like this.
public class ImageFile : ImageData, IAnalyzableImage
{
[AnalyzeImageForDescription]
public virtual string Description { get; set; }
public virtual string Copyright { get; set; }
public virtual bool ImageAnalysisCompleted { get; set; }
}The property Description will hold the image description and the attribute AnalyzeImageForDescription tells the addon to automatically populate it.
The interface IAnalyzableImage and the property ImageAnalysisCompleted will help us add descriptions to existing images later. The bool property will be set to true for all images that have had their Description generated.
The addon is capable of generating different types of metadata, not just descriptions. If interested in the other options, please refer to the documentation.
In startup.cs add services.AddAutomaticImageDescription() inside the method ConfigureServices so the result looks like this.
services
.AddCmsAspNetIdentity<ApplicationUser>()
.AddCms()
.AddAlloy()
.AddAdminUserRegistration()
.AddEmbeddedLocalization<Startup>()
.AddAutomaticImageDescription();Create a Computer Vision resource in Azure, and add it to your configuration files as described in the documentation. It could look like this.
"Gulla": {
"AutomaticImageDescription": {
"ComputerVisionEndpoint": "https://mycomputervision.cognitiveservices.azure.com/",
"ComputerVisionSubscriptionKey": "0123456789abcdef",
}
}Now, the Description property will be populated every time a new image is uploaded to Optimizely CMS. But for now, it will only be visible inside edit mode.
Let's bring it to our site's visitors too!
Add alt text to images in content areas
Update the view component Components.ImageFileViewConponent.cs and add the Description property. The result could look like this.
protected override IViewComponentResult InvokeComponent(ImageFile currentContent)
{
var model = new ImageViewModel
{
Url = _urlResolver.GetUrl(currentContent.ContentLink),
Name = currentContent.Name,
Copyright = currentContent.Copyright,
Description = currentContent.Description
};
return View(model);
}Add a new Description-property to the Models.ViewModels.ImageViewModel. The result could look like this.
public class ImageViewModel
{
public string Url { get; set; }
public string Name { get; set; }
public string Copyright { get; set; }
public string Description { get; set; }
}Update the view component for ImageFiles, found in Views.Shared.Components.ImageFile.Default.cshtml and add the alt-attribute. The result might look like this.
@model ImageViewModel
<img src="@Model.Url" alt="@Model.Description" class="image-file" />
This view component will render the alt text for images placed inside content areas.
Add alt text to images in rich text
In order to render the alt text for images placed inside rich text (XhtmlString properties that are rendered with the TinyMCE editor in edit mode), we have to add a custom rendering for XhtmlString.
One option is creating an extension method like this to rewrite the alt texts for XhtmlString inputs.
public static class XhtmlStringExtensions
{
public static XhtmlString RewriteAltTexts(this XhtmlString xhtmlString)
{
var contextModeResolver = ServiceLocator.Current.GetInstance<IContextModeResolver>();
if (contextModeResolver.CurrentMode == ContextMode.Edit)
{
return xhtmlString;
}
var processedText = xhtmlString.ToInternalString();
var dictionary = GetImageDictionary(xhtmlString);
foreach (var (internalUrl, image) in dictionary.Select(x => (x.Key, x.Value)))
{
// Handle both old style with file name as alt text, and new style empty alt text
var regex = new Regex($"<img src=\"{Regex.Escape(internalUrl)}\" alt=\"({Regex.Escape(image.Name)})?\"", RegexOptions.IgnoreCase);
processedText = regex.Replace(processedText, $"<img src=\"{internalUrl}\" alt=\"{image.Description}\"");
}
return new XhtmlString(processedText);
}
private static Dictionary<string, ImageFile> GetImageDictionary(XhtmlString xhtmlString)
{
var contentLoader = ServiceLocator.Current.GetInstance<IContentLoader>();
var imageDictionary = new Dictionary<string, ImageFile>();
foreach (var urlFragment in xhtmlString.Fragments.Where(x => x is UrlFragment))
{
foreach (var guid in urlFragment.ReferencedPermanentLinkIds)
{
if (!contentLoader.TryGet(guid, out ImageFile image))
{
continue;
}
var url = $"~/link/{image.ContentGuid:N}.aspx";
if (!imageDictionary.ContainsKey(url))
{
imageDictionary.Add(url, image);
}
}
}
return imageDictionary;
}
}Create a new display template Views.Shared.DisplayTemplates.XhtmlString.cshtml to hijack the rendering of XhtmlString properties.
@using EPiServer.Core
@model XhtmlString
@{ if (Model == null) { return; }; }
@{Html.RenderXhtmlString(Model.RewriteAltTexts());}Now, we have alt texts for images in content areas and rich text, but we are still missing some images.
Add alt text to images referenced by ContentReference properties
Create a new view file in Views.Shared.DisplayTemplates.Image.cshtml with the following content. This will be used for the images references by ContentReference properties when they are rendered with Html.PropertyFor() or Html.DisplayFor().
@using EPiServer
@using EPiServer.ServiceLocation
@model EPiServer.Core.ContentReference
@if (Model != null)
{
var contentLoader = ServiceLocator.Current.GetInstance<IContentLoader>();
var image = contentLoader.Get<ImageFile>(Model);
<img src="@Url.ContentUrl(Model)" alt="@image.Description" />
}Add alt text to the rest of the images
There are still some images left that are rendered without alt text. That's those located inside blocks where the HTML is custom-built. One example is the TeaserBlock. In order to separate the logic from the view, we should consider converting this to a view component.
Create a new file Models.ViewModels.TeaserBlockModel.cs with content like this.
public class TeaserBlockModel
{
[UIHint(UIHint.Image)]
public ContentReference Image { get; set; }
public string ImageDescription { get; set; }
public string Heading { get; set; }
public string Text { get; set; }
public PageReference Link { get; set; }
}
Then create a new file with the logic, Components.TeaserBlockComponent.cs like this.
public class TeaserBlockComponent : BlockComponent<TeaserBlock>
{
private readonly IContentLoader _contentLoader;
public TeaserBlockComponent(IContentLoader contentLoader)
{
_contentLoader = contentLoader;
}
protected override IViewComponentResult InvokeComponent(TeaserBlock currentContent)
{
var model = new TeaserBlockModel()
{
Heading = currentContent.Heading,
Text = currentContent.Text,
Link = currentContent.Link,
Image = currentContent.Image,
ImageDescription = _contentLoader.Get<ImageFile>(currentContent.Image)?.Description
};
return View(model);
}
}And finally, move the two view files from /Views/Blocks/ to Views.Shared.Components.TeaserBlock.Default.cshtml and Views.Shared.Components.TeaserBlockWide.Default.cshtml. Example from TeaserBlock.Default.cshtml.
@using EPiServer.Core
@model TeaserBlockModel
<div>
@*Link the teaser block only if a link has been set and not displayed in preview*@
@using (Html.BeginConditionalLink(
!ContentReference.IsNullOrEmpty(Model.Link) && !(Html.ViewContext.IsPreviewMode()),
Url.PageLinkUrl(Model.Link),
Model.Heading))
{
<div class="img-wrapper mb-3" epi-property="@Model.Image">
<img src="@Url.ContentUrl(Model.Image)" alt="@Model.ImageDescription" />
</div>
<h2 epi-property="@Model.Heading">@Model.Heading</h2>
<p epi-property="@Model.Text">@Model.Text</p>
}
</div>
All new images will now get image descriptions added on upload, and the description will be shown as alt text when the image is rendered.
Update existing images
If you're not starting a brand new project, you will likely have some existing images. If you want to generate descriptions for them, install this NuGet package.
Install-Package Gulla.Episerver.AutomaticImageDescription.ScheduledJobThis will add a new scheduled job to Optimizely admin mode called «Analyze all images, update metadata». If you run the job, it will update existing images with metadata.
When using this job, you should always implement IAnalyzableImage on your image model, so that the job can keep track of what images have metadata generated and not. The job can be configured to add metadata over time, to stay inside the free tier of the Azure service.
Advanced features
Want a different language than English, or more than one language? Want to support other types of metadata, not just image descriptions? Other property types? Have a look at the documentation!
A final warning
I recommend treating AI-generated content as a draft or starting point. It's wise to review the output before publishing. However, in most cases, a suboptimal alt text is better than no alt text.
If you have any questions, feel free to reach out!