Smart Tags and AI-Based Tagging in AEM Assets

Tagging thousands of assets by hand is slow, inconsistent and rarely finished. Smart Tags use Adobe Sensei, Adobe's AI framework, to analyze assets and apply descriptive tags automatically, so your library becomes searchable without a manual tagging effort. This post covers how it works, how to configure and curate it, and how to read the results from code.

What Smart Tags Do

When an asset is uploaded, AEM can analyze its content and attach tags based on what it sees. An image of a woman running along a beach might get tags like running, beach, outdoor and sport, each with a confidence score.

Smart Tags improve:

• Search: assets match more of the words people actually type
• Consistency: the same visual content gets the same vocabulary
• Speed: new uploads are tagged without waiting for someone to do it

Two Flavors: Smart Tags and Enhanced Smart Tags

AEM offers two related capabilities:

Generic tags might call an asset "shoe". Enhanced tags, once trained, can call it "trail-runner-v2" if that's how your catalog names it.

What Gets Tagged

• Images: analyzed by Adobe Sensei's image recognition. In AEM as a Cloud Service, Smart Tags for images are provisioned automatically
• Videos and text-based assets: supported as well, with tagging enabled by default in Cloud Service
• Other asset types: support depends on MIME type, so check the current Adobe documentation for the supported list before promising it to a client

Enabling and Disabling Smart Tag Enhancements

On Cloud Service, the enhanced AI-generated metadata is controlled from the Assets configuration console:

1. Go to Tools → Assets → Assets Configurations → Smart Tag Enhancement Configuration
2. Review the available options, including AI-generated titles and keywords
3. To turn the feature off for an environment, choose the option to disable Smart Tag Enhancements

You can also disable it per folder through the folder's Asset Processing settings. That's useful for folders where AI tagging adds no value, such as internal documents or sensitive content.

A common practice is to disable enhancements in Stage and Production while you are experimenting, so training and tagging behavior stay isolated, and to enable them where you've decided to use them. When the feature is disabled, newly uploaded assets stop getting Smart Tags or AI-generated metadata, but existing assets keep theirs.

Training Smart Tags on Your Taxonomy

For tags to match your business vocabulary, you train the service on assets you have already tagged correctly.

Prepare a training set:

1. Build a tag structure (taxonomy) that reflects how your business describes its content
2. Tag a representative set of assets manually. Quality matters more than volume, because the AI learns from whatever you give it
3. Adobe recommends training on at least two distinct tags, with at least ten images per tag, and about thirty examples per tag for good results

Run training:

1. Apply your tags to the training assets
2. Run the Smart Tags Training workflow
3. Check the Smart Tags Training report in the Reports console. A green status next to a tag means the service is trained for it

Review before you train. Adobe recommends curating the tags on your training assets before training, because mistakes in the training set become mistakes in every future prediction.

Curating and Moderating Smart Tags

AI tags aren't perfect, so plan to review them:

• Open an asset's Properties and look at the Smart Tags in the metadata view
• Remove tags that are wrong or misleading
• Add tags the AI missed. These improve search results and, for enhanced tags, can feed back into training
• Review tags on high-value or high-visibility assets first

Moderation also improves asset search, since your search results then reflect curated tags rather than raw predictions.

Where Smart Tags Are Stored

Smart Tags are stored as metadata on the asset, separately from the tags authors apply by hand. They're typically found under the asset's metadata node as child nodes of predictedTags, each with a tag name and a confidence value.

To see this on your own instance:

1. Open CRXDE Lite (or the repository browser available in your environment)
2. Navigate to /content/dam/<your-folder>/<asset>/jcr:content/metadata
3. Look for the predictedTags node and inspect its children

Node and property names can vary between releases, so verify the structure on your environment before writing code that depends on it.

Reading Smart Tags in a Sling Model

Once you've confirmed the structure, you can read the predicted tags like this:

@Model(adaptables = Resource.class,
       defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class AssetSmartTags {

    @Self
    private Resource resource;

    public List<String> getTopTags(double minConfidence) {
        List<String> tags = new ArrayList<>();
        Resource predicted = resource.getChild("jcr:content/metadata/predictedTags");
        if (predicted == null) {
            return tags;
        }
        for (Resource tag : predicted.getChildren()) {
            ValueMap props = tag.getValueMap();
            String name = props.get("name", String.class);
            double confidence = props.get("confidence", 0.0);
            if (name != null && confidence >= minConfidence) {
                tags.add(name);
            }
        }
        return tags;
    }
}

Filtering by minConfidence keeps low-quality guesses out of anything customer-facing.

Using Smart Tags in Search

Because Smart Tags are indexed with the asset, they show up in Assets search automatically, and you can combine them with other filters. For a better experience:

• Teach authors to search by concept, not just file name
• Curate the top searched assets so their tags are accurate
• Combine Smart Tags with metadata schemas (covered in the next post) for structured filtering

Governance Tips

• Decide who owns the taxonomy. Changing tag names later affects training and search
• Set a review cadence. Sample new uploads, check tag quality, and retrain when your taxonomy changes
• Be deliberate about sensitive content. Disable AI tagging on folders where automated analysis isn't appropriate
• Check licensing. Smart Tag features depend on your Adobe entitlements, so confirm what's included before planning around them
• Don't treat tags as ground truth. Use confidence thresholds for anything automated, such as filtered feeds or recommendations

Troubleshooting

• No Smart Tags on new uploads: check that enhancements aren't disabled at environment or folder level, and that asset processing completed
• Tags are generic, not on-brand: train enhanced Smart Tags on your taxonomy
• Training status isn't green: your training set may be too small, or a tag may have too few examples. Add more consistent examples and retrain
• Tags look wrong: check the training set for mislabeled assets, then curate and retrain

What's Next

Smart Tags work best when your library is well organized. In the next post we step back from automation to DAM folder structure best practices, the foundation that makes permissions, processing profiles, metadata and search all work smoothly.

Next in this series: [DAM Folder Structure Best Practices]