Create Data 360 Reports to Monitor Instruction Adherence
Create Data 360 reports to monitor how well agents are interpreting subagent instructions.
Required Editions
| Available in: Lightning Experience |
| Available in: Enterprise, Performance, Unlimited, and Developer Editions. Required add-on licenses vary by agent type. |
| User Permissions Needed | |
|---|---|
| To create, edit, and delete reports in private folders: | Create and Customize Reports |
| To create, edit, and delete reports in public and private folders: | Report Builder OR Report Builder (Lightning Experience) |
| To turn on data collection | View Setup and Customize Application |
| To view and access Data 360 reports: | Data 360 User |
See Also
Set Up Audit and Feedback Data Collection
Instruction adherence scores depend on generative AI audit and feedback data.
Turn on data collection for generative AI audit and feedback data (see Set Up Einstein Generative AI Audit and Feedback). Thereafter, data becomes visible in Data 360 within 24 hours. Data for agent generations and user feedback (including instruction adherence metrics) streams continuously on an hourly basis to Data 360. Instruction adherence data is stored in the Content Category (GenAIContentCategory) data library object (DLO) in the Data Model for Generative AI Audit and Feedback.
Create a Report to List Instruction Adherence Scores with Explanations
Use this report to monitor and analyze how well the agent interprets the subagent instructions.
- On the Reports tab, click New Report.
- For the category, select Data Cloud, and select GenAIContentCategory for the report type.
- Click Start Report.
-
On the Outline tab, select these columns.
- Detector Type
- Category
- Value
- Parent
- Timestamp
-
On the Filters tab, select Detector Type for
the field, equals for the Operator, and type
InstructionAdherence for the value.
When you run the report, the Category column indicates the level of instruction adherence: high, low, or uncertain. To limit the report to show only the instances of high or low adherence, you can add another filter.
- Click Save and Run to generate the report. When prompted, give the report a name and description.
- From the generated report, select one of the items for which you want to see the related agent response. Copy the value (parent ID) from the Parent column.
Create a Report to View Agent Responses for Individual Scores
Use this report to see the related agent response for an individual instruction adherence score.
- From the Reports tab, click New Report.
- For the category, select Data Cloud, and select GenAIGeneration for the report type.
- Click Start Report.
-
On the Outline tab, select these columns.
- Generation Id
- Response Text
- Timestamp
-
On the Filters tab, create a filter for
GenerationIdthat uses the parent ID that you copied from the first report you created. - Click Save and Run to run the report. The report includes the agent response for the item you selected from the instruction adherence report.
