Reviews and feedback

Feedback from the people who use a Coworker is the raw material for its reviews. A Supervisor works through that feedback one item at a time, adjusts the Coworker, marks how each item turned out, and records the outcome.

Giving feedback#

Under each reply from a Coworker in chat are a thumbs-up and a thumbs-down.

• Select the thumbs-up when the reply was good.
• Select the thumbs-down when it was not. You can then pick the kind of problem, from Not sure (the default), Hallucination, Stale data, Misread, Gap and Error or failure, and describe the problem in the "What went wrong?" box. Select Send.

Anyone who can chat with a Coworker can give feedback on its replies. A thumb you give is counted on its own: that conversation is not also reviewed automatically. Your Coworker's Supervisor can read conversations that have been flagged, as part of reviewing the Coworker.

How helpful the Coworker has been#

Besides thumbs, a sample of a Coworker's finished interactions is reviewed automatically to estimate how often people were helped, and for which of its duties. The result is the Helped figure on the Signals card of the Coworker's details page, next to the thumbs-up rate. It is visible to the Supervisor and to admins and executives; see Coworker details. The Supervisor can also open the interactions that went badly from there, and select Review conversations now there.

Finding reviews that are due#

Open Governance and choose Reviews.

• Upcoming lists reviews due in the next two weeks, or already overdue, for the Coworkers you can see. A monthly review falls due 30 days after the last one, and a quarterly review 90 days after the last one. For a Coworker that has had no review, the clock starts from when it was hired. If nothing is due, the page says "Nothing due in the next two weeks."
• Recent reviews lists the ten latest recorded reviews with their date, outcome, who recorded them and the summary. Selecting a Coworker's name opens its review page.
• Choose a Coworker to review… with Start review opens a review for any Coworker you pick.

A Coworker's details page also has a Reviews list with its full history, and a Start a review button for its Supervisor.

The review page#

The review page is titled with the Coworker's name and the path Governance / Guided review. Choose Monthly review or Quarterly deep review at the top.

Only the Coworker's Supervisor can read its feedback, mark items and record a review. If you are someone else, the page tells you that working through feedback requires being the Supervisor.

Working through the feedback#

Feedback to work through is one list of everything that went wrong since the last review of this type: every thumbs-down and flagged item from users, and every interaction automatic review judged did not help the person (tagged Automatic review, with the reason). The list is grouped into themes by the weekly feedback synthesis, and Run synthesis now runs one straight away. Items that are not in a theme are listed last under Not in a theme. A line above the list shows your progress, such as "3 of 8 worked", and menus narrow the list by status and by reason.

Select an item to open its conversation right under it, then try to reproduce the problem and fix the Coworker the way you normally would. Two links at the top of the page, Open chat with the Coworker and Coworker details, open a chat and the details page (where instructions and Knowledge Bases are edited) in a new tab, so you keep your place in the review.

Marking each item#

Under the conversation, choose how the item turned out: Fixed, Could not reproduce, Transient error, Not fixable, Not a problem or Other, add a comment if you like (it is required for Other), and select Save, or Save and open next to move on to the next item that has no marker. An item with no marker shows Open. Mark as open removes a marker. Markers stay with the Coworker, so items you marked in an earlier review stay marked. See Running a guided review for the full page.

Finish review#

When you are done:

1. Choose an outcome from the list: Healthy, Watch, Sunset or Due.
2. Write a Summary.
3. Select Save review.

You return to Governance, where the review appears under Recent reviews. Recording a review does not change the Coworker's phase; to retire it, see Advancing a Coworker and retiring it.