Skip to main content

Feedback and Finding Tuning

Buzzbin can capture 👍 and 👎 reactions on finding comments in GitLab as feedback. This helps admins and the product team understand which findings are useful and which ones are noisy.

Where feedback is captured​

A reviewer reacts on the same GitLab thread Buzzbin created for a finding. Buzzbin receives the emoji webhook event and maps it back to the original finding.

Only these reactions are used as v1 signals:

  • thumbsup for useful
  • thumbsdown for not useful or noisy

Other emoji are ignored.

How votes are counted​

For each finding and username, only one current vote counts. If a user reacts with 👍 and later switches to 👎, the new vote replaces the old one. If the user revokes the reaction, the vote is removed from aggregates.

This makes webhook replay and vote changes idempotent instead of double-counting.

What is stored​

Buzzbin stores the finding link, reacting username, sentiment, and timestamps. It does not duplicate finding text or code onto the feedback row.

Helpful rate​

Helpful rate is calculated as:

positive / (positive + negative)

When a slice has too few votes, Buzzbin does not show the percentage as a firm number. It shows raw counts or a not-enough-data state so decisions are not based on tiny samples.

What tuning means​

Today, feedback is primarily used for analytics and decision-making. Automatically changing prompts or review categories from votes should be handled separately and carefully.

Safer short-term uses include:

  • finding categories with many downvotes
  • comparing model quality across repositories or time windows
  • identifying noisy rules or paths
  • manually adjusting enabled categories or custom instructions

To see how feedback appears in dashboards, see Insights and Analytics.