Sleep More; Triage Faster with Sentry
As a developer, triage duty week was often the worst week of my month. Anytime a bug was reported, I’d search for the right environment, wander through logs, pray there was an associated stack trace, use my mental mapping of our code base, and route bugs to the right teams.
Developers on triage rotation need to ensure bugs are routed to the correct team along with adequate information to help the team investigate the bug. If there’s not enough context, the precious time developers would rather spend building their product will be spent debugging instead.
Today, Sentry streamlines the triaging and context-gathering process for more than four million developers. Nonetheless, in recent months we challenged ourselves to do better and asked ourselves - how can Sentry help developers triage issues faster?
We’ve been hard at work to enable exactly this outcome for our users by improving time to resolution and issue resolution rate.
Improving issue resolution rate and time to resolution
Triaging an issue can be broken down into four phases:
Detection
Notification
Assignment
Resolution
Based on user feedback and interviews, we determined that improvements to notifications and assignments result in better outcomes for our users.
We focussed on two specific improvement areas:
Improving issue assignment with Suggested Assignees
Reducing noise due to un-targeted notifications
Improving Assignment With Suggested Assignees and Reduced Alert Noise
Suggested Assignees allow you to easily identify and assign issues to the right team or developer.
For each issue, Suggested Assignees are members or teams in your Sentry org who likely have the most context about that issue.
Sentry determines the Suggested Assignee for an issue based on three signals:
Suspect Commits - Sentry identifies the developer that most recently changed the code that triggered this issue
Ownership Rules - Sentry evaluates rules specified by you to identify the appropriate Sentry team/member based on an issue’s tags, code paths, etc.
Code Owners - Sentry evaluates your CODEOWNERS files from GitHub/GitLab to identify the appropriate Sentry team/member for your issue based on its code path.
Based on customer feedback we learned that Suggested Assignees help our users reliably and quickly assign and resolve issues. To enable Suggested Assignees for more users, we launched a number of features in recent months:
Suspect Commits via Git Blame. Suspect Commits used to just show you who last modified the file causing the Sentry issue. By integrating with the Git Blame API, Sentry will now identify who changed the specific line of code, and the pull request which introduced the code causing the issue. You can now also automatically assign issues to the Suspect Commit author.
This feature is available for any customer that uses our GitHub or GitLab integration. Please read this section for more details on enabling this feature for your Organization.
Automatically Setting up Code Mappings. In order to derive Suspect Commits for your issues, Sentry needs to connect the file that causes the error in an issue to the source code file in your repository. This setup can be tricky to get right and we noticed that our users often missed setting this up.
We’ve automated this setup for users that use Sentry’s GitHub integration. Sentry can automatically map the code causing your errors to your source code for Python, JavaScript, Node, and Ruby platforms. This automated setup has allowed Sentry to derive Suspect Commits for numerous more issues.
Reducing un-targeted Sentry notifications. Sentry creates a default Issue Alert rule when you create a new project. We noticed that for issues without Suggested Assignees, this rule can lead to several untargeted notifications to your team members working on the same project.
We’ve changed this alert rule to use a more limited fallback that informs your teams without spamming all members of that project.
Reaching Better Outcomes
Here’s how these improvements helped users achieve better issue resolution outcomes:
When an issue has a Suggested Assignee, Sentry can notify the right developer, team, or Slack channel based on your Alert Rule settings.
This helps you cut down unnecessary notifications for your developers and helps draw their attention toward issues that matter to their teams. These benefits compound as more issues have Suggested Assignee.
Additionally, Sentry’s changes to our default Issue Alert rule reduced un-targeted notifications by ~33% across our customers without hampering their ability to triage issues promptly.
Suggested Assignees help you take better ownership of fixing errors. Developers pay more attention to errors when they see themselves or their team as a Suggested Assignee for a Sentry issue.
Our data shows that Sentry customers that installed our GitHub integration resolved issues with Suspect Commits at a 20% higher rate and with a 24-hour shorter time to resolution compared to issues without Suspect Commits.
What's Next
Sentry will continue investing in making triaging more efficient for our users. Stay tuned for the following improvements which are currently available for early adopters.
Targeted notifications to Suspect Commits.
Sentry’s alert rule actions today allow you to notify Suggested Assignees identified through Codeowners or Ownership rules but not Suspect Commits.
We’ve made improvements to Sentry’s default Issue Alert rule to notify all Suggested Assignees including authors of Suspect Commits. By including both Suggested Assignees and authors of Suspect Commits, un-targeted notifications will be reduced, improving the signal:noise value from your notifications.
Support for larger CODEOWNERS files.
We will support CODEOWNERS files, up to the GitHub limit of 3MB. This means we will be able to identify Suggested Assignees for more issues as your code base continues to grow.
Easily add Ownership Rules.
Developers care about particular code paths or transactions in a Sentry issue and often prefer having similar issues assigned to their teams in the future.
We’ve shipped improvements that make it easier for you to view who’s owning what code areas in your Sentry project. You will also be able to easily claim ownership from the Sentry issue itself. Using these tools will make your triage much faster as you continue seeing new errors
Your feedback is much appreciated
As you try out these new features, we want to hear from you. Reach out via Discord, GitHub, or email us directly at feedback-issues@sentry.io to share your experience and feedback.
And if you’re new to Sentry, you can try it for free or request a demo to get started.
