AI Skills v2 is coming (on 11.9)

We’re launching AI Skills v2 to give you a more consistent, representative picture of how your engineers work with AI and where they can grow.

What’s changing?

Each day, v2 randomly selects one eligible session per engineer for analysis. To be eligible, the Weave plugin must be installed and the engineer must send more than four prompts (at least five) in the session. This approach helps ensure all engineers with eligible sessions are represented, instead of letting a few people with many sessions dominate the picture.

We’ve also made some significant updates to the AI Skills UI so don’t be too surprised! We’ve improved the insight breakdowns of the Skills and now allow you to see your scores progress over time.

How v2 analyzes skills differently

V1 estimates skills from summarized and classified conversation turns. V2 assesses the engineer’s actions against specific criteria, considering the full conversation for planning and nearby turns for other skills.

V2 is designed to produce more precise, actionable assessments in three ways:

Together, these changes are designed to make assessments more precise and easier to understand.

What does that look like?

A smooth transition

For the last two weeks, we’ve been running v1 and v2 in the background for your org. On 11.9, we will be initiating the cutover, and giving you the ability to explore both v1 and v2 results side by side from the two weeks. If you’d prefer to opt out of v2, reach out to us directly and we can keep your org on v1. If you like what you are seeing, v2 will be the default moving forward.