---
title: How Bryq Builds AI Skills Assessments
description: How Bryq builds AI Skills Assessments
---
[Skip to content](https://help.bryq.com/en/how-bryq-builds-ai-skills-assessments#main-content)
[](https://help.bryq.com/?hsLang=en)
Open main navigation
Close main navigation
How can we help you?
- There are no suggestions because the search field is empty.
1. [Bryq](https://help.bryq.com/en?hsLang=en)
2. [Talent Acquisition](https://help.bryq.com/en/talent-acquisition?hsLang=en)
3. [Skills Assessments](https://help.bryq.com/en/talent-acquisition?hsLang=en#skills-assessments)
November 9, 2026
# How Bryq Builds AI Skills Assessments
### Introduction
Every Bryq AI skills assessment measures what people do with AI at work. No question tests definitions, theory or how AI models are built. Knowing about AI and working well with it are two different skills. Only the second one is scored.
The general AI Fluency Assessment is built on six peer-reviewed research frameworks, including UNESCO, SFIA and OECD competency models. Role-specific assessments follow the method below. Product management is used as the example throughout.
#### Step 1: Study the Role
Each role-specific assessment starts from two kinds of evidence:
- **Published research** on AI in that kind of work. Each claim about AI rests on two studies. One shows the human habit behind the problem. The other shows the same problem with AI and was published in 2023 or later.
- **A job analysis** of live US job postings for the role that mention AI, all from the last 12 months. Bryq reads at least 100 different postings per role.
The job analysis shows what the role produces, such as roadmaps and stakeholder updates for a product manager. For each of these outputs, Bryq writes down what good work looks like when AI is involved. It also lists the ways AI makes that work go wrong. These are called failure modes.
A failure mode records three things: what goes wrong, why nobody notices, and what a capable person without AI fluency does instead.
#### Step 2: Remove the Job Itself
Every failure mode faces one test: take the AI out of the work. If the problem still happens, it belongs to the job in general, so it is left out. Only problems that exist because AI is involved stay. This keeps the assessment measuring AI fluency inside the role, separate from skill at the role itself.
#### Step 3: Choose the Skills
Failure modes are grouped by the skill that would have prevented them. Each group must pass five tests to become a skill the assessment measures:
| Test | What it checks |
| --- | --- |
| Specific to the role | Only this role runs into the problem. Problems every professional meets belong in the general AI Fluency Assessment. |
| Not already covered | The general AI Fluency Assessment does not measure it, or not at this depth. |
| Measurable in a work scenario | One scenario from the role's work can measure it, with a graded score. |
| Needs AI fluency to answer | A strong professional with little AI experience gets it wrong. |
| Enough evidence | Several scenarios, each from separate research, can measure it. |
A role-specific assessment keeps three to five skills, with five questions for each.
#### Step 4: Write the Scenarios
Each scenario is a real moment from the role's work, with AI already part of it. Before writing, Bryq fixes four rules for every scenario:
- **Difficulty:** an easier scenario turns on one AI behavior. A core one weighs two concerns against each other. A harder one rewards a response that goes against common practice.
- **Plain language:** someone outside the role can follow it on a first read. Every role term is explained where it appears.
- **Fairness:** nothing gives anyone an edge for reasons unrelated to AI fluency. That includes first language, cultural background, access to a paid AI tool or experience at one kind of company.
#### Step 5: Weight the Options
Every question has four options, and every option carries a weight:
| Option | What it does | Weight |
| --- | --- | --- |
| Best | Splits the work correctly between the person and the AI, in line with what is at stake. | Full |
| Workable | Reaches an acceptable result at a real cost, such as an extra round of work. | Part |
| Plausible | Sounds reasonable but rests on a misunderstanding of how AI works. | Small part |
| AI-naive | What a capable professional without AI fluency would do. It sounds sensible. | None |
The best and AI-naive options always differ on one checkable fact about how AI behaves. That might be something the AI cannot know, or information it never sees. It might also be a gap between how reliable its output looks and how reliable it is. They never differ on style or tone.
What earns full weight changes from question to question. Sometimes it is giving the AI more of the work than instinct suggests. Sometimes it is changing what the AI is given, instead of checking what it returns. Sometimes it is not using AI for that step at all. No single habit, such as always picking the most careful option, leads to a high score.
#### Step 6: Check Every Question
Every question and option must pass these checks, or it is rewritten or cut:
- **Needs the AI:** with the AI taken out, the question stops working.
- **Specific to the role:** with another role swapped in, the question stops working.
- **No clue in the options:** with the scenario hidden, nobody can pick the best option from how the options are written.
- **Every wrong option is a real choice:** a capable professional would pick each one. None can be ruled out without knowing how AI works.
#### Step 7: Review and Calibrate
Two rounds of independent review follow. Bryq's in-house industrial and organizational (I/O) psychologists review every scenario for readability. An automated review then applies every rule to every scenario, several times over.
Calibration continues after launch. As people take the assessment, Bryq compares each question's results with its planned difficulty. It also checks how well each question separates people who work well with AI from those who do not. Any question that does not measure as intended is revised or replaced.
- [Integrations](https://help.bryq.com/en/integrations?hsLang=en)
- [About Bryq](https://help.bryq.com/en/about-bryq?hsLang=en#main-content)
- [Overview](https://help.bryq.com/en/about-bryq?hsLang=en#overview)
- [Compliance](https://help.bryq.com/en/about-bryq?hsLang=en#compliance)
- [Set up and Configuration](https://help.bryq.com/en/set-up-and-configuration?hsLang=en#main-content)
- [3rd Party Integrations](https://help.bryq.com/en/set-up-and-configuration?hsLang=en#3rd-party-integrations)
- [Setting up Bryq](https://help.bryq.com/en/set-up-and-configuration?hsLang=en#setting-up-bryq)
- [Talent Acquisition](https://help.bryq.com/en/talent-acquisition?hsLang=en#main-content)
- [Understanding assessment results](https://help.bryq.com/en/talent-acquisition?hsLang=en#understanding-assessment-results)
- [Skills Assessments](https://help.bryq.com/en/talent-acquisition?hsLang=en#skills-assessments)
- [Setting up jobs](https://help.bryq.com/en/talent-acquisition?hsLang=en#setting-up-jobs)
- [Bryq Assessment (Profile Fit)](https://help.bryq.com/en/talent-acquisition?hsLang=en#bryq-assessment-profile-fit)
- [Candidate Experience](https://help.bryq.com/en/talent-acquisition?hsLang=en#candidate-experience)
- [Using Bryq - Best Practices](https://help.bryq.com/en/talent-acquisition?hsLang=en#using-bryq-best-practices)
- [Hiring Managers](https://help.bryq.com/en/talent-acquisition?hsLang=en#hiring-managers)
- [Talent Management](https://help.bryq.com/en/talent-management?hsLang=en#main-content)
- [Growing your Talent with Bryq](https://help.bryq.com/en/talent-management?hsLang=en#growing-your-talent-with-bryq)
- [Setup](https://help.bryq.com/en/talent-management?hsLang=en#setup)
- [Team Insights](https://help.bryq.com/en/talent-management?hsLang=en#team-insights)
- [Role Validation](https://help.bryq.com/en/talent-management?hsLang=en#role-validation)
- [Bryq Science](https://help.bryq.com/en/bryq-science?hsLang=en#main-content)
- [The Science behind Bryq](https://help.bryq.com/en/bryq-science?hsLang=en#the-science-behind-bryq)
- [FAQs](https://help.bryq.com/en/faqs?hsLang=en)
- [Settings](https://help.bryq.com/en/settings?hsLang=en)
- [Product Tutorials](https://help.bryq.com/en/product-tutorials?hsLang=en)
[](https://help.bryq.com/?hsLang=en)
[Bryq App](https://app.bryq.com/) • [Bryq Website •](https://www.bryq.com/) [App Updates](https://news.bryq.com/product-updates?hsLang=en)
Copyright © 2026, bryq.com