Reliability and validity are two of the key properties of any psychometric test. In this article, you can find information on the different types of reliability and validity and the relationship between them. We also discuss what Bryq does to ensure the tool is psychometrically sound.
How was the Bryq assessment developed?
Bryq was developed based on established I-O psychology research results, with the goal of leveraging proven methods and maximizing the psychometric properties of the resulting assessment. Every new development is evaluated by two distinct teams of I-O psychologists, significantly reducing the number of potential blind spots.
Explaining Reliability
When using a tool we need to ensure that it provides us with accurate results.
Reliability refers to the accuracy of the measurements we make. It is an essential property of any psychometric test and indicates the degree to which a person’s actual test score results from their ‘true score’ on the construct being measured, and how much it is due to ‘error’. Considering that no measurement can be assumed to be perfectly accurate, reliability estimates help us to measure the degree of error, hence allowing us to quantify the accuracy of the test items.
Let’s dive deep into the main types of reliability and specific reliability estimates of the Bryq assessment.
There are two ways to assess reliability: Test-retest & Internal consistency reliability.
Test-retest reliability is a method that involves administering the same assessment on two different occasions, in order to determine the extent to which individuals will obtain similar scores. At Bryq, we used a group of 200 individuals to assess our Test-retest reliability, assessing subjects twice, 3 months apart. Stability coefficients were above the commonly accepted thresholds.
When it comes to Internal consistency reliability, it assesses if an individual will respond similarly to all items of the same trait or ability. Our multiple and frequent analyses revealed consistent results over time, with the vast majority of the scales ranging above the designated Cronbach alpha thresholds.
Explaining Validity
When using a tool, we need to ensure that it is suitable for the designated purpose.
Validity should be an ultimate consideration when evaluating an assessment. Essentially, validity answers the question ‘Is the test suitable for my purpose?’. It is common for validity evidence to be established via a number of sources, with these commonly being known as content validity, construct validity, and predictive validity.
Let’s dive deep into these sources:
Content validity is a prerequisite to the other sources of validity and it refers to the degree in which the content of the items is related to the content domain of the construct to measure. In order to assess content validity, the most common procedure involves evaluation of the assessment by a panel of I-O psychologists. That’s what we also did at Bryq, where the two teams of I-Os carried out an interjudge agreement analysis to validate the content of the assessment.
Construct validity refers to how well the assessment measures what it is supposed to measure. One of the most common ways of measuring construct validity is to compare the assessment with other major, valid and reliable assessments and check for the correlation between them. The Bryq assessment was compared with two of the most well-established personality inventories, namely Cattell’s 16PF and the NEO PI-R. The results revealed strong correlations among the inventories.
Predictive validity refers to how precise the tool is in formulating predictions. To ensure predictive validity, one must first start with a strong reliable basis. The Bryq assessment is built on the solid basis that cognitive ability, combined with personality, are one of the strongest and most valid predictors of future job performance. Findings from several talent benchmarking exercises carried out on Bryq customers support the aforementioned notion. Specifically, the majority of the talent benchmarking studies showed a strong positive correlation between the overall Bryq score and top performers.
To sum up:
Reliability and validity are key components of a psychometric test and are published in technical manuals of the assessments. An assessment must have both of these properties to be classified as a psychometric tool.
The reliability confirms the accuracy of the tool and the validity confirms that it is measuring the constructs it claims to measure.
It is important to note that a test cannot be valid without being reliable, although it is possible to be reliable and not valid.
Our Ongoing Goals
Bryq starts with a reliable basis and we rely on evidence-based research findings.
The theoretical frameworks Bryq uses are well-documented.
We ensure that we remainreliable with quarterly reliability check-ins on all Bryq items.
We ensure that our translations are done in a systematic way by accredited professionals to maintain the reliability of the items across languages.
We conduct benchmarking exercises to ensure the predictive validity of our tool.
As always, if you have any further questions please don’t hesitate to reach our Bryq support. We are always happy to help.