Reliability and validity are two of the key properties of any psychometric test. In this article, you may find information on the different types of reliability and validity as well as the relationship between them and discuss what Bryq does to ensure the psychometric soundness of the tool.
How was the Bryq assessment developed?
Bryq was developed based on established I/O psychology research results, with the goal of leveraging proven methods and maximizing the psychometric properties of the resulting assessment. Every new development is evaluated by two distinct teams of I/O psychologists, significantly reducing the number of potential blind spots.
Explaining Reliability
When using a tool we need to ensure that it provides us with accurate results.
Reliability refers to the accuracy of the measurements we make. It is an essential property of any psychometric test and indicates the degree to which a person’s actual test score results from their ‘true score’ on the construct being measured, and how much it is due to ‘error’. Considering that no measurement can be assumed to be perfectly accurate, reliability estimates help us to measure the degree of error, hence allowing us to quantify the accuracy of the test items.
Let’s deep dive into the main types of reliability and specific reliability estimates of the Bryq assessment.
There are two ways to assess reliability: Test-retest & Internal consistency reliability.
Test-retest reliability is a method that involves administering the same assessment on two different occasions, in order to determine the extent to which individuals will obtain similar scores. For Bryq Test-retest reliability was examined in a group of 200 individuals, with three months between administrations. Stability coefficients were above the commonly accepted thresholds.
When it comes to Internal consistency reliability, it assesses if an individual will respond similarly to all items of the same trait or ability. Our several and frequent analyses reveal consistent results over time with the vast majority of the scales ranging above the designated Cronbach alpha thresholds.
Explaining Validity
When using a tool we need to ensure that it is suitable for the designated purpose.
Validity should be an ultimate consideration when evaluating an assessment. Essentially, validity answers the question ‘Is the test suitable for my purpose?’. It is common for validity evidence to be established via a number of sources, commonly known as content validity, construct validity, and predictive validity.
Let’s deep dive into these sources:
Content validity is a prerequisite to the other sources of validity and it refers to the degree to which the content of the items is related to the content domain of the construct to measure. In order to assess content validity the most common procedure involves evaluation of the assessment by a panel of I/O psychologists. That’s what we also did in Bryq, where the two teams of I/O’s carried out an interjudge agreement analysis to validate the content of the assessment.
Construct validity refers to how well the assessment measures what it is supposed to measure. One of the most common ways of measuring construct validity is to compare the assessment with other major, valid and reliable assessments and check for the correlation between them. The Bryq assessment was compared with two of the most well-established personality inventories namely: Cattell’s 16PF (ranging and the NEO PI-R. The results revealed strong correlations among the inventories.
Predictive validity refers to how exactly the tool is in formulating predictions. To ensure predictive validity you first start with a strong reliable basis. The Bryq assessment is built on the solid basis that cognitive ability combined with personality is one of the strongest and most valid predictors of future job performance. Findings from several talent benchmarking exercises conducted with Bryq’s customers support the aforementioned notion. Specifically, the majority of the talent benchmarking studies conducted by the Bryq team showed a strong positive correlation between the overall Bryq score and top performers.
To sum up,
Reliability and validity are key components of a psychometric test and are published in technical manuals of the assessments. An assessment must have both of these properties to be classified as a psychometric tool.
The reliability confirms the accuracy of the tool and the validity confirms that it is measuring the constructs which it claims to measure.
It is important to note that a test cannot be valid without being reliable, although it is possible to be reliable and not valid.
Our Ongoing Goals
Bryq starts with a reliable basis and we rely on evidence-based research findings.
The theoretical frameworks Bryq uses are well-documented.
We ensure that we are always reliable with quarterly reliability checks on all of the Bryq items.
We ensure that our translations are done systematically by accredited professionals to keep the reliability of the items across languages.
We conduct benchmarking exercises to ensure the predictive validity of our tool.
As always, if you have any further questions please don’t hesitate to reach our Bryq support. We are always happy to help!