DEVELOPING A TEST FOR THE ACQUISITION OF PSYCHOMETRIC CONCEPTS
https://doi-001.org/1025/17611402842141
Fatima Zohra Lachraf *1, Samira Missoun 2, Nabila Benzine 3, Asma Khouiled 4, Yassia Zebeir 5, Sarra Khouiled 6, Hind Ghedhaifi 7, Abdelaziz Khouiled 8
1 Martyr Hamma Lakhdar University – El Oued, Algeria
2 University Center – El Bayadh, Algeria
3 Kasdi Merbah University – Ouargla, Algeria
4 Ziane Achour University – Djelfa, Algeria
5 University of Biskra, Algeria
6 Ecole Normale superieure de Ouargla, Algeria
7 Social Development and Community Service Laboratory, El Oued university, Algeria
8 University Oran 1 Ahmed Ben Balla, Algeria
Emails: fatimalachraf@gmail.com
Received: 10/07/2025 ; Accepted : 02/10/2025
Abstract
The present study aims to develop a test to measure the extent to which students of psychology and educational sciences have acquired concepts of psychometrics. To achieve this objective, a sample of 156 male and female second-year undergraduate students in Educational Sciences at Martyr Hamma Lakhdar University in El Oued was selected using a simple random sampling method. To determine the psychometric properties of this test, its validity was verified using two methods: face validity and internal consistency validity. Its reliability was examined using the split-half method and internal consistency by calculating Cronbach’s alpha coefficient. Using these statistical methods, the results revealed that the test developed in this study to measure the acquisition of psychometric concepts demonstrated acceptable levels of validity and reliability.
Keywords: Test, concept acquisition, psychometrics.
INTRODUCTION:
Life is witnessing significant cognitive developments and emerging trends in various fields of human development, whether in concept development, thinking styles, or other innovations that have affected different aspects of our lives, particularly the field of education. This situation now requires those responsible for the educational process to keep pace with these changes by providing learners with methods that help them advance, adapt to ongoing developments, and acquire concepts that can contribute to enhancing their knowledge base. Through the accumulation and interconnection of concepts, a person learns principles and laws, and then arrives at theories, thereby building a cognitive system that distinguishes them from others and helps shape their personality, which in turn determines their attitudes toward people, topics, and objects in the external world.
Accordingly, many researchers, including Al-Ta’i and Al-Salifani (2014), point out that learning and developing concepts among learners has become one of the most prominent goals of educational institutions at all levels, especially higher education. Therefore, academic achievement is no longer the sole objective of the educational process; more importantly, it is to equip learners with the concepts of academic subjects.
RESEARCH PROBLEM:
Today, the world is experiencing an enormous information revolution, making nations and societies measured not only by the knowledge and information they possess but also by the extent to which they can extract the concepts and relationships embedded within this knowledge. Concepts represent one of the fundamental levels of cognitive and scientific structures, to which the other components of knowledge are added. This has placed the topic of concept acquisition at the center of attention for individuals and societies, making it a key indicator of students’ ability to engage in various forms of scientific thinking and to achieve success and excellence in different fields.
Furthermore, concepts play a major role in transferring the effects of learning, expanding students’ perceptions, and directing their attention beyond mere rote memorization of the educational material—particularly in the field of psychometrics. Psychometrics occupies an important place among the subjects taught to students of psychology and educational sciences at the university, deriving its significance from its nature and relevance to both their academic and professional lives. The concepts in psychometrics are numerous and strongly interrelated to the extent that they may sometimes be used for the same purpose; for example, some may use the term “measurement” to mean “evaluation” or “assessment,” or vice versa. However, each concept has its own specific use that differs from others, while still maintaining a degree of complementarity. Moreover, the field of psychometrics, with its set of concepts, helps individuals make more appropriate decisions and aids psychological specialists in understanding and diagnosing problems.
Several researchers have also linked psychometrics to the level of progress or backwardness of nations, noting significant differences among countries in the extent to which psychological measurements are widespread. When making decisions about any psychological trait or educational phenomenon, it is necessary to resort to measurement tools that help in obtaining accurate data and information. Among the most prominent of these tools are psychological and educational tests, which allow us to describe phenomena quantitatively and to draw conclusions that help in evaluating them.
Based on the above, and despite the considerable importance of psychometric concepts, a review of the available tests and measurements related to this variable revealed the existence of instruments concerned with measuring learning and the acquisition of concepts in various other subjects. However, regarding tools specifically designed to measure university students’ acquisition of psychometric concepts, we have observed within the limits of our review—that they are scarce, with only one notable study, Nazzal (2012), who developed a test to measure measurement and evaluation concepts.
Accordingly, this study seeks to answer the following main question:
Does the psychometric concepts acquisition test developed in this research possess good psychometric properties?
To answer this question, it has been broken down into the following sub-questions:
Does the psychometric concepts acquisition test developed in this research have acceptable validity coefficients consistent with the characteristics of a good test?
Does the psychometric concepts acquisition test developed in this research have acceptable reliability coefficients consistent with the characteristics of a good test?
Research Hypotheses:
The psychometric concepts acquisition test developed in this research has acceptable validity coefficients consistent with the characteristics of a good test.
The psychometric concepts acquisition test developed in this research has acceptable reliability coefficients consistent with the characteristics of a good test.
Research Significance:
From a scientific perspective, this research is among the first local contributions to develop a test that measures the extent to which university students have acquired psychometric concepts. It also highlights the importance of scientific concepts and their role in the transfer of learning effects, along with the growing focus on teaching and learning concepts given that these concepts represent the building blocks of educational material. Moreover, the subject of psychological and educational measurement is one of the core courses for psychology and educational sciences students, as it helps develop students’ abilities and provides them with knowledge and concepts.
From a practical perspective, this research aims to provide a tool for measuring students’ acquisition of psychometric concepts, which can be used in future research and studies in this field.
Research Objectives:
The present research sought to achieve the following objectives:
To identify a list of scientific concepts that should be taught to psychology and educational sciences students in the psychometrics course, scheduled for the third semester.
To enrich the psychological library with a tool for measuring the acquisition of psychometric concepts, designed in accordance with the scientific standards of good psychological tests, which may contribute to supporting the field of psychometrics in Algeria.
Research Limits:
Spatial limits: The current research was conducted within the Department of Psychology and Educational Sciences at Martyr Hamma Lakhdar University in El Oued.
Human limits: The study sample consisted of second-year undergraduate students in Educational Sciences at the Department of Psychology and Educational Sciences, Martyr Hamma Lakhdar University in El Oued.
Temporal limits: The study was conducted during the first semester of the 2018/2019 academic year.
Course limits: The research was based on the third-semester syllabus of the psychometrics course prescribed for undergraduate students specializing in Educational Sciences by the Ministry of Higher Education and Scientific Research in Algeria. In this study, the course will be referred to simply as Psychometrics.
Operational Definition of Research Concepts:
Concept Acquisition:
Acquisition: Defined by Abu Jadu (2003) as “the initial stage of learning during which the organism internalizes new behavior so that it becomes part of its behavioral repertoire” (Nazzal, 2012, p. 497).
Operational definition: The students’ ability to know, understand, use, and analyze the concepts of the psychometrics course. It is measured by the score obtained by students on the test specifically developed for this purpose.
Concepts: Defined by Al-Sharif (1996) as “an abstract mental image formed in the individual as a result of generalizing the common properties among the examples of a concept” (Al-Saadi, 2009, p. 789).
Operational definition: The mental image formed by students through abstracting the common characteristics of objects and symbols in the psychometrics course, consisting of the name and its verbal meaning. It is measured by the score obtained by students on the test specifically developed in this research for this purpose.
Operational definition of psychometric concepts acquisition: The extent to which students are able to define, distinguish, and apply the concepts of the psychometrics course, and use them in subsequent educational situations. This is expressed by the score each student obtains on the test developed in this research for this purpose.
Psychometric Properties: Refers to the essential characteristics related mainly to validity and reliability, which are calculated after administering the test to a representative sample of the population. In this research, it refers to:
Validity: Measured through two methods face validity and internal consistency validity.
Reliability: Measured through the split-half method and internal consistency using Cronbach’s alpha coefficient.
Theoretical Framework
Definition of a Psychological Test
Psychological tests hold a prominent position in most psychological and educational practices, and thus have received several definitions, including:
Leona E. Tayler (1982): “A standardized situation designed specifically to obtain a sample of behavior” (Tayler, 1988, p. 48).
Anne Anastasi (1982): “An objective, standardized measure of a particular sample of behavior” (Maameria, 2012, p. 104).
From these definitions, it can be concluded that a psychological test is an objective, standardized measure that aids in understanding, predicting, and anticipating behavior, based on a sample of an individual’s behavior that accurately represents the trait being measured.
Psychometric Properties of Psychological Tests
According to Bou Salem, the psychometric properties of a test are “the essential characteristics related to the effectiveness of the test items, as well as validity and reliability, and their associated indices such as discrimination indices, difficulty and ease levels in achievement and aptitude tests, and norms for interpreting results. These properties are verified after a pilot administration of the test to a representative sample of the population called the standardization sample. The quality and objectivity of the test depend on the appropriateness of these properties” (Bou Salem, 2014, p. 60).
Validity: Defined by Cronbach as the degree to which a test explains the trait intended to be measured (Al-Nimr, 2013, p. 69). The most common definition is that a test must measure the ability, trait, attitude, or aptitude it was designed to measure—that is, it must measure exactly what it claims to measure.
Reliability: Defined by Anastasi as the consistency of measurements obtained from the same individuals when retested with the same instrument on different occasions under the same conditions (SUSANA, 1997, p. 84). Reliability therefore refers to the stability of individuals’ results on the same test regardless of how many times it is administered, and to the internal consistency of the test, maintaining a high level of accuracy in measuring the intended trait.
Acquisition of Psychometric Concepts
Concepts are all that an individual develops in terms of meaning and understanding, expressed through words, phrases, or specific processes that lead to the development of thinking abilities and the construction of ideas sufficient to understand experiences of the world around them. Abu Hatab (1996, p. 597) defined them as a category of information or variables sharing common properties, involving processes of discrimination, evaluation, and classification. These may refer to objects, events, people, ideas, or other entities.
From this, they can be defined as the mental image formed by the student through abstracting common characteristics, linked to the terms and expressions of the psychometrics course, and consisting of the name and its verbal meaning.
The process of concept acquisition is considered a natural process that begins before children enter school. They attempt to discover concepts through their environment. Raouf (1978) notes that sensory perception is the child’s means of discovering the environment and its contents. Through the senses, the child perceives relationships or properties of the objects they interact with. As their experiences grow, they enter a stage of understanding and cognitive perception, classifying objects into categories based on shared characteristics and expressing them verbally (Mansour, 2014, p. 101).
According to Aram (2012), concept acquisition has become a primary goal at all educational levels, as it provides students with meaningful learning experiences, equips them with knowledge to keep pace with scientific developments, and enables them to make informed daily decisions.
Development of Number Concepts
Maameria (2012, p. 33) defines psychometrics as the process by which the amount of a given trait present in an individual, object, or phenomenon is determined. Quantifying traits helps students make accurate comparisons between objects, properties, or individuals.
Piaget believes that number concepts develop in stages corresponding to the child’s developmental phases and that instruction should align with these stages. According to Badawi (2016, p. 239), Piaget divided the development of number concepts into three stages:
Stage 1: Ages 4–5 years
Stage 2: Ages 5–6 years
Stage 3: Ages 6–7 years
In later stages of cognitive development, the child can classify, order, and count objects mentally. In the formal operational stage, they can manipulate symbols and diagrams without relying on physical objects (Boutros, 2015, p. 254).
Initially, learners may fail to recognize that each number follows another in a continuous series. Over time, they understand numerical sequencing—an important step toward meaningful counting—and then grasp the cardinal value of a set. Gradually, they develop number discrimination skills, enabling them to compare quantities between sets and determine ordinal positions.
Even if these skills are acquired, it is crucial to assess whether the learner understands number conservation the recognition that quantity remains constant despite changes in arrangement. German & Bayridsl confirm that once a child understands quantity conservation, they can engage in meaningful counting and identify the cardinal value of sets (Badawi, 2016, p. 248).
Development of Measurement Concepts
Measurement concepts one of the most frequently used mathematical skills in daily life involve using numbers to compare objects based on their properties. Boutros (2015, p. 269) states that in the concrete operational stage (ages 4–12), measurement skills develop fully. The process begins with a self-defined unit of measurement and evolves toward the use of standardized units for accurate measurement.
Through growth and experience, children become familiar with measurable properties and understand that differences among objects depend on variations in these properties. Measurement concepts thus mature during the concrete operational stage.
Importance of Psychometrics
Psychometrics is essential for individuals and societies, as it aims to identify methods for directing experiences toward desired objectives. Measurement and evaluation serve as crucial tools for identifying strengths to build upon and weaknesses to address.
Psychometrics has undergone significant advancements, enabling the quantification of psychological phenomena. In the behavioral sciences, measurement is not a mere physical process; it assesses psychological traits and educational and social behaviors. Through it, hypotheses can be tested, and more informed decisions can be made about individuals—a process increasingly vital in modern societies.
Lehmann & Mehrenz (2003) highlight that the importance of measurement and evaluation lies in making sound educational decisions and determining the types of change we aim to achieve in learners. This can only be done by measuring the extent to which objectives have been met. Measurement clarifies realistic objectives for each student, enhances motivation, encourages new study habits, and provides feedback on strengths and weaknesses (Nazzal, 2012, p. 493).
Accordingly, greater emphasis should be placed on psychometric concepts and on equipping every student with them, as they serve as the primary gateway to any study or research in psychology.
Field Procedures of the Research
Research Method:
This research adopted the descriptive method, as it is the most suitable for answering the research questions. It also provides accurate data and information that reveal the psychometric properties of the psychometric concepts acquisition test developed in this study, after its administration to the sample.
Research Sample:
The sample consisted of 15 male students and 141 female students, making a total of 156 second-year undergraduate students majoring in Psychology and Educational Sciences. The sample was selected using a simple random sampling method from the original population, whose members had completed the topics specified in this research from the psychometrics course.
Research Tool:
Steps of Developing the Test:
The process of constructing and developing the psychometric concepts acquisition test followed these steps:
Selection of the Course Content:
The selected material was the topics prescribed in the third-semester syllabus of the psychometrics course, totaling five main topics. Each topic was assigned a relative weight based on the total number of scheduled sessions and its relative importance.
Identifying the Course Concepts:
After selecting the psychometrics course and determining the topics to be taught in the third semester, we conducted a conceptual content analysis of the course to extract a set of concepts included in the material. This was done to achieve the study’s objective and to develop a test measuring students’ acquisition of psychometric concepts. We proposed 67 concepts distributed among the five main topics mentioned above. These extracted concepts were then presented to a group of specialized professors in psychometrics or those who had previously taught the course, who provided their feedback on them.
Constructing the Psychometric Concepts Acquisition Test:
To ensure the development of a quality test that serves the purpose of this study, we:
Reviewed previous tests and measurements that addressed the acquisition of scientific and mathematical concepts (Tawfiq, 2007; Nazzal, 2012; Aram, 2012; Mohammed, 2013; Touba, 2014).
Reviewed studies, opinions, and theories that addressed concepts and strategies for teaching and imparting them to students (Qushta, 2008; Al-Harahsha, 2008; Al-Saadi, 2008; Fendi & Ali, 2011; Aseeri, 2015; Al-Subaie & Al-Turki, 2016; Kawara, 2017).
Followed the steps for preparing an achievement test in drafting the test items. In this regard, Malham (2002, p. 204) emphasized that a test should be a representative sample of questions measuring the objectives and content according to their priority in the educational process. The objectives of the achievement test can be formulated based on the learned content or syllabus, ensuring an appropriate distribution of questions. In this case, a test specification table can be developed, providing a framework for constructing a test that better measures the learning material.
Based on this, Moshri (2018, p. 22) summarized the steps of constructing an achievement test into three main stages:
Defining the purpose of the test
Stage One: Test Blueprint
Analyzing the course content
Identifying the specific learning objectives
Designing the test specification table
Formulating the questions
Stage Two: Formulating the Questions and Instructions
Writing the instructions
Designing and formatting the test paper
Stage Three: Preparing the Test for Administration
Administering the test
Scoring
By following the above-mentioned steps, the Psychometric Concepts Test in this study was prepared as follows:
Stage One: Test Blueprint
Our objective in developing the psychometric concepts test was to measure the extent to which students have acquired the concepts of this subject.
Since the present study aimed to develop a test to determine students’ acquisition of psychometric concepts, we conducted a content analysis of the psychometrics course included within the methodological teaching units for Educational Sciences students in the third semester, as stated in the “L.M.D Academic Bachelor / Educational Sciences Program Alignment Model” for 2016 (canevas), approved by the Ministry of Higher Education and Scientific Research. The aim was to identify and extract the topics and concepts contained therein.
The content analysis process went through the following steps:
Purpose of the content analysis:
We sought to determine the subtopics for each main topic included in the psychometrics course for the third semester, as well as their relative importance and the number of sessions allocated to each. This aimed to develop the psychometric concepts test as a data collection tool for this study.
Content analysis sample:
The five main topics scheduled for teaching in the third semester of the psychometrics course were selected, considered as the main topics:
Development of the psychometrics movement
The concept of psychometrics
The importance and levels of psychometrics
Methods of measuring academic achievement
Psychometric properties of school tests
Category of content analysis:
According to Roshdy (2004), the category of analysis refers to “the main or secondary elements in which the units of analysis are placed—whether they are words, topics, values, or others—on which each attribute of the content is classified.” In light of the objective of this study, the category of analysis used was the scientific concept in this course, defined here as: the mental image formed by the student through abstracting common characteristics linked to the terms in the psychometrics course (words or phrases), consisting of the name and its verbal meaning.
Content analysis criteria:
To achieve an accurate analysis, the following guidelines were applied:
The analysis covered the five scheduled topics in the psychometrics course during the third semester, taught to university students in Educational Sciences.
The analysis was carried out within the scope of the course’s scientific content as prescribed in the 2016 canevas by the Ministry of Higher Education and Scientific Research, and as found in psychometrics textbooks available in the faculty library, referenced in the model. Two additional textbooks not mentioned in the canevas were also used: Fundamentals of Psychometrics and Instrument Design (Maameria, 2012) and Measurement in Psychology and Education (Bou Salem, 2014).
Content analysis procedures:
Identify the content of the main topics included in the course program under study.
Identify the content of the subtopics for each main topic and review them thoroughly.
Determine the number of sessions allocated to each main topic and its relative importance: This was based on the psychometrics course program (canevas), setting the number of sessions at 12 out of the programmed 14–16 weeks in the semester, taking into account the actual start of classes at the beginning of the academic year.
Validity of the content analysis:
To ensure more valid tools, the validity of the content analysis was verified by presenting the analyzed content to a panel of 13 professors specialized in psychometrics or who had previously taught the course, asking for their feedback on the following points:
Agreement with the proposed subtopics for each main topic.
Suggesting more appropriate subtopics if they disagreed.
Adequacy of the number of subtopics for each main topic.
Adequacy of the number of sessions allocated to each topic and their relative importance.
They were also asked to write any additional comments or suggestions for modifications.
Key feedback from the reviewers included:
Adding one more session, raising the total from 12 to 13 sessions, with the additional session allocated to the first main topic, Development of the psychometrics movement.
Adding two subtopics to the fifth main topic, Psychometric properties of educational tests:
Methods of calculating validity
Item analysis
Some reviewers suggested additional subtopics, but upon revisiting the canevas (the basis of the analysis), these were found to belong to fourth-semester topics and were therefore not added.
Results of the content analysis:
Based on the proposed subtopics, the assigned sessions for each, and the feedback from the reviewers, the main and subtopics scheduled for the psychometrics course in the third semester were finalized as shown in the following table:
Table (01): Main and Subtopics and the Number of Scheduled Sessions for Each Topic
No.
Main Topics
Subtopics
Number of Sessions
01
Development of the Psychometrics Movement
Development of the Psychometrics Movement (during the 18th and 19th centuries)
02
Development of the Psychometrics Movement (during the 20th century)
02
Concept of Psychometrics
Definition and Components of Measurement
02
Its Characteristics and Assumptions
03
Importance and Levels of Psychometrics
Purposes and Fields of Psychometrics
02
Levels of Psychometrics
04
Methods of Measuring Academic Achievement
Definition of the Achievement Test
02
Conditions and Types of Achievement Tests
05
Psychometric Properties of Psychological Tests
Definition of Validity, Its Characteristics, and Types
05
Factors Affecting Validity and Methods of Calculating It
Definition of Reliability and Methods of Calculating It
Factors Affecting Reliability
Item Analysis
Defining the Specific Learning Objectives:
Moshri (2018) indicated that general objectives should be translated into clear, well-formulated operational behavioral objectives, which help in preparing an achievement test that directly measures learning outcomes. Defining objectives facilitates the construction of a test that is representative of these objectives and their levels. Accordingly, we formulated a set of operational behavioral objectives for the third-semester psychometrics course.
Designing the Table of Specifications:
After analyzing the content of the course and identifying the learning objectives for each of the topics mentioned, we constructed a table of specifications following the steps outlined by Moshri (2018), as follows:
Identify the topics of the course.
Determine the relative weight of the course topics: This was done by calculating the relative weight of each topic’s importance through dividing the number of sessions allocated to each topic by the total number of sessions for the course as a whole, then multiplying the result by 100.
Determine the relative weight of the course objectives: This was done after identifying the learning objectives of the course topics according to the six cognitive levels mentioned by Bloom.
The relative weight of the importance of each topic’s objectives was calculated by dividing the number of objectives for each topic by the total number of course objectives, then multiplying the result by 100.
Determine the relative weight of behavioral objectives at different levels by dividing the number of objectives at a given level by the total number of course objectives, then multiplying the result by 100.
Determine the number of questions: The number of questions for each topic at each level of objectives was calculated by multiplying the total number of questions by the relative weight of the topic’s importance and the relative weight of the topic’s objectives.
Table (02): Table of Specifications for the Psychometrics Test
Levels of Objectives
Relative Weight of Topics
Remember19 %
Understand 23%
Apply 12%
Analyze 25 %
Synthesize 2 %
Evaluate 19 %
Total 100 %
Relative Weight of Topic 1 (15 %)
Number of Questions
1
1
0
1
0
1
4
Relative Weight of Topic 2 (15 %)
Number of Questions
1
1
0
1
0
1
4
Relative Weight of Topic 3 (15 %)
Number of Questions
1
1
0
1
0
1
4
Relative Weight of Topic 4 (15 %)
Number of Questions
1
1
0
1
0
1
4
Relative Weight of Topic 5 (40%)
Number of Questions
2
2
1
2
0
2
9
Total 100 %
Total Number
6
6
1
6
0
6
25
Stage Two: Formulating the Questions and Instructions
Formulating the Questions:
After constructing the above table of specifications, we proceeded to formulate the questions for the concepts test, basing them on the objective test format as the foundation for drafting the test items. This type of test “includes specific answers, thus reducing the chances of subjective judgment, and is characterized by its comprehensive coverage of the prescribed syllabus and ease of scoring” (Majeed, 2007, p. 223).
This type of test has various formats, and we chose the multiple-choice format, “which is used to measure different types of knowledge such as terms, facts, concepts, and methods. It is also suitable for measuring understanding and complex learning processes, as it can assess all cognitive objectives from knowledge, understanding, application, analysis, synthesis, to evaluation” (Majeed, 2007, p. 227).
Therefore, the questions for the present test were prepared in the form of multiple-choice items, with each question having four answer options, one of which represented the correct answer. The total number of questions was 25, distributed across five main topics (dimensions) as shown in the following table:
Table (03): Number and Serial Numbers of the Questions Belonging to Each Dimension of the Test
No.
Topic (Dimension)
Number of Questions
Item Numbers Belonging to It
01
Development of the Psychometrics Movement
03
1، 4، 8، 10
02
Concept of Psychometrics
04
2، 5، 6، 7
03
Importance and Levels of Psychometrics
03
9، 15، 17، 18
04
Methods of Measuring Academic Achievement
04
11، 12، 13، 14
05
Psychometric Properties of Educational Tests
10
3، 16، 19، 20، 21، 22، 23، 24، 25
Total
25
Setting the Instructions:
After determining and drafting the test questions, we prepared the test instructions, which clarified the purpose of the test and explained how to answer it, emphasizing that the information would be used solely for scientific research purposes. These instructions included:
General information about the student: e.g., gender.
Specific information about the test: number of questions, number of options.
Instructions for answering: marking (×) in front of the correct option.
Stage Three: Preparing the Test for Administration:
After completing the above-mentioned steps for constructing the psychometric concepts acquisition test, we reviewed the questions and answer options in the proposed initial version of the test to avoid any ambiguity, overlap, or redundancy in the questions.
Psychometric Properties of the Test:
In this section, we address the methods used to verify the validity and reliability of the test and to analyze its items.
Test Validity:
To verify the validity of the psychometric concepts acquisition test developed in this study, we used two types of validity: face validity and internal consistency validity.
Face Validity:
After preparing the initial version of the test, it was presented to a panel of 13 reviewers professors specialized in psychometrics, professors who had taught the course, and university teaching experts in order to obtain their opinions and suggestions regarding:
Approval of the extracted concepts and their adequacy.
Approval of the appropriateness of the test questions and their answer options.
Approval of the linguistic accuracy of the test questions.
Approval of the adequacy and appropriateness of the questions for the students.
Each reviewer was also asked to provide any comments or suggestions for modifying any question.
After collecting all the evaluation forms, the percentages of agreement on the questions and the suggested modifications for each item were calculated. All questions (items) were approved except for Question 13, which received an agreement rate of 66% from the reviewers and was therefore excluded from the final version of the test. As a result, the fourth dimension, Methods of Measuring Academic Achievement, consisted of three questions.
The remaining questions achieved agreement rates above 80% for approval, although some were recommended for rewording to suit the students’ level, clarifying ambiguities, or simplifying wording for questions considered somewhat complex.
The final version of the test thus contained 24 questions out of the original 25.
Internal Consistency Validity:
We attempted to extract internal consistency validity indicators by calculating Pearson correlation coefficients between each question and the total score of the dimension to which it belongs, between each question and the total test score, and between the total score of each dimension and the overall test score.
The results for the first dimension Development of the Psychometrics Movement—are presented in the following table:
Table (04): Correlation Coefficients of the First Dimension’s Questions
Dimension
Question Numbers
Correlation Coefficient of the Question with:
Dimension
Test
01
01
0.57**
0.63**
04
0.51**
0.56**
08
0.62**
0.71**
10
0.53**
0.54**
Correlation of the Dimension with the Test: 0.67**
** Correlation significance at the 0.01 significance level.
Based on the results shown in Table (04), it was found that the correlation coefficients between the scores of the questions and the total score of the dimension to which they belong ranged between 0.51 and 0.62, all of which were positive and statistically significant at the 0.01 level.
It was also found that the correlation coefficients between the scores of the questions and the total score of the concepts test ranged between 0.54 and 0.71, all of which were positive and statistically significant at the 0.01 level.
Furthermore, the results in the same table showed that the correlation coefficient between the total score of the first dimension and the total test score was 0.67, which is positive and statistically significant at the 0.01 level. From this, we concluded that construct validity was achieved for the dimension “Development of the Psychometrics Movement” in the psychometric concepts test.
Results of the Second Dimension – “Concept of Psychometrics”: The results obtained are presented in the following table:
Table (05): Correlation Coefficients of the Second Dimension’s Questions
Dimension
Question Numbers
Correlation Coefficient of the Question with:
Dimension
Test
02
02
0.61**
0.56**
05
0.53**
0.47*
06
0.51*
0.44*
07
0.60**
0.67**
Correlation of the Dimension with the Test: 0.54**
** Correlation significance at the 0.01 level* Correlation significance at the 0.05 level
Based on the results shown in Table (05), it was found that the correlation coefficients between the scores of the questions and the total score of the dimension to which they belong ranged between 0.51 and 0.61, all of which were positive and statistically significant at the 0.01 level, except for Question 06, which was significant at the 0.05 level.
It was also found that the correlation coefficients between the scores of the questions and the total score of the concepts test ranged between 0.44 and 0.67, all of which were positive and statistically significant, with Questions 02 and 07 significant at the 0.01 level, and Questions 05 and 06 significant at the 0.05 level.
Furthermore, the results in the same table showed that the correlation coefficient between the total score of the second dimension and the total test score was 0.54, which is positive and statistically significant at the 0.01 level. From this, we concluded that construct validity was achieved for the dimension “Concept of Psychometrics” in the psychometric concepts test.
Results of the Third Dimension – “Importance and Levels of Psychometrics”:
The results obtained are presented in the following table:
Table (06): Correlation Coefficients of the Third Dimension’s Question
Dimension
Question Numbers
Correlation Coefficient of the Question with:
Dimension
Test
03
09
0.63**
0.57**
14
0.56**
0.53**
16
0.62**
0.71**
17
0.60**
0.54**
Correlation of the Dimension with the Test 0.67**
** Correlation significance at the 0.01 level
Based on the results shown in Table (06), it was found that the correlation coefficients between the scores of the questions and the total score of the dimension to which they belong ranged between 0.56 and 0.63, all of which were positive and statistically significant at the 0.01 level.
It was also found that the correlation coefficients between the scores of the questions and the total score of the concepts test ranged between 0.53 and 0.71, all of which were positive and statistically significant at the 0.01 level.
Furthermore, the results in the same table showed that the correlation coefficient between the total score of the third dimension and the total test score was 0.67, which is positive and statistically significant at the 0.01 level. From this, we concluded that construct validity was achieved for the dimension “Importance and Levels of Psychometrics” in the psychometric concepts test.
Results of the Fourth Dimension – “Methods of Measuring Academic Achievement”:
The results obtained are presented in the following table:
Table (07): Correlation Coefficients of the Fourth Dimension’s Questions
Dimension
Question Numbers
Correlation Coefficient of the Question with the Dimension
Dimension
Test
04
11
0.57**
0.62**
12
0.41*
0.44*
13
0.71**
0.73**
Correlation of the Dimension with the Test: 0.71**
** Correlation significance at the 0.01 level* Correlation significance at the significance level(0.05).
Based on the results shown in Table (12), it was found that the correlation coefficients between the scores of the questions and the total score of the dimension to which they belong ranged between 0.41 and 0.71, all of which were positive and statistically significant at the 0.01 level, except for Question 12, which was positive and statistically significant at the 0.05 level.
The results in the same table also showed that the correlation coefficients between the scores of the questions and the total score of the concepts test ranged between 0.44 and 0.73, all of which were positive and statistically significant at the 0.01 level, except for Question 12, which was positive and statistically significant at the 0.05 level.
Furthermore, the results in the same table indicated that the correlation coefficient between the total score of the fourth dimension and the total test score was 0.71, which is positive and statistically significant at the 0.01 level. From this, we concluded that construct validity was achieved for the dimension “Methods of Measuring Academic Achievement” in the psychometric concepts test.
Results of the Fifth Dimension – “Psychometric Properties of Educational Tests”:
The results obtained are presented in the following table:
Table (08): Correlation Coefficients of the Fifth Dimension’s Questions
Dimension
Question Numbers
Correlation Coefficient of the Question with:
Dimension
Test
05
03
0.57**
0.64**
15
0.43*
0.57**
18
0.63**
0.72**
19
0.71**
0.73**
20
0.61**
0.56**
21
0.44*
0.65**
22
0.73**
0.57**
23
0.56**
0.72**
24
0.60**
0.63**
Correlation of the Dimension with the Test: 0.72**
** Correlation significance at the 0.01 level* Correlation significance at the 0.05 level
Based on the results shown in Table (08), it was found that the correlation coefficients between the scores of the questions and the total score of the dimension to which they belong ranged between 0.43 and 0.73, most of which were positive and statistically significant at the 0.01 level, except for Questions 15 and 21, which were positive and statistically significant at the 0.05 level.
The results in the same table also indicated that the correlation coefficients between the scores of the questions and the total score of the concepts test ranged between 0.56 and 0.73, all of which were positive and statistically significant at the 0.01 level.
Furthermore, the results in this table showed that the correlation coefficient between the total score of the fifth dimension and the total test score was 0.72, which is positive and statistically significant at the 0.01 level. From this, we concluded that construct validity was achieved for the dimension “Psychometric Properties of Educational Tests” in the psychometric concepts test.
By reviewing the results of the validity of the psychometric concepts acquisition test developed in this study using the two methods face validity and internal consistency validity (construct validity) it became clear that this test has acceptable validity indicators consistent with the characteristics of a good test, after being administered to a sample of students.
Test Reliability:
To verify the reliability of the concepts test developed in this research, reliability coefficients were calculated using two methods: the split-half method and internal consistency through Cronbach’s alpha coefficient for the test scores. The results for each method are presented below:
Test reliability using the split-half method:
This method relies on splitting the test into two equivalent halves after administering it to the same group. It has several formulas for calculating the reliability coefficient. In this study, we used the split-half reliability adjusted by the Spearman-Brown and Gettman formulas. The results were as follows:
Table (09): Test Reliability Coefficient Using the Split-Half Method
Formula
Reliability Coefficient Value
Spearman-Brown
0.67
Gettman
0.67
Based on Table (14), it was found that the reliability coefficient obtained using the Spearman-Brown formula was 0.67, and likewise, the Gettman coefficient was also 0.67. This reliability coefficient is considered acceptable.
Test Reliability Indicator Using Internal Consistency (Cronbach’s Alpha Coefficient):
The results obtained in this regard are presented in the following table:
Table (10): Reliability Coefficients of the Dimensions of the Concepts Test and the Overall Test Using Cronbach’s Alpha Coefficient
No
Dimensions
Coefficient (α)
1
Development of the Psychometrics Movement
0.86
2
Concept of Psychometrics
0.73
3
Importance and Levels of Psychometrics
0.51
4
Methods of Measuring Academic Achievement
0.70
5
Psychometric Properties of School Tests
0.62
The Scale as a Whole
0.76
Based on the results in the previous table, it was found that the reliability coefficients of the dimensions of the concepts test using Cronbach’s alpha ranged between 0.51 and 0.86, with the highest value (0.86) recorded for the first dimension and the lowest value (0.51) recorded for the third dimension. The reliability coefficient for the entire test, calculated using Cronbach’s alpha, was 0.76, which is considered acceptable.
From the results presented regarding the reliability indicators of the concepts test after its administration to a sample of students, we conclude that this test possesses an acceptable reliability coefficient, which we interpret as evidence that the test has the characteristics of a good test.
Research Conclusion:
In this study, we aimed to develop a test to measure the extent to which university students have acquired psychometric concepts. To achieve this, a sample of 156 male and female students from the Department of Psychology and Educational Sciences at the University of El Oued was selected using a simple random sampling method. The descriptive method was employed, along with a content analysis form for the psychometrics course, based on the 2016 canevas approved by the Algerian Ministry of Higher Education and Scientific Research, as well as information from selected psychometrics textbooks. After conducting the analysis, preparing the initial version of the test, administering it to the sample, obtaining students’ scores, and calculating the validity and reliability coefficients, the results indicated good validity and reliability indicators for the psychometric concepts test, consistent with the characteristics of a good test.
Based on these results, we propose the following:
Emphasizing the importance of psychometric concepts and the need to teach and instill them in learners.
Utilizing the psychometric concepts acquisition test developed in this study in further research and studies.
Developing additional psychological tests and measurements to contribute to enriching the psychological literature.
REFERENCES:
Abu Hatab, Fouad, & Sadiq, Amal. (1996). Educational Psychology. Cairo: The Anglo Egyptian Bookshop.
Badawi, Ramadan Massaad. (2016). Developing Mathematical Concepts and Skills for Preschool Children. Dar Al-Fikr: Jordan.
Boutros, Hafez Boutros. (2015). Developing Scientific and Mathematical Concepts for Kindergarten Children. Dar Al-Maseera for Publishing, Distribution, and Printing: Jordan.
Bou Salem, Abdelaziz. (2014). Measurement in Psychology and Education. 1st ed. Algeria: Publications of the Laboratory of Measurement and Psychological Studies, Dar Kortoba for Publishing and Distribution.
Tyler, Leona. (1988). Tests and Measurements. (Trans. Saad Abdel Rahman). Library of the Fundamentals of Modern Psychology, Al-Israa Publishing and Distribution: Cairo, Egypt.
Roshdy, Ahmed Taima. (2004). Content Analysis in the Humanities. Egypt: Dar Al-Fikr Al-Arabi for Printing and Publishing.
Al-Saadi, Youssef Faleh Mohammed. (2009). The Effect of Using Daniel and Klusmeyer’s Educational Models on Acquiring Biological Concepts and Attitudes towards the Subject among Middle School Students. Journal of Basic Education College, (56), 785–812.
Al-Ta’i, Fadel Khalil Ibrahim, & Al-Salifani, Sattar Jabbar Haji. (2014). The Effectiveness of an Instructional Design According to the Gerlach & Ely Model in Acquiring Temporal Concepts among Eleventh Grade History Students and Developing Their Habits of Mind and Historical Empathy. International Specialized Educational Journal, 3(4), 122–143.
Aram, Mervat Suleiman. (2012). The Effect of Using the KWL Strategy in Acquiring Concepts and Critical Thinking Skills in Science among Seventh Grade Female Students. Unpublished Master’s Thesis, Islamic University of Gaza: Palestine.
Majeed, Sawsan Shaker. (2007). Fundamentals of Developing Psychological and Educational Tests and Measurements. 1st ed., Hashemite Kingdom of Jordan: De Bono for Printing, Publishing, and Distribution.
Moshri, Solaf. (2018). Lectures on the Evaluation and Preparation of Competency Frameworks. Pedagogical Support Unit for Newly Recruited Teachers at the University of El Oued – Algeria.
Maameria, Bachir. (2012). Fundamentals of Psychometrics and Instrument Design. Dar Al-Khaldounia for Publishing and Distribution: Algeria.
Malham, Sami. (2002). Research Methods in Education and Psychology. Amman: Dar Al-Maseera.
Mansour, Mostafa. (2014). The Importance of Scientific Concepts in Science Teaching and the Difficulties in Learning Them.
Nazzal, Nasser Khazal. (2012). The Effect of the Frayer Model on Acquiring Measurement and Evaluation Concepts among Female Students of the Teachers’ Training Institute. Journal of Basic Education College, 20(83), 491–524.
Al-Nimr, Essam. (2013). Measurement and Evaluation in Special Education. Jordan: Dar Al-Yazouri Scientific for Publishing and Distribution.
Anastasi, Susana. A. (1997). Psychological Testing. New York, USA: Prentice Hall.