Criteria for assessing the competencies of university students. Methods for assessing the competencies of students of higher professional education
Description: Student performance assessment is a measure by which the degree to which each student achieves established desired educational outcomes is determined. Typically, such assessments are carried out by teachers within the disciplines they teach. When effectively assessing student performance, a variety of methods are used with a focus on the desired learning outcomes: disciplinary knowledge, personal, interpersonal competencies, the ability to create products and systems (see Standard 2). These methods include written and oral exams and test work, control sections, drawing up progress charts, keeping journals and portfolios for each student, self-control and students’ opinions about the classes being conducted.
Rationality: If we prioritize the personal, interpersonal competencies of students, their ability to create products and systems, if we establish them as indicators of the effectiveness of education and taken into account when drawing up curricula and educational assignments, then we need to work out effective methods assessment of these skills. It is necessary to develop your own assessment criteria for each of the designated educational outcomes. For example, the effectiveness of mastering disciplinary knowledge can be assessed by conducting oral and written examinations and verification work, but the ability to design and create products and systems is better assessed when performing practical work. The use of a variety of methods for assessing student performance helps to obtain reliable and complete information about student performance. Thus, the degree to which each student achieves the desired learning outcomes will be determined with maximum accuracy.
Data:
· assessment methods directly depend on the established CDIO learning outcomes;
· successful application of selected assessment methods;
· a high percentage of teachers using appropriate assessment methods;
· determination of the degree to which each student achieves the desired learning outcome, based on reliable and complete data.
Standard 12 – CDIO Program Evaluation
A system by which the entire program is assessed according to the listed twelve standards for students and teachers
and other key participants for the purpose of continuous improvement of the educational process.
Description: Program evaluation refers to the compliance of the entire program with established success indicators. The assessment must be carried out in accordance with the twelve approved CDIO standards. Collecting statistical data on the success of the program can be done by assessing the success of an individual course, obtaining advice from members of the teaching staff, conducting surveys before and after the program, analyzing reports from external auditors, and conducting surveys of graduates and employers over time, after completion of training. This information may be collected regularly by faculty, students, program administrators, alumni, or any other key stakeholders. All these statistics together make it possible to make an overall assessment of the program and contribute to its further improvement and development.
Rationality: The main objective of conducting a program evaluation is to evaluate its effectiveness and the extent to which it has achieved its goals. Statistical evaluation data collected to produce a global assessment is also necessary for continuous program improvement. For example, if at the end of the program the majority of students believe that they were unable to achieve some of the desired results, then the program can be revised, the reasons why the results were not achieved are identified and eliminated. In addition, many accreditation and auditing agencies often require that program success statistics be collected systematically.
A distinctive feature of modern professional education is personality-oriented training aimed at developing mobile, competitive, high-quality specialists characterized by responsibility, creative initiative, and the ability to take constructive and competent actions in professional activity. Focusing on this result requires a special system for assessing students’ general and professional competencies.
The purpose of the study is to study the methodological approaches of domestic teachers of higher educational institutions to assessing the development of students’ competencies in the process of studying the disciplines they teach.
Research objectives:
Consider the essence of the concept of “competence” of students and the mechanism of its formation;
To study the dependence of the level of formation of professional competence among students on the use of various teaching technologies and a complex of organizational and pedagogical conditions in teaching methods;
Consider methodological approaches to monitoring the development of competencies and its documentation support;
Determine the stages of the process of assessing the development of student competencies
Doctor pedagogical sciences A.V. Khutorskoy offers the following definition of competence: a set of interrelated personality qualities (knowledge, abilities, skills, methods of activity), specified in relation to a certain range of objects and processes, and necessary for high-quality productive activity in relation to them.
Visual experience in implementing the organizational basis for ensuring the process of developing professional competencies of graduates can be traced by analyzing the work on the implementation of the project “Promotion of raising the level of financial literacy population and development of financial education in Russian Federation" Currently, the Kaliningrad region is a pilot region for the implementation of the project. The government of the Kaliningrad region has adopted a target program “Increasing the level of financial literacy of residents of the Kaliningrad region in 2011-2016.” One of the main tasks is: organizing a system of financial education and awareness that promotes the transfer of knowledge and skills of financial literacy to all categories of the Russian population.
To assess the formation of experience in transmitting financial and economic knowledge to various target audiences, a number of activities were developed to study the level of professional competencies of students. The work consisted of 2 stages. At the first stage, a survey of students was conducted. At the second stage, a round table was held with the involvement of specialists from the partner bank of the educational institution. Teachers of economic and legal disciplines and educational institutions actively took part in the work of the round table.
When developing and holding the Round Table, the organizers needed to plan the work in advance and carry out preparatory work. Students prepared reports on current topics. By summarizing opinions, the conversation participants analyzed theses and antitheses. We solved practical situations that anyone can encounter in life. A representative of the bank assessed the development of professional competencies of students and teachers.
A study devoted to assessing the dependence of the level of formation of professional competence among students on its use in the methodology economic training accounting and economic disciplines of a set of various teaching technologies and a complex of organizational and pedagogical conditions, conducted at the Department of Management Accounting and Control, Faculty of Finance and Economics, Orenburg State University.
The experimental work involved full-time students majoring in “Finance and Credit” and teachers from the Department of Management Accounting and Control, Faculty of Finance and Economics, Orenburg State University. The study presents a quantitative assessment of indicators of the level of effectiveness of the formation of professional competence, determined through coefficients based on methods of mathematical statistics.
A number of experiments were carried out, namely:
Conducting active lectures using tools information technologies and Internet resources, which showed that it has a lesser impact on the development of professional competence in students;
Experimental testing of methods of economic teaching in accounting and analytical disciplines, which include a set of new educational technologies in the context of a competency-based approach, ensuring that students are placed in the position of subjects of activity. This experiment showed: students, to a greater extent than in the first two cases, are focused on the successful implementation of the economic knowledge acquired during the learning process in their future professional work.
This approach to assessing the effectiveness of developing professional competence among economic students in the process of studying accounting and analytical disciplines allows us to create such a model of economic training, and such professionally oriented didactic support for accounting and analytical disciplines that ensure the effective formation and development of professionally significant qualities of the future economist.
The development of competencies presupposes the active participation of the student in the educational process. It is appropriate to emphasize here that a student cannot be taught, he can only learn on his own. Teaching disciplines should be in a “case-study” format, where students acquire skills in teamwork, decision-making, evaluation and presentation of results. In addition to mastering disciplines, to assess the development of competencies, it is necessary to take into account all types of practices and extracurricular activities.
Melnikova S.V. proposes to develop and approve the following documentation for the successful development of competencies in an educational institution:
1. Regulations on monitoring the development of competencies.
2. Program for the formation of OK and PC in all specialties.
3. List of competencies for mastering an academic discipline, interdisciplinary course and professional module.
4. Individual map of monitoring the development of student competencies.
The program for the formation of OC and PC is necessary to coordinate all teachers involved in the formation of student competencies.
The program should determine disciplines, types of practices, extracurricular activities that contribute to the development of competencies, frequency of information collection, approximate dates of milestones, those responsible for collecting, summarizing and storing information, recording the results in an individual competencies development monitoring card, assessment criteria, assessment technologies, maturity indicators competencies. To monitor progress in an academic discipline, interdisciplinary course, professional module, academic or production practice The teacher fills out the competency sheet.
Competency sheet
Academic year ______________, semester _______________________
Speciality ___________________________________________
Course ______________, group ______________________________
Discipline _____________________________________________
Last name and initials of the teacher _________________________________
Surnames and initials of experts______________________________
Date of competency assessment _______________________
In the “Rating” column you can use the following notations:
0 – according to the selected criterion, competence is not demonstrated
1 – competence is partially demonstrated;
2 – competence is fully demonstrated.
The given points will allow you to judge the basic, advanced or creative level of competencies. The conclusion about the level of development of OK and PC is entered into the individual personal card of the student’s competence development.
Competency development monitoring map
Full name________________________ Specialty ____________________
Date of admission to educational institution____________________
The organizational and legal conditions created in this way will improve the professionalism of assessing the maturity of the competencies of vocational education students, and therefore will make the educational process more effective.
Assessment of the process of developing competencies in the practice of Volgograd State Social and Pedagogical University was implemented using two types of competency cards: maps of developed competencies and a student competency profile.
A map of the competence being formed is designed for each competence being formed. The “Knowledge” and “Skills” columns of the card display the points received by students for completing tests and tasks that allow them to evaluate knowledge and skills. The received points are summed up and translated into a qualitative assessment of the level of the cognitive and operational basis of competence (high, sufficient, minimal, below acceptable). In the “Attitude” column, the teacher, based on the results of observation of activities and based on analysis creative works students gives a qualitative assessment of the level of manifestation of the motivational-value component of students’ educational achievements. In the column “Level of Competence Mastery,” the teacher sets an integral assessment, which depends on the indicators of the three components of educational achievements and the analysis of the student’s quasi-professional activities.
Map of the competence being formed
|
1. Cognitive basis of competence (knowledge) |
2. Operational basis of competence (skills) |
3. Attitude to the process, content and result of the activity (attitude) |
Level of mastery of competence |
||||||||||
|
Activity* |
Group work, cooperation* |
Motivation and attitude* |
|||||||||||
* – approximate observable qualities are given.
A – “high level”, B – “ average level", C – “minimum level”, D – “below the minimum standard”.
If a student was not diagnosed or did not pass the work due to some circumstances, he is given a mark “F”. This does not mean that the level of his knowledge, skills, and developed competencies is below the standard, but means that this student needs additional diagnostics.
To record the level of competencies demonstrated in the work, an assessment card of demonstrated competencies was designed - “Student Competencies Profile”. This assessment is also cumulative in nature and changes throughout the entire period of mastering the discipline. The assignment of marks A-D or F to the competency profile is carried out by the teacher based on observation of students’ activities in the classroom, studying products educational activities.
Map “Student Competency Profile”
|
Competency indices from the Federal State Educational Standard for Higher Professional Education in the direction of “Pedagogical Education” |
|||||||||||
In the future, this “competency profile” is filled out by each teacher and compiled into a single database. The final assessment of the formation of one or another general cultural or general professional competence would be determined by the method of expert assessments based on a set of assessments of the “individual competence profile.”
For students at the Volgograd State Social and Pedagogical University, a version of the “portfolio of competencies” has been developed, which is built like a self-assessment diary. For each competency, the student fills out the sections “I Know,” “I Can,” and “I Have Qualities.” Next, the “Certificate” section is filled out, where the student either submits work or describes where, in what classes or during teaching practice they demonstrated this competence. The last section to be filled in is the “Level of Formation” section, where the student, assessing his achievements in this area comprehensively, makes a general assessment from “complete mastery” to “not proficient.” It is possible to carry out such an assessment procedure based on the results of studying each topic, but research has shown that it is more advisable to complete the portfolio after studying the entire psychological and pedagogical cycle.
In order to create an interactive platform for a broad discussion of new approaches to the development of diagnostic technology for external independent assessment of competencies along the entire path of mastering the content of educational programs at a university, it was developed and implemented innovative project"Federal Internet Exam: Competency-Based Approach." The project is focused on conducting an external independent assessment of student learning outcomes within the framework of the requirements of the Federal State Educational Standard. The proposed technology for conducting an Internet exam (on-line mode) was tested in December 2011 - January 2012. 1,577 educational programs from 164 universities and 37 colleges from 64 regions of Russia and the Republic of Kazakhstan took part in the testing; the number of testing sessions was 45,842.
Based on the requirements of the Federal State Educational Standard, the new model of the pedagogical meter in the project is presented in three interconnected blocks. First block of tasks (thematic content) checks the student’s level of mastery of the discipline material at the “know” level. This block contains tasks in which the method of solution learned by the student while studying the discipline is obvious. Second block tasks (modular content) assess the degree of mastery of the discipline material at the “know” and “be able” levels. This block is represented by tasks in which there is no explicit indication of the method of execution, and the student independently chooses one of the studied methods to solve them. The tasks in this block allow you to evaluate not only knowledge in the discipline, but also the ability to use it when solving standard (typical) problems. Third block (case filling) evaluates the mastery of the discipline at the level of “know”, “be able”, “possess”. It is represented by case meters, the content of which involves the use of a set of skills necessary for independently constructing a method for solving a task. Students’ completion of this kind of non-standard practice-oriented tasks indicates the degree of influence of the learning process on the formation of students’ general cultural and professional competencies in accordance with the requirements of the Federal State Educational Standard.
The developed criteria for the implementation of pedagogical measuring materials made it possible to draw conclusions about the level of mastery of the disciplines of each individual student and give him recommendations for further successful progress in his studies. It is proposed to distinguish five levels of mastering disciplines: critical, reproductive, basic, productive and high. The critical level characterizes the insufficient level of knowledge in a given discipline to continue its study. The reproductive level shows knowledge of basic concepts, identifies them and reproduces them. A basic level of– this is the student’s level of mastery in performing a sequence of actions, the student reproduces previously learned information and decides typical tasks, performing the necessary actions independently. The productive level indicates the presence of a system of cumulative integrated knowledge that allows the student to have a flexible approach to solving complex tasks, justify and prove the correctness of the chosen method of solving them. A high level of mastery of disciplines indicates the student’s ability to think creatively (outside the box), to use the existing system of integrated knowledge and skills in a new non-standard situation. Student showing this highest degree mastering disciplines (at this stage of training) is capable of generalizing and transferring established patterns to new phenomena.
In the future, the Federal Internet Exam: a competency-based approach will allow us to solve another very important task - to implement diagnostic technology for external assessment of competencies along the entire path of mastering the content of university study programs.
Thus, research shows that the process of assessing the development of students’ competencies is multi-stage, including assessment of the level of competence by the teacher, student self-assessment, discussion of the results and a set of measures to eliminate deficiencies. In addition, it is necessary to observe the frequency of assessing the level of development of student competencies (from a first-year student starting to master OOP to a university graduate) at each stage of training and upon completion of studying the OOP module.
conference.osu.ru › assets/files/conf_reports/Karmanova E. V. Technology for assessing the level of development of competencies of students studying in the system distance learning university // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 1
vyyyyyyyyyyyifART15417UDC 378.146
Karmanova Ekaterina Vladimirovna,
Candidate of Pedagogical Sciences, Associate Professor of the Department of Business Informatics and Information Technologies FSBEI HPE ©Magnitogorsk State Technical University named after. G. I. Nosova, Magnitogorsk [email protected]
Technology for assessing the level of development of competencies of students studying in the university’s distance learning system
Abstract. The article discusses the problem of organizing an assessment of the level of development of competencies of students studying using distance educational technologies. The author substantiates a mathematical model for assessing the level of development of students' competencies, which takes into account the structure of existing electronic courses in the distance learning system of Magnitogorsk State Technical University. The main provisions of the technology for assessing the level of development of students' competencies are revealed. Key words: competency-based approach, competency assessment, mathematical model, technology, distance learning system. Section: (01) pedagogy; history of pedagogy and education; theory and methods of teaching and education (by subject areas).
With the introduction of a competency-based approach to the system Russian education Pre-educational institutions have faced a number of tasks that cannot be solved using traditional methods; one of these tasks includes the need to organize an assessment of the level of development of students’ competencies. Today, each university independently decides what will be the methodology for the formation and use of assessment funds to assess the level of development of students’ competencies. This problem also extends to the organization of the learning process using distance learning technologies, since this form of learning is legitimized at the level of the Law on Education of the Russian Federation, and therefore must comply with other requirements of Russian legislation in the field of education. An analysis of the literature showed that universities do not have a generally accepted technology that allows them to track the process of forming a separate competency; in addition, existing methods for assessing competencies are quite cumbersome and often do not take into account the specifics of the formation of the competencies themselves within the framework of the university’s distance learning system. Search possible ways solving the identified problem and determined the choice of the goal of our research: the need to develop a technology for assessing the level of development of competencies of students studying using the distance learning system (hereinafter referred to as DLS).
When developing the technology for assessing competencies, we relied on the following provisions: since disciplines are the main form of educational activity (not a single university has introduced changes), and it is the level of students’ mastery of academic disciplines that is assessed by teachers, it is obvious that by assessing the level of training in disciplines it is possible to ultimately correctly to establish the level of formation of competencies, in addition, in this case it is more convenient to adapt previously accumulated experience and use adapted diagnostic techniques Karmanova E. V. Technology for assessing the level of formation of competencies of students studying in the university’s distance learning system // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 2
within the framework of a competency-based approach. Also, within the framework of our research, the task was to develop a technology that would allow, based on the existing structure of electronic courses, to “painlessly” introduce an assessment of the level of development of students’ competencies for a university’s LMS. For a clear understanding of the technology proposed below, we will describe the existing structure of electronic courses in the DMS of Magnitogorsk State technical university. In the LMS, the electronic course is a set of modules, each of which contains theoretical material, a practical task and a test for self-control. The electronic course necessarily ends with a final test for the entire discipline being studied (Fig. 1).
Rice. 1. Elements of an electronic course in the LMS
Let us describe the assessment methodology. When developing a mathematical model for assessing the levels of development of competencies, we used the points received by students based on the results of performing diagnostic materials for a separate competency, and the result of a control event (test or exam). Let us need to find out the level of development () of competence. This competence is formed through disciplines =(1,2,…,), where the discipline directly influences the formation of competence. The influence of a discipline on the formation of competence can be considered at the level of the number of modules (sections) that form a given competence and, accordingly, check the level of its development. As a rule, the number of such sections for an electronic course is determined by teachers (authors of the discipline), who, in turn, are guided by the labor intensity of the discipline. In accordance with the discipline, an electronic course is created in the LMS, which, based on work program The discipline contains modules and necessarily ends with a final test. Since each discipline module in the electronic course contains a practical task and a test, these components will be diagnostic materials for assessing competence within a given discipline: (1≤≤2+1) (see Fig. 2). Let us explain this entry: at a minimum, in a separate electronic course, the selected competence must be formed by at least one module of the course and, therefore, tested by at least one diagnostic material (either a test or a practical task), the maximum number of diagnostic materials can be all practical tasks and tests for each module plus a final test. However, the final test may not necessarily test this competency, and therefore is not necessarily included in the block of diagnostic materials for this competency. The influence of the results obtained from tests and practical tasks is different, so we will separately enter our weight for each of them: Determination of weights for test materials and Karmanova E.V. Technology for assessing the level of competency formation of students studying in the university’s distance learning system // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 3
Fig. 2. Example of a set of diagnostic materials for a specific competency
The result is also influenced by the final score obtained as a result of the final control (test or exam):. Thus, the mathematical model for assessing the level of competence formation will be presented in the following form: = ∑ (∑∙ = 1+), = 1 where n is the number of disciplines that form k competence; t is the number of diagnostic materials of the i discipline that test k competence; weight separate diagnostic material (test or practical task) on the result of control in the i-th discipline; the score received by the student as a result of passing the r-th DM in the i-th discipline; the score received in an exam or test in the i-th discipline (this may also include a grade in course work, educational/industrial practice, final state certification). The result obtained can be compared with the maximum number of points that a student can receive based on the results of training, and presented as a percentage (70% work during the semester in the i-th discipline and 30% result of the examination test/test in the i discipline). For example, let the i-th discipline have ∑∙=550=1 points, this is 70% of the total score of the discipline, then the maximum value = 236 points. As a rule, the assessment of Karmanova E.V. Technology for assessing the level of development of competencies of students studying in the university’s distance learning system // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 4
on the exam is given on a five-point scale, therefore, to convert it into course points, you must either indicate in percentage the rules for converting grades obtained on the exam to course points (for example, ©2ª 0%; ©3ª 55%; ©4ª 80%; ©5ª 100%), or the teacher also evaluate the exam results according to the course scale, which is more the right decision, however, increases the complexity of assessment. The procedure for calculating the level of competence development gives the final result after studying all the disciplines that form the competence. However, in the process of developing competence, it is very important to monitor and predict its possible assessment at any stage of training with subsequent correction of the process of developing competence. In addition, according to the requirements of federal government educational standards assessment of the quality of training of students and graduates should include current, intermediate and final state certification, which means that the level of competence development should be assessed. Therefore, a procedure is proposed for calculating the characteristics of the level of competence in the process of its formation. The contribution of the discipline to the competence: =∑∙=1+. Accordingly, the maximum and minimum possible contribution of the discipline to the competence is calculated using the formulas: brmin is the minimum possible score for assessing the knowledge and skills of students. Then the maximum and minimum possible assessment of competence at the current time is calculated using the formulas:=∑=1,
=∑=1,
where k is the number of disciplines that form competence and are studied at this moment time. This assessment will be calculated in points that teachers assigned in the disciplines that form this competency. The current assessment of competence is calculated according to formula (7) and measured in points of the score rating system adopted at the university for assessing disciplines: = ∑. = 1 The current contribution of all disciplines studied at a given time to the formation of competence is found using the expression:
Competencies can be developed at various levels; as a rule, there are three levels of formation of a separate competency: threshold (initial), basic and advanced. To convert students' quantitative assessments into competency levels, it is enough to determine the intervals of each level. As a rule, E. V. Karmanova, Technology for assessing the level of competency formation of students studying in the university's distance learning system // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 5
In the provisions of the university's point rating system, such intervals are presented as percentages. Any assessment technology must answer primarily the following questions: 1. What will be assessed? 2.Who or what evaluates? 3.When should an assessment be carried out? 4.Where and how should the assessment results be recorded? 5. By what criteria to evaluate (what is the evaluation methodology)? Figure 3 shows the main provisions of the proposed technology for assessing the level of development of competencies of students studying in the university's distance learning system.
Rice. 3. Technology for assessing the level of development of students’ competencies in the university’s LMS
The proposed technology for assessing the level of development of competencies of students studying in the university’s educational and educational system can also be used for full-time students. However, it is worth noting that the procedure for assessing student competencies must be supported by the automated control system that currently exists in almost every university. Automation in this case is necessary because it is necessary to track a fairly large number of competencies (on average 40) that need to be developed for each graduate; in addition, as a rule, the battery of diagnostic materials for assessing competencies is also large in size, and processing it manually is labor-intensive.
Links to sources 1. Romanova E.P., Romanova M.V. About the concept of ©open Remote educationª in modern conditions training of students of classical universities // VIII International Conference ©Strategy of excellence in industry and masteryª: Materials: 3 volumes. T.II / administrators: T.S. Khokhlova, V.O. Khokhlov, Yu.A. Stupak. Dnipropetrovsk; Varna, 2012. P. 434436. Karmanova E. V. Technology for assessing the level of competency formation of students studying in the university’s distance learning system // Concept. 22015.2No. 12(December).2ART15417. 20.3 p.l. 2URL: http://ekoncept.ru/2015/15417.htm.2ISSN 2304120X. 6
2. Eliseev I.N. Mathematical models and software packages for automated assessment of learning outcomes using latent variables: dis. ... dra tech. Sciences: 05.13.18. Novocherkassk: SRSTU, 2013. 371 p. 3. Zakirova E.I. Information support for decision-making in the selection of students for a university master’s program based on a competency-based approach: dis. ...cand. tech. Sciences: 05.13.10.PNRPU: Tchaikovsky, 2014. 198 p.4.Permyakov O.E., Menkova S.V. Diagnostics of the formation of professional competencies. M.: FIRO, 2010. 114 p. 5.Pirskaya A.S. Methodology for assessing graduate competencies // Scientific and Technical Bulletin of the St. Petersburg State University of Information Technologies, Mechanics and Optics. 2012. No. 1(77). WITH. 124132.6.Sibikina I.V. Models and algorithms for the formation and assessment of competencies of a university graduate: dissertation... candidate of technical sciences. Sciences: 05.13.10. Astrakhan, 2012. 200 p.
Ekaterina Karmanova, Candidate of Pedagogic Sciences, Associate Professor at the chair of Business Informatics and Information Technologies, Nosov’s Magnitogorsk State Technical University, [email protected] for assessmentthe competenceslevel of students enrolled in university distance learning Abstract.The paper considers the problemofassessing the level of formationof students’organization competences using distance learning technologies. The author establishes thematic model for evaluating the level of formation of competences, taking into account the structure of the existing elearning courses in distance learning system of Nosov’sMagnitogorsk State Technical University. The basic provisions technologies assessing level of formation competences of students are revealed. Key words: competence approach, assessment of competence, mathematical model, technology, distance learning system. References 1. Romanova, E. P. & Romanova, M. V. (2012). “O ponjatii ‘otkrytoe distancionnoe obrazovanie’v sovremennyh uslovijah podgotovki studentsov klassicheskih universitetov”, VIII Mizhnarodna konferencija “Strategija jakosti u promislovosti i osviti”: Materiali: u 3 t.T. II / uporjadniki: T. S. Hohlova, V. O. Hohlov, Ju. A. Stupak, Dnipropetrovs "k, Varna, pp. 434436 (in Ukrainian). 2. Eliseev, I. N. (2013). Matematicheskie modeli i kompleksy programm dlja avtomatizirovannoj ocenki rezul" tatov obuchenija s ispol "zovaniem latentnyh peremennyh: dis. ... dra tehn. nauk: 05.13.18, JuRGTU, Novocherkassk, 371 p. (in Russian). 3. Zakirova, Je. I. (2014). . ... kand. tehn. nauk: 05.13.10, Chajkovskij, PNIPU, 198 p. (in Russian). 4. Permjakov, O. & Men "kova, S. V. (2010). Diagnostika formirovanija professional"nyh kompetencij, FIRO, Moscow, 114p. (in Russian). 5. Pirskaja, A. S. (2012). “Metodika ocenivanija kompetencij vypusknika”, Nauchnotehnicheskij vestnik SanktPeterburgskogo gosudarstvennogo universiteta informacionnyh tehnologij, mehaniki i opt iki, №1(77 ), pp. 124132 (in Russian). 6. Sibikina, I. V. (2012). (in Russian).
Gorev P. M., candidate of pedagogical sciences, editor-in-chief of the magazine ©Conceptª
1The article describes an approach to assessing and presenting the results of mastering basic professional educational programs within the framework of the competency-based paradigm. The proposed method for assessing the final generalized result of the development of competencies meets the interests of participants in the educational process. The method is based on a “model and profile of competencies” accepted in the professional environment and adapted to educational practice; disciplinary foundation for the formation of competencies at the university; scientific qualimetric formalization of educational results. The method is quite simple and universal. The form of presentation of intermediate results allows, at the stage of developing a university graduate competency model (model and profile of competencies) with the participation of the employer, to balance the volumes, list and sequence of disciplines and practices that develop both individual and group competences. It allows you to change the balance of competencies in the interests of the professional community of the industry, and is not limited by the number of competencies. The visibility of the final results will allow a third-party employer to see the level of development of competencies that are most significant for the graduate - a potential employee of the organization. The system-activity approach involves students’ participation in the process of self-formation and application of rating cumulative results of competencies development and their presentation in a portfolio.
learning outcome
competency model
competency profile
qualimetry
academic discipline
employer
educational institution
portfolio.
1. ECTS Users’ Guide Brussels: Directorate – General for Education and Culture. – 2005. [Electronic resource]. - URL: http://ec.europa.eu/education/programmes/socrates/ects/doc/ guide _en.pdf (access date: 10/08/2017).
2. Savelyev B.A. Model of assessment tools of the Federal State Educational Standard for Higher Professional Education [Electronic resource]. - URL: http://op-new.rguts.ru/templates/Default/images/sector/project_oop/questions/metod_rekomends/1.pdf (access date: 10/02/2017).
3. Kartushina E.N. Features of constructing a competency model in an organization // Socio-economic processes and phenomena. – 2012. – No. 7-8. – P. 60-64.
4. Guidelines on the development of basic professional educational programs and additional professional programs taking into account relevant professional standards [Electronic resource]. - URL: http://docs.cntd.ru/document/420264612 (access date: 10/02/2017).
5. Kozlova T.V. How to evaluate an employee's competence? [Electronic resource]. - URL: https://hrliga.com/index.php?module=profession&op=view&id=1447 (access date: 10/9/2017).
6. Tatur Yu.G. Competence-based approach to describing results and designing standards for higher professional education. – M.: Research Center for Problems of Quality of Training of Specialists, 2004. – P. 18.
7. Azgaldov G.G., Raikhman E.P. About qualimetry. M.: Standards Publishing House, 1973. 172 p.
The concept of “learning outcomes” as the most important component The ultimate goal of higher vocational education was introduced into educational terminology in 2005 during the development of the European Qualifications Framework. It is this phenomenon that is assigned a central system-forming role in the education system. The result of education expresses the interests of the individual and the state, ensuring their consistency; reflects the priorities of the socio-economic state and development of society, aims at the formation of adequate mechanisms for assessment, recognition of education and transparency for all participants in the educational process, improving its elements.
The ECTS User Guide defines learning outcomes as “statements of what a learner is expected to know, understand and/or be able to demonstrate (do) after completing a period of study.”
Modern model education, adapted to the needs of a post-industrial society, digital and information economy with dynamically changing technologies, substantively, and since the adoption of the Federal Law “On Education in the Russian Federation” of December 29, 2012 and formally, interpreted as a “competency-based paradigm”. Within the framework of this approach, when selecting and planning the educational outcomes of graduates of the higher education system, the foreground is not the volume, versatility and depth of “ready-made” acquired knowledge and algorithms for reproducing it according to samples (knowledge-oriented approach), but competencies. The emphasis in characterizing the results has shifted to the graduate’s ability to adapt to the professional community, the willingness to creatively solve educational and life problems, the ability to independently acquire and update knowledge and apply it in situations close to future professional activity, etc.
The strategic result of the competency-based paradigm is the formation of a model of a graduate as an organic unity of a set of competencies based on his acquired conscious knowledge, skills, methods of activity and constantly striving for self-realization, ready for self-education and improvement, implementing the principle of “lifelong learning”.
In relation to Russian educational practice, since the transition to the Federal State Educational Standard for Higher Professional Education, and then the Federal State Educational Standard for Higher Education, Federal State Educational Standard for Higher Education 3+ and Federal State Educational Standard for Higher Education 3++, the requirements for the educational results of a graduate, that is, mastering a bachelor’s (master’s) program, are established educational organization in the form of developed competencies.
The complexity and versatility of the phenomenon of “learning outcome” - competence, have led to a variety of directions for its study. A significant number of pedagogical studies have been noted in relation to the triad: “competencies as a result of education ↔ educational technologies as a way of their formation ↔ assessment tools as a tool for proving the achievement of the stated educational results.” IN the specified list, in our opinion, the logical final point is missing - “measurement (assessment) and recording of the personal result of education achieved by the graduate.”
A review of Russian sources on the development of models for measuring the integral educational results of university graduates revealed: 1) their limited number. Of greatest interest are the works of N.E. Bestuzheva, M.V. Bedilo, E.S. Dzhevitskaya, T.P. Kategorskaya, V.I. Nikolaeva, A.S. Platonova, M.N. Ryzhkova, I.V. Sibikina, Z.V. Yakimova and others, in some moments close to the problem under study; 2) the lack of a unified approach and universal criteria for the achievability of results, the difference in units of measurement of the result; 3) incompleteness of methods that provide a visual representation and quantitative indicator of the level of development of competencies of a university graduate. At the same time, the Federal State Educational Standard for Higher Education 3++ requires “to record the results of mastering a bachelor’s degree program,” and the learning results are determined and assessed by the university itself. Employers are interested in the availability of objective data for selecting employees from among young specialists based on competencies, and not in the academic assessment of a particular discipline. In our opinion, it is appropriate to recall the practice of the period of the industrial, planned Russian economy and the knowledge paradigm of education, when the criterion for ranking a graduate when assigning a job after graduation was the “arithmetic average of the diploma” as a final, formal, but objective result.
The purpose of the article is to substantiate the approach and methodology for calculating the outcome indicator for mastering the basic professional educational program, based on the competency model adopted in professional field and adapted for the education sector.
The scientific and methodological essence of the method under discussion is to present a qualitative target result in the form of a cognitive model of competencies and derive a formalized educational result on its basis, guided by the principles of qualimetry.
The method is considered in the form of an algorithm that consistently records objective scientific and pedagogical foundations, correlated with the substantive elements and provisions of the calculation methodology.
I. Orientation of educational results to the interests of the professional community; bringing together the practice of designing and assessing the competencies of university graduates with HR management tools used by recruitment agencies and organizations. Reasonable borrowing of best practices, methods and concepts from this area is quite natural and advisable within the framework of a single educational and professional space.
The starting point, the key concept, is the “competency model” - the complete set of competencies necessary for company employees to successfully achieve the organization’s strategic goals. In vocational education, it is transformed into a “graduate competency model” or a set of universal (UK), general professional (GPC) competencies in the field of training, approved by the Federal State Educational Standard for Higher Education, and professional (PC), legitimately called “profile”, selected by the university from the relevant professional standards (PS ) .
II. A holistic understanding of both the employee’s competency model and the graduate’s “competency model” is that it is an integrated result in which not a single competency can be excluded, all competencies are interrelated, since one discipline forms several competencies. Developers of HR competency models note its multi-level structure. It is necessary to clarify that “multi-level” should not be understood “in the concept of a physical model, which, according to Yu.G. Tatura, V.I. Bidenko, represents a “layer cake” (specialist education), in which there is a “layer of dough” (knowledge), a “layer of jam” (skills), a “layer of glaze” (value orientations), etc. If you cut it into pieces (competence), then each of them will repeat in miniature the structure of the “cake.” In our opinion, this is hierarchical structure, which decomposes competence into basic elements - certain knowledge, abilities, skills, experience and other characteristics, which is absolutely consistent with the structure of professional standards (Fig. 1).
Rice. 1. Hierarchical decomposition of achieving the main goal of professional activity within the framework of a professional standard (fragment)
On the other hand, levels can also be considered as different degrees development of fundamental elements and, accordingly, the competencies formed by them.
III. Competency-based performance in education is built and implemented on the basis of the traditional (knowledge-based) approach. It is possible to evaluate only what is measured, which means not the competence itself, but the actual learning results that contribute to its formation. From the “competency model” of a graduate (Fig. 2) it follows that the results of mastering independent academic disciplines, subject areas and practices, interdisciplinary modules are fundamental, primary knowledge, abilities, skills, which at a higher level of the hierarchy are integrated into individual competencies, then into blocks and a comprehensive indicator of competencies.
Rice. 2. Hierarchical decomposition of the competency model as a result of mastering the main professional educational program (fragment)
The requirements of Federal State Educational Standard 3++, which provide that the university’s “indicators of achievement of competencies and learning outcomes in disciplines and practices must be correlated with each other,” are contained in the funds of assessment tools for the discipline. Knowing the intermediate assessments of students’ achievement of planned results in individual academic subjects included in the competence, we form an assessment of the development of competencies.
The multi-level nature of the model is also included in the multi-stage algorithm for assessing the level of development of competencies. On its basis, qualitative indicators of results are consistently formed, and then quantitative ones according to the same principle. Quantitative assessment of mastery of the discipline, that is, knowledge, skills and abilities, is carried out in absolute points () on a 100-point scale. Its use eliminates the uncertainty of academic grades of “pass” and “fail.” The resulting complex indicator of the formation of the entire set of competencies is also expressed in points with the maximum possible value of 100, which also corresponds to the logic of qualimetry.
IV. The significance of each competency and the block of homogeneous competencies in the structure of the model are not equal. In the professional sphere, this is determined by the position of the organization, in the Federal State Educational Standards of Higher Education - by profile (specialization), and the specific requirements for graduates of professional communities in the region. This property of competencies is expressed by an indicator - the weighting coefficient (), which allows you to use the concept of “competency profile” in the method, having previously specified it.
Competency profile - the structure of a competency model, taking into account the weight (significance) coefficients of a single competency or a group of homogeneous competencies (for example, QM, OPK and PC). The sum of all weight coefficients characterizes integrity and, based on point II, is 1.0 or 100. For the same competency model, the profiles will be different. The competency profile is most often drawn up in the form of a diagram that allows you to clearly illustrate the role of each of them.
Determining a weighting factor is essentially ranking competencies by importance. Knowledge is necessary for subsequent calculations: the level of competence development (), competency profile () and a comprehensive indicator () of graduate training. Ranking can be carried out using the expert method by involving experts from among teachers, employers, etc. using the method of pairwise comparison, sequential placements, etc. It is more rational to carry out this procedure without the personal participation of specialists, based on the information presented in educational and methodological documents (sections “Competencies” and “Competencies (2)” of the curriculum).
According to the conclusions of experts - developers of educational programs, the importance of competence in pedagogical process determined by: the number of disciplines and practices ( and ), their volume ( and ) in credit units or hours required for its formation. The form of control, the position of the discipline by semester, the number of didactic units and others (A.V. Onoprienko, I.V. Sibikina) turn out to be much less significant. Then the weighting coefficient of competence is calculated according to formula (1):
, (1)
where is the number of academic disciplines and practices that provide a single competency;
All academic disciplines and practices included in the basic professional educational program.
If a discipline provides several competencies, we assume that the volume of discipline hours for a single competency is proportional to the number of competencies it forms.
The consistency of the ranks and numerical values of the coefficients of significance of competencies of a particular profile, obtained by the heuristic method and calculated using the formula, was checked by calculating the Pearson correlation coefficient and comparing it with the critical value of the coefficient for the condition (31 variables by the number of competencies and statistical significance of 0.01). The calculated value of the Pearson criterion of 0.892 significantly exceeds the critical table value (52.19), which confirms the reliability of the calculations and the admissibility of our chosen approach to determining .
Figure 3 shows the competency profile for a “real” basic professional educational program.
Rice. 3. Competency profile
Developing a profile is a good way to decide on adjusting a graduate’s training in the direction of a specific competency or block. Thus, in the competency profile of a bachelor of commerce, the weight coefficient general cultural competencies is 26.83, clearly inferior to the OPK and PC blocks, which have indicators of 18.72 and 44.45. Taking into account recommendations that have been tested by time, practice, confirmed by science, in the competency profile of a bachelor of commerce, soft skills (flexible competencies) should be a priority: readiness to implement a set of communication skills, manifestation of social responsibility, speed of decision-making, which are mainly positioned in the OK and are formed by social and humanitarian disciplines. It can be concluded that the competency profile requires clarification.
The task of the stage is to determine the list and number of disciplines () and practices (), the mastery of which forms a single competence and their contribution ( and ) to this result. The logic and content of the formation of competencies is reflected in the “Competencies” section of the curriculum, in which the horizontal line is a competency, and the vertical column is a list educational subjects(practice), as well as the number of credits for mastering the discipline (practice).
Qualimetric assessment of a student’s educational results involves the calculation of several indicators, primarily focused on the employer, their comparison and decision-making.
1. The degree of development of a single competency is calculated as a weighted arithmetic mean score, taking into account the number of academic subjects and practices and the number of hours in the competency structure. This indicator is important for the student during intermediate control during the educational process.
According to the proposed assessment model, the degree of development of individual competence is calculated using formula (2):
, (2)
The weight coefficients of a discipline or practice in the competency structure are calculated using formulas (3, 4), and the sum of the weight coefficients of disciplines is equal to 1.0 or 100%:
, 
(3, 4)
2. Professionally oriented level of single competence (). The value of the indicator lies in the fact that it more accurately reflects the overall result of the competency-oriented educational process that is significant for the professional community. The level of competence development ( - educational educational result) is adjusted to the significance of the competence for a profile (specialization) or potential position in a particular organization ( - weight coefficient of the competence).
The indicator is calculated using previously calculated indicators and their values according to formula (5):
(5)
A visual representation of the level of development of individual competencies as an educational result and the possibility of comparing them for, for example, two graduates is provided by a radar diagram (Fig. 4a).
Rice. 4. The result of the development of competencies: a - educational (arithmetic mean weighted score); b - professionally oriented (score based on competency profile)
A comparison of the diagrams illustrates the contribution of the competency profile (Fig. 4b), the transformation of the educational result into a professionally oriented indicator. A high academic score (PC-5, PC-14) is not always relevant for a specific position, and vice versa (GPC-3). An employer can easily navigate and, when allocating job responsibilities, take into account that graduates are more inclined to work with regulatory, legal, technical documentation (GPC-2), are able to participate in the development and innovative methods, tools and technologies in the field of professional activity (PC-11, 13), rather than managing the organization’s personnel (PC-5). The employer may take into account the individual professionally oriented achievements of graduates. Bachelor with more high value(ability for self-organization and self-education) can be considered as a candidate for the position of line manager.
3. A complex generalized indicator of the final educational result at a university () is presented in a formalized form by analogy with the “average grade point of a diploma,” but from the standpoint of a competency-based approach. The calculation is carried out according to formula (6):
(6)
However, the information content of the indicator is limited, since it does not allow identifying the development of individual competencies among graduates in a situation where the values are equal. This requires decomposition, that is, a return to the indicator and its visual representation in the form of a competency profile.
Despite the apparent numerousness and complexity of the calculations, the technique is quite simple and efficient, since as a tool it is enough to use the standard Microsoft Excel program with an approved algorithm, accessible not only to the teacher, but also to the student. The electronic program allows, after introducing semester points in disciplines and practices, to accumulate an integral quantitative result and track the dynamics of competence development.
Among the positive aspects of the proposed method, we note the main ones.
The universality of the method and its independence from dynamic changes in the professional field, reflected in the educational sphere in the Federal State Educational Standard for Higher Education. The method is not limited by the number or name of competencies. These can be: corporate, managerial and functional (technical). It is separately possible to position competence in the field of digital communication - digital competence. The method and methodology are acceptable as an assessment mechanism at all levels of vocational education.
Scientific, pedagogical validity of the method, validity of the obtained calculated data, reliability (use of uniform standards or criteria within the university); fairness (all students should have equal opportunities to achieve, learn the result and build a trajectory for its improvement); timeliness, efficiency.
The professional community is provided with an accessible and understandable information base, since the result is displayed in the graduate student’s personal online portfolio. It is advisable for students to start forming a portfolio personally by the student, displaying the results of intermediate control in the context of disciplines and competencies, from the 1st year of study within the discipline “Introduction to the profile of the field” or similar in content.
Pedagogical value of the approach. When using it in methodological work and the educational process, the student becomes involved not only in the process of independent calculations of the result (with subsequent control by the teacher), but becomes an active observer of the step-by-step development of success as he consistently masters knowledge, skills, and abilities from discipline to discipline, from practice to practice in courses of study. A student can build a personal educational trajectory based on the final result planned by “yesterday’s applicant.” The student accumulates experience in personal self-organization related to his performance of self-assessment, life-planning, reflective and other functions. The intermediate results reflected in the portfolio and the orientation of industry (enterprise) organizations on them are useful in selecting students for internships, for completing final work on real problems of the enterprise, and an invitation to work depending on the results of training. This also increases motivation for academic work.
Conclusion. The method and methodology for assessing educational results in a competency-based format presented for discussion and the calculations based on them for a specific basic professional educational program confirm the didactic validity of this approach, usefulness for the professional community, strengthening the relationship between work and vocational education, as well as the need to develop this applied area in didactics of higher education.
Bibliographic link
Vasilyeva N.O. ASSESSMENT OF STUDENTS’ EDUCATIONAL RESULTS BASED ON THE COMPETENCY MODEL // Modern problems of science and education. – 2017. – No. 6.;URL: http://science-education.ru/ru/article/view?id=27188 (access date: 02/01/2020). We bring to your attention magazines published by the publishing house "Academy of Natural Sciences"