Documents | Twinning

Ms Helka Kekäläinen, Ms Heli Mattisen, Ms Kirsi Hiltunen, Ms Kati Isoaho, Mr Hannu Apajalahti, Mr Tauno Otto, Mr Heikki Malinen, Mr Touko Apajalahti, Ms Tiia Bach, Ms Lagle Zobel Annex II to the mission report of Activity: 4.6 Analysis of the Pilots

Annex II to the mission report of Activity: 4.6 Analysis of the Pilots

ANNEX II

Component 4: Standards and Guidelines for QA in Higher Education

Activity 4.6: Analysis of the Pilots 5-9 June 2017

Recommendations for further activities of ANO supporting the implementation of an EHEA compatible QA system in Azerbaijan

The Assessment Criteria

The key terms in the Accreditation Manual (e.g the Quality Assurance system) need to be explained more so that everyone involved in the evaluations can have a joint understanding.
According to the international experts, the number of sub-criteria under the assessment criteria is far to high. However, which of these criteria need to be removed or merged, needs further discussion. The number of sub-criteria under different criteria should be balanced out more (under some criteria there are currently 5 or 6 sub-criteria, whereas under others there are as much as 8 or 9), as within the timeframe given, it was difficult for the experts to get information about all the criteria. On the other hand, the university representatives found all the criteria to be very important. It is important to eliminate all possible overlapping between the criteria.
The pilot universities suggested that the criteria should be graded in a more measurable way instead of „conforming“/“not conforming“, e.g in percentages. However, each criteria might have its strengths and weakesses and measuring them in such a strict manner might reduce the overall anaytical quality of the report.
It is very important to make sure that all the stakeholders can understand the assessment criteria in a uniform way. ANO has to remember that criteria has to be applicable to all kinds of universities specialized in humanities, social sciences, law, etc.
The universities suggested to highlight as separate assessment areas the professional development of staff (in-house training) and the cooperation between universities and the industry/labour market. However, these topics are already covered by the existing assessment areas.
Maybe there could be some sort of a „free assessment area“ that ANO and the HEI could decide upon together – one optional evaluation area on which the HEI could get some enhancement-based feedback.

Self-evaluation

It is highly recommended to provide public self-evaluation trainings to HEIs in order to raise their capacity in how to conduct SER on an institutional level. ANO could take responsibility for that. Also, institutions themselves could appoint external experts as „critical friends“ for consulting purposes. Institutions could also benefit from benchmarking with previous self-evaluation reports.

ANO is advised to apply a road map approach to development of self-evaluation capacity of the HEIs. For the coming years it is necessary to provide concise support for the HEIs in this matter, but it might be possible to increase the level of autonomy in self-evaluation process later in future, when the institutional capacity has reach the more mature level.

Before sending the SER to the expert panel, it should be checked whether 1) all the areas are covered, 2) all requirements set for SER are met, 3) additional data are there, and 4) the report is clear for readers. A balance has to be struck by ANO not to influence the content of the report but make sure it gives all information necessary for assessment. One option would be to include the requirements towards SER in the assessment criteria to be fulfilled as a prerequisite before checking other assessment areas. It is important to keep one eye on the ESG and the other on the local context.
As there were some differences between the institutions how they organized the SER process internally, there were some challenges reaching the right persons and getting their input. Therefore, it is recommended to introduce some guidelines regarding the organization of the process in a HEI.
More background data (statistical data) is needed for evaluations.Various factual information should be included as annexes (e.g job descriptions). However, appropriate context should be provided for the quantitative data. It is recommended to always provide in the SERs very general background data regarding the finances, students, teaching staff. Similar information about every single university should be available to the panels.

Another recommendation is that he institution itself should have some targets for different assessment areas (e.g student numbers). Some it could also be quantitative indicatiors about reaching targets in certain areas which should be supported by data in appendixes. Which kind of targets should be added to whic areas, should be elaborated by ANO. The data should then be analysed and connected to the targets. Standardized indicators could be used in the process (dropout rate, applications per study place). Information important of institutions’ point of view should be included.

Composition of panels and training

The foreign members of the panel pointed out that they would need some more information on the local HE system. Therefore, it is recommended to have mixed teams of both foreign and local experts.
It is suggested to adopte a code of ethics for experts: ANO should establish selection criteria for the best available experts to provide objectivity (there should be a mechanism for conflict of interest) before the evaluation process.
As it is a very important requirement of the ESG to have a student on the evaluation panel, it is the task of ANO to find the students interested in the external evaluation process and also allocate appropriate resources for their training. Also, the need of a student perspective and the importance of the role of the student in a team should be better explained in the process. Very active cooperation with student unions is suggested to build up joint understanding of the whole assessment system.
It is highly recommended to consider including the labor market representative in the panel. It is necessary to build trust between the society and the university. ANO should start working into the direction of involving the labor market representatives in the process.
It is very important to have at least as the chair to the panel an international expert who can demonstrate new skills and provide a learning experience for local experts.

It is also recommended to hire international trainers to train ANO experts.

When ANO succeeds in adopting new accreditation standards, it is important to communicate with different stakeholders to dissaminate information about the new standards.

The site visit

Regarding the whole evaluation process:

the timeframe (from the compilation of SER to writing the report) could be reconsidered: whether there should be more time for writing the report/submitting comments by the HEIs. ANO should thinking this through quite carefully: agreeing on the dates of the visit, submitting the report, deadlines/timelines should be in place.
It could be good to have the role and tasks of the Project Managers set out in some kind of format so that it would be also clear to them – e.g, should they take notes during site visit, how much are they allowed to interfere with writing the report, etc.
It is important to have one contact person in the HEI in order during the whole evaluation process in to ensure the continuity of the process.

Before the site visit:

It is crucial to have a detailed visit schedule ready well before the site visit. It is also important to get all the names of the interwiees before the site visit so that the team could check whether they shall meet all the necessary people.
In the future, in order to improve that everything in the institution is well understood and prepared in time, the Project Manager could maybe meet/talk to the contact person before the visit so that everything is coordinated. Guidelines should be in place how to prepare for the visit. Also, all the people who come to the interviews could be informed about the panel members.

During the site visit:

Deep consideration should be given to how much time should be allocated to see the facilities and have a tour of the institution.
If necessary, it is suggested to have a meeting at the end of the visit to clarify some issues that remained unclear, depending on the topic.
It could also be considered to aski feedback right after the visit about the panel members and performance of the experts. This would also be useful feedback for the agency in deciding whether to use these experts again.

As a general remark, it was suggested that EKKA and FINEEC could maybe support this new evaluation process in Azerbaijan at the beginning to make sure it starts successfully.

The report

Although the Manual seems to provide enough information, more guidelines are needed for the experts how to write the report – e.g, what is the expected level of detail, in what style the report should be written and what kind of background information is needed. It is of essence that all the reports are written in a consistent manner.
It is recommended to develop a method/toolkit to help the Project Manager how to work in different situations for the teams.
The evaluation reports should also be public in the future, discussed in the universities and also with the whole sector. It is important for ANO to spread the knowledge system-wide, organize seminars, meta-analyses.
There should be given clear guidelines to the institutions how to comment on the reports.
After the committee receives the comments and decides which ones to consider, there could be a cover letter why some comments have/haven’t been accepted.
It would be useful for ANO to follow up on its own activities, e.g conduct a their metasynthesis on their own operations and develop their own actions, as this is a requirement of the new ESG.

Ms. Heli Mattisen, Ms. Helka Kekalainen Recommendations on doctoral studies in Azerbaijan as part of Activity 3.3 Aligning Higher Education Qualifications

Recommendations on doctoral studies in Azerbaijan as part of Activity 3.3 Aligning Higher Education Qualifications

DEVELOPMENT OF DOCTORAL EDUCATION

BASED ON HIGH LEVEL RESEARCH

PREVIOUS TWINNING MISSION conducted by Professor Mati Heidmets and Karl-Erik Michelsen: No time should be wasted, because scientific environment today is challenged by fundamental changes in society, economy and environment. Hence, the capability to produce new knowledge is and will be tremendously valuable asset and it cannot be replaced by transfer of knowledge or any other means.

NIZAMI project: There is an urgent need for Azerbaijan universities to revise the rules and content of organizing doctoral studies in Azerbaijan within the Bologna process documents in terms of their consistency, especially in ensuring science and research provisions in relevance with EU standards, to fulfill the students' mobility - one of the basic provisions of Bologna process.

OVERVIEW OF THE CURRENT SITUATION FOCUSING ON MAIN AREAS OF IMPROVEMENT
Functioning of the system:

Currently the doctoral education in Azerbaijan is too fragmented and provided by very diverse types of institutions. In 2015, 116 institutions were offering PhD studies. Academy of Sciences employs more than 11 000 staff members and additionally there are 53 universities (21 of them offering PhD studies) and several research institutions. In 2015, there were 145 organisations carrying out research.
There is a lack of research infrastructure and support services.
There are several governmental institutions that govern and micro-manage universities. This creates overlapping activities that eat up scarce resources. Higher attestation commission has strong powers to control dissertation process and to provide diplomas.
Only about 10% of PhD students graduate in time.

Human resources:

The heterogeneous system of research results in-efficient use of financial and human resources. Salaries are low and PhD studies are also suffering from low status. Universities and Academy of Science have difficulties in attracting best talents and brains.
Supervisors for PhD candidates must have Doctor of Science degree (3.19). This is an outdated requirement, because about 70% of potential supervisors are older than 65 (2014 total 2510 academic staff members having Doctor of Science degree and involved in research, total number of persons involved in doctoral studies 2014 – around 3000)

Structure of doctoral programmes

The system of two level doctoral degrees, PhD and Doctor of Sciences, is not in accordance with Bologna principles and doesn’t have a clear reasoning.
PhD programs have no clear structure and don’t use credit system.
For the level of Doctor of Sciences there is no “study programme” – this consists of research conducted by an academic professional individually or in a research group, comparable with the work of the post-doctoral researcher. Thus, it should be considered as the continuation of the academic career rather than part of formal education.

Internationalization

There is weak international component in the PhD studies, involvement of foreign scientists/professors in teaching process, research projects or as co-supervisors is not common
Dissertations are written in the form of monograph instead of articles in pre-reviewed international journals. Students (and the supervisors) publish mostly in national language.
There is no free access to electronic databases and foreign scientific journals.
Very few doctoral students participate in foreign exchange programs.
Doctoral studies don’t provide a doctoral candidate with the skills of acting in an international work and study environment on their own and develop the ability to express themselves in their field of specialisation in at least one foreign language both in writing as well as orally.
Recognition of doctoral qualifications from foreign countries in order to apply for the position of the professor is not possible (?)

RECOMMENDATIONS

A precondition for doctoral education should be high level research capacity and internationally recognised research output of the institution in the area/specialty of the concrete doctoral programme. In order to define institutions eligible for offering doctoral studies, first, the evaluation of research output and capacity is needed.

Training of high level researchers is currently done in too many institutions (116!). Therefore, we find it necessary that doctoral programs should be concentrated in the large and well equipped universities that have passed rigorous quality assurance evaluations. Quality assurance of doctoral education has been “outcome-based” (provided by the High Attestation Committee), there are no strict requirements regarding the scope, level and capacity of research for opening/developing a doctoral programme.

An example of assessment criteria for research evaluation can be found in Annex.

Criteria for research universities should include criteria for quality of research as well as quality of educational process, bearing in mind the main goal of integrating research and education. Publishing in High Impact Journals shall not replace but support student centred and learning outcome based teaching and learning.

Cooperation between universities and research institutes of the Academy of Sciences and other’s needs to be enhanced and motivated with the means of funding. Higher education and research in the Republic of Azerbaijan should be able to compete at the global level, not nationally. Building strong research universities by bringing together the best talents and brains and high level infrastructure would make it possible.

The differentiation should be made between PhD as doctoral degree acquired after the completion of doctoral studies, and Doctor of Sciences as a result of an individual research effort. Inherited from the Soviet system, candidate for Doctor of Sciences can partly be compared with the post-doctoral researcher in westerns countries, but it should be considered as the continuation of the academic career rather than part of formal education.

The academic content of the doctoral programs should be re-design to assist PhD students to conduct high quality research. For instance, current mandatory course on philosophy should be replaced by courses that enhance skills in research design, methodology and writing a peer review article in English language.

Doctoral studies should also provide students with management, teamwork and project development skills as well as the knowledge of legal protection of intellectual property.

Dissertation work should evolve from monographs into article based dissertations. At least four peer-review articles + intellectually stimulating summary is needed to complete doctoral studies. Article based dissertation connects PhD students in the international scientific networks. In addition, objective peer review process validates dissertations and improves supervision.

Universities should be given full autonomy in scientific matters. They are the place where scientific research is done in the future. This process requires quality assurance system that will evaluate scientific functions in the universities. Those institutions that pass the rigorous evaluation should be granted rights to design and conduct programs that aim at BA, MA and also PhD degrees. High Attestation Commission could be reorganized into research evaluation authority offering regular research evaluation procedures rather than re-checking the quality of every single dissertation.

Research infrastructure and environment should be improved by providing all PhD students and professional researchers a free access to electronic databases and foreign publications. In addition new laboratory and research facilities should be built and contracts with other countries signed to get access to national and transnational research infrastructure.

PhD students should have good and open access to international exchange programs and at least six months training abroad should be required from all students. In addition supervision of PhD studies should be given to domestic as well as international scientific experts.

To promote the internationalization of doctoral programs there should be considerable investments in improving English language skills of students and staff. Additionally foreign experts need to be invited to serve as opponents and members of the defence panels.

ANNEX: Description of assessment criteria and rating scale

	Scientific impact of research	Sustainability and potential of research	Societal importance of research
Very good	Most of the R&D outcomes are of high international standard: the results generate considerable international interest in the field. Publications have been issued by leading international publishers and/or in highly recognized international journals. The number of publications per research staff member and the bibliometric indicators of the publications indicate consistently high level of R&D. Additional information provided by the institution confirm the great impact of the R&D.	The organization and management of R&D are clear and effective and take into account the specifics of the field. Measures for assuring funding in the field and the amount and structure of funding as well as the gender, age, and ethnic/national diversity among staff are testimony to the sustainability of the R&D from a future perspective. The institution has a clear and focused vision in relation to the development of the field which strengthens the sustainability and potential of the field. Research directions have clear potential from a scientific, socioeconomic, and/or environmental perspective. Infrastructure is in very good condition and provides very good conditions for R&D in the field and for doctoral studies (in the case of universities). (For universities) The R&D on which all of the doctoral programmes in the field are based, is at high international level.	R&D in the field takes into account societal development trends and needs (e.g. through research in vital issues or commission applied research, research staff participating in external development and decision-making committees, significant societal developments based on the results of R&D, etc.). Research staff in the corresponding field is proactively involved in societally and globally important research, raising issues and initiating discussions. The institution clearly values, in its R&D management, the role and responsibility it has in society and takes into account the societal importance of its R&D.
Good	Majority of the R&D outcomes are of good international standard: the results generate international interest in the field. Publications have been issued by internationally recognized publishers and/or in internationally recognised journals. The number of high-level publications per research staff member indicates some disparity in the level of R&D within research directions and/or structural units. Additional information provided by the institution confirm the scientific impact of R&D.	The organization and management of R&D are generally clear and effective and take into account the specifics of the field where possible. Measures for assuring funding in the field, the amount of funding as well as the composition of staff are testimony to the sustainability of the R&D from the future perspective, however it may be necessary to implement measures to strengthen the structure of funding and the diversity among staff. The institution has a clear vision of strengths and development needs of the corresponding R&D field and the desire to strengthen the potential of the field. Research directions from a scientific, socioeconomic, and/or environmental perspective are at times clear but also at times unapparent and require more attention. Infrastructure is in good condition and provides good conditions for R&D in the field and for doctoral studies (in the case of universities). (For universities) The R&D on which all of the doctoral programmes in the field are based, is generally at good or very good international level but uneven across different programmes. (In this case, the committee indicates which programmes are at satisfactory level).	Various R&D directions in the field take into account societal development trends and needs (e.g., through research in vital issues or commission applied research, research staff participating in external development and decision-making committees, significant societal developments based on the results of R&D, etc.). The choice of research in the field is partly based on societally and globally important topics. Discussion on vital issues is initiated. The institution values the role and responsibility of its R&D in society and it endeavours to align its activities with the needs of society and takes into account the societal importance of its R&D.
Satisfactory	The R&D outcomes are in general of satisfactory international standard: they provide international interest in certain areas. Publications have been issued by international publishers or recognised by domestic publishers or in domestic scientific journals. Research staff members are active in publication of outcomes but the level of publication is very uneven across staff members and/or sub-fields.	The organization and management of R&D are generally clear, however there are areas that require more focused reflection, including opportunities for taking into account the specifics of the field. Measures for assuring funding in the field and the amount of funding as well as the composition of staff permit conducting R&D and doctoral studies, however require significant effort from the institution to ensure sustainability and strengthen the potential from the future perspective. Institution has room for improvement in defining strengths and development needs of the corresponding R&D field. Research direction potential from a scientific, socioeconomic, and/or environmental perspective is at times unapparent and requires greater attention. Some directions of research are exhausted/becoming exhausted and there have been no demonstrable efforts in expanding these. Infrastructure is in satisfactory condition for carrying out research and providing doctoral studies (in the case of universities) but improvement is needed to increase quality. (For universities) The R&D on which most of the doctoral programmes in the field are based, is at least at a satisfactory international level, however the level of R&D in case of some programmes is unsatisfactory. (In this case, the committee indicates which programmes are at an unsatisfactory level) or the variation in levels is so large across programmes that a rating of “good” cannot be given.	Development of R&D in some directions takes into account societal development trends and needs (e.g., through research in vital issues or commission applied research, research staff participating in external development and decision-making committees, etc.). The direction of R&D at the institution deals with its role and responsibility in society and takes into account the societal importance of its R&D in some aspects.
Unsatisfactory	Satisfactory R&D at an international level is practically non-existent or few high-level outcomes do not provide evidence of being at satisfactory level overall. There is a failure to implement measures to raise the level of research in the field.	Funding in the field is very uneven or insufficient. The composition of R&D staff does not indicate sustainability of the R&D from a future perspective. R&D development lacks potential from a research and/or socioeconomic perspective. Institution has not been able to define clear strengths and development needs of corresponding R&D field. Research direction potential from a scientific, socioeconomic, and/or environmental perspective is exhausted/becoming exhausted in various research directions; significant restructuring is necessary in order to generate further potential. Infrastructure is in very uneven or in poor condition (does not meet current standards in the field in large part, or depends on another institution’s infrastructure in large part). (For universities) The R&D on which most of the doctoral programmes in the field are based, is at a weak international level.	The institution lacks a clear and focused approach of development of the corresponding R&D field according to societal development trends and needs, and lacks an understanding of consideration of these aspects. Societal impact of R&D is not being evaluated.
Areas of special note	Specified by the evaluation committee as appropriate	Specified by the evaluation committee as appropriate	Specified by the evaluation committee as appropriate
Areas in need of improvement	Specified by the evaluation committee as appropriate	Specified by the evaluation committee as appropriate	Specified by the evaluation committee as appropriate

Ms Heli Mattisen Mission report of Activity: 4.6 Analysis of the Pilots

Mission report of Activity: 4.6 Analysis of the Pilots

TWINNING AZERBAIJAN

Support to the Ministry of Education of the Republic of Azerbaijan for Further Adherence of the Higher Education System to the European Higher Education Area (AZ-ad-EHEA)

AZ/14/ENP/OT/31

EU Short Term Expert Mission Report

Basic Information

Component and Activity:

Component: 4. Standards and Guidelines for QA in HE

Activity: 4.6 Analysis of the Pilots

Name of the Expert: Ms Heli Mattisen

Dates of the Mission: 19 – 21 June 2017

Contractor: Finnish Education Evaluation Centre (FINEEC) /

Estonian Quality Agency for Higher and Vocational Education (EKKA)

Relevant Background Information/State of Affairs

The objective of Activity 4.6 was to coordinate a trial/test run of the Azerbaijani Standards and Guidelines for QA of higher education (AzSG) at three higher education institutions in Azerbaijan. The main aims of the pilot evaluations was to support the strategic management of institutions, provide external feedback to the institutions’ own internal quality assurance procedures, as well as inform the internal and external stakeholders of the compliance of the institutions’ quality assurance with the ESG.

In April 2016 a Drafting Group was appointed by the Ministry of Education to work on a proposal for AzSG. A draft manual for the pilot evaluations was discussed with the Advisory Group in a seminar in June 2016. The draft was published on the Twinning project’s website in order to get feedback on the assessment areas and criteria. Amendments were made to the manual based on the feedback. The capacity of the three pilot universities (Azerbaijan State Pedagogical University, Azerbaijan State Economic University and Azerbaijan Technical University) to conduct a self-evaluation was supported through trainings and events in several previous missions.

The pilot institutions submitted their self-evaluation reports in January – February, 2017. The site visits of the pilot universities took place in April, 2017 and the assessment reports were finalized in May – June, 2017. Feedback seminar was organised in each university 5-6 June. International and local experts in cooperation with university representatives and ANO staff analysed the results of the pilot evaluations and prepared recommendations for the future activities of the Accreditation and Nostrification Office (ANO) in Azerbaijan in 7-9 June.

The pilot evaluations had an institutional approach with the focus on teaching and learning. The evaluation reports provide the pilot institutions with information regarding their strengths and good practices, as well as recommendations for the institutions’ further development and suggestions to the MoE regarding the regulations governing higher education in Azerbaijan.

Objectives and Tasks of the Mission

The overall aim of the Component 4 is to develop Standards and Guidelines for Quality Assurance in Higher Education in Azerbaijan, in line with the European Standards and Guidelines for Quality Assurance (ESG) and test them with three higher education institutions.

The special objective of the last missions in Activity 4.6 was to disseminate the experiences of the pilot evaluations and the Twinning project achievements to the larger higher education community.

Time Schedule of the Mission

Date	Activities/Meetings BC experts met (title and institution)	Remarks
19.06.2017	Meeting with the head of higher education unit of the Department of Higher Education and Research Sülhaddin Gözalov and RTA team Preparation of the joint report about the outcomes of the component 1
20.06.2017	CONFERENCE ON ENHANCING HIGHER EDUCATION QUALITY IN AZERBAIJAN Participants: Representatives of the universities, MoE, Twinning STE-s
21.06.2017	Preparation or the mission report Meeting with the RTA team at MoE Planning the closing conference

Achievement of the Expected Results

The expected results of the mission were achieved.

The Opening words of the conference were given by Mr. Jeyhun Bayramov, the Deputy Minister of Education, H.E. Ambassador Malena Mård, Head of Delegation of the European Union and Mr. Indrek Kiverik, Charge d’Affaires, Estonia. The MS project leaders gave a joint presentation. The mandatory results were reported and the main outcomes of the project were identified as mapping the situation in HE in a comprehensive way, realistic and applicable recommendations widely discussed with stakeholders, supporting collaborative learning in the HE community of Azerbaijan

and supporting some changes in mind-set regarding the autonomy of HEI-s combined with a rigorous, meaningful and improvement led approach in quality assurance. BC Project Leader Emin Amrullayev reflected the success of the project.

Each of the component leaders or their substitutes with their beneficiary counterparts presented the results of each component.

Component 1 Legislation by Heli Mattisen and Sülhaddin Gözalov.

The main partners in the component have been Department of Higher Education and Research and Accreditation and Nostrification Office (ANO) as well as Legal Department of the MoE, Universities, Academy of Sciences and the leaders of other components.

STE-s have given concrete recommendations given regarding

Overall structure and coherence of the legal framework
Amendments to the Statute for Higher Education Institutions – governance structure, autonomy in designing management structure clear division of responsibilities, student involvement etc.
New State Standard and Programme for Higher Education – learning outcomes, student assessment, doctoral education etc.
New Accreditation Standards
New Accreditation Rules and Procedures

Most of the recommendations of the STE’s have been taken into account, some documents (like State Standard and Programme for Higher Education) are still under development.

Sülhaddin Gözalov provided some examples of new documents developed based on the recommendations of Twinning experts – like new Statutes for the Engineering University and the University on Languages have giving more autonomy to the universities in building up the management system, developing the programmes, involving students and external stakeholders in the government of the university. The new regulation on credit system will give more freedom to the university and the teachers to design the learning and teaching process.

On the whole, it has been an useful learning process - „formative assessment“ regarding compliance with Bologna and ESG has been offered by European experts, but the main drafting work has been done by local experts.

Component 2 - Co-ordinating and communication by Kauko Hämäläinen and Vusala Gurbanova

The main objectives and recommendations of seven different activities were reflected and also, how these recommendations are used. Main objectives of this component were the following:

Coordination and networking capacity of the MoE and relevant stakeholders is enhanced on the basis of good practice examples in the EHEA.
Assess progress and the current status of EHEA/Bologna reforms in Az.
Assess the institutional set up within the MoE with best practices and improvements for the overall institutional architecture.
Developing communication and networking between universities, stakeholders and MoE based on the experiences from Finland and Estonia.
Conducting a training needs analysis and train experts.
Create communication strategy.

The main recommendations from different activities were the following:

- Systematizing doctoral education.

- Redesigning national standards.

- Establishing networking between Az HEIs.

- Developing more lifelong learning services.

- Drafting a mobility program and removing obstacles for student mobility.

- Division of work and sharing responsibilities between the ministry, universities and external stakeholders.

- Prioritizing areas for the Ministry.

- Developing quality practices of the internal QA process.

- Establish rector´s conferences.

- Analyse needs and options to join international associations in HE.

- A newsletter as a medium of communication.

- Increase students´ and stakeholders involvement in internal QA process and administration of HEIs.

- Improve pedagogical training for university teachers.

- Ensuring coordination of EHEA related trainings.

Many of these recommendations have been implemented or started planning how to proceed. E.g. study visits and training has been organised to increase the capacity of people in central positions. The main results in practice of component 2 are according the summary by Vusala Gurbanova the following:

1.MoE is realizing Nizami project parallel with the Twinning project where they are planning to restructure doctoral education in Az in line with requirements of EHEA. On respect of these 2 projects recommendations regulations about doctoral studies are going to be improved. The main changes that they are going to implement are the following: PhD will be the last level of HE. At the same time, doctor of science will be open for everybody who would like to continue his/her research. The third one is implementation of ECTS at PhD level.

MoE started to draft new standards of HE, where learning outcomes of all levels of HE will be reflected. Additionally, descriptors of all levels of HE will be included to this document. We are planning to combine 5 regulations in one and draft the general standards for HE. 12 recommendations out of 14 have been taken into consideration in the new draft.
The main achievement in the framework of component 2 was increasing cooperation of the representatives of different HEs. This project helped HEIs´ representatives to come and work together, to learn from each other, involve students to the decision making processes of universities and to cooperate with different stakeholders.
In the framework of the Twinning project the participants had an opportunity to meet with different rectors and to learn their opinions regarding to the establishment of a Rector´s conference. Their reflection were positive and supportive. MoE understand the importance of the Rector´s conference and they already drafted its statute.

Component 3 - Developing AzQF by Maiki Udam and Azad Akhunov

1.Trainings on QF-EHEA

The awareness among academic staff about the basics of QF-EHEA, including constructive alignment between learning outcomes, student assessment and teaching methods, vary notably.

Recommendation: At the national level, a more systematic approach to trainings of academic staff on various topics (learning outcomes, student assessment, teaching methods, e-learning, recognition of prior learning (RPL) etc) is needed.

Analysing the correspondence between EQF and AzQF

Although the draft of AzQF has some minor contradictions both concerning content and formulations, the document is ready to be approved and implemented.

Aligning HE Qualifications and Curricula (in Doctoral Studies)

It has been the most challenging area in Component 3 as the organisation of doctoral studies in Azerbaijan differs from the model used in most other European countries.

There are several issues that need to be solved:

the outcomes of 2-level doctoral studies (PhD and Doctor of Science) and their position in the AzQF;
cooperation between universities and research institutes of the Academy of Science;
funding of research at the universities;
international cooperation and visibility;
quality assurance of research and doctoral studies.

Enhancing Practices and Procedures of Academic Recognition

It is commendable that the Accreditation and Nostrification Office (ANO) has been created and manned with dedicated staff.

The main recommendation is to move from nostrification (looking for equivalence of (mainly) quantitative indicators) to the recognition (looking at general correspondence of learning outcomes).

It is also advisable not to regulate the recognition issues too strictly on the national level but to give HEI-s more autonomy to decide about cases related to academic mobility and its recognition.

Further Implementation of the AzQF

The Draft Action Plan on Implementation of National Qualifications Framework for Lifelong Learning as well as The Draft Concept of Establishment of National Commission and Secretariat for Implementation of AzQF have been developed.

Recommendations:

the future projects and national activities should focus more on RPL and respective trainings of academic staff;
development of databases/IT system(s) for managing the information about students, staff, study progress, admission, graduation etc would support implementation of NQF and QA.

Component 4 – Pilot evaluations by Helka Kekäläinen and Tofig Ahmadov

The main acitivites of the component were reflected: Trainings for the EHEA context, ESG and EHEA trends and practices, Drafting the Standards and Guidelines for Quality Assurance in Azerbaijan, Fostering the Self-Evaluation Capacity in 3 Pilot institutions and The test run of the Standards and Guidelines with 3 universities in Azerbaijan.

The common strengths of the 3 pilot institutions were identified as

The roles and focuses of the universities – pedagogy, economy, technology – are clear and well justified,
Willingness to improve, awareness of the need for change
Orientation towards improving the management structure and create conditions for the senior management staff to take more responsibility and ownership in fulfilling their role
Awareness of the need to improve management and leadership competences.

The recommendations for Strategic Planning and Study Programmes and their development were explained.

Tofig Ahmadov introduced in detail the new approach to external quality assurance that ANO will carry out in future.

Unexpected Results

No unexpected results

Issues Left Open After the Mission

There were no issues left open after the mission.

Recommendations for Future Missions

The last PSC will take place 28 August and the Closing Conference 29 August 2017. RTA office will look for a venue that would combine the event and catering in a satisfactory way.

Conclusions and General Remarks Concerning the Project

The Project has achieved all the mandatory results and more.

Ms Helka Kekäläinen, Ms Maiki Udam, Mr Kauko Hämäläinen Mission report of Activity: 4.6 Analysis of the Pilots

Mission report of Activity: 4.6 Analysis of the Pilots

TWINNING AZERBAIJAN

Support to the Ministry of Education of the Republic of Azerbaijan for Further Adherence of the Higher Education System to the European Higher Education Area (AZ-ad-EHEA)

AZ/14/ENP/OT/31

EU Short Term Expert Mission Report

Basic Information

Component and Activity:

Component: 4. Standards and Guidelines for QA in HE

Activity: 4.6 Analysis of the Pilots

Name of the Expert: Ms Helka Kekäläinen, Ms Maiki Udam, Mr Kauko Hämäläinen

Dates of the Mission: 20 – 21 June 2017

Contractor: Finnish Education Evaluation Centre (FINEEC) /

Estonian Quality Agency for Higher and Vocational Education (EKKA)

Relevant Background Information/State of Affairs

The objective of Activity 4.6 was to coordinate a trial/test run of the Azerbaijani Standards and Guidelines for QA of higher education (AzSG) at three higher education institutions in Azerbaijan. The main aims of the pilot evaluations was to support the strategic management of institutions, provide external feedback to the institutions’ own internal quality assurance procedures, as well as inform the internal and external stakeholders of the compliance of the institutions’ quality assurance with the ESG.

In April 2016 a Drafting Group was appointed by the Ministry of Education to work on a proposal for AzSG. A draft manual for the pilot evaluations was discussed with the Advisory Group in a seminar in June 2016. The draft was published on the Twinning project’s website in order to get feedback on the assessment areas and criteria. Amendments were made to the manual based on the feedback. The capacity of the three pilot universities (Azerbaijan State Pedagogical University, Azerbaijan State Economic University and Azerbaijan Technical University) to conduct a self-evaluation was supported through trainings and events in several previous missions.

The pilot institutions submitted their self-evaluation reports in January – February, 2017. The site visits of the pilot universities took place in April, 2017 and the assessment reports were finalized in May – June, 2017. Feedback seminar was organised in each university 5-6 June. International and local experts in cooperation with university representatives and ANO staff analysed the results of the pilot evaluations and prepared recommendations for the future activities of the Accreditation and Nostrification Office (ANO) in Azerbaijan in 7-9 June.

The pilot evaluations had an institutional approach with the focus on teaching and learning. The evaluation reports provide the pilot institutions with information regarding their strengths and good practices, as well as recommendations for the institutions’ further development and suggestions to the MoE regarding the regulations governing higher education in Azerbaijan.

Objectives and Tasks of the Mission

The overall aim of the Component 4 is to develop Standards and Guidelines for Quality Assurance in Higher Education in Azerbaijan, in line with the European Standards and Guidelines for Quality Assurance (ESG) and test them with three higher education institutions.

The special objective of the last missions in Activity 4.6 was to disseminate the experiences of the pilot evaluations and the Twinning project achievements to the larger higher education community.

Time Schedule of the Mission

Date

Activities/Meetings

BC experts met

(title and institution)

Remarks

20 June

CONFERENCE ON ENHANCING HIGHER EDUCATION QUALITY IN AZERBAIJAN

List of participants Annexed

Azerbaijan State Economic University

21 June

Planning the Closing Conference, Reporting

RTA office, Hotel Austin

Achievement of the Expected Results

The Opening words of the conference were given by Mr. Jeyhun Bayramov, the Deputy Minister of Education, H.E. Ambassador Malena Mård, Head of Delegation of the European Union and Mr. Indrek Kiverik, Charge d’Affaires, Estonia. The MS project leaders gave a joint presentation. The mandatory results were reported and the main outcomes of the project were identified as mapping the situation in HE in a comprehensive way, realistic and applicable recommendations widely discussed with stakeholders, supporting collaborative learning in the HE community of Azerbaijan

and supporting some changes in mind-set regarding the autonomy of HEI-s combined with a rigorous, meaningful and improvement led approach in quality assurance. BC Project Leader Emin Amrullayev reflected the success of the project.

Each of the component leaders or their substitutes with their beneficiary counterparts presented the results of each component.

Component 1 Legislation by Heli Mattisen and Sülhaddin Gözalov.

The main partners in the component have been Department of Higher Education and Research and Accreditation and Nostrification Office (ANO) as well as Legal Department of the MoE, Universities, Academy of Sciences and the leaders of other components.

STE-s have given concrete recommendations given regarding

Overall structure and coherence of the legal framework
Amendments to the Statute for Higher Education Institutions – governance structure, autonomy in designing management structure clear division of responsibilities, student involvement etc.
New State Standard and Programme for Higher Education – learning outcomes, student assessment, doctoral education etc.
New Accreditation Standards
New Accreditation Rules and Procedures

Most of the recommendations of the STE’s have been taken into account, some documents (like State Standard and Programme for Higher Education) is still under development.

Sülhaddin Gözalov provided some examples of new documents developed based on the recommendations of Twinning experts – like new Statutes for the Engineering University and the University on Languages have giving more autonomy to the universities in building up the management system, developing the programmes, involving students and external stakeholders in the government of the university. The new regulation on credit system will give more freedom to the university and the teachers to design the learning and teaching process.

On the whole, it has been an useful learning process - „formative assessment“ regarding compliance with Bologna and ESG has been offered by European experts, but the main drafting work has been done by local experts.

Component 2 - Co-ordinating and communication by Kauko Hämäläinen and Vusala Gurbanova

The main objectives and recommendations of seven different activities were reflected and also, how these recommendations are used. Main objectives of this component were the following:

Coordination and networking capacity of the MoE and relevant stakeholders is enhanced on the basis of good practice examples in the EHEA.
Assess progress and the current status of EHEA/Bologna reforms in Az.
Assess the institutional set up within the MoE with best practices and improvements for the overall institutional architecture.
Developing communication and networking between universities, stakeholders and MoE based on the experiences from Finland and Estonia.
Conducting a training needs analysis and train experts.
Create communication strategy.

The main recommendations from different activities were the following:

- Systematizing doctoral education.

- Redesigning national standards.

- Establishing networking between Az HEIs.

- Developing more lifelong learning services.

- Drafting a mobility program and removing obstacles for student mobility.

- Division of work and sharing responsibilities between the ministry, universities and external stakeholders.

- Prioritizing areas for the Ministry.

- Developing quality practices of the internal QA process.

- Establish rector´s conferences.

- Analyse needs and options to join international associations in HE.

- A newsletter as a medium of communication.

- Increase students´ and stakeholders involvement in internal QA process and administration of HEIs.

- Improve pedagogical training for university teachers.

- Ensuring coordination of EHEA related trainings.

Many of these recommendations have been implemented or started planning how to proceed. E.g. study visits and training has been organised to increase the capacity of people in central positions. The main results in practice of component 2 are according the summary by Vusala Gurbanova the following:

1.MoE is realizing Nizami project parallel with the Twinning project where they are planning to restructure doctoral education in Az in line with requirements of EHEA. On respect of these 2 projects recommendations regulations about doctoral studies are going to be improved. The main changes that they are going to implement are the following: PhD will be the last level of HE. At the same time, doctor of science will be open for everybody who would like to continue his/her research. The third one is implementation of ECTS at PhD level.

MoE started to draft new standards of HE, where learning outcomes of all levels of HE will be reflected. Additionally, descriptors of all levels of HE will be included to this document. We are planning to combine 5 regulations in one and draft the general standards for HE. 12 recommendations out of 14 have been taken into consideration in the new draft.
The main achievement in the framework of component 2 was increasing cooperation of the representatives of different HEs. This project helped HEIs´ representatives to come and work together, to learn from each other, involve students to the decision making processes of universities and to cooperate with different stakeholders.
In the framework of the Twinning project the participants had an opportunity to meet with different rectors and to learn their opinions regarding to the establishment of a Rector´s conference. Their reflection were positive and supportive. MoE understand the importance of the Rector´s conference and they already drafted its statute.

Component 3 - Developing AzQF by Maiki Udam and Azad Akhunov

1.Trainings on QF-EHEA

The awareness among academic staff about the basics of QF-EHEA, including constructive alignment between learning outcomes, student assessment and teaching methods, vary notably.

Recommendation: At the national level, a more systematic approach to trainings of academic staff on various topics (learning outcomes, student assessment, teaching methods, e-learning, recognition of prior learning (RPL) etc) is needed.

Analysing the correspondence between EQF and AzQF

Although the draft of AzQF has some minor contradictions both concerning content and formulations, the document is ready to be approved and implemented.

Aligning HE Qualifications and Curricula (in Doctoral Studies)

It has been the most challenging area in Component 3 as the organisation of doctoral studies in Azerbaijan differs from the model used in most other European countries.

There are several issues that need to be solved:

the outcomes of 2-level doctoral studies (PhD and Doctor of Science) and their position in the AzQF;
cooperation between universities and research institutes of the Academy of Science;
funding of research at the universities;
international cooperation and visibility;
quality assurance of research and doctoral studies.

Enhancing Practices and Procedures of Academic Recognition

It is commendable that the Accreditation and Nostrification Office (ANO) has been created and manned with dedicated staff.

The main recommendation is to move from nostrification (looking for equivalence of (mainly) quantitative indicators) to the recognition (looking at general correspondence of learning outcomes).

It is also advisable not to regulate the recognition issues too strictly on the national level but to give HEI-s more autonomy to decide about cases related to academic mobility and its recognition.

Further Implementation of the AzQF

The Draft Action Plan on Implementation of National Qualifications Framework for Lifelong Learning as well as The Draft Concept of Establishment of National Commission and Secretariat for Implementation of AzQF have been developed.

Recommendations:

the future projects and national activities should focus more on RPL and respective trainings of academic staff;
development of databases/IT system(s) for managing the information about students, staff, study progress, admission, graduation etc would support implementation of NQF and QA.

Component 4 – Pilot evaluations by Helka Kekäläinen and Tofig Ahmadov

The main acitivites of the component were reflected: Trainings for the EHEA context, ESG and EHEA trends and practices, Drafting the Standards and Guidelines for Quality Assurance in Azerbaijan, Fostering the Self-Evaluation Capacity in 3 Pilot institutions and The test run of the Standards and Guidelines with 3 universities in Azerbaijan.

The common strengths of the 3 pilot institutions were identified as

The roles and focuses of the universities – pedagogy, economy, technology – are clear and well justified,
Willingness to improve, awareness of the need for change
Orientation towards improving the management structure and create conditions for the senior management staff to take more responsibility and ownership in fulfilling their role
Awareness of the need to improve management and leadership competences.

The recommendations for Strategic Planning and Study Programmes and their development were explained.

Tofig Ahmadov introduced in detail the new approach to external quality assurance that ANO will carry out in future.

Unexpected Results

The experts did not come across to any unexpected results.

Issues Left Open After the Mission

No issues were left open after the mission.

Recommendations for Future Missions

The last PSC will take place 28 August and the Closing Conference 29 August 2017. RTA office will look for a venue that would combine the event and catering in a satisfactory way.

Conclusions and General Remarks Concerning the Project

The Project has achieved all the mandatory results and more.

Ms Helka Kekäläinen European Style Pilot Evaluations in Azerbaijan Higher Education

European Style Pilot Evaluations in Azerbaijan Higher Education

Ms Maiki Udam Developing Azerbaijani Qualifications Framework (AzQF)

Developing Azerbaijani Qualifications Framework (AzQF)

Mr Kauko Hämäläinen, Ms Vusala Gurbanova Twinning in Azerbaijan - Component II

Twinning in Azerbaijan - Component II

Ms Heli Mattisen, Mr Sülhaddin Gözalov Developing the Legal and Regulative Framework

Developing the Legal and Regulative Framework

Ms Helka Kekäläinen, Ms Heli Mattisen Main Outcomes of the Twinning Project - AZ-ad-EHEA

Main Outcomes of the Twinning Project - AZ-ad-EHEA

Ms. Helka Kekäläinen, Ms. Heli Mattisen Main Outcomes of the Twinning Project