Implementation and aims
All five GOAL intervention strategies were carried out in Slovenia. To make it easier to review and define the activities specific to a particular intervention strategy, the national team compiled a protocol for the three-phase monitoring of the course and realisation of the defined tasks:
- activities that needed to be carried out prior to the GOAL programme;
- activities carried out during the GOAL programme;
- activities carried out after completion of the GOAL programme, including Wave 1 and Wave 2 evaluation.
During preparations for and the implementation of GOAL guidance, all five intervention strategies were integrated in such a way that they supported each other. In particular, the strategies of “partnership”, “tools”, “counsellor’s competences” and “outreach – reaching target groups of adults” together supported the strategy of providing adults in education with a “high-quality guidance process”.
Within the GOAL project and as part of the “quality of guidance” intervention strategy, we set the following objectives with the aim of producing high-quality guidance services for low-skilled adults:
- to upgrade counsellors’ skills to enable them provide guidance to low-skilled adults;
- to improve the use of tools within the guidance process;
- to integrate within a network of organisations in the local environment that work with the selected GOAL target groups;
- to more easily reach these target groups on this basis and provide counsellors with more comprehensive information on the selected target groups’ needs and characteristics on the one hand, and the activities offered by specific partners on the other.
We describe the activities carried out, the outcomes achieved and the proposals formulated for further development in detail in individual sections of this report. The overall assessment by all those involved in the GOAL project is that all these activities together led to an increase in the quality of guidance services at ISIO centres and secondary education centres alike, and to better integration with the partners involved – which, in turn, led to an increase in the quality of the partner organisations’ operations.
We can conclude, from the discussions with stakeholders that took place in the Wave 1 and Wave 2 focus groups, that we did manage to foster among the participants a greater awareness of the importance of adult education guidance and of the quality of this activity from a variety of aspects. Some of them were of the opinion that focus group discussions should continue in the future to foster the exchange of opinions and views between stakeholders and enable new (common) practical solutions to be formulated.
Strengths and achievements
Very good results were achieved with the 160 clients involved in the GOAL programme – testament to the quality of the work of counsellors in both organisations (the two ISIO centres and two secondary education centres). An analysis of the results in Wave 2 indicates that GOAL guidance did help clients identify their own wishes and interests, formulate their objectives and sketch out a plan for achieving those objectives, with a large majority achieving their objectives.
Most clients (90%) entered the guidance process with clearly defined objectives, and for the majority it was clear what their next steps would be after guidance if they wished to achieve their objective. Eighty-seven per cent of clients believed that guidance had helped them arrive at this realisation. The majority (90%) also said that they were more aware of their options after guidance and 87% said that they were more motivated to achieve their objectives. The majority (90%) felt that guidance had made it clearer to them what their next steps towards their objective should be.
The GOAL guidance process generally took place across multiple sessions, with all but two of the 160 clients involved taking part in at least two sessions. The majority (95%) of clients made progress towards achieving their objective fully or at least in part. The high quality of the guidance provided is also indicated by the following client outcomes, as outlined in Section 9:
- 82% of clients obtained information on their educational options in the course of the guidance process;
- 60% of clients enrolled in a course or an education or training programme;
- 45% of clients became more self-confident;
- 19% of clients improved their job-specific skills.
The majority of those who achieved their objectives, or achieved them at least in part, saw the greatest increase in their self-efficacy score. The data shows that clients were more self-confident, more sure of themselves and more motivated to achieve their objectives after guidance.
As we have mentioned on several occasions (see e.g. sections 4, 6 and 9): we can also conclude from the opinions expressed by clients in the client satisfaction survey and the follow-up survey that they were very satisfied with the work of the counsellors, and particularly with the personal, discreet and professional approach they took to the guidance process. Clients liked the fact that counsellors took time for them, proceeded from their needs and interests, and gave them the feeling that they remained available for consultation after the completion of the guidance process. The advantage for GOAL counsellors was that guidance became part of their regular activities so that they could provide guidance support during the client’s education process (if the client had indeed enrolled in education or training).
From the point of view of ensuring high-quality guidance, GOAL counsellors also made particular mention of the importance of proper training in the guidance of different target groups. An analysis of the situation prior to the commencement of GOAL guidance indicated that counsellors did have the knowledge, skills and competences to provide guidance to low-skilled adults (prior to the programme, counsellors at ISIO centres had had slightly more training to this end than counsellors at secondary education centres). They also pointed out that they required additional competences precisely in order to provide guidance to low-skilled adults, as the provision of guidance to these groups of adults was more demanding for them (these target groups required more incentives and motivation and a more individually tailored approach) (Needs and Strengths Analysis, 2015).
At the end of the GOAL guidance process, counsellors were of the opinion that they had acquired new knowledge, skills and competences through the additional training organised for them by the national GOAL team during implementation of the GOAL programme. They welcomed the possibility, at working meetings between all the counsellors and the national team, of exchanging information and examples of good practice from their respective organisations and of learning about examples of good practice in the participating GOAL countries presented to them by the national team after every international meeting.
The study visit made by GOAL providers to Iceland at the end of the GOAL project (spring 2018), organised as part of the Erasmus+ K1 Mobility programme, will also help to raise the quality of further guidance work at GOAL providers.
From the point of view of the use of specialist tools in the guidance process, counsellors were highly positive about the three questionnaires to aid management of the guidance process with clients from the first to the final session, with emphasis on the thorough acquisition of data on clients so as to enable them to formulate their objectives and their steps towards those objectives more easily (at the same time, these questionnaires were the basis for the recording of data on the client and the guidance process in the monitoring database). The national team compiled a short protocol for counsellors for the management of the GOAL guidance process to make it easier for them to manage a first, subsequent or final session. Counsellors identified this protocol as a tool that could support them in their efforts towards high-quality guidance process management. We therefore proposed that a standard form for the three questionnaires and the protocol be prepared for use in adult education guidance. In addition, a form for the preparation of a personal education plan should be compiled in standard form, as this is a tool that is used by all counsellors, albeit with different form and content (with greater or fewer elements). All adult education counsellors would welcome the standardisation and upgrading of the form. We have outlined in Section 7 a few more proposals for the upgrading of adult education guidance tools that will help to improve the quality of the guidance process.
During GOAL implementation and partner cooperation, partners emphasised in Wave 1 that the GOAL project could improve the guidance options for vulnerable groups of adults if clients/adults from other partner organisations could become involved in GOAL counsellors’ guidance activities. They believed that this would enable them to address their clients (vulnerable target groups) properly in terms of the content and time required by these groups (more time for sessions, and more thorough and individually tailored sessions).
They stressed that it was also important for the project findings (examples of good practice) and the tools developed to be disseminated to all partners in the network for their further use. Partners also expect information on the work at ISIO and secondary education centres to be disseminated to the other organisations. Some partners highlighted, as an important aspect of project quality, greater mutual recognition, which made it easier to establish mutual trust in work with vulnerable target groups in the local environment. In Wave 2, partners from both regional networks highlighted the following achievements affecting quality, in addition to the possibility of referring clients to a guidance provider (ISIO or secondary education centre): the formation of solid partnership, which made it easier to realise their own programmes; the upgrading of activities that they otherwise performed for their target groups; the acquisition of new experiences and knowledge. Some also pointed out the recognition and exchange of various guidance tools that they were able to use in their own professional work.
All partners emphasised the very good cooperation with both guidance providers, i.e. the ISIO centre and the secondary education centre. One important factor in relation to quality is the fact that all partners participating in the evaluation process proposed that the local guidance networks continue with formalised modes of operation, clear objectives and an approach that attempted to reach different target groups within the local environment.
Stakeholders further discussed the factors affecting the quality of adult education guidance in the Wave 2 focus group.. The stakeholders believed that a well-trained counsellor was the key factor in quality and that an adult education counsellor therefore required additional specialist knowledge. They proposed that the professionalism of the counsellor’s work be strengthened and the profile of “adult education counsellor” be defined, alongside the tasks and competences associated with the post. They proposed the preparation of initial basic training and continuous additional professional training. They also highlighted, in relation to the issue of quality, the importance of tools, the careful planning and management of the guidance process, the development of different approaches to make it easier to reach vulnerable groups of adults and the establishment of partner cooperation.
They highlighted the results achieved at the level of individual clients as an important internal and external aspect of quality. They emphasised not only the importance of quantitative outcomes (how many were involved in guidance, the characteristics of the groups, etc.) but also the importance of qualitative (“soft”) outcomes, e.g. whether guidance also led to greater social inclusion and prevented poverty, improved job opportunities over the longer period, etc. At the meeting, the stakeholders were of one mind: that it was difficult to measure qualitative outcomes and effects of guidance, as other factors influenced the client, in addition to the guidance process. Nevertheless, it was important for guidance monitoring and the measurement of outcomes and effects to include the measurement of the qualitative aspects of the outcomes of the guidance process at the level of the client.
Challenges and barriers
It emerged from the monitoring and evaluation of the programme and results of GOAL guidance that one of the key challenges and, at the same time, obstacles to ensuring that guidance was of the requisite quality was the provision of sufficient numbers of adult education counsellors who had undergone high-quality training.
For all GOAL counsellors except one, guidance was only one of their tasks. They therefore pointed out that they would like to have sufficient time available for guidance; only then would they be able to offer clients a thorough guidance process of longer duration. This would also make it easier to monitor and evaluate the outcomes and effects of guidance (this was also important from the point of view of the assessment of the quality of the guidance process at the level of the client). Therefore, one of the challenges for counsellors is how to include the monitoring and measurement of the outcomes and effects of guidance in their regular work. GOAL counsellors gave a positive assessment to the approach developed in the GOAL project to the monitoring of the guidance process and the measurement of outcomes and effects, and proposed that it be considered a part of the further development of adult education guidance.
From the point of view of the use of tools, the challenge, after Wave 1 (when counsellors did not use tools to any great extent), was to improve this. As we have already mentioned above, it is still necessary to do more work to develop this aspect of the guidance process in order to further develop a high-quality guidance process. The challenge lies in how to increase access to the tools that have already been developed but are not widely available to all counsellors, either because they are unfamiliar with them or because their use requires additional training. The challenge is also to develop and introduce new tools, train counsellors properly in their use, monitor the use of tools and, on the basis of this, upgrade them, and examine which tools are more effective, which are most frequently used by counsellors, which still need to be designed, and so on.
The partnerships established within the two regional networks also supported high-quality guidance, chiefly from two aspects: the referral of clients to GOAL guidance by partners in cases where they required more thorough guidance on education and on improving their job opportunities, and the greater familiarity of GOAL counsellors (and partners) with the activities of participating partners and with the characteristics and needs of the selected GOAL target groups. The large number of partners within the networks (ten partners in each) whose activities covered different target groups proved to be an obstacle; they would show less interest in taking part in partnership meetings that did not address the target groups they themselves were involved with. The challenge for the future, therefore, is whether to formulate a partnership solely on the basis of cooperation in the joint expert treatment of one selected target group set up for a definite period of time so that partners might carry out concrete activities that contribute to the greater involvement of the selected target group in lifelong learning.
In the first focus group in Wave 1, the stakeholders highlighted two factors that could hinder the provision of high-quality guidance services. The first was the issue of funding: insufficient funds to secure an adequate number of counsellors and all the conditions they required for their work, and the lack of free-of-charge adult education and learning opportunities for vulnerable groups of clients (stakeholders stressed that the local community had to become involved in the funding of guidance and adult education, in addition to the state). The second factor was the still-insufficient level of integration and cooperation of policy actors at the national and local levels in their approach to the field of adult education guidance.
If adult education guidance is properly placed within a systemic framework and adult education guidance activities, in their various forms and at various organisations, are given adequate and stable funding, this will also help to increase the quality of adult education guidance. The Ministry of Education, Science and Sport should remain the entity responsible for systemic dimension of adult education guidance at the national level. It should also strengthen cooperation with other line ministries. The opinion of stakeholders in the discussions in both focus groups in Wave 1 and Wave 2 was that this cooperation was still insufficient at the national level and was therefore one of the obstacles to a number of already well-developed adult education guidance activities in Slovenia becoming a proper and stable part of the practice of all adult educators.
Baseline and progress across GOAL’s five intervention strategies
The table below provides a brief evaluative summary of the quality of different aspects of the GOAL programme in Slovenia, comparing quality at the start of the evaluation (baseline) and at the end. In this table, we provide numerical ratings for each of the five intervention areas, and an explanation of that rating for each category. These ratings and explanations are provided for the start of the evaluation and the end, with the aim of briefly summarising key issues and change over time. In addition to provide ratings and commentary for the five core GOAL intervention areas, we also address overall service quality and policy interest/support. The latter is a key factor in determining future programme sustainability.