Below is the rest of our ALEC summary document for the upcoming NZQA review. You can read our revised ALEC Policies here and the intro to this document here and Section 1 here.

Hopefully, I’ll finish the revisions to the new system I’m using for our ALEC internal processes and post those shortly as well.

2. What is the value of our outcomes for our key stakeholders

One of our key stakeholders is the TEC. The TEC provide a unique funding stream (ALEG) for the specific purpose of upskilling educators via the ALNE qualifications including the current NCALNE (Voc) that we deliver.

We believe that the value of our outcomes and training for the TEC is evidenced in:

• The Adult Literacy and Numeracy Implementation Strategy 2012. This document which takes us through until around 2015 highlights the place of literacy and numeracy as a national priority. The Implementation Strategy outlines the need for ongoing literacy and numeracy professional development for the sector supported by the ALEC funding and including via the existing NCALNE (Voc) and other qualifications.
• The allocation of ALG funding and preferred provider status with the TEC. Our understanding is that in 2014 the TEC allocated around 140 grants to organisations delivering ALNE professional development. Our funding contract states that we were allocated 88 of these. This means that ALEC has over a 60% share of the current market for TEC funded ALEG provision. Again, we see this as evidence that the TEC values our work and outcomes. Also, 60% is a conservative estimate due to the fact that some proportion of the grants also go to postgraduate programmes around the country.
• The suggestion from TEC that the current revisions to Section 159 of the Education Act will formalise the intent expressed in the Strategy document on p.34. This is that the NCALNE (Voc) will become a requirement for all foundation level tutors and a condition of funding for their organisations.
• The inclusion in 2013 of the ALEC model for NCALNE (Voc) provision called “The Baseline Knowledge and Skills for Embedding Literacy & Numeracy” in the TEC’s Adult Literacy and Numeracy: An Overview of the Evidence (p.48).

In terms of our learners, there is evidence collected via student evaluations that suggests that graduates are satisfied with the training. The table below highlights a number of different comments from graduates or those about the complete the training:

Year	Sample comments expressing satisfaction with the NCALNE (Voc) training
2011	“Great course, many thanks to …, …, and …” “Had great fun.” “Learnt a lot and keen to put it into action.” “Very good overall. I look forward to implementing and developing what I have learnt” “Great course that will benefit the workplace” “Engaging presenter with high level credibility;l excellent materials” “Awesome – fun and interesting” “Enlightening”
2012	“Well worthwhile” “Overall, I found the course very informative and enlightening. This will definitely encourage me to embed LN throughout the programmes I am involved in” “… is an enthusiastic tutor and fully equipped with knowledge” “Are there any more LN courses I can do?” “Thanks heaps and have really enjoyed the whole course.” “Overall there was some very useful information and the resources provided were thorough and interesting. Thanks.” “Well done and thank you” “I suggest all tutors, whether first year or more do this course. It has opened my eyes and it is excellent knowledge to have”
2013	“Thank you so much for including me” “It was great!” “Thank you and keep in touch  about the next step NDip ALNE. That would be great, cheers” “I am now keen to learn more with regards to LN” “…beginning to see how this training will improve my learners’ experience – which helps me with my work” “Great class. Thanks for opening up new ways to look at things” “It was a good pace, good humour, very understandable” “Great course, well worth the effort to attend”
2014	“Very impressed with the delivery and content. Well done!” “Look forward to the next one thanks.” “Very good experience. Highly recommended. Thank you!” “Great work. Well done!” “Loved it – feel very enthused, reinvigorated for returning to work and developing some of this stuff” “Very good programme and lecturers” “Have definitely learnt new material” “Enjoyed the course/content. Recommend this for all tertiary educators.”

We would also suggest that there is other evidence that key stakeholders value our outcomes. For example:

• The inclusion in 2013 of our NCALNE (Voc) model and programme in the form of the NCALNE Lite modules now available on www.pathwaysawarua.com.
• TEC support in 2014 to develop the initial modules for the revised Diploma at level six in collaboration with Pathways Awarua
• An invitation in 2013 to put Graeme’s name forward for the TRoQ governance group and then a request for Graeme to join the TRoQ working party for the ALNE qualifications in 2014.
• The adoption of the ALEC framework and model for embedding for the TRoQ review across all of the qualifications being reviewed and revised in 2014.
• Repeat work with several large organisations between 2011 and 2014 including more recently Northtec and the Department of Corrections. We have now trained over 100 tutors working for the Department of Corrections. Corrections staff see our work with tutors as a key factor in creating a change in the culture of education and training inside the prison system.
• Unpublished research from Unitec from Dr John Benseman that cautiously notes the success and generally positive impact of the NCALNE (Voc) according to respondents to his survey.

Trades and vocational tutors often vote with their feet. In other words, if they don’t like something, or the delivery is overly academic, they will leave. Again, our outcomes data indicates that we consistently achieve higher than the 80% required by the TEC. This seems to be another indication that participants value the training and will persist even if it means taking much longer than anticipated or they have to complete by distance once the face-to-face workshops are over.

This 80% threshold is not exactly benchmarking data but in the absence of completion data from our competitors it’s the closest that we have to doing this. Our assumption is that our competitors are being held to the same contractual requirements – thus the 80% is a kind of de facto benchmark.

3. How well do programmes and activities match the needs of learners and other stakeholders

Stakeholder feedback

One indication that our NCALNE (Voc) professional development meets the needs of our learners are their comments around what they find most helpful or useful from the training. Again, here is a selection of comments drawn from our learner evaluation data the point to a good match between our activities and the needs of our learners and their work environment:

Year	What learners found most helpful or useful
2011	“Sharing info with colleagues” “To learn how to embed numeracy” “Looking at assessments and designing these to suit your area of work to highlight learner skill” “Getting away [from job site] and working with my peers” “Creating resources and tools” “Introduction and exposure to new innovative tools to help with L/N delivery” “Designing the activities” “Maths plus Maori learning [styles]”
2012	“This will definitely encourage me to embed throughout the programmes I am involved in” “I learnt a lot of new things” “The activities and being supported [was helpful]” “Learning a new way of looking at teaching prisoners” “The great set of info given to take home is proving to be a great asset” “[furthering] my own journey in LN especially place value”
2013	“Great, most enjoyable course I have attended” “Was the best course I have done” “Finding out what literacy and numeracy were really about… had no idea how big it was” “Thank you. I feel really excited about thie course. Cannot wait to come back.” “Everything [was challenging] but it was broken down to my level of understanding” “All areas covered gave me “food” to think about when dealing with our clients. Found the course most informative”
2014	“Overall concept [was helpful]” “Numeracy unpacking [was useful]” “Learning something very relevant to our students. The atmosphere was great with a strong class, which also helped” “[Gaining] a clearer understanding of learners and my role” “Very good experience. Highly recommended. Thank you!”

Entry requirements

Entry requirements to this training are dictated by the TEC funding requirements. This means that candidates need to be employed full time, part time, or in a voluntary capacity in the adult education sector. For our learners, we expect them to be adult educators with some kind of content specialisation, e.g. a trade or other vocational training expertise.

Employer and graduate feedback

For the 2014 EER we’ll supply a list of key contacts for reviewers to communicate with.

Learner achievement information

Consistently reaching the 80% threshold for our outcomes indicates that there is a good match between the NCALNE (Voc) delivery and the needs of our stakeholders including the TEC, employers, and the tutors that we train.

4. How effective is the teaching?

Learner feedback on teaching

We collect feedback from learners at the end of every workshop. The evaluation has two parts. The first part uses a rating scale across a range of categories relating to the trainers and the programme. The second part involves answers to more open questions about what was useful, challenging or that could be changed.

Typically, our learners rate our trainers and the programme at 4 or 5 out of 5 every time. This is consistent year to year and with different trainers, so we’re confident that we’re delivering at a high level. This data is available in it’s raw form as we didn’t see much value in time spent crunching it further.

The learner answer to the open questions are more interesting and we’ve used a selection of these to paint a picture of what our learners think of our training below and elsewhere in this document.

Year	Learner comments on the teaching
2011	“The course structure was excellent, kept on track, not too many games or icebreakers” “Enjoyed the numeracy side… learning a lot of LN stuff”. “[Enjoyed] knowledge and techniques to teach from the trainers” “I appreciate the open conversation to the level of expertise; gives me an excited view of my way forward” “Rodney, you are Tuake! (Awesome)” “Great facilitation that is current, relevant, and informative” “The trainers were excellent and the group was also great”
2012	“I found the course very informative and enlightening” “… is an enthusiastic tutor” “send [more] principal instructors on the course – they need to understand what we do and help others when they train” “Thank you so much. I did learn a lot. Enjoyed the course” “Reference to the big picture to keep things on track and to know the purpose [was helpful]” “Well structured” “Perhaps ALEC should start offering the level 6 Diploma”
2013	“Great workshop thanks” “Helpful resources” “Great demonstration, modern and great designed materials” “How … broke everything down and related it to our individual courses [was helpful]” “… was a fantastic trainer” “I gained a lot of very valuable info and it was relaxed yet very professional” “The Maori element [was challenging]. I also found this the most interesting” “How the course was broken down [was helpful]. It made it more interesting for myself to be engaged in and learn something that I thought I wasn’t going to enjoy or thought I would struggle with content”
2014	“[Enjoyed] enthusiasm and knowledge of facilitators” “[It was helpful] the way the course was broken down into small easy sections” “The delivery style and lots of activities made the long days seem quite manageable” “Programme is well organised” “… is an amazing educator” “Thanks… worth a lot”

Results of peer observation of teaching

As a small organisation providing national delivery of training often in different locations around New Zealand we work with trainers who are already highly skilled facilitators. We’ll provide a list of these facilitators for NZQA to contact. We don’t conduct formal peer observations but we often co-facilitate training together. This co-facilitation also acts as a kind of internal professional development due to the different areas of specialisation that we each have.

Feedback from other stakeholders

This is a quote from Dr John Benseman’s recent but unpublished survey of NCALNE (Voc) graduates:

“This study shows that the national adult literacy and numeracy certificates have been received very positively by this group of practitioners. Only a very small number of the respondents have criticised its content or delivery to any degree. Most have found it a positive experience and report that it has also influenced their practice. These findings indicate that NCALNE has made a positive contribution towards developing a more knowledgeable and skilled workforce for this emerging educational sector.” p.12 Benseman, Unpublished study

The TEC has also commissioned research to look into the value of professional development in general in the sector. This included a focus on the NCALNE (Voc). This is similar to benchmarking data. We also know that we command more than 60% of the ALEG funded delivery so the comments below apply:

“Most TEOs believe the National Certificate in Adult Literacy and Numeracy Education  (NCALNE) is an important part of tutors’ professional development. However a number of TEOs with mature practice thought that NCALNE is just the start of what is required by vocational tutors. The majority of TEOs use an external provider to deliver NCALNE…

Several TEOs commented that NCALNE was the best professional development they had. One of the TEOs queried why the National Centre, as the sector leader, is not responsible for delivering this qualification. ”. p.20 Literacy and Numeracy Professional Development Research Findings

“Studying for NCALNE was cited as being the most effective form of professional development that those interviewed had undertaken. p.25 Literacy and Numeracy Professional Development Research Findings

Teaching and learning plans

Our delivery of the NCALNE (Voc) is largely determined by the content we need to assess for the relevant unit standards. However, we strive to deliver the content in a coherent, project-based, problem solving approach.

Typically, we deliver 5 full days of training spread out over about 4 months. The format is as follows:

• Orientation to the training (Day 1)
• Literacy and numeracy in Aotearoa New Zealand: Context and issues (Day 1).
• Maori literacy and numeracy (Day 2)
• Mapping with the Learning Progressions (Day 2)
• Using literacy and numeracy Diagnostic assessment (Day 3)
• Embedding literacy and numeracy into training (Day 3 and 4)
• Assessing and measuring literacy and numeracy gains (Day 4)
• Evaluating embedded literacy and numeracy (Day 4 and 5)

We will provide a full sample of our delivery resources and course content to the NZQA evaluators either before the review date if requested or during the site visit. We can also supply the email coaching programme and show how it works in relation to the learning management system we designed as part of the Basecamp project management software we use.

5. How well are learners guided and supported?

Learner feedback on learning environment

Year	Learner comments on the learning environment
2011	“Different teaching method, games, videos, group work, individual work… positive modelling of teaching strategies which I endeavour to model in our teaching environment” “[Need] more time to do assessments” “Nice balance of practical and theory” “The process is OK, it’s just completing the work in the timeframe” “It was the right pace” “Trainer enthusiasm, clear layout, and helpful materials” “[Enjoyed] the interaction between trainers and trainees”
2012	“Participants must be keen and motivated in order to participate effectively” “Practical approach. Lots of discussions [was helpful]” “The general discussion from the tutor [was helpful] – it makes it real. Clear explanations” “Good communication” “Hands on working in small groups [was helpful]” “Encouraging tutors” “Encouragement of feedback and participation” “Respectful environment”
2013	“Good helpful learning and assessment” Feel that this was a really high quality workshop” “It was a really valuable and well run course, thanks” “I enjoyed the learning environment group format”
2014	“All of it was great – a really good mix of delivery/activities. Great approach of putting everyone at easy by doing group work, i.e. no one freaking out about having to answer a question in front of everyone” “I have been to and been put off many learning environments and this was easily the best. Very impressed with delivery and content. Well done?” “Fantastic working with 3 facilitators. Great tag team” “Great people both training and in the course” “Teaching style [was helpful]”

Feedback on learner support

Year	Comments on learner support
2011	“Understanding how much everyone has improved” “Challenging to complete assignments” “Would be good to receive a pack and logistical information about the time comittment, training day content, assessment etc prior to the course” “A few more unprompted emails from the trainers would be helpful and encouraging”
2012	“Thank you for the time you have to put in to give us quick results on assignments” “[Enjoyed] the activities and being supported” “The visit to my working environment [was helpful]” “Indri [admins support] was very helpful” “Support pack provided [was helpful” “Accessibility to … [was helpful]”
2013	“Nice having … visit and see my teaching environment” “Email support on a regular basis [was helpful]” “Understanding sections of course requirements [was challenging]. However, this dissipated once I talked one on one with tutors.” “Excellent course and support provided by ALEC presenters and associated tutors”
2014	“Yes, thank you, I have received your emails. They are very helpful.” “Thanks a lot. I am encouraged.” “Thank you very much for all the support and help.” “Appreciate you keeping on eye on things!” “Thank you for all your help during this training. Wonderfully organised.”

We know that many of our learners struggle with the coursework in the time between our sessions. We also know that some of our remote learners also struggle with not having access to peers or tutors to bounce around the ideas in the training. In order to deal with these issues, we implemented a full email coaching programme at the end of 2013. We track each learner through the coaching programme individually via the learning management system that we set up in our ALEC Basecamp account.

Learner and staff opinion

We’ll provide a list of key people in our workforce and wider network for NZQA to contact with regards to their opinions about our programme, what needs it meets, the effectiveness of our training, the learning environments that we create or work in, and the support that we offer.

6. How effective are governance and management in supporting education achievement?

Strategic and business plans

Key sources of evidence for the effectiveness of governance and management in supporting educational achievement with evidence of the approach to meeting identified needs and aspirations would include the following ALEC documents:

• Company Profile and Business Plan. This is in the process of becoming a live online document.
• Investment plans for the TEC. These are stored in our TEC workspace.
• Policies. This is an ongoing online project in our ALEC Basecamp account which replaces the various sections on policies in our QMS at the time of our last EER.
• Processes. This is an ongoing online project in our LucidChart account where we are in the process of updating and visually mapping out all of our major organisational process.
• Procedures & Practices.  Most of our procedures and the ways in which we do things including record keeping are now incorporated into our ALEC learning management system which we designed using the Basecamp project management application. This system is now in it’s 3rd major evolution online and we can provide access and a tour at any stage.

Needs Assessment

A summary of needs assessment considerations and information is available in our current Company Profile and Business Plan. However, because our work connects with the TEC Implementation Strategy for literacy and numeracy we also refer to the following documents as TEC-led needs assessment in our field as we develop our business and training content:

• TEC theoretical framework for strengthening literacy and numeracy.
• TEC mature organisation embedding practices