Since I posted our revised ALEC Policies the other day and the introduction to our ALEC Organisational Self Assessment and Internal Review Summary I thought I probably don’t have much to lose to post the rest of it. Here’s part 1. Parts 2 – 6 to follow shortly.

Please comment. It shows stakeholder engagement… You’ll have my thanks. I’ll also respond to your comments.

1. How well do our learners achieve?

Learner achievement data

Learner achievement data, specifically qualification completions is our main source of evidence for tracking learner outcomes in relation to our NCALNE (Voc) professional development work.

The TEC requires us to report on this information at key milestones through the year. As per our contract with the TEC, the minimum achievement levels for 2014 for adult literacy educator grant (ALEG) funding are:

• that a minimum of 80% of learners complete 40 credits towards an approved ALNE qualification over a 12 month period; or
• that a minimum of 80% of learners complete an approved ALE qualification within the required timeframe.

The table below summarises this data on outcomes for recent years

Year	Number of Learners/Funded Places	% Achieved Qualification
2011	89/100	89%
2012	83/99	84%
2013	75/86	87%
2014	TBA/88	TBA%

Feedback from our learners

We also look at feedback from our learners, workforce, and other interested communities or individuals. Some of this happens formally, for example through our student evaluation process. And some feedback happens more organically through conversations with colleagues, professionals, and others working in the field including at government departments such as the TEC, NZQA, and Department of Corrections.

The table below summarises key themes, both positive and negative, as they tend to show up in the various student evaluations that we undertake with our cohorts of learners around the country.

Positive themes in the learner feedback	Negative themes in the learner feedback
Training is enjoyable Presenters are engaging and high quality Content is informative Approach is often transformative for learners Skills learned are directly applicable to classroom experience for most	Timeframe sometimes feels too short Some material is complex Assessment tasks are time consuming Some participants are compelled to attend and/or not motivated There is a lot of content to digest

We keep hard copies of all student evaluations. The data is collated and summarised. We do analyse it, but because we’ve been doing this since 2007 we can almost predict in advance what the negative themes will be.

There isn’t a lot of negative feedback and the comments we do get usually relate to the pace of the training which. For this we are usually constrained by other factors such as available time. Often we can’t do much about this. One thing that we have done though is that we do talk about the time constraints right at the beginning of the training when we are doing the delivery.

It’s also worth noting here that our learners are often hostile when we start our training. This doesn’t tend to show up in their comments as we tend to win them over. However, many of our learners are sent to this training due to TEC funding and compliance requirements on courses taught at levels 1 – 3. This means they are often unmotivated, particularly when they start.

Where we can we work in managers as well, as this makes a huge difference if our learners are supported internally by their supervisor or manager. Their final presentations and assessments tend to reflect the changes in their thinking and practices.

Some learners come into the training hostile to the Maori content which is currently in Outcome 2 of unit standard 21204. Our approach to this content is practical and inclusive and many Pakeha tutors tell us informally afterwards about the change in their thinking after looking at some basic Matauranga Maori concepts and approaches.

As noted elsewhere, we’ve also implemented an email coaching programme that coaches them through the various aspects of their assessments and evidence collection as a way of keeping them focused and motivated during the times in between training sessions.

Another part of our analysis is that our learners are often stepping up to a higher level of work in an academic sense than they have ever done before. We see evidence for this in the lack of study and organisational skills in some candidates.

Feedback from colleagues and other interested communities or individuals

Feedback from our colleagues, particularly those doing the same kind of work, happens continually, but in a much more organic way. For example, when we co-facilitate training with other trainers we often meet afterwards and deconstruct the day’s training over dinner. This also happens via phone, skype, and email as we communicate with others including at TEC, NZQA, the Department of Corrections, and at other similar organisations. It’s a small field and we tend to stay well informed via this network.

By contrast, the student evaluations that happen at the end of a face-to-face delivery workshop are more formal, the ongoing process of giving and receiving feedback from those in our network is not a formal process, but it’s something that happens continually as we meet and communicate with others. Much of this is anecdotal and sometimes information is given or received in confidence.

However, the table below is a summary of what we see as some key trends arising from feedback from colleagues and others in our network

Year	Information emerging organically from our network that has informed decisions
2011	The uncertain nature of LN related funding moving forward.
2012	The need for an online interactive version of our NCALNE (Voc) training in some format. The need for a national register of NCALNE graduates currently active in the field. Concerns around the massive churn factor in trades and vocational training around the country
2013	The willingness of Pathways Awarua to partner in the process of creating online modules for the NCALNE (Voc) using their platform. The need for a much more tightly defined and coherent level 6 Diploma that offers a pathway not just for more academically inclined tutors, but for trades tutors who are developing skills and expertise in embedded literacy and numeracy provision.
2014	Revisions to the Education Act Section 159 may require all tutors teachings at levels one to three to complete the NCALNE (Voc) or its revised format from 2015. This will increase demand for the training. An indication that tutors teaching in ILN programmes may be required to complete the NCALNE (Ed) training as a condition of funding. Initial interest from graduates and others with regards to starting the Diploma even as we go through the process of creating it.

Another avenue for feedback that we have only just started to tap is via our social media channels including Facebook, Twitter, LinkedIn, and Graeme’s Blog. This technology is still rather new to many in our target audience so we don’t get a lot of comments. However, there are growing numbers of people who read our content online.

Source	Details
Facebook	The ALEC Facebook page has a total of 490 total page likes. The reach is still relatively small with perhaps between 20 to 30 people reached when we post content here. Engagement is low with two to three people commenting, liking, or clicking per post.
LinkedIn	ALEC gets some coverage on LinkedIn via a company page as well as Graeme’s personal page. However, LinkedIn is not a huge presence for us at the moment. More people comment on our postings on LinkedIn than through our Facebook channels.
Twitter	From 4016 tweets, Graeme has 179 followers on twitter and is following 568. Twitter works more like a news feed and we haven’t promoted it as a major channel for our content. Currently, it’s still too unknown in education in New Zealand.
Blog	The blog data is more interesting. From nearly 250 blog posts since we started it in February 2012 the blog has had more than 15,600 views. A brief breakdown is as follows: 5819 from New Zealand, 3295 from the United States, 1226 from the United Kingdon, 640 from Canada, and 637 from Australia with the balance from countries like India, Germany, the Philippines, South Africa, Ireland and many other countries. While this doesn’t exactly provide a rationale for exporting the ALNE qualifications internationally, it does indicate that large numbers of people in many countries are interested in the content. Also, our blog readers, who are generally not our learners, are more likely to comment online than our Facebook or LinkedIn connections. If we expand our online presence the focus is likely to be on leveraging blog content as this is where the widest and most engaged audience is.

The reason for including this analysis of our social media channels is as follows:

1. It seems good business practice to maintain some kind of active social media presence. This is especially relevant for us when our competitors have a very insignificant digital footprint.
2. We are interested in exporting the ALNE qualifications at stage and social media channels allow us to connect with a potential global audience.
3. Social media, and in particular our blog, allow us to maintain and promote our content and brands, including in ways that go beyond the NCALNE (Voc) training.

Destination data

Destination data for our graduates doesn’t tell us much as our learners are typically employed and working full time in their current position when they undertake a course of study with us and are usually still employed and working in the same role when they finish the training. Our existing NCALNE (Voc) training is a part time course of study typically supported by their employer.

Having said that we do visit some of our NCALNE (Voc) candidates on their own training sites. When this happens we do sometimes connect with our graduates and get to see what they are doing further down the track.

Our time with our learners is actually very short and we are not required to report anything further to the TEC once they have graduated. Also, currently there is no professional body for our graduates to join. We have recommended to the TEC that there would be enormous value in the creation of a national register of NCALNE graduates who are actively embedding literacy and numeracy into their training and this is now listed in the TEC current work plan.

Also, many of our graduates complete our training because it’s a compliance requirement for their organisations. As such, they often start the NCALNE (Voc) as reluctant learners. We think the student evaluation data validates our sense that they leave the training reasonably enthusiastic about the work. However, there is often very little incentive for our graduates to continue on to higher levels of study in this field.

We do often hear anecdotally that our graduates will complete courses like the Adult Tertiary Teaching certificates at level 4 or 5. As practically focused trades and vocational trainers they are usually not interested in completing high level academic study like the various Master or postgraduate courses on offer from Auckland University of Technology or the University of Waitkato.

Also, until now no provider has delivered the level 6 National Diploma in Adult Literacy & Numeracy Education. This level 6 course was developed at NZQA several years ago in response to demand from the ALNE sector. Our analysis of the current diploma is that it reflects the biases of those providers delivering the postgraduate and Masters level courses, is overly academic and lacking in practical application.

We started working on our application for course approval for the Diploma in 2014. We’ve also started investigating directions for possible course content.

We are seeking to remedy this at the current time. As part of the NZQA TRoQ working group, Graeme was involved in April 2014 in re-writing the outcomes and strategic purpose of the Diploma to align more closely with the other ALNE qualifications under review and with a focus on creating a practical pathway for graduates of either the Vocational or Educator qualifications at level 5. As noted elsewhere, the TRoQ review will major ramifications for all ALNE providers regarding delivery of these qualifications.