Why Education Pilots Don't Usually Get Off The Ground

Part 1: Education Pilots Don't Usually "Get you there"

I speak with a lot of education technology, products and services vendors who say they are going to do a pilot (free or low-cost trial) in one or more schools. They tell me they will place the product in the school, the kids will use it and the school promises that if it works, they will buy it on a large scale.

The vendors are usually excited by this supposed "win" and have visions of orders pouring in as a result. Those vendors looking to raise money or sell out are sure that pilots will be a valuable proving ground and investors and acquirers being impressed the vendor has teachers, principals, etc. willing and motivated to try it out.

 But then they are surprised when I tell them that the pilot won't prove what it's supposed to, they will get mixed results -- if they get any results at all, investors will yawn and not assign any value (or valuation) to the pilot programs, and they will waste their time, money and other resources. Pilots don't usually work the way they are supposed to.

Free pilots are most often a waste

My experience shows that pilots/trials are a bad idea for many reasons:

          (1) If a school or district isn't paying for it or paying very little,, they aren't likely to use the product as religiously or put in the effort required to get a new project off the ground

          (2) If you are hoping to get investors excited, free or reduced price pilots don't convince investors that there's a market (E.g. that people want the product enough to pay for it)

          (3) Pilots are typically run either at some start date or for a duration that might not give you enough time to collect data and show gains or other "proof of life"

          (4) Even if the pilot runs the whole school year you are waiting a long time to run the program, collect and analyze data and determine effectiveness. Do you really have a year to wait to demonstrate the product works and will work for others who the vendor is convinced will be "lining up" to buy once they see the pilot is successful? And when does this opportunity to move from pilot to paid happen in the budgeting and buying cycle? Probably not when they are allocating money to these products.

          (5) There are a lot of things that go into product success. The teachers have to use the product correctly, devote as much time as is necessary to get results and figure things out on their own because they are the "Guinea pigs." A lot has to happen to get a "thumbs-up" on the product. Very little has to happen (or not happen) to get a "thumbs-down."

          (6) Another important factor is "who wants and is driving the pilot." If a principal wants to do it s/he often tells a teacher "Congratulations! You get to pilot this in your classroom!" What the teacher hears is they have to learn a new tool (that probably isn't fully baked and documented," they won't get enough support, they don't get more time or relief in other areas to offset the additional time this trial will take and they are to be held accountable for student learning even though they don't know if or how the new product will work.

          (7) And best case -- and it still doesn't usually work out -- Pilots are often served up as free or reduced in price until such time as the product has proven itself and the teacher is happy -- then the vendor will expect to get paid and the teacher or principal is supposed to be a spokesperson and endorse and help sell the product to others. Guess what? The teacher or principal usually figures out that as long as they are unhappy about something, they get free use forever until the vendor gets anxious enough to demand payment. At that point -- I've seen it a 100 times before -- the teacher or principal is enamored with another product from Vendor 2 that they can "pilot" for free and the cycle (sans payoff for Vendor 1) repeats.

In short, PILOTS JUST DON'T WORK THE WAY VENDORS HOPE -- I don't usually see pilots paying off. The school doesn't have enough skin in the game, that moment where the school is supposed to pay never happens and it's often difficult to measure effectiveness with a new product that isn't totally worked out, or at the least, can't move the needle sufficiently in the alloted time to demonstrate it works.

Now FUNDED pilots are another thing! They do work! I'll cover that in Part 2..