There is a disconnect between districts and educational technology providers on the effectiveness of current procurement processes according to a new report. Sponsored by Digital Promise and the Education Industry Association and paid for by the Bill and Melinda Gates Foundation, Improving Ed-tech Purchasing states that 66% of providers are unhappy with the procurement process while only 20% of districts are unhappy.
70% of superintendents report that procurement processes meet their product acquisition needs and 77% say they meet instructional needs, although there is a sense of overwhelm as ed-tech products flood the market.
How schools discover, acquire, and evaluate learning technology is crucial to whether students and teachers can readily access the tools that support their goals…at times, teachers and students don’t end up with the best learning technology tools to meet their needs. We can do more to ensure the promise of personalized learning is fulfilled.
The report reveals that technology directors, often the ones most involved in ed-tech purchasing, are the least satisfied with the credibility of evidence provided by tech companies. On the other hand, providers face a wall of fragmented policy procedures that differ from district to district. One provider commented, “It’s not as if the districts are really broadcasting what they are looking for and sometimes they don’t know what they’re looking for until they see it.”
The report highlights what educational marketers already know – that third-party evidence concerning a product’s effectiveness is costly to develop. Then there is the question of credibility. Only 29% of districts report that they are satisfied with the evidence about product effectiveness. And, in practice, districts rely more on peer recommendations and pilots than rigorous evidence.
On the provider side there is concern that districts don’t know how to evaluate external evidence when it is available. One provider stated, “Our internal efficacy rigor is rarely an asset because few districts know how to assess or differentiate vendor efficacy claims.” And many providers can’t afford to invest in randomized control trials that the schools want.
In the absence of an independent third-party clearinghouse, the report writers recommend the following:
- Technology providers should consider instructional needs when developing and marketing their products.
- Create guidelines for districts to evaluate evidence about products, leverage peer recommendations, and conduct well-structured pilots that increase rigor and inform purchase decisions but don’t reduce instructional time.
- Providers should focus on faster, less expensive alternatives to randomized trials and formal pilots such as case studies and small comparison group designs.
- Schools and districts should publicize their instructional needs and goals so providers can better match them notably by releasing more RFIs or Requests for Information.
The report writers highlight the need for a third-party evaluation entity, which both providers and buyers would find useful. Specifically they envision on online “Ed-tech Product Information Exchange” where districts can learn about products and providers can learn about districts’ instructional needs and purchasing policies. It certainly would have the potential to create better matches between buyers and providers.
But, given the previous attempts to establish this kind of entity, it is hard to imagine who might step forward. Also, there is clearly an opportunity for better clarity between buyer and provider, but not much tolerance for more bureaucracy.
What is your challenge in conveying the efficacy of your products to districts?