Friday, 25 May 2018

Milestones, rewards and a few surprises

The celebrations are over and many schools said farewell this week to students graduating or moving on to the next level. We have a week or so before summer school begins, so it’s a perfect time to take stock of another year’s lessons.

This was definitely a year of firsts for us at BrightFish. We transitioned from half-year pilots and trials to our first full year of implementation with thousands of students using BrightFish Reading across the country. In November, we set our first milestone of 100 students working in BrightFish hourly for an entire school day. A week later we broke that high mark and continued to reset it throughout the year. The year ended on a high note – we had our first student crack 600,000 points to take the top prize in our national BrightFish All-Star Award.

Lessons learned - and a few new tricks

We learned a lot this year. I’ve compiled a few of those lessons or what I like to call “stuff I didn’t know for sure.”

1. Teachers really like live chat. When our head of development suggested we add a live chat app to BrightFish, I politely nodded and tried to look enthusiastic. (All the while thinking that teachers really wouldn’t use it but what harm could it do.) I was so wrong – teachers loved it! Typical questions were about login issues, adding new students, reading reports, and changing or adding content levels. Most issues were resolved in under three minutes and teachers could chat right from their BrightFish dashboards. 

Excerpt from a live chat session with a BrightFish teacher.

2. Sometimes no news really is good news. In the first few weeks after the initial training, we typically hear from teachers a fair bit as they learn to use the tools and gain more experience with the program. When my team doesn’t hear from a school, we usually conclude that they haven’t started or aren’t using the tools. Our development team created a nice little monitoring tool for us to see how many times teachers have accessed their BrightFish pages and when. Guess what? Many of the “silent” teachers were managing on their own, using the tools and getting along just fine without us.

3. Scheduling is a big issue in middle schools. Invariably, one the biggest barriers to using a new technology program – or any curriculum resource – is time. What’s interesting is that this seems to be a mostly middle school challenge, especially when the district introduces a new program and teachers are asked to integrate it into their curriculum. We’re still analyzing data from this year, but the trend is definitely leaning towards the lowest weekly usage in grades 6-8.

4. Fluency rates improve from story to story. Across our user populations, we have measured an increase in both accuracy and speed in reading words and phrases as students complete multiple stories. The slowest trials are recorded in the first story, with improvements after 2 or more stories. We expected this result, but we’re now able to establish baselines for each grade level and story, which helps us to predict how students will do on the activities. As students begin to process words and phrases more efficiently, they can focus their cognitive resources on comprehending text and deriving meaning from it.

5. Vocabulary scores get better too. For English learners, vocabulary scores improve after three or more stories. In isolating data from English Learners in high school and adult programs, we’re seeing scores improving in vocabulary after students have completed two or more stories. What surprised us was the rate of improvement, indicating that students are able to transfer word skills from one story to another, regardless of subject matter.

6. Subject matter really does matter. Generally speaking, nonfiction text is more difficult, especially for ELL students. It takes longer on average to complete nonfiction stories and comprehension scores are typically lower than in fiction stories. We’re not really surprised by this – and we definitely need to collect more data before making any pronouncements – but it does support the idea that students need a wide exposure to different types of text to strengthen their reading skills and develop content knowledge.

7. Games are still hot. We periodically ask a few questions of our students to find out what they like and want to add to BrightFish. The #1 favorite feature of the program for a second year running is “The Games Store.” Story selection and reading different kinds of stories took the #2 spot, followed closely by learning new words.
Tracking points and redeeming them for games - still #1 for BrightFish students

8. Don’t mess with my points! We knew that students pay very close attention to how they are doing on each story. Students loved the new feature that allowed them to track their points versus everyone in the class. We really found out the value of points when we had to do a software reset one weekend to make an adjustment to the Games Store. It resulted in some students having their values changed because points hadn’t been deducted from their store balance. I think we heard from every single one of them the following Monday.

9. Infrastructure is getting better – or we are. When we first started two years ago with a few trial schools, we spent a lot of time helping schools test their devices and setup. Browser-based software is only as good as its network connection, and reports of “getting kicked out” and some devices not working at all consumed about 50 percent of our support time. From those early experiences, we learned a lot about improving our own infrastructure and the way we communicate with teachers on technical issues. By building intuitive “self-test” tools for bandwidth and browser updates, implementing server checks to bridge temporary disconnections, and improving display messages that let teachers know if there’s an issue, we’ve improved the experience. And those support calls? We’re down 10 percent of our support requests being about technical issues. Next year, we’re striving for less than 5 percent.

10. It’s good to test. This year we had really fantastic partnerships with districts on testing the use of BrightFish with different schools, teachers and student populations. No one was shy about making changes when there were issues with a class getting started or with students at a particular school getting enough access. It’s a great lesson that experimentation is healthy and sometimes you just need to try new things and see what sticks. It just takes one great teacher to adopt a program for success to take hold – and from there you can create a local model that flourishes.

Well, enough looking back. Now it’s time to get ready for summer school!