Hackweek Spring 2017

Coming up next week is Brightcove’s 8th Hackweek. At Hackweek, all engineers take breaks from their regular sprint to work on something new and different. We want to encourage innovation and it’s often nice to just take a break from your regular duties for a while.

Plus, we get these awesome T-shirts!

During Hackweek, we often tackle problems that we think could lead us in a new direction. These projects sometimes succeed and we sometimes are able to follow through with those after Hackweek. Other times, we fail and realize we shouldn’t proceed in that direction. In all cases, we learn a lot during Hackweek.

Here are a few highlights from previous Hackweeks, in the words of the team members:

MSE-HLS

This project came to be to see whether we could play back HLS content on desktop browsers using the new Media Source Extensions (MSE) APIs.

How did you come up with the idea?

Adaptive bitrate streaming provided a big playback experience improvement over progressive download and was a key advantage for Flash. MSE was a really experimental technology but the only way to do adaptive bitrate formats on any of the non-Safari browsers. HLS was a natural choice for the format since we were already producing it by default to serve iOS viewers. HLS over MSE seemed like the easiest way to provide the professional playback experience people had come to expect on HTML5 video. Of course, “easiest” is relative in this case.

Why did you choose to work on it rather than something else?

It seemed like a real technical challenge and was completely outside my area of expertise since I hadn’t dealt with the internals of video formats before. Also, it felt like a necessary step to truly kill Flash and nobody else was clamoring to do it.

How far did you get during Hackweek?

I spent two full Hackweeks working on the project. I think we might have had video-only playback at the end of the second Hackweek. We didn’t actually have anything usable until a solid amount of additional nights-and-holidays hacking.

What was required to take it from Hackweek project to production?

A huge amount of debugging and help. After basic playback was proven out, our team felt it was the right choice for the future and we got to work making it production ready. It remains a big part of the work we do to make the Brightcove Player and Video.js awesome.

Dynamic Delivery

One of the biggest projects we’ve run in the last 5 years originated in a Hackweek 2 years ago. The next generation Ingest and Delivery technology called “Dynamic Delivery” (codenamed "Bolt") had humble beginnings.

How did you come up with the idea?

I’d talked to customers who wanted to reduce their storage footprint, and be able to react to the rapidly changing device landscape more efficiently, it was clear that dynamic media packaging was a technology that could help solve this problem for our customers, but the existing solutions in the market didn’t meet our requirements, or our customer’s

The idea of building a dynamic media packager wasn’t a new one, but it had never been done successfully as a global service, so that was the obvious challenge, to create a SaaS-based dynamic media packager.

Why did you choose to work on it rather than something else?

This project had such an obvious benefit to our customers that it was very compelling to work on.

How far did you get during Hackweek?

We got a fairly comprehensive prototype working during Hackweek, we managed to get the barebones we needed for just-in-time re-packaging of fragmented MP4 (fMP4) media to MPEG-DASH Live Profile and HLS V3/4 delivery technologies, and we demo’d it running it on a comprehensive set of players and devices. We even managed to squeeze in a just-in-time image thumbnailing service on our final morning.

Here’s our Dynamic Delivery Demo from Hackweek 2 years ago:

What was required to take it from Hackweek project to production?

About 2 years, and several people. As we built out the project, the scope quickly expanded to include a very significant number of extra delivery and DRM technologies. We also ultimately took the opportunity to redesign our ingest solution at the same time to ensure that we could optimally leverage the new delivery technology that we’d designed. We also built out a whole new global cloud infrastructure to ensure the best video experiences for our customers, regardless of where they are.

We now have a team working on Dynamic Delivery in London, Phoenix, Boston, and Tasmania, making sure we stay ahead of the curve on the latest delivery technologies.

Dynamic Delivery is currently being rolled out to Video Cloud customers worldwide.

CodeCommit

AWS CodeCommit is Amazon’s managed source control as a service product. Brightcove’s Player Management service has a git backend and so we thought maybe we could swap it out with CodeCommit.

Tell me a bit about the project, how did you come up with the idea?

The goal was to solve a longstanding issue: better horizontal scaling for player publishing.  We especially wanted to be able to update or roll back all players in 15 minutes in the event of a bad release in order to limit our customers' exposure to issues. At the same time we wanted to take advantage of pre-built solutions as well as reduce our costs.

Our main bottleneck was our NAS solution - SoftNAS. It proved to be our single biggest liability in terms of service stability and limited us greatly in how much we were able to scale.  We used SoftNAS as a place for shared storage of player repositories. Our thinking at the time was that we wanted to find a drop in replacement for either the NAS with EFS so we could continue doing what we were already doing or to use CodeCommit and have it manage repositories entirely.

Why did you choose to work on it rather than something else?

The payoff for solving this issue was too great not to work on it. Faster player updates and rollbacks save us engineer hours and reduce our liability when something goes wrong.  Removing SoftNAS from our system would reduce cost and improve system reliability. And doing it during Hackweek allowed us to have engineers from two different teams tackle the project together.

How far did you get during Hackweek?

We ended the week with a working prototype but did not get to the point of performance testing.

What was required to take it from Hackweek project to production?

Performance and load testing I believe were all that remained after Hackweek and unfortunately that was what ended up doing the project in. CodeCommit could not serve the level of traffic we needed to send its way.

Player Testing Enhancement

This project had a grand vision that for simplifying testing of players both during development but also in the wild, particularly, to help out our Customer Support team. It would’ve been a page where you could pass in a local player or a website that has the player loaded and it’ll run a series of tests against this player and show you the results. This way, Customer Support can gain more information and understanding about a misbehaving player. It could have also been extended to be a module in the Brightcove Studio which allows users to test their own players in the same manner.

How did you come up with the idea?

With such a grand vision, you had to start somewhere. We started the tests for ads and other plugins across a wide variety of player configurations

Why did you choose to work on it rather than something else?

We realized that we had quite a few ideas for improving our testing but were never able to fully prioritize the work during our regular sprints. Hackweek seemed like a great time to try out some of our ideas.

How far did you get during Hackweek?

We managed to get an initial Proof of Concept done in a couple of days once we switched to using Capybara. Before that, we tried to use casper.js but it has some limitations that prevented it from being ideal for the type of testing we wanted. A member from another team was invaluable in helping us come up to speed with Capybara.

What was required to take it from Hackweek project to production?

Unfortunately, it isn’t in production. We have action items in our planning but it is hard to get prioritized with all the other work that we have. Though, once complete it will probably be very useful. There’s already another project that was made that creates a wide variety of players with different configurations and we think this would work well with that project to create a complete end-to-end testing solution with a nice UI. We think to complete the project will not take more than a couple of weeks but would require buy-in from stakeholders and team-members to prioritize it during planning.