Back in January, PC World posted an article entitled Facebook uses 10,000 Blu-ray discs to store ‘cold’ data which discussed Facebook’s strategy of using Blu-ray disk libraries for long-term data retention. More recently, the Facebook engineer who developed the design left and started a company to focus exclusively on this strategy, and so naturally many people are left wondering what this is and what it means to IT. Simply put, the use of Blu-ray for large scale archiving represents a new approach to long-term storage and appears to be positioned as a competitor to disk and tape-based archives. This position is controversial and naturally, many vendors and pundits chimed in on the new use case. In this blog post, I wanted to highlight some of the salient blogs and articles. In each case, I added my perspective on the topic.
Spectra Logic #1 – Can Consumer Technologies Meet the Demands of Commercial Data Storage Applications – This blog discusses the strategy from a high level and focuses specifically on the duty cycle of Blu-ray drives and media and whether optical hardware is robust enough for enterprise class use.
Jay’s opinion – Spectra Logic raises an interesting point. While the Blu-ray archive is storing infrequently accessed data, Blu-ray drives will still be required to access information and as data volumes increase, the burden on those drives will likely increase as well. This increase in drive utilization over time could put more stress on the drives and thus increase failure rates.
Spectra Logic #2 – The Facebook Prototype: Economics of Blu-ray Disc versus Tape Storage – This post contains a pricing analysis of the Blu-ray solution versus tape and highlights the fact that Blu-ray is 20x+ more expensive than LTO-6 in terms of $/GB.
Jay’s opinion – 20x+ price difference is huge especially when storing petabytes of archive of information. This difference is so massive that it potentially overshadows any other benefits of Blu-ray. Others have suggested that there could be some new technology to improve the economics, but one of the facets of cost relates to volume (as volumes go up, prices usually go down). Plus, it is hard for me to imagine that Blu-ray media shipments would ever rise to levels to compete with LTO shipments. I suppose that there could be some software technologies to improve the economics like deduplication, but those solutions are not typically optimized for removable media like Blu-ray or tape. It will be interesting to see what techniques are used to reduce costs going forward.
SearchStorage – Facebook storage: Reviews vary for Blu-ray cold storage, M.2 SSD plans (Note: registration required) – This article contains multiple analyst quotes and commentary on the Blu-ray architecture and is particularly informative.
Jay’s opinion – The views in this story are pretty consistent and align with my perspectives that the price differences between Blu-ray and tape are so vast as to be difficult to overcome. In fact my favorite quote from this article is from my friend, John Webster from the Evaluator Group who said, “The vendor community is littered with the dead bodies of optical storage companies trying to sell optical [drives] to enterprise storage buyers.” I could not agree more with John’s perspectives.
Forbes – Who Saves Everything Forever? – This article is written by John Webster and discusses the potential benefits of long-term storage on Blu-ray due to the long lifespan of the media.
Jay’s opinion – John makes the point that Blu-ray could be beneficial because it potentially has a longer lifespan than tape thus would require fewer migrations over its lifecycle. This is an interesting position and I would agree that fewer migrations would be required as compared to tape; however, the question comes back to economics. While fewer migrations are certainly better, is a reduction in migrations worth a 20x premium? For most use cases, I would say no.
In summary, it is clear that a Blu-ray archive strategy faces some significant challenges when compared with traditional tape archive. This is not to say that Blu-ray technology or optical archival is a bad thing, but like all technologies, it brings significant trade-offs. I am very interested in watching how the optical archival market changes over time. What do you think, is mainstream optical archiving the wave of the future or just a short term fad?
On a side note, in a future blog, I will compare optical disk performance with LTO. Stay tuned, it should be interesting.