March 11, 2013
All I ever hear about these days is “big data”. It’s a like saying “old Neil” – pretty much a natural consequence of getting out of bed each day.
Big data is a nice term trying to point out that we have lots of data, and we are more and more often putting it in the cloud (that invisible thing out there in la-la land). I do, and while pushing that data out there I’ve made an interesting (and somewhat obvious) observation.
Big data in my place is photography. I have a mac (great machine), which stores all my family data. These days that consists of music (not as dominant as it used to be) and photos. Once upon a time video was the big driver of space in my systems, but now it’s still photos. How do I know all this? The mac has a fantastic component called “time machine” which is very similar in nature to say ShadowProtect (which many of you server lads would be familiar with) – the ability to roll back to a certain point in time etc, while keeping a copy of all your data on an external device (in my case a USB drive) in case the hard drive in the mac takes a powder.
But is that safe enough? Not for me, so I purchase some space up in the cloud and make a copy of my data to that location. Because I’m anal and somewhat organised (at least I like to think so), I’ve filed everything into year folders, then each folder has a separate folder within it for each “event” throughout that year.
I then push a year at a time up to the cloud repository, so that when I’m finished I only need to update the last year, then make a new year folder on the mac and start putting files in that. OK some clever person is going to tell me there is some wonderful software available to do all this but you know what? I actually like being in control of what is happening and what is going were, so like Frank said “I’ll do it my way thank you very much”.
So the interesting thing has been looking at the data that is in each of these folders. 6-8 years ago it was grainy video from dedicated video cameras that we lugged around from BMX event to BMX event. Those days are gone. With the price reduction of quality Digital SLR cameras I know find three of them lying around the house (none of which are mind and who knows how they work out whose is whose), but these babies are the generators of storage requirements big time for me.
With the speed of the cameras, the size of the memory cards and the resolution of the photos – it’s a perfect storm of data being dumped into the mac. Yes it is supposedly possible to delete unwanted copies of horrible photos that will never be printed, but that never seems to happy in my house. So I have to be organised, manage the proliferation of data happening on the machine in the other corner of my office and have a regularly checked system to keep track of and copy of, these photos. You can bet your bottom dollar that I’ll be the one in trouble if something does and something is lost (I am, after all, the “computer person” in the house).
All this is great business for computer companies. The average business is starting to put it’s compute power in the cloud, store it’s data in the cloud etc, but I believe the real driver of cloud data is not business, but social media and personal data. At Adaptec we are just going along for the ride – happy to sell product to whichever cloud vendor (often called datacenter) needs more storage so no complaints from a corporate perspective, but … and here’s what I want to know …
What is your “Big Data”? … and how are you making sure it’s safe.