Cloud Storage

For chat about everything, from the world of gadgets, to improving your home theatre setup.

Moderators: DreamensioN, Splash

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Sat 21 Dec, 2013 7:33 pm

spitex wrote:Cameron, we are testing 6x 100Mbit/s connections between our DC's (customer gear) and Amazon Direct Connect - they are a private layer2 link between our network and boxes at Amazon. It's pretty sexy stuff ;)

Our link to Amazon is capable of 10Gig.

But you are right, I do worry about security of people's files on the NAS. Like, I would hate to have Eugenixx's adult collection visible haha.
I've looked at the amazon direct connect stuff, and while it does look good I think that unless the customer is REALLY big, they'd probably just invest more in their Internet link and get a double dose of speed.

I am only partially concerned about other people's files from a security point of view (all my stuff would be encrypted), but I am concerned about size of storage and basic SLA's. I just think it would be easier if people who bought in got given x00gb of storage, and that was it. That way I don't have to get cranky when I cant upload an archive because dan's collection of cheese pr0n has grown another 100gb.

From a realistic point of view, I think a few hundred gig would be enough for most people. If you want unlimited storage you can go pay for it, likewise if you want a few gig for some home documents then you can get a free account from dropbox.

Some quick maths:
- QNAP TS-420U is $799. 4 bay, 1RU, 2xgigabit eth ports. Only 1 power supply, but unless we want to spend ANOTHER $1400 for an 869 with dual power, 2RU, 8 bays, then its not worth it. (No, I don't think its worth it)
- 4x 3tb Seagate barracuda's would be $139x4= $556. WD Red's are $183 a piece, but I'm not sure if they're worth it.

Raid 5 would be sufficient for a 4 bay NAS given its essentially an offsite backup, so 4x 3tb drives would give 9tb of storage for $1356 ($1532 with reds).

If we went the 2RU option with 8x 3tb drives it would be $3312 ($3664 with reds) for 21tb. To be honest, I'd prefer to buy 2x 1RU kits and geographically disperse them, than have 1x 2RU sitting in one spot. For one thing, it would be a few hundred dollars cheaper straight away.

So quick price breakdown (based on raid 5):
420U with seagates = $150/tb
420U with reds = $170/tb
869U with seagates = $157/tb
869U with reds = $175/tb

To be honest if we went with the 8 bay, I'd suggest RAID 6 (allowing for 2 drives to fail), which would make it:
869U with seagates = $184/tb
869U with reds = $203/tb

So, on the basic option (1RU with 4x seagates), costs would come to something like this:
5x 1tb allocations @ $150 per allocation
+
8 x 500gb allocations @ $75 per allocation

Giving 13 allocations/users in total. Can easily make 18 allocations @ $75/each, or 9 allocations @ $150/each.

Based on a 4 year life cycle, it would end up costing ~$40/$20 per year.

If there is enough interest I'm happy to sort out the logistics (including initial seeding of data) and then give it to spite for racking and comms.

User avatar
dan
argument inc
Posts: 6154
Joined: Wed 08 May, 2002 5:46 pm
Drives: German's mum........
Location: Brisbane

Re: Cloud Storage

Post by dan » Sun 22 Dec, 2013 12:52 am

daBOODA wrote:Has anyone looked into transporter? You buy a transporter for each location (your house, your parents' house, your mate's house etc). Each transporter has a hard drive in it. The transporters will then sync between each other. There's no ongoing fees because it's your bandwidth. I'm pretty sure you start the hard drives off with data before you switch it all on so you don't have to do a big initial transfer.
isn't that what i am already doing with crashplan for free??
camo means mogg, and mogg means business

User avatar
Eu-GenixX
5000RPM
Posts: 5978
Joined: Tue 30 Apr, 2002 8:37 pm
Drives: CJ Lancer VR
Location: Brisbane
Contact:

Re: Cloud Storage

Post by Eu-GenixX » Sun 22 Dec, 2013 3:14 pm

spitex wrote:But you are right, I do worry about security of people's files on the NAS. Like, I would hate to have Eugenixx's adult collection visible haha.
You don't have enough space or bandwidth to cope with that... :P

User avatar
tipper
male housemaid
Posts: 2842
Joined: Thu 03 Jul, 2003 5:48 pm
Location: Brisbane

Re: Cloud Storage

Post by tipper » Mon 23 Dec, 2013 8:43 am

Bloods, I just looked at transporter. What's with websites like this being so vague about what they do?

Looks like a useful way of syncing data if the point is I could have one at say my parents house and sync from my home PC to the transporter there as a backup. But the real use is just as a remote access solution I am guessing.

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Fri 24 Jan, 2014 9:10 pm

cGr wrote:I think the biggest issue I'm going to have is uploading all the data, which based on a couple of hundred gig will take a couple of months :o I could always send amazon a hard drive with all the files on it - for a couple of hundred bucks its not a bad option.
So, ~1 month on, and I've got half my data backed up to Amazon's Glacier service.

What have I learnt?
- Uploading data using an ADSL link sucks massive amounts of balls. To be fair, the same problem exists for any cloud based backup.
- I've been using Cloudberry mostly, with fastglacier occasionally, and its delayed me somewhat. Because Glacier only updates its inventory ~once a day (and then there's the ~4 hour retrieval time of that inventory), glacier clients tend to keep their own inventories to help with what files are where. I got caught on a directory because (impatient) I tried fastglacier to see if it were any better, and the whole thing got confused. Ironically, after I left it for a couple of days and the inventories updated on both of the clients, everything was sweet.
- Cloudberry has been a bit of a bitch. A new version came out which was deleting files that it had just uploaded (despite it knowing that it had just uploaded them), and it was only after a few irate emails that they admitted that there was a bug and that the "updated" version on the website (which had the same version number as the previous version) fixed the problem.

On the upside:
- I've had to load test some Internet links lately, and having 5MB/s upload speeds has saved me some time.
- It's cheap. I think the first 'bill' (which was actually just a debit of my credit) was 28c.
- The new cloudberry client is much better, and knowing the limitation on inventory times I can now use fastglacier in parallel (just uploading a separate directory).

I'm going to keep using Glacier, but crashplans interface is by far a better thing to use (and dropbox kills them all). If we all had 100mbps upload speeds, then glacier would be far easier to use.

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Fri 24 Jan, 2014 9:42 pm

And not unrelated, backblaze blog on the most reliable hard drives: http://blog.backblaze.com/2014/01/21/wh ... uld-i-buy/

User avatar
dorift
I love lamp
Posts: 8251
Joined: Mon 24 Jun, 2002 8:21 pm
Drives: Toyota Hilux Diablo
Location: Not near you
Contact:

Re: Cloud Storage

Post by dorift » Sun 26 Jan, 2014 10:16 pm

Apart from just paying a premium price, this shit is just all too complicated for the average Joe hey.

So I'll end up just losing all my shit in a fiery blaze of HDD sadness.
89 Corolla SX
90 Pulsar GTiR
95 Barina GSi
98 200sx
04 Forester XT
99 Hilux ute (woohoo!)

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Sun 26 Jan, 2014 11:13 pm

Go here: https://www.code42.com/store/
Buy individual for 1 year.
Start uploading.

Even @ the premium price of $5.78 AUD/mth, its pretty cheap insurance.

or

I'll help you setup glacier for cheaps.

User avatar
dan
argument inc
Posts: 6154
Joined: Wed 08 May, 2002 5:46 pm
Drives: German's mum........
Location: Brisbane

Re: Cloud Storage

Post by dan » Mon 27 Jan, 2014 4:29 pm

dan wrote:as an update, i have recently fscked off amazon s3 in favour of crash plan because it is cheaper and has unlimited storage.

crashplan has a free version that lets you backup to other computers (either on your local network, or somewhere remote over the internet like your parents/friends/etc computer), so i have all my computers create local "little" backups on my HTPC, and then i have a paid version of crashplan on my HTPC that creates a "big" backup of the HTPC and all the backups on the HTPC to the cloud. i also have it do the same "big" backup to a local PC so that i don't need to rip everything down from the cloud for something simple like a hardware failure.

it works pretty well. i even have my old man running crashplan free and it backs up his music/documents/etc over the internet to the HTPC, that is in turn backed up to the cloud.

it costs $60/yr and i am putting 300gb on it. valuable insurance for photos/etc.

the only fear i have is that crashplan will go broke. but i guess all of the local backups reduce that risk somewhat. the chances of someone stealing/burning all of my computers and crashplan going broke, all in the same week, is pretty remote.

fwiw, i also use dropbox at work, but mostly for collaboration. it is good at what it does, but i would't use it for backups unless i got it for free... too expensive and not enough features!!
and, over 1mo later, my 140gb crashplan cloud backup is completed..!! man uploading on DSL sucks!!

it costs $60/yr, which i am happy to pay because i would be very sad if i lost all my photos.
camo means mogg, and mogg means business

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Thu 13 Feb, 2014 9:21 pm

dan wrote:and, over 1mo later, my 140gb crashplan cloud backup is completed..!! man uploading on DSL sucks!!
I hear ya. I've been burning my ADSL all night to get a few gig up. Lucky to have a day like today where I get 20gb up in a couple of hours @ 160mbps :drool

120gb down, only 50gb to go!

User avatar
dorift
I love lamp
Posts: 8251
Joined: Mon 24 Jun, 2002 8:21 pm
Drives: Toyota Hilux Diablo
Location: Not near you
Contact:

Re: Cloud Storage

Post by dorift » Thu 13 Feb, 2014 9:57 pm

Still living life on the edge here!
89 Corolla SX
90 Pulsar GTiR
95 Barina GSi
98 200sx
04 Forester XT
99 Hilux ute (woohoo!)

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Fri 14 Feb, 2014 7:33 am

Rebel.

User avatar
dorift
I love lamp
Posts: 8251
Joined: Mon 24 Jun, 2002 8:21 pm
Drives: Toyota Hilux Diablo
Location: Not near you
Contact:

Re: Cloud Storage

Post by dorift » Fri 14 Feb, 2014 8:58 am

Image
89 Corolla SX
90 Pulsar GTiR
95 Barina GSi
98 200sx
04 Forester XT
99 Hilux ute (woohoo!)

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Sat 08 Mar, 2014 7:26 pm

3 months and 187gb later, the backup has finished. Patience is a virtue.

Now for the rest of the family backups...

User avatar
dorift
I love lamp
Posts: 8251
Joined: Mon 24 Jun, 2002 8:21 pm
Drives: Toyota Hilux Diablo
Location: Not near you
Contact:

Re: Cloud Storage

Post by dorift » Sun 09 Mar, 2014 12:21 am

So what if you'd lost everything during the upload?
89 Corolla SX
90 Pulsar GTiR
95 Barina GSi
98 200sx
04 Forester XT
99 Hilux ute (woohoo!)

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Sun 09 Mar, 2014 9:19 am

I would have cried like a baby. Just like what you're going to do if you don't get your shit sorted.

User avatar
Evan
Nice WRX Fuckhead
Posts: 3818
Joined: Fri 03 May, 2002 10:11 am
Drives: audi. what a c##t!
Location: Vancouver, BC
Contact:

Re: Cloud Storage

Post by Evan » Tue 11 Mar, 2014 7:21 am

i had an 'oh fuck' moment over the weekend and thought i'd nearly lost nearly 300gb worth of stuff (mainly photos).

backups are done, now just to get a copy offsite in case the worst really does happen.
aka gti-R

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Fri 14 Mar, 2014 7:39 pm

Google drive storage price drop: http://googleblog.blogspot.com.au/2014/ ... drive.html
15gb: Free
100gb: $1.99/mth
1tb: $9.99/mth
10tb+: $99.99/mth

User avatar
spitex
GTiR wannabe
Posts: 10721
Joined: Sun 28 Apr, 2002 8:17 am
Drives: FPV F6
Location: Brisneyland
Contact:

Re: Cloud Storage

Post by spitex » Fri 14 Mar, 2014 8:48 pm

Drive is so easy! :D We use it for work.

User avatar
cGr
Intuition
Posts: 10110
Joined: Fri 19 Jul, 2002 9:08 am

Re: Cloud Storage

Post by cGr » Fri 28 Mar, 2014 7:26 pm

Amazon S3 storage pricing has dropped. First terabyte used to be $0.094/gb and now its $0.033/gb (which is a fairly decent reducation).

No change for glacier so its still $0.012/gb.

There is a reduction in computing power too, though I'm not sure who on here would care.

Post Reply